From patchwork Mon Aug 24 09:06:18 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28012 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3A0BEC3339 for ; Mon, 24 Aug 2026 09:06:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DC1AA68397; Mon, 24 Aug 2026 11:06:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nSjSEPG5"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EE0F467FEC for ; Mon, 24 Aug 2026 11:06:55 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 853CEC24; Mon, 24 Aug 2026 11:05:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562331; bh=byIGjRhenI/x8BOCx2/Bw1OU6vwKjYygTpEJfXzKfRU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nSjSEPG54lccoaJrPtxRDeaLNogArJsSkcHSPIfkiSj1kjkXm6cYIQYX+wCFBZcEA J4guEiw390/i+xYIZO9kSOhMnQ1BtrGFMjf1tzkAngQfYf6/uv40q2+k7TNtApnt5l h1WgZGG068cCfnpnOmMVR5YG3knHELneXe9/PLpQ= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 01/20] utils: update-kernel-headers: Add rkisp2-config.h Date: Mon, 24 Aug 2026 18:06:18 +0900 Message-ID: <20260824090641.3705246-2-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add rkisp2-config.h to the list of kernel headers to update, as we will soon be adding support for the rkisp2 pipeline. Signed-off-by: Paul Elder --- No change in v2 --- utils/update-kernel-headers.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/utils/update-kernel-headers.sh b/utils/update-kernel-headers.sh index 3fdea5f1eb5d..75eeee263418 100755 --- a/utils/update-kernel-headers.sh +++ b/utils/update-kernel-headers.sh @@ -58,6 +58,7 @@ headers=" linux/media/arm/mali-c55-config.h linux/media/v4l2-isp.h linux/rkisp1-config.h + linux/rkisp2-config.h linux/stddef.h linux/udmabuf.h linux/v4l2-common.h From patchwork Mon Aug 24 09:06:19 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28013 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 47F08C3339 for ; Mon, 24 Aug 2026 09:07:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E4A3A683A4; Mon, 24 Aug 2026 11:07:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="B6KcR7ch"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A137067FEC for ; Mon, 24 Aug 2026 11:06:59 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 47578C25; Mon, 24 Aug 2026 11:05:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562335; bh=IUzxXOl45WZFJcJxIeDZysixrWbxJrd+XnUEvzWSsck=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=B6KcR7chWMmzf6URMjCZ2GZTr7IM5l6fV9TedvXiOjXaBm5mUoVi9v1JxHb7o+kUI zhS5jyhGAhxN7F9kA0Th8AVPMlBV2ieKNA7mgdmDxyf5QpaH/3ppNWZfVkABtpudNp MT72/ZnrYXhLMYAlpg6x/lkO2YHa0xV19zQ8kzXE= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 02/20] include: linux: rkisp2-config.h: update to v7.2-rc1 Date: Mon, 24 Aug 2026 18:06:19 +0900 Message-ID: <20260824090641.3705246-3-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Update rkisp2-config.h to the most recent version. Signed-off-by: Paul Elder --- This comes from v2 of "media: rockchip: rkisp2: Add rkisp2 driver": https://lore.kernel.org/all/20260824090406.3695880-1-paul.elder@ideasonboard.com/ Changes in v2: - update to the headers from v2 of the kernel patches (obviously...) - use extensible stats instead of non-extensible stats - add crop parameter --- include/linux/README | 2 +- include/linux/rkisp2-config.h | 602 ++++++++++++++++++++++++++++++++++ include/linux/videodev2.h | 4 + 3 files changed, 607 insertions(+), 1 deletion(-) create mode 100644 include/linux/rkisp2-config.h diff --git a/include/linux/README b/include/linux/README index 9303e0f14ae6..0f580b0e5183 100644 --- a/include/linux/README +++ b/include/linux/README @@ -1,4 +1,4 @@ # SPDX-License-Identifier: CC0-1.0 -Files in this directory are imported from v7.1 of the Linux kernel. Do not +Files in this directory are imported from v7.1-14720-g4e2580d7e333 of the Linux kernel. Do not modify them manually. diff --git a/include/linux/rkisp2-config.h b/include/linux/rkisp2-config.h new file mode 100644 index 000000000000..56f7ca05d868 --- /dev/null +++ b/include/linux/rkisp2-config.h @@ -0,0 +1,602 @@ +/* SPDX-License-Identifier: ((GPL-2.0-or-later WITH Linux-syscall-note) OR MIT) */ +/* + * Rockchip ISP2 userspace API + * Copyright (C) 2017 Rockchip Electronics Co., Ltd. + * Copyright (C) 2026 Ideas on Board Oy. + */ + +#ifndef _RKISP2_CONFIG_H +#define _RKISP2_CONFIG_H + +#include + +#include + +#define RKISP2_ISP_GAMMA_OUT_MAX_SEGMENTS 49 + +#define RKISP2_ISP_LSC_SAMPLES_MAX 17 +#define RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX 16 + +#define RKISP2_ISP_AE_MEAN_MAX_LITE 25 +#define RKISP2_ISP_AE_MEAN_MAX_BIG 225 + +#define RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_LITE 25 +#define RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_BIG 225 +#define RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_MAX RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_BIG + +#define RKISP2_ISP_HIST_BIN_N_MAX 256 + +#define RKISP2_ISP_AWB_COUNTS_SIZE 225 + +/** + * enum rkisp2_isp_version - ISP variants + * + * @RKISP3_V0: Used at least in RK3588 + */ +enum rkisp2_isp_version { + RKISP3_V0 = 30, +}; + +/* See enum rkisp2_isp_goc_segments for logarithmic segment sizes */ +enum rkisp2_isp_goc_mode { + RKISP2_ISP_GOC_MODE_LOGARITHMIC, + RKISP2_ISP_GOC_MODE_EQUIDISTANT +}; + +/* + * The segments are: + * 1 x8, 2 x4, 4 x4, 8 x4, 16 x4, 32 x4, 64 x4, 128 x4, 256 x4, 512 x4 + * In 48-segment mode, the last group of 512 x4 becomes 256 x8 + */ +enum rkisp2_isp_goc_segments { + RKISP2_ISP_GOC_SEGMENTS_44, + RKISP2_ISP_GOC_SEGMENTS_48 +}; + +enum rkisp2_isp_lsc_config { + RKISP2_ISP_LSC_CONFIG_8X8, + RKISP2_ISP_LSC_CONFIG_16X16 +}; + +enum rkisp2_isp_set_active_table_when { + RKISP2_ISP_LSC_SET_ACTIVE_TABLE_AFTER, + RKISP2_ISP_LSC_SET_ACTIVE_TABLE_BEFORE, +}; + +enum rkisp2_isp_crop_en { + RKISP2_ISP_CROP_ENABLE_MAIN = 0x1, + RKISP2_ISP_CROP_ENABLE_SELF = 0x2, +}; + +enum rkisp2_isp_histogram_mode { + RKISP2_ISP_HISTOGRAM_MODE_DISABLE, + RKISP2_ISP_HISTOGRAM_MODE_R_HISTOGRAM = 2, + RKISP2_ISP_HISTOGRAM_MODE_G_HISTOGRAM, + RKISP2_ISP_HISTOGRAM_MODE_B_HISTOGRAM, + RKISP2_ISP_HISTOGRAM_MODE_Y_HISTOGRAM +}; + +/* + * This selects which bits are used from the input data to compute the + * histogram + */ +enum rkisp2_isp_histogram_data_sel { + RKISP2_ISP_HISTOGRAM_DATA_SEL_11_4, + RKISP2_ISP_HISTOGRAM_DATA_SEL_10_3, + RKISP2_ISP_HISTOGRAM_DATA_SEL_9_2, + RKISP2_ISP_HISTOGRAM_DATA_SEL_8_1, + RKISP2_ISP_HISTOGRAM_DATA_SEL_7_0, +}; + +/*---------- Statistics ------------*/ + +/** + * enum rkisp2_stats_block_type - RkISP2 extensible stats block types + * + * @RKISP2_STATS_BLOCK_AE_LITE: Auto exposite (lite) measurements + * @RKISP2_STATS_BLOCK_HIST_LITE: Histogram (lite) + * @RKISP2_STATS_BLOCK_HIST_BIG0: Histogram (big 0) + * @RKISP2_STATS_BLOCK_HIST_BIG1: Histogram (big 1) + * @RKISP2_STATS_BLOCK_HIST_BIG2: Histogram (big 2) + * @RKISP2_STATS_BLOCK_AWB: Auto white balance counts + */ +enum rkisp2_stats_block_type { + RKISP2_STATS_BLOCK_AE_LITE, + RKISP2_STATS_BLOCK_HIST_LITE, + RKISP2_STATS_BLOCK_HIST_BIG0, + RKISP2_STATS_BLOCK_HIST_BIG1, + RKISP2_STATS_BLOCK_HIST_BIG2, + RKISP2_STATS_BLOCK_AWB, +}; + +/** + * struct rkisp2_stats_ae_lite - statistics auto exposure data + * + * @header: block header (type = RKISP2_STATS_BLOCK_AE_LITE) + * @exp_mean_r: Mean luminance value of block xy for r channel + * @exp_mean_g: Mean luminance value of block xy for g channel + * @exp_mean_b: Mean luminance value of block xy for b channel + * @done: This set to nonzero when the stats are ready + * + * Image is divided into 5x5 blocks on lite and 15x15 blocks on big. + */ +struct rkisp2_stats_ae_lite { + struct v4l2_isp_block_header header; + __u16 exp_mean_r[RKISP2_ISP_AE_MEAN_MAX_LITE]; + __u16 exp_mean_g[RKISP2_ISP_AE_MEAN_MAX_LITE]; + __u16 exp_mean_b[RKISP2_ISP_AE_MEAN_MAX_LITE]; + __u8 done; +}; + +/** + * struct rkisp2_stats_hist - statistics histogram data + * + * @header: block header (type = RKISP2_STATS_BLOCK_HIST_{LITE,BIG0,BIG1,BIG2}) + * @hist_bins: measured bin counters. Each bin is a 28 bits unsigned fixed point + * type. Bits 0-4 are the fractional part and bits 5-27 are the + * integer part. + * @done: This set to nonzero when the stats are ready + * + * There are 256 bins, at least on 3.x. + */ +struct rkisp2_stats_hist { + struct v4l2_isp_block_header header; + __u32 hist_bins[RKISP2_ISP_HIST_BIN_N_MAX]; + __u8 done; +}; + +/** + * struct rkisp2_stats_awb - statistics auto white balance data + * + * @header: block header (type = RKISP2_STATS_BLOCK_AWB) + * @counts_r: Counts of red (18-bits) + * @counts_g: Counts of green (18-bits) + * @counts_b: Counts of blue (18-bits) + * @counts_w: Counts of white point (10-bits) + * @done: This set to nonzero when the stats are ready + * + * TODO Figure out what is being counted + */ +struct rkisp2_stats_awb { + struct v4l2_isp_block_header header; + __u32 counts_r[RKISP2_ISP_AWB_COUNTS_SIZE]; + __u32 counts_g[RKISP2_ISP_AWB_COUNTS_SIZE]; + __u32 counts_b[RKISP2_ISP_AWB_COUNTS_SIZE]; + __u16 counts_w[RKISP2_ISP_AWB_COUNTS_SIZE]; + __u8 done; +}; + +#define RKISP2_STATS_MAX_SIZE \ + (sizeof(struct rkisp2_stats_ae_lite) +\ + sizeof(struct rkisp2_stats_hist) * 4 +\ + sizeof(struct rkisp2_stats_awb)) + +/*---------- Parameters ------------*/ + +/** + * enum rkisp2_params_block_type - RkISP2 extensible params block types + * + * @RKISP2_PARAMS_BLOCK_BLS: Black level subtraction + * @RKISP2_PARAMS_BLOCK_AWB_GAINS: AWB gains + * @RKISP2_PARAMS_BLOCK_CSM: Color conversion coefficients (in the ISP block) + * @RKISP2_PARAMS_BLOCK_CCM: Color correction matrix (in the CCM block) + * @RKISP2_PARAMS_BLOCK_GOC: Gamma out correction + * @RKISP2_PARAMS_BLOCK_LSC: Lens shading correction + * @RKISP2_PARAMS_BLOCK_CROP: Crop config + * @RKISP2_PARAMS_BLOCK_AE_LITE: AE measurement config (lite) + * @RKISP2_PARAMS_BLOCK_HIST_LITE: Histogram measurement config (lite) + * @RKISP2_PARAMS_BLOCK_HIST_BIG0: Histogram measurement config (zeroth big block) + * @RKISP2_PARAMS_BLOCK_HIST_BIG1: Histogram measurement config (first big block) + * @RKISP2_PARAMS_BLOCK_HIST_BIG2: Histogram measurement config (second big block) + * @RKISP2_PARAMS_BLOCK_AWB_MEAS: AWB measurements config + */ +enum rkisp2_params_block_type { + RKISP2_PARAMS_BLOCK_BLS, + RKISP2_PARAMS_BLOCK_AWB_GAINS, + RKISP2_PARAMS_BLOCK_CSM, + RKISP2_PARAMS_BLOCK_CCM, + RKISP2_PARAMS_BLOCK_GOC, + RKISP2_PARAMS_BLOCK_LSC, + RKISP2_PARAMS_BLOCK_CROP, + RKISP2_PARAMS_BLOCK_AE_LITE, + RKISP2_PARAMS_BLOCK_HIST_LITE, + RKISP2_PARAMS_BLOCK_HIST_BIG0, + RKISP2_PARAMS_BLOCK_HIST_BIG1, + RKISP2_PARAMS_BLOCK_HIST_BIG2, + RKISP2_PARAMS_BLOCK_AWB_MEAS, +}; + +/** + * struct rkisp2_isp_window - measurement window. + * + * Measurements are calculated per window inside the frame. + * This struct represents a window for a measurement. + * + * @h_offs: the horizontal offset of the window from the left of the frame in pixels. + * @v_offs: the vertical offset of the window from the top of the frame in pixels. + * @h_size: the horizontal size of the window in pixels + * @v_size: the vertical size of the window in pixels. + */ +struct rkisp2_isp_window { + __u16 h_offs; + __u16 v_offs; + __u16 h_size; + __u16 v_size; +}; + +/** + * struct rkisp2_isp_bls_fixed_val - BLS fixed subtraction values + * + * These are signed 13-bit (-4096 to +4095). + * + * @a: Fixed black level value for Bayer channel 0 + * @b: Fixed black level value for Bayer channel 1 + * @c: Fixed black level value for Bayer channel 2 + * @d: Fixed black level value for Bayer channel 3 + */ +struct rkisp2_isp_bls_fixed_val { + __s16 a; + __s16 b; + __s16 c; + __s16 d; +}; + +/** + * struct rkisp2_isp_awb_gains - Auto white balance gain in the ISP block + * + * All fields in this struct are 16 bit, where: + * 0x100h = 1, unsigned integer value, range 0 to 63 with 8 bit fractional part. + * + * This leaves the upper two msb unaccounted for; it is unknown if these are + * unused or misdocumented. + * + * TODO investigate the upper two bits + * + * @r: gain value for red component. + * @gr: gain value for green component in red line. + * @b: gain value for blue component. + * @gb: gain value for green component in blue line. + */ +struct rkisp2_isp_awb_gains { + __u16 r; + __u16 gr; + __u16 b; + __u16 gb; +}; + +/** + * struct rkisp2_isp_color_cc - Color coefficients + * + * @r: Red coefficient + * @g: Green coefficient + * @b: Blue coefficient + */ +struct rkisp2_isp_color_cc { + __u8 r; + __u8 g; + __u8 b; +}; + +/** + * struct rkisp2_isp_awb_color_quad - Group of RGB and luminance for AWB + * + * TODO redesign this? + * + * @r: Red + * @g: Green + * @b: Blue + * @y: Y (luminance) + */ +struct rkisp2_isp_awb_color_quad { + __u8 r; + __u8 g; + __u8 b; + __u8 y; +}; + +/** + * struct rkisp2_params_bls - RkISP2 params BLS config + * + * RkISP2 parameters Black Level Subtraction configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_BLS`. + * + * TODO Check if auto-mode and window selection is for both blocks or just for + * one block (it might be the same as 2.x) + * + * @header: The RkISP2 parameters block header + * @enable_auto: Automatic mode activated means that the measured values + * are subtracted. Otherwise the fixed subtraction + * values will be subtracted. + * @enabled_windows: enabled window (bit 0 for window 1, bit 1 for window 2) + * @bls_window1: Measurement window 1 size + * @bls_window2: Measurement window 2 size + * @bls_samples: Set amount of measured pixels for each Bayer position + * (A, B, C and D) to 2^bls_samples. (TODO needs confirmation) + * @bls_fixed_val: Black Level Subtraction fixed values for the BLS module at + * the front of the pipeline + * @bls1_fixed_val: Black Level Subtraction fixed values for the BLS module after + * bayer noise reduction + */ +struct rkisp2_params_bls { + struct v4l2_isp_params_block_header header; + __u8 enable_auto; + __u8 enabled_windows; + struct rkisp2_isp_window bls_window1; + struct rkisp2_isp_window bls_window2; + __u8 bls_samples; + struct rkisp2_isp_bls_fixed_val bls_fixed_val; + struct rkisp2_isp_bls_fixed_val bls1_fixed_val; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_awb_gains - RKISP2 params AWB gains config + * + * RkISP2 parameters auto white balance gains configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_AWB_GAINS`. + * + * TODO investigate what the different blocks mean + * + * Block 0 is equivalent to the awb gains block on 2.x, but blocks 1 and + * 2 do not exist on 2.x. + * + * @header: The RkISP2 parameters block header + * @gains: Gains configuration for block i + */ +struct rkisp2_params_awb_gains { + struct v4l2_isp_params_block_header header; + struct rkisp2_isp_awb_gains gains[3]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_csm - Configuration used by Color Space Conversion + * + * RkISP2 parameters histogram configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_CSM`. + * + * @header: The RkISP2 parameters block header + * @coeff: color correction matrix. Values are 9-bit signed fixed-point numbers with 2 bit integer + * and 7 bit fractional part, ranging from -2 (0x100) to +1.992 (0x0FF). 0 is + * represented by 0x000 and a coefficient value of 1 as 0x080. + */ +struct rkisp2_params_csm { + struct v4l2_isp_params_block_header header; + __u16 coeff[3][3]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_ccm - Configuration used by Color Correction Matrix + * + * RkISP2 parameters histogram configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_CCM`. + * + * @header: The RkISP2 parameters block header + * @high_y_alpha_adj_en: Enable CCM high Y alpha adjustment (TODO figure out what this does) + * @coeff: color correction matrix. Values are 11-bit signed fixed-point numbers with 4 bit integer + * and 7 bit fractional part, ranging from -8 (0x400) to +7.992 (0x3FF). 0 is + * represented by 0x000 and a coefficient value of 1 as 0x080. The + * value is expanded 128 times (TODO figure out what this means). + * @offset: Red, Green, Blue offsets for the color correction matrix. 12-bits + * wide ranging from -4096 to 4095, but only for red; green and blue are 11-bit + * signed fixed-point like coeff, but are still 12-bits wide. + * @y_coeff: Red, Green, Blue coefficients for RGB2Y calculation. red and green + * are 11-bits wide and blue is 12-bits wide. The value is expanded 128 times. + * @alp: CCM curve y-axis point definition for ccm input pixel's luminance. + * 11-bit unsigned ranging from 0 to 1024. The value is expanded 128 times. + * @inflection_point: Inflection point of the ccm alpha interpolation curve. + * The inflection point is 2^inflection_point. Since the maximum y-value is + * 1024, the maximum value of this field is expected to be 10 (0xa), but the + * documentation says 4'b10. + */ +struct rkisp2_params_ccm { + struct v4l2_isp_params_block_header header; + __u8 high_y_alpha_adj_en; + __u16 coeff[3][3]; + __u16 offset[3]; + __u16 y_coeff[3]; + __u16 alp[17]; + __u8 inflection_point; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_goc - Configuration used by Gamma Out correction + * + * RkISP2 parameters gamma out correction configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_GOC`. + * + * @header: The RkISP2 parameters block header + * @mode: goc mode (from enum rkisp2_isp_goc_mode) + * @segments: segments mode (from enum rkisp2_isp_goc_segments) + * @offset: offset value of the gamma out curve + * @gamma_y: gamma out curve y-axis for all color components + * + * The number of entries of @gamma_y depends on the segments mode. The entries + * are 12-bit unsigned. + */ +struct rkisp2_params_goc { + struct v4l2_isp_params_block_header header; + __u8 mode; + __u8 segments; + __u16 offset; + __u16 gamma_y[RKISP2_ISP_GAMMA_OUT_MAX_SEGMENTS]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_lsc - Configuration used by Lens shading correction + * + * RkISP2 parameters lens shading correction configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_LSC`. + * + * The LSC module on the rkisp2 two tables: the 0th table and the 1th table. + * They can be programmed independently and (somewhat) simultaneously, and can be + * swapped by setting a single register. Hence the UAPI here is designed so + * that all these components can be controlled independently. + * + * In the first dimension of {r,gr,gb,b}_data_tbl we can designate which table + * to write the data to. write_table is then used to signal whether to write + * the data, and this can be controlled for both tables. active_table chooses + * which table to activate. set_active_table_when signals whether to set the + * active_table before or after programming the table. This allows + * optimizations such as setting a future table in one parameter buffer while + * swapping before setting it. + * + * This design gives us more control. For example, if we want to only program + * the 0th table without modifying the 1th table, we do not need to also + * populate the 1th table and we can use write_table to designate that we only + * want to program the 0th table. We can also swap tables without needing to + * re-populate the tables by setting active_table and unsetting write_table. + * + * {x,y}_sizes designates the grid of the LSC, and the table entries above + * correspond to the *vertices* of the grid. {x,y}_grads control the bilinear + * interpolation within the grid. + * + * @header: The RkISP2 parameters block header + * @r_data_tbl: Sample table red + * @gr_data_tbl: Sample table green (red) + * @gb_data_tbl: Sample table green (blue) + * @b_data_tbl: Sample table blue + * @write_table: Set to 1 to signal to write the respective table from above + * @active_table: Choose which of the two tables is active (0 or 1) + * @set_active_table_when: From rkisp2_isp_set_active_table_when; switch to the + * active table before or after programming the table + * @x_sizes: Sizes x + * @y_sizes: Sizes y + * @x_grads: Gradients x + * @y_grads: Gradients y + * @window_mode: From enum rkisp2_isp_lsc_config + */ +struct rkisp2_params_lsc { + struct v4l2_isp_params_block_header header; + + __u16 r_data_tbl[2][RKISP2_ISP_LSC_SAMPLES_MAX][RKISP2_ISP_LSC_SAMPLES_MAX]; + __u16 gr_data_tbl[2][RKISP2_ISP_LSC_SAMPLES_MAX][RKISP2_ISP_LSC_SAMPLES_MAX]; + __u16 gb_data_tbl[2][RKISP2_ISP_LSC_SAMPLES_MAX][RKISP2_ISP_LSC_SAMPLES_MAX]; + __u16 b_data_tbl[2][RKISP2_ISP_LSC_SAMPLES_MAX][RKISP2_ISP_LSC_SAMPLES_MAX]; + __u8 write_table[2]; + __u8 active_table; + __u8 set_active_table_when; + + __u16 x_sizes[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + __u16 y_sizes[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + __u16 x_grads[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + __u16 y_grads[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + + __u8 window_mode; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_crop - Crop configuration + * + * RkISP2 parameters crop configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_CROP`. + * This block is responsible for configuring main and self path crops. The + * driver will automatically compute the scaling factors for the resizer and + * any format conversions (eg. for converting subsampling factors). + * + * @header: The RkISP2 parameters block header + * @crop_en: Enable bits for main path and self path crop below. Comes from enum rkisp2_isp_crop_en + * @mp_crop: Crop window for the main path, relative to the ISP source format size + * @sp_crop: Crop window for the self path, relative to the ISP source format size + */ +struct rkisp2_params_crop { + struct v4l2_isp_params_block_header header; + __u8 crop_en; + struct rkisp2_isp_window mp_crop; + struct rkisp2_isp_window sp_crop; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_ae_lite - RKISP2 params AE lite config + * + * RkISP2 parameters auto exposure measurement configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_AE_LITE`. + * + * TODO change window_num to enum? + * + * @header: The RkISP2 parameters block header + * @window_num: 0 for 1x1, 1 for 5x5 + * @meas_window: Size of measurement window. First window for 5x5. + */ +struct rkisp2_params_ae_lite { + struct v4l2_isp_params_block_header header; + __u8 window_num; + struct rkisp2_isp_window meas_window; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_hist_lite RKISP2 params histogram lite config + * + * RkISP2 parameters histogram configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_HIST_LITE`. + * + * @header: The RkISP2 parameters block header + * @data_sel: Data selection mode (from enum rkisp2_isp_histogram_data_sel) + * @mode: Histogram mode (from enum rkisp2_isp_histogram_mode) + * @stepsize: Predivider (count every pixel) + * @waterline: Waterline for region statics + * @coeffs: Coefficients for raw2y formula + * @meas_window: Size of first measurement subwindow + * @weights: Weights + */ +struct rkisp2_params_hist_lite { + struct v4l2_isp_params_block_header header; + __u8 data_sel; + __u8 mode; + __u8 stepsize; + __u16 waterline; + struct rkisp2_isp_color_cc coeffs; + struct rkisp2_isp_window meas_window; + __u8 weights[RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_LITE]; +} __attribute__((aligned(8))); + +/** + * Same as struct rkisp2_params_hist_lite but for big channel + * + * RkISP2 parameters histogram configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_HIST_BIG{0,1,2}`. + * + * @window_num: 0 or 1 for 5x5, 2 or 3 for 15x15 + */ +struct rkisp2_params_hist_big { + struct v4l2_isp_params_block_header header; + __u8 window_num; + __u8 data_sel; + __u8 mode; + __u8 stepsize; + __u16 waterline; + struct rkisp2_isp_color_cc coeffs; + struct rkisp2_isp_window meas_window; + __u8 weights[RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_BIG]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_awb_meas - Configuration used by rawawb + * + * RkISP2 parameters AWB measurement configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_AWB_MEAS`. + * + * @header: The RkISP2 parameters block header + * @meas_window: Size of first measurement subwindow (13 bits) + * @limits: Limits for white point detection [min, max] (8 bits) + * @weights: Weights (6-bits) + */ +struct rkisp2_params_awb_meas { + struct v4l2_isp_params_block_header header; + struct rkisp2_isp_window meas_window; + struct rkisp2_isp_awb_color_quad limits[2]; + __u8 weights[RKISP2_ISP_AWB_COUNTS_SIZE]; +} __attribute__((aligned(8))); + +#define RKISP2_PARAMS_MAX_SIZE \ + (sizeof(struct rkisp2_params_bls) +\ + sizeof(struct rkisp2_params_awb_gains) +\ + sizeof(struct rkisp2_params_csm) +\ + sizeof(struct rkisp2_params_ccm) +\ + sizeof(struct rkisp2_params_goc) +\ + sizeof(struct rkisp2_params_lsc) +\ + sizeof(struct rkisp2_params_crop) +\ + sizeof(struct rkisp2_params_ae_lite) +\ + sizeof(struct rkisp2_params_hist_lite) +\ + sizeof(struct rkisp2_params_hist_big) * 3 +\ + sizeof(struct rkisp2_params_awb_meas)) + +#endif /* _RKISP2_CONFIG_H */ diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h index 8bccad84ff83..d242f3c0ab5f 100644 --- a/include/linux/videodev2.h +++ b/include/linux/videodev2.h @@ -851,6 +851,10 @@ struct v4l2_pix_format { #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ #define V4L2_META_FMT_RK_ISP1_EXT_PARAMS v4l2_fourcc('R', 'K', '1', 'E') /* Rockchip ISP1 3a Extensible Parameters */ +/* Vendor specific - used for RKISP2 camera sub-system */ +#define V4L2_META_FMT_RKISP2_PARAMS v4l2_fourcc('R', 'K', '2', 'P') /* Rockchip ISP2 Parameters */ +#define V4L2_META_FMT_RKISP2_STATS v4l2_fourcc('R', 'K', '2', 'S') /* Rockchip ISP2 3A Statistics */ + /* Vendor specific - used for C3_ISP */ #define V4L2_META_FMT_C3ISP_PARAMS v4l2_fourcc('C', '3', 'P', 'M') /* Amlogic C3 ISP Parameters */ #define V4L2_META_FMT_C3ISP_STATS v4l2_fourcc('C', '3', 'S', 'T') /* Amlogic C3 ISP Statistics */ From patchwork Mon Aug 24 09:06:20 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28014 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 509BAC3339 for ; Mon, 24 Aug 2026 09:07:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F3B65683AD; Mon, 24 Aug 2026 11:07:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="f1UHonua"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 54EB967FEC for ; Mon, 24 Aug 2026 11:07:03 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F2CC3C2D; Mon, 24 Aug 2026 11:05:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562339; bh=n/lYAk74Bw2T3OivRbYmWVvTdtrJWUwHCkOWpaTCr+E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f1UHonua1BrhzHD6Lo/soY6dpK1asO7xLdHcxeAqz2yDPgaPE7mcN5iY3OYGWhViE w9ln6+9cfpA/gdBBT4PG8E3DmAkakUqG4tPI8LGpB9YZfsJYQoIN2iFZNLCcOXCO6B GIHK/NVIk1yakcAMvnjLU2AZg5ZVZJJqveJA/PAo= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 03/20] ipa: rkisp2: params: Add rkisp2 extensible parameters wrapper Date: Mon, 24 Aug 2026 18:06:20 +0900 Message-ID: <20260824090641.3705246-4-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the libipa wrapper for extensible parameters for the rkisp2. This wraps the extensible parameters in the rkisp2 uapi to make them easier to use within libcamera. Signed-off-by: Paul Elder --- This is in a separate patch to reduce the size of the other patches to make them easier to review. Changes in v2: - rename things so that it doesn't conflict with extensible stats that have been added - add crop --- src/ipa/rkisp2/algorithms/algorithm.h | 27 +++++++++ src/ipa/rkisp2/params.h | 85 +++++++++++++++++++++++++++ 2 files changed, 112 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/algorithm.h create mode 100644 src/ipa/rkisp2/params.h diff --git a/src/ipa/rkisp2/algorithms/algorithm.h b/src/ipa/rkisp2/algorithms/algorithm.h new file mode 100644 index 000000000000..2b945c930161 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/algorithm.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 control algorithm interface + */ + +#pragma once + +#include + +#include + +#include "module.h" + +namespace libcamera { + +namespace ipa::rkisp2 { + +class Algorithm : public libcamera::ipa::Algorithm +{ +}; + +} /* namespace ipa::rkisp2 */ + +} /* namespace libcamera */ + diff --git a/src/ipa/rkisp2/params.h b/src/ipa/rkisp2/params.h new file mode 100644 index 000000000000..96452cf19414 --- /dev/null +++ b/src/ipa/rkisp2/params.h @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 ISP Parameters + */ + +#pragma once + +#include + +#include +#include + +#include + +namespace libcamera { + +namespace ipa::rkisp2 { + +enum class RkISP2ParamsBlocks : uint16_t { + Bls, + AwbGains, + Csm, + Ccm, + Goc, + Lsc, + Crop, + AeLite, + HistLite, + HistBig0, + HistBig1, + HistBig2, + AwbMeas, +}; + +namespace details { + +template +struct block_type { +}; + +#define RkISP2_DEFINE_PARAMS_BLOCK_TYPE(id, cfgType, blkType) \ +template<> \ +struct block_type { \ + using type = struct rkisp2_params_##cfgType; \ + static constexpr rkisp2_params_block_type blockType = \ + rkisp2_params_block_type::RKISP2_PARAMS_BLOCK_##blkType;\ +} + +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(Bls, bls, BLS); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(AwbGains, awb_gains, AWB_GAINS); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(Csm, csm, CSM); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(Ccm, ccm, CCM); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(Goc, goc, GOC); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(Lsc, lsc, LSC); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(Crop, crop, CROP); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(AeLite, ae_lite, AE_LITE); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(HistLite, hist_lite, HIST_LITE); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(HistBig0, hist_big, HIST_BIG0); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(HistBig1, hist_big, HIST_BIG1); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(HistBig2, hist_big, HIST_BIG2); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(AwbMeas, awb_meas, AWB_MEAS); + +struct param_traits { + using id_type = RkISP2ParamsBlocks; + + template + using id_to_details = block_type; +}; + +} /* namespace details */ + +class RkISP2Params : public V4L2Params +{ +public: + RkISP2Params(Span data) + : V4L2Params(data, V4L2_ISP_VERSION_V1) + { + } +}; + +} /* namespace ipa::rkisp2 */ + +} /* namespace libcamera */ From patchwork Mon Aug 24 09:06:21 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28015 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 16248C3339 for ; Mon, 24 Aug 2026 09:07:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C5D9C68393; Mon, 24 Aug 2026 11:07:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HszAR/1h"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0DAA467FEC for ; Mon, 24 Aug 2026 11:07:07 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id ABF86C24; Mon, 24 Aug 2026 11:05:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562342; bh=wE+G8FOfk2l9WPTDhebO2dGae0MfXf1PpV7BtEI4mWE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HszAR/1hhtR1rfcFh6i/emXWaYAHmUGL02m4Xaw7E+2DEDxutTe1cK8Q9WPIl6SfK xlCK3f5gQeFsLvfa8Yok5ABDsqVzgPJwD5IAgDGR05WfrbdXDFd0+iHm5tN74NU6QT qBOSG3mkWBpIM9eTrUnt3zKPqw0PhfB9Txh3qQzk= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 04/20] ipa: rkisp2: stats: Add rkisp2 extensible statistics wrapper Date: Mon, 24 Aug 2026 18:06:21 +0900 Message-ID: <20260824090641.3705246-5-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the libipa wrapper for extensible statistics for the rkisp2. This wraps the extensible statistics in the rkisp2 uapi to make them easier to use within libcamera. Signed-off-by: Paul Elder --- This is in a separate patch to reduce the size of the other patches to make them easier to review. New in v2 --- src/ipa/rkisp2/stats.h | 65 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 src/ipa/rkisp2/stats.h diff --git a/src/ipa/rkisp2/stats.h b/src/ipa/rkisp2/stats.h new file mode 100644 index 000000000000..1a63041ebe10 --- /dev/null +++ b/src/ipa/rkisp2/stats.h @@ -0,0 +1,65 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 ISP Statistics + */ + +#pragma once + +#include + +#include +#include + +#include + +namespace libcamera { + +namespace ipa::rkisp2 { + +enum class RkISP2StatsBlocks : uint16_t { + AeLite, + HistLite, + HistBig0, + HistBig1, + HistBig2, + Awb, +}; + +template +struct block_type { +}; + +#define RKISP2_DEFINE_STATS_BLOCK_TYPE(id, cfgType, blkType) \ +template<> \ +struct block_type { \ + using type = struct rkisp2_stats_##cfgType; \ + static constexpr rkisp2_stats_block_type blockType = \ + rkisp2_stats_block_type::RKISP2_STATS_BLOCK_##blkType; \ +} + +RKISP2_DEFINE_STATS_BLOCK_TYPE(AeLite, ae_lite, AE_LITE); +RKISP2_DEFINE_STATS_BLOCK_TYPE(HistLite, hist, HIST_LITE); +RKISP2_DEFINE_STATS_BLOCK_TYPE(HistBig0, hist, HIST_BIG0); +RKISP2_DEFINE_STATS_BLOCK_TYPE(HistBig1, hist, HIST_BIG1); +RKISP2_DEFINE_STATS_BLOCK_TYPE(HistBig2, hist, HIST_BIG2); +RKISP2_DEFINE_STATS_BLOCK_TYPE(Awb, awb, AWB); + +struct stats_traits { + using id_type = RkISP2StatsBlocks; + template using id_to_details = block_type; +}; + +class RkISP2Stats : public V4L2Stats +{ +public: + RkISP2Stats(Span data) + : V4L2Stats(data, V4L2_ISP_VERSION_V1) + { + } +}; + +} /* namespace ipa::rkisp2 */ + +} /* namespace libcamera */ From patchwork Mon Aug 24 09:06:22 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28016 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1E9AFC3339 for ; Mon, 24 Aug 2026 09:07:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C3EBA68393; Mon, 24 Aug 2026 11:07:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bxc79KSc"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B243A67FEC for ; Mon, 24 Aug 2026 11:07:10 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 60714C23; Mon, 24 Aug 2026 11:05:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562346; bh=RCr7Egp/mMIrcJAlCiT/38vZ5WdMFnsHjXF/U8nDX8I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bxc79KScjX2D+v3DHLgFht6vBTz6gIcwd8dW/ntBlBTvQX6ABz3X3suCNPy0nNFZk tSdyEWQcz0vyPDYJmM66HU9ky4K0VqI/BpoKY9sgrQHZsh5qoJp3GvHcKxuD0clAAV t4iKGTto80mXO2Tc7WkmA5gbNG1PQubScH2GaSKU= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 05/20] include: ipa: rkisp2: Add rkisp2 ipa interface Date: Mon, 24 Aug 2026 18:06:22 +0900 Message-ID: <20260824090641.3705246-6-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the IPA interface for the rkisp2. It is based on and is nearly identical to the rkisp1 IPA interface. Signed-off-by: Paul Elder --- This is also split into a separate patch to make the other patches smaller and easier to review. No change in v2 --- include/libcamera/ipa/meson.build | 1 + include/libcamera/ipa/rkisp2.mojom | 47 ++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+) create mode 100644 include/libcamera/ipa/rkisp2.mojom diff --git a/include/libcamera/ipa/meson.build b/include/libcamera/ipa/meson.build index fb518526a2a0..32854928fdc5 100644 --- a/include/libcamera/ipa/meson.build +++ b/include/libcamera/ipa/meson.build @@ -66,6 +66,7 @@ pipeline_ipa_mojom_mapping = { 'ipu3': 'ipu3.mojom', 'mali-c55': 'mali-c55.mojom', 'rkisp1': 'rkisp1.mojom', + 'rkisp2': 'rkisp2.mojom', 'rpi/pisp': 'raspberrypi.mojom', 'rpi/vc4': 'raspberrypi.mojom', 'simple': 'softisp.mojom', diff --git a/include/libcamera/ipa/rkisp2.mojom b/include/libcamera/ipa/rkisp2.mojom new file mode 100644 index 000000000000..52284a29cd24 --- /dev/null +++ b/include/libcamera/ipa/rkisp2.mojom @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +/* + * \todo Document the interface and remove the related EXCLUDE_PATTERNS entry. + */ + +module ipa.rkisp2; + +import "include/libcamera/ipa/core.mojom"; + +/* + * colorSpaceEncoding comes from libcamera::ColorSpace::YcbcrEncoding; -1 for invalid + * colorSpaceRange comes from libcamera::ColorSpace::Range; -1 for invalid + */ +struct IPAConfigInfo { + libcamera.IPACameraSensorInfo sensorInfo; + libcamera.ControlInfoMap sensorControls; + int32 colorSpaceEncoding; + int32 colorSpaceRange; +}; + +interface IPARkISP2Interface { + init(libcamera.IPASettings settings, + libcamera.IPACameraSensorInfo sensorInfo, + libcamera.ControlInfoMap sensorControls) + => (int32 ret, libcamera.ControlInfoMap ipaControls); + start() => (int32 ret); + stop(); + + configure(IPAConfigInfo configInfo) + => (int32 ret, libcamera.ControlInfoMap ipaControls); + + mapBuffers(array buffers); + unmapBuffers(array ids); + + [async] queueRequest(uint32 frame, libcamera.ControlList reqControls); + [async] computeParams(uint32 frame, uint32 bufferId); + [async] processStats(uint32 frame, uint32 bufferId, + libcamera.ControlList sensorControls); +}; + +interface IPARkISP2EventInterface { + paramsComputed(uint32 frame, uint32 bufferId, uint32 bytesused); + setSensorControls(uint32 frame, libcamera.ControlList sensorControls); + metadataReady(libcamera.ControlList metadata); +}; + From patchwork Mon Aug 24 09:06:23 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28017 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 028F4C3339 for ; Mon, 24 Aug 2026 09:07:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BA5F968393; Mon, 24 Aug 2026 11:07:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="qSOFO4y0"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AB8AD67FEC for ; Mon, 24 Aug 2026 11:07:14 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 170A6C25; Mon, 24 Aug 2026 11:05:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562350; bh=3B2xkjEm/RCN5RjoZhRTZbw9FEZ+QMH8jnfQ9CxGYVg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qSOFO4y0jx3pV4L+WObTOst6R7mEO3lK4tO4t6R3srLx4MpTWeoMFHVNrCAhczTfd RUbAofVDN1S0x/AviwOuuuqlfTizm1ZOopokARohV+CQBeGJbotpS9Zf+68eETkhKY 6t5V/TXzNelj5XXrM7UuI6z6WG4sZuO6bq8P6kYQ= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 06/20] ipa: rkisp2: Add rkisp2 ipa Date: Mon, 24 Aug 2026 18:06:23 +0900 Message-ID: <20260824090641.3705246-7-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement the rkisp2 IPA. It is based on the rkisp1 IPA. No control algorithms are implemented yet, but this lays the foundation and plumbing for the IPA to function. The control algorithms will be added in later patches. Signed-off-by: Paul Elder --- This is also split from other patches to make them all easier to review. I put this patch before the pipeline handler patch so that the pipeline handler doesn't error out when it fails to find an IPA. Changes in v2: - initialize/declare ScalerCrop control - use extensible stats --- meson_options.txt | 2 +- src/ipa/meson.build | 1 + src/ipa/rkisp2/algorithms/meson.build | 5 + src/ipa/rkisp2/algorithms/module.h | 29 ++ src/ipa/rkisp2/ipa_context.h | 211 ++++++++++++ src/ipa/rkisp2/meson.build | 29 ++ src/ipa/rkisp2/rkisp2.cpp | 440 ++++++++++++++++++++++++++ 7 files changed, 716 insertions(+), 1 deletion(-) create mode 100644 src/ipa/rkisp2/algorithms/meson.build create mode 100644 src/ipa/rkisp2/algorithms/module.h create mode 100644 src/ipa/rkisp2/ipa_context.h create mode 100644 src/ipa/rkisp2/meson.build create mode 100644 src/ipa/rkisp2/rkisp2.cpp diff --git a/meson_options.txt b/meson_options.txt index e829ebe60490..56a5e2085990 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -48,7 +48,7 @@ option('gstreamer', option('ipas', type : 'array', - choices : ['ipu3', 'mali-c55', 'rkisp1', 'rpi/pisp', 'rpi/vc4', + choices : ['ipu3', 'mali-c55', 'rkisp1', 'rkisp2', 'rpi/pisp', 'rpi/vc4', 'softisp', 'vimc'], description : 'Select which IPA modules to build') diff --git a/src/ipa/meson.build b/src/ipa/meson.build index ab4a1897f58a..3e8f5bffcd3a 100644 --- a/src/ipa/meson.build +++ b/src/ipa/meson.build @@ -28,6 +28,7 @@ supported_ipas = { 'ipu3': 'ipu3', 'mali-c55': 'mali-c55', 'rkisp1': 'rkisp1', + 'rkisp2': 'rkisp2', 'rpi/pisp': 'rpi/pisp', 'rpi/vc4': 'rpi/vc4', 'simple': 'softisp', diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build new file mode 100644 index 000000000000..4bb81e48eb01 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +rkisp2_ipa_algorithms = files([ +]) + diff --git a/src/ipa/rkisp2/algorithms/module.h b/src/ipa/rkisp2/algorithms/module.h new file mode 100644 index 000000000000..fc681dd0aef1 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/module.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board Oy. + * + * RkISP2 IPA Module + */ + +#pragma once + +#include + +#include + +#include + +#include "ipa_context.h" +#include "params.h" +#include "stats.h" + +namespace libcamera { + +namespace ipa::rkisp2 { + +using Module = ipa::Module; + +} /* namespace ipa::rkisp2 */ + +} /* namespace libcamera*/ diff --git a/src/ipa/rkisp2/ipa_context.h b/src/ipa/rkisp2/ipa_context.h new file mode 100644 index 000000000000..57d796e3265d --- /dev/null +++ b/src/ipa/rkisp2/ipa_context.h @@ -0,0 +1,211 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board Oy. + * + * RkISP2 IPA Context + * + */ + +#pragma once + +#include + +#include + +#include + +#include +#include +#include + +#include + +#include "libcamera/internal/debug_controls.h" +#include "libcamera/internal/matrix.h" +#include "libcamera/internal/vector.h" + +#include "libipa/agc_mean_luminance.h" +#include "libipa/awb.h" +#include "libipa/camera_sensor_helper.h" +#include "libipa/ccm.h" +#include "libipa/fc_queue.h" +#include "libipa/fixedpoint.h" +#include "libipa/lsc.h" + +namespace libcamera { + +namespace ipa::rkisp2 { + +struct IPAHwSettings { + unsigned int numAeCells; + unsigned int numHistogramBins; + unsigned int numHistogramWeights; + unsigned int numGammaOutSamples; + uint32_t supportedBlocks; + bool compand; +}; + +struct RKISP2AwbSession { + struct rkisp2_isp_window measureWindow; + bool enabled; +}; + +struct IPASessionConfiguration { + struct { + struct rkisp2_isp_window measureWindow; + struct rkisp2_isp_window measureWindow15; + } agc; + + struct RKISP2AwbSession awb; + + struct { + utils::Duration minExposureTime; + utils::Duration maxExposureTime; + double minAnalogueGain; + double maxAnalogueGain; + + int32_t defVBlank; + utils::Duration lineDuration; + Size size; + } sensor; + + struct { + int32_t colorSpaceEncoding; + int32_t colorSpaceRange; + } csm; + + bool raw; +}; + +struct IPAActiveState { + struct { + struct { + uint32_t exposure; + double gain; + } manual; + struct { + uint32_t exposure; + double gain; + double quantizationGain; + double yTarget; + } automatic; + + bool autoExposureEnabled; + bool autoGainEnabled; + double exposureValue; + controls::AeConstraintModeEnum constraintMode; + controls::AeExposureModeEnum exposureMode; + controls::AeMeteringModeEnum meteringMode; + utils::Duration minFrameDuration; + utils::Duration maxFrameDuration; + } agc; + + ipa::awb::ActiveState awb; + + struct { + double gamma; + } goc; + + ipa::ccm::ActiveState ccm; + + struct { + double lux; + } lux; + + struct { + controls::WdrModeEnum mode; + AgcMeanLuminance::AgcConstraint constraint; + double gain; + double strength; + } wdr; + + ipa::lsc::ActiveState lsc; + + struct { + Matrix csm; + bool update; + } csm; + + struct { + Rectangle crop; + bool set; + } crop; +}; + +struct IPAFrameContext : public FrameContext { + struct { + uint32_t exposure; + double gain; + double exposureValue; + double quantizationGain; + uint32_t vblank; + double yTarget; + bool autoExposureEnabled; + bool autoGainEnabled; + controls::AeConstraintModeEnum constraintMode; + controls::AeExposureModeEnum exposureMode; + controls::AeMeteringModeEnum meteringMode; + utils::Duration minFrameDuration; + utils::Duration maxFrameDuration; + utils::Duration frameDuration; + bool updateMetering; + bool autoExposureModeChange; + bool autoGainModeChange; + } agc; + + ipa::awb::FrameContext awb; + + struct { + double gamma; + bool update; + } goc; + + struct { + uint32_t exposure; + double gain; + } sensor; + + ipa::ccm::FrameContext ccm; + + struct { + double lux; + } lux; + + struct { + controls::WdrModeEnum mode; + double strength; + double gain; + } wdr; + + ipa::lsc::FrameContext lsc; + + struct { + Rectangle crop; + bool set; + } crop; +}; + +struct IPAContext { + IPAContext(unsigned int frameContextSize) + : frameContexts(frameContextSize) + { + } + + IPAHwSettings hw; + IPACameraSensorInfo sensorInfo; + IPASessionConfiguration configuration; + IPAActiveState activeState; + + FCQueue frameContexts; + + ControlInfoMap::Map ctrlMap; + + DebugMetadata debugMetadata; + + /* Interface to the Camera Helper */ + std::unique_ptr camHelper; +}; + +} /* namespace ipa::rkisp2 */ + +} /* namespace libcamera*/ diff --git a/src/ipa/rkisp2/meson.build b/src/ipa/rkisp2/meson.build new file mode 100644 index 000000000000..f2f435c2d66f --- /dev/null +++ b/src/ipa/rkisp2/meson.build @@ -0,0 +1,29 @@ +# SPDX-License-Identifier: CC0-1.0 + +subdir('algorithms') + +ipa_name = 'ipa_rkisp2' + +rkisp2_ipa_sources = files([ + 'rkisp2.cpp', +]) + +rkisp2_ipa_sources += rkisp2_ipa_algorithms + +mod = shared_module(ipa_name, rkisp2_ipa_sources, + name_prefix : '', + include_directories : [ipa_includes], + dependencies : [libcamera_private, libipa_dep], + install : true, + install_dir : ipa_install_dir) + +if ipa_sign_module + custom_target(ipa_name + '.so.sign', + input : mod, + output : ipa_name + '.so.sign', + command : [ipa_sign, ipa_priv_key, '@INPUT@', '@OUTPUT@'], + install : false, + build_by_default : true) +endif + +ipa_names += ipa_name diff --git a/src/ipa/rkisp2/rkisp2.cpp b/src/ipa/rkisp2/rkisp2.cpp new file mode 100644 index 000000000000..3085671008f7 --- /dev/null +++ b/src/ipa/rkisp2/rkisp2.cpp @@ -0,0 +1,440 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board Oy. + * + * RkISP2 Image Processing Algorithms + */ + +#include +#include +#include +#include +#include +#include + +#include +#include + +#include +#include + +#include +#include +#include +#include + +#include +#include +#include + +#include "libcamera/internal/formats.h" +#include "libcamera/internal/mapped_framebuffer.h" +#include "libcamera/internal/yaml_parser.h" + +#include "algorithms/algorithm.h" + +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(IPARkISP2) + +using namespace std::literals::chrono_literals; + +namespace ipa::rkisp2 { + +/* Maximum number of frame contexts to be held */ +static constexpr uint32_t kMaxFrameContexts = 16; + +class IPARkISP2 : public IPARkISP2Interface, public Module +{ +public: + IPARkISP2(); + + int init(const IPASettings &settings, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) override; + int start() override; + void stop() override; + + int configure(const IPAConfigInfo &ipaConfig, + ControlInfoMap *ipaControls) override; + void mapBuffers(const std::vector &buffers) override; + void unmapBuffers(const std::vector &ids) override; + + void queueRequest(const uint32_t frame, const ControlList &controls) override; + void computeParams(const uint32_t frame, const uint32_t bufferId) override; + void initializeFrameContext(IPAFrameContext &frameContext, + const ControlList &controls); + void processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls) override; + +protected: + std::string logPrefix() const override; + +private: + void updateControls(const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls); + + void setControls(unsigned int frame, const IPAFrameContext &frameContext); + + std::map buffers_; + std::map mappedBuffers_; + + ControlInfoMap sensorControls_; + + /* Local parameter storage */ + struct IPAContext context_; +}; + +namespace { + +/* List of controls handled by the RkISP2 IPA */ +const ControlInfoMap::Map rkisp2Controls{ + { &controls::DebugMetadataEnable, ControlInfo(false, true, false) }, +}; + +} /* namespace */ + +IPARkISP2::IPARkISP2() + : context_(kMaxFrameContexts) +{ + context_.frameContexts.setInitCallback( + [this](IPAFrameContext &fc, const ControlList &c) { + this->initializeFrameContext(fc, c); + }); +} + +std::string IPARkISP2::logPrefix() const +{ + return "rkisp2"; +} + +int IPARkISP2::init(const IPASettings &settings, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) +{ + context_.sensorInfo = sensorInfo; + + context_.camHelper = CameraSensorHelperFactoryBase::create(settings.sensorModel); + if (!context_.camHelper) { + LOG(IPARkISP2, Error) + << "Failed to create camera sensor helper for " + << settings.sensorModel; + return -ENODEV; + } + + context_.configuration.sensor.lineDuration = + sensorInfo.minLineLength * 1.0s / sensorInfo.pixelRate; + + /* Load the tuning data file. */ + File file(settings.configurationFile); + if (!file.open(File::OpenModeFlag::ReadOnly)) { + int ret = file.error(); + LOG(IPARkISP2, Error) + << "Failed to open configuration file " + << settings.configurationFile << ": " << strerror(-ret); + return ret; + } + + std::unique_ptr data = YamlParser::parse(file); + if (!data) + return -EINVAL; + + unsigned int version = (*data)["version"].get(0); + if (version != 1) { + LOG(IPARkISP2, Error) + << "Invalid tuning file version " << version; + return -EINVAL; + } + + if (!data->contains("algorithms")) { + LOG(IPARkISP2, Error) + << "Tuning file doesn't contain any algorithm"; + return -EINVAL; + } + + int ret = createAlgorithms(context_, (*data)["algorithms"]); + if (ret) + return ret; + + /* Initialize controls. */ + updateControls(sensorInfo, sensorControls, ipaControls); + + return 0; +} + +int IPARkISP2::start() +{ + /* \todo Properly handle startup controls. */ + return 0; +} + +void IPARkISP2::stop() +{ + context_.frameContexts.clear(); +} + +int IPARkISP2::configure(const IPAConfigInfo &ipaConfig, + ControlInfoMap *ipaControls) +{ + sensorControls_ = ipaConfig.sensorControls; + + const auto itExp = sensorControls_.find(V4L2_CID_EXPOSURE); + int32_t minExposure = itExp->second.min().get(); + int32_t maxExposure = itExp->second.max().get(); + + const auto itGain = sensorControls_.find(V4L2_CID_ANALOGUE_GAIN); + int32_t minGain = itGain->second.min().get(); + int32_t maxGain = itGain->second.max().get(); + + LOG(IPARkISP2, Debug) + << "Exposure: [" << minExposure << ", " << maxExposure + << "], gain: [" << minGain << ", " << maxGain << "]"; + + /* Clear the IPA context before the streaming session. */ + context_.configuration = {}; + context_.activeState = {}; + context_.frameContexts.clear(); + + const IPACameraSensorInfo &info = ipaConfig.sensorInfo; + const ControlInfo vBlank = sensorControls_.find(V4L2_CID_VBLANK)->second; + context_.configuration.sensor.defVBlank = vBlank.def().get(); + context_.configuration.sensor.size = info.outputSize; + context_.configuration.sensor.lineDuration = info.minLineLength * 1.0s / info.pixelRate; + + /* Update the camera controls using the new sensor settings. */ + updateControls(info, sensorControls_, ipaControls); + + /* + * When the AGC computes the new exposure values for a frame, it needs + * to know the limits for exposure time and analogue gain. As it depends + * on the sensor, update it with the controls. + * + * \todo take VBLANK into account for maximum exposure time + */ + context_.configuration.sensor.minExposureTime = + minExposure * context_.configuration.sensor.lineDuration; + context_.configuration.sensor.maxExposureTime = + maxExposure * context_.configuration.sensor.lineDuration; + context_.configuration.sensor.minAnalogueGain = + context_.camHelper->gain(minGain); + context_.configuration.sensor.maxAnalogueGain = + context_.camHelper->gain(maxGain); + + context_.configuration.csm.colorSpaceEncoding = ipaConfig.colorSpaceEncoding; + context_.configuration.csm.colorSpaceRange = ipaConfig.colorSpaceRange; + + for (const auto &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + + int ret = algo->configure(context_, info); + if (ret) + return ret; + } + + return 0; +} + +void IPARkISP2::mapBuffers(const std::vector &buffers) +{ + for (const IPABuffer &buffer : buffers) { + auto elem = buffers_.emplace(std::piecewise_construct, + std::forward_as_tuple(buffer.id), + std::forward_as_tuple(buffer.planes)); + const FrameBuffer &fb = elem.first->second; + + MappedFrameBuffer mappedBuffer(&fb, MappedFrameBuffer::MapFlag::ReadWrite); + if (!mappedBuffer.isValid()) { + LOG(IPARkISP2, Fatal) << "Failed to mmap buffer: " + << strerror(mappedBuffer.error()); + } + + mappedBuffers_.emplace(buffer.id, std::move(mappedBuffer)); + } +} + +void IPARkISP2::unmapBuffers(const std::vector &ids) +{ + for (unsigned int id : ids) { + const auto fb = buffers_.find(id); + if (fb == buffers_.end()) + continue; + + mappedBuffers_.erase(id); + buffers_.erase(id); + } +} + +void IPARkISP2::queueRequest(const uint32_t frame, const ControlList &controls) +{ + context_.debugMetadata.enableByControl(controls); + + context_.frameContexts.getOrInitContext(frame, controls); +} + +void IPARkISP2::initializeFrameContext(IPAFrameContext &frameContext, + const ControlList &controls) +{ + for (const auto &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + algo->queueRequest(context_, frameContext.frame(), frameContext, controls); + } +} + +void IPARkISP2::computeParams(const uint32_t frame, const uint32_t bufferId) +{ + IPAFrameContext &frameContext = context_.frameContexts.getOrInitContext(frame); + + RkISP2Params params(mappedBuffers_.at(bufferId).planes()[0]); + + for (const auto &algo : algorithms()) + algo->prepare(context_, frame, frameContext, ¶ms); + + paramsComputed.emit(frame, bufferId, params.bytesused()); +} + +void IPARkISP2::processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls) +{ + IPAFrameContext &frameContext = context_.frameContexts.getOrInitContext(frame); + + RkISP2Stats stats(mappedBuffers_.at(bufferId).planes()[0]); + + frameContext.sensor.exposure = + sensorControls.get(V4L2_CID_EXPOSURE).get(); + frameContext.sensor.gain = + context_.camHelper->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + + ControlList metadata(controls::controls); + + for (const auto &algo : algorithms()) + algo->process(context_, frame, frameContext, &stats, metadata); + + setControls(frame, frameContext); + + metadataReady.emit(metadata); +} + +void IPARkISP2::setControls(unsigned int frame, const IPAFrameContext &frameContext) +{ + /* + * \todo The frame number is most likely wrong here, we need to take + * internal sensor delays and other timing parameters into account. + */ + + uint32_t exposure = frameContext.agc.exposure; + uint32_t gain = context_.camHelper->gainCode(frameContext.agc.gain); + uint32_t vblank = frameContext.agc.vblank; + + LOG(IPARkISP2, Debug) + << "Set controls for frame " << frame << ": exposure " << exposure + << ", gain " << frameContext.agc.gain << ", vblank " << vblank; + + ControlList ctrls(sensorControls_); + if (frameContext.agc.exposure * frameContext.agc.gain > 0) { + ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); + ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain)); + } + ctrls.set(V4L2_CID_VBLANK, static_cast(vblank)); + + setSensorControls.emit(frame, ctrls); +} + +void IPARkISP2::updateControls(const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) +{ + ControlInfoMap::Map ctrlMap = rkisp2Controls; + + /* + * Compute exposure time limits from the V4L2_CID_EXPOSURE control + * limits and the line duration. + */ + double lineDuration = context_.configuration.sensor.lineDuration.get(); + const ControlInfo &v4l2Exposure = sensorControls.find(V4L2_CID_EXPOSURE)->second; + int32_t minExposure = v4l2Exposure.min().get() * lineDuration; + int32_t maxExposure = v4l2Exposure.max().get() * lineDuration; + int32_t defExposure = v4l2Exposure.def().get() * lineDuration; + ctrlMap.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::ExposureTime), + std::forward_as_tuple(minExposure, maxExposure, defExposure)); + + /* Compute the analogue gain limits. */ + const ControlInfo &v4l2Gain = sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; + float minGain = context_.camHelper->gain(v4l2Gain.min().get()); + float maxGain = context_.camHelper->gain(v4l2Gain.max().get()); + float defGain = context_.camHelper->gain(v4l2Gain.def().get()); + ctrlMap.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::AnalogueGain), + std::forward_as_tuple(minGain, maxGain, defGain)); + + /* + * Compute the frame duration limits. + * + * The frame length is computed assuming a fixed line length combined + * with the vertical frame sizes. + */ + const ControlInfo &v4l2HBlank = sensorControls.find(V4L2_CID_HBLANK)->second; + uint32_t hblank = v4l2HBlank.def().get(); + uint32_t lineLength = sensorInfo.outputSize.width + hblank; + + const ControlInfo &v4l2VBlank = sensorControls.find(V4L2_CID_VBLANK)->second; + std::array frameHeights{ + v4l2VBlank.min().get() + sensorInfo.outputSize.height, + v4l2VBlank.max().get() + sensorInfo.outputSize.height, + v4l2VBlank.def().get() + sensorInfo.outputSize.height, + }; + + std::array frameDurations; + for (unsigned int i = 0; i < frameHeights.size(); ++i) { + uint64_t frameSize = lineLength * frameHeights[i]; + frameDurations[i] = frameSize / (sensorInfo.pixelRate / 1000000U); + } + + /* \todo Move this (and other agc-related controls) to agc */ + context_.ctrlMap[&controls::FrameDurationLimits] = + ControlInfo(frameDurations[0], frameDurations[1], + ControlValue(Span{ { frameDurations[2], frameDurations[2] } })); + + Rectangle ispMinCrop{ 0, 0, 32, 32 }; + /* + * No need to clamp this as the hardware will hang anyway if sensor + * size > isp max size + */ + Rectangle ispMaxCrop{ 0, 0, sensorInfo.outputSize }; + /* + * \todo Either always enable Crop algo or make this conditional on + * when the Crop algo is present + */ + context_.ctrlMap[&controls::ScalerCrop] = + ControlInfo(ispMinCrop, ispMaxCrop, ispMaxCrop); + + ctrlMap.insert(context_.ctrlMap.begin(), context_.ctrlMap.end()); + *ipaControls = ControlInfoMap(std::move(ctrlMap), controls::controls); +} + +} /* namespace ipa::rkisp2 */ + +/* + * External IPA module interface + */ + +extern "C" { +const struct IPAModuleInfo ipaModuleInfo = { + IPA_MODULE_API_VERSION, + 1, + "rkisp2", +}; + +IPAInterface *ipaCreate() +{ + return new ipa::rkisp2::IPARkISP2(); +} +} + +} /* namespace libcamera */ From patchwork Mon Aug 24 09:06:24 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28018 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EF863C3339 for ; Mon, 24 Aug 2026 09:07:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A52EF68393; Mon, 24 Aug 2026 11:07:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="S/LNh2is"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 633D367FEC for ; Mon, 24 Aug 2026 11:07:18 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0F5B2C2D; Mon, 24 Aug 2026 11:05:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562354; bh=QxUz/zCRZt5t7Df8ulREtPTwJD6sQQtHXsy00sONejo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S/LNh2isH0pcyY+oQRkxBUfdphtAYZXYtJ1XjQTopGn3ymxYejbrZDcuGvDxxXQPZ 08CNnwfMc25pTzdVtftbMJyQkYK/dvh/NHBsN4j2IrnAN0B/DWS4E7yOq8wpT+DX67 bsq4R5yc/O4LdRKPwQu8GcIBJ8dl5WfrOxH5+F+c= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 07/20] pipeline: rkisp2: Implement pipeline handler for rkisp2 Date: Mon, 24 Aug 2026 18:06:24 +0900 Message-ID: <20260824090641.3705246-8-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement the pipeline handler for rkisp2. This currently supports the ISP version integrated on the Rockchip RK3588. The current version only supports memory-to-memory mode and only the main path and only raw sensors. Signed-off-by: Paul Elder --- Changes in v2: - split dma rx pads - split main/self path capture pads - no more configuring crop rectangles, as they have been moved to parameters buffers --- Documentation/runtime_configuration.rst | 9 + meson.build | 1 + meson_options.txt | 1 + src/libcamera/pipeline/rkisp2/meson.build | 5 + src/libcamera/pipeline/rkisp2/rkisp2.cpp | 1293 +++++++++++++++++++++ 5 files changed, 1309 insertions(+) create mode 100644 src/libcamera/pipeline/rkisp2/meson.build create mode 100644 src/libcamera/pipeline/rkisp2/rkisp2.cpp diff --git a/Documentation/runtime_configuration.rst b/Documentation/runtime_configuration.rst index a71b715a1724..53a5a29744c1 100644 --- a/Documentation/runtime_configuration.rst +++ b/Documentation/runtime_configuration.rst @@ -81,6 +81,8 @@ Configuration file example supported_devices: - driver: mxc-isi software_isp: true + rkisp2: + isp_enable: true software_isp: copy_input_buffer: false measure: @@ -157,6 +159,13 @@ pipelines.simple.supported_devices.driver, pipelines.simple.supported_devices.so Example `software_isp` value: ``true`` +pipelines.rkisp2.isp_enable + Configure whether or not to use the ISP. Default (when unconfigured) is + true. When set to false the ISP will not be used, so only the VICAP will be + used for capture. + + Example value: ``false`` + software_isp.copy_input_buffer Define whether input buffers should be copied into standard (cached) memory in software ISP. This is done by default to prevent very slow diff --git a/meson.build b/meson.build index d8e78ed6c01a..516014ffc93a 100644 --- a/meson.build +++ b/meson.build @@ -219,6 +219,7 @@ pipelines_support = { 'ipu3': arch_x86, 'mali-c55': arch_arm, 'rkisp1': arch_arm, + 'rkisp2': arch_arm, 'rpi/pisp': arch_arm, 'rpi/vc4': arch_arm, 'simple': ['any'], diff --git a/meson_options.txt b/meson_options.txt index 56a5e2085990..55ab557d2953 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -82,6 +82,7 @@ option('pipelines', 'ipu3', 'mali-c55', 'rkisp1', + 'rkisp2', 'rpi/pisp', 'rpi/vc4', 'simple', diff --git a/src/libcamera/pipeline/rkisp2/meson.build b/src/libcamera/pipeline/rkisp2/meson.build new file mode 100644 index 000000000000..86f43fb7fcc9 --- /dev/null +++ b/src/libcamera/pipeline/rkisp2/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +libcamera_internal_sources += files([ + 'rkisp2.cpp', +]) diff --git a/src/libcamera/pipeline/rkisp2/rkisp2.cpp b/src/libcamera/pipeline/rkisp2/rkisp2.cpp new file mode 100644 index 000000000000..fa17d08381f2 --- /dev/null +++ b/src/libcamera/pipeline/rkisp2/rkisp2.cpp @@ -0,0 +1,1293 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board Oy. + * + * Pipeline handler for Rockchip ISP2 + */ + +#include +#include +#include +#include +#include + +#include +#include + +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "libcamera/internal/buffer_queue.h" +#include "libcamera/internal/camera.h" +#include "libcamera/internal/camera_manager.h" +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/delayed_controls.h" +#include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/formats.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/global_configuration.h" +#include "libcamera/internal/ipa_manager.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/request.h" +#include "libcamera/internal/sequence_sync_helper.h" +#include "libcamera/internal/v4l2_subdevice.h" +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +static const Size ispMaxSize = Size(4416, 3312); +static const Size vicapMaxSize = Size(8192, 8192); + +/* \todo Re-add NV21 and YUV422 and YVU422 after it's fixed in the driver */ +const std::map formatToMediaBus = { + { formats::UYVY, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::NV12, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::NV16, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::NV61, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::YUV420, MEDIA_BUS_FMT_YUYV8_1_5X8 }, + { formats::YVU420, MEDIA_BUS_FMT_YUYV8_1_5X8 }, + { formats::R8, MEDIA_BUS_FMT_YUYV8_2X8 }, +}; + +/* \todo Deduplicate this (we have the same thing in rkisp1) */ +const std::map rawFormats = { + { formats::SBGGR8, MEDIA_BUS_FMT_SBGGR8_1X8 }, + { formats::SGBRG8, MEDIA_BUS_FMT_SGBRG8_1X8 }, + { formats::SGRBG8, MEDIA_BUS_FMT_SGRBG8_1X8 }, + { formats::SRGGB8, MEDIA_BUS_FMT_SRGGB8_1X8 }, + { formats::SBGGR10, MEDIA_BUS_FMT_SBGGR10_1X10 }, + { formats::SGBRG10, MEDIA_BUS_FMT_SGBRG10_1X10 }, + { formats::SGRBG10, MEDIA_BUS_FMT_SGRBG10_1X10 }, + { formats::SRGGB10, MEDIA_BUS_FMT_SRGGB10_1X10 }, + { formats::SBGGR12, MEDIA_BUS_FMT_SBGGR12_1X12 }, + { formats::SGBRG12, MEDIA_BUS_FMT_SGBRG12_1X12 }, + { formats::SGRBG12, MEDIA_BUS_FMT_SGRBG12_1X12 }, + { formats::SRGGB12, MEDIA_BUS_FMT_SRGGB12_1X12 }, +}; + +LOG_DEFINE_CATEGORY(RkISP2) + +class PipelineHandlerRkISP2; + +struct RkISP2FrameInfo { + RkISP2FrameInfo(Request *_request, + FrameBuffer *_buffer, + const ControlList &_metadata = ControlList(controls::controls), + bool _metadataProcessed = false) + : request(_request), mainPathBuffer(_buffer), metadataProcessed(_metadataProcessed) + { + metadata.merge(_metadata); + } + + Request *request = nullptr; + FrameBuffer *mainPathBuffer = nullptr; + ControlList metadata = ControlList(controls::controls); + bool metadataProcessed = false; +}; + +struct RkISP2RequestInfo { + Request *request = nullptr; + size_t sequence = 0; + bool sequenceValid = false; +}; + +class RkISP2CameraData : public Camera::Private +{ +public: + RkISP2CameraData(PipelineHandler *pipe, MediaDevice *media) + : Camera::Private(pipe), media_(media), frame_(0) + { + } + + ~RkISP2CameraData() + { + } + + int init(bool usingIsp, CameraSensor *sensor, V4L2VideoDevice *video, + V4L2VideoDevice *rawrd, V4L2Subdevice *isp, + V4L2VideoDevice *mainPath, V4L2VideoDevice *param, + V4L2VideoDevice *stat); + PixelFormat getSensorFormat(unsigned int mbusCode, const Size &size, + const Size &maxSize); + + PipelineHandlerRkISP2 *pipe(); + const PipelineHandlerRkISP2 *pipe() const; + int loadIPA(); + + void computeParamBuffers(unsigned int frame); + + void tryCompleteRequest(Request *request, FrameBuffer *buffer); + void tryCompleteRequest(const ControlList &metadata); + + void vicapBufferReady(FrameBuffer *buffer); + void rawrdBufferReady(FrameBuffer *buffer); + void ispBufferReady(FrameBuffer *buffer); + void statBufferReady(FrameBuffer *buffer); + + void paramsComputed(unsigned int frame, unsigned int bufferId, unsigned int bytesused); + void setSensorControls(unsigned int frame, const ControlList &sensorControls); + void metadataReady(const ControlList &metadata); + + /* These are the buffers that go between the vicap and the isp */ + std::vector> internalBuffers_; + + std::vector> statBuffers_; + std::unique_ptr paramQueue_; + + std::unique_ptr delayedCtrls_; + + /* + * These are the buffers that are half-complete, as in either metadata + * is ready or the frame has been completed from the ISP. Both + * ready-handlers populate and complete from the front of the queue. + * + * The stat-ready handler flushes the queue however, because in general + * more images complete than stats do, so this prevents the queue from + * perpetually growing and making the metadata out of date. + */ + std::deque pendingCompleteBuffers_; + + /* + * This is for tracking sequence numbers of requests for synchronizing + * between stats and params and images for any dropped frames + */ + std::deque pendingCompleteRequests_; + + std::vector ipaBuffers_; + + bool usingIsp_; + bool isRaw_; + + MediaDevice *media_; + CameraSensor *sensor_; + V4L2VideoDevice *video_; + V4L2VideoDevice *rawrd_; + V4L2Subdevice *isp_; + V4L2VideoDevice *mainPath_; + V4L2VideoDevice *param_; + V4L2VideoDevice *stat_; + Stream stream_; + + std::unique_ptr ipa_; + ControlInfoMap ipaControls_; + + /* + * The sensor frame sequence of the last request queued to the pipeline + * handler + */ + unsigned int frame_; + SequenceSyncHelper syncHelper_; +}; + +class RkISP2CameraConfiguration : public CameraConfiguration +{ +public: + RkISP2CameraConfiguration(const RkISP2CameraData *data); + + Status validate() override; + + const V4L2SubdeviceFormat &sensorFormat() { return sensorFormat_; } + +private: + const RkISP2CameraData *data_; + + V4L2SubdeviceFormat sensorFormat_; +}; + +namespace { + +/* + * This many internal buffers (or rather parameter and statistics buffer + * pairs) ensures that the pipeline runs smoothly, without frame drops. + */ +static constexpr unsigned int kRkISP2MinBufferCount = 4; + +} /* namespace */ + +class PipelineHandlerRkISP2 : public PipelineHandler +{ +public: + PipelineHandlerRkISP2(CameraManager *manager); + + std::unique_ptr + generateConfiguration(Camera *camera, + Span roles) override; + + int configure(Camera *camera, CameraConfiguration *config) override; + + int exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) override; + + int start(Camera *camera, const ControlList *controls) override; + void stopDevice(Camera *camera) override; + + int queueRequestDevice(Camera *camera, Request *request) override; + + bool match(DeviceEnumerator *enumerator) override; + +private: + RkISP2CameraData *cameraData(Camera *camera) + { + return static_cast(camera->_d()); + } + + int updateControls(RkISP2CameraData *data); + bool createCamera(bool usingIsp); + int processControls(RkISP2CameraData *data, const ControlList &ctrls); + + int allocateBuffers(Camera *camera); + int freeBuffers(Camera *camera); + + Size clampSensorSize(RkISP2CameraData *data, unsigned int mbus, + const Size &maxSize); + + std::shared_ptr media_; + std::unique_ptr sensor_; + std::unique_ptr csi_; + std::unique_ptr cif_; + std::unique_ptr video_; + + std::shared_ptr ispMedia_; + std::unique_ptr rawrd_; + std::unique_ptr isp_; + std::unique_ptr mainPath_; + std::unique_ptr param_; + std::unique_ptr stat_; +}; + +int RkISP2CameraData::init(bool usingIsp, CameraSensor *sensor, + V4L2VideoDevice *video, V4L2VideoDevice *rawrd, + V4L2Subdevice *isp, + V4L2VideoDevice *mainPath, V4L2VideoDevice *param, + V4L2VideoDevice *stat) +{ + usingIsp_ = usingIsp; + sensor_ = sensor; + video_ = video; + rawrd_ = rawrd; + isp_ = isp; + mainPath_ = mainPath; + param_ = param; + stat_ = stat; + + ControlInfoMap::Map ctrls; + + auto &testPatterns = sensor_->testPatternModes(); + if (testPatterns.size()) { + ctrls.emplace(&controls::draft::TestPatternMode, + ControlInfo(testPatterns.front(), testPatterns.back(), + testPatterns.front())); + } else + LOG(RkISP2, Warning) << "Sensor doesn't support any test pattern modes"; + + controlInfo_ = ControlInfoMap(std::move(ctrls), controls::controls); + + video_->bufferReady.connect(this, &RkISP2CameraData::vicapBufferReady); + if (mainPath_) { + rawrd_->bufferReady.connect(this, &RkISP2CameraData::rawrdBufferReady); + mainPath_->bufferReady.connect(this, &RkISP2CameraData::ispBufferReady); + stat_->bufferReady.connect(this, &RkISP2CameraData::statBufferReady); + } + + return 0; +} + +PixelFormat RkISP2CameraData::getSensorFormat(unsigned int mbusCode, + const Size &size, + const Size &maxSize) +{ + std::vector mbusCodes = { mbusCode }; + V4L2SubdeviceFormat format = sensor_->getFormat(mbusCodes, size, maxSize); + const auto &ret = std::find_if(rawFormats.begin(), rawFormats.end(), + [format](const auto &value) { return value.second == format.code; }); + if (ret != rawFormats.end()) + return (*ret).first; + + LOG(RkISP2, Error) << "No raw format supported by sensor"; + return formats::SRGGB10; +} + +PipelineHandlerRkISP2 *RkISP2CameraData::pipe() +{ + return static_cast(Camera::Private::pipe()); +} + +const PipelineHandlerRkISP2 *RkISP2CameraData::pipe() const +{ + return static_cast(Camera::Private::pipe()); +} + +int RkISP2CameraData::loadIPA() +{ + ipa_ = pipe()->createIPA(1, 1); + if (!ipa_) + return -ENOENT; + + ipa_->paramsComputed.connect(this, &RkISP2CameraData::paramsComputed); + ipa_->setSensorControls.connect(this, &RkISP2CameraData::setSensorControls); + ipa_->metadataReady.connect(this, &RkISP2CameraData::metadataReady); + + /* The IPA tuning file is made from the sensor name. */ + std::string ipaTuningFile = + ipa_->configurationFile(sensor_->model() + ".yaml", "uncalibrated.yaml"); + + IPACameraSensorInfo sensorInfo{}; + int ret = sensor_->sensorInfo(&sensorInfo); + if (ret) { + LOG(RkISP2, Error) << "Camera sensor information not available"; + return ret; + } + + ret = ipa_->init({ ipaTuningFile, sensor_->model() }, + sensorInfo, sensor_->controls(), + &ipaControls_); + if (ret < 0) { + LOG(RkISP2, Error) << "IPA initialization failure"; + return ret; + } + + return 0; +} + +void RkISP2CameraData::computeParamBuffers(unsigned int frame) +{ + while (paramQueue_->nextSequence() <= frame) { + if (paramQueue_->empty(BufferQueue::Idle)) { + LOG(RkISP2, Warning) << "Out of param buffers"; + return; + } + + uint32_t paramsSequence; + FrameBuffer *paramBuffer = paramQueue_->front(BufferQueue::Idle); + paramQueue_->prepareBuffer(¶msSequence); + ipa_->computeParams(paramsSequence, paramBuffer->cookie()); + } +} + +void RkISP2CameraData::vicapBufferReady(FrameBuffer *buffer) +{ + Request *request = buffer->request(); + + if (!mainPath_ || isRaw_) { + pipe()->completeBuffer(request, buffer); + pipe()->completeRequest(request); + return; + } + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) + return; + + rawrd_->queueBuffer(buffer); +} + +void RkISP2CameraData::rawrdBufferReady(FrameBuffer *buffer) +{ + if (buffer->metadata().status == FrameMetadata::FrameCancelled) + return; + + video_->queueBuffer(buffer); +} + +void RkISP2CameraData::ispBufferReady(FrameBuffer *buffer) +{ + Request *request = buffer->request(); + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) { + syncHelper_.cancelFrame(); + pipe()->completeBuffer(request, buffer); + pipe()->completeRequest(request); + return; + } + + RkISP2RequestInfo info = pendingCompleteRequests_.front(); + pendingCompleteRequests_.pop_front(); + ASSERT(info.request == buffer->request()); + + int droppedFrames = syncHelper_.gotFrame(info.sequence, buffer->metadata().sequence); + if (droppedFrames > 0) { + LOG(RkISP2, Debug) + << "Dropped frames " << droppedFrames << " expected " + << info.sequence << " got " << buffer->metadata().sequence; + } + + pipe()->completeBuffer(request, buffer); + tryCompleteRequest(request, buffer); +} + +void RkISP2CameraData::statBufferReady(FrameBuffer *buffer) +{ + size_t sequence = buffer->metadata().sequence; + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) + return; + + ipa_->processStats(sequence, buffer->cookie(), + delayedCtrls_->get(sequence)); + + stat_->queueBuffer(buffer); +} + +void RkISP2CameraData::paramsComputed(unsigned int frame, + unsigned int bufferId, + unsigned int bytesused) +{ + FrameBuffer *buffer = paramQueue_->front(BufferQueue::Preparing); + + ASSERT(buffer->cookie() == bufferId); + + buffer->_d()->metadata().planes()[0].bytesused = bytesused; + + int ret = paramQueue_->preparedBuffer(); + if (ret < 0) { + LOG(RkISP2, Error) + << "Failed to queue parameter buffer for frame " + << frame << ": " << strerror(-ret); + } +} + +void RkISP2CameraData::setSensorControls([[maybe_unused]] unsigned int frame, + const ControlList &sensorControls) +{ + delayedCtrls_->push(sensorControls); +} + +void RkISP2CameraData::metadataReady(const ControlList &metadata) +{ + tryCompleteRequest(metadata); +} + +void RkISP2CameraData::tryCompleteRequest(Request *request, FrameBuffer *buffer) +{ + if (pendingCompleteBuffers_.empty()) { + pendingCompleteBuffers_.emplace_back(request, buffer); + return; + } + + RkISP2FrameInfo &info = pendingCompleteBuffers_.front(); + if (info.request != nullptr || info.mainPathBuffer != nullptr || !info.metadataProcessed) { + pendingCompleteBuffers_.emplace_back(request, buffer); + return; + } + + info = pendingCompleteBuffers_.front(); + request->_d()->metadata().merge(info.metadata); + pendingCompleteBuffers_.pop_front(); + + pipe()->completeRequest(request); +} + +void RkISP2CameraData::tryCompleteRequest(const ControlList &metadata) +{ + if (pendingCompleteBuffers_.empty()) { + pendingCompleteBuffers_.emplace_back(nullptr, nullptr, metadata, true); + return; + } + + RkISP2FrameInfo &info = pendingCompleteBuffers_.front(); + if (!info.metadata.empty() || info.metadataProcessed) { + /* + * Generally more metadata complete than images, so flush the + * queue when adding new metadata to to the queue to prevent + * metadata in the queue from becoming too old + */ + pendingCompleteBuffers_.clear(); + pendingCompleteBuffers_.emplace_back(nullptr, nullptr, metadata, true); + return; + } + + Request *request = info.request; + request->_d()->metadata().merge(metadata); + pendingCompleteBuffers_.pop_front(); + + pipe()->completeRequest(request); +} + +RkISP2CameraConfiguration::RkISP2CameraConfiguration(const RkISP2CameraData *data) + : CameraConfiguration(), data_(data) +{ +} + +CameraConfiguration::Status RkISP2CameraConfiguration::validate() +{ + const CameraSensor *sensor = data_->sensor_; + std::vector mbusCodes; + Status status = Valid; + + if (config_.empty()) + return Invalid; + + /* + * Make sure that if a sensor configuration has been requested it + * is valid. + */ + if (sensorConfig) { + if (!sensorConfig->isValid()) { + LOG(RkISP2, Error) + << "Invalid sensor configuration request"; + + return Invalid; + } + + unsigned int bitDepth = sensorConfig->bitDepth; + if (bitDepth != 8 && bitDepth != 10 && bitDepth != 12) { + LOG(RkISP2, Error) + << "Invalid sensor configuration bit depth"; + + return Invalid; + } + } + + /* \todo Support self path */ + if (config_.size() != 1) { + config_.resize(1); + status = Adjusted; + } + + StreamConfiguration &cfg = config_[0]; + + /* \todo Support YUV sensors */ + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + bool usingIsp = data_->usingIsp_; + if (usingIsp && info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) + usingIsp = false; + + const Size &maxSize = usingIsp ? ispMaxSize : vicapMaxSize; + + if (!usingIsp) { + if (!rawFormats.count(cfg.pixelFormat)) { + cfg.pixelFormat = formats::SRGGB10; + status = Adjusted; + } + + unsigned int mbusCode = rawFormats.at(cfg.pixelFormat); + auto sizes = sensor->sizes(mbusCode); + + Size bestSize; + for (const Size &s : sizes) { + /* Ignore smaller sizes. */ + if (s.width < cfg.size.width || + s.height < cfg.size.height) + continue; + + /* Make sure the width stays in the limits. */ + if (s.width > maxSize.width) + continue; + + bestSize = s; + break; + } + + if (bestSize.isNull()) { + LOG(RkISP2, Error) << "Unable to find a suitable sensor format"; + return Invalid; + } + + if (bestSize != cfg.size) + status = Adjusted; + cfg.size = bestSize; + + mbusCodes = { mbusCode }; + sensorFormat_ = sensor->getFormat(mbusCodes, cfg.size, maxSize); + + ASSERT(sensorFormat_.code == mbusCode); + ASSERT(sensorFormat_.size == cfg.size); + + return status; + } + + if (!formatToMediaBus.count(cfg.pixelFormat)) { + cfg.pixelFormat = formats::UYVY; + status = Adjusted; + } + + /* \todo Adjust sizes a bit better */ + if (cfg.size > ispMaxSize) { + cfg.size = ispMaxSize; + status = Adjusted; + } + V4L2DeviceFormat format; + format.fourcc = data_->video_->toV4L2PixelFormat(cfg.pixelFormat); + format.size = cfg.size; + + int ret = data_->mainPath_->tryFormat(&format); + if (ret) + return Invalid; + + cfg.bufferCount = 4; + + std::transform(rawFormats.begin(), rawFormats.end(), + std::back_inserter(mbusCodes), + [](const auto &value) { return value.second; }); + sensorFormat_ = sensor->getFormat(mbusCodes, cfg.size, maxSize); + if (sensorFormat_.size.isNull()) + status = Invalid; + + return status; +} + +PipelineHandlerRkISP2::PipelineHandlerRkISP2(CameraManager *manager) + : PipelineHandler(manager) +{ +} + +Size PipelineHandlerRkISP2::clampSensorSize(RkISP2CameraData *data, unsigned int mbus, + const Size &maxSize) +{ + const std::vector &sizes = data->sensor_->sizes(mbus); + + for (auto it = sizes.rbegin(); it != sizes.rend(); ++it) { + if (it->width <= maxSize.width && + it->height <= maxSize.height) + return *it; + } + + return *sizes.begin(); +} + +std::unique_ptr +PipelineHandlerRkISP2::generateConfiguration(Camera *camera, + Span roles) +{ + RkISP2CameraData *data = cameraData(camera); + auto config = std::make_unique(data); + if (roles.empty()) + return config; + + if (roles.size() > 1) { + LOG(RkISP2, Error) << "Too many roles requested"; + return config; + } + + const StreamRole &role = roles[0]; + + bool isRaw = !data->usingIsp_ || role == StreamRole::Raw; + Size maxSize = isRaw ? vicapMaxSize : ispMaxSize; + + Size defaultSize = { 1920, 1080 }; + unsigned int defaultMbusCode = MEDIA_BUS_FMT_SRGGB10_1X10; + PixelFormat rawFormat = data->getSensorFormat(defaultMbusCode, + defaultSize, maxSize); + + /* Enumerate formats */ + std::vector sizes = { { Size(32, 32), maxSize } }; + auto makeStream = [sizes](auto const &pair) { + return std::make_pair(pair.first, sizes); + }; + std::map> streamFormats; + std::transform(formatToMediaBus.begin(), formatToMediaBus.end(), + std::inserter(streamFormats, streamFormats.end()), + makeStream); + std::transform(rawFormats.begin(), rawFormats.end(), + std::inserter(streamFormats, streamFormats.end()), + makeStream); + + StreamFormats formats(streamFormats); + StreamConfiguration cfg(formats); + /* UYVY is always supported by this ISP */ + cfg.pixelFormat = isRaw ? rawFormat : formats::UYVY; + cfg.size = clampSensorSize(data, defaultMbusCode, maxSize); + cfg.colorSpace = isRaw ? ColorSpace::Raw : ColorSpace::Sycc; + cfg.bufferCount = 4; + + config->addConfiguration(cfg); + + config->validate(); + + return config; +} + +int PipelineHandlerRkISP2::configure(Camera *camera, + CameraConfiguration *c) +{ + RkISP2CameraData *data = cameraData(camera); + RkISP2CameraConfiguration *config = + static_cast(c); + /* \todo Support multiple streams */ + StreamConfiguration &cfg = config->at(0); + int ret; + + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + data->isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; + + V4L2SubdeviceFormat format = config->sensorFormat(); + LOG(RkISP2, Debug) << "Configuring sensor with " << format; + + if (config->sensorConfig) + ret = sensor_->applyConfiguration(*config->sensorConfig, + Transform::Identity, &format); + else + ret = sensor_->setFormat(&format); + if (ret < 0) + return ret; + + LOG(RkISP2, Debug) << "Sensor configured with " << format; + + LOG(RkISP2, Debug) << "Configuring CSI with : " << format; + ret = csi_->setFormat(0, &format); + if (ret) + return ret; + + LOG(RkISP2, Debug) << "Configuring VICAP with : " << format; + ret = cif_->setFormat(0, &format); + if (ret) + return ret; + + ret = cif_->setFormat(1, &format); + if (ret) + return ret; + + Size maxSize = data->isRaw_ ? vicapMaxSize : ispMaxSize; + PixelFormat vicapPixelFormat = + data->getSensorFormat(format.code, format.size, maxSize); + + V4L2DeviceFormat vicapOutputFormat; + vicapOutputFormat.fourcc = video_->toV4L2PixelFormat(vicapPixelFormat); + vicapOutputFormat.size = format.size; + + LOG(RkISP2, Debug) << "Configuring VICAP capture node with : " << vicapOutputFormat; + ret = data->video_->setFormat(&vicapOutputFormat); + if (ret) + return ret; + + if (!data->usingIsp_ || data->isRaw_) { + cfg.setStream(&data->stream_); + cfg.stride = vicapOutputFormat.planes[0].bpl; + return 0; + } + + /* + * \todo Figure out if these should go in the pipeline handler + * or camera data + */ + LOG(RkISP2, Debug) << "Configuring rawrd0 with: " << vicapOutputFormat; + ret = rawrd_->setFormat(&vicapOutputFormat); + if (ret) + return ret; + + LOG(RkISP2, Debug) << "Configuring ISP input with: " << format; + ret = isp_->setFormat(0, &format); + if (ret) + return ret; + + format.code = formatToMediaBus.at(cfg.pixelFormat); + format.size = cfg.size; + LOG(RkISP2, Debug) << "Configuring ISP output with: " << format; + ret = isp_->setFormat(5, &format); + if (ret) + return ret; + + V4L2DeviceFormat outputFormat; + outputFormat.fourcc = mainPath_->toV4L2PixelFormat(cfg.pixelFormat); + outputFormat.size = cfg.size; + + LOG(RkISP2, Debug) << "Configuring main path with: " << outputFormat; + ret = mainPath_->setFormat(&outputFormat); + if (ret) + return ret; + + if (outputFormat.size != cfg.size || + outputFormat.fourcc != data->mainPath_->toV4L2PixelFormat(cfg.pixelFormat)) { + LOG(RkISP2, Error) + << "Unable to configure capture in " << cfg.toString(); + return -EINVAL; + } + + cfg.setStream(&data->stream_); + cfg.stride = outputFormat.planes[0].bpl; + + V4L2DeviceFormat paramFormat; + paramFormat.fourcc = V4L2PixelFormat(V4L2_META_FMT_RKISP2_PARAMS); + ret = param_->setFormat(¶mFormat); + if (ret) + return ret; + + V4L2DeviceFormat statFormat; + statFormat.fourcc = V4L2PixelFormat(V4L2_META_FMT_RKISP2_STATS); + ret = stat_->setFormat(&statFormat); + if (ret) + return ret; + + IPACameraSensorInfo sensorInfo; + ret = data->sensor_->sensorInfo(&sensorInfo); + if (ret) + return ret; + + int colorSpaceEncoding = -1; + int colorSpaceRange = -1; + if (cfg.colorSpace) { + colorSpaceEncoding = static_cast(cfg.colorSpace->ycbcrEncoding); + colorSpaceRange = static_cast(cfg.colorSpace->range); + } + + /* Inform IPA of stream configuration and sensor controls. */ + ipa::rkisp2::IPAConfigInfo ipaConfig{ sensorInfo, + data->sensor_->controls(), + colorSpaceEncoding, + colorSpaceRange }; + + ret = data->ipa_->configure(ipaConfig, &data->ipaControls_); + if (ret) { + LOG(RkISP2, Error) << "failed configuring IPA (" << ret << ")"; + return ret; + } + + return updateControls(data); +} + +int PipelineHandlerRkISP2::exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) +{ + unsigned int count = stream->configuration().bufferCount; + RkISP2CameraData *data = cameraData(camera); + + if (!data->usingIsp_ || data->isRaw_) + return data->video_->exportBuffers(count, buffers); + + return data->mainPath_->exportBuffers(count, buffers); +} + +int PipelineHandlerRkISP2::start(Camera *camera, + const ControlList *controls) +{ + RkISP2CameraData *data = cameraData(camera); + unsigned int count = data->stream_.configuration().bufferCount; + bool useMP = data->usingIsp_ && !data->isRaw_; + utils::ScopeExitActions actions; + int ret; + + data->frame_ = 0; + + LOG(RkISP2, Debug) << (useMP ? "Using" : "Not using") << " main path"; + + if (useMP) { + /* Allocate buffers for params and stats */ + ret = allocateBuffers(camera); + if (ret) { + LOG(RkISP2, Error) << "Failed to allocate buffers"; + return ret; + } + actions += [&]() { freeBuffers(camera); }; + + /* \todo Support start controls */ + ret = data->ipa_->start(); + if (ret) { + LOG(RkISP2, Error) + << "Failed to start IPA " << camera->id(); + return ret; + } + actions += [&]() { data->ipa_->stop(); }; + + ret = data->param_->streamOn(); + if (ret) { + LOG(RkISP2, Error) + << "Failed to start parameters " << camera->id(); + return ret; + } + actions += [&]() { data->param_->streamOff(); }; + } + + ret = data->video_->importBuffers(kRkISP2MinBufferCount); + if (ret < 0) { + LOG(RkISP2, Error) << "Failed to import buffers to vicap"; + return ret; + } + + actions += [&]() { data->video_->releaseBuffers(); }; + + if (useMP) { + ret = data->rawrd_->importBuffers(count); + if (ret < 0) { + LOG(RkISP2, Error) << "Failed to import buffers to rawrd"; + return ret; + } + + actions += [&]() { data->rawrd_->releaseBuffers(); }; + + ret = data->mainPath_->importBuffers(count); + if (ret < 0) { + LOG(RkISP2, Error) << "Failed to import buffers to main path"; + return ret; + } + + actions += [&]() { data->mainPath_->releaseBuffers(); }; + + auto queueBuffers = [&](const std::vector> &buffers, + V4L2VideoDevice *device, std::string_view name) { + for (const std::unique_ptr &buffer : buffers) { + ret = device->queueBuffer(buffer.get()); + if (ret < 0) { + LOG(RkISP2, Warning) + << "Failed to queue buffer " + << &buffer << " to " << name + << ": " << ret; + } + } + }; + + queueBuffers(data->internalBuffers_, data->video_, "vicap"); + queueBuffers(data->statBuffers_, data->stat_, "stat"); + } + + ret = data->video_->streamOn(); + if (ret < 0) + return ret; + + actions += [&]() { data->video_->streamOff(); }; + + if (useMP) { + ret = data->rawrd_->streamOn(); + if (ret < 0) + return ret; + + actions += [&]() { data->rawrd_->streamOff(); }; + + ret = data->stat_->streamOn(); + if (ret) { + LOG(RkISP2, Error) + << "Failed to start stats " << camera->id(); + return ret; + } + actions += [&]() { data->stat_->streamOff(); }; + + ret = data->mainPath_->streamOn(); + if (ret < 0) + return ret; + + actions += [&]() { data->mainPath_->streamOff(); }; + } + + if (controls) { + ret = processControls(data, *controls); + if (ret < 0) + return ret; + } + + if (useMP) + data->isp_->setFrameStartEnabled(true); + + actions.release(); + return 0; +} + +void PipelineHandlerRkISP2::stopDevice(Camera *camera) +{ + RkISP2CameraData *data = cameraData(camera); + bool useMP = data->usingIsp_ && !data->isRaw_; + + if (useMP) + data->isp_->setFrameStartEnabled(false); + + data->video_->streamOff(); + data->video_->releaseBuffers(); + + if (!useMP) + return; + + data->ipa_->stop(); + + data->rawrd_->streamOff(); + data->rawrd_->releaseBuffers(); + + data->mainPath_->streamOff(); + data->mainPath_->releaseBuffers(); + + data->stat_->streamOff(); + data->stat_->releaseBuffers(); + + data->param_->streamOff(); + + data->internalBuffers_.clear(); + data->statBuffers_.clear(); + freeBuffers(camera); +} + +int PipelineHandlerRkISP2::queueRequestDevice(Camera *camera, Request *request) +{ + RkISP2CameraData *data = cameraData(camera); + FrameBuffer *buffer = request->findBuffer(&data->stream_); + if (!buffer) { + LOG(RkISP2, Error) + << "Attempt to queue request with invalid stream"; + return -ENOENT; + } + + int ret = processControls(data, request->controls()); + if (ret < 0) + return ret; + + if (!data->usingIsp_ || data->isRaw_) + return data->video_->queueBuffer(buffer); + + int correction = data->syncHelper_.correction(); + data->frame_ += correction; + data->syncHelper_.pushCorrection(correction); + + data->pendingCompleteRequests_.push_back({ request, data->frame_, true }); + + data->ipa_->queueRequest(data->frame_, request->controls()); + data->computeParamBuffers(data->frame_); + data->frame_++; + + return data->mainPath_->queueBuffer(buffer); +} + +int PipelineHandlerRkISP2::updateControls(RkISP2CameraData *data) +{ + ControlInfoMap::Map controls; + + /* Add the pipeline handler registered controls to list of camera controls. */ + for (const auto &phControl : data->controlInfo_) + controls[phControl.first] = phControl.second; + + /* Add the IPA registered controls to list of camera controls. */ + for (const auto &ipaControl : data->ipaControls_) + controls[ipaControl.first] = ipaControl.second; + + data->controlInfo_ = ControlInfoMap(std::move(controls), + controls::controls); + + return 0; +} + +bool PipelineHandlerRkISP2::createCamera(bool usingIsp) +{ + std::unique_ptr data = + std::make_unique(this, media_.get()); + if (data->init(usingIsp, sensor_.get(), video_.get(), rawrd_.get(), + isp_.get(), mainPath_.get(), param_.get(), stat_.get())) { + LOG(RkISP2, Error) << "Failed to initialize data"; + return false; + } + + /* Initialize the camera properties. */ + data->properties_ = data->sensor_->properties(); + + const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, + { V4L2_CID_VBLANK, { delays.vblankDelay, true } }, + }; + + if (usingIsp) { + data->delayedCtrls_ = + std::make_unique(data->sensor_->device(), + params); + isp_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + + int ret = data->loadIPA(); + if (ret) { + LOG(RkISP2, Error) << "Failed to load IPA"; + return false; + } + + updateControls(data.get()); + + data->paramQueue_ = + std::make_unique(std::make_unique>(param_.get()), + BufferQueue::PrepareStage, "Params"); + } + + const std::string &id = data->sensor_->id(); + std::set streams{ &data->stream_ }; + std::shared_ptr camera = + Camera::create(std::move(data), id, streams); + registerCamera(std::move(camera)); + + LOG(RkISP2, Debug) + << "RkISP2 device registered " + << (usingIsp ? "with" : "without") << " ISP"; + + return true; +} + +bool PipelineHandlerRkISP2::match(DeviceEnumerator *enumerator) +{ + DeviceMatch dm("rockchip-cif"); + /* \todo Generalize this for the other csi ports */ + /* + * I think we will have one camera per csi port, and then 4 streams + * each that correspond to the 4 channels. Not sure how to handle + * routing to the ISP though. For now we'll just assume one camera. + */ + dm.add("rkcif-mipi2"); + /* \todo Generalize this for the other channels */ + dm.add("rkcif-mipi2-id0"); + dm.add("dw-mipi-csi2rx fdd30000.csi"); + + media_ = acquireMediaDevice(enumerator, dm); + if (!media_) + return false; + + csi_ = V4L2Subdevice::fromEntityName(media_.get(), "dw-mipi-csi2rx fdd30000.csi"); + if (!csi_ || csi_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open csi"; + return false; + } + + cif_ = V4L2Subdevice::fromEntityName(media_.get(), "rkcif-mipi2"); + if (!cif_ || cif_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open cif"; + return false; + } + + /* \todo Support multiple streams */ + video_ = V4L2VideoDevice::fromEntityName(media_.get(), "rkcif-mipi2-id0"); + if (!video_ || video_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open capture device"; + return false; + } + + for (MediaEntity *entity : media_->locateEntities(MEDIA_ENT_F_CAM_SENSOR)) { + LOG(RkISP2, Debug) << "Identified " << entity->name(); + sensor_ = CameraSensorFactoryBase::create(entity); + /* Just get the first sensor for now */ + if (sensor_) + break; + } + + if (!sensor_) { + LOG(RkISP2, Error) << "Failed to find sensor"; + return false; + } + + const GlobalConfiguration &configuration = cameraManager()->_d()->configuration(); + bool usingIsp = configuration.configuration()["pipelines"]["rkisp2"]["isp_enable"].get(true); + if (!usingIsp) { + LOG(RkISP2, Info) << "ISP disabled in configuration file"; + return createCamera(usingIsp); + } + + /* Match ISP */ + + DeviceMatch dmIsp("rkisp2"); + dmIsp.add("rkisp2_isp"); + /* \todo Generalize this for the other channels */ + dmIsp.add("rkisp2_rawrd0"); + /* \todo Support self path */ + dmIsp.add("rkisp2_mainpath"); + + ispMedia_ = acquireMediaDevice(enumerator, dmIsp); + if (!ispMedia_) { + usingIsp = false; + LOG(RkISP2, Debug) << "ISP not found"; + return createCamera(usingIsp); + } + + /* \todo Support the other rawrd nodes */ + rawrd_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_rawrd0"); + if (!rawrd_ || rawrd_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 rawrd device"; + return false; + } + + isp_ = V4L2Subdevice::fromEntityName(ispMedia_.get(), "rkisp2_isp"); + if (!isp_ || isp_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 isp"; + return false; + } + + /* \todo Support self path */ + mainPath_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_mainpath"); + if (!mainPath_ || mainPath_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 main path"; + return false; + } + + param_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_params"); + if (!param_ || param_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 params"; + return false; + } + + stat_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_stats"); + if (!stat_ || stat_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 stats"; + return false; + } + + return createCamera(true); +} + +int PipelineHandlerRkISP2::processControls(RkISP2CameraData *data, const ControlList &ctrls) +{ + const auto &testPattern = ctrls.get(controls::draft::TestPatternMode); + if (testPattern) + data->sensor_->setTestPatternMode(static_cast(*testPattern)); + + return 0; +} + +/* This is only called when using the ISP */ +int PipelineHandlerRkISP2::allocateBuffers(Camera *camera) +{ + RkISP2CameraData *data = cameraData(camera); + unsigned int ipaBufferId = 1; + utils::ScopeExitActions actions; + + int ret = data->video_->exportBuffers(kRkISP2MinBufferCount, &data->internalBuffers_); + if (ret < 0) + return ret; + + actions += [&]() { data->video_->releaseBuffers(); }; + + ret = data->stat_->allocateBuffers(kRkISP2MinBufferCount, &data->statBuffers_); + if (ret < 0) + return ret; + + actions += [&]() { data->stat_->releaseBuffers(); }; + + ret = data->paramQueue_->allocateBuffers(kRkISP2MinBufferCount); + if (ret < 0) + return ret; + + actions += [&]() { data->paramQueue_->releaseBuffers(); }; + + auto pushBuffers = [&](const std::vector> &buffers) { + for (const std::unique_ptr &buffer : buffers) { + Span planes = buffer->planes(); + + buffer->setCookie(ipaBufferId++); + data->ipaBuffers_.emplace_back(buffer->cookie(), + std::vector{ planes.begin(), + planes.end() }); + } + }; + + pushBuffers(data->paramQueue_->buffers()); + pushBuffers(data->statBuffers_); + + data->ipa_->mapBuffers(data->ipaBuffers_); + + actions.release(); + return 0; +} + +int PipelineHandlerRkISP2::freeBuffers(Camera *camera) +{ + RkISP2CameraData *data = cameraData(camera); + + std::vector ids; + for (IPABuffer &ipabuf : data->ipaBuffers_) + ids.push_back(ipabuf.id); + + data->ipa_->unmapBuffers(ids); + data->ipaBuffers_.clear(); + + data->paramQueue_->releaseBuffers(); + + return 0; +} + +REGISTER_PIPELINE_HANDLER(PipelineHandlerRkISP2, "rkisp2") + +} /* namespace libcamera */ From patchwork Mon Aug 24 09:06:25 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28019 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9BA24C3339 for ; Mon, 24 Aug 2026 09:07:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4FD9A683B2; Mon, 24 Aug 2026 11:07:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DsMo6F33"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1367D68392 for ; Mon, 24 Aug 2026 11:07:22 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BB000C23; Mon, 24 Aug 2026 11:05:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562357; bh=LwU47QUkDJUKvTxCsltyLYQegARHTOtCF6u48eSM+bE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DsMo6F33wAPEM7F0vK6e+wo/0K3QvnzjBGwGJNSiGiP2w1r81qQj87XaXT9k/JELE 7sdFlbK1zNDgfd6nPA0ZHjE94jIqQDA96XZoRDO0p6tWKC5VWcaJ6nkQL+0a5m8oZK 3QOxnGOgumO24vSklEB4+ZUvsyABvO4nrXhxo7GU= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 08/20] ipa: rkisp2: algo: bls: Implement black level subtraction Date: Mon, 24 Aug 2026 18:06:25 +0900 Message-ID: <20260824090641.3705246-9-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a static black level subtraction algorithm for the rkisp2 IPA. Signed-off-by: Paul Elder --- No change in v2 --- src/ipa/rkisp2/algorithms/bls.cpp | 154 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/bls.h | 41 +++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 196 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/bls.cpp create mode 100644 src/ipa/rkisp2/algorithms/bls.h diff --git a/src/ipa/rkisp2/algorithms/bls.cpp b/src/ipa/rkisp2/algorithms/bls.cpp new file mode 100644 index 000000000000..316c057d1046 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/bls.cpp @@ -0,0 +1,154 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Black Level Subtraction control + */ + +#include "bls.h" + +#include + +#include + +#include + +/** + * \file bls.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class BlackLevelSubtraction + * \brief RkISP2 Black Level Subtraction control + * + * The pixels output by the camera normally include a black level, because + * sensors do not always report a signal level of '0' for black. Pixels at or + * below this level should be considered black. To achieve that, the RkISP2 BLS + * algorithm subtracts a configurable offset from all pixels. + * + * The black level can be measured at runtime from an optical dark region of the + * camera sensor, or measured during the camera tuning process. The first option + * isn't currently supported. + * + * \todo Add support for black level in the metadata so that we can capture + * proper raw images for tuning + */ + +LOG_DEFINE_CATEGORY(RkISP2Bls) + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int BlackLevelSubtraction::init(IPAContext &context, const ValueNode &tuningData) +{ + std::optional levelRed = tuningData["R"].get(); + std::optional levelGreenR = tuningData["Gr"].get(); + std::optional levelGreenB = tuningData["Gb"].get(); + std::optional levelBlue = tuningData["B"].get(); + bool tuningHasLevels = levelRed && levelGreenR && levelGreenB && levelBlue; + + auto blackLevel = context.camHelper->blackLevel(); + if (!blackLevel) { + /* + * Not all camera sensor helpers have been updated with black + * levels. Print a warning and fall back to the levels from the + * tuning data to preserve backward compatibility. This should + * be removed once all helpers provide the data. + */ + LOG(RkISP2Bls, Warning) + << "No black levels provided by camera sensor helper" + << ", please fix"; + + blackLevelRed_ = levelRed.value_or(4096); + blackLevelGreenR_ = levelGreenR.value_or(4096); + blackLevelGreenB_ = levelGreenB.value_or(4096); + blackLevelBlue_ = levelBlue.value_or(4096); + } else if (tuningHasLevels) { + /* + * If black levels are provided in the tuning file, use them to + * avoid breaking existing camera tuning. This is deprecated and + * will be removed. + */ + LOG(RkISP2Bls, Warning) + << "Deprecated: black levels overwritten by tuning file"; + + blackLevelRed_ = *levelRed; + blackLevelGreenR_ = *levelGreenR; + blackLevelGreenB_ = *levelGreenB; + blackLevelBlue_ = *levelBlue; + } else { + blackLevelRed_ = *blackLevel; + blackLevelGreenR_ = *blackLevel; + blackLevelGreenB_ = *blackLevel; + blackLevelBlue_ = *blackLevel; + } + + LOG(RkISP2Bls, Debug) + << "Black levels: red " << blackLevelRed_ + << ", green (red) " << blackLevelGreenR_ + << ", green (blue) " << blackLevelGreenB_ + << ", blue " << blackLevelBlue_; + + return 0; +} + +int BlackLevelSubtraction::configure([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + /* \todo Save the cfa */ + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void BlackLevelSubtraction::prepare(IPAContext &context, + const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + RkISP2Params *params) +{ + if (context.configuration.raw) + return; + + if (frame > 1) + return; + + auto config = params->block(); + config.setEnabled(true); + + config->enable_auto = 0; + + /* Scale down to the 12-bit black levels used by the BLS block. */ + /* \todo Handle cfa properly */ + config->bls_fixed_val.a = blackLevelRed_ >> 4; + config->bls_fixed_val.b = blackLevelGreenR_ >> 4; + config->bls_fixed_val.c = blackLevelGreenB_ >> 4; + config->bls_fixed_val.d = blackLevelBlue_ >> 4; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void BlackLevelSubtraction::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] const RkISP2Stats *stats, + ControlList &metadata) +{ + metadata.set(controls::SensorBlackLevels, + { static_cast(blackLevelRed_), + static_cast(blackLevelGreenR_), + static_cast(blackLevelGreenB_), + static_cast(blackLevelBlue_) }); +} + +REGISTER_IPA_ALGORITHM(BlackLevelSubtraction, "BlackLevelSubtraction") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/bls.h b/src/ipa/rkisp2/algorithms/bls.h new file mode 100644 index 000000000000..31f3fe21c8c7 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/bls.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Black Level Subtraction control + */ + +#pragma once + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class BlackLevelSubtraction : public Algorithm +{ +public: + BlackLevelSubtraction() = default; + ~BlackLevelSubtraction() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) override; + +private: + int16_t blackLevelRed_; + int16_t blackLevelGreenR_; + int16_t blackLevelGreenB_; + int16_t blackLevelBlue_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 4bb81e48eb01..36996918b39e 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 rkisp2_ipa_algorithms = files([ + 'bls.cpp', ]) From patchwork Mon Aug 24 09:06:26 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28020 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7085DC3339 for ; Mon, 24 Aug 2026 09:07:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2E785683B2; Mon, 24 Aug 2026 11:07:27 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rLlRyXpV"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C22B867FEC for ; Mon, 24 Aug 2026 11:07:25 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6C9BDC31; Mon, 24 Aug 2026 11:05:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562361; bh=H2qLjTk1QDZE1mPMgumstaTmrps3NMDIHuuUl/2jFVE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rLlRyXpVdhTMq4w3UITm3DAJYSwPymZXWJJfI3OPEApZ2ypMAQfC6UmeRsOdEUppb bn1R6MTfBH/n7cbpH3GW9fdPbry9q8a2c8/maKfpXIWeegykzBZNtZ5RC0NP+S6R7B TAYQFt+3Pp1GXlgQDWsGOFZmPrb8PqB+dffFXNBk= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 09/20] ipa: rkisp2: algo: awb: Implement automatic white balance control Date: Mon, 24 Aug 2026 18:06:26 +0900 Message-ID: <20260824090641.3705246-10-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement auto white balance control algorithm for the rkisp2 IPA. This leverages the libipa awb, so it supports baysian awb, grey world awb, as well as manual white balance. Signed-off-by: Paul Elder --- Changes in v2: - rebased on libipa rework - use extensible stats --- src/ipa/rkisp2/algorithms/awb.cpp | 211 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/awb.h | 56 +++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 268 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/awb.cpp create mode 100644 src/ipa/rkisp2/algorithms/awb.h diff --git a/src/ipa/rkisp2/algorithms/awb.cpp b/src/ipa/rkisp2/algorithms/awb.cpp new file mode 100644 index 000000000000..e3b284a2d79f --- /dev/null +++ b/src/ipa/rkisp2/algorithms/awb.cpp @@ -0,0 +1,211 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * AWB control algorithm + */ + +#include "awb.h" + +#include + +#include + +#include + +#include "libcamera/internal/vector.h" + +/** + * \file awb.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +LOG_DEFINE_CATEGORY(RkISP2Awb) + +class RkISP2AwbStats final : public AwbStats +{ +public: + RkISP2AwbStats() = default; + RkISP2AwbStats(const RGB means) + { + rgbMeans_[0] = means.r(); + rgbMeans_[1] = means.g(); + rgbMeans_[2] = means.b(); + + rg_ = rgbMeans_.r(); + bg_ = rgbMeans_.b(); + } + + double computeColourError(const RGB &gains) const override + { + /* + * Compute the sum of the squared colour error (non-greyness) as + * it appears in the log likelihood equation. + */ + double deltaR = gains.r() * rg_ - 1.0; + double deltaB = gains.b() * bg_ - 1.0; + double delta2 = deltaR * deltaR + deltaB * deltaB; + + return delta2; + } + + RGB rgbMeans() const override + { + return rgbMeans_; + } + + bool valid() const override + { + /* Minimum mean value below which AWB can't operate. */ + constexpr double minValue = 0.2; + + return rgbMeans_.r() > minValue || rgbMeans_.g() > minValue || + rgbMeans_.b() > minValue; + } + +private: + RGB rgbMeans_; + double rg_; + double bg_; +}; + +namespace { + +} /* namespace */ + +/** + * \class Awb + * \brief Manage the white balance with automatic and manual controls + */ + +Awb::Awb() +{ +} + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Awb::init(IPAContext &context, const ValueNode &tuningData) +{ + return awbAlgo_.init(tuningData, context.ctrlMap); +} + +int Awb::configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) +{ + awbAlgo_.configure(context.activeState.awb); + + context.configuration.awb.measureWindow.h_offs = 0; + context.configuration.awb.measureWindow.v_offs = 0; + /* + * Unlike ae lite, this seems to still work when height == full window + * height + */ + context.configuration.awb.measureWindow.h_size = configInfo.outputSize.width / 15; + context.configuration.awb.measureWindow.v_size = configInfo.outputSize.height / 15; + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Awb::queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + awbAlgo_.queueRequest(context.activeState.awb, frame, frameContext.awb, + controls); +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Awb::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, RkISP2Params *params) +{ + awbAlgo_.prepare(context.activeState.awb, frameContext.awb); + + auto gainConfig = params->block(); + gainConfig.setEnabled(true); + + RGB gains = frameContext.awb.gains; + + /* \todo Use quantized class */ + gainConfig->gains[0].gb = std::clamp(256 * 1.0, 0, 0x3fff); + gainConfig->gains[0].b = std::clamp(256 * gains.b(), 0, 0x3fff); + gainConfig->gains[0].r = std::clamp(256 * gains.r(), 0, 0x3fff); + gainConfig->gains[0].gr = std::clamp(256 * 1.0, 0, 0x3fff); + + auto measConfig = params->block(); + measConfig.setEnabled(true); + + measConfig->meas_window = context.configuration.awb.measureWindow; + struct rkisp2_isp_awb_color_quad minLimits = { 0, 0, 0, 0 }; + struct rkisp2_isp_awb_color_quad maxLimits = { 255, 255, 255, 255 }; + measConfig->limits[0] = minLimits; + measConfig->limits[1] = maxLimits; + + for (unsigned int i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + measConfig->weights[i] = 0x20; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Awb::process([[maybe_unused]] IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) +{ + /* The first frame has no stats so skip processing */ + if (frame < 1) + return; + + RkISP2AwbStats awbStats = calculateRgbMeans(frameContext, stats); + + awbAlgo_.process(context.activeState.awb, frameContext.awb, awbStats, + frameContext.lux.lux, metadata); +} + +RkISP2AwbStats Awb::calculateRgbMeans([[maybe_unused]] const IPAFrameContext &frameContext, + const RkISP2Stats *stats) const +{ + auto awb = stats->block(); + + if (!awb->done) { + LOG(RkISP2Awb, Error) << "No awb stats"; + return {}; + } + + std::array counts; + RGB means; + + for (size_t i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + counts[i] = static_cast(awb->counts_r[i]); + means.r() = std::accumulate(counts.begin(), counts.end(), 0) / counts.size(); + + for (size_t i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + counts[i] = static_cast(awb->counts_g[i]); + means.g() = std::accumulate(counts.begin(), counts.end(), 0) / counts.size(); + + for (size_t i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + counts[i] = static_cast(awb->counts_b[i]); + means.b() = std::accumulate(counts.begin(), counts.end(), 0) / counts.size(); + + for (size_t i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + counts[i] = static_cast(awb->counts_w[i]); + + return RkISP2AwbStats(means); +} + +REGISTER_IPA_ALGORITHM(Awb, "Awb") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/awb.h b/src/ipa/rkisp2/algorithms/awb.h new file mode 100644 index 000000000000..42c9f1193795 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/awb.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * AWB control algorithm + */ + +#pragma once + +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include "libipa/awb.h" +#include "libipa/fixedpoint.h" + +#include "algorithm.h" +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class RkISP2AwbStats; + +class Awb : public Algorithm +{ +public: + Awb(); + ~Awb() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) override; + +private: + RkISP2AwbStats calculateRgbMeans(const IPAFrameContext &frameContext, + const RkISP2Stats *stats) const; + + AwbAlgorithm> awbAlgo_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 36996918b39e..30133ac4fa68 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 rkisp2_ipa_algorithms = files([ + 'awb.cpp', 'bls.cpp', ]) From patchwork Mon Aug 24 09:06:27 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28021 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3109BC3339 for ; Mon, 24 Aug 2026 09:07:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E1172683B2; Mon, 24 Aug 2026 11:07:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HGJCgXLR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7127067FEC for ; Mon, 24 Aug 2026 11:07:29 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 21F9FC34; Mon, 24 Aug 2026 11:06:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562365; bh=8HvF8tEaeTyLqSEedTpGGr37MmorUJm5t1uhwUDMDho=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HGJCgXLRDSXANKaXnn9UPAuNuYfPP/egC1aywtg5F/F6ttsazAYs5dWDRLWBhEkGt XU6fXjzhXgQxYL9kB9GM1T0fKaPanFznpDnslGo+dOjBPflpL9mhTv8OxawNefrWuv zJazy5NG2fF+soNOyTt7o3WQyZA4v9JX4+bYc318= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 10/20] ipa: rkisp2: algo: agc: Implement automatic gain control Date: Mon, 24 Aug 2026 18:06:27 +0900 Message-ID: <20260824090641.3705246-11-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement auto gain control algorithm for the rkisp2 IPA. This leverages the libipa agc, and supports manual gain and exposure control as well. It currently does not function in raw mode, meaning that there is no manual gain and exposure control in raw mode, as no IPA is created when running in raw mode for the rkisp2. Signed-off-by: Paul Elder --- todo: support manual gain and exposure in raw mode Changes in v2: - rebased on libipa rework - use hist lite instead of big0 - use extensible stats --- src/ipa/rkisp2/algorithms/agc.cpp | 479 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/agc.h | 58 ++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 538 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/agc.cpp create mode 100644 src/ipa/rkisp2/algorithms/agc.h diff --git a/src/ipa/rkisp2/algorithms/agc.cpp b/src/ipa/rkisp2/algorithms/agc.cpp new file mode 100644 index 000000000000..4a7f111ccf0d --- /dev/null +++ b/src/ipa/rkisp2/algorithms/agc.cpp @@ -0,0 +1,479 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board Oy. + * + * AGC/AEC mean-based control algorithm + */ + +#include "agc.h" + +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include +#include +#include + +#include "libipa/histogram.h" + +/** + * \file agc.h + */ + +namespace libcamera { + +using namespace std::literals::chrono_literals; + +namespace ipa::rkisp2::algorithms { + +/** + * \class Agc + * \brief A mean-based auto-exposure algorithm + */ + +LOG_DEFINE_CATEGORY(RkISP2Agc) + +int Agc::init(IPAContext &context, const ValueNode &tuningData) +{ + int ret = parseTuningData(tuningData); + if (ret) + return ret; + + context.ctrlMap[&controls::ExposureTimeMode] = + ControlInfo({ { ControlValue(controls::ExposureTimeModeAuto), + ControlValue(controls::ExposureTimeModeManual) } }, + ControlValue(controls::ExposureTimeModeAuto)); + context.ctrlMap[&controls::AnalogueGainMode] = + ControlInfo({ { ControlValue(controls::AnalogueGainModeAuto), + ControlValue(controls::AnalogueGainModeManual) } }, + ControlValue(controls::AnalogueGainModeAuto)); + context.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); + /* \todo Support AnalogueGain and ExposureTime in raw mode */ + context.ctrlMap.merge(controls()); + + return 0; +} + +/** + * \brief Configure the AGC given a configInfo + * \param[in] context The shared IPA context + * \param[in] configInfo The IPA configuration data + * + * \return 0 + */ +int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) +{ + context.configuration.agc.measureWindow.h_offs = 0; + context.configuration.agc.measureWindow.v_offs = 0; + context.configuration.agc.measureWindow.h_size = (configInfo.outputSize.width / 5); + /* + * ae lite needs the -2 because the total window height must be + * divisible by 2, and it cannot be equal to or greater than the frame + * size, or else the hardware hangs + * + * \todo Move this to the kernel? + * \todo Check if hist lite also needs this + */ + context.configuration.agc.measureWindow.v_size = (configInfo.outputSize.height / 5) - 2; + + context.configuration.agc.measureWindow15.h_size = (configInfo.outputSize.width / 15); + context.configuration.agc.measureWindow15.v_size = (configInfo.outputSize.height / 15) - 2; + + /* Configure the default exposure and gain. */ + context.activeState.agc.automatic.gain = context.configuration.sensor.minAnalogueGain; + context.activeState.agc.automatic.exposure = + 10ms / context.configuration.sensor.lineDuration; + context.activeState.agc.automatic.quantizationGain = 1.0; + context.activeState.agc.manual.gain = context.activeState.agc.automatic.gain; + context.activeState.agc.manual.exposure = context.activeState.agc.automatic.exposure; + context.activeState.agc.autoExposureEnabled = true; + context.activeState.agc.autoGainEnabled = true; + context.activeState.agc.exposureValue = 0.0; + + /* Limit the frame duration to match current initialisation */ + ControlInfo &frameDurationLimits = context.ctrlMap[&controls::FrameDurationLimits]; + context.activeState.agc.minFrameDuration = std::chrono::microseconds(frameDurationLimits.min().get()); + context.activeState.agc.maxFrameDuration = std::chrono::microseconds(frameDurationLimits.max().get()); + + AgcMeanLuminance::configure(context.configuration.sensor.lineDuration, + context.camHelper.get()); + + setLimits(context.configuration.sensor.minExposureTime, + context.configuration.sensor.maxExposureTime, + context.configuration.sensor.minAnalogueGain, + context.configuration.sensor.maxAnalogueGain, {}); + + context.activeState.agc.automatic.yTarget = effectiveYTarget(); + + resetFrameCount(); + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Agc::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + auto &agc = context.activeState.agc; + + const auto &aeEnable = controls.get(controls::ExposureTimeMode); + if (aeEnable && + (*aeEnable == controls::ExposureTimeModeAuto) != agc.autoExposureEnabled) { + agc.autoExposureEnabled = (*aeEnable == controls::ExposureTimeModeAuto); + + LOG(RkISP2Agc, Debug) + << (agc.autoExposureEnabled ? "Enabling" : "Disabling") + << " AGC (exposure)"; + + /* + * If we go from auto -> manual with no manual control + * set, use the last computed value, which we don't + * know until prepare() so save this information. + * + * \todo Check the previous frame at prepare() time + * instead of saving a flag here + */ + if (!agc.autoExposureEnabled && !controls.get(controls::ExposureTime)) + frameContext.agc.autoExposureModeChange = true; + } + + const auto &agEnable = controls.get(controls::AnalogueGainMode); + if (agEnable && + (*agEnable == controls::AnalogueGainModeAuto) != agc.autoGainEnabled) { + agc.autoGainEnabled = (*agEnable == controls::AnalogueGainModeAuto); + + LOG(RkISP2Agc, Debug) + << (agc.autoGainEnabled ? "Enabling" : "Disabling") + << " AGC (gain)"; + /* + * If we go from auto -> manual with no manual control + * set, use the last computed value, which we don't + * know until prepare() so save this information. + */ + if (!agc.autoGainEnabled && !controls.get(controls::AnalogueGain)) + frameContext.agc.autoGainModeChange = true; + } + + const auto &exposure = controls.get(controls::ExposureTime); + if (exposure && !agc.autoExposureEnabled) { + agc.manual.exposure = *exposure * 1.0us + / context.configuration.sensor.lineDuration; + + LOG(RkISP2Agc, Debug) + << "Set exposure to " << agc.manual.exposure; + } + + const auto &gain = controls.get(controls::AnalogueGain); + if (gain && !agc.autoGainEnabled) { + agc.manual.gain = *gain; + + LOG(RkISP2Agc, Debug) << "Set gain to " << agc.manual.gain; + } + + frameContext.agc.autoExposureEnabled = agc.autoExposureEnabled; + frameContext.agc.autoGainEnabled = agc.autoGainEnabled; + + if (!frameContext.agc.autoExposureEnabled) + frameContext.agc.exposure = agc.manual.exposure; + if (!frameContext.agc.autoGainEnabled) + frameContext.agc.gain = agc.manual.gain; + + if (!frameContext.agc.autoExposureEnabled && + !frameContext.agc.autoGainEnabled) + frameContext.agc.quantizationGain = 1.0; + + const auto &exposureValue = controls.get(controls::ExposureValue); + if (exposureValue) + agc.exposureValue = *exposureValue; + frameContext.agc.exposureValue = agc.exposureValue; + + const auto &frameDurationLimits = controls.get(controls::FrameDurationLimits); + if (frameDurationLimits) { + /* Limit the control value to the limits in ControlInfo */ + ControlInfo &limits = context.ctrlMap[&controls::FrameDurationLimits]; + int64_t minFrameDuration = + std::clamp((*frameDurationLimits).front(), + limits.min().get(), + limits.max().get()); + int64_t maxFrameDuration = + std::clamp((*frameDurationLimits).back(), + limits.min().get(), + limits.max().get()); + + agc.minFrameDuration = std::chrono::microseconds(minFrameDuration); + agc.maxFrameDuration = std::chrono::microseconds(maxFrameDuration); + } + frameContext.agc.minFrameDuration = agc.minFrameDuration; + frameContext.agc.maxFrameDuration = agc.maxFrameDuration; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Agc::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, RkISP2Params *params) +{ + uint32_t activeAutoExposure = context.activeState.agc.automatic.exposure; + double activeAutoGain = context.activeState.agc.automatic.gain; + double activeAutoQGain = context.activeState.agc.automatic.quantizationGain; + + /* Populate exposure and gain in auto mode */ + if (frameContext.agc.autoExposureEnabled) { + frameContext.agc.exposure = activeAutoExposure; + frameContext.agc.quantizationGain = activeAutoQGain; + } + if (frameContext.agc.autoGainEnabled) { + frameContext.agc.gain = activeAutoGain; + frameContext.agc.quantizationGain = activeAutoQGain; + } + + /* + * Populate manual exposure and gain from the active auto values when + * transitioning from auto to manual + */ + if (!frameContext.agc.autoExposureEnabled && frameContext.agc.autoExposureModeChange) { + context.activeState.agc.manual.exposure = activeAutoExposure; + frameContext.agc.exposure = activeAutoExposure; + } + if (!frameContext.agc.autoGainEnabled && frameContext.agc.autoGainModeChange) { + context.activeState.agc.manual.gain = activeAutoGain; + frameContext.agc.gain = activeAutoGain; + frameContext.agc.quantizationGain = activeAutoQGain; + } + + frameContext.agc.yTarget = context.activeState.agc.automatic.yTarget; + + if (frame > 1) + return; + + /* + * Configure the AEC measurements. Set the window, measure + * continuously, and estimate Y as (R + G + B) x (85/256). + */ + auto aeLiteConfig = params->block(); + aeLiteConfig.setEnabled(true); + + aeLiteConfig->window_num = 1; + aeLiteConfig->meas_window = context.configuration.agc.measureWindow; + + auto hstConfig = params->block(); + hstConfig.setEnabled(true); + + hstConfig->window_num = 0; + /* \todo choose this based on the bitdepth */ + hstConfig->data_sel = RKISP2_ISP_HISTOGRAM_DATA_SEL_9_2; + hstConfig->mode = RKISP2_ISP_HISTOGRAM_MODE_Y_HISTOGRAM; + /* waterline means to exclude everything above this value */ + hstConfig->waterline = 0x0; + hstConfig->stepsize = 0; + hstConfig->coeffs.r = 0x21; + hstConfig->coeffs.g = 0x20; + hstConfig->coeffs.b = 0x0d; + + hstConfig->meas_window = context.configuration.agc.measureWindow15; + + /* \todo Support configuring the weights */ + for (size_t i = 0; i < RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_BIG; i++) + hstConfig->weights[i] = 0x20; + + auto hstConfigLite = params->block(); + hstConfigLite.setEnabled(false); +} + +void Agc::fillMetadata(IPAContext &context, IPAFrameContext &frameContext, + ControlList &metadata, [[maybe_unused]] const RkISP2Stats *stats) +{ + utils::Duration exposureTime = context.configuration.sensor.lineDuration + * frameContext.sensor.exposure; + metadata.set(controls::AnalogueGain, frameContext.sensor.gain); + metadata.set(controls::ExposureTime, exposureTime.get()); + metadata.set(controls::FrameDuration, frameContext.agc.frameDuration.get()); + metadata.set(controls::ExposureTimeMode, + frameContext.agc.autoExposureEnabled + ? controls::ExposureTimeModeAuto + : controls::ExposureTimeModeManual); + metadata.set(controls::AnalogueGainMode, + frameContext.agc.autoGainEnabled + ? controls::AnalogueGainModeAuto + : controls::AnalogueGainModeManual); + + metadata.set(controls::ExposureValue, frameContext.agc.exposureValue); +} + +double Agc::estimateLuminance(double gain) const +{ + /* + * \todo Enforce this check, since lite-like is 5x5 while big is 15x15, + * but I haven't yet figured out how to use all 15x15 weights. At the + * moment we're running everything lite-like + */ + ASSERT(expMeans_.size() == weights_.size()); + double ySum = 0.0; + double wSum = 0.0; + + /* Sum the averages, saturated to 4095. */ + for (unsigned i = 0; i < expMeans_.size(); i++) { + double w = weights_[i] / 0x10; + ySum += std::min(expMeans_[i] * gain, 4095.0) * w; + wSum += w; + } + + /* \todo Weight with the AWB gains */ + + return ySum / wSum / 4095; +} + +void Agc::processFrameDuration(IPAContext &context, + IPAFrameContext &frameContext, + utils::Duration frameDuration) +{ + IPACameraSensorInfo &sensorInfo = context.sensorInfo; + utils::Duration lineDuration = context.configuration.sensor.lineDuration; + + frameContext.agc.vblank = (frameDuration / lineDuration) - sensorInfo.outputSize.height; + + /* Update frame duration accounting for line length quantization. */ + frameContext.agc.frameDuration = (sensorInfo.outputSize.height + frameContext.agc.vblank) * lineDuration; +} + +/** + * \brief Process RkISP2 statistics, and run AGC operations + * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number + * \param[in] frameContext The current frame context + * \param[in] stats The RKISP2 statistics and ISP results + * \param[out] metadata Metadata for the frame, to be filled by the algorithm + * + * Identify the current image brightness, and use that to estimate the optimal + * new exposure and gain for the scene. + */ +void Agc::process([[maybe_unused]] IPAContext &context, + const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) +{ + /* The first frame has no stats so skip processing */ + if (frame < 1) + return; + + const utils::Duration &lineDuration = context.configuration.sensor.lineDuration; + + auto aeLite = stats->block(); + auto histBig = stats->block(); + + /* + * \todo Verify that the exposure and gain applied by the sensor for + * this frame match what has been requested. This isn't a hard + * requirement for stability of the AGC (the guarantee we need in + * automatic mode is a perfect match between the frame and the values + * we receive), but is important in manual mode. + */ + + /* The lower 5 bits are fractional and meant to be discarded. */ + Histogram hist({ histBig->hist_bins, RKISP2_ISP_HIST_BIN_N_MAX }, + [](uint32_t x) { return x >> 5; }); + + expMeans_.resize(RKISP2_ISP_AE_MEAN_MAX_LITE); + for (size_t i = 0; i < RKISP2_ISP_AE_MEAN_MAX_LITE; i++) { + /* r and b are 0~1023; g is 255~4095 so multiply r and b to match g */ + uint16_t r = aeLite->exp_mean_r[i] * 4; + uint16_t g = aeLite->exp_mean_g[i]; + uint16_t b = aeLite->exp_mean_b[i] * 4; + expMeans_[i] = 0.2126 * r + 0.7152 * g + 0.0722 * b; + } + + /* \todo Support configuring the weights */ + std::vector modeWeights(25, 0x10); + weights_ = { modeWeights.data(), modeWeights.size() }; + + /* + * Set the AGC limits using the fixed exposure time and/or gain in + * manual mode, or the sensor limits in auto mode. + */ + utils::Duration minExposureTime; + utils::Duration maxExposureTime; + double minAnalogueGain; + double maxAnalogueGain; + + if (frameContext.agc.autoExposureEnabled) { + minExposureTime = context.configuration.sensor.minExposureTime; + maxExposureTime = std::clamp(frameContext.agc.maxFrameDuration, + context.configuration.sensor.minExposureTime, + context.configuration.sensor.maxExposureTime); + } else { + minExposureTime = context.configuration.sensor.lineDuration + * frameContext.agc.exposure; + maxExposureTime = minExposureTime; + } + + if (frameContext.agc.autoGainEnabled) { + minAnalogueGain = context.configuration.sensor.minAnalogueGain; + maxAnalogueGain = context.configuration.sensor.maxAnalogueGain; + } else { + minAnalogueGain = frameContext.agc.gain; + maxAnalogueGain = frameContext.agc.gain; + } + + setLimits(minExposureTime, maxExposureTime, minAnalogueGain, maxAnalogueGain, {}); + + /* + * The Agc algorithm needs to know the effective exposure value that was + * applied to the sensor when the statistics were collected. + */ + utils::Duration exposureTime = lineDuration * frameContext.sensor.exposure; + double analogueGain = frameContext.sensor.gain; + utils::Duration effectiveExposureValue = exposureTime * analogueGain; + if (effectiveExposureValue == 0ms) { + LOG(RkISP2Agc, Warning) + << "frame " << frame << ": Effective exposure value is 0: sensor exposure: " + << exposureTime << ", analogue gain: " << analogueGain; + } + + /* \todo Support lux estimation */ + + /* \todo Support setting constraint and exposure modes */ + utils::Duration newExposureTime; + double aGain, qGain, dGain; + std::tie(newExposureTime, aGain, qGain, dGain) = + calculateNewEv(0, 0, hist, effectiveExposureValue); + + LOG(RkISP2Agc, Debug) + << "Divided up exposure time, analogue gain, quantization gain" + << " and digital gain are " << newExposureTime << ", " << aGain + << ", " << qGain << " and " << dGain; + + IPAActiveState &activeState = context.activeState; + /* Update the estimated exposure and gain. */ + activeState.agc.automatic.exposure = newExposureTime / lineDuration; + activeState.agc.automatic.gain = aGain; + activeState.agc.automatic.quantizationGain = qGain; + activeState.agc.automatic.yTarget = effectiveYTarget(); + /* + * Expand the target frame duration so that we do not run faster than + * the minimum frame duration when we have short exposures. + */ + processFrameDuration(context, frameContext, + std::max(frameContext.agc.minFrameDuration, newExposureTime)); + + fillMetadata(context, frameContext, metadata, stats); + expMeans_ = {}; +} + +REGISTER_IPA_ALGORITHM(Agc, "Agc") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/agc.h b/src/ipa/rkisp2/algorithms/agc.h new file mode 100644 index 000000000000..5288b1e7963f --- /dev/null +++ b/src/ipa/rkisp2/algorithms/agc.h @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board Oy. + * + * RkISP2 AGC/AEC mean-based control algorithm + */ + +#pragma once + +#include + +#include + +#include +#include + +#include "libipa/agc_mean_luminance.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class Agc : public Algorithm, public AgcMeanLuminance +{ +public: + Agc() = default; + ~Agc() = default; + + int init(IPAContext &context, [[maybe_unused]] const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) override; + +private: + void fillMetadata(IPAContext &context, IPAFrameContext &frameContext, + ControlList &metadata, const RkISP2Stats *stats); + double estimateLuminance(double gain) const override; + void processFrameDuration(IPAContext &context, + IPAFrameContext &frameContext, + utils::Duration frameDuration); + + std::vector expMeans_; + Span weights_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 30133ac4fa68..027e40dd0346 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 rkisp2_ipa_algorithms = files([ + 'agc.cpp', 'awb.cpp', 'bls.cpp', ]) From patchwork Mon Aug 24 09:06:28 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28022 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 05124C3339 for ; Mon, 24 Aug 2026 09:07:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AE4F0683B2; Mon, 24 Aug 2026 11:07:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BBg8zWY1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2415067FEC for ; Mon, 24 Aug 2026 11:07:33 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CBF55C25; Mon, 24 Aug 2026 11:06:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562368; bh=HfBBxlseHzjRbPUNbWq0VbLNDNjxTj3sV1GradrbhiU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BBg8zWY161jNGkzmSDC23bZqP3Uzh1hew76FxGfiKQgAD8DMhauZjr0VTc+8YOMCA m4dVlUag/W/C7nkb5vmwEq9HSE2B2fmqR2P2Wna6X7iBLRpV8pRwBFWTod+vhjUilg Uzf0nEJnutZGnEVmUJQEFXXxLMbi3i/sMUj/vaGU= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 11/20] ipa: rkisp2: algo: ccm: Implement color correction matrix Date: Mon, 24 Aug 2026 18:06:28 +0900 Message-ID: <20260824090641.3705246-12-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a color correction matrix control algorithm for the rkisp2 IPA. This uses the libipa color correction matrix algorithm. Signed-off-by: Paul Elder --- Changes in v2: - rebased on libipa rework --- src/ipa/rkisp2/algorithms/ccm.cpp | 126 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/ccm.h | 57 ++++++++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 184 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/ccm.cpp create mode 100644 src/ipa/rkisp2/algorithms/ccm.h diff --git a/src/ipa/rkisp2/algorithms/ccm.cpp b/src/ipa/rkisp2/algorithms/ccm.cpp new file mode 100644 index 000000000000..803a22264c04 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/ccm.cpp @@ -0,0 +1,126 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Color Correction Matrix control algorithm + */ + +#include "ccm.h" + +#include + +#include +#include + +#include + +#include + +#include "libipa/interpolator.h" + +/** + * \file ccm.h + * \brief RkISP2 CCM algorithm implementation + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class Ccm + * \brief RkISP2 color correction matrix algorithm + */ + +LOG_DEFINE_CATEGORY(RkISP2Ccm) + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Ccm::init([[maybe_unused]] IPAContext &context, const ValueNode &tuningData) +{ + return ccmAlgo_.init(tuningData, context.ctrlMap); +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Ccm::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + return ccmAlgo_.configure(context.activeState.ccm, + context.activeState.awb.automatic.colourTemperature); +} + +void Ccm::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + ccmAlgo_.queueRequest(context.activeState.ccm, frameContext.ccm, controls); +} + +void Ccm::setParameters(RkISP2Params *params, IPAFrameContext &context) +{ + const Matrix &matrix = context.ccm.ccm; + const Matrix &offsets = context.ccm.offsets; + + auto config = params->block(); + config.setEnabled(true); + + /* + * 4 bit integer and 7 bit fractional, ranging from -8 (0x400) to + * +7.9921875 (0x3ff) + */ + for (unsigned int i = 0; i < 3; i++) { + for (unsigned int j = 0; j < 3; j++) + config->coeff[i][j] = Q<4, 7>(matrix[i][j]).quantized(); + } + + for (unsigned int i = 0; i < 3; i++) + config->offset[i] = offsets[i][0] & 0xfff; + + config->y_coeff[0] = 0x26; + config->y_coeff[1] = 0x4b; + config->y_coeff[2] = 0x0f; + config->inflection_point = 0xa; + + for (unsigned int i = 0; i < 17; i++) + config->alp[i] = 0x400; + + /* \todo Support alp */ + + LOG(RkISP2Ccm, Debug) << "Setting matrix " << matrix; + LOG(RkISP2Ccm, Debug) << "Setting offsets " << offsets; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Ccm::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, RkISP2Params *params) +{ + if (frameContext.awb.autoEnabled) + ccmAlgo_.prepare(context.activeState.ccm, frameContext.ccm, + frame, frameContext.awb.colourTemperature); + + setParameters(params, frameContext); +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Ccm::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const RkISP2Stats *stats, + ControlList &metadata) +{ + ccmAlgo_.process(frameContext.ccm, metadata); +} + +REGISTER_IPA_ALGORITHM(Ccm, "Ccm") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/ccm.h b/src/ipa/rkisp2/algorithms/ccm.h new file mode 100644 index 000000000000..27878f0c187c --- /dev/null +++ b/src/ipa/rkisp2/algorithms/ccm.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Color Correction Matrix control algorithm + */ + +#pragma once + +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include "libipa/ccm.h" +#include "libipa/fixedpoint.h" + +#include "algorithm.h" +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class Ccm : public Algorithm +{ +public: + Ccm() {} + ~Ccm() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) override; + +private: + void parseYaml(const ValueNode &tuningData); + void setParameters(RkISP2Params *params, IPAFrameContext &context); + + CcmAlgorithm> ccmAlgo_; +}; + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 027e40dd0346..e7ae3d163bfb 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -4,5 +4,6 @@ rkisp2_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', 'bls.cpp', + 'ccm.cpp', ]) From patchwork Mon Aug 24 09:06:29 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28023 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8C0C6C3339 for ; Mon, 24 Aug 2026 09:07:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 46EFB683B2; Mon, 24 Aug 2026 11:07:38 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cElIEZiR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CD04D67FEC for ; Mon, 24 Aug 2026 11:07:36 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 81EA9C24; Mon, 24 Aug 2026 11:06:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562372; bh=MQNDnCciWi8vA2LvMmW1b4l7pkNMfCC62ZxWLrcjCiA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cElIEZiRCMD3OuckLsw/Fn35qvsIczOcAqx09aZSFdRrQpCK4ElUOHYxpiq8+ar7x 6jv3G1ppiU9umefB/8xtxkyXCVE0/GFD9AJXUFr/OITkFEncKu1Xkclar5FrA4aShz rUHnExXnuVbPlQT/dzu7+uSb+u/v+McqcfNzIL6Y= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 12/20] ipa: rkisp2: algo: csm: Implement color space conversion Date: Mon, 24 Aug 2026 18:06:29 +0900 Message-ID: <20260824090641.3705246-13-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a static color space conversion algorithm for the rkisp2 IPA. Signed-off-by: Paul Elder --- No change in v2 --- src/ipa/rkisp2/algorithms/csm.cpp | 177 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/csm.h | 34 +++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 212 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/csm.cpp create mode 100644 src/ipa/rkisp2/algorithms/csm.h diff --git a/src/ipa/rkisp2/algorithms/csm.cpp b/src/ipa/rkisp2/algorithms/csm.cpp new file mode 100644 index 000000000000..9befa81bdf30 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/csm.cpp @@ -0,0 +1,177 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Color space conversion + */ + +#include "csm.h" + +#include +#include + +#include +#include + +#include +#include + +#include "linux/rkisp2-config.h" + +/** + * \file csm.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class ColorSpaceConversion + * \brief RkISP2 Color space conversion + * + * This algorithm implements the color space conversion for the RkISP2. + */ + +LOG_DEFINE_CATEGORY(RkISP2Csm) + +namespace { + struct CsmCoeffs { + uint16_t limited[9]; + uint16_t full[9]; + }; + + static const struct CsmCoeffs rec601Coeffs = { + { + 0x0021, 0x0042, 0x000d, + 0x01ed, 0x01db, 0x0038, + 0x0038, 0x01d1, 0x01f7, + }, + { + 0x0026, 0x004b, 0x000f, + 0x01ea, 0x01d6, 0x0040, + 0x0040, 0x01ca, 0x01f6, + }, + }; + + static const struct CsmCoeffs rec709Coeffs = { + { + 0x0018, 0x0050, 0x0008, + 0x01f3, 0x01d5, 0x0038, + 0x0038, 0x01cd, 0x01fb, + }, + { + 0x001b, 0x005c, 0x0009, + 0x01f1, 0x01cf, 0x0040, + 0x0040, 0x01c6, 0x01fa, + }, + }; + + static const struct CsmCoeffs rec2020Coeffs = { + { + 0x001d, 0x004c, 0x0007, + 0x01f0, 0x01d8, 0x0038, + 0x0038, 0x01cd, 0x01fb, + }, + { + 0x0022, 0x0057, 0x0008, + 0x01ee, 0x01d2, 0x0040, + 0x0040, 0x01c5, 0x01fb, + }, + }; + + static const struct CsmCoeffs smpte240mCoeffs = { + { + 0x0018, 0x004f, 0x000a, + 0x01f3, 0x01d5, 0x0038, + 0x0038, 0x01ce, 0x01fa, + }, + { + 0x001b, 0x005a, 0x000b, + 0x01f1, 0x01cf, 0x0040, + 0x0040, 0x01c7, 0x01f9, + }, + }; + + uint16_t identityCsm[9] = { + 1, 0, 0, + 0, 1, 0, + 0, 0, 1, + }; +} /* namespace */ + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int ColorSpaceConversion::init([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const ValueNode &tuningData) +{ + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int ColorSpaceConversion::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + const CsmCoeffs *coeffs; + switch (context.configuration.csm.colorSpaceEncoding) { + case static_cast(ColorSpace::YcbcrEncoding::Rec601): + coeffs = &rec601Coeffs; + break; + case static_cast(ColorSpace::YcbcrEncoding::Rec709): + coeffs = &rec709Coeffs; + break; + case static_cast(ColorSpace::YcbcrEncoding::Rec2020): + coeffs = &rec2020Coeffs; + break; + default: + coeffs = nullptr; + break; + } + + if (!coeffs) { + context.activeState.csm.csm = Matrix(identityCsm); + return 0; + } + + if (context.configuration.csm.colorSpaceRange == static_cast(ColorSpace::Range::Limited)) + context.activeState.csm.csm = Matrix(coeffs->limited); + else + context.activeState.csm.csm = Matrix(coeffs->full); + context.activeState.csm.update = true; + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void ColorSpaceConversion::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + RkISP2Params *params) +{ + if (!context.activeState.csm.update) + return; + + auto config = params->block(); + config.setEnabled(true); + + /* + * We'll use the active state directly as we don't support runtime + * configuration of the csm + */ + for (size_t i = 0; i < 3; i++) + for (size_t j = 0; j < 3; j++) + config->coeff[i][j] = context.activeState.csm.csm[i][j]; + + context.activeState.csm.update = false; +} + +REGISTER_IPA_ALGORITHM(ColorSpaceConversion, "ColorSpaceConversion") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/csm.h b/src/ipa/rkisp2/algorithms/csm.h new file mode 100644 index 000000000000..d34e4d3959e8 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/csm.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Color space conversion + */ + +#pragma once + +#include "libcamera/internal/matrix.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class ColorSpaceConversion : public Algorithm +{ +public: + ColorSpaceConversion() = default; + ~ColorSpaceConversion() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; +}; + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index e7ae3d163bfb..01829e25d254 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -5,5 +5,6 @@ rkisp2_ipa_algorithms = files([ 'awb.cpp', 'bls.cpp', 'ccm.cpp', + 'csm.cpp', ]) From patchwork Mon Aug 24 09:06:30 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28024 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 46BF7C3339 for ; Mon, 24 Aug 2026 09:07:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 01B76683B2; Mon, 24 Aug 2026 11:07:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HMgteYpN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8457C67FEC for ; Mon, 24 Aug 2026 11:07:40 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 32E76C3B; Mon, 24 Aug 2026 11:06:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562376; bh=ZVpnNwtrrruMy2dZ9VGF1Qr91MDAID+w2Gxdho6yCEo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HMgteYpNgrQ5ug6hf0FcsNkIr9SUaSUXSOBNIp1wXCZlyOLyJW2zTifZ70FeK6vhN O5APQWXNmiwxrOzrieckKP+TfGaYndqjIuEAYCNuDoXnkXIectQMCVXVz14KSybvZF ygTjuVu9sxBKUPbM/zw1hJkSpHjs+O+2mu57OVxc= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 13/20] ipa: rkisp2: algo: goc: Implement gamma out correction Date: Mon, 24 Aug 2026 18:06:30 +0900 Message-ID: <20260824090641.3705246-14-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a manual gamma out correction algorithm for the rkisp2 IPA. It takes the gamma control and applies the gamma out correction curve to the ISP. Signed-off-by: Paul Elder --- No change in v2 --- src/ipa/rkisp2/algorithms/goc.cpp | 143 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/goc.h | 42 ++++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 186 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/goc.cpp create mode 100644 src/ipa/rkisp2/algorithms/goc.h diff --git a/src/ipa/rkisp2/algorithms/goc.cpp b/src/ipa/rkisp2/algorithms/goc.cpp new file mode 100644 index 000000000000..d5d10e2dfa19 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/goc.cpp @@ -0,0 +1,143 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Gamma out control + */ + +#include "goc.h" + +#include + +#include +#include + +#include + +#include "linux/rkisp2-config.h" + +/** + * \file goc.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class GammaOutCorrection + * \brief RkISP2 Gamma out correction + * + * This algorithm implements the gamma out curve for the RkISP2. It defaults to + * a gamma value of 2.2. + * + * As gamma is internally represented as a piecewise linear function with only + * 17 knots, the difference between gamma=2.2 and sRGB gamma is minimal. + * Therefore sRGB gamma was not implemented as special case. + * + * Useful links: + * - https://www.cambridgeincolour.com/tutorials/gamma-correction.htm + * - https://en.wikipedia.org/wiki/SRGB + */ + +LOG_DEFINE_CATEGORY(RkISP2Gamma) + +const float kDefaultGamma = 2.2f; + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int GammaOutCorrection::init(IPAContext &context, const ValueNode &tuningData) +{ + defaultGamma_ = tuningData["gamma"].get(kDefaultGamma); + context.ctrlMap[&controls::Gamma] = ControlInfo(0.1f, 10.0f, defaultGamma_); + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int GammaOutCorrection::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + context.activeState.goc.gamma = defaultGamma_; + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void GammaOutCorrection::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + frameContext.goc.gamma = context.activeState.goc.gamma; + + const auto &gamma = controls.get(controls::Gamma); + if (!gamma) + return; + + context.activeState.goc.gamma = *gamma; + + frameContext.goc.gamma = context.activeState.goc.gamma; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void GammaOutCorrection::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP2Params *params) +{ + /* \todo Optimize so we don't have to write it every frame */ + + /* + * The logarithmic segments as specified in the reference, plus an + * additional 0 to make the loop easier + * + * Just use 44 mode for now. 48 mode is when the last four 512 are + * split into eight pieces of 256. + */ + static constexpr std::array segments = { + 0, 1, 1, 1, 1, 1, 1, 1, 1, + 2, 2, 2, 2, 4, 4, 4, 4, + 8, 8, 8, 8, 16, 16, 16, 16, + 32, 32, 32, 32, 64, 64, 64, 64, + 128, 128, 128, 128, 256, 256, 256, 256, + 512, 512, 512, 512 + }; + + auto config = params->block(); + config.setEnabled(true); + + unsigned x = 0; + for (const auto [i, size] : utils::enumerate(segments)) { + config->gamma_y[i] = static_cast(std::pow(x / 4096.0, 1.0 / frameContext.goc.gamma) * 4096.0); + x += size; + } + + config->mode = RKISP2_ISP_GOC_MODE_LOGARITHMIC; + config->segments = RKISP2_ISP_GOC_SEGMENTS_44; + config->offset = 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void GammaOutCorrection::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const RkISP2Stats *stats, + ControlList &metadata) +{ + metadata.set(controls::Gamma, frameContext.goc.gamma); +} + +REGISTER_IPA_ALGORITHM(GammaOutCorrection, "GammaOutCorrection") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/goc.h b/src/ipa/rkisp2/algorithms/goc.h new file mode 100644 index 000000000000..4353dda5ff14 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/goc.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Gamma out control + */ + +#pragma once + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class GammaOutCorrection : public Algorithm +{ +public: + GammaOutCorrection() = default; + ~GammaOutCorrection() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) override; + +private: + float defaultGamma_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 01829e25d254..3d73b0a99320 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -6,5 +6,6 @@ rkisp2_ipa_algorithms = files([ 'bls.cpp', 'ccm.cpp', 'csm.cpp', + 'goc.cpp', ]) From patchwork Mon Aug 24 09:06:31 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28025 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E1E6BC3339 for ; Mon, 24 Aug 2026 09:07:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9D59D67FEC; Mon, 24 Aug 2026 11:07:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PxAPydVH"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3A2A067FEC for ; Mon, 24 Aug 2026 11:07:44 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DBBFBC2D; Mon, 24 Aug 2026 11:06:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562380; bh=qAYSdB/mXXLmazhDSD8IF6cwxwTOAWWG4ksbYTXylt8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PxAPydVHo/jCdvvoYk3dmGMdysmcDULLoE1Dz/gD3F5eIFUkqV4R1Fb27hT+1+pB4 UsHjJzm1pRN9BrlLelkwmBa76Xs8lYpByia89r0LXVFCnm6q61g3ZEjzp+EQ+35VVM KD06seq6eJDSTy8uZ1KeNUxlap6siHaEkzKwu/lU= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 14/20] ipa: rkisp2: algo: lsc: Implement lens shading correction Date: Mon, 24 Aug 2026 18:06:31 +0900 Message-ID: <20260824090641.3705246-15-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a lens shading correction algorithm for the rkisp2 IPA. It uses the libipa lens shading correction. Signed-off-by: Paul Elder --- Changes in v2: - rebased on libipa rework --- src/ipa/rkisp2/algorithms/lsc.cpp | 263 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/lsc.h | 71 +++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 335 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/lsc.cpp create mode 100644 src/ipa/rkisp2/algorithms/lsc.h diff --git a/src/ipa/rkisp2/algorithms/lsc.cpp b/src/ipa/rkisp2/algorithms/lsc.cpp new file mode 100644 index 000000000000..b7c67dc983f7 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/lsc.cpp @@ -0,0 +1,263 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Lens Shading Correction control + */ + +#include "lsc.h" + +#include +#include +#include + +#include +#include + +/** + * \file lsc.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class LensShadingCorrection + * \brief RkISP2 Lens Shading Correction control + */ + +LOG_DEFINE_CATEGORY(RkISP2Lsc) + +namespace { + +constexpr int kColourTemperatureQuantization = 10; + +unsigned int quantize(unsigned int value, unsigned int step) +{ + return std::lround(value / static_cast(step)) * step; +} + +} /* namespace */ + +LensShadingCorrection::LensShadingCorrection() + : lastAppliedCt_(0), lastAppliedQuantizedCt_(0) +{ +} + +std::vector LensShadingCorrection::parseSizes(const ValueNode &tuningData, + const char *prop) +{ + std::vector sizes = + tuningData[prop].get>().value_or(utils::defopt); + /* Nobody cares about 8x8 mirrored mode; we'll just use 16x16 mode */ + if (sizes.size() != RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX) { + LOG(RkISP2Lsc, Error) + << "Invalid '" << prop << "' values: expected " + << RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX + << " elements, got " << sizes.size(); + return {}; + } + + /* + * The sum of all elements must be 1 to satisfy hardware constraints. + * Validate it here, allowing a 1% tolerance as rounding errors may + * prevent an exact match (further adjustments will be performed in + * LensShadingCorrection::prepare()). + * + * If we were in 8x8 mode then we'd have to mirror the quadrants like + * in rkisp1, but in 16x16 mode we get to configure the entire table. + * Since 8x8 table support is a todo, we only need to handle the 16x16 + * case here thus the sum should be 1. + * + * \todo Support 8x8 mode? + */ + double sum = std::accumulate(sizes.begin(), sizes.end(), 0.0); + if (sum < 0.95 || sum > 1.05) { + LOG(RkISP2Lsc, Error) + << "Invalid '" << prop << "' values: sum of the elements" + << " should be 1.0, got " << sum; + return {}; + } + + return sizes; +} + +std::vector LensShadingCorrection::sizesToPositions(Span sizes) +{ + std::vector positions(sizes.size() + 1); + + positions[0] = 0.0; + for (size_t i = 1; i < positions.size(); i++) + positions[i] = positions[i - 1] + sizes[i - 1]; + + return positions; +} + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int LensShadingCorrection::init(IPAContext &context, + const ValueNode &tuningData) +{ + xSize_ = parseSizes(tuningData, "x-size"); + ySize_ = parseSizes(tuningData, "y-size"); + + if (xSize_.empty() || ySize_.empty()) + return -EINVAL; + + xPos_ = sizesToPositions(xSize_); + yPos_ = sizesToPositions(ySize_); + + return lscAlgo_.init(tuningData, context.ctrlMap, LscDescriptor{ + .keys = { "r", "gr", "gb", "b" }, + .numHSamples = RKISP2_ISP_LSC_SAMPLES_MAX, + .numVSamples = RKISP2_ISP_LSC_SAMPLES_MAX, + .sensorSize = context.sensorInfo.activeAreaSize + }); +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int LensShadingCorrection::configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) +{ + const Size &size = context.configuration.sensor.size; + Size totalSize{}; + + /* Calculate gradients. */ + for (unsigned int i = 0; i < RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX; ++i) { + xSizes_[i] = xSize_[i] * size.width; + ySizes_[i] = ySize_[i] * size.height; + + /* + * To prevent unexpected behavior of the ISP, the sum of + * x_sizes and y_sizes items shall be equal to + * respectively size.width and size.height. Enforce it by + * computing the last tables value to avoid + * rounding-induced errors. + */ + if (i == RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX - 1) { + xSizes_[i] = size.width - totalSize.width; + ySizes_[i] = size.height - totalSize.height; + } + + totalSize.width += xSizes_[i]; + totalSize.height += ySizes_[i]; + + xGrad_[i] = std::round(32768 / xSizes_[i]); + yGrad_[i] = std::round(32768 / ySizes_[i]); + } + + return lscAlgo_.configure(context.activeState.lsc, configInfo.analogCrop, + xPos_, yPos_); +} + +void LensShadingCorrection::setParameters(rkisp2_params_lsc &config) +{ + memcpy(config.x_grads, xGrad_, sizeof(config.x_grads)); + memcpy(config.y_grads, yGrad_, sizeof(config.y_grads)); + memcpy(config.x_sizes, xSizes_, sizeof(config.x_sizes)); + memcpy(config.y_sizes, ySizes_, sizeof(config.y_sizes)); +} + +void LensShadingCorrection::copyTable(rkisp2_params_lsc &config, + const ipa::lsc::Components &set) +{ + const auto &r = set.at("r"); + std::copy(r.begin(), r.end(), &config.r_data_tbl[0][0][0]); + const auto &gr = set.at("gr"); + std::copy(gr.begin(), gr.end(), &config.gr_data_tbl[0][0][0]); + const auto &gb = set.at("gb"); + std::copy(gb.begin(), gb.end(), &config.gb_data_tbl[0][0][0]); + const auto &b = set.at("b"); + std::copy(b.begin(), b.end(), &config.b_data_tbl[0][0][0]); +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void LensShadingCorrection::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + lscAlgo_.queueRequest(context.activeState.lsc, frameContext.lsc, + controls); +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void LensShadingCorrection::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) +{ + uint32_t ct = frameContext.awb.colourTemperature; + unsigned int quantizedCt = quantize(ct, kColourTemperatureQuantization); + + /* Check if we can skip the update. */ + if (!frameContext.lsc.update) { + if (!frameContext.lsc.enabled) + return; + + /* + * Add a threshold so that oscillations around a quantization + * step don't lead to constant changes. + */ + if (utils::abs_diff(ct, lastAppliedCt_) < kColourTemperatureQuantization / 2) + return; + + if (quantizedCt == lastAppliedQuantizedCt_) + return; + } + + auto config = params->block(); + config.setEnabled(frameContext.lsc.enabled); + + if (!frameContext.lsc.enabled) + return; + + /* + * \todo Should add support for the lsc table swapping functionality? + * Or maybe we don't need it because the lsc doesn't change very + * frequently. Just use the 0th table for now. + */ + config->window_mode = RKISP2_ISP_LSC_CONFIG_16X16; + config->write_table[0] = 1; + config->write_table[1] = 0; + config->active_table = 0; + config->set_active_table_when = RKISP2_ISP_LSC_SET_ACTIVE_TABLE_AFTER; + + setParameters(*config); + + const auto &set = lscAlgo_.interpolateComponents(quantizedCt); + copyTable(*config, set); + + lastAppliedCt_ = ct; + lastAppliedQuantizedCt_ = quantizedCt; + + LOG(RkISP2Lsc, Debug) + << "ct is " << ct << ", quantized to " + << quantizedCt; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void LensShadingCorrection::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const RkISP2Stats *stats, + ControlList &metadata) +{ + lscAlgo_.process(frameContext.lsc, metadata); +} + +REGISTER_IPA_ALGORITHM(LensShadingCorrection, "LensShadingCorrection") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/lsc.h b/src/ipa/rkisp2/algorithms/lsc.h new file mode 100644 index 000000000000..d64c04ae6842 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/lsc.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Lens Shading Correction algorithm + */ + +#pragma once + +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include "libipa/fixedpoint.h" +#include "libipa/lsc.h" + +#include "algorithm.h" +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class LensShadingCorrection : public Algorithm +{ +public: + LensShadingCorrection(); + ~LensShadingCorrection() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) override; + +private: + std::vector parseSizes(const ValueNode &tuningData, + const char *prop); + std::vector sizesToPositions(Span sizes); + + void setParameters(rkisp2_params_lsc &config); + void copyTable(rkisp2_params_lsc &config, + const ipa::lsc::Components &set0); + + std::vector xSize_; + std::vector ySize_; + uint16_t xGrad_[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + uint16_t yGrad_[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + uint16_t xSizes_[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + uint16_t ySizes_[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + std::vector xPos_; + std::vector yPos_; + + unsigned int lastAppliedCt_; + unsigned int lastAppliedQuantizedCt_; + + LscAlgorithm> lscAlgo_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 3d73b0a99320..bcc947fabdb4 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -7,5 +7,6 @@ rkisp2_ipa_algorithms = files([ 'ccm.cpp', 'csm.cpp', 'goc.cpp', + 'lsc.cpp', ]) From patchwork Mon Aug 24 09:06:32 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28026 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 940F5C3339 for ; Mon, 24 Aug 2026 09:07:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3AA6D683B2; Mon, 24 Aug 2026 11:07:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Hd0RRoUF"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E71E468393 for ; Mon, 24 Aug 2026 11:07:47 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 92B38C23; Mon, 24 Aug 2026 11:06:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562383; bh=XG+S1xdnPRs8D/FDSuxuXnD0Mh+1iKkem31+3QqCxvc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Hd0RRoUFTHKpxQHT3B8dFqnfZMN4pRzbwN7q3eDWaiB2SgMd647S5hctWz0mhnsji dkULL9yQIxZmAng5azLdmDWFht+1kAb+g3MlyXdyFPI85kz5fobw1xSMZTgy3ahu+w 8UsTkNz9DTc8U/D+tfTTKM5q+J2zX5ZpOnMNpODk= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 15/20] ipa: rkisp2: algo: crop: Implement scaler crop Date: Mon, 24 Aug 2026 18:06:32 +0900 Message-ID: <20260824090641.3705246-16-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a thin algorithm for handling the ScalerCrop control. As rkisp2 uses parameter buffers to configure scaling and cropping (as opposed to v4l2 subdev pads), it is cleaner to implement scaler crop as an algorithm in the IPA. Signed-off-by: Paul Elder --- New in v2 --- src/ipa/rkisp2/algorithms/crop.cpp | 99 +++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/crop.h | 39 +++++++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 139 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/crop.cpp create mode 100644 src/ipa/rkisp2/algorithms/crop.h diff --git a/src/ipa/rkisp2/algorithms/crop.cpp b/src/ipa/rkisp2/algorithms/crop.cpp new file mode 100644 index 000000000000..8b39821c711e --- /dev/null +++ b/src/ipa/rkisp2/algorithms/crop.cpp @@ -0,0 +1,99 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Crop + */ + +#include "crop.h" + +#include + +#include +#include + +#include + +#include "linux/rkisp2-config.h" + +/** + * \file crop.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class Crop + * \brief RkISP2 Crop + * + * This is a thin algorithm that implements ScalerCrop for the RkISP2. + * + * As the cropping of the RkISP2 is controlled by parameter buffers instead of + * by V4L2 crop rectangles, it is more practical to implement the ScalerCrop + * control here instead of in the pipeline handler. This also has the positive + * side effect of supporting per-frame ScalerCrop. + */ + +LOG_DEFINE_CATEGORY(RkISP2Crop) + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Crop::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + frameContext.crop.crop = context.activeState.crop.crop; + + const auto &scalerCrop = controls.get(controls::ScalerCrop); + if (!scalerCrop) + return; + + context.activeState.crop.crop = *scalerCrop; + + frameContext.crop.crop = context.activeState.crop.crop; + frameContext.crop.set = true; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Crop::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] RkISP2Params *params) +{ + if (!frameContext.crop.set) + return; + + auto config = params->block(); + config.setEnabled(true); + + config->crop_en = RKISP2_ISP_CROP_ENABLE_MAIN; + config->mp_crop.h_offs = frameContext.crop.crop.x; + config->mp_crop.v_offs = frameContext.crop.crop.y; + config->mp_crop.h_size = frameContext.crop.crop.width; + config->mp_crop.v_size = frameContext.crop.crop.height; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Crop::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const RkISP2Stats *stats, + ControlList &metadata) +{ + /* \todo Adjust this to match the spec */ + metadata.set(controls::ScalerCrop, frameContext.crop.crop); +} + +REGISTER_IPA_ALGORITHM(Crop, "Crop") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/crop.h b/src/ipa/rkisp2/algorithms/crop.h new file mode 100644 index 000000000000..7c21e20c86ec --- /dev/null +++ b/src/ipa/rkisp2/algorithms/crop.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Crop + */ + +#pragma once + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class Crop : public Algorithm +{ +public: + Crop() = default; + ~Crop() = default; + + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) override; + +private: + Rectangle defaultCrop_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index bcc947fabdb4..9ccc9653233a 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -5,6 +5,7 @@ rkisp2_ipa_algorithms = files([ 'awb.cpp', 'bls.cpp', 'ccm.cpp', + 'crop.cpp', 'csm.cpp', 'goc.cpp', 'lsc.cpp', From patchwork Mon Aug 24 09:06:33 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28027 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 137D4C3339 for ; Mon, 24 Aug 2026 09:07:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C7DB8683DC; Mon, 24 Aug 2026 11:07:52 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="kbN8JATN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9D32068393 for ; Mon, 24 Aug 2026 11:07:51 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4849FC24; Mon, 24 Aug 2026 11:06:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562387; bh=ANkI6m1mn4QEmtkEsIpMdABg4JfnT3xfJK0XI6ZekSQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kbN8JATNsNLg1eXNFQlMDsge/irr0uxTiDoR2VXZb0t7Q4QUYZK5w3VIAU2AnNqOH 63b2X5px6X6FFePBeuaMOp0gwRKbccYD3Tbl48MzDS/EAgmjb9t0YYjonfccVhdKVE MxBpQu7RAIJWzl9Eiwj/K2M0LcstE0ncVR1aUlPw= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 16/20] utils: tuning: libtuning: image: Enable images missing some metadata Date: Mon, 24 Aug 2026 18:06:33 +0900 Message-ID: <20260824090641.3705246-17-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Some pipeline handlers, like the rkisp2 pipeline handler, are capable of taking raw images but cannot set: - exposure time - gain - black level yet as they have not been implemented for raw mode yet. Arguably black mode will never be supported in raw mode as it bypasses the ISP completely and black level subtraction is in the ISP. This prevents the images from being used for tuning in the current libtuning-based tuning scripts as it requires exposure time and iso and black level, so modify libtuning to allow images with no exposure time and iso and black level. Signed-off-by: Paul Elder --- No change in v2 --- utils/tuning/libtuning/image.py | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/utils/tuning/libtuning/image.py b/utils/tuning/libtuning/image.py index ecd334bdc67f..b1ada0d0867c 100644 --- a/utils/tuning/libtuning/image.py +++ b/utils/tuning/libtuning/image.py @@ -70,12 +70,27 @@ class Image: white = metadata[f'Exif.{subimage}.WhiteLevel'].value self.sigbits = int(white).bit_length() self.fmt = (self.sigbits - 4) // 2 - self.exposure = int(metadata[f'Exif.{photo}.ExposureTime'].value * 1000000) - self.againQ8 = metadata[f'Exif.{photo}.ISOSpeedRatings'].value * 256 / 100 - self.againQ8_norm = self.againQ8 / 256 + + self.exposure = None + exposure_key = f'Exif.{photo}.ExposureTime' + if exposure_key in metadata: + self.exposure = int(metadata[exposure_key].value * 1000000) + + self.againQ8 = None + self.againQ8_norm = None + iso_key = f'Exif.{photo}.ISOSpeedRatings' + if iso_key in metadata: + self.againQ8 = metadata[iso_key].value * 256 / 100 + self.againQ8_norm = self.againQ8 / 256 + + self.blacklevel = 0 + self.blacklevel_16 = 0 + bl_key = f'Exif.{subimage}.BlackLevel' + if bl_key in metadata: + self.blacklevel = int(metadata[bl_key].value[0]) + self.blacklevel_16 = self.blacklevel << (16 - self.sigbits) + self.camName = metadata['Exif.Image.Model'].value - self.blacklevel = int(metadata[f'Exif.{subimage}.BlackLevel'].value[0]) - self.blacklevel_16 = self.blacklevel << (16 - self.sigbits) # Channel order depending on bayer pattern # The key is the order given by exif, where 0 is R, 1 is G, and 2 is B From patchwork Mon Aug 24 09:06:34 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28028 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D0784C3339 for ; Mon, 24 Aug 2026 09:07:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8A0FB683DC; Mon, 24 Aug 2026 11:07:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uO754VqI"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4D1736838A for ; Mon, 24 Aug 2026 11:07:55 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F3FDEC25; Mon, 24 Aug 2026 11:06:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562391; bh=F/Y/d2aaUo/KjEVQEBSfZBGUTXdyBMfYDvcP8qtFxsY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uO754VqInlo7A6RpmkYdVOreUCRzHEe/vFz2isIQEk5RoqvgU+1ACrWvGXV2LNLKf 3wo30L+bvZXfrBgi0hb+fk0evIlR5eFE/1oLxCRxjsY2afIPqjV3X27MNBEe6udr0c xefpyqNBll28t3OOmG74YPo/DV5Aq7QMjEn8xejQ= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 17/20] utils: tuning: rkisp2.py: Add tuning script for rkisp2 Date: Mon, 24 Aug 2026 18:06:34 +0900 Message-ID: <20260824090641.3705246-18-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a tuning script for the rkisp2. It is based on the rkisp1 tuning script. Signed-off-by: Paul Elder --- lsc is not output currently as it needs a bit more investigation. No change in v2 --- utils/tuning/rkisp2.py | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100755 utils/tuning/rkisp2.py diff --git a/utils/tuning/rkisp2.py b/utils/tuning/rkisp2.py new file mode 100755 index 000000000000..13feff03e6cc --- /dev/null +++ b/utils/tuning/rkisp2.py @@ -0,0 +1,42 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Copyright (C) 2026, Paul Elder +# Copyright (C) 2026, Ideas On Board +# +# Tuning script for rkisp2, based on rkisp1 + +import logging +import sys + +import coloredlogs +import libtuning as lt +from libtuning.generators import YamlOutput +from libtuning.modules.agc import AGCRkISP1 +from libtuning.modules.awb import AWBRkISP1 +from libtuning.modules.ccm import CCMRkISP1 +from libtuning.modules.lsc import LSCRkISP1 +from libtuning.modules.lux import LuxRkISP1 +from libtuning.modules.static import StaticModule +from libtuning.parsers import YamlParser + +coloredlogs.install(level=logging.INFO, fmt='%(name)s %(levelname)s %(message)s') + +csm = StaticModule('ColorSpaceConversion') +agc = AGCRkISP1(debug=[lt.Debug.Plot]) +awb = AWBRkISP1(debug=[lt.Debug.Plot]) +bls = StaticModule('BlackLevelSubtraction') +ccm = CCMRkISP1(debug=[lt.Debug.Plot]) +goc = StaticModule('GammaOutCorrection', {'gamma': 2.2}) + +tuner = lt.Tuner('RkISP2') +tuner.add([csm, agc, awb, bls, ccm, goc]) +tuner.set_input_parser(YamlParser()) +tuner.set_output_formatter(YamlOutput()) + +# Bayesian AWB uses the lux value, so insert the lux algorithm before AWB. +# Compress is parameterized by others, so add it at the end. +tuner.set_output_order([csm, agc, awb, bls, ccm, goc]) + +if __name__ == '__main__': + sys.exit(tuner.run(sys.argv)) From patchwork Mon Aug 24 09:06:35 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28029 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 901D4C3339 for ; Mon, 24 Aug 2026 09:08:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4AA11683DC; Mon, 24 Aug 2026 11:08:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="hMgIPaXA"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F0CFB6838A for ; Mon, 24 Aug 2026 11:07:58 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A41E3C23; Mon, 24 Aug 2026 11:06:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562394; bh=shC7mLhx5RhE1nFOnvWMHiGL38nfZ9U8mCVUf4nrfvI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hMgIPaXAfeZExBBS2LHy8JjnZOMe8qd6/+mdUBO3wsylsaN92cTty3ajGxBIJARzA QJ37iB/mEkRNa6yAlvbOf2HJEH7y7CFK38K7iOD6NINLcHjPhgM4URuMLWCshgWzLO itIda6qwyjtjysQ2vylOsdKwqbr9HdlmMHDdXlMQ= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 18/20] ipa: rkisp2: data: Add uncalibrated tuning file Date: Mon, 24 Aug 2026 18:06:35 +0900 Message-ID: <20260824090641.3705246-19-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add uncalibrated tuning file for rkisp2 that just enables black level subtraction. Signed-off-by: Paul Elder --- No change in v2 --- src/ipa/rkisp2/data/meson.build | 10 ++++++++++ src/ipa/rkisp2/data/uncalibrated.yaml | 7 +++++++ src/ipa/rkisp2/meson.build | 1 + 3 files changed, 18 insertions(+) create mode 100644 src/ipa/rkisp2/data/meson.build create mode 100644 src/ipa/rkisp2/data/uncalibrated.yaml diff --git a/src/ipa/rkisp2/data/meson.build b/src/ipa/rkisp2/data/meson.build new file mode 100644 index 000000000000..1ecd8b20ec90 --- /dev/null +++ b/src/ipa/rkisp2/data/meson.build @@ -0,0 +1,10 @@ +# SPDX-License-Identifier: CC0-1.0 + +conf_files = files([ + 'uncalibrated.yaml', +]) + +install_data(conf_files, + install_dir : ipa_data_dir / 'rkisp2', + install_tag : 'runtime') + diff --git a/src/ipa/rkisp2/data/uncalibrated.yaml b/src/ipa/rkisp2/data/uncalibrated.yaml new file mode 100644 index 000000000000..6d60bf1e6e98 --- /dev/null +++ b/src/ipa/rkisp2/data/uncalibrated.yaml @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +version: 1 +algorithms: + - BlackLevelSubtraction: +... diff --git a/src/ipa/rkisp2/meson.build b/src/ipa/rkisp2/meson.build index f2f435c2d66f..db6b06aeec25 100644 --- a/src/ipa/rkisp2/meson.build +++ b/src/ipa/rkisp2/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 subdir('algorithms') +subdir('data') ipa_name = 'ipa_rkisp2' From patchwork Mon Aug 24 09:06:36 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28030 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3DA06C3339 for ; Mon, 24 Aug 2026 09:08:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DF559683EB; Mon, 24 Aug 2026 11:08:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="wnx738IM"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A5795683B2 for ; Mon, 24 Aug 2026 11:08:02 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 55343C24; Mon, 24 Aug 2026 11:06:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562398; bh=HOXfdKeFzRjzgCEZHHnv5k6VATnM/WzKxgLKB6/AU1E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wnx738IM7EGl37Y6ntWX2d+IRaqknt3N9DJSw8SI1XpcdWbjpojYV7KeKtEp3/BVJ iKf57qN18bQBvVnqPQYMD4LNsAb2oDaRscB24lJuRHswHV3nHcLvdIKr+USUAMlVh9 yVZz7RsbukbLz89IoBrredhSM7DWCYjVffO/wlJs= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 19/20] ipa: rkisp2: data: Add rudimentary tuning file for imx219 Date: Mon, 24 Aug 2026 18:06:36 +0900 Message-ID: <20260824090641.3705246-20-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a tuning file for rkisp2 for imx219. The ccms come from rpi, the lsc tables are just all 1.0 for now, and the agc parameters are copied from the output of the rkisp1 tuning script. Signed-off-by: Paul Elder --- No change in v2 --- src/ipa/rkisp2/data/imx219.yaml | 139 ++++++++++++++++++++++++++++++++ src/ipa/rkisp2/data/meson.build | 1 + 2 files changed, 140 insertions(+) create mode 100644 src/ipa/rkisp2/data/imx219.yaml diff --git a/src/ipa/rkisp2/data/imx219.yaml b/src/ipa/rkisp2/data/imx219.yaml new file mode 100644 index 000000000000..eef413a490cb --- /dev/null +++ b/src/ipa/rkisp2/data/imx219.yaml @@ -0,0 +1,139 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +version: 1 +algorithms: + - LensShadingCorrection: + x-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, + 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ] + y-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, + 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ] + sets: + - ct: 5800 + r: [ + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + ] + gr: [ + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + ] + gb: [ + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + ] + b: [ + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + ] + - Agc: + AeMeteringMode: + MeteringCentreWeighted: [0, 0, 0, 0, 0, 0, 6, 8, 6, 0, 0, 8, 16, 8, 0, 0, 6, 8, 6, 0, 0, 0, 0, 0, 0] + MeteringSpot: [0, 0, 0, 0, 0, 0, 2, 4, 2, 0, 0, 4, 16, 4, 0, 0, 2, 4, 2, 0, 0, 0, 0, 0, 0] + MeteringMatrix: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] + AeExposureMode: + ExposureNormal: + exposureTime: [ 100, 10000, 30000, 60000, 120000 ] + gain: [ 1.0, 2.0, 4.0, 6.0, 6.0 ] + ExposureShort: + exposureTime: [ 100, 5000, 10000, 20000, 120000 ] + gain: [ 1.0, 2.0, 4.0, 6.0, 6.0 ] + AeConstraintMode: + ConstraintNormal: + lower: + qLo: 0.98 + qHi: 1.0 + yTarget: [0, 0.65, 1000, 0.65] + ConstraintHighlight: + lower: + qLo: 0.98 + qHi: 1.0 + yTarget: [0, 0.5, 1000, 0.5] + upper: + qLo: 0.98 + qHi: 1.0 + yTarget: [0, 0.8, 1000, 0.5] + relativeLuminanceTarget: 0.5 + - Awb: + - BlackLevelSubtraction: + R: 4096 + Gr: 4096 + Gb: 4096 + B: 4096 + - Ccm: + ccms: + - ct: 2860 + ccm: [ 2.12089, -0.52461, -0.59629, -0.85342, 2.80445, -0.95103, -0.26897, -1.14788, 2.41685 ] + - ct: 2960 + ccm: [ 2.26962, -0.54174, -0.72789, -0.77008, 2.60271, -0.83262, -0.26036, -1.51254, 2.77289 ] + - ct: 3603 + ccm: [ 2.18644, -0.66148, -0.52496, -0.77828, 2.69474, -0.91645, -0.25239, -0.83059, 2.08298 ] + - ct: 4650 + ccm: [ 2.18174, -0.70887, -0.47287, -0.70196, 2.76426, -1.06231, -0.25157, -0.71978, 1.97135 ] + - ct: 5858 + ccm: [ 2.32392, -0.88421, -0.43971, -0.63821, 2.58348, -0.94527, -0.28541, -0.54112, 1.82653 ] + - ct: 7580 + ccm: [ 2.21175, -0.53242, -0.67933, -0.57875, 3.07922, -1.50047, -0.27709, -0.73338, 2.01048 ] + - GammaOutCorrection: + - ColorSpaceConversion: +... diff --git a/src/ipa/rkisp2/data/meson.build b/src/ipa/rkisp2/data/meson.build index 1ecd8b20ec90..6b21718e408f 100644 --- a/src/ipa/rkisp2/data/meson.build +++ b/src/ipa/rkisp2/data/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 conf_files = files([ + 'imx219.yaml', 'uncalibrated.yaml', ]) From patchwork Mon Aug 24 09:06:37 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28031 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4A7AFC3308 for ; Mon, 24 Aug 2026 09:08:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EDDCF683ED; Mon, 24 Aug 2026 11:08:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="v9hNwhQs"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5575F6838A for ; Mon, 24 Aug 2026 11:08:06 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:e616:f172:fe90:f1b5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 05D26C23; Mon, 24 Aug 2026 11:06:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562402; bh=H0RiDQw1XheJP92AQlO2vD0Fh2CkLcw/Ytfjtw9z6F8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=v9hNwhQsEJanaNjyeVgFj2zyDrBPFhgyIfUmSv9tpF9nWB9obNU2MGBMNk2QyM7ep rmhvutnk7SjAU7rwYlXTT7/ctnvIuAdwgoVunE1HJ2BA7CWP/O9FJOiWjdjG0Fo4fY rJqiBAvBS3PQ54azCaFZPg02I+wOhfeJ5fUw1AiQ= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v2 20/20] ipa: rkisp2: data: Add tuning file for imx708 Date: Mon, 24 Aug 2026 18:06:37 +0900 Message-ID: <20260824090641.3705246-21-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260824090641.3705246-1-paul.elder@ideasonboard.com> References: <20260824090641.3705246-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a tuning file for rkisp2 for imx708. This was tuned using the rkisp2 tuning script using images from [0] (at branch raspberrypi/imx708 in directory 2026-05-22), with some hand-modifications, such as removing lsc because it needs further investigation to function properly. [0] https://gitlab.freedesktop.org/camera/tuning Signed-off-by: Paul Elder --- No change in v2 --- src/ipa/rkisp2/data/imx708.yaml | 155 ++++++++++++++++++++++++++++++++ src/ipa/rkisp2/data/meson.build | 1 + 2 files changed, 156 insertions(+) create mode 100644 src/ipa/rkisp2/data/imx708.yaml diff --git a/src/ipa/rkisp2/data/imx708.yaml b/src/ipa/rkisp2/data/imx708.yaml new file mode 100644 index 000000000000..3296ca58b71e --- /dev/null +++ b/src/ipa/rkisp2/data/imx708.yaml @@ -0,0 +1,155 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +version: 1 +algorithms: + - Crop: + - ColorSpaceConversion: + - Agc: + AeMeteringMode: + MeteringCentreWeighted: [ 0, 0, 0, 0, 0, 0, 6, 8, 6, 0, 0, 8, 16, 8, 0, 0, 6, 8, 6, 0, 0, 0, 0, 0, 0 ] + AeExposureMode: + ExposureNormal: + exposureTime: [ 100, 10000, 30000, 60000, 120000 ] + gain: [ 1.2, 2.0, 4.0, 6.0, 6.0 ] + AeConstraintMode: + ConstraintNormal: + lower: + qLo: 0.98 + qHi: 1.0 + yTarget: 0.1 + relativeLuminanceTarget: 0.1 + - Awb: + algorithm: "grey" + AwbMode: + AwbAuto: + lo: 2500 + hi: 9000 + AwbIncandescent: + lo: 2500 + hi: 3000 + AwbTungsten: + lo: 3000 + hi: 3500 + AwbFluorescent: + lo: 4000 + hi: 4700 + AwbIndoor: + lo: 3000 + hi: 5000 + AwbDaylight: + lo: 5500 + hi: 6500 + AwbCloudy: + lo: 6500 + hi: 8000 + priors: + - lux: 0 + ct: [ 2000, 13000 ] + probability: [ 1.0, 1.0 ] + colourGains: + - ct: 2500 + gains: [ 1.1199462425803561, 1.9142419601837672 ] + - ct: 3000 + gains: [ 1.1242270938729624, 1.9015021867275146 ] + - ct: 3500 + gains: [ 1.1583458820803894, 1.8109380659181455 ] + - ct: 4000 + gains: [ 1.2714558169103625, 1.6165535079211122 ] + - ct: 4500 + gains: [ 1.476886722788362, 1.4486455164421266 ] + - ct: 5000 + gains: [ 1.64446637066272, 1.3819789939192924 ] + - ct: 5500 + gains: [ 1.7822135091783995, 1.3487995683841383 ] + - ct: 6000 + gains: [ 1.8677624206200971, 1.33422281521014 ] + - ct: 6500 + gains: [ 1.9219680953296177, 1.3266118333775536 ] + - ct: 7000 + gains: [ 1.9516003122560501, 1.3229263130043656 ] + - ct: 7500 + gains: [ 1.9673421207948063, 1.321178491214163 ] + - ct: 8000 + gains: [ 1.9845207382417145, 1.3190871916633689 ] + - ct: 8500 + gains: [ 1.993620414673046, 1.3182177695755337 ] + - ct: 9000 + gains: [ 2.006823198876179, 1.3170025023047545 ] + transversePos: 0.01269 + transverseNeg: 0.01561 + - BlackLevelSubtraction: + - Ccm: + ccms: + - ct: 2500 + ccm: [ + 1.72201, 0.27067, -0.99269, -0.64829, 2.48777, -0.83947, -0.08905, -0.54176, + 1.63081, + ] + - ct: 3000 + ccm: [ + 1.80587, 0.27132, -1.07719, -0.67445, 2.55729, -0.88284, -0.11544, -0.50189, + 1.61734, + ] + - ct: 3500 + ccm: [ + 1.83389, 0.37559, -1.20949, -0.69373, 2.67018, -0.97646, -0.15308, -0.47581, + 1.62889, + ] + - ct: 4000 + ccm: [ + 1.86271, 0.39818, -1.26089, -0.66884, 2.62966, -0.96082, -0.19298, -0.45096, + 1.64394, + ] + - ct: 4500 + ccm: [ + 1.78213, 0.40302, -1.18514, -0.61892, 2.47338, -0.85445, -0.24296, -0.38901, + 1.63197, + ] + - ct: 5000 + ccm: [ + 1.75328, 0.39113, -1.14441, -0.60457, 2.40168, -0.79711, -0.28013, -0.33477, + 1.61489, + ] + - ct: 5500 + ccm: [ + 1.67989, 0.42725, -1.10714, -0.60576, 2.35811, -0.75235, -0.30859, -0.29885, + 1.60744, + ] + - ct: 6000 + ccm: [ + 1.62571, 0.45052, -1.07622, -0.60271, 2.32574, -0.72302, -0.31931, -0.27971, + 1.59902, + ] + - ct: 6500 + ccm: [ + 1.60018, 0.43641, -1.03658, -0.59726, 2.29742, -0.70016, -0.31271, -0.28088, + 1.59358, + ] + - ct: 7000 + ccm: [ + 1.59364, 0.41222, -1.00585, -0.58752, 2.27837, -0.69085, -0.30352, -0.28323, + 1.58675, + ] + - ct: 7500 + ccm: [ + 1.59057, 0.40006, -0.99063, -0.58525, 2.26603, -0.68078, -0.29994, -0.28105, + 1.58099, + ] + - ct: 8000 + ccm: [ + 1.59551, 0.36878, -0.96428, -0.57195, 2.24572, -0.67377, -0.28793, -0.28858, + 1.57651, + ] + - ct: 8500 + ccm: [ + 1.60576, 0.33929, -0.94504, -0.56235, 2.23411, -0.67176, -0.27817, -0.29265, + 1.57082, + ] + - ct: 9000 + ccm: [ + 1.60867, 0.30611, -0.91478, -0.54939, 2.21074, -0.66135, -0.26915, -0.30127, + 1.57042, + ] + - GammaOutCorrection: + gamma: 2.2 diff --git a/src/ipa/rkisp2/data/meson.build b/src/ipa/rkisp2/data/meson.build index 6b21718e408f..f4dc569db277 100644 --- a/src/ipa/rkisp2/data/meson.build +++ b/src/ipa/rkisp2/data/meson.build @@ -2,6 +2,7 @@ conf_files = files([ 'imx219.yaml', + 'imx708.yaml', 'uncalibrated.yaml', ])