From patchwork Tue Jul 9 14:49:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20632 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 88F0FBD87C for ; Tue, 9 Jul 2024 14:50:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 30FF563375; Tue, 9 Jul 2024 16:50:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BCzTvyE5"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9216462C95 for ; Tue, 9 Jul 2024 16:49:55 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E306616C3; Tue, 9 Jul 2024 16:49:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720536563; bh=f0dh+OIPlOANSzGLrn8+XlzDDvnfpKrx/6/OXSbRwiw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BCzTvyE5PVOfwgux4X3SHUGeTk6Z34jBnIdAjyzN2v08IpFb3A7bNDdOL432Rs8P/ OwDwNqwQmZCrhHlMY0Vn1GEOujNMhSyADXoRYLLjYJuu4und8Cx1+KFeaSvyAhbaX/ tyiGvJFwAJ1B29tlJSpnXL9oRqztsgi2rvlGDwA0= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Nayden Kanchev , Jacopo Mondi Subject: [PATCH v2 01/10] include: linux: Add mali-c55-config.h Date: Tue, 9 Jul 2024 15:49:41 +0100 Message-Id: <20240709144950.3277837-2-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709144950.3277837-1-dan.scally@ideasonboard.com> References: <20240709144950.3277837-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the header file describing the Mali C55's 3A statistics and parameters structures so that they can be used by the new IPA module. The header has been taken from the v6 of the patchset for the Mali C55 ISP driver submitted to linux-media, found at the link below: https://lore.kernel.org/linux-media/20240709132906.3198927-1-dan.scally@ideasonboard.com/ Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Acked-by: Kieran Bingham --- include/linux/mali-c55-config.h | 945 ++++++++++++++++++++++++++++++++ 1 file changed, 945 insertions(+) create mode 100644 include/linux/mali-c55-config.h diff --git a/include/linux/mali-c55-config.h b/include/linux/mali-c55-config.h new file mode 100644 index 00000000..ea9872e9 --- /dev/null +++ b/include/linux/mali-c55-config.h @@ -0,0 +1,945 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * ARM Mali-C55 ISP Driver - Userspace API + * + * Copyright (C) 2023 Ideas on Board Oy + */ + +#ifndef __UAPI_MALI_C55_CONFIG_H +#define __UAPI_MALI_C55_CONFIG_H + +#include + +/* + * Frames are split into zones of almost equal width and height - a zone is a + * rectangular tile of a frame. The metering blocks within the ISP collect + * aggregated statistics per zone. A maximum of 15x15 zones can be configured, + * and so the statistics buffer within the hardware is sized to accommodate + * that. + * + * The utilised number of zones is runtime configurable. + */ +#define MALI_C55_MAX_ZONES (15 * 15) + +/** + * struct mali_c55_ae_1024bin_hist - Auto Exposure 1024-bin histogram statistics + * + * @bins: 1024 element array of 16-bit pixel counts. + * + * The 1024-bin histogram module collects image-global but zone-weighted + * intensity distributions of pixels in fixed-width bins. The modules can be + * configured into different "plane modes" which affect the contents of the + * collected statistics. In plane mode 0, pixel intensities are taken regardless + * of colour plane into a single 1024-bin histogram with a bin width of 4. In + * plane mode 1, four 256-bin histograms with a bin width of 16 are collected - + * one for each CFA colour plane. In plane modes 4, 5, 6 and 7 two 512-bin + * histograms with a bin width of 8 are collected - in each mode one of the + * colour planes is collected into the first histogram and all the others are + * combined into the second. The histograms are stored consecutively in the bins + * array. + * + * The 16-bit pixel counts are stored as a 4-bit exponent in the most + * significant bits followed by a 12-bit mantissa. Conversion to a usable + * format can be done according to the following pseudo-code:: + * + * if (e == 0) { + * bin = m * 2; + * } else { + * bin = (m + 4096) * 2^e + * } + * + * where + * e is the exponent value in range 0..15 + * m is the mantissa value in range 0..4095 + * + * The pixels used in calculating the statistics can be masked using three + * methods: + * + * 1. Pixels can be skipped in X and Y directions independently. + * 2. Minimum/Maximum intensities can be configured + * 3. Zones can be differentially weighted, including 0 weighted to mask them + * + * The data for this histogram can be collected from different tap points in the + * ISP depending on configuration - after the white balance or digital gain + * blocks, or immediately after the input crossbar. + */ +struct mali_c55_ae_1024bin_hist { + __u16 bins[1024]; +} __attribute__((packed)); + +/** + * struct mali_c55_ae_5bin_hist - Auto Exposure 5-bin histogram statistics + * + * @hist0: 16-bit normalised pixel count for the 0th intensity bin + * @hist1: 16-bit normalised pixel count for the 1st intensity bin + * @hist3: 16-bit normalised pixel count for the 3rd intensity bin + * @hist4: 16-bit normalised pixel count for the 4th intensity bin + * + * The ISP generates a 5-bin histogram of normalised pixel counts within bins of + * pixel intensity for each of 225 possible zones within a frame. The centre bin + * of the histogram for each zone is not available from the hardware and must be + * calculated by subtracting the values of hist0, hist1, hist3 and hist4 from + * 0xffff as in the following equation: + * + * hist2 = 0xffff - (hist0 + hist1 + hist3 + hist4) + */ +struct mali_c55_ae_5bin_hist { + __u16 hist0; + __u16 hist1; + __u16 hist3; + __u16 hist4; +} __attribute__((packed)); + +/** + * struct mali_c55_awb_average_ratios - Auto White Balance colour ratios + * + * @avg_rg_gr: Average R/G or G/R ratio in Q4.8 format. + * @avg_bg_br: Average B/G or B/R ratio in Q4.8 format. + * @num_pixels: The number of pixels used in the AWB calculation + * + * The ISP calculates and collects average colour ratios for each zone in an + * image and stores them in Q4.8 format (the lowest 8 bits are fractional, with + * bits [11:8] representing the integer). The exact ratios collected (either + * R/G, B/G or G/R, B/R) are configurable through the parameters buffer. The + * value of the 4 high bits is undefined. + */ +struct mali_c55_awb_average_ratios { + __u16 avg_rg_gr; + __u16 avg_bg_br; + __u32 num_pixels; +} __attribute__((packed)); + +/** + * struct mali_c55_af_statistics - Auto Focus edge and intensity statistics + * + * @intensity_stats: Packed mantissa and exponent value for pixel intensity + * @edge_stats: Packed mantissa and exponent values for edge intensity + * + * The ISP collects the squared sum of pixel intensities for each zone within a + * configurable Region of Interest on the frame. Additionally, the same data are + * collected after being passed through a bandpass filter which removes high and + * low frequency components - these are referred to as the edge statistics. + * + * The intensity and edge statistics for a zone can be used to calculate the + * contrast information for a zone + * + * C = E2 / I2 + * + * Where I2 is the intensity statistic for a zone and E2 is the edge statistic + * for that zone. Optimum focus is reached when C is at its maximum. + * + * The intensity and edge statistics are stored packed into a non-standard 16 + * bit floating point format, where the 7 most significant bits represent the + * exponent and the 9 least significant bits the mantissa. This format can be + * unpacked with the following pseudocode:: + * + * if (e == 0) { + * x = m; + * } else { + * x = 2^e-1 * (m + 2^9) + * } + * + * where + * e is the exponent value in range 0..127 + * m is the mantissa value in range 0..511 + */ +struct mali_c55_af_statistics { + __u16 intensity_stats; + __u16 edge_stats; +} __attribute__((packed)); + +/** + * struct mali_c55_stats_buffer - 3A statistics for the mali-c55 ISP + * + * @ae_1024bin_hist: 1024-bin frame-global pixel intensity histogram + * @iridix_1024bin_hist: Post-Iridix block 1024-bin histogram + * @ae_5bin_hists: 5-bin pixel intensity histograms for AEC + * @reserved1: Undefined buffer space + * @awb_ratios: Color balance ratios for Auto White Balance + * @reserved2: Undefined buffer space + * @af_statistics: Pixel intensity statistics for Auto Focus + * @reserved3: Undefined buffer space + * + * This struct describes the metering statistics space in the Mali-C55 ISP's + * hardware in its entirety. The space between each defined area is marked as + * "unknown" and may not be 0, but should not be used. The @ae_5bin_hists, + * @awb_ratios and @af_statistics members are arrays of statistics per-zone. + * The zones are arranged in the array in raster order starting from the top + * left corner of the image. + */ + +struct mali_c55_stats_buffer { + struct mali_c55_ae_1024bin_hist ae_1024bin_hist; + struct mali_c55_ae_1024bin_hist iridix_1024bin_hist; + struct mali_c55_ae_5bin_hist ae_5bin_hists[MALI_C55_MAX_ZONES]; + __u32 reserved1[14]; + struct mali_c55_awb_average_ratios awb_ratios[MALI_C55_MAX_ZONES]; + __u32 reserved2[14]; + struct mali_c55_af_statistics af_statistics[MALI_C55_MAX_ZONES]; + __u32 reserved3[15]; +} __attribute__((packed)); + +/** + * enum mali_c55_param_buffer_version - Mali-C55 parameters block versioning + * + * @MALI_C55_PARAM_BUFFER_V1: First version of Mali-C55 parameters block + */ +enum mali_c55_param_buffer_version { + MALI_C55_PARAM_BUFFER_V1, +}; + +/** + * enum mali_c55_param_block_type - Enumeration of Mali-C55 parameter blocks + * + * This enumeration defines the types of Mali-C55 parameters block. Each block + * configures a specific processing block of the Mali-C55 ISP. The block + * type allows the driver to correctly interpret the parameters block data. + * + * It is the responsibility of userspace to correctly set the type of each + * parameters block. + * + * @MALI_C55_PARAM_BLOCK_SENSOR_OFFS: Sensor pre-shading black level offset + * @MALI_C55_PARAM_BLOCK_AEXP_HIST: Auto-exposure 1024-bin histogram + * configuration + * @MALI_C55_PARAM_BLOCK_AEXP_IHIST: Post-Iridix auto-exposure 1024-bin + * histogram configuration + * @MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS: Auto-exposure 1024-bin histogram + * weighting + * @MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS: Post-Iridix auto-exposure 1024-bin + * histogram weighting + * @MALI_C55_PARAM_BLOCK_DIGITAL_GAIN: Digital gain + * @MALI_C55_PARAM_BLOCK_AWB_GAINS: Auto-white balance gains + * @MALI_C55_PARAM_BLOCK_AWB_CONFIG: Auto-white balance statistics config + * @MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP: Auto-white balance gains for AEXP-0 tap + * @MALI_C55_PARAM_MESH_SHADING_CONFIG : Mesh shading tables configuration + * @MALI_C55_PARAM_MESH_SHADING_SELECTION: Mesh shading table selection + * @MALI_C55_PARAM_BLOCK_SENTINEL: First non-valid block index + */ +enum mali_c55_param_block_type { + MALI_C55_PARAM_BLOCK_SENSOR_OFFS, + MALI_C55_PARAM_BLOCK_AEXP_HIST, + MALI_C55_PARAM_BLOCK_AEXP_IHIST, + MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS, + MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS, + MALI_C55_PARAM_BLOCK_DIGITAL_GAIN, + MALI_C55_PARAM_BLOCK_AWB_GAINS, + MALI_C55_PARAM_BLOCK_AWB_CONFIG, + MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP, + MALI_C55_PARAM_MESH_SHADING_CONFIG, + MALI_C55_PARAM_MESH_SHADING_SELECTION, + MALI_C55_PARAM_BLOCK_SENTINEL, +}; + +/** + * struct mali_c55_params_block_header - Mali-C55 parameter block header + * + * This structure represents the common part of all the ISP configuration + * blocks. Each parameters block embeds an instance of this structure type + * as its first member, followed by the block-specific configuration data. The + * driver inspects this common header to discern the block type and its size and + * properly handle the block content by casting it to the correct block-specific + * type. + * + * The @type field is one of the values enumerated by + * :c:type:`mali_c55_param_block_type` and specifies how the data should be + * interpreted by the driver. The @size field specifies the size of the + * parameters block and is used by the driver for validation purposes. The + * @enabled field specifies if the ISP block should be enabled (and configured + * according to the provided parameters) or disabled. + * + * Userspace is responsible for correctly populating the parameters block header + * fields (@type, @enabled and @size) and correctly populate the block-specific + * parameters. + * + * For example: + * + * .. code-block:: c + * + * void populate_sensor_offs(struct mali_c55_params_block_header *block) { + * block->type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS; + * block->enabled = true; + * block->size = sizeof(struct mali_c55_params_sensor_off_preshading); + * + * struct mali_c55_params_sensor_off_preshading *sensor_offs = + * (struct mali_c55_params_sensor_off_preshading *)block; + * + * sensor_offs->chan00 = offset00; + * sensor_offs->chan01 = offset01; + * sensor_offs->chan10 = offset10; + * sensor_offs->chan11 = offset11; + * } + * + * @type: The parameters block type from :c:type:`mali_c55_param_block_type` + * @enabled: Block enabled/disabled flag + * @size: Size (in bytes) of the parameters block + */ +struct mali_c55_params_block_header { + __u8 type; + __u8 enabled; + __u32 size; +} __attribute__((aligned(8))); + +/** + * struct mali_c55_params_sensor_off_preshading - offset subtraction for each + * color channel + * + * Provides removal of the sensor black level from the sensor data. Separate + * offsets are provided for each of the four Bayer component color channels + * which are defaulted to R, Gr, Gb, B. + * + * header.type should be set to MALI_C55_PARAM_BLOCK_SENSOR_OFFS from + * :c:type:`mali_c55_param_block_type` for this block. + * + * @header: The Mali-C55 parameters block header + * @chan00: Offset for color channel 00 (default: R) + * @chan01: Offset for color channel 01 (default: Gr) + * @chan10: Offset for color channel 10 (default: Gb) + * @chan11: Offset for color channel 11 (default: B) + */ +struct mali_c55_params_sensor_off_preshading { + struct mali_c55_params_block_header header; + __u32 chan00; + __u32 chan01; + __u32 chan10; + __u32 chan11; +}; + +/** + * enum mali_c55_aexp_hist_tap_points - Tap points for the AEXP histogram + * @MALI_C55_AEXP_HIST_TAP_WB: After static white balance + * @MALI_C55_AEXP_HIST_TAP_FS: After WDR Frame Stitch + * @MALI_C55_AEXP_HIST_TAP_TPG: After the test pattern generator + */ +enum mali_c55_aexp_hist_tap_points { + MALI_C55_AEXP_HIST_TAP_WB = 0, + MALI_C55_AEXP_HIST_TAP_FS, + MALI_C55_AEXP_HIST_TAP_TPG, +}; + +/** + * enum mali_c55_aexp_skip_x - Horizontal pixel skipping + * @MALI_C55_AEXP_SKIP_X_EVERY_2ND: Collect every 2nd pixel horizontally + * @MALI_C55_AEXP_SKIP_X_EVERY_3RD: Collect every 3rd pixel horizontally + * @MALI_C55_AEXP_SKIP_X_EVERY_4TH: Collect every 4th pixel horizontally + * @MALI_C55_AEXP_SKIP_X_EVERY_5TH: Collect every 5th pixel horizontally + * @MALI_C55_AEXP_SKIP_X_EVERY_8TH: Collect every 8th pixel horizontally + * @MALI_C55_AEXP_SKIP_X_EVERY_9TH: Collect every 9th pixel horizontally + */ +enum mali_c55_aexp_skip_x { + MALI_C55_AEXP_SKIP_X_EVERY_2ND, + MALI_C55_AEXP_SKIP_X_EVERY_3RD, + MALI_C55_AEXP_SKIP_X_EVERY_4TH, + MALI_C55_AEXP_SKIP_X_EVERY_5TH, + MALI_C55_AEXP_SKIP_X_EVERY_8TH, + MALI_C55_AEXP_SKIP_X_EVERY_9TH +}; + +/** + * enum mali_c55_aexp_skip_y - Vertical pixel skipping + * @MALI_C55_AEXP_SKIP_Y_ALL: Collect every single pixel vertically + * @MALI_C55_AEXP_SKIP_Y_EVERY_2ND: Collect every 2nd pixel vertically + * @MALI_C55_AEXP_SKIP_Y_EVERY_3RD: Collect every 3rd pixel vertically + * @MALI_C55_AEXP_SKIP_Y_EVERY_4TH: Collect every 4th pixel vertically + * @MALI_C55_AEXP_SKIP_Y_EVERY_5TH: Collect every 5th pixel vertically + * @MALI_C55_AEXP_SKIP_Y_EVERY_8TH: Collect every 8th pixel vertically + * @MALI_C55_AEXP_SKIP_Y_EVERY_9TH: Collect every 9th pixel vertically + */ +enum mali_c55_aexp_skip_y { + MALI_C55_AEXP_SKIP_Y_ALL, + MALI_C55_AEXP_SKIP_Y_EVERY_2ND, + MALI_C55_AEXP_SKIP_Y_EVERY_3RD, + MALI_C55_AEXP_SKIP_Y_EVERY_4TH, + MALI_C55_AEXP_SKIP_Y_EVERY_5TH, + MALI_C55_AEXP_SKIP_Y_EVERY_8TH, + MALI_C55_AEXP_SKIP_Y_EVERY_9TH +}; + +/** + * enum mali_c55_aexp_row_column_offset - Start from the first or second row or + * column + * @MALI_C55_AEXP_FIRST_ROW_OR_COL: Start from the first row / column + * @MALI_C55_AEXP_SECOND_ROW_OR_COL: Start from the second row / column + */ +enum mali_c55_aexp_row_column_offset { + MALI_C55_AEXP_FIRST_ROW_OR_COL = 1, + MALI_C55_AEXP_SECOND_ROW_OR_COL = 2, +}; + +/** + * enum mali_c55_aexp_hist_plane_mode - Mode for the AEXP Histograms + * @MALI_C55_AEXP_HIST_COMBINED: All color planes in one 1024-bin histogram + * @MALI_C55_AEXP_HIST_SEPARATE: Each color plane in one 256-bin histogram with a bin width of 16 + * @MALI_C55_AEXP_HIST_FOCUS_00: Top left plane in the first bank, rest in second bank + * @MALI_C55_AEXP_HIST_FOCUS_01: Top right plane in the first bank, rest in second bank + * @MALI_C55_AEXP_HIST_FOCUS_10: Bottom left plane in the first bank, rest in second bank + * @MALI_C55_AEXP_HIST_FOCUS_11: Bottom right plane in the first bank, rest in second bank + * + * In the "focus" modes statistics are collected into two 512-bin histograms + * with a bin width of 8. One colour plane is in the first histogram with the + * remainder combined into the second. The four options represent which of the + * four positions in a bayer pattern are the focused plane. + */ +enum mali_c55_aexp_hist_plane_mode { + MALI_C55_AEXP_HIST_COMBINED = 0, + MALI_C55_AEXP_HIST_SEPARATE = 1, + MALI_C55_AEXP_HIST_FOCUS_00 = 4, + MALI_C55_AEXP_HIST_FOCUS_01 = 5, + MALI_C55_AEXP_HIST_FOCUS_10 = 6, + MALI_C55_AEXP_HIST_FOCUS_11 = 7, +}; + +/** + * struct mali_c55_params_aexp_hist - configuration for AEXP metering hists + * + * This struct allows users to configure the 1024-bin AEXP histograms. Broadly + * speaking the parameters allow you to mask particular regions of the image and + * to select different kinds of histogram. + * + * The skip_x, offset_x, skip_y and offset_y fields allow users to ignore or + * mask pixels in the frame by their position relative to the top left pixel. + * First, the skip_y, offset_x and offset_y fields define a 2x2 pixel pattern of + * which pixels covered by the pattern will be counted in the statistics. + * + * If skip_y == 0 then two pixels from each coveredregion will be counted. If + * both offset_x and offset_y are zero, then the two left-most pixels in each + * 2x2 pixel region covered by the pattern will be counted. Setting offset_x = 1 + * will discount the top left pixel and count the top right pixel. Setting + * offset_y = 1 will discount the bottom left pixel and count the bottom right + * pixel. + * + * If skip_y != 0 then only a single pixel from each region covered by the + * pattern will be counted. In this case offset_x controls whether the pixel + * that's counted is in the left (if offset_x == 0) or right (if offset_x == 1) + * column and offset_y controls whether the pixel that's counted is in the top + * (if offset_y == 0) or bottom (if offset_y == 1) row. + * + * The skip_x and skip_y fields control how the pixel masking pattern is + * repeated across the image data. The first instance of the pattern is always + * in the top left of the image data. The skip_x field controls how many pixels + * are ignored in the x direction before the pixel masking pattern is repeated. + * The skip_y field controls how many pixels are ignored in the y direction + * before the pixel masking pattern is repeated. + * + * These fields can be used to reduce the number of pixels counted for the + * statistics, but it's important to be careful to configure them correctly. + * Some combinations of values will result in colour components from the input + * data being ignored entirely, for example in the following configuration: + * + * skip_x = 0 + * offset_x = 0 + * skip_y = 0 + * offset_y = 0 + * + * Only the R and Gb components of RGGB data that was input would be collected. + * Similarly in the following configuration: + * + * skip_x = 0 + * offset_x = 0 + * skip_y = 1 + * offset_y = 1 + * + * Only the Gb component of RGGB data that was input would be collected. To + * correct things such that all 4 colour components were included it would be + * necessary to set the skip_x and skip_y fields in a way that resulted in all + * four colour components being collected: + * + * skip_x = 1 + * offset_x = 0 + * skip_y = 1 + * offset_y = 1 + * + * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AEXP_HIST or + * MALI_C55_PARAM_BLOCK_AEXP_IHIST from :c:type:`mali_c55_param_block_type`. + * + * @header: The Mali-C55 parameters block header + * @skip_x: Horizontal decimation. See enum mali_c55_aexp_skip_x + * @offset_x: Skip the first column, or not. See enum mali_c55_aexp_row_column_offset + * @skip_y: Vertical decimation. See enum mali_c55_aexp_skip_y + * @offset_y: Skip the first row, or not. See enum mali_c55_aexp_row_column_offset + * @scale_bottom: Scale pixels in bottom half of intensity range: 0=1x ,1=2x, 2=4x, 4=8x, 4=16x + * @scale_top: scale pixels in top half of intensity range: 0=1x ,1=2x, 2=4x, 4=8x, 4=16x + * @plane_mode: Plane separation mode. See enum mali_c55_aexp_hist_plane_mode + * @tap_point: Tap point for histogram from enum mali_c55_aexp_hist_tap_points. + * This parameter is unused for the post-Iridix Histogram + */ +struct mali_c55_params_aexp_hist { + struct mali_c55_params_block_header header; + __u8 skip_x; + __u8 offset_x; + __u8 skip_y; + __u8 offset_y; + __u8 scale_bottom; + __u8 scale_top; + __u8 plane_mode; + __u8 tap_point; +}; + +/** + * struct mali_c55_params_aexp_weights - Array of weights for AEXP metering + * + * This struct allows users to configure the weighting for both of the 1024-bin + * AEXP histograms. The pixel data collected for each zone is multiplied by the + * corresponding weight from this array, which may be zero if the intention is + * to mask off the zone entirely. + * + * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS + * or MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS from :c:type:`mali_c55_param_block_type`. + * + * @header: The Mali-C55 parameters block header + * @nodes_used_horiz: Number of active zones horizontally [0..15] + * @nodes_used_vert: Number of active zones vertically [0..15] + * @zone_weights: Zone weighting. Index is row*col where 0,0 is the top + * left zone continuing in raster order. Each zone can be + * weighted in the range [0..15]. The number of rows and + * columns is defined by @nodes_used_vert and + * @nodes_used_horiz + */ +struct mali_c55_params_aexp_weights { + struct mali_c55_params_block_header header; + __u8 nodes_used_horiz; + __u8 nodes_used_vert; + __u8 zone_weights[MALI_C55_MAX_ZONES]; +}; + +/** + * struct mali_c55_params_digital_gain - Digital gain value + * + * This struct carries a digital gain value to set in the ISP. + * + * header.type should be set to MALI_C55_PARAM_BLOCK_DIGITAL_GAIN from + * :c:type:`mali_c55_param_block_type` for this block. + * + * @header: The Mali-C55 parameters block header + * @gain: The digital gain value to apply, in Q5.8 format. + */ +struct mali_c55_params_digital_gain { + struct mali_c55_params_block_header header; + __u16 gain; +}; + +/** + * enum mali_c55_awb_stats_mode - Statistics mode for AWB + * @MALI_C55_AWB_MODE_GRBR: Statistics collected as Green/Red and Blue/Red ratios + * @MALI_C55_AWB_MODE_RGBG: Statistics collected as Red/Green and Blue/Green ratios + */ +enum mali_c55_awb_stats_mode { + MALI_C55_AWB_MODE_GRBR = 0, + MALI_C55_AWB_MODE_RGBG, +}; + +/** + * struct mali_c55_params_awb_gains - Gain settings for auto white balance + * + * This struct allows users to configure the gains for auto-white balance. There + * are four gain settings corresponding to each colour channel in the bayer + * domain. Although named generically, the association between the gain applied + * and the colour channel is done automatically within the ISP depending on the + * input format, and so the following mapping always holds true:: + * + * gain00 = R + * gain01 = Gr + * gain10 = Gb + * gain11 = B + * + * All of the gains are stored in Q4.8 format. + * + * header.type should be set to one of either MALI_C55_PARAM_BLOCK_AWB_GAINS or + * MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP from :c:type:`mali_c55_param_block_type`. + * + * @header: The Mali-C55 parameters block header + * @gain00: Multiplier for colour channel 00 + * @gain01: Multiplier for colour channel 01 + * @gain10: Multiplier for colour channel 10 + * @gain11: Multiplier for colour channel 11 + */ +struct mali_c55_params_awb_gains { + struct mali_c55_params_block_header header; + __u16 gain00; + __u16 gain01; + __u16 gain10; + __u16 gain11; +}; + +/** + * enum mali_c55_params_awb_tap_points - Tap points for the AWB statistics + * @MALI_C55_AWB_STATS_TAP_PF: Immediately after the Purple Fringe block + * @MALI_C55_AWB_STATS_TAP_CNR: Immediately after the CNR block + */ +enum mali_c55_params_awb_tap_points { + MALI_C55_AWB_STATS_TAP_PF = 0, + MALI_C55_AWB_STATS_TAP_CNR, +}; + +/** + * struct mali_c55_params_awb_config - Stats settings for auto-white balance + * + * This struct allows the configuration of the statistics generated for auto + * white balance. Pixel intensity limits can be set to exclude overly bright or + * dark regions of an image from the statistics entirely. Colour ratio minima + * and maxima can be set to discount pixels who's ratios fall outside the + * defined boundaries; there are two sets of registers to do this - the + * "min/max" ratios which bound a region and the "high/low" ratios which further + * trim the upper and lower ratios. For example with the boundaries configured + * as follows, only pixels whos colour ratios falls into the region marked "A" + * would be counted:: + * + * cr_high + * 2.0 | | + * | cb_max --> _________________________v_____ + * 1.8 | | \ | + * | | \ | + * 1.6 | | \ | + * | | \ | + * c 1.4 | cb_low -->|\ A \|<-- cb_high + * b | | \ | + * 1.2 | | \ | + * r | | \ | + * a 1.0 | cb_min --> |____\_________________________| + * t | ^ ^ ^ + * i 0.8 | | | | + * o | cr_min | cr_max + * s 0.6 | | + * | cr_low + * 0.4 | + * | + * 0.2 | + * | + * 0.0 |_______________________________________________________________ + * 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 + * cr ratios + * + * header.type should be set to MALI_C55_PARAM_BLOCK_AWB_CONFIG from + * :c:type:`mali_c55_param_block_type` for this block. + * + * @header: The Mali-C55 parameters block header + * @tap_point: The tap point from enum mali_c55_params_awb_tap_points + * @stats_mode: AWB statistics collection mode, see :c:type:`mali_c55_awb_stats_mode` + * @white_level: Upper pixel intensity (I.E. raw pixel values) limit + * @black_level: Lower pixel intensity (I.E. raw pixel values) limit + * @cr_max: Maximum R/G ratio (Q4.8 format) + * @cr_min: Minimum R/G ratio (Q4.8 format) + * @cb_max: Maximum B/G ratio (Q4.8 format) + * @cb_min: Minimum B/G ratio (Q4.8 format) + * @nodes_used_horiz: Number of active zones horizontally [0..15] + * @nodes_used_vert: Number of active zones vertically [0..15] + * @cr_high: R/G ratio trim high (Q4.8 format) + * @cr_low: R/G ratio trim low (Q4.8 format) + * @cb_high: B/G ratio trim high (Q4.8 format) + * @cb_low: B/G ratio trim low (Q4.8 format) + */ +struct mali_c55_params_awb_config { + struct mali_c55_params_block_header header; + __u8 tap_point; + __u8 stats_mode; + __u16 white_level; + __u16 black_level; + __u16 cr_max; + __u16 cr_min; + __u16 cb_max; + __u16 cb_min; + __u8 nodes_used_horiz; + __u8 nodes_used_vert; + __u16 cr_high; + __u16 cr_low; + __u16 cb_high; + __u16 cb_low; +}; + +#define MALI_C55_NUM_MESH_SHADING_ELEMENTS 3072 + +/** + * struct mali_c55_params_mesh_shading_config - Mesh shading configuration + * + * The mesh shading correction module allows programming a separate table of + * either 16x16 or 32x32 node coefficients for 3 different light sources. The + * final correction coefficients applied are computed by blending the + * coefficients from two tables together. + * + * A page of 1024 32-bit integers is associated to each colour channel, with + * pages stored consecutively in memory. Each 32-bit integer packs 3 8-bit + * correction coefficients for a single node, one for each of the three light + * sources. The 8 most significant bits are unused. The following table + * describes the layout:: + * + * +----------- Page (Colour Plane) 0 -------------+ + * | @mesh[i] | Mesh Point | Bits | Light Source | + * +-----------+------------+-------+--------------+ + * | 0 | 0,0 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +-----------+------------+-------+--------------+ + * | 1 | 0,1 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +-----------+------------+-------+--------------+ + * | ... | ... | ... | ... | + * +-----------+------------+-------+--------------+ + * | 1023 | 31,31 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +----------- Page (Colour Plane) 1 -------------+ + * | @mesh[i] | Mesh Point | Bits | Light Source | + * +-----------+------------+-------+--------------+ + * | 1024 | 0,0 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +-----------+------------+-------+--------------+ + * | 1025 | 0,1 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +-----------+------------+-------+--------------+ + * | ... | ... | ... | ... | + * +-----------+------------+-------+--------------+ + * | 2047 | 31,31 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +----------- Page (Colour Plane) 2 -------------+ + * | @mesh[i] | Mesh Point | Bits | Light Source | + * +-----------+------------+-------+--------------+ + * | 2048 | 0,0 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +-----------+------------+-------+--------------+ + * | 2049 | 0,1 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +-----------+------------+-------+--------------+ + * | ... | ... | ... | ... | + * +-----------+------------+-------+--------------+ + * | 3071 | 31,31 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +-----------+------------+-------+--------------+ + * + * The @mesh_scale member determines the precision and minimum and maximum gain. + * For example if @mesh_scale is 0 and therefore selects 0 - 2x gain, a value of + * 0 in a coefficient means 0.0 gain, a value of 128 means 1.0 gain and 255 + * means 2.0 gain. + * + * header.type should be set to MALI_C55_PARAM_MESH_SHADING_CONFIG from + * :c:type:`mali_c55_param_block_type` for this block. + * + * @header: The Mali-C55 parameters block header + * @mesh_show: Output the mesh data rather than image data + * @mesh_scale: Set the precision and maximum gain range of mesh shading + * - 0 = 0-2x gain + * - 1 = 0-4x gain + * - 2 = 0-8x gain + * - 3 = 0-16x gain + * - 4 = 1-2x gain + * - 5 = 1-3x gain + * - 6 = 1-5x gain + * - 7 = 1-9x gain + * @mesh_page_r: Mesh page select for red colour plane [0..2] + * @mesh_page_g: Mesh page select for green colour plane [0..2] + * @mesh_page_b: Mesh page select for blue colour plane [0..2] + * @mesh_width: Number of horizontal nodes minus 1 [15,31] + * @mesh_height: Number of vertical nodes minus 1 [15,31] + * @mesh: Mesh shading correction tables + */ +struct mali_c55_params_mesh_shading_config { + struct mali_c55_params_block_header header; + __u8 mesh_show; + __u8 mesh_scale; + __u8 mesh_page_r; + __u8 mesh_page_g; + __u8 mesh_page_b; + __u8 mesh_width; + __u8 mesh_height; + __u32 mesh[MALI_C55_NUM_MESH_SHADING_ELEMENTS]; +}; + +/** enum mali_c55_params_mesh_alpha_bank - Mesh shading table bank selection + * @MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS1 - Select Light Sources 0 and 1 + * @MALI_C55_MESH_ALPHA_BANK_LS1_AND_LS2 - Select Light Sources 1 and 2 + * @MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS2 - Select Light Sources 0 and 2 + */ +enum mali_c55_params_mesh_alpha_bank { + MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS1 = 0, + MALI_C55_MESH_ALPHA_BANK_LS1_AND_LS2 = 1, + MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS2 = 4 +}; + +/** + * struct mali_c55_params_mesh_shading_selection - Mesh table selection + * + * The module computes the final correction coefficients by blending the ones + * from two light source tables, which are selected (independently for each + * colour channel) by the @mesh_alpha_bank_r/g/b fields. + * + * The final blended coefficients for each node are calculated using the + * following equation: + * + * Final coefficient = (a * LS\ :sub:`b`\ + (256 - a) * LS\ :sub:`a`\) / 256 + * + * Where a is the @mesh_alpha_r/g/b value, and LS\ :sub:`a`\ and LS\ :sub:`b`\ + * are the node cofficients for the two tables selected by the + * @mesh_alpha_bank_r/g/b value. + * + * The scale of the applied correction may also be controlled by tuning the + * @mesh_strength member. This is a modifier to the final coefficients which can + * be used to globally reduce the gains applied. + * + * header.type should be set to MALI_C55_PARAM_MESH_SHADING_SELECTION from + * :c:type:`mali_c55_param_block_type` for this block. + * + * @header: The Mali-C55 parameters block header + * @mesh_alpha_bank_r: Red mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`) + * @mesh_alpha_bank_g: Green mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`) + * @mesh_alpha_bank_b: Blue mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`) + * @mesh_alpha_r: Blend coefficient for R [0..255] + * @mesh_alpha_g: Blend coefficient for G [0..255] + * @mesh_alpha_b: Blend coefficient for B [0..255] + * @mesh_strength: Mesh strength in Q4.12 format [0..4096] + */ +struct mali_c55_params_mesh_shading_selection { + struct mali_c55_params_block_header header; + __u8 mesh_alpha_bank_r; + __u8 mesh_alpha_bank_g; + __u8 mesh_alpha_bank_b; + __u8 mesh_alpha_r; + __u8 mesh_alpha_g; + __u8 mesh_alpha_b; + __u16 mesh_strength; +}; + +/** + * union mali_c55_params_block - Generalisation of a parameter block + * + * This union allows the driver to treat a block as a generic pointer to this + * union and safely access the header and block-specific struct without having + * to resort to casting. The header member is accessed first, and the type field + * checked which allows the driver to determine which of the other members + * should be used. The data member at the end allows a pointer to an address + * within the data member of :c:type:`mali_c55_params_buffer` to initialise a + * union variable. + * + * @header: Pointer to the shared header struct embedded as the + * first member of all the possible other members (except + * @data). This member would be accessed first and the type + * field checked to determine which of the other members + * should be accessed. + * @sensor_offs: For header->type == MALI_C55_PARAM_BLOCK_SENSOR_OFFS + * @aexp_hist: For header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST and + * header->type == MALI_C55_PARAM_BLOCK_AEXP_IHIST + * @aexp_weights: For header->type == MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS + * and header->type = MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS + * @digital_gain: For header->type == MALI_C55_PARAM_BLOCK_DIGITAL_GAIN + * @awb_gains: For header->type == MALI_C55_PARAM_BLOCK_AWB_GAINS and + * header->type = MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP + * @awb_config: For header->type == MALI_C55_PARAM_MESH_SHADING_CONFIG + * @shading_config: For header->type == MALI_C55_PARAM_MESH_SHADING_SELECTION + * @shading_selection: For header->type == MALI_C55_PARAM_BLOCK_SENSOR_OFFS + * @data: Allows easy initialisation of a union variable with a + * pointer into a __u8 array. + */ +union mali_c55_params_block { + struct mali_c55_params_block_header *header; + struct mali_c55_params_sensor_off_preshading *sensor_offs; + struct mali_c55_params_aexp_hist *aexp_hist; + struct mali_c55_params_aexp_weights *aexp_weights; + struct mali_c55_params_digital_gain *digital_gain; + struct mali_c55_params_awb_gains *awb_gains; + struct mali_c55_params_awb_config *awb_config; + struct mali_c55_params_mesh_shading_config *shading_config; + struct mali_c55_params_mesh_shading_selection *shading_selection; + __u8 *data; +}; + +/** + * define MALI_C55_PARAMS_MAX_SIZE - Maximum size of all Mali C55 Parameters + * + * Though the parameters for the Mali-C55 are passed as optional blocks, the + * driver still needs to know the absolute maximum size so that it can allocate + * a buffer sized appropriately to accommodate userspace attempting to set all + * possible parameters in a single frame. + * + * Some structs are in this list multiple times. Where that's the case, it just + * reflects the fact that the same struct can be used with multiple different + * header types from :c:type:`mali_c55_param_block_type`. + */ +#define MALI_C55_PARAMS_MAX_SIZE \ + (sizeof(struct mali_c55_params_sensor_off_preshading) + \ + sizeof(struct mali_c55_params_aexp_hist) + \ + sizeof(struct mali_c55_params_aexp_weights) + \ + sizeof(struct mali_c55_params_aexp_hist) + \ + sizeof(struct mali_c55_params_aexp_weights) + \ + sizeof(struct mali_c55_params_digital_gain) + \ + sizeof(struct mali_c55_params_awb_gains) + \ + sizeof(struct mali_c55_params_awb_config) + \ + sizeof(struct mali_c55_params_awb_gains) + \ + sizeof(struct mali_c55_params_mesh_shading_config) + \ + sizeof(struct mali_c55_params_mesh_shading_selection)) + +/** + * struct mali_c55_params_buffer - 3A configuration parameters + * + * This struct contains the configuration parameters of the Mali-C55 ISP + * algorithms, serialized by userspace into a data buffer. Each configuration + * parameter block is represented by a block-specific structure which contains a + * :c:type:`mali_c55_params_block_header` entry as first member. Userspace + * populates the @data buffer with configuration parameters for the blocks that + * it intends to configure. As a consequence, the data buffer effective size + * changes according to the number of ISP blocks that userspace intends to + * configure. + * + * The parameters buffer is versioned by the @version field to allow modifying + * and extending its definition. Userspace shall populate the @version field to + * inform the driver about the version it intends to use. The driver will parse + * and handle the @data buffer according to the data layout specific to the + * indicated version and return an error if the desired version is not + * supported. + * + * For each ISP block that userspace wants to configure, a block-specific + * structure is appended to the @data buffer, one after the other without gaps + * in between nor overlaps. Userspace shall populate the @total_size field with + * the effective size, in bytes, of the @data buffer. + * + * The expected memory layout of the parameters buffer is:: + * + * +-------------------- struct mali_c55_params_buffer ------------------+ + * | version = MALI_C55_PARAM_BUFFER_V1; | + * | total_size = sizeof(struct mali_c55_params_sensor_off_preshading) | + * | sizeof(struct mali_c55_params_aexp_hist); | + * | +------------------------- data ---------------------------------+ | + * | | +--------- struct mali_c55_params_sensor_off_preshading ------+ | | + * | | | +-------- struct mali_c55_params_block_header header -----+ | | | + * | | | | type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS; | | | | + * | | | | enabled = 1; | | | | + * | | | | size = | | | | + * | | | | sizeof(struct mali_c55_params_sensor_off_preshading);| | | | + * | | | +---------------------------------------------------------+ | | | + * | | | chan00 = ...; | | | + * | | | chan01 = ...; | | | + * | | | chan10 = ...; | | | + * | | | chan11 = ...; | | | + * | | +------------ struct mali_c55_params_aexp_hist ---------------+ | | + * | | | +-------- struct mali_c55_params_block_header header -----+ | | | + * | | | | type = MALI_C55_PARAM_BLOCK_AEXP_HIST; | | | | + * | | | | enabled = 1; | | | | + * | | | | size = sizeof(struct mali_c55_params_aexp_hist); | | | | + * | | | +---------------------------------------------------------+ | | | + * | | | skip_x = ...; | | | + * | | | offset_x = ...; | | | + * | | | skip_y = ...; | | | + * | | | offset_y = ...; | | | + * | | | scale_bottom = ...; | | | + * | | | scale_top = ...; | | | + * | | | plane_mode = ...; | | | + * | | | tap_point = ...; | | | + * | | +-------------------------------------------------------------+ | | + * | +-----------------------------------------------------------------+ | + * +---------------------------------------------------------------------+ + * + * @version: The version from :c:type:`mali_c55_param_buffer_version` + * @total_size: The Mali-C55 configuration data effective size, excluding this + * header + * @data: The Mali-C55 configuration blocks data + */ +struct mali_c55_params_buffer { + __u8 version; + __u32 total_size; + __u8 data[MALI_C55_PARAMS_MAX_SIZE]; +}; + +#endif /* __UAPI_MALI_C55_CONFIG_H */ From patchwork Tue Jul 9 14:49:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20633 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 12BCBBD87C for ; Tue, 9 Jul 2024 14:50:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 338B863381; Tue, 9 Jul 2024 16:50:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Kf3gviK8"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E340C6336D for ; Tue, 9 Jul 2024 16:49:55 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4B0141894; Tue, 9 Jul 2024 16:49:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720536563; bh=l+ruGOtUEsf+oom+1AAtuuNzfM0E7wBmzYXQw3sMT5s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Kf3gviK8nLoeEr47/bBnYCNaENCx0F85npsjWERS5DmVLazm0qItWlUOaIXCTRKhY mWYpr6iXQfjeNfRpX59iFwFZNKTP7lxezDdDOYHEeO6MGyl69hL+h/WkIzWxLhhwma DbX3Pv9u9qpNf9ycWOXpZR0UVO8PyiwQVVnH7TZQ= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Nayden Kanchev , Jacopo Mondi Subject: [PATCH v2 02/10] include: linux: Add Mali-C55 Stats and Params V4L2 format Date: Tue, 9 Jul 2024 15:49:42 +0100 Message-Id: <20240709144950.3277837-3-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709144950.3277837-1-dan.scally@ideasonboard.com> References: <20240709144950.3277837-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the new format describing the Mali C55's Statistics and Parameters to videodev2. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Acked-by: Kieran Bingham --- Changes in v2: - None include/linux/videodev2.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h index 7fe522e8..fbb8fbcd 100644 --- a/include/linux/videodev2.h +++ b/include/linux/videodev2.h @@ -839,6 +839,9 @@ struct v4l2_pix_format { /* The metadata format identifier for FE stats buffers. */ #define V4L2_META_FMT_RPI_FE_STATS v4l2_fourcc('R', 'P', 'F', 'S') +#define V4L2_META_FMT_MALI_C55_PARAMS v4l2_fourcc('C', '5', '5', 'P') /* ARM Mali-C55 Parameters */ +#define V4L2_META_FMT_MALI_C55_3A_STATS v4l2_fourcc('C', '5', '5', 'S') /* ARM Mali-C55 3A Statistics */ + /* priv field value to indicates that subsequent fields are valid. */ #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe From patchwork Tue Jul 9 14:49:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20634 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AB103BD87C for ; Tue, 9 Jul 2024 14:50:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B1FDA63378; Tue, 9 Jul 2024 16:50:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VytxQNf7"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1EE0B6336E for ; Tue, 9 Jul 2024 16:49:56 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9937E13D7; Tue, 9 Jul 2024 16:49:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720536563; bh=b8+81gbruk7rCiSmK/LBIHCqd9+uD2sWnh6Osk3O8mw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VytxQNf7sHKx4KqccQlCekWz1iidtbdJPw9/viootE8/9yxMpaEPyCv2Si8Spw+UB igySf2ntqoSY6yHd8xNULcfQst3a2f6Q8A6BgNtS67JWFkLkGrcAinu1WQzj19hici raN/BTYn+m61KOuX51HX3jWbpsJOaGig/0aTDPpc= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Nayden Kanchev , Jacopo Mondi Subject: [PATCH v2 03/10] libcamera: mali-c55: Acquire and plumb in 3a params and stats Date: Tue, 9 Jul 2024 15:49:43 +0100 Message-Id: <20240709144950.3277837-4-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709144950.3277837-1-dan.scally@ideasonboard.com> References: <20240709144950.3277837-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Acquire the mali-c55 3a stats and parameters video devices during ::match() and plumb them in. For this commit we simply allocate and release buffers for the statistics and parameters. Statistics buffers are queue and dequeued from the stats video device but their contents are for now untouched. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Reviewed-by: Kieran Bingham --- Changes in v2: - Links between the ISP subdevice and the statistics and parameters video devices are now enabled (previously they were immutable) src/libcamera/pipeline/mali-c55/mali-c55.cpp | 174 ++++++++++++++++++- 1 file changed, 171 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 916b1d30..1e5674fc 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -12,6 +12,7 @@ #include #include +#include #include #include @@ -26,6 +27,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/framebuffer.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/v4l2_subdevice.h" @@ -510,6 +512,8 @@ public: int exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) override; + int allocateBuffers(Camera *camera); + void freeBuffers(Camera *camera); int start(Camera *camera, const ControlList *controls) override; void stopDevice(Camera *camera) override; @@ -517,6 +521,7 @@ public: int queueRequestDevice(Camera *camera, Request *request) override; void bufferReady(FrameBuffer *buffer); + void statsBufferReady(FrameBuffer *buffer); bool match(DeviceEnumerator *enumerator) override; @@ -576,6 +581,14 @@ private: MediaDevice *media_; std::unique_ptr isp_; + std::unique_ptr stats_; + std::unique_ptr params_; + + std::vector> statsBuffers_; + std::queue availableStatsBuffers_; + + std::vector> paramsBuffers_; + std::queue availableParamsBuffers_; std::array pipes_; @@ -835,6 +848,16 @@ int PipelineHandlerMaliC55::configure(Camera *camera, return ret; } + V4L2DeviceFormat statsFormat; + ret = stats_->getFormat(&statsFormat); + if (ret) + return ret; + + if (statsFormat.planes[0].size != sizeof(struct mali_c55_stats_buffer)) { + LOG(MaliC55, Error) << "3a stats buffer size invalid"; + return -EINVAL; + } + /* * Propagate the format to the ISP sink pad and configure the input * crop rectangle (no crop at the moment). @@ -896,6 +919,27 @@ int PipelineHandlerMaliC55::configure(Camera *camera, pipe->stream = stream; } + /* + * Enable the media link between the ISP subdevice and the statistics + * video device. + */ + const MediaEntity *ispEntity = isp_->entity(); + ret = ispEntity->getPadByIndex(3)->links()[0]->setEnabled(true); + if (ret) { + LOG(MaliC55, Error) << "Couldn't enable statistics link"; + return ret; + } + + /* + * Enable the media link between the ISP subdevice and the parameters + * video device. + */ + ret = ispEntity->getPadByIndex(4)->links()[0]->setEnabled(true); + if (ret) { + LOG(MaliC55, Error) << "Couldn't enable parameters link"; + return ret; + } + data->updateControls(); return 0; @@ -910,27 +954,110 @@ int PipelineHandlerMaliC55::exportFrameBuffers(Camera *camera, Stream *stream, return pipe->cap->exportBuffers(count, buffers); } -int PipelineHandlerMaliC55::start([[maybe_unused]] Camera *camera, [[maybe_unused]] const ControlList *controls) +void PipelineHandlerMaliC55::freeBuffers([[maybe_unused]] Camera *camera) { + while (!availableStatsBuffers_.empty()) + availableStatsBuffers_.pop(); + while (!availableParamsBuffers_.empty()) + availableParamsBuffers_.pop(); + + statsBuffers_.clear(); + paramsBuffers_.clear(); + + if (stats_->releaseBuffers()) + LOG(MaliC55, Error) << "Failed to release stats buffers"; + + if (params_->releaseBuffers()) + LOG(MaliC55, Error) << "Failed to release stats buffers"; + + return; +} + +int PipelineHandlerMaliC55::allocateBuffers(Camera *camera) +{ + MaliC55CameraData *data = cameraData(camera); + unsigned int bufferCount; + int ret; + + bufferCount = std::max({ + data->frStream_.configuration().bufferCount, + data->dsStream_.configuration().bufferCount, + }); + + ret = stats_->allocateBuffers(bufferCount, &statsBuffers_); + if (ret < 0) + return ret; + + for (std::unique_ptr &buffer : statsBuffers_) + availableStatsBuffers_.push(buffer.get()); + + ret = params_->allocateBuffers(bufferCount, ¶msBuffers_); + if (ret < 0) + return ret; + + for (std::unique_ptr &buffer : paramsBuffers_) + availableParamsBuffers_.push(buffer.get()); + + return 0; +} + +int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const ControlList *controls) +{ + int ret; + + ret = allocateBuffers(camera); + if (ret) + return ret; + for (MaliC55Pipe &pipe : pipes_) { if (!pipe.stream) continue; Stream *stream = pipe.stream; - int ret = pipe.cap->importBuffers(stream->configuration().bufferCount); + ret = pipe.cap->importBuffers(stream->configuration().bufferCount); if (ret) { LOG(MaliC55, Error) << "Failed to import buffers"; + freeBuffers(camera); return ret; } ret = pipe.cap->streamOn(); if (ret) { LOG(MaliC55, Error) << "Failed to start stream"; + freeBuffers(camera); return ret; } } + ret = stats_->streamOn(); + if (ret) { + LOG(MaliC55, Error) << "Failed to start stats stream"; + + for (MaliC55Pipe &pipe : pipes_) { + if (pipe.stream) + pipe.cap->streamOff(); + } + + freeBuffers(camera); + return ret; + } + + ret = params_->streamOn(); + if (ret) { + LOG(MaliC55, Error) << "Failed to start params stream"; + + stats_->streamOff(); + + for (MaliC55Pipe &pipe : pipes_) { + if (pipe.stream) + pipe.cap->streamOff(); + } + + freeBuffers(camera); + return ret; + } + return 0; } @@ -943,6 +1070,10 @@ void PipelineHandlerMaliC55::stopDevice([[maybe_unused]] Camera *camera) pipe.cap->streamOff(); pipe.cap->releaseBuffers(); } + + stats_->streamOff(); + params_->streamOff(); + freeBuffers(camera); } void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, @@ -1043,8 +1174,24 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) { + FrameBuffer *statsBuffer; int ret; + if (availableStatsBuffers_.empty()) { + LOG(MaliC55, Error) << "Stats buffer underrun"; + return -ENOENT; + } + + statsBuffer = availableStatsBuffers_.front(); + availableStatsBuffers_.pop(); + + /* + * We need to associate the Request to this buffer even though it's a + * purely internal one because we will need to use request->sequence() + * later. + */ + statsBuffer->_d()->setRequest(request); + for (auto &[stream, buffer] : request->buffers()) { MaliC55Pipe *pipe = pipeFromStream(cameraData(camera), stream); @@ -1062,6 +1209,10 @@ int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) */ applyScalerCrop(camera, request->controls()); + ret = stats_->queueBuffer(statsBuffer); + if (ret) + return ret; + return 0; } @@ -1073,6 +1224,11 @@ void PipelineHandlerMaliC55::bufferReady(FrameBuffer *buffer) completeRequest(request); } +void PipelineHandlerMaliC55::statsBufferReady(FrameBuffer *buffer) +{ + availableStatsBuffers_.push(buffer); +} + void PipelineHandlerMaliC55::registerMaliCamera(std::unique_ptr data, const std::string &name) { @@ -1150,7 +1306,7 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) const MediaPad *ispSink; /* - * We search for just the ISP subdevice and the full resolution pipe. + * We search for just the always-available elements of the media graph. * The TPG and the downscale pipe are both optional blocks and may not * be fitted. */ @@ -1158,6 +1314,8 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) dm.add("mali-c55 isp"); dm.add("mali-c55 resizer fr"); dm.add("mali-c55 fr"); + dm.add("mali-c55 3a stats"); + dm.add("mali-c55 3a params"); media_ = acquireMediaDevice(enumerator, dm); if (!media_) @@ -1167,6 +1325,14 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) if (isp_->open() < 0) return false; + stats_ = V4L2VideoDevice::fromEntityName(media_, "mali-c55 3a stats"); + if (stats_->open() < 0) + return false; + + params_ = V4L2VideoDevice::fromEntityName(media_, "mali-c55 3a params"); + if (params_->open() < 0) + return false; + MaliC55Pipe *frPipe = &pipes_[MaliC55FR]; frPipe->resizer = V4L2Subdevice::fromEntityName(media_, "mali-c55 resizer fr"); if (frPipe->resizer->open() < 0) @@ -1195,6 +1361,8 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) dsPipe->cap->bufferReady.connect(this, &PipelineHandlerMaliC55::bufferReady); } + stats_->bufferReady.connect(this, &PipelineHandlerMaliC55::statsBufferReady); + ispSink = isp_->entity()->getPadByIndex(0); if (!ispSink || ispSink->links().empty()) { LOG(MaliC55, Error) << "ISP sink pad error"; From patchwork Tue Jul 9 14:49:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20635 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EEE7ABD87C for ; Tue, 9 Jul 2024 14:50:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F042A6337F; Tue, 9 Jul 2024 16:50:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dN7PDmpm"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 69D5263372 for ; Tue, 9 Jul 2024 16:49:56 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EBFCE2393; Tue, 9 Jul 2024 16:49:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720536564; bh=CYwt4r5Pv9k929OcWB7FLw32F/rvqYtCohYrE/tlfi4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dN7PDmpmwW6HOmSd0dkAQt+GuCRvATHagNBZNLTWz+8Y+cOFv/AAYpeUsVpBlzfjC jPK9dKm0gYehYwfV0uw2vA0HSroVmEWYOjvXoJWs2WNRobinnQd6vSpmPTIS/+H2SJ XyDxNszANhOC0RMmYExZre/KzLl7x2HNaaClE7CU= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Nayden Kanchev , Jacopo Mondi Subject: [PATCH v2 04/10] ipa: mali-c55: Add Mali-C55 ISP IPA module Date: Tue, 9 Jul 2024 15:49:44 +0100 Message-Id: <20240709144950.3277837-5-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709144950.3277837-1-dan.scally@ideasonboard.com> References: <20240709144950.3277837-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a barebones IPA module for the Mali-C55 ISP. In this initial implementation pretty much only buffer plumbing is implemented. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Documentation/Doxyfile.in | 1 + include/libcamera/ipa/mali-c55.mojom | 34 +++ include/libcamera/ipa/meson.build | 1 + meson_options.txt | 2 +- src/ipa/mali-c55/algorithms/algorithm.h | 24 ++ src/ipa/mali-c55/algorithms/meson.build | 4 + src/ipa/mali-c55/data/meson.build | 8 + src/ipa/mali-c55/data/uncalibrated.yaml | 6 + src/ipa/mali-c55/ipa_context.cpp | 101 +++++++ src/ipa/mali-c55/ipa_context.h | 42 +++ src/ipa/mali-c55/mali-c55.cpp | 341 ++++++++++++++++++++++++ src/ipa/mali-c55/meson.build | 33 +++ src/ipa/mali-c55/module.h | 27 ++ 13 files changed, 623 insertions(+), 1 deletion(-) create mode 100644 include/libcamera/ipa/mali-c55.mojom create mode 100644 src/ipa/mali-c55/algorithms/algorithm.h create mode 100644 src/ipa/mali-c55/algorithms/meson.build create mode 100644 src/ipa/mali-c55/data/meson.build create mode 100644 src/ipa/mali-c55/data/uncalibrated.yaml create mode 100644 src/ipa/mali-c55/ipa_context.cpp create mode 100644 src/ipa/mali-c55/ipa_context.h create mode 100644 src/ipa/mali-c55/mali-c55.cpp create mode 100644 src/ipa/mali-c55/meson.build create mode 100644 src/ipa/mali-c55/module.h diff --git a/Documentation/Doxyfile.in b/Documentation/Doxyfile.in index abafcf6c..62ca622d 100644 --- a/Documentation/Doxyfile.in +++ b/Documentation/Doxyfile.in @@ -51,6 +51,7 @@ EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/span.h \ EXCLUDE_PATTERNS = @TOP_BUILDDIR@/include/libcamera/ipa/*_serializer.h \ @TOP_BUILDDIR@/include/libcamera/ipa/*_proxy.h \ @TOP_BUILDDIR@/include/libcamera/ipa/ipu3_*.h \ + @TOP_BUILDDIR@/include/libcamera/ipa/mali-c55_*.h \ @TOP_BUILDDIR@/include/libcamera/ipa/raspberrypi_*.h \ @TOP_BUILDDIR@/include/libcamera/ipa/rkisp1_*.h \ @TOP_BUILDDIR@/include/libcamera/ipa/vimc_*.h diff --git a/include/libcamera/ipa/mali-c55.mojom b/include/libcamera/ipa/mali-c55.mojom new file mode 100644 index 00000000..4f6f461c --- /dev/null +++ b/include/libcamera/ipa/mali-c55.mojom @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +module ipa.mali_c55; + +import "include/libcamera/ipa/core.mojom"; + +struct IPAConfigInfo { + libcamera.IPACameraSensorInfo sensorInfo; + libcamera.ControlInfoMap sensorControls; +}; + +interface IPAMaliC55Interface { + init(libcamera.IPASettings settings, IPAConfigInfo configInfo) + => (int32 ret, libcamera.ControlInfoMap ipaControls); + start() => (int32 ret); + stop(); + + configure(IPAConfigInfo configInfo, uint8 bayerOrder) + => (int32 ret, libcamera.ControlInfoMap ipaControls); + + mapBuffers(array buffers, bool readOnly); + unmapBuffers(array buffers); + + [async] queueRequest(uint32 frame, libcamera.ControlList reqControls); + [async] fillParams(uint32 request, uint32 bufferId); + [async] processStats(uint32 request, uint32 bufferId, + libcamera.ControlList sensorControls); +}; + +interface IPAMaliC55EventInterface { + paramsComputed(uint32 request); + statsProcessed(uint32 request, libcamera.ControlList metadata); + setSensorControls(libcamera.ControlList sensorControls); +}; diff --git a/include/libcamera/ipa/meson.build b/include/libcamera/ipa/meson.build index 3352d08f..3b872a7f 100644 --- a/include/libcamera/ipa/meson.build +++ b/include/libcamera/ipa/meson.build @@ -63,6 +63,7 @@ libcamera_generated_ipa_headers += custom_target('core_ipa_serializer_h', # Mapping from pipeline handler name to mojom file pipeline_ipa_mojom_mapping = { 'ipu3': 'ipu3.mojom', + 'mali-c55': 'mali-c55.mojom', 'rkisp1': 'rkisp1.mojom', 'rpi/vc4': 'raspberrypi.mojom', 'simple': 'soft.mojom', diff --git a/meson_options.txt b/meson_options.txt index 7aa41249..5365601c 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -32,7 +32,7 @@ option('gstreamer', option('ipas', type : 'array', - choices : ['ipu3', 'rkisp1', 'rpi/vc4', 'simple', 'vimc'], + choices : ['ipu3', 'mali-c55', 'rkisp1', 'rpi/vc4', 'simple', 'vimc'], description : 'Select which IPA modules to build') option('lc-compliance', diff --git a/src/ipa/mali-c55/algorithms/algorithm.h b/src/ipa/mali-c55/algorithms/algorithm.h new file mode 100644 index 00000000..afd90982 --- /dev/null +++ b/src/ipa/mali-c55/algorithms/algorithm.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas On Board + * + * algorithm.h - Mali-C55 control algorithm interface + */ + +#pragma once + +#include + +#include "module.h" + +namespace libcamera { + +namespace ipa::mali_c55 { + +class Algorithm : public libcamera::ipa::Algorithm +{ +}; + +} /* namespace ipa::mali_c55 */ + +} /* namespace libcamera */ diff --git a/src/ipa/mali-c55/algorithms/meson.build b/src/ipa/mali-c55/algorithms/meson.build new file mode 100644 index 00000000..f2203b15 --- /dev/null +++ b/src/ipa/mali-c55/algorithms/meson.build @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: CC0-1.0 + +mali_c55_ipa_algorithms = files([ +]) diff --git a/src/ipa/mali-c55/data/meson.build b/src/ipa/mali-c55/data/meson.build new file mode 100644 index 00000000..88109d17 --- /dev/null +++ b/src/ipa/mali-c55/data/meson.build @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: CC0-1.0 + +conf_files = files([ + 'uncalibrated.yaml' +]) + +install_data(conf_files, + install_dir : ipa_data_dir / 'mali-c55') diff --git a/src/ipa/mali-c55/data/uncalibrated.yaml b/src/ipa/mali-c55/data/uncalibrated.yaml new file mode 100644 index 00000000..2cdc39a8 --- /dev/null +++ b/src/ipa/mali-c55/data/uncalibrated.yaml @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +version: 1 +algorithms: +... diff --git a/src/ipa/mali-c55/ipa_context.cpp b/src/ipa/mali-c55/ipa_context.cpp new file mode 100644 index 00000000..99f76ecd --- /dev/null +++ b/src/ipa/mali-c55/ipa_context.cpp @@ -0,0 +1,101 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas On Board + * + * ipa_context.cpp - MaliC55 IPA Context + */ + +#include "ipa_context.h" + +/** + * \file ipa_context.h + * \brief Context and state information shared between the algorithms + */ + +namespace libcamera::ipa::mali_c55 { + +/** + * \struct IPASessionConfiguration + * \brief Session configuration for the IPA module + * + * The session configuration contains all IPA configuration parameters that + * remain constant during the capture session, from IPA module start to stop. + * It is typically set during the configure() operation of the IPA module, but + * may also be updated in the start() operation. + */ + +/** + * \struct IPAActiveState + * \brief Active state for algorithms + * + * The active state contains all algorithm-specific data that needs to be + * maintained by algorithms across frames. Unlike the session configuration, + * the active state is mutable and constantly updated by algorithms. The active + * state is accessible through the IPAContext structure. + * + * The active state stores two distinct categories of information: + * + * - The consolidated value of all algorithm controls. Requests passed to + * the queueRequest() function store values for controls that the + * application wants to modify for that particular frame, and the + * queueRequest() function updates the active state with those values. + * The active state thus contains a consolidated view of the value of all + * controls handled by the algorithm. + * + * - The value of parameters computed by the algorithm when running in auto + * mode. Algorithms running in auto mode compute new parameters every + * time statistics buffers are received (either synchronously, or + * possibly in a background thread). The latest computed value of those + * parameters is stored in the active state in the process() function. + * + * Each of the members in the active state belongs to a specific algorithm. A + * member may be read by any algorithm, but shall only be written by its owner. + */ + +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + * + * The frame context stores two distinct categories of information: + * + * - The value of the controls to be applied to the frame. These values are + * typically set in the queueRequest() function, from the consolidated + * control values stored in the active state. The frame context thus stores + * values for all controls related to the algorithm, not limited to the + * controls specified in the corresponding request, but consolidated from all + * requests that have been queued so far. + * + * For controls that can be set manually or computed by an algorithm + * (depending on the algorithm operation mode), such as for instance the + * colour gains for the AWB algorithm, the control value will be stored in + * the frame context in the queueRequest() function only when operating in + * manual mode. When operating in auto mode, the values are computed by the + * algorithm in process(), stored in the active state, and copied to the + * frame context in prepare(), just before being stored in the ISP parameters + * buffer. + * + * The queueRequest() function can also store ancillary data in the frame + * context, such as flags to indicate if (and what) control values have + * changed compared to the previous request. + * + * - Status information computed by the algorithm for a frame. For instance, + * the colour temperature estimated by the AWB algorithm from ISP statistics + * calculated on a frame is stored in the frame context for that frame in + * the process() function. + */ + +/** + * \struct IPAContext + * \brief Global IPA context data shared between all algorithms + * + * \var IPAContext::configuration + * \brief The IPA session configuration, immutable during the session + * + * \var IPAContext::activeState + * \brief The IPA active state, storing the latest state for all algorithms + * + * \var IPAContext::frameContexts + * \brief Ring buffer of per-frame contexts + */ + +} /* namespace libcamera::ipa::mali_c55 */ diff --git a/src/ipa/mali-c55/ipa_context.h b/src/ipa/mali-c55/ipa_context.h new file mode 100644 index 00000000..9e408a17 --- /dev/null +++ b/src/ipa/mali-c55/ipa_context.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas On Board + * + * ipa_context.h - Mali-C55 IPA Context + */ + +#pragma once + +#include + +#include + +namespace libcamera { + +namespace ipa::mali_c55 { + +struct IPASessionConfiguration { +}; + +struct IPAActiveState { +}; + +struct IPAFrameContext : public FrameContext { + struct { + uint32_t exposure; + double sensorGain; + } agc; +}; + +struct IPAContext { + IPASessionConfiguration configuration; + IPAActiveState activeState; + + FCQueue frameContexts; + + ControlInfoMap::Map ctrlMap; +}; + +} /* namespace ipa::mali_c55 */ + +} /* namespace libcamera*/ diff --git a/src/ipa/mali-c55/mali-c55.cpp b/src/ipa/mali-c55/mali-c55.cpp new file mode 100644 index 00000000..3daac3af --- /dev/null +++ b/src/ipa/mali-c55/mali-c55.cpp @@ -0,0 +1,341 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Ideas on Board Oy + * + * mali-c55.cpp - Mali-C55 ISP image processing algorithms + */ + +#include +#include +#include + +#include +#include + +#include +#include + +#include +#include +#include +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/mapped_framebuffer.h" +#include "libcamera/internal/yaml_parser.h" + +#include "algorithms/algorithm.h" +#include "libipa/camera_sensor_helper.h" + +#include "ipa_context.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(IPAMaliC55) + +namespace ipa::mali_c55 { + +/* Maximum number of frame contexts to be held */ +static constexpr uint32_t kMaxFrameContexts = 16; + +class IPAMaliC55 : public IPAMaliC55Interface, public Module +{ +public: + IPAMaliC55(); + + int init(const IPASettings &settings, const IPAConfigInfo &ipaConfig, + ControlInfoMap *ipaControls) override; + int start() override; + void stop() override; + int configure(const IPAConfigInfo &ipaConfig, uint8_t bayerOrder, + ControlInfoMap *ipaControls) override; + void mapBuffers(const std::vector &buffers, bool readOnly) override; + void unmapBuffers(const std::vector &buffers) override; + void queueRequest(const uint32_t frame, const ControlList &controls) override; + void fillParams(unsigned int request, uint32_t bufferId) override; + void processStats(unsigned int request, unsigned int bufferId, + const ControlList &sensorControls) override; + +protected: + std::string logPrefix() const override; + +private: + void updateControls(const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls); + void setControls(); + + std::map buffers_; + + ControlInfoMap sensorControls_; + + /* Interface to the Camera Helper */ + std::unique_ptr camHelper_; + + /* Local parameter storage */ + struct IPAContext context_; +}; + +static constexpr uint32_t kNumFrameContexts = 16; + +namespace { + +} /* namespace */ + +IPAMaliC55::IPAMaliC55() + : context_({ {}, {}, { kMaxFrameContexts }, {} }) +{ +} + +std::string IPAMaliC55::logPrefix() const +{ + return "mali-c55"; +} + +int IPAMaliC55::init(const IPASettings &settings, const IPAConfigInfo &ipaConfig, + ControlInfoMap *ipaControls) +{ + camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); + if (!camHelper_) { + LOG(IPAMaliC55, Error) + << "Failed to create camera sensor helper for " + << settings.sensorModel; + return -ENODEV; + } + + File file(settings.configurationFile); + if (!file.open(File::OpenModeFlag::ReadOnly)) { + int ret = file.error(); + LOG(IPAMaliC55, Error) + << "Failed to open configuration file " + << settings.configurationFile << ": " << strerror(-ret); + return ret; + } + + std::unique_ptr data = YamlParser::parse(file); + if (!data) + return -EINVAL; + + if (!data->contains("algorithms")) { + LOG(IPAMaliC55, Error) + << "Tuning file doesn't contain any algorithm"; + return -EINVAL; + } + + int ret = createAlgorithms(context_, (*data)["algorithms"]); + if (ret) + return ret; + + updateControls(ipaConfig.sensorInfo, ipaConfig.sensorControls, ipaControls); + + return 0; +} + +void IPAMaliC55::setControls() +{ + ControlList ctrls(sensorControls_); + + setSensorControls.emit(ctrls); +} + +int IPAMaliC55::start() +{ + return 0; +} + +void IPAMaliC55::stop() +{ + context_.frameContexts.clear(); +} + +void IPAMaliC55::updateControls(const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) +{ + ControlInfoMap::Map ctrlMap; + + /* + * Compute the frame duration limits. + * + * The frame length is computed assuming a fixed line length combined + * with the vertical frame sizes. + */ + const ControlInfo &v4l2HBlank = sensorControls.find(V4L2_CID_HBLANK)->second; + uint32_t hblank = v4l2HBlank.def().get(); + uint32_t lineLength = sensorInfo.outputSize.width + hblank; + + const ControlInfo &v4l2VBlank = sensorControls.find(V4L2_CID_VBLANK)->second; + std::array frameHeights{ + v4l2VBlank.min().get() + sensorInfo.outputSize.height, + v4l2VBlank.max().get() + sensorInfo.outputSize.height, + v4l2VBlank.def().get() + sensorInfo.outputSize.height, + }; + + std::array frameDurations; + for (unsigned int i = 0; i < frameHeights.size(); ++i) { + uint64_t frameSize = lineLength * frameHeights[i]; + frameDurations[i] = frameSize / (sensorInfo.pixelRate / 1000000U); + } + + ctrlMap[&controls::FrameDurationLimits] = ControlInfo(frameDurations[0], + frameDurations[1], + frameDurations[2]); + + /* + * Compute exposure time limits from the V4L2_CID_EXPOSURE control + * limits and the line duration. + */ + double lineDuration = sensorInfo.minLineLength / sensorInfo.pixelRate; + + const ControlInfo &v4l2Exposure = sensorControls.find(V4L2_CID_EXPOSURE)->second; + int32_t minExposure = v4l2Exposure.min().get() * lineDuration; + int32_t maxExposure = v4l2Exposure.max().get() * lineDuration; + int32_t defExposure = v4l2Exposure.def().get() * lineDuration; + ctrlMap[&controls::ExposureTime] = ControlInfo(minExposure, maxExposure, defExposure); + + /* Compute the analogue gain limits. */ + const ControlInfo &v4l2Gain = sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; + float minGain = camHelper_->gain(v4l2Gain.min().get()); + float maxGain = camHelper_->gain(v4l2Gain.max().get()); + float defGain = camHelper_->gain(v4l2Gain.def().get()); + ctrlMap[&controls::AnalogueGain] = ControlInfo(minGain, maxGain, defGain); + + /* + * Merge in any controls that we support either statically or from the + * algorithms. + */ + ctrlMap.merge(context_.ctrlMap); + + *ipaControls = ControlInfoMap(std::move(ctrlMap), controls::controls); +} + +int IPAMaliC55::configure(const IPAConfigInfo &ipaConfig, + [[maybe_unused]] uint8_t bayerOrder, + ControlInfoMap *ipaControls) +{ + sensorControls_ = ipaConfig.sensorControls; + + /* Clear the IPA context before the streaming session. */ + context_.configuration = {}; + context_.activeState = {}; + context_.frameContexts.clear(); + + const IPACameraSensorInfo &info = ipaConfig.sensorInfo; + + updateControls(info, ipaConfig.sensorControls, ipaControls); + + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + + int ret = algo->configure(context_, info); + if (ret) + return ret; + } + + return 0; +} + +void IPAMaliC55::mapBuffers(const std::vector &buffers, bool readOnly) +{ + for (const IPABuffer &buffer : buffers) { + const FrameBuffer fb(buffer.planes); + buffers_.emplace( + buffer.id, + MappedFrameBuffer( + &fb, + readOnly ? MappedFrameBuffer::MapFlag::Read + : MappedFrameBuffer::MapFlag::ReadWrite)); + } +} + +void IPAMaliC55::unmapBuffers(const std::vector &buffers) +{ + for (const IPABuffer &buffer : buffers) { + auto it = buffers_.find(buffer.id); + if (it == buffers_.end()) + continue; + + buffers_.erase(buffer.id); + } +} + +void IPAMaliC55::queueRequest(const uint32_t frame, const ControlList &controls) +{ + IPAFrameContext &frameContext = context_.frameContexts.alloc(frame); + + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + + algo->queueRequest(context_, frame, frameContext, controls); + } +} + +void IPAMaliC55::fillParams(unsigned int request, + [[maybe_unused]] uint32_t bufferId) +{ + struct mali_c55_params_buffer *params; + IPAFrameContext &frameContext = context_.frameContexts.get(request); + + params = reinterpret_cast( + buffers_.at(bufferId).planes()[0].data()); + memset(params, 0, sizeof(mali_c55_params_buffer)); + + params->version = MALI_C55_PARAM_BUFFER_V1; + + for (auto const &algo : algorithms()) { + algo->prepare(context_, request, frameContext, params); + + ASSERT(params->total_size <= MALI_C55_PARAMS_MAX_SIZE); + } + + paramsComputed.emit(request); +} + +void IPAMaliC55::processStats(unsigned int request, unsigned int bufferId, + const ControlList &sensorControls) +{ + IPAFrameContext &frameContext = context_.frameContexts.get(request); + const mali_c55_stats_buffer *stats = nullptr; + + stats = reinterpret_cast( + buffers_.at(bufferId).planes()[0].data()); + + frameContext.agc.exposure = + sensorControls.get(V4L2_CID_EXPOSURE).get(); + frameContext.agc.sensorGain = + camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + + ControlList metadata(controls::controls); + + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + + algo->process(context_, request, frameContext, stats, metadata); + } + + setControls(); + + statsProcessed.emit(request, metadata); +} + +} /* namespace ipa::mali_c55 */ + +/* + * External IPA module interface + */ +extern "C" { +const struct IPAModuleInfo ipaModuleInfo = { + IPA_MODULE_API_VERSION, + 1, + "mali-c55", + "mali-c55", +}; + +IPAInterface *ipaCreate() +{ + return new ipa::mali_c55::IPAMaliC55(); +} + +} /* extern "C" */ + +} /* namespace libcamera */ diff --git a/src/ipa/mali-c55/meson.build b/src/ipa/mali-c55/meson.build new file mode 100644 index 00000000..9e9b269c --- /dev/null +++ b/src/ipa/mali-c55/meson.build @@ -0,0 +1,33 @@ +# SPDX-License-Identifier: CC0-1.0 + +subdir('algorithms') +subdir('data') + +ipa_name = 'ipa_mali_c55' + +mali_c55_ipa_sources = files([ + 'ipa_context.cpp', + 'mali-c55.cpp' +]) + +mali_c55_ipa_sources += mali_c55_ipa_algorithms + +mod = shared_module(ipa_name, + [mali_c55_ipa_sources, libcamera_generated_ipa_headers], + name_prefix : '', + include_directories : [ipa_includes, libipa_includes], + dependencies : libcamera_private, + link_with : libipa, + install : true, + install_dir : ipa_install_dir) + +if ipa_sign_module + custom_target(ipa_name + '.so.sign', + input : mod, + output : ipa_name + '.so.sign', + command : [ipa_sign, ipa_priv_key, '@INPUT@', '@OUTPUT@'], + install : false, + build_by_default : true) +endif + +ipa_names += ipa_name diff --git a/src/ipa/mali-c55/module.h b/src/ipa/mali-c55/module.h new file mode 100644 index 00000000..1d85ec1f --- /dev/null +++ b/src/ipa/mali-c55/module.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas On Board + * + * module.h - Mali-C55 IPA Module + */ + +#pragma once + +#include + +#include + +#include + +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::mali_c55 { + +using Module = ipa::Module; + +} /* namespace ipa::mali_c55 */ + +} /* namespace libcamera*/ From patchwork Tue Jul 9 14:49:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20636 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DB2FBBD87C for ; Tue, 9 Jul 2024 14:50:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4767C63377; Tue, 9 Jul 2024 16:50:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Lyc0fbG0"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B44B263375 for ; Tue, 9 Jul 2024 16:49:56 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4CD9A2394; Tue, 9 Jul 2024 16:49:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720536564; bh=tZgXjm38kExnhxEfORqXhoKHn7iONYa2ubpNOE21TeI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Lyc0fbG0lSDjs+EFyTkKVB1D8hNgD+/LZVhpTykgvAnFG3zKzX611CQ4oMQenmKe1 oBdtPUTSmQxNB+jPomLZY4odbcoIJ8YbSTEgkt5oseh7Is9VVzyO0/R6+HHiJ/i1Wg OV+qzMv3+o+CFrB65cYnH9+IacoH8fdrxMYnlyjY= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Nayden Kanchev Subject: [PATCH v2 05/10] mali-c55: Plumb the IPA module in Date: Tue, 9 Jul 2024 15:49:45 +0100 Message-Id: <20240709144950.3277837-6-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709144950.3277837-1-dan.scally@ideasonboard.com> References: <20240709144950.3277837-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Plumb the Pipeline-IPA loop in. Load the IPA module at camera creation time and create the loop between the pipeline and the IPA. When a new Request is queued the IPA is asked to prepare the parameters buffer, once ready it notifies the pipeline which queues the parameters to the ISP along with a buffer for statistics and frames, Once statistics are ready they get passed to the IPA which upates its settings for the next frame. Acked-by: Nayden Kanchev Signed-off-by: Jacopo Mondi --- Changes in v2: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 403 +++++++++++++++++-- 1 file changed, 360 insertions(+), 43 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 1e5674fc..dd523d8d 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -23,13 +23,19 @@ #include #include +#include +#include + #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/request.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" @@ -70,6 +76,16 @@ constexpr Size kMaliC55MinSize = { 128, 128 }; constexpr Size kMaliC55MaxSize = { 8192, 8192 }; constexpr unsigned int kMaliC55ISPInternalFormat = MEDIA_BUS_FMT_RGB121212_1X36; +struct MaliC55FrameInfo { + Request *request; + + FrameBuffer *paramBuffer; + FrameBuffer *statBuffer; + + bool paramsDone; + bool statsDone; +}; + class MaliC55CameraData : public Camera::Private { public: @@ -79,6 +95,7 @@ public: } int init(); + int loadIPA(); /* Deflect these functionalities to either TPG or CameraSensor. */ const std::vector sizes(unsigned int mbusCode) const; @@ -87,7 +104,7 @@ public: int pixfmtToMbusCode(const PixelFormat &pixFmt) const; const PixelFormat &bestRawFormat() const; - void updateControls(); + void updateControls(const ControlInfoMap &ipaControls); PixelFormat adjustRawFormat(const PixelFormat &pixFmt) const; Size adjustRawSizes(const PixelFormat &pixFmt, const Size &rawSize) const; @@ -100,8 +117,15 @@ public: Stream frStream_; Stream dsStream_; + std::unique_ptr ipa_; + std::vector ipaStatBuffers_; + std::vector ipaParamBuffers_; + + std::unique_ptr delayedCtrls_; + private: void initTPGData(); + void setSensorControls(const ControlList &sensorControls); std::string id_; std::vector tpgCodes_; @@ -166,6 +190,11 @@ void MaliC55CameraData::initTPGData() tpgResolution_ = tpgSizes_.back(); } +void MaliC55CameraData::setSensorControls(const ControlList &sensorControls) +{ + delayedCtrls_->push(sensorControls); +} + const std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const { if (sensor_) @@ -268,7 +297,7 @@ const PixelFormat &MaliC55CameraData::bestRawFormat() const return invalidPixFmt; } -void MaliC55CameraData::updateControls() +void MaliC55CameraData::updateControls(const ControlInfoMap &ipaControls) { if (!sensor_) return; @@ -286,6 +315,9 @@ void MaliC55CameraData::updateControls() ControlInfo(ispMinCrop, sensorInfo.analogCrop, sensorInfo.analogCrop); + for (auto const &c : ipaControls) + controls.emplace(c.first, c.second); + controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); } @@ -339,6 +371,46 @@ Size MaliC55CameraData::adjustRawSizes(const PixelFormat &rawFmt, const Size &si return bestSize; } +int MaliC55CameraData::loadIPA() +{ + int ret; + + /* Do not initialize IPA for TPG. */ + if (!sensor_) + return 0; + + ipa_ = IPAManager::createIPA(pipe(), 1, 1); + if (!ipa_) + return -ENOENT; + + ipa_->setSensorControls.connect(this, &MaliC55CameraData::setSensorControls); + + std::string ipaTuningFile = ipa_->configurationFile(sensor_->model() + ".yaml"); + if (ipaTuningFile.empty()) + ipaTuningFile = ipa_->configurationFile("uncalibrated.yaml"); + + /* We need to inform the IPA of the sensor configuration */ + ipa::mali_c55::IPAConfigInfo ipaConfig{}; + + ret = sensor_->sensorInfo(&ipaConfig.sensorInfo); + if (ret) + return ret; + + ipaConfig.sensorControls = sensor_->controls(); + + ControlInfoMap ipaControls; + ret = ipa_->init({ ipaTuningFile, sensor_->model() }, ipaConfig, + &ipaControls); + if (ret) { + LOG(MaliC55, Error) << "Failed to initialise the Mali-C55 IPA"; + return ret; + } + + updateControls(ipaControls); + + return 0; +} + class MaliC55CameraConfiguration : public CameraConfiguration { public: @@ -348,6 +420,7 @@ public: } Status validate() override; + const Transform &combinedTransform() { return combinedTransform_; } V4L2SubdeviceFormat sensorFormat_; @@ -355,6 +428,7 @@ private: static constexpr unsigned int kMaxStreams = 2; const MaliC55CameraData *data_; + Transform combinedTransform_; }; CameraConfiguration::Status MaliC55CameraConfiguration::validate() @@ -364,6 +438,19 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() if (config_.empty()) return Invalid; + /* + * The TPG doesn't support flips, so we only need to calculate a + * transform if we have a sensor. + */ + if (data_->sensor_) { + Orientation requestedOrientation = orientation; + combinedTransform_ = data_->sensor_->computeTransform(&orientation); + if (orientation != requestedOrientation) + status = Adjusted; + } else { + combinedTransform_ = Transform::Rot0; + } + /* Only 2 streams available. */ if (config_.size() > kMaxStreams) { config_.resize(kMaxStreams); @@ -521,7 +608,10 @@ public: int queueRequestDevice(Camera *camera, Request *request) override; void bufferReady(FrameBuffer *buffer); + void paramsBufferReady(FrameBuffer *buffer); void statsBufferReady(FrameBuffer *buffer); + void paramsComputed(unsigned int requestId); + void statsProcessed(unsigned int requestId, const ControlList &metadata); bool match(DeviceEnumerator *enumerator) override; @@ -565,6 +655,10 @@ private: pipe.stream = nullptr; } + MaliC55FrameInfo *findFrameInfo(FrameBuffer *buffer); + MaliC55FrameInfo *findFrameInfo(Request *request); + void tryComplete(MaliC55FrameInfo *info); + int configureRawStream(MaliC55CameraData *data, const StreamConfiguration &config, V4L2SubdeviceFormat &subdevFormat); @@ -574,7 +668,7 @@ private: void applyScalerCrop(Camera *camera, const ControlList &controls); - void registerMaliCamera(std::unique_ptr data, + bool registerMaliCamera(std::unique_ptr data, const std::string &name); bool registerTPGCamera(MediaLink *link); bool registerSensorCamera(MediaLink *link); @@ -590,6 +684,8 @@ private: std::vector> paramsBuffers_; std::queue availableParamsBuffers_; + std::map frameInfoMap_; + std::array pipes_; bool dsFitted_; @@ -838,6 +934,13 @@ int PipelineHandlerMaliC55::configure(Camera *camera, if (ret) return ret; + if (data->sensor_) { + ret = data->sensor_->setFormat(&subdevFormat, + maliConfig->combinedTransform()); + if (ret) + return ret; + } + if (data->csi_) { ret = data->csi_->setFormat(0, &subdevFormat); if (ret) @@ -940,7 +1043,34 @@ int PipelineHandlerMaliC55::configure(Camera *camera, return ret; } - data->updateControls(); + if (!data->ipa_) + return 0; + + /* We need to inform the IPA of the sensor configuration */ + ipa::mali_c55::IPAConfigInfo ipaConfig{}; + + ret = data->sensor_->sensorInfo(&ipaConfig.sensorInfo); + if (ret) + return ret; + + ipaConfig.sensorControls = data->sensor_->controls(); + + /* + * And we also need to tell the IPA the bayerOrder of the data (as + * affected by any flips that we've configured) + */ + const Transform &combinedTransform = maliConfig->combinedTransform(); + BayerFormat::Order bayerOrder = data->sensor_->bayerOrder(combinedTransform); + + ControlInfoMap ipaControls; + ret = data->ipa_->configure(ipaConfig, utils::to_underlying(bayerOrder), + &ipaControls); + if (ret) { + LOG(MaliC55, Error) << "Failed to configure IPA"; + return ret; + } + + data->updateControls(ipaControls); return 0; } @@ -954,8 +1084,10 @@ int PipelineHandlerMaliC55::exportFrameBuffers(Camera *camera, Stream *stream, return pipe->cap->exportBuffers(count, buffers); } -void PipelineHandlerMaliC55::freeBuffers([[maybe_unused]] Camera *camera) +void PipelineHandlerMaliC55::freeBuffers(Camera *camera) { + MaliC55CameraData *data = cameraData(camera); + while (!availableStatsBuffers_.empty()) availableStatsBuffers_.pop(); while (!availableParamsBuffers_.empty()) @@ -964,11 +1096,18 @@ void PipelineHandlerMaliC55::freeBuffers([[maybe_unused]] Camera *camera) statsBuffers_.clear(); paramsBuffers_.clear(); + if (data->ipa_) { + data->ipa_->unmapBuffers(data->ipaStatBuffers_); + data->ipa_->unmapBuffers(data->ipaParamBuffers_); + } + data->ipaStatBuffers_.clear(); + data->ipaParamBuffers_.clear(); + if (stats_->releaseBuffers()) LOG(MaliC55, Error) << "Failed to release stats buffers"; if (params_->releaseBuffers()) - LOG(MaliC55, Error) << "Failed to release stats buffers"; + LOG(MaliC55, Error) << "Failed to release params buffers"; return; } @@ -976,6 +1115,7 @@ void PipelineHandlerMaliC55::freeBuffers([[maybe_unused]] Camera *camera) int PipelineHandlerMaliC55::allocateBuffers(Camera *camera) { MaliC55CameraData *data = cameraData(camera); + unsigned int ipaBufferId = 1; unsigned int bufferCount; int ret; @@ -988,27 +1128,51 @@ int PipelineHandlerMaliC55::allocateBuffers(Camera *camera) if (ret < 0) return ret; - for (std::unique_ptr &buffer : statsBuffers_) + for (std::unique_ptr &buffer : statsBuffers_) { + buffer->setCookie(ipaBufferId++); + data->ipaStatBuffers_.emplace_back(buffer->cookie(), + buffer->planes()); availableStatsBuffers_.push(buffer.get()); + } ret = params_->allocateBuffers(bufferCount, ¶msBuffers_); if (ret < 0) return ret; - for (std::unique_ptr &buffer : paramsBuffers_) + for (std::unique_ptr &buffer : paramsBuffers_) { + buffer->setCookie(ipaBufferId++); + data->ipaParamBuffers_.emplace_back(buffer->cookie(), + buffer->planes()); availableParamsBuffers_.push(buffer.get()); + } + + if (data->ipa_) { + data->ipa_->mapBuffers(data->ipaStatBuffers_, true); + data->ipa_->mapBuffers(data->ipaParamBuffers_, false); + } return 0; } int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const ControlList *controls) { + MaliC55CameraData *data = cameraData(camera); int ret; ret = allocateBuffers(camera); if (ret) return ret; + if (data->ipa_) { + ret = data->ipa_->start(); + if (ret) { + LOG(MaliC55, Error) + << "Failed to start IPA" << camera->id(); + freeBuffers(camera); + return ret; + } + } + for (MaliC55Pipe &pipe : pipes_) { if (!pipe.stream) continue; @@ -1018,6 +1182,8 @@ int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const Control ret = pipe.cap->importBuffers(stream->configuration().bufferCount); if (ret) { LOG(MaliC55, Error) << "Failed to import buffers"; + if (data->ipa_) + data->ipa_->stop(); freeBuffers(camera); return ret; } @@ -1025,6 +1191,8 @@ int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const Control ret = pipe.cap->streamOn(); if (ret) { LOG(MaliC55, Error) << "Failed to start stream"; + if (data->ipa_) + data->ipa_->stop(); freeBuffers(camera); return ret; } @@ -1034,6 +1202,9 @@ int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const Control if (ret) { LOG(MaliC55, Error) << "Failed to start stats stream"; + if (data->ipa_) + data->ipa_->stop(); + for (MaliC55Pipe &pipe : pipes_) { if (pipe.stream) pipe.cap->streamOff(); @@ -1048,6 +1219,8 @@ int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const Control LOG(MaliC55, Error) << "Failed to start params stream"; stats_->streamOff(); + if (data->ipa_) + data->ipa_->stop(); for (MaliC55Pipe &pipe : pipes_) { if (pipe.stream) @@ -1058,11 +1231,19 @@ int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const Control return ret; } + ret = isp_->setFrameStartEnabled(true); + if (ret) + LOG(MaliC55, Error) << "Failed to enable frame start events"; + return 0; } -void PipelineHandlerMaliC55::stopDevice([[maybe_unused]] Camera *camera) +void PipelineHandlerMaliC55::stopDevice(Camera *camera) { + MaliC55CameraData *data = cameraData(camera); + + isp_->setFrameStartEnabled(false); + for (MaliC55Pipe &pipe : pipes_) { if (!pipe.stream) continue; @@ -1073,6 +1254,8 @@ void PipelineHandlerMaliC55::stopDevice([[maybe_unused]] Camera *camera) stats_->streamOff(); params_->streamOff(); + if (data->ipa_) + data->ipa_->stop(); freeBuffers(camera); } @@ -1174,64 +1357,179 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) { - FrameBuffer *statsBuffer; - int ret; + MaliC55CameraData *data = cameraData(camera); + + /* Do not run the IPA if the TPG is in use. */ + if (!data->ipa_) { + MaliC55FrameInfo frameInfo; + frameInfo.request = request; + frameInfo.statBuffer = nullptr; + frameInfo.paramBuffer = nullptr; + frameInfo.paramsDone = true; + frameInfo.statsDone = true; + + frameInfoMap_[request->sequence()] = frameInfo; + + for (auto &[stream, buffer] : request->buffers()) { + MaliC55Pipe *pipe = pipeFromStream(data, stream); + + pipe->cap->queueBuffer(buffer); + } + + return 0; + } if (availableStatsBuffers_.empty()) { LOG(MaliC55, Error) << "Stats buffer underrun"; return -ENOENT; } - statsBuffer = availableStatsBuffers_.front(); + if (availableParamsBuffers_.empty()) { + LOG(MaliC55, Error) << "Params buffer underrun"; + return -ENOENT; + } + + MaliC55FrameInfo frameInfo; + frameInfo.request = request; + + frameInfo.statBuffer = availableStatsBuffers_.front(); availableStatsBuffers_.pop(); + frameInfo.paramBuffer = availableParamsBuffers_.front(); + availableParamsBuffers_.pop(); - /* - * We need to associate the Request to this buffer even though it's a - * purely internal one because we will need to use request->sequence() - * later. - */ - statsBuffer->_d()->setRequest(request); + frameInfo.paramsDone = false; + frameInfo.statsDone = false; - for (auto &[stream, buffer] : request->buffers()) { - MaliC55Pipe *pipe = pipeFromStream(cameraData(camera), stream); + frameInfoMap_[request->sequence()] = frameInfo; - ret = pipe->cap->queueBuffer(buffer); - if (ret) - return ret; + data->ipa_->queueRequest(request->sequence(), request->controls()); + data->ipa_->fillParams(request->sequence(), + frameInfo.paramBuffer->cookie()); + + return 0; +} + +MaliC55FrameInfo *PipelineHandlerMaliC55::findFrameInfo(Request *request) +{ + for (auto &[sequence, info] : frameInfoMap_) { + if (info.request == request) + return &info; } - /* - * Some controls need to be applied immediately, as in example, - * the ScalerCrop one. - * - * \todo Move it buffer queue time (likely after the IPA has filled in - * the parameters buffer) once we have plumbed the IPA loop in. - */ - applyScalerCrop(camera, request->controls()); + return nullptr; +} - ret = stats_->queueBuffer(statsBuffer); - if (ret) - return ret; +MaliC55FrameInfo *PipelineHandlerMaliC55::findFrameInfo(FrameBuffer *buffer) +{ + for (auto &[sequence, info] : frameInfoMap_) { + if (info.paramBuffer == buffer || + info.statBuffer == buffer) + return &info; + } - return 0; + return nullptr; +} + +void PipelineHandlerMaliC55::tryComplete(MaliC55FrameInfo *info) +{ + if (!info->paramsDone) + return; + if (!info->statsDone) + return; + + Request *request = info->request; + if (request->hasPendingBuffers()) + return; + + if (info->statBuffer) + availableStatsBuffers_.push(info->statBuffer); + if (info->paramBuffer) + availableParamsBuffers_.push(info->paramBuffer); + + frameInfoMap_.erase(request->sequence()); + + completeRequest(request); } void PipelineHandlerMaliC55::bufferReady(FrameBuffer *buffer) { Request *request = buffer->request(); + MaliC55FrameInfo *info = findFrameInfo(request); + ASSERT(info); if (completeBuffer(request, buffer)) - completeRequest(request); + tryComplete(info); +} + +void PipelineHandlerMaliC55::paramsBufferReady(FrameBuffer *buffer) +{ + MaliC55FrameInfo *info = findFrameInfo(buffer); + ASSERT(info); + + info->paramsDone = true; + + tryComplete(info); } void PipelineHandlerMaliC55::statsBufferReady(FrameBuffer *buffer) { - availableStatsBuffers_.push(buffer); + MaliC55FrameInfo *info = findFrameInfo(buffer); + ASSERT(info); + + Request *request = info->request; + MaliC55CameraData *data = cameraData(request->_d()->camera()); + + ControlList sensorControls = data->delayedCtrls_->get(buffer->metadata().sequence); + + data->ipa_->processStats(request->sequence(), buffer->cookie(), + sensorControls); } -void PipelineHandlerMaliC55::registerMaliCamera(std::unique_ptr data, +void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId) +{ + MaliC55FrameInfo &frameInfo = frameInfoMap_[requestId]; + Request *request = frameInfo.request; + MaliC55CameraData *data = cameraData(request->_d()->camera()); + + /* + * Queue buffers for stats and params, then queue buffers to the capture + * video devices. + */ + + frameInfo.paramBuffer->_d()->metadata().planes()[0].bytesused = + sizeof(struct mali_c55_params_buffer); + params_->queueBuffer(frameInfo.paramBuffer); + stats_->queueBuffer(frameInfo.statBuffer); + + for (auto &[stream, buffer] : request->buffers()) { + MaliC55Pipe *pipe = pipeFromStream(data, stream); + + pipe->cap->queueBuffer(buffer); + } +} + +void PipelineHandlerMaliC55::statsProcessed(unsigned int requestId, + const ControlList &metadata) +{ + MaliC55FrameInfo &frameInfo = frameInfoMap_[requestId]; + + frameInfo.statsDone = true; + frameInfo.request->metadata().merge(metadata); + + tryComplete(&frameInfo); +} + +bool PipelineHandlerMaliC55::registerMaliCamera(std::unique_ptr data, const std::string &name) { + if (data->loadIPA()) + return false; + + if (data->ipa_) { + data->ipa_->statsProcessed.connect(this, &PipelineHandlerMaliC55::statsProcessed); + data->ipa_->paramsComputed.connect(this, &PipelineHandlerMaliC55::paramsComputed); + } + std::set streams{ &data->frStream_ }; if (dsFitted_) streams.insert(&data->dsStream_); @@ -1239,6 +1537,8 @@ void PipelineHandlerMaliC55::registerMaliCamera(std::unique_ptr camera = Camera::create(std::move(data), name, streams); registerCamera(std::move(camera)); + + return true; } /* @@ -1264,9 +1564,7 @@ bool PipelineHandlerMaliC55::registerTPGCamera(MediaLink *link) if (data->init()) return false; - registerMaliCamera(std::move(data), name); - - return true; + return registerMaliCamera(std::move(data), name); } /* @@ -1293,9 +1591,27 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) return false; data->properties_ = data->sensor_->properties(); - data->updateControls(); - registerMaliCamera(std::move(data), sensor->name()); + /* + * \todo Read delay values from the sensor itself or from a + * a sensor database. For now use generic values taken from + * the Raspberry Pi and listed as 'generic values'. + */ + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { 1, false } }, + { V4L2_CID_EXPOSURE, { 2, false } }, + }; + + data->delayedCtrls_ = + std::make_unique(data->sensor_->device(), + params); + isp_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + + /* \todo: Init properties. */ + + if (!registerMaliCamera(std::move(data), sensor->name())) + return false; } return true; @@ -1362,6 +1678,7 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) } stats_->bufferReady.connect(this, &PipelineHandlerMaliC55::statsBufferReady); + params_->bufferReady.connect(this, &PipelineHandlerMaliC55::paramsBufferReady); ispSink = isp_->entity()->getPadByIndex(0); if (!ispSink || ispSink->links().empty()) { From patchwork Tue Jul 9 14:49:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20637 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8F1A4C328C for ; Tue, 9 Jul 2024 14:50:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2AB5563387; Tue, 9 Jul 2024 16:50:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="QzURKxgy"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 08BE363377 for ; Tue, 9 Jul 2024 16:49:57 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 94EBE2395; Tue, 9 Jul 2024 16:49:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720536564; bh=1Woc6L1aMTKznMg98pJn+b77rCkkD4KH5Ty+CT3Logw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QzURKxgyGM3vpkpw07/KbCq0aKrl157xDRsErvW5BxB12iZGVjRQlR3N8B6M9PRTR M9IDpwv8ZMtyd/ogYmegBMsHKakxf7uuzakl4Vd1OGLHOdPjS5kanhzq71RhX4bK5B ySPraB17fjHoOR3N2R1MsBwcOc5D7KO1PeGjy/hU= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Nayden Kanchev Subject: [PATCH v2 06/10] ipa: mali-c55: Add BLC Algorithm Date: Tue, 9 Jul 2024 15:49:46 +0100 Message-Id: <20240709144950.3277837-7-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709144950.3277837-1-dan.scally@ideasonboard.com> References: <20240709144950.3277837-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add a Black Level Correction algorithm. Acked-by: Nayden Kanchev Jacopo Mondi Reviewed-by: Kieran Bingham --- Changes in v2: - Use the union rather than reinterpret_cast<>() to abstract the block src/ipa/mali-c55/algorithms/blc.cpp | 121 ++++++++++++++++++++++++ src/ipa/mali-c55/algorithms/blc.h | 40 ++++++++ src/ipa/mali-c55/algorithms/meson.build | 1 + 3 files changed, 162 insertions(+) create mode 100644 src/ipa/mali-c55/algorithms/blc.cpp create mode 100644 src/ipa/mali-c55/algorithms/blc.h diff --git a/src/ipa/mali-c55/algorithms/blc.cpp b/src/ipa/mali-c55/algorithms/blc.cpp new file mode 100644 index 00000000..aebb9dc4 --- /dev/null +++ b/src/ipa/mali-c55/algorithms/blc.cpp @@ -0,0 +1,121 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2022, Ideas On Board + * + * blc.cpp - Mali-C55 sensor offset (black level) correction + */ + +#include "blc.h" + +#include +#include + +#include "libcamera/internal/yaml_parser.h" + +/** + * \file blc.h + */ + +namespace libcamera { + +namespace ipa::mali_c55::algorithms { + +/** + * \class BlackLevelCorrection + * \brief MaliC55 Black Level Correction control + * + * todo + */ + +LOG_DEFINE_CATEGORY(MaliC55Blc) + +BlackLevelCorrection::BlackLevelCorrection() + : tuningParameters_(false) +{ +} + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int BlackLevelCorrection::init([[maybe_unused]] IPAContext &context, + const YamlObject &tuningData) +{ + offset00 = tuningData["offset00"].get(256); + offset01 = tuningData["offset01"].get(256); + offset10 = tuningData["offset10"].get(256); + offset11 = tuningData["offset11"].get(256); + + if (offset00 > kMaxOffset || offset01 > kMaxOffset || + offset10 > kMaxOffset || offset11 > kMaxOffset) { + LOG(MaliC55Blc, Error) << "Invalid black level offsets"; + return -EINVAL; + } + + tuningParameters_ = true; + + LOG(MaliC55Blc, Debug) + << "Black levels: 00 " << offset00 << ", 01 " << offset01 + << ", 10 " << offset10 << ", 11 " << offset11; + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context, + const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + mali_c55_params_buffer *params) +{ + mali_c55_params_block block; + block.data = ¶ms->data[params->total_size]; + + if (frame > 0) + return; + + if (!tuningParameters_) + return; + + block.header->type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS; + block.header->enabled = true; + block.header->size = sizeof(mali_c55_params_sensor_off_preshading); + + block.sensor_offs->chan00 = offset00; + block.sensor_offs->chan01 = offset01; + block.sensor_offs->chan10 = offset10; + block.sensor_offs->chan11 = offset11; + + params->total_size += block.header->size; +} + +void BlackLevelCorrection::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] const mali_c55_stats_buffer *stats, + ControlList &metadata) +{ + /* + * Black Level Offsets in tuning data need to be 20-bit, whereas the + * metadata expects values from a 16-bit range. Right-shift to remove + * the 4 least significant bits. + * + * The black levels should be reported in the order R, Gr, Gb, B. We + * ignore that here given we're using matching values so far, but it + * would be safer to check the sensor's bayer order. + * + * \todo Account for bayer order. + */ + metadata.set(controls::SensorBlackLevels, { + static_cast(offset00 >> 4), + static_cast(offset01 >> 4), + static_cast(offset10 >> 4), + static_cast(offset11 >> 4), + }); +} + +REGISTER_IPA_ALGORITHM(BlackLevelCorrection, "BlackLevelCorrection") + +} /* namespace ipa::mali_c55::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/mali-c55/algorithms/blc.h b/src/ipa/mali-c55/algorithms/blc.h new file mode 100644 index 00000000..e73056e0 --- /dev/null +++ b/src/ipa/mali-c55/algorithms/blc.h @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2022, Ideas On Board + * + * blc.h - Mali-C55 sensor offset (black level) correction + */ + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::mali_c55::algorithms { + +class BlackLevelCorrection : public Algorithm +{ +public: + BlackLevelCorrection(); + ~BlackLevelCorrection() = default; + + int init(IPAContext &context, const YamlObject &tuningData) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + mali_c55_params_buffer *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const mali_c55_stats_buffer *stats, + ControlList &metadata) override; + +private: + static constexpr uint32_t kMaxOffset = 0xfffff; + + bool tuningParameters_; + uint32_t offset00; + uint32_t offset01; + uint32_t offset10; + uint32_t offset11; +}; + +} /* namespace ipa::mali_c55::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/mali-c55/algorithms/meson.build b/src/ipa/mali-c55/algorithms/meson.build index f2203b15..d84432b9 100644 --- a/src/ipa/mali-c55/algorithms/meson.build +++ b/src/ipa/mali-c55/algorithms/meson.build @@ -1,4 +1,5 @@ # SPDX-License-Identifier: CC0-1.0 mali_c55_ipa_algorithms = files([ + 'blc.cpp', ]) From patchwork Tue Jul 9 14:49:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20638 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E91ACBD87C for ; Tue, 9 Jul 2024 14:50:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1DCC663374; Tue, 9 Jul 2024 16:50:13 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="KQohLwi1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5C1296337A for ; Tue, 9 Jul 2024 16:49:57 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DCE5516C3; Tue, 9 Jul 2024 16:49:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720536565; bh=xfu0Pcyn2lPtAxyOI3x+w5hn1ZycRDUZwCX2xVEWjWk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KQohLwi1BKlvi0SOVG/U6zW2bz8f1dSoek0TW0tE6ulJY/093uC6ZkctGcSnlPCOX ZXBCIZ3DT8VHJTsmHtl2XAz4QMKBmK8zD1pQZTEGYcu1O+QTSfymIZVwmih21RIh3+ 0HBNiFDpzueWPmcGxWrgIUWlMR5Mqz+d5mrtRhGk= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Nayden Kanchev , Jacopo Mondi Subject: [PATCH v2 07/10] ipa: mali-c55: Add Agc algorithm Date: Tue, 9 Jul 2024 15:49:47 +0100 Message-Id: <20240709144950.3277837-8-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709144950.3277837-1-dan.scally@ideasonboard.com> References: <20240709144950.3277837-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new algorithm and associated infrastructure for Agc. The tuning files for uncalibrated sensors is extended to enable the algorithm. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v2: - Use the union rather than reinterpret_cast<>() to abstract the block src/ipa/mali-c55/algorithms/agc.cpp | 440 ++++++++++++++++++++++++ src/ipa/mali-c55/algorithms/agc.h | 82 +++++ src/ipa/mali-c55/algorithms/meson.build | 1 + src/ipa/mali-c55/data/uncalibrated.yaml | 1 + src/ipa/mali-c55/ipa_context.h | 32 ++ src/ipa/mali-c55/mali-c55.cpp | 54 ++- 6 files changed, 608 insertions(+), 2 deletions(-) create mode 100644 src/ipa/mali-c55/algorithms/agc.cpp create mode 100644 src/ipa/mali-c55/algorithms/agc.h diff --git a/src/ipa/mali-c55/algorithms/agc.cpp b/src/ipa/mali-c55/algorithms/agc.cpp new file mode 100644 index 00000000..bbd3118c --- /dev/null +++ b/src/ipa/mali-c55/algorithms/agc.cpp @@ -0,0 +1,440 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas On Board Oy + * + * agc.cpp - AGC/AEC mean-based control algorithm + */ + +#include "agc.h" + +#include + +#include + +#include +#include + +namespace libcamera { + +using namespace std::literals::chrono_literals; + +namespace ipa::mali_c55::algorithms { + +LOG_DEFINE_CATEGORY(MaliC55Agc) + +/* + * Number of histogram bins. This is only true for the specific configuration we + * set to the ISP; 4 separate histograms of 256 bins each. If that configuration + * ever changes then this constant will need updating. + */ +static constexpr unsigned int kNumHistogramBins = 256; + +/* + * The Mali-C55 ISP has a digital gain block which allows setting gain in Q5.8 + * format, a range of 0.0 to (very nearly) 32.0. We clamp from 1.0 to the actual + * max value. + */ +static constexpr double kMinDigitalGain = 1.0; +static constexpr double kMaxDigitalGain = 8191 * pow(2, -8); + +uint32_t AgcStatistics::decodeBinValue(uint16_t binVal) +{ + int exponent = (binVal & 0xf000) >> 12; + int mantissa = binVal & 0xfff; + + if (!exponent) + return mantissa * 2; + else + return (mantissa + 4096) * pow(2, exponent); +} + +/* + * We configure the ISP to give us 4 histograms of 256 bins each, with + * a single histogram per colour channel (R/Gr/Gb/B). The memory space + * containing the data is a single block containing all 4 histograms + * with the position of each colour's histogram within it dependent on + * the bayer pattern of the data input to the ISP. + * + * NOTE: The validity of this function depends on the parameters we have + * configured. With different skip/offset x, y values not all of the + * colour channels would be populated, and they may not be in the same + * planes as calculated here. + */ +int AgcStatistics::setBayerOrderIndices(BayerFormat::Order bayerOrder) +{ + switch (bayerOrder) { + case BayerFormat::Order::RGGB: + rIndex_ = 0; + grIndex_ = 1; + gbIndex_ = 2; + bIndex_ = 3; + break; + case BayerFormat::Order::GRBG: + grIndex_ = 0; + rIndex_ = 1; + bIndex_ = 2; + gbIndex_ = 3; + break; + case BayerFormat::Order::GBRG: + gbIndex_ = 0; + bIndex_ = 1; + rIndex_ = 2; + grIndex_ = 3; + break; + case BayerFormat::Order::BGGR: + bIndex_ = 0; + gbIndex_ = 1; + grIndex_ = 2; + rIndex_ = 3; + break; + default: + LOG(MaliC55Agc, Error) + << "Invalid bayer format " << bayerOrder; + return -EINVAL; + } + + return 0; +} + +void AgcStatistics::parseStatistics(const mali_c55_stats_buffer *stats) +{ + uint32_t r[256], g[256], b[256], y[256]; + + /* + * We need to decode the bin values for each histogram from their 16-bit + * compressed values to a 32-bit value. We also take the average of the + * Gr/Gb values into a single green histogram. + */ + for (unsigned int i = 0; i < 256; i++) { + r[i] = decodeBinValue(stats->ae_1024bin_hist.bins[i + (256 * rIndex_)]); + g[i] = (decodeBinValue(stats->ae_1024bin_hist.bins[i + (256 * grIndex_)]) + + decodeBinValue(stats->ae_1024bin_hist.bins[i + (256 * gbIndex_)])) / 2; + b[i] = decodeBinValue(stats->ae_1024bin_hist.bins[i + (256 * bIndex_)]); + + y[i] = (r[i] * .299) + (g[i] * .587) + (b[i] * .114); + } + + rHist = Histogram(Span(r, kNumHistogramBins)); + gHist = Histogram(Span(g, kNumHistogramBins)); + bHist = Histogram(Span(b, kNumHistogramBins)); + yHist = Histogram(Span(y, kNumHistogramBins)); +} + +Agc::Agc() + : AgcMeanLuminance() +{ +} + +int Agc::init(IPAContext &context, const YamlObject &tuningData) +{ + int ret = parseTuningData(tuningData); + if (ret) + return ret; + + context.ctrlMap[&controls::AeEnable] = ControlInfo(false, true); + context.ctrlMap[&controls::DigitalGain] = ControlInfo( + static_cast(kMinDigitalGain), + static_cast(kMaxDigitalGain), + static_cast(kMinDigitalGain) + ); + context.ctrlMap.merge(controls()); + + return 0; +} + +int Agc::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + int ret = statistics_.setBayerOrderIndices(context.configuration.sensor.bayerOrder); + if (ret) + return ret; + + /* + * Defaults; we use whatever the sensor's default exposure is and the + * minimum analogue gain. AEGC is _active_ by default. + */ + context.activeState.agc.autoEnabled = true; + context.activeState.agc.automatic.sensorGain = context.configuration.agc.minAnalogueGain; + context.activeState.agc.automatic.exposure = context.configuration.agc.defaultExposure; + context.activeState.agc.automatic.ispGain = kMinDigitalGain; + context.activeState.agc.manual.sensorGain = context.configuration.agc.minAnalogueGain; + context.activeState.agc.manual.exposure = context.configuration.agc.defaultExposure; + context.activeState.agc.manual.ispGain = kMinDigitalGain; + context.activeState.agc.constraintMode = constraintModes().begin()->first; + context.activeState.agc.exposureMode = exposureModeHelpers().begin()->first; + + /* \todo Run this again when FrameDurationLimits is passed in */ + setLimits(context.configuration.agc.minShutterSpeed, + context.configuration.agc.maxShutterSpeed, + context.configuration.agc.minAnalogueGain, + context.configuration.agc.maxAnalogueGain); + + resetFrameCount(); + + return 0; +} + +void Agc::queueRequest(IPAContext &context, const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + const ControlList &controls) +{ + auto &agc = context.activeState.agc; + + const auto &constraintMode = controls.get(controls::AeConstraintMode); + agc.constraintMode = constraintMode.value_or(agc.constraintMode); + + const auto &exposureMode = controls.get(controls::AeExposureMode); + agc.exposureMode = exposureMode.value_or(agc.exposureMode); + + const auto &agcEnable = controls.get(controls::AeEnable); + if (agcEnable && *agcEnable != agc.autoEnabled) { + agc.autoEnabled = *agcEnable; + + LOG(MaliC55Agc, Info) + << (agc.autoEnabled ? "Enabling" : "Disabling") + << " AGC"; + } + + /* + * If the automatic exposure and gain is enabled we have no further work + * to do here... + */ + if (agc.autoEnabled) + return; + + /* + * ...otherwise we need to look for exposure and gain controls and use + * those to set the activeState. + */ + const auto &exposure = controls.get(controls::ExposureTime); + if (exposure) { + agc.manual.exposure = *exposure * 1.0us / context.configuration.sensor.lineDuration; + + LOG(MaliC55Agc, Debug) + << "Exposure set to " << agc.manual.exposure + << " on request sequence " << frame; + } + + const auto &analogueGain = controls.get(controls::AnalogueGain); + if (analogueGain) { + agc.manual.sensorGain = *analogueGain; + + LOG(MaliC55Agc, Debug) + << "Analogue gain set to " << agc.manual.sensorGain + << " on request sequence " << frame; + } + + const auto &digitalGain = controls.get(controls::DigitalGain); + if (digitalGain) { + agc.manual.ispGain = *digitalGain; + + LOG(MaliC55Agc, Debug) + << "Digital gain set to " << agc.manual.ispGain + << " on request sequence " << frame; + } +} + +size_t Agc::fillGainParamBlock(IPAContext &context, IPAFrameContext &frameContext, + mali_c55_params_block block) +{ + IPAActiveState &activeState = context.activeState; + double gain; + + if (activeState.agc.autoEnabled) + gain = activeState.agc.automatic.ispGain; + else + gain = activeState.agc.manual.ispGain; + + block.header->type = MALI_C55_PARAM_BLOCK_DIGITAL_GAIN; + block.header->enabled = true; + block.header->size = sizeof(struct mali_c55_params_digital_gain); + + block.digital_gain->gain = int(gain * pow(2, 8)); + frameContext.agc.ispGain = gain; + + return block.header->size; +} + +size_t Agc::fillParamsBuffer(mali_c55_params_block block, + enum mali_c55_param_block_type type) +{ + block.header->type = type; + block.header->enabled = true; + block.header->size = sizeof(struct mali_c55_params_aexp_hist); + + /* Collect every 3rd pixel horizontally */ + block.aexp_hist->skip_x = 1; + /* Start from first column */ + block.aexp_hist->offset_x = 0; + /* Collect every pixel vertically */ + block.aexp_hist->skip_y = 0; + /* Start from the first row */ + block.aexp_hist->offset_y = 0; + /* 1x scaling (i.e. none) */ + block.aexp_hist->scale_bottom = 0; + block.aexp_hist->scale_top = 0; + /* Collect all Bayer planes into 4 separate histograms */ + block.aexp_hist->plane_mode = 1; + /* Tap the data immediately after the digital gain block */ + block.aexp_hist->tap_point = MALI_C55_AEXP_HIST_TAP_FS; + + return block.header->size; +} + +size_t Agc::fillWeightsArrayBuffer(mali_c55_params_block block, + enum mali_c55_param_block_type type) +{ + block.header->type = type; + block.header->enabled = true; + block.header->size = sizeof(struct mali_c55_params_aexp_weights); + + /* We use every zone - a 15x15 grid */ + block.aexp_weights->nodes_used_horiz = 15; + block.aexp_weights->nodes_used_vert = 15; + + /* + * We uniformly weight the zones to 1 - this results in the collected + * histograms containing a true pixel count, which we can then use to + * approximate colour channel averages for the image. + */ + Span weights{ + block.aexp_weights->zone_weights, + MALI_C55_MAX_ZONES + }; + std::fill(weights.begin(), weights.end(), 1); + + return block.header->size; +} + +void Agc::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, mali_c55_params_buffer *params) +{ + mali_c55_params_block block; + + block.data = ¶ms->data[params->total_size]; + params->total_size += fillGainParamBlock(context, frameContext, block); + + if (frame > 0) + return; + + block.data = ¶ms->data[params->total_size]; + params->total_size += fillParamsBuffer(block, + MALI_C55_PARAM_BLOCK_AEXP_HIST); + + block.data = ¶ms->data[params->total_size]; + params->total_size += fillWeightsArrayBuffer(block, + MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS); + + block.data = ¶ms->data[params->total_size]; + params->total_size += fillParamsBuffer(block, + MALI_C55_PARAM_BLOCK_AEXP_IHIST); + + block.data = ¶ms->data[params->total_size]; + params->total_size += fillWeightsArrayBuffer(block, + MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS); +} + +double Agc::estimateLuminance(const double gain) const +{ + double rAvg = statistics_.rHist.interQuantileMean(0, 1) * gain; + double gAvg = statistics_.gHist.interQuantileMean(0, 1) * gain; + double bAvg = statistics_.bHist.interQuantileMean(0, 1) * gain; + double yAvg = (rAvg * .299) + (gAvg * .587) + (bAvg * .114); + + return yAvg / kNumHistogramBins; +} + +/** + * The function estimates the correlated color temperature using + * from RGB color space input. + * In physics and color science, the Planckian locus or black body locus is + * the path or locus that the color of an incandescent black body would take + * in a particular chromaticity space as the blackbody temperature changes. + * + * If a narrow range of color temperatures is considered (those encapsulating + * daylight being the most practical case) one can approximate the Planckian + * locus in order to calculate the CCT in terms of chromaticity coordinates. + * + * More detailed information can be found in: + * https://en.wikipedia.org/wiki/Color_temperature#Approximation + */ +uint32_t Agc::estimateCCT() const +{ + double red = statistics_.rHist.interQuantileMean(0, 1); + double green = statistics_.gHist.interQuantileMean(0, 1); + double blue = statistics_.bHist.interQuantileMean(0, 1); + + /* Convert the RGB values to CIE tristimulus values (XYZ) */ + double X = (-0.14282) * (red) + (1.54924) * (green) + (-0.95641) * (blue); + double Y = (-0.32466) * (red) + (1.57837) * (green) + (-0.73191) * (blue); + double Z = (-0.68202) * (red) + (0.77073) * (green) + (0.56332) * (blue); + + /* Calculate the normalized chromaticity values */ + double x = X / (X + Y + Z); + double y = Y / (X + Y + Z); + + /* Calculate CCT */ + double n = (x - 0.3320) / (0.1858 - y); + uint32_t ct = 449 * n * n * n + 3525 * n * n + 6823.3 * n + 5520.33; + + LOG(MaliC55Agc, Debug) << "Estimated Colour Temperature: " << ct; + + return ct; +} + +void Agc::process(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const mali_c55_stats_buffer *stats, + [[maybe_unused]] ControlList &metadata) +{ + IPASessionConfiguration &configuration = context.configuration; + IPAActiveState &activeState = context.activeState; + + if (!stats) { + LOG(MaliC55Agc, Error) << "No statistics buffer passed to Agc"; + return; + } + + statistics_.parseStatistics(stats); + context.activeState.agc.temperatureK = estimateCCT(); + + /* + * The Agc algorithm needs to know the effective exposure value that was + * applied to the sensor when the statistics were collected. + */ + uint32_t exposure = frameContext.agc.exposure; + double analogueGain = frameContext.agc.sensorGain; + double digitalGain = frameContext.agc.ispGain; + double totalGain = analogueGain * digitalGain; + utils::Duration currentShutter = exposure * configuration.sensor.lineDuration; + utils::Duration effectiveExposureValue = currentShutter * totalGain; + + utils::Duration shutterTime; + double aGain, dGain; + std::tie(shutterTime, aGain, dGain) = + calculateNewEv(activeState.agc.constraintMode, + activeState.agc.exposureMode, statistics_.yHist, + effectiveExposureValue); + + dGain = std::clamp(dGain, kMinDigitalGain, kMaxDigitalGain); + + LOG(MaliC55Agc, Debug) + << "Divided up shutter, analogue gain and digital gain are " + << shutterTime << ", " << aGain << " and " << dGain; + + activeState.agc.automatic.exposure = shutterTime / configuration.sensor.lineDuration; + activeState.agc.automatic.sensorGain = aGain; + activeState.agc.automatic.ispGain = dGain; + + metadata.set(controls::ExposureTime, currentShutter.get()); + metadata.set(controls::AnalogueGain, frameContext.agc.sensorGain); + metadata.set(controls::DigitalGain, frameContext.agc.ispGain); + metadata.set(controls::ColourTemperature, context.activeState.agc.temperatureK); +} + +REGISTER_IPA_ALGORITHM(Agc, "Agc") + +} /* namespace ipa::mali_c55::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/mali-c55/algorithms/agc.h b/src/ipa/mali-c55/algorithms/agc.h new file mode 100644 index 00000000..91e257a3 --- /dev/null +++ b/src/ipa/mali-c55/algorithms/agc.h @@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Ideas on Board Oy + * + * agc.h - Mali C55 AGC/AEC mean-based control algorithm + */ + +#pragma once + +#include + +#include "libcamera/internal/bayer_format.h" + +#include "libipa/agc_mean_luminance.h" +#include "libipa/histogram.h" + +#include "algorithm.h" +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::mali_c55::algorithms { + +class AgcStatistics +{ +public: + AgcStatistics() + { + } + + int setBayerOrderIndices(BayerFormat::Order bayerOrder); + uint32_t decodeBinValue(uint16_t binVal); + void parseStatistics(const mali_c55_stats_buffer *stats); + + Histogram rHist; + Histogram gHist; + Histogram bHist; + Histogram yHist; +private: + unsigned int rIndex_; + unsigned int grIndex_; + unsigned int gbIndex_; + unsigned int bIndex_; +}; + +class Agc : public Algorithm, public AgcMeanLuminance +{ +public: + Agc(); + ~Agc() = default; + + int init(IPAContext &context, const YamlObject &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + mali_c55_params_buffer *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const mali_c55_stats_buffer *stats, + ControlList &metadata) override; + +private: + double estimateLuminance(const double gain) const override; + size_t fillGainParamBlock(IPAContext &context, + IPAFrameContext &frameContext, + mali_c55_params_block block); + size_t fillParamsBuffer(mali_c55_params_block block, + enum mali_c55_param_block_type type); + size_t fillWeightsArrayBuffer(mali_c55_params_block block, + enum mali_c55_param_block_type type); + uint32_t estimateCCT() const; + + AgcStatistics statistics_; +}; + +} /* namespace ipa::mali_c55::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/mali-c55/algorithms/meson.build b/src/ipa/mali-c55/algorithms/meson.build index d84432b9..96808431 100644 --- a/src/ipa/mali-c55/algorithms/meson.build +++ b/src/ipa/mali-c55/algorithms/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 mali_c55_ipa_algorithms = files([ + 'agc.cpp', 'blc.cpp', ]) diff --git a/src/ipa/mali-c55/data/uncalibrated.yaml b/src/ipa/mali-c55/data/uncalibrated.yaml index 2cdc39a8..6dcc0295 100644 --- a/src/ipa/mali-c55/data/uncalibrated.yaml +++ b/src/ipa/mali-c55/data/uncalibrated.yaml @@ -3,4 +3,5 @@ --- version: 1 algorithms: + - Agc: ... diff --git a/src/ipa/mali-c55/ipa_context.h b/src/ipa/mali-c55/ipa_context.h index 9e408a17..73a7cd78 100644 --- a/src/ipa/mali-c55/ipa_context.h +++ b/src/ipa/mali-c55/ipa_context.h @@ -7,8 +7,11 @@ #pragma once +#include #include +#include "libcamera/internal/bayer_format.h" + #include namespace libcamera { @@ -16,15 +19,44 @@ namespace libcamera { namespace ipa::mali_c55 { struct IPASessionConfiguration { + struct { + utils::Duration minShutterSpeed; + utils::Duration maxShutterSpeed; + uint32_t defaultExposure; + double minAnalogueGain; + double maxAnalogueGain; + } agc; + + struct { + BayerFormat::Order bayerOrder; + utils::Duration lineDuration; + } sensor; }; struct IPAActiveState { + struct { + struct { + uint32_t exposure; + double sensorGain; + double ispGain; + } automatic; + struct { + uint32_t exposure; + double sensorGain; + double ispGain; + } manual; + bool autoEnabled; + uint32_t constraintMode; + uint32_t exposureMode; + uint32_t temperatureK; + } agc; }; struct IPAFrameContext : public FrameContext { struct { uint32_t exposure; double sensorGain; + double ispGain; } agc; }; diff --git a/src/ipa/mali-c55/mali-c55.cpp b/src/ipa/mali-c55/mali-c55.cpp index 3daac3af..56936b86 100644 --- a/src/ipa/mali-c55/mali-c55.cpp +++ b/src/ipa/mali-c55/mali-c55.cpp @@ -33,6 +33,8 @@ namespace libcamera { LOG_DEFINE_CATEGORY(IPAMaliC55) +using namespace std::literals::chrono_literals; + namespace ipa::mali_c55 { /* Maximum number of frame contexts to be held */ @@ -60,6 +62,9 @@ protected: std::string logPrefix() const override; private: + void updateSessionConfiguration(const IPACameraSensorInfo &info, + const ControlInfoMap &sensorControls, + BayerFormat::Order bayerOrder); void updateControls(const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls); @@ -133,7 +138,21 @@ int IPAMaliC55::init(const IPASettings &settings, const IPAConfigInfo &ipaConfig void IPAMaliC55::setControls() { + IPAActiveState &activeState = context_.activeState; + uint32_t exposure; + uint32_t gain; + + if (activeState.agc.autoEnabled) { + exposure = activeState.agc.automatic.exposure; + gain = camHelper_->gainCode(activeState.agc.automatic.sensorGain); + } else { + exposure = activeState.agc.manual.exposure; + gain = camHelper_->gainCode(activeState.agc.manual.sensorGain); + } + ControlList ctrls(sensorControls_); + ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); + ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain)); setSensorControls.emit(ctrls); } @@ -148,6 +167,36 @@ void IPAMaliC55::stop() context_.frameContexts.clear(); } +void IPAMaliC55::updateSessionConfiguration(const IPACameraSensorInfo &info, + const ControlInfoMap &sensorControls, + BayerFormat::Order bayerOrder) +{ + context_.configuration.sensor.bayerOrder = bayerOrder; + + const ControlInfo &v4l2Exposure = sensorControls.find(V4L2_CID_EXPOSURE)->second; + int32_t minExposure = v4l2Exposure.min().get(); + int32_t maxExposure = v4l2Exposure.max().get(); + int32_t defExposure = v4l2Exposure.def().get(); + + const ControlInfo &v4l2Gain = sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; + int32_t minGain = v4l2Gain.min().get(); + int32_t maxGain = v4l2Gain.max().get(); + + /* + * When the AGC computes the new exposure values for a frame, it needs + * to know the limits for shutter speed and analogue gain. + * As it depends on the sensor, update it with the controls. + * + * \todo take VBLANK into account for maximum shutter speed + */ + context_.configuration.sensor.lineDuration = info.minLineLength * 1.0s / info.pixelRate; + context_.configuration.agc.minShutterSpeed = minExposure * context_.configuration.sensor.lineDuration; + context_.configuration.agc.maxShutterSpeed = maxExposure * context_.configuration.sensor.lineDuration; + context_.configuration.agc.defaultExposure = defExposure; + context_.configuration.agc.minAnalogueGain = camHelper_->gain(minGain); + context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain); +} + void IPAMaliC55::updateControls(const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls) @@ -209,8 +258,7 @@ void IPAMaliC55::updateControls(const IPACameraSensorInfo &sensorInfo, *ipaControls = ControlInfoMap(std::move(ctrlMap), controls::controls); } -int IPAMaliC55::configure(const IPAConfigInfo &ipaConfig, - [[maybe_unused]] uint8_t bayerOrder, +int IPAMaliC55::configure(const IPAConfigInfo &ipaConfig, uint8_t bayerOrder, ControlInfoMap *ipaControls) { sensorControls_ = ipaConfig.sensorControls; @@ -222,6 +270,8 @@ int IPAMaliC55::configure(const IPAConfigInfo &ipaConfig, const IPACameraSensorInfo &info = ipaConfig.sensorInfo; + updateSessionConfiguration(info, ipaConfig.sensorControls, + static_cast(bayerOrder)); updateControls(info, ipaConfig.sensorControls, ipaControls); for (auto const &a : algorithms()) { From patchwork Tue Jul 9 14:49:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20641 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0CEAAC328C for ; Tue, 9 Jul 2024 14:50:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ADA4B63385; Tue, 9 Jul 2024 16:50:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="kLxDd+xw"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AD16C6337B for ; Tue, 9 Jul 2024 16:49:57 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 40B5E1894; Tue, 9 Jul 2024 16:49:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720536565; bh=7gjfvjBVQCAT1ThN0qkIGAF70eZPTch8lYeJj+9PUHY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kLxDd+xwXCLLug7CT4Zj1XdOtU5abfYI2Az6cWziBZlIbCEmwXmnSp/Bg8k5BX324 xHVvDurMNRxP0dEVTgTSy2iWwozXfDVPZkDsXR9kvig9+Z16pqSR7GvcZ+fVFbM1Ou 4svtbC4eotwxOpQbOaB9kBIuWrjbUsZMTX0WVAJs= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Nayden Kanchev , Jacopo Mondi Subject: [PATCH v2 08/10] ipa: mali-c55: Add AWB Algorithm Date: Tue, 9 Jul 2024 15:49:48 +0100 Message-Id: <20240709144950.3277837-9-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709144950.3277837-1-dan.scally@ideasonboard.com> References: <20240709144950.3277837-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a simple grey-world auto white balance algorithm to the mali-c55 IPA. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Reviewed-by: Kieran Bingham --- Changes in v2: - Use the union rather than reinterpret_cast<>() to abstract the block src/ipa/mali-c55/algorithms/awb.cpp | 227 ++++++++++++++++++++++++ src/ipa/mali-c55/algorithms/awb.h | 40 +++++ src/ipa/mali-c55/algorithms/meson.build | 1 + src/ipa/mali-c55/ipa_context.h | 10 ++ 4 files changed, 278 insertions(+) create mode 100644 src/ipa/mali-c55/algorithms/awb.cpp create mode 100644 src/ipa/mali-c55/algorithms/awb.h diff --git a/src/ipa/mali-c55/algorithms/awb.cpp b/src/ipa/mali-c55/algorithms/awb.cpp new file mode 100644 index 00000000..b7b74992 --- /dev/null +++ b/src/ipa/mali-c55/algorithms/awb.cpp @@ -0,0 +1,227 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas On Board Oy + * + * awb.cpp - Mali C55 grey world auto white balance algorithm + */ + +#include "awb.h" + +#include + +#include + +#include + +namespace libcamera { + +namespace ipa::mali_c55::algorithms { + +LOG_DEFINE_CATEGORY(MaliC55Awb) + +/* Number of frames at which we should run AWB at full speed */ +static constexpr uint32_t kNumStartupFrames = 4; + +Awb::Awb() +{ +} + +int Awb::configure([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + /* + * Initially we have no idea what the colour balance will be like, so + * for the first frame we will make no assumptions and leave the R/B + * channels unmodified. + */ + context.activeState.awb.rGain = 1.0; + context.activeState.awb.bGain = 1.0; + + return 0; +} + +size_t Awb::fillGainsParamBlock(mali_c55_params_block block, IPAContext &context, + IPAFrameContext &frameContext) +{ + block.header->type = MALI_C55_PARAM_BLOCK_AWB_GAINS; + block.header->enabled = true; + block.header->size = sizeof(struct mali_c55_params_awb_gains); + + double rGain = context.activeState.awb.rGain; + double bGain = context.activeState.awb.bGain; + + /* + * The gains here map as follows: + * gain00 = R + * gain01 = Gr + * gain10 = Gb + * gain11 = B + * + * This holds true regardless of the bayer order of the input data, as + * the mapping is done internally in the ISP. + */ + block.awb_gains->gain00 = int(rGain * pow(2, 8)); + block.awb_gains->gain01 = 256; /* Otherwise known as 1.0 */ + block.awb_gains->gain10 = 256; + block.awb_gains->gain11 = int(bGain * pow(2, 8)); + + frameContext.awb.rGain = rGain; + frameContext.awb.bGain = bGain; + + return sizeof(struct mali_c55_params_awb_gains); +} + +size_t Awb::fillConfigParamBlock(mali_c55_params_block block) +{ + block.header->type = MALI_C55_PARAM_BLOCK_AWB_CONFIG; + block.header->enabled = true; + block.header->size = sizeof(struct mali_c55_params_awb_config); + + /* Tap the stats after the purple fringe block */ + block.awb_config->tap_point = MALI_C55_AWB_STATS_TAP_PF; + + /* Get R/G and B/G ratios as statistics */ + block.awb_config->stats_mode = MALI_C55_AWB_MODE_RGBG; + + /* Default white level */ + block.awb_config->white_level = 1023; + + /* Default black level */ + block.awb_config->black_level = 0; + + /* + * By default pixels are included who's colour ratios are bounded in a + * region (on a cr ratio x cb ratio graph) defined by four points: + * (0.25, 0.25) + * (0.25, 1.99609375) + * (1.99609375, 1.99609375) + * (1.99609375, 0.25) + * + * The ratios themselves are stored in Q4.8 format. + * + * \todo should these perhaps be tunable? + */ + block.awb_config->cr_max = 511; + block.awb_config->cr_min = 64; + block.awb_config->cb_max = 511; + block.awb_config->cb_min = 64; + + /* We use the full 15x15 zoning scheme */ + block.awb_config->nodes_used_horiz = 15; + block.awb_config->nodes_used_vert = 15; + + /* + * We set the trimming boundaries equivalent to the main boundaries. In + * other words; no trimming. + */ + block.awb_config->cr_high = 511; + block.awb_config->cr_low = 64; + block.awb_config->cb_high = 511; + block.awb_config->cb_low = 64; + + return sizeof(struct mali_c55_params_awb_config); +} + +void Awb::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, mali_c55_params_buffer *params) +{ + mali_c55_params_block block; + block.data = ¶ms->data[params->total_size]; + + params->total_size += fillGainsParamBlock(block, context, frameContext); + + if (frame > 0) + return; + + block.data = ¶ms->data[params->total_size]; + params->total_size += fillConfigParamBlock(block); +} + +void Awb::process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const mali_c55_stats_buffer *stats, + [[maybe_unused]] ControlList &metadata) +{ + const struct mali_c55_awb_average_ratios *awb_ratios = stats->awb_ratios; + + /* + * The ISP produces average R:G and B:G ratios for zones. We take the + * average of all the zones with data and simply invert them to provide + * gain figures that we can apply to approximate a grey world. + */ + unsigned int counted_zones = 0; + double rgSum = 0, bgSum = 0; + + for (unsigned int i = 0; i < 225; i++) { + if (!awb_ratios[i].num_pixels) + continue; + + /* + * The statistics are in Q4.8 format, so we convert to double + * here. + */ + rgSum += (awb_ratios[i].avg_rg_gr * pow(2, -8)); + bgSum += (awb_ratios[i].avg_bg_br * pow(2, -8)); + counted_zones++; + } + + /* + * Sometimes the first frame's statistics have no valid pixels, in which + * case we'll just assume a grey world until they say otherwise. + */ + double rgAvg, bgAvg; + if (!counted_zones) { + rgAvg = 1.0; + bgAvg = 1.0; + } else { + rgAvg = rgSum / counted_zones; + bgAvg = bgSum / counted_zones; + } + + /* + * The statistics are generated _after_ white balancing is performed in + * the ISP. To get the true ratio we therefore have to adjust the stats + * figure by the gains that were applied when the statistics for this + * frame were generated. + */ + double rRatio = rgAvg / frameContext.awb.rGain; + double bRatio = bgAvg / frameContext.awb.bGain; + + /* + * And then we can simply invert the ratio to find the gain we should + * apply. + */ + double rGain = 1 / rRatio; + double bGain = 1 / bRatio; + + /* + * Running at full speed, this algorithm results in oscillations in the + * colour balance. To remove those we dampen the speed at which it makes + * changes in gain, unless we're in the startup phase in which case we + * want to fix the miscolouring as quickly as possible. + */ + double speed = frame < kNumStartupFrames ? 1.0 : 0.2; + rGain = speed * rGain + context.activeState.awb.rGain * (1.0 - speed); + bGain = speed * bGain + context.activeState.awb.bGain * (1.0 - speed); + + context.activeState.awb.rGain = rGain; + context.activeState.awb.bGain = bGain; + + metadata.set(controls::ColourGains, { + static_cast(frameContext.awb.rGain), + static_cast(frameContext.awb.bGain), + }); + + LOG(MaliC55Awb, Debug) << "For frame number " << frame << ": " + << "Average R/G Ratio: " << rgAvg + << ", Average B/G Ratio: " << bgAvg + << "\nrGain applied to this frame: " << frameContext.awb.rGain + << ", bGain applied to this frame: " << frameContext.awb.bGain + << "\nrGain to apply: " << context.activeState.awb.rGain + << ", bGain to apply: " << context.activeState.awb.bGain; +} + +REGISTER_IPA_ALGORITHM(Awb, "Awb") + +} /* namespace ipa::mali_c55::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/mali-c55/algorithms/awb.h b/src/ipa/mali-c55/algorithms/awb.h new file mode 100644 index 00000000..800c2e83 --- /dev/null +++ b/src/ipa/mali-c55/algorithms/awb.h @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas on Board Oy + * + * awb.h - Mali C55 grey world auto white balance algorithm + */ + +#include "algorithm.h" +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::mali_c55::algorithms { + +class Awb : public Algorithm +{ +public: + Awb(); + ~Awb() = default; + + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + mali_c55_params_buffer *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const mali_c55_stats_buffer *stats, + ControlList &metadata) override; + +private: + size_t fillGainsParamBlock(mali_c55_params_block block, + IPAContext &context, + IPAFrameContext &frameContext); + size_t fillConfigParamBlock(mali_c55_params_block block); +}; + +} /* namespace ipa::mali_c55::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/mali-c55/algorithms/meson.build b/src/ipa/mali-c55/algorithms/meson.build index 96808431..f11791aa 100644 --- a/src/ipa/mali-c55/algorithms/meson.build +++ b/src/ipa/mali-c55/algorithms/meson.build @@ -2,5 +2,6 @@ mali_c55_ipa_algorithms = files([ 'agc.cpp', + 'awb.cpp', 'blc.cpp', ]) diff --git a/src/ipa/mali-c55/ipa_context.h b/src/ipa/mali-c55/ipa_context.h index 73a7cd78..105e5776 100644 --- a/src/ipa/mali-c55/ipa_context.h +++ b/src/ipa/mali-c55/ipa_context.h @@ -50,6 +50,11 @@ struct IPAActiveState { uint32_t exposureMode; uint32_t temperatureK; } agc; + + struct { + double rGain; + double bGain; + } awb; }; struct IPAFrameContext : public FrameContext { @@ -58,6 +63,11 @@ struct IPAFrameContext : public FrameContext { double sensorGain; double ispGain; } agc; + + struct { + double rGain; + double bGain; + } awb; }; struct IPAContext { From patchwork Tue Jul 9 14:49:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20640 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C1AA2BD87C for ; Tue, 9 Jul 2024 14:50:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 12F1663380; Tue, 9 Jul 2024 16:50:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="lGEZDsek"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1434063374 for ; Tue, 9 Jul 2024 16:49:58 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 92AA313D7; Tue, 9 Jul 2024 16:49:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720536565; bh=DnqmmdK8j0cogtyHk+OI/hrdJn0771hTlfAQ21dJ8Is=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lGEZDsekVDaldDdXbTiiBVaFmBlzXOGGfBntAoWk0rlG9I9N3wKokMuHuERDumJLX zpHILBOeRh43KC0pItJP/Dh2d5GwI8KBdT8QztcfnrSEd+nMv74wa2j6WsU7j2YNw6 FAbmM8Vf7edNkvqiYgvqZxG9qdb7pK9MoqeMmwnU= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Nayden Kanchev , Jacopo Mondi Subject: [PATCH v2 09/10] ipa: mali-c55: Add Lens Shading Correction algorithm Date: Tue, 9 Jul 2024 15:49:49 +0100 Message-Id: <20240709144950.3277837-10-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709144950.3277837-1-dan.scally@ideasonboard.com> References: <20240709144950.3277837-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a lens shading correction algorithm to the mali-c55 IPA. This algorithm parses tables from Yaml in a easy to follow format before munging them into Arm's interleaved mesh to be copied to the ISP. A colour temperature estimate from the AGC statistics is used to select the appropriate table to apply; this can be some interpolation of two tables, in which case the colour temperature estimate is also used to derive the coefficient that does the blending. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Reviewed-by: Kieran Bingham --- Changes in v2: - Use the union rather than reinterpret_cast<>() to abstract the block src/ipa/mali-c55/algorithms/lsc.cpp | 216 ++++++++++++++++++++++++ src/ipa/mali-c55/algorithms/lsc.h | 45 +++++ src/ipa/mali-c55/algorithms/meson.build | 1 + 3 files changed, 262 insertions(+) create mode 100644 src/ipa/mali-c55/algorithms/lsc.cpp create mode 100644 src/ipa/mali-c55/algorithms/lsc.h diff --git a/src/ipa/mali-c55/algorithms/lsc.cpp b/src/ipa/mali-c55/algorithms/lsc.cpp new file mode 100644 index 00000000..8d574779 --- /dev/null +++ b/src/ipa/mali-c55/algorithms/lsc.cpp @@ -0,0 +1,216 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas On Board Oy + * + * lsc.cpp - Mali-C55 Lens shading correction algorithm + */ + +#include "lsc.h" + +#include "libcamera/internal/yaml_parser.h" + +namespace libcamera { + +namespace ipa::mali_c55::algorithms { + +LOG_DEFINE_CATEGORY(MaliC55Lsc) + +Lsc::Lsc() +{ +} + +int Lsc::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) +{ + if (!tuningData.contains("meshScale")) { + LOG(MaliC55Lsc, Error) << "meshScale missing from tuningData"; + return -EINVAL; + } + + meshScale_ = tuningData["meshScale"].get(0); + + const YamlObject &yamlSets = tuningData["sets"]; + if (!yamlSets.isList()) { + LOG(MaliC55Lsc, Error) << "LSC tables missing or invalid"; + return -EINVAL; + } + + size_t tableSize = 0; + const auto &sets = yamlSets.asList(); + for (const auto &yamlSet : sets) { + uint32_t ct = yamlSet["ct"].get(0); + + if (!ct) { + LOG(MaliC55Lsc, Error) << "Invalid colour temperature"; + return -EINVAL; + } + + if (std::count(colourTemperatures_.begin(), + colourTemperatures_.end(), ct)) { + LOG(MaliC55Lsc, Error) + << "Multiple sets found for colour temperature"; + return -EINVAL; + } + + std::vector rTable = + yamlSet["r"].getList().value_or(std::vector{}); + std::vector gTable = + yamlSet["g"].getList().value_or(std::vector{}); + std::vector bTable = + yamlSet["b"].getList().value_or(std::vector{}); + + /* + * Some validation to do; only 16x16 and 32x32 tables of + * coefficients are acceptable, and all tables across all of the + * sets must be the same size. The first time we encounter a + * table we check that it is an acceptable size and if so make + * sure all other tables are of equal size. + */ + if (!tableSize) { + if (rTable.size() != 256 && rTable.size() != 1024) { + LOG(MaliC55Lsc, Error) + << "Invalid table size for colour temperature " << ct; + return -EINVAL; + } + tableSize = rTable.size(); + } + + if (rTable.size() != tableSize || + gTable.size() != tableSize || + bTable.size() != tableSize) { + LOG(MaliC55Lsc, Error) + << "Invalid or mismatched table size for colour temperature " << ct; + return -EINVAL; + } + + if (colourTemperatures_.size() >= 3) { + LOG(MaliC55Lsc, Error) + << "A maximum of 3 colour temperatures are supported"; + return -EINVAL; + } + + for (unsigned int i = 0; i < tableSize; i++) { + mesh_[kRedOffset + i] |= + (rTable[i] << (colourTemperatures_.size() * 8)); + mesh_[kGreenOffset + i] |= + (gTable[i] << (colourTemperatures_.size() * 8)); + mesh_[kBlueOffset + i] |= + (bTable[i] << (colourTemperatures_.size() * 8)); + } + + colourTemperatures_.push_back(ct); + } + + if (tableSize == 256) + meshSize_ = 15; + else + meshSize_ = 31; + + return 0; +} + +size_t Lsc::fillConfigParamsBlock(mali_c55_params_block block) const +{ + block.header->type = MALI_C55_PARAM_MESH_SHADING_CONFIG; + block.header->enabled = true; + block.header->size = sizeof(struct mali_c55_params_mesh_shading_config); + + block.shading_config->mesh_show = false; + block.shading_config->mesh_scale = meshScale_; + block.shading_config->mesh_page_r = 0; + block.shading_config->mesh_page_g = 1; + block.shading_config->mesh_page_b = 2; + block.shading_config->mesh_width = meshSize_; + block.shading_config->mesh_height = meshSize_; + + std::copy(mesh_.begin(), mesh_.end(), block.shading_config->mesh); + + return block.header->size; +} + +size_t Lsc::fillSelectionParamsBlock(mali_c55_params_block block, uint8_t bank, + uint8_t alpha) const +{ + block.header->type = MALI_C55_PARAM_MESH_SHADING_SELECTION; + block.header->enabled = true; + block.header->size = sizeof(struct mali_c55_params_mesh_shading_selection); + + block.shading_selection->mesh_alpha_bank_r = bank; + block.shading_selection->mesh_alpha_bank_g = bank; + block.shading_selection->mesh_alpha_bank_b = bank; + block.shading_selection->mesh_alpha_r = alpha; + block.shading_selection->mesh_alpha_g = alpha; + block.shading_selection->mesh_alpha_b = alpha; + block.shading_selection->mesh_strength = 0x1000; /* Otherwise known as 1.0 */ + + return block.header->size; +} + +std::tuple Lsc::findBankAndAlpha(uint32_t ct) const +{ + unsigned int i; + + ct = std::clamp(ct, colourTemperatures_.front(), + colourTemperatures_.back()); + + for (i = 0; i < colourTemperatures_.size() - 1; i++) { + if (ct >= colourTemperatures_[i] && + ct <= colourTemperatures_[i + 1]) + break; + } + + /* + * With the clamping, we're guaranteed an index into colourTemperatures_ + * that's <= colourTemperatures_.size() - 1. + */ + uint8_t alpha = (255 * (ct - colourTemperatures_[i])) / + (colourTemperatures_[i + 1] - colourTemperatures_[i]); + + return { i, alpha }; +} + +void Lsc::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + mali_c55_params_buffer *params) +{ + /* + * For each frame we assess the colour temperature of the **last** frame + * and then select an appropriately blended table of coefficients based + * on that ct. As a bit of a shortcut, if we've only a single table the + * handling is somewhat simpler; if it's the first frame we just select + * that table and if we're past the first frame then we can just do + * nothing - the config will never change. + */ + uint32_t temperatureK = context.activeState.agc.temperatureK; + uint8_t bank, alpha; + + if (colourTemperatures_.size() == 1) { + if (frame > 0) + return; + + bank = 0; + alpha = 0; + } else { + std::tie(bank, alpha) = findBankAndAlpha(temperatureK); + } + + mali_c55_params_block block; + block.data = ¶ms->data[params->total_size]; + + params->total_size += fillSelectionParamsBlock(block, bank, alpha); + + if (frame > 0) + return; + + /* + * If this is the first frame, we need to load the parsed coefficient + * tables from tuning data to the ISP. + */ + block.data = ¶ms->data[params->total_size]; + params->total_size += fillConfigParamsBlock(block); +} + +REGISTER_IPA_ALGORITHM(Lsc, "Lsc") + +} /* namespace ipa::mali_c55::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/mali-c55/algorithms/lsc.h b/src/ipa/mali-c55/algorithms/lsc.h new file mode 100644 index 00000000..ed356f4a --- /dev/null +++ b/src/ipa/mali-c55/algorithms/lsc.h @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas On Board Oy + * + * lsc.h - Mali-C55 Lens shading correction algorithm + */ + +#include +#include + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::mali_c55::algorithms { + +class Lsc : public Algorithm +{ +public: + Lsc(); + ~Lsc() = default; + + int init(IPAContext &context, const YamlObject &tuningData) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + mali_c55_params_buffer *params) override; +private: + static constexpr unsigned int kRedOffset = 0; + static constexpr unsigned int kGreenOffset = 1024; + static constexpr unsigned int kBlueOffset = 2048; + + size_t fillConfigParamsBlock(mali_c55_params_block block) const; + size_t fillSelectionParamsBlock(mali_c55_params_block block, + uint8_t bank, uint8_t alpha) const; + std::tuple findBankAndAlpha(uint32_t ct) const; + + std::vector mesh_ = std::vector(3072); + std::vector colourTemperatures_; + uint32_t meshScale_; + uint32_t meshSize_; +}; + +} /* namespace ipa::mali_c55::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/mali-c55/algorithms/meson.build b/src/ipa/mali-c55/algorithms/meson.build index f11791aa..1665da07 100644 --- a/src/ipa/mali-c55/algorithms/meson.build +++ b/src/ipa/mali-c55/algorithms/meson.build @@ -4,4 +4,5 @@ mali_c55_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', 'blc.cpp', + 'lsc.cpp', ]) From patchwork Tue Jul 9 14:49:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20639 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8530EC328C for ; Tue, 9 Jul 2024 14:50:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0555863394; Tue, 9 Jul 2024 16:50:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uEd+5Mm2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6B3E263376 for ; Tue, 9 Jul 2024 16:49:58 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EA4702397; Tue, 9 Jul 2024 16:49:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720536566; bh=wbfbB9u4j7w+/aL5cYJo7eurFEQ2bvLeERunKOjX+pI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uEd+5Mm21oNz2nHpEzDwk1Ty4jylOjkGjnqGM4XD3PHrav57+ZhOG6hFmsXeJpmwC Juzg99P4jUIprHA38j77g9zsLvMNew7AmrnhExScPwA2koIOxQScs6dmxUDe5yLIQz i0gJwEmmxOV7NXI7T9AaOPte/tiVEE7pJrpdeGuQ= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Nayden Kanchev , Jacopo Mondi Subject: [PATCH v2 10/10] ipa: mali-c55: Add IMX415 tuning data file Date: Tue, 9 Jul 2024 15:49:50 +0100 Message-Id: <20240709144950.3277837-11-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709144950.3277837-1-dan.scally@ideasonboard.com> References: <20240709144950.3277837-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a tuning data file for the IMX415 camera sensor. The black level offset data is drawn from the camera sensor's datasheet. The lens shading tables are generated through the libtuning LSC modules. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Acked-by: Kieran Bingham --- Changes in v2: - None src/ipa/mali-c55/data/imx415.yaml | 325 ++++++++++++++++++++++++++++++ src/ipa/mali-c55/data/meson.build | 1 + 2 files changed, 326 insertions(+) create mode 100644 src/ipa/mali-c55/data/imx415.yaml diff --git a/src/ipa/mali-c55/data/imx415.yaml b/src/ipa/mali-c55/data/imx415.yaml new file mode 100644 index 00000000..126b427a --- /dev/null +++ b/src/ipa/mali-c55/data/imx415.yaml @@ -0,0 +1,325 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +version: 1 +algorithms: + - Agc: + - Awb: + - BlackLevelCorrection: + offset00: 51200 + offset01: 51200 + offset10: 51200 + offset11: 51200 + - Lsc: + meshScale: 4 # 1.0 - 2.0 Gain + sets: + - ct: 2500 + r: [ + 21, 20, 19, 17, 15, 14, 12, 11, 9, 9, 9, 9, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 13, 13, 10, 10, 13, 16, 17, 18, 21, 22, + 21, 20, 18, 16, 14, 13, 12, 11, 10, 9, 9, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 12, 15, 17, 18, 21, 21, + 20, 19, 17, 16, 14, 13, 12, 11, 10, 9, 8, 8, 8, 7, 7, 7, 8, 8, 8, 8, 8, 9, 9, 8, 8, 8, 11, 15, 17, 18, 21, 21, + 19, 19, 17, 15, 14, 13, 12, 11, 10, 8, 8, 7, 7, 7, 6, 6, 7, 7, 8, 8, 8, 8, 8, 7, 7, 8, 10, 14, 17, 18, 20, 22, + 19, 18, 17, 15, 14, 13, 11, 11, 9, 8, 8, 7, 7, 6, 5, 5, 5, 5, 6, 7, 8, 7, 7, 6, 7, 7, 10, 12, 16, 18, 20, 22, + 18, 18, 16, 15, 14, 12, 11, 10, 9, 8, 6, 6, 5, 5, 4, 4, 5, 5, 5, 5, 6, 6, 6, 6, 6, 7, 8, 12, 16, 18, 19, 20, + 18, 18, 16, 14, 13, 12, 11, 9, 9, 7, 6, 5, 5, 5, 4, 4, 4, 5, 5, 4, 4, 5, 5, 5, 5, 6, 8, 11, 15, 18, 18, 19, + 18, 17, 15, 14, 13, 12, 11, 9, 8, 7, 6, 5, 5, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 6, 7, 9, 14, 17, 18, 18, + 18, 17, 15, 14, 13, 12, 11, 9, 8, 7, 5, 5, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 6, 8, 12, 17, 18, 18, + 18, 16, 15, 13, 12, 11, 10, 9, 8, 7, 5, 4, 4, 4, 4, 3, 3, 4, 4, 4, 3, 3, 4, 4, 4, 5, 6, 8, 12, 16, 19, 19, + 17, 16, 15, 13, 12, 11, 10, 8, 7, 6, 4, 4, 3, 3, 3, 3, 3, 3, 4, 4, 3, 3, 3, 4, 4, 5, 6, 9, 12, 16, 19, 20, + 17, 15, 15, 13, 12, 11, 10, 8, 6, 6, 5, 4, 3, 3, 3, 2, 3, 3, 4, 4, 3, 3, 2, 3, 4, 5, 6, 9, 11, 16, 19, 20, + 17, 15, 15, 14, 11, 11, 10, 8, 6, 5, 5, 4, 3, 3, 2, 2, 2, 3, 3, 3, 3, 3, 2, 3, 4, 5, 6, 8, 11, 16, 18, 19, + 16, 16, 15, 13, 11, 11, 10, 7, 6, 5, 4, 4, 3, 2, 2, 2, 2, 3, 3, 3, 3, 2, 2, 2, 3, 4, 6, 8, 11, 14, 17, 18, + 16, 16, 14, 13, 11, 10, 9, 7, 6, 5, 4, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 2, 2, 2, 3, 4, 6, 8, 10, 14, 17, 18, + 16, 15, 14, 13, 13, 10, 9, 7, 6, 4, 4, 2, 2, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 4, 4, 6, 8, 11, 17, 18, 19, + 16, 15, 14, 14, 13, 12, 9, 8, 7, 5, 4, 2, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 3, 4, 5, 6, 8, 12, 17, 19, 20, + 17, 15, 15, 14, 13, 12, 9, 8, 7, 5, 3, 2, 1, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 4, 5, 6, 8, 13, 16, 19, 21, + 17, 16, 15, 13, 13, 12, 9, 8, 7, 5, 3, 2, 0, 0, 0, 0, 1, 1, 1, 2, 2, 3, 3, 4, 5, 5, 7, 8, 13, 16, 19, 20, + 17, 16, 15, 14, 13, 12, 9, 8, 7, 5, 3, 2, 0, 0, 0, 0, 1, 1, 1, 2, 3, 3, 3, 4, 5, 6, 7, 9, 13, 17, 19, 20, + 18, 16, 15, 14, 13, 12, 9, 8, 7, 5, 4, 2, 1, 0, 0, 0, 0, 1, 2, 2, 3, 3, 3, 4, 5, 6, 8, 9, 13, 17, 20, 20, + 18, 16, 16, 15, 14, 12, 10, 9, 7, 6, 5, 3, 2, 1, 0, 0, 0, 1, 2, 3, 3, 3, 4, 5, 6, 7, 9, 10, 14, 18, 20, 20, + 18, 17, 16, 15, 14, 12, 10, 9, 8, 7, 6, 5, 3, 3, 1, 0, 0, 1, 2, 3, 4, 4, 5, 6, 6, 7, 9, 12, 15, 19, 20, 20, + 18, 18, 17, 16, 14, 13, 11, 10, 9, 8, 7, 6, 5, 5, 3, 1, 1, 1, 2, 3, 5, 5, 5, 6, 7, 9, 12, 15, 17, 19, 20, 20, + 18, 18, 17, 16, 15, 13, 12, 10, 10, 9, 8, 7, 6, 5, 4, 2, 1, 2, 3, 4, 5, 5, 6, 6, 8, 10, 13, 16, 18, 20, 20, 21, + 19, 18, 17, 16, 15, 14, 13, 11, 10, 10, 9, 8, 7, 6, 5, 4, 3, 2, 3, 3, 5, 5, 6, 7, 10, 11, 14, 17, 19, 20, 21, 22, + 20, 19, 18, 17, 16, 15, 13, 12, 11, 10, 10, 9, 8, 7, 6, 5, 4, 3, 3, 3, 5, 6, 6, 7, 10, 12, 14, 18, 20, 21, 22, 23, + 21, 20, 19, 18, 17, 16, 14, 13, 12, 11, 10, 10, 9, 7, 7, 5, 5, 4, 4, 5, 6, 6, 7, 8, 11, 13, 16, 19, 21, 22, 22, 22, + 22, 21, 20, 19, 18, 17, 16, 14, 13, 12, 12, 10, 9, 8, 7, 6, 6, 5, 5, 6, 7, 7, 8, 9, 12, 14, 18, 20, 21, 22, 22, 22, + 23, 22, 21, 20, 19, 17, 16, 15, 14, 14, 13, 12, 10, 9, 8, 7, 6, 5, 5, 6, 7, 8, 8, 10, 12, 15, 18, 20, 21, 22, 22, 22, + 24, 23, 22, 21, 20, 18, 17, 16, 15, 15, 14, 14, 13, 11, 9, 8, 6, 6, 6, 6, 7, 8, 9, 11, 14, 17, 19, 20, 21, 21, 21, 21, + 24, 24, 23, 21, 20, 19, 17, 16, 15, 15, 15, 14, 14, 14, 11, 9, 6, 5, 5, 6, 8, 8, 10, 12, 15, 17, 20, 20, 21, 21, 21, 21, + ] + g: [ + 19, 18, 17, 15, 13, 12, 10, 9, 8, 8, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 12, 12, 9, 9, 11, 15, 15, 16, 19, 20, + 19, 18, 16, 15, 12, 12, 10, 10, 8, 7, 7, 7, 7, 6, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 7, 7, 11, 14, 15, 16, 19, 19, + 18, 17, 16, 14, 12, 12, 10, 10, 8, 7, 7, 6, 6, 5, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 6, 6, 10, 14, 15, 17, 19, 19, + 17, 17, 16, 14, 12, 12, 10, 10, 8, 7, 6, 6, 6, 5, 5, 4, 5, 5, 6, 6, 6, 7, 7, 5, 6, 6, 9, 13, 15, 17, 18, 20, + 17, 17, 15, 14, 12, 11, 10, 9, 8, 7, 6, 5, 5, 4, 4, 4, 4, 4, 5, 6, 6, 6, 5, 5, 5, 6, 8, 11, 15, 17, 18, 20, + 17, 17, 15, 13, 12, 11, 9, 9, 8, 7, 5, 5, 4, 4, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 7, 11, 15, 17, 18, 18, + 17, 16, 15, 13, 12, 11, 9, 8, 8, 6, 5, 4, 4, 4, 3, 3, 4, 4, 3, 3, 3, 4, 4, 4, 4, 5, 6, 9, 14, 17, 17, 18, + 17, 16, 14, 13, 12, 11, 9, 8, 7, 6, 5, 4, 4, 4, 4, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 6, 8, 13, 16, 17, 17, + 17, 15, 14, 13, 12, 11, 9, 8, 7, 6, 5, 4, 4, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 5, 7, 12, 16, 17, 17, + 17, 15, 14, 12, 11, 10, 9, 8, 7, 6, 4, 4, 3, 3, 3, 2, 2, 3, 3, 3, 2, 2, 3, 3, 3, 4, 5, 7, 11, 15, 18, 18, + 16, 14, 13, 12, 11, 10, 9, 7, 7, 5, 4, 3, 3, 3, 2, 2, 2, 3, 3, 3, 2, 2, 2, 3, 3, 4, 5, 8, 11, 15, 18, 19, + 16, 14, 13, 12, 11, 10, 9, 7, 5, 5, 4, 3, 3, 2, 2, 2, 2, 3, 3, 3, 2, 2, 2, 2, 3, 4, 5, 8, 10, 15, 18, 19, + 16, 14, 14, 13, 11, 10, 9, 7, 5, 5, 4, 3, 2, 2, 2, 2, 2, 2, 3, 3, 2, 2, 2, 2, 3, 3, 5, 7, 10, 15, 17, 18, + 16, 15, 14, 12, 11, 10, 9, 7, 5, 5, 4, 3, 2, 2, 1, 1, 2, 2, 2, 2, 2, 2, 1, 2, 2, 3, 5, 6, 10, 14, 17, 17, + 15, 15, 13, 12, 11, 10, 9, 7, 5, 4, 3, 2, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 1, 2, 2, 3, 5, 7, 10, 14, 17, 18, + 15, 14, 13, 12, 12, 10, 9, 7, 6, 4, 3, 2, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 3, 4, 5, 7, 10, 17, 18, 18, + 15, 14, 14, 13, 12, 11, 9, 7, 6, 4, 3, 2, 1, 0, 1, 1, 1, 2, 2, 2, 2, 2, 2, 3, 3, 4, 5, 7, 12, 17, 19, 19, + 16, 14, 14, 13, 12, 12, 9, 7, 6, 4, 3, 2, 0, 0, 0, 1, 1, 1, 2, 2, 2, 2, 2, 3, 4, 4, 5, 8, 12, 17, 19, 20, + 16, 15, 14, 13, 12, 12, 9, 7, 7, 4, 3, 1, 0, 0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 4, 4, 6, 8, 12, 17, 19, 20, + 17, 15, 14, 13, 12, 12, 9, 7, 7, 5, 3, 1, 0, 0, 0, 0, 1, 1, 1, 2, 2, 3, 3, 4, 4, 5, 6, 8, 12, 17, 19, 20, + 18, 15, 15, 14, 13, 12, 9, 8, 7, 5, 4, 2, 1, 0, 0, 0, 0, 1, 2, 2, 3, 3, 3, 4, 5, 6, 7, 9, 13, 17, 19, 20, + 18, 16, 15, 14, 13, 12, 9, 9, 7, 6, 5, 3, 2, 1, 0, 0, 0, 1, 2, 3, 3, 3, 4, 5, 6, 7, 9, 10, 14, 18, 20, 20, + 18, 16, 16, 15, 13, 12, 10, 9, 8, 7, 6, 5, 3, 3, 1, 0, 0, 1, 2, 3, 4, 4, 5, 5, 6, 7, 9, 12, 15, 19, 20, 20, + 18, 18, 16, 16, 14, 13, 10, 10, 9, 8, 7, 6, 5, 5, 3, 1, 1, 1, 2, 3, 5, 5, 5, 6, 6, 8, 11, 15, 17, 19, 20, 20, + 18, 18, 16, 16, 14, 13, 12, 10, 9, 9, 8, 7, 6, 5, 4, 3, 1, 2, 3, 5, 5, 5, 5, 6, 7, 10, 12, 15, 18, 20, 20, 20, + 18, 18, 17, 16, 15, 14, 12, 11, 10, 10, 9, 8, 7, 6, 5, 4, 3, 2, 3, 3, 5, 5, 5, 6, 9, 11, 14, 16, 19, 20, 20, 21, + 19, 19, 18, 17, 16, 15, 13, 12, 11, 10, 10, 9, 8, 7, 5, 4, 4, 3, 3, 3, 5, 5, 6, 7, 10, 12, 14, 18, 20, 20, 21, 22, + 21, 20, 18, 18, 17, 16, 14, 12, 11, 11, 10, 10, 8, 7, 7, 5, 4, 4, 4, 5, 6, 6, 6, 7, 11, 13, 16, 19, 20, 21, 21, 22, + 22, 21, 20, 19, 18, 16, 15, 14, 12, 12, 12, 10, 9, 8, 7, 6, 5, 4, 5, 6, 6, 7, 7, 8, 12, 13, 17, 19, 21, 21, 21, 21, + 23, 22, 21, 20, 18, 17, 16, 16, 14, 14, 13, 12, 10, 10, 8, 7, 6, 5, 5, 6, 7, 7, 8, 9, 12, 15, 18, 19, 21, 21, 21, 20, + 23, 22, 22, 21, 20, 18, 17, 16, 15, 15, 15, 14, 13, 11, 9, 8, 6, 6, 6, 6, 7, 8, 9, 10, 13, 16, 19, 20, 20, 21, 21, 20, + 24, 23, 22, 21, 20, 19, 17, 17, 16, 16, 15, 15, 14, 14, 11, 9, 6, 6, 6, 6, 8, 8, 10, 12, 15, 17, 19, 20, 20, 21, 21, 20, + ] + b: [ + 11, 9, 9, 7, 6, 5, 4, 4, 3, 3, 3, 3, 3, 4, 5, 5, 5, 4, 5, 4, 4, 4, 7, 7, 3, 3, 5, 8, 8, 9, 11, 11, + 11, 10, 8, 7, 5, 5, 5, 4, 3, 3, 3, 3, 3, 3, 4, 4, 5, 4, 5, 4, 4, 4, 4, 4, 2, 2, 5, 8, 8, 9, 11, 11, + 10, 10, 7, 7, 5, 5, 5, 4, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 3, 1, 1, 5, 7, 9, 9, 11, 11, + 10, 9, 8, 7, 6, 5, 5, 5, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 3, 2, 1, 1, 4, 7, 9, 10, 12, 13, + 9, 9, 8, 7, 6, 5, 5, 5, 4, 3, 3, 3, 3, 2, 2, 3, 3, 3, 3, 3, 4, 3, 2, 1, 1, 1, 4, 6, 9, 10, 12, 13, + 9, 9, 9, 7, 6, 6, 5, 4, 4, 3, 3, 3, 2, 2, 2, 2, 3, 3, 2, 1, 2, 2, 2, 1, 1, 1, 2, 6, 9, 10, 11, 12, + 8, 9, 9, 7, 7, 6, 5, 4, 4, 3, 3, 2, 2, 2, 2, 2, 3, 3, 2, 1, 1, 1, 1, 0, 0, 1, 2, 5, 9, 11, 11, 11, + 8, 9, 9, 7, 7, 6, 5, 4, 4, 3, 3, 2, 2, 2, 2, 2, 3, 3, 2, 1, 1, 1, 1, 0, 0, 0, 1, 3, 8, 11, 11, 11, + 9, 9, 8, 7, 7, 7, 6, 4, 4, 3, 3, 2, 2, 2, 2, 2, 3, 3, 2, 1, 1, 1, 1, 0, 0, 0, 1, 3, 7, 11, 11, 11, + 9, 9, 8, 7, 7, 7, 6, 5, 4, 3, 2, 2, 2, 2, 2, 2, 2, 3, 2, 1, 1, 0, 0, 0, 0, 0, 1, 3, 7, 10, 12, 13, + 9, 9, 8, 8, 7, 7, 6, 5, 4, 3, 2, 1, 2, 1, 2, 2, 2, 3, 2, 1, 0, 0, 0, 0, 0, 0, 1, 4, 7, 10, 13, 13, + 9, 8, 8, 8, 7, 7, 6, 5, 3, 3, 2, 1, 1, 1, 1, 1, 2, 3, 3, 2, 1, 0, 0, 0, 0, 1, 2, 4, 7, 11, 13, 13, + 9, 8, 8, 7, 7, 6, 6, 5, 3, 3, 2, 2, 2, 1, 1, 1, 2, 3, 3, 2, 1, 0, 0, 0, 0, 1, 2, 4, 6, 11, 13, 13, + 9, 8, 8, 7, 7, 6, 6, 4, 3, 3, 2, 2, 1, 1, 1, 1, 2, 3, 3, 3, 1, 1, 0, 0, 1, 1, 2, 3, 6, 10, 12, 13, + 9, 8, 8, 7, 7, 6, 6, 4, 3, 3, 2, 1, 1, 1, 1, 1, 1, 3, 3, 3, 1, 1, 1, 1, 1, 2, 2, 4, 6, 10, 13, 13, + 9, 9, 8, 8, 8, 6, 6, 4, 4, 3, 2, 1, 1, 1, 1, 1, 2, 2, 3, 3, 1, 1, 1, 1, 2, 2, 3, 4, 6, 13, 14, 14, + 9, 9, 8, 8, 8, 8, 6, 5, 4, 3, 2, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 1, 1, 2, 2, 3, 3, 5, 9, 13, 15, 15, + 10, 9, 9, 9, 10, 10, 6, 6, 5, 3, 2, 1, 0, 0, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 4, 5, 9, 13, 15, 16, + 10, 10, 9, 9, 10, 10, 7, 6, 5, 3, 2, 1, 0, 0, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 3, 3, 4, 5, 9, 13, 15, 16, + 11, 10, 9, 9, 10, 10, 7, 6, 6, 3, 2, 1, 0, 0, 1, 1, 1, 2, 2, 2, 2, 2, 2, 3, 3, 4, 5, 6, 9, 13, 15, 16, + 12, 10, 10, 10, 10, 10, 7, 6, 6, 4, 3, 1, 1, 1, 1, 1, 1, 2, 2, 3, 3, 3, 3, 3, 5, 5, 6, 7, 10, 14, 15, 16, + 12, 11, 10, 10, 10, 10, 8, 7, 6, 6, 4, 3, 2, 1, 1, 1, 1, 2, 2, 3, 4, 4, 4, 4, 5, 5, 6, 8, 11, 15, 16, 16, + 12, 12, 11, 11, 10, 10, 9, 8, 7, 6, 6, 4, 4, 3, 2, 2, 2, 2, 3, 4, 4, 4, 5, 5, 5, 6, 7, 10, 13, 15, 16, 15, + 12, 12, 12, 12, 11, 10, 10, 8, 8, 7, 7, 6, 5, 5, 3, 2, 2, 2, 3, 4, 5, 5, 5, 5, 6, 7, 10, 13, 14, 16, 16, 16, + 12, 12, 13, 12, 12, 11, 11, 9, 8, 8, 8, 7, 6, 6, 5, 4, 3, 3, 3, 5, 6, 6, 6, 6, 7, 9, 11, 14, 15, 17, 17, 16, + 13, 13, 13, 13, 12, 12, 11, 11, 10, 10, 9, 8, 7, 7, 6, 5, 5, 4, 4, 4, 5, 6, 6, 6, 9, 10, 12, 14, 16, 17, 17, 18, + 13, 13, 14, 13, 13, 13, 12, 12, 11, 10, 10, 9, 8, 7, 6, 6, 6, 5, 4, 4, 6, 6, 6, 7, 9, 11, 12, 16, 17, 17, 17, 18, + 15, 15, 15, 15, 14, 13, 13, 13, 12, 11, 11, 10, 9, 9, 8, 7, 6, 5, 5, 5, 6, 7, 7, 8, 10, 12, 14, 17, 17, 18, 17, 17, + 16, 16, 16, 16, 15, 14, 14, 14, 13, 13, 12, 11, 11, 9, 8, 7, 7, 6, 6, 6, 7, 7, 8, 8, 10, 12, 16, 17, 18, 18, 17, 17, + 18, 17, 17, 16, 16, 15, 14, 14, 14, 14, 14, 13, 11, 11, 9, 8, 7, 7, 6, 6, 7, 7, 8, 9, 10, 13, 16, 17, 17, 18, 17, 16, + 18, 17, 17, 17, 16, 15, 15, 15, 15, 15, 15, 14, 14, 12, 10, 9, 7, 7, 6, 6, 7, 7, 8, 9, 11, 14, 16, 16, 17, 17, 16, 15, + 18, 18, 17, 17, 17, 16, 15, 15, 15, 16, 15, 15, 14, 14, 12, 9, 7, 6, 6, 6, 7, 7, 9, 10, 12, 14, 15, 16, 16, 16, 15, 15, + ] + - ct: 5500 + r: [ + 19, 18, 17, 16, 15, 13, 11, 10, 9, 9, 9, 8, 8, 8, 8, 8, 8, 9, 10, 10, 8, 8, 9, 9, 9, 11, 14, 15, 16, 16, 18, 18, + 18, 18, 17, 15, 14, 13, 11, 11, 9, 9, 8, 8, 7, 7, 7, 7, 7, 8, 9, 8, 8, 8, 9, 9, 8, 8, 11, 14, 16, 17, 17, 18, + 18, 17, 17, 15, 14, 13, 12, 11, 9, 9, 8, 7, 7, 6, 6, 6, 6, 8, 8, 8, 8, 8, 8, 8, 7, 7, 10, 13, 16, 17, 18, 18, + 17, 17, 16, 15, 14, 12, 11, 11, 9, 8, 7, 7, 6, 5, 5, 5, 5, 6, 8, 7, 8, 8, 8, 6, 6, 6, 9, 12, 16, 16, 18, 19, + 17, 17, 16, 14, 13, 12, 11, 10, 10, 8, 7, 7, 5, 5, 5, 5, 5, 5, 6, 7, 7, 7, 6, 6, 6, 6, 8, 11, 15, 16, 17, 19, + 18, 17, 16, 14, 13, 12, 11, 10, 10, 8, 7, 6, 5, 5, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 7, 10, 14, 16, 17, 18, + 18, 17, 16, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 4, 4, 4, 4, 4, 3, 4, 5, 5, 5, 5, 5, 6, 10, 13, 16, 16, 17, + 18, 16, 15, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 4, 4, 4, 4, 4, 3, 3, 3, 4, 4, 4, 4, 5, 6, 8, 12, 16, 16, 16, + 17, 16, 15, 13, 12, 11, 10, 9, 8, 7, 5, 4, 4, 4, 4, 4, 4, 4, 3, 3, 3, 3, 3, 3, 4, 4, 5, 7, 11, 16, 16, 16, + 17, 16, 15, 12, 12, 12, 10, 8, 7, 7, 4, 4, 4, 4, 4, 4, 4, 4, 3, 3, 3, 3, 3, 3, 3, 4, 5, 7, 10, 16, 16, 16, + 16, 16, 14, 12, 12, 11, 10, 8, 7, 6, 4, 4, 3, 3, 3, 3, 3, 4, 3, 3, 2, 2, 2, 3, 3, 4, 4, 7, 10, 14, 16, 16, + 16, 15, 14, 13, 12, 11, 10, 8, 7, 6, 5, 4, 3, 3, 3, 3, 3, 4, 3, 3, 3, 2, 2, 3, 3, 3, 5, 7, 10, 14, 15, 16, + 16, 15, 15, 13, 11, 11, 11, 9, 7, 5, 5, 4, 3, 2, 2, 2, 3, 3, 3, 3, 2, 2, 2, 2, 3, 3, 5, 6, 9, 14, 15, 16, + 16, 15, 15, 13, 11, 11, 11, 10, 7, 5, 4, 4, 2, 2, 2, 2, 2, 3, 3, 3, 2, 2, 2, 2, 3, 3, 5, 6, 10, 13, 15, 17, + 15, 15, 14, 12, 11, 11, 11, 10, 7, 4, 4, 3, 2, 2, 2, 2, 2, 2, 3, 3, 2, 2, 2, 3, 3, 3, 5, 6, 9, 13, 16, 17, + 15, 15, 14, 12, 11, 11, 10, 9, 7, 4, 4, 3, 1, 1, 1, 1, 2, 2, 3, 3, 2, 2, 3, 3, 3, 3, 5, 7, 10, 15, 17, 17, + 15, 15, 14, 12, 11, 11, 10, 9, 7, 4, 4, 3, 1, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 5, 7, 11, 16, 17, 18, + 16, 15, 15, 12, 12, 11, 10, 9, 6, 4, 4, 3, 1, 0, 0, 1, 1, 2, 2, 2, 2, 3, 3, 3, 4, 4, 5, 7, 12, 15, 17, 18, + 16, 16, 15, 12, 12, 11, 10, 8, 6, 5, 4, 3, 1, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 5, 6, 8, 12, 15, 17, 18, + 15, 15, 15, 13, 12, 12, 10, 8, 7, 5, 4, 3, 2, 0, 0, 0, 1, 1, 2, 2, 3, 3, 3, 3, 4, 5, 7, 9, 12, 16, 17, 18, + 15, 15, 15, 13, 13, 12, 10, 8, 7, 5, 5, 3, 2, 1, 0, 0, 1, 1, 1, 2, 3, 3, 3, 4, 5, 5, 7, 9, 13, 16, 18, 18, + 15, 15, 15, 14, 13, 12, 10, 8, 7, 6, 5, 5, 3, 2, 1, 0, 1, 1, 1, 2, 3, 3, 3, 5, 5, 6, 8, 10, 14, 17, 18, 19, + 16, 16, 16, 15, 13, 12, 10, 9, 8, 7, 6, 6, 5, 4, 2, 1, 1, 1, 1, 2, 3, 3, 5, 5, 6, 7, 9, 12, 15, 18, 18, 19, + 17, 16, 16, 16, 13, 12, 11, 10, 9, 8, 7, 7, 6, 5, 4, 2, 1, 1, 2, 3, 4, 4, 5, 5, 6, 8, 11, 14, 16, 18, 19, 19, + 18, 18, 17, 16, 14, 13, 12, 10, 9, 8, 8, 7, 7, 5, 5, 3, 2, 2, 4, 4, 5, 5, 5, 5, 7, 10, 12, 16, 17, 19, 19, 20, + 18, 18, 17, 16, 15, 13, 12, 10, 10, 9, 9, 9, 7, 6, 5, 4, 3, 3, 4, 4, 5, 5, 5, 6, 7, 11, 15, 17, 18, 19, 19, 20, + 19, 18, 18, 17, 16, 14, 13, 11, 10, 10, 9, 9, 8, 6, 5, 5, 4, 4, 4, 4, 6, 6, 6, 7, 9, 11, 15, 17, 19, 19, 20, 21, + 20, 19, 19, 19, 17, 16, 13, 12, 11, 11, 10, 9, 9, 7, 6, 5, 5, 4, 4, 5, 6, 7, 7, 8, 9, 12, 15, 18, 19, 19, 20, 20, + 21, 20, 20, 19, 19, 16, 16, 13, 12, 12, 11, 11, 9, 8, 7, 6, 6, 5, 5, 6, 7, 7, 8, 8, 10, 13, 17, 19, 19, 20, 20, 20, + 22, 21, 20, 20, 19, 17, 16, 14, 13, 13, 14, 12, 11, 9, 8, 7, 6, 5, 5, 6, 7, 7, 8, 9, 11, 15, 17, 19, 19, 20, 20, 20, + 22, 22, 21, 20, 19, 18, 16, 15, 14, 14, 15, 15, 13, 11, 9, 8, 7, 5, 5, 6, 7, 8, 9, 10, 13, 16, 18, 18, 19, 20, 19, 19, + 22, 22, 21, 20, 19, 19, 16, 16, 15, 15, 15, 15, 15, 13, 10, 9, 7, 5, 5, 6, 8, 8, 10, 11, 14, 16, 18, 18, 19, 19, 19, 19, + ] + g: [ + 16, 16, 15, 14, 13, 11, 10, 9, 8, 7, 7, 7, 6, 6, 6, 6, 6, 7, 8, 8, 6, 6, 7, 7, 7, 9, 12, 13, 13, 14, 14, 14, + 16, 16, 15, 14, 13, 11, 10, 9, 8, 7, 7, 6, 6, 6, 6, 6, 6, 6, 7, 7, 6, 6, 7, 7, 6, 6, 10, 12, 13, 14, 14, 14, + 16, 15, 15, 13, 13, 11, 10, 9, 8, 7, 7, 6, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 5, 5, 8, 12, 14, 15, 15, 16, + 15, 15, 14, 13, 12, 11, 10, 10, 8, 7, 6, 6, 5, 4, 4, 4, 4, 5, 6, 6, 6, 6, 6, 5, 5, 5, 7, 11, 14, 15, 15, 17, + 16, 15, 14, 13, 12, 11, 10, 10, 9, 7, 6, 6, 4, 4, 4, 3, 3, 4, 4, 6, 6, 5, 5, 4, 4, 5, 6, 10, 14, 14, 16, 16, + 16, 15, 15, 13, 12, 11, 10, 10, 9, 7, 6, 6, 4, 4, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 9, 13, 14, 15, 16, + 16, 15, 15, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 4, 3, 3, 3, 3, 3, 3, 3, 4, 3, 3, 3, 4, 5, 8, 13, 14, 15, 15, + 16, 15, 14, 12, 11, 10, 9, 8, 7, 6, 6, 5, 4, 3, 3, 3, 3, 3, 2, 2, 2, 3, 3, 3, 3, 4, 5, 7, 11, 14, 15, 15, + 16, 15, 14, 12, 11, 10, 9, 8, 7, 6, 5, 3, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 3, 3, 4, 6, 10, 14, 15, 14, + 16, 15, 14, 12, 11, 11, 9, 8, 7, 6, 4, 3, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 2, 3, 4, 6, 10, 15, 15, 15, + 15, 15, 13, 12, 11, 11, 10, 8, 7, 5, 4, 3, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 2, 3, 3, 6, 9, 13, 15, 15, + 15, 14, 13, 12, 11, 11, 10, 8, 7, 5, 5, 3, 2, 2, 2, 2, 3, 3, 3, 3, 2, 2, 2, 2, 2, 2, 4, 6, 9, 13, 14, 15, + 15, 14, 14, 13, 11, 11, 10, 8, 7, 5, 5, 3, 2, 2, 2, 2, 2, 3, 3, 3, 2, 2, 1, 2, 2, 2, 4, 5, 8, 13, 14, 15, + 15, 14, 14, 13, 11, 11, 11, 10, 7, 5, 4, 3, 2, 2, 2, 2, 2, 2, 3, 3, 2, 2, 1, 2, 2, 2, 4, 5, 8, 13, 14, 15, + 15, 14, 13, 12, 11, 11, 10, 9, 7, 4, 4, 3, 2, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 4, 5, 8, 13, 15, 16, + 15, 15, 13, 12, 11, 11, 10, 9, 7, 4, 4, 2, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 4, 6, 8, 15, 16, 16, + 15, 15, 14, 12, 11, 11, 10, 9, 7, 4, 4, 2, 1, 0, 1, 1, 1, 2, 2, 2, 2, 2, 2, 3, 3, 3, 4, 6, 11, 15, 16, 17, + 15, 15, 15, 12, 12, 11, 10, 10, 7, 4, 4, 2, 1, 0, 0, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 4, 4, 6, 11, 15, 17, 17, + 15, 15, 15, 12, 12, 12, 10, 9, 7, 5, 4, 2, 1, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 3, 4, 5, 7, 11, 15, 17, 17, + 15, 15, 15, 13, 12, 12, 10, 9, 7, 5, 4, 3, 2, 0, 0, 0, 1, 1, 1, 2, 3, 3, 3, 3, 4, 4, 6, 8, 12, 15, 17, 17, + 15, 15, 15, 13, 13, 12, 10, 9, 7, 6, 5, 3, 2, 1, 0, 0, 1, 1, 1, 2, 3, 3, 3, 3, 4, 5, 6, 9, 13, 16, 17, 18, + 15, 15, 15, 14, 13, 13, 10, 9, 8, 6, 6, 5, 3, 2, 1, 1, 1, 1, 2, 2, 3, 3, 4, 4, 5, 6, 7, 10, 14, 17, 18, 18, + 15, 16, 16, 15, 13, 13, 11, 9, 8, 7, 6, 6, 5, 4, 2, 1, 1, 1, 2, 2, 3, 4, 5, 5, 6, 7, 8, 12, 15, 17, 18, 18, + 16, 16, 16, 16, 13, 13, 11, 10, 9, 8, 7, 7, 6, 6, 4, 2, 2, 2, 2, 3, 5, 5, 5, 5, 6, 8, 11, 14, 16, 18, 18, 18, + 17, 17, 17, 16, 14, 13, 13, 11, 10, 9, 8, 8, 7, 6, 5, 4, 2, 2, 4, 5, 5, 5, 5, 5, 7, 9, 12, 15, 17, 18, 18, 18, + 18, 18, 17, 16, 15, 14, 13, 11, 10, 10, 9, 9, 7, 6, 5, 5, 4, 3, 4, 4, 5, 5, 5, 6, 7, 10, 15, 16, 18, 18, 18, 19, + 19, 18, 18, 17, 16, 14, 13, 12, 11, 10, 10, 9, 9, 7, 6, 5, 5, 4, 4, 4, 6, 6, 6, 6, 8, 10, 15, 16, 18, 18, 18, 19, + 20, 19, 19, 19, 17, 16, 14, 13, 12, 11, 11, 10, 9, 7, 7, 6, 5, 4, 4, 5, 6, 6, 6, 7, 9, 11, 15, 17, 18, 18, 18, 18, + 22, 20, 20, 20, 19, 17, 16, 14, 13, 13, 12, 11, 10, 9, 7, 6, 6, 5, 5, 6, 7, 7, 7, 8, 9, 12, 16, 18, 18, 19, 18, 18, + 22, 22, 21, 20, 19, 18, 17, 16, 14, 14, 15, 13, 11, 10, 9, 8, 7, 6, 5, 6, 7, 8, 8, 9, 10, 14, 17, 18, 18, 19, 18, 17, + 22, 22, 22, 21, 20, 19, 17, 17, 16, 16, 16, 16, 14, 12, 10, 8, 7, 6, 6, 7, 8, 8, 8, 10, 13, 16, 18, 18, 18, 18, 18, 17, + 22, 22, 22, 21, 20, 20, 18, 17, 16, 16, 17, 17, 16, 14, 11, 10, 8, 6, 6, 7, 8, 8, 10, 11, 14, 17, 18, 18, 18, 18, 18, 17, + ] + b: [ + 13, 12, 12, 12, 11, 9, 8, 7, 7, 7, 7, 6, 6, 6, 6, 5, 5, 6, 7, 7, 6, 6, 5, 5, 5, 6, 9, 9, 9, 10, 9, 8, + 13, 13, 12, 11, 11, 9, 9, 8, 7, 7, 7, 6, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 5, 5, 4, 4, 7, 9, 9, 10, 10, 9, + 13, 13, 12, 11, 11, 9, 9, 8, 7, 7, 6, 6, 5, 4, 4, 4, 4, 5, 6, 6, 6, 5, 5, 5, 3, 3, 6, 9, 10, 11, 10, 11, + 13, 13, 12, 11, 11, 10, 9, 9, 7, 7, 6, 5, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 4, 3, 3, 5, 8, 10, 11, 11, 12, + 13, 13, 12, 11, 11, 10, 9, 9, 8, 7, 6, 6, 4, 4, 3, 3, 3, 4, 4, 5, 5, 5, 4, 3, 2, 3, 4, 8, 11, 11, 11, 12, + 13, 13, 13, 11, 11, 10, 9, 9, 8, 7, 6, 6, 4, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 3, 7, 11, 11, 11, 12, + 14, 14, 13, 11, 11, 10, 9, 8, 8, 7, 6, 6, 4, 3, 3, 3, 3, 3, 2, 2, 2, 3, 2, 2, 2, 2, 3, 6, 11, 11, 11, 11, + 14, 14, 13, 11, 11, 10, 9, 8, 7, 7, 6, 5, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 2, 2, 2, 2, 4, 9, 11, 11, 11, + 14, 14, 13, 12, 11, 11, 9, 8, 7, 6, 5, 4, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 2, 2, 2, 2, 4, 8, 12, 12, 11, + 14, 13, 13, 12, 12, 11, 10, 8, 7, 6, 4, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 2, 2, 2, 2, 4, 8, 12, 12, 13, + 13, 13, 13, 12, 12, 12, 11, 9, 7, 6, 4, 3, 2, 2, 2, 2, 3, 3, 3, 2, 2, 2, 2, 2, 2, 2, 2, 4, 8, 12, 12, 13, + 14, 13, 13, 12, 12, 11, 11, 9, 7, 6, 5, 3, 2, 2, 2, 2, 2, 3, 3, 3, 2, 2, 2, 2, 2, 2, 3, 4, 8, 12, 13, 13, + 14, 13, 13, 13, 12, 11, 11, 9, 7, 6, 5, 4, 2, 1, 1, 1, 2, 3, 3, 3, 2, 2, 2, 2, 2, 2, 3, 4, 7, 12, 13, 13, + 14, 13, 13, 13, 12, 12, 12, 11, 7, 5, 5, 4, 2, 1, 1, 1, 1, 3, 3, 3, 2, 2, 2, 2, 2, 3, 3, 4, 7, 12, 13, 13, + 14, 14, 13, 13, 12, 12, 11, 11, 7, 5, 4, 3, 1, 1, 1, 1, 1, 2, 3, 3, 2, 2, 2, 3, 3, 3, 4, 4, 7, 12, 13, 14, + 14, 14, 13, 13, 12, 12, 11, 11, 8, 5, 4, 3, 1, 1, 0, 0, 1, 1, 3, 3, 3, 3, 3, 3, 3, 4, 4, 5, 8, 13, 15, 15, + 14, 15, 14, 13, 13, 12, 11, 11, 8, 5, 4, 3, 1, 0, 0, 0, 0, 1, 2, 2, 3, 3, 3, 4, 4, 4, 5, 6, 10, 15, 15, 16, + 15, 15, 15, 14, 13, 13, 12, 11, 8, 5, 4, 3, 1, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 4, 4, 5, 6, 11, 15, 16, 16, + 15, 15, 15, 14, 14, 14, 12, 11, 8, 6, 5, 3, 1, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 4, 5, 6, 7, 11, 15, 16, 16, + 15, 15, 15, 15, 14, 14, 12, 11, 9, 7, 5, 3, 2, 1, 0, 1, 1, 1, 2, 3, 3, 4, 4, 4, 4, 5, 6, 8, 12, 15, 16, 16, + 15, 16, 15, 15, 15, 14, 13, 11, 9, 7, 6, 5, 3, 2, 1, 1, 1, 1, 2, 3, 3, 4, 4, 4, 5, 6, 7, 9, 12, 16, 16, 16, + 15, 16, 16, 15, 15, 15, 13, 11, 10, 8, 7, 6, 4, 3, 2, 1, 1, 2, 2, 3, 4, 4, 5, 5, 6, 7, 8, 10, 14, 16, 16, 16, + 16, 16, 17, 16, 15, 15, 14, 12, 11, 9, 8, 8, 6, 5, 3, 2, 2, 2, 3, 3, 5, 5, 6, 6, 7, 8, 9, 12, 15, 16, 16, 16, + 16, 17, 18, 17, 16, 15, 14, 13, 11, 10, 9, 9, 8, 6, 5, 3, 3, 3, 3, 4, 6, 6, 6, 6, 7, 8, 11, 14, 16, 16, 16, 16, + 17, 18, 18, 18, 17, 16, 16, 14, 12, 11, 10, 9, 8, 7, 6, 5, 3, 3, 5, 6, 6, 6, 6, 6, 8, 10, 12, 16, 17, 17, 17, 16, + 18, 18, 18, 18, 18, 17, 16, 14, 13, 12, 11, 11, 8, 8, 6, 6, 5, 4, 5, 5, 6, 6, 6, 7, 8, 11, 15, 16, 17, 17, 16, 16, + 18, 19, 19, 19, 19, 17, 17, 15, 14, 13, 12, 11, 11, 8, 7, 6, 6, 5, 5, 5, 7, 7, 7, 8, 9, 11, 15, 17, 17, 17, 16, 16, + 20, 20, 20, 20, 19, 19, 17, 17, 15, 14, 14, 12, 11, 9, 8, 7, 7, 6, 6, 6, 8, 8, 8, 8, 9, 12, 15, 18, 18, 16, 16, 16, + 22, 22, 22, 22, 21, 20, 19, 18, 17, 16, 15, 14, 12, 11, 10, 8, 8, 7, 7, 7, 8, 8, 8, 9, 10, 13, 17, 18, 18, 16, 16, 15, + 23, 22, 22, 22, 22, 21, 20, 20, 18, 18, 19, 16, 14, 13, 11, 10, 9, 8, 7, 7, 8, 9, 9, 10, 11, 15, 17, 18, 17, 17, 16, 14, + 23, 23, 23, 23, 23, 22, 21, 21, 20, 20, 20, 19, 18, 15, 12, 11, 10, 8, 8, 8, 9, 9, 10, 11, 13, 17, 17, 17, 17, 16, 15, 13, + 23, 23, 24, 24, 23, 23, 22, 21, 21, 21, 20, 20, 19, 17, 14, 12, 11, 9, 8, 9, 9, 10, 10, 12, 15, 17, 17, 17, 16, 16, 15, 13, + ] + - ct: 8500 + r: [ + 18, 17, 16, 15, 13, 12, 10, 9, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 12, 12, 8, 8, 10, 13, 14, 15, 17, 18, + 17, 17, 16, 14, 12, 11, 10, 10, 8, 8, 8, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 7, 7, 10, 13, 14, 15, 17, 17, + 17, 16, 15, 13, 12, 11, 10, 10, 8, 8, 7, 7, 7, 6, 7, 7, 8, 8, 8, 7, 7, 7, 7, 7, 6, 6, 9, 12, 14, 15, 17, 17, + 16, 16, 15, 13, 12, 11, 10, 10, 9, 7, 7, 7, 6, 6, 5, 5, 6, 6, 7, 7, 7, 7, 7, 5, 5, 5, 8, 11, 14, 15, 17, 19, + 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 7, 6, 6, 5, 5, 5, 5, 5, 5, 6, 6, 6, 5, 5, 5, 5, 7, 10, 13, 15, 17, 19, + 15, 15, 14, 13, 12, 11, 9, 9, 8, 7, 6, 5, 5, 5, 4, 4, 5, 5, 5, 4, 4, 4, 4, 4, 4, 5, 6, 9, 13, 15, 16, 17, + 15, 15, 13, 12, 11, 11, 9, 8, 8, 6, 5, 5, 4, 4, 4, 4, 4, 4, 4, 3, 3, 4, 4, 4, 4, 4, 5, 8, 13, 15, 15, 16, + 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 4, 4, 4, 4, 4, 4, 4, 3, 3, 3, 3, 3, 3, 4, 5, 7, 11, 14, 15, 15, + 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 4, 4, 4, 3, 3, 4, 3, 3, 3, 2, 3, 3, 3, 3, 4, 6, 10, 14, 15, 15, + 15, 13, 13, 11, 11, 10, 9, 8, 7, 6, 4, 4, 4, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 3, 4, 6, 10, 13, 16, 16, + 14, 13, 12, 11, 10, 10, 9, 7, 7, 5, 4, 3, 3, 3, 3, 2, 2, 3, 3, 3, 2, 2, 2, 2, 2, 3, 4, 6, 9, 13, 16, 17, + 14, 13, 12, 11, 10, 10, 9, 7, 6, 5, 4, 3, 3, 3, 2, 2, 2, 3, 3, 3, 2, 2, 1, 2, 2, 3, 4, 6, 9, 13, 16, 17, + 14, 13, 12, 12, 10, 10, 9, 7, 5, 5, 4, 3, 3, 2, 2, 2, 2, 3, 3, 3, 2, 2, 1, 1, 2, 3, 4, 6, 9, 13, 15, 17, + 14, 13, 12, 11, 10, 9, 9, 7, 5, 5, 4, 3, 2, 2, 2, 2, 2, 3, 3, 3, 2, 2, 1, 1, 2, 3, 4, 6, 9, 12, 15, 16, + 13, 13, 12, 11, 10, 9, 8, 7, 5, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 2, 2, 3, 4, 6, 8, 12, 15, 16, + 13, 13, 12, 11, 11, 9, 8, 7, 6, 4, 3, 2, 1, 1, 1, 1, 2, 2, 2, 2, 2, 1, 1, 2, 3, 3, 4, 6, 9, 15, 16, 16, + 13, 13, 12, 12, 11, 10, 9, 7, 6, 4, 3, 2, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 4, 4, 6, 10, 15, 17, 18, + 14, 13, 13, 12, 12, 11, 9, 7, 6, 4, 3, 2, 1, 0, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 3, 4, 5, 7, 11, 15, 17, 18, + 14, 13, 13, 12, 12, 11, 9, 7, 7, 4, 3, 1, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 4, 4, 5, 7, 11, 15, 17, 18, + 15, 13, 13, 12, 12, 11, 8, 7, 6, 4, 3, 1, 0, 0, 0, 0, 1, 1, 1, 2, 2, 2, 2, 3, 4, 5, 6, 7, 11, 15, 17, 18, + 15, 14, 13, 13, 12, 11, 8, 8, 7, 5, 4, 2, 1, 0, 0, 0, 0, 1, 2, 2, 2, 2, 3, 3, 5, 5, 7, 8, 11, 15, 17, 18, + 15, 14, 14, 13, 12, 11, 9, 8, 7, 6, 5, 3, 2, 1, 0, 0, 0, 1, 2, 2, 3, 3, 3, 5, 5, 6, 8, 9, 13, 16, 18, 18, + 15, 14, 14, 13, 12, 11, 9, 9, 8, 7, 6, 5, 3, 3, 1, 0, 0, 1, 2, 2, 3, 4, 4, 5, 5, 6, 8, 11, 14, 17, 18, 18, + 16, 16, 15, 14, 13, 12, 10, 9, 8, 8, 7, 6, 5, 5, 3, 1, 1, 1, 2, 3, 4, 4, 5, 5, 6, 8, 10, 13, 15, 17, 18, 18, + 16, 16, 15, 14, 14, 12, 11, 10, 9, 9, 8, 7, 6, 5, 4, 3, 1, 1, 2, 4, 4, 5, 5, 5, 7, 9, 11, 14, 16, 18, 18, 19, + 16, 16, 15, 15, 14, 13, 12, 11, 10, 10, 9, 8, 7, 6, 5, 4, 3, 2, 3, 3, 4, 5, 5, 6, 9, 10, 13, 15, 18, 18, 19, 19, + 17, 17, 16, 15, 15, 14, 12, 11, 11, 10, 10, 9, 8, 7, 6, 5, 4, 3, 3, 3, 5, 5, 6, 6, 9, 11, 13, 17, 18, 19, 19, 20, + 19, 18, 17, 17, 15, 15, 13, 12, 11, 11, 10, 10, 9, 8, 7, 5, 5, 4, 4, 5, 6, 6, 6, 7, 10, 12, 15, 18, 19, 19, 19, 20, + 19, 19, 18, 17, 17, 16, 14, 13, 12, 12, 11, 10, 9, 8, 7, 6, 6, 5, 5, 6, 6, 6, 7, 8, 11, 12, 16, 18, 19, 19, 19, 19, + 20, 19, 19, 18, 17, 16, 15, 14, 13, 13, 12, 12, 10, 9, 8, 7, 6, 5, 5, 6, 6, 7, 8, 9, 11, 14, 17, 18, 19, 19, 19, 19, + 20, 20, 19, 18, 18, 17, 15, 15, 14, 14, 14, 13, 12, 11, 9, 7, 6, 5, 5, 6, 7, 7, 8, 9, 12, 15, 17, 18, 18, 19, 18, 18, + 21, 20, 20, 19, 18, 18, 15, 15, 14, 14, 14, 14, 13, 13, 11, 9, 6, 5, 5, 6, 7, 7, 9, 10, 13, 15, 18, 18, 18, 18, 18, 18, + ] + g: [ + 16, 16, 15, 13, 12, 10, 9, 8, 7, 7, 7, 6, 6, 6, 7, 7, 7, 6, 6, 6, 6, 6, 11, 11, 6, 6, 9, 12, 12, 13, 15, 15, + 16, 15, 14, 13, 11, 10, 9, 9, 8, 7, 7, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 5, 5, 9, 12, 12, 13, 15, 15, + 15, 15, 14, 12, 11, 11, 9, 9, 8, 7, 6, 6, 6, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 5, 4, 5, 8, 11, 13, 14, 15, 15, + 15, 15, 14, 12, 11, 10, 9, 9, 8, 6, 6, 6, 5, 5, 4, 4, 4, 4, 6, 6, 6, 6, 6, 4, 4, 4, 7, 11, 13, 14, 15, 17, + 15, 15, 13, 12, 11, 10, 9, 9, 8, 6, 6, 5, 5, 4, 4, 3, 3, 4, 4, 5, 5, 5, 4, 4, 3, 4, 6, 9, 13, 14, 15, 17, + 15, 14, 13, 12, 11, 10, 9, 8, 8, 6, 5, 5, 4, 4, 3, 3, 3, 3, 3, 3, 3, 4, 3, 3, 3, 4, 5, 8, 13, 14, 14, 15, + 14, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 4, 3, 3, 3, 3, 3, 3, 3, 2, 3, 3, 3, 3, 3, 4, 7, 12, 14, 14, 14, + 14, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 3, 3, 4, 6, 11, 14, 14, 14, + 14, 13, 12, 11, 11, 10, 9, 8, 7, 6, 4, 4, 3, 3, 3, 2, 3, 3, 3, 3, 2, 2, 2, 2, 2, 3, 3, 5, 10, 14, 14, 14, + 15, 13, 12, 11, 11, 10, 9, 8, 7, 6, 4, 4, 3, 3, 2, 2, 2, 3, 3, 3, 2, 2, 2, 2, 2, 2, 3, 5, 10, 13, 15, 16, + 14, 13, 12, 11, 11, 10, 9, 7, 7, 5, 4, 3, 2, 2, 2, 2, 2, 2, 3, 2, 2, 1, 2, 2, 2, 2, 3, 6, 9, 13, 16, 16, + 14, 13, 12, 11, 10, 10, 9, 7, 5, 5, 4, 3, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 2, 2, 2, 4, 6, 8, 13, 16, 16, + 14, 13, 13, 12, 10, 10, 9, 7, 5, 5, 4, 3, 2, 2, 1, 1, 1, 2, 2, 2, 2, 1, 1, 1, 2, 2, 4, 5, 8, 13, 15, 16, + 14, 13, 13, 12, 10, 10, 9, 7, 5, 5, 4, 3, 2, 1, 1, 1, 1, 2, 2, 2, 2, 1, 1, 1, 1, 2, 3, 5, 8, 12, 14, 15, + 14, 13, 12, 11, 10, 10, 9, 7, 6, 4, 3, 2, 1, 1, 1, 1, 1, 2, 2, 2, 2, 1, 1, 1, 2, 3, 3, 5, 8, 12, 15, 15, + 14, 13, 12, 11, 11, 10, 9, 7, 6, 4, 3, 2, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1, 1, 2, 2, 3, 4, 5, 8, 14, 16, 16, + 14, 13, 13, 12, 12, 10, 9, 7, 6, 4, 3, 2, 1, 0, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 3, 3, 4, 5, 10, 15, 16, 17, + 14, 13, 13, 13, 12, 12, 9, 7, 7, 4, 3, 2, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 4, 6, 11, 15, 17, 17, + 14, 14, 13, 12, 12, 12, 9, 8, 7, 5, 3, 2, 0, 0, 0, 0, 1, 1, 1, 2, 2, 2, 2, 3, 3, 4, 5, 7, 11, 15, 17, 17, + 15, 14, 13, 13, 12, 12, 9, 8, 7, 5, 3, 2, 0, 0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 4, 5, 5, 7, 11, 15, 17, 17, + 16, 14, 14, 13, 13, 12, 9, 8, 7, 5, 4, 2, 1, 0, 0, 0, 0, 1, 2, 2, 3, 3, 3, 3, 5, 5, 6, 8, 12, 16, 17, 17, + 16, 15, 14, 14, 13, 12, 10, 9, 7, 6, 5, 4, 2, 1, 0, 0, 0, 1, 2, 3, 3, 3, 3, 5, 5, 6, 8, 9, 13, 17, 17, 17, + 16, 15, 15, 14, 13, 12, 10, 10, 8, 7, 6, 5, 4, 3, 2, 0, 1, 1, 2, 3, 3, 4, 5, 5, 5, 6, 8, 11, 14, 17, 17, 17, + 16, 16, 15, 15, 14, 13, 11, 10, 9, 8, 7, 6, 5, 5, 3, 2, 1, 1, 2, 3, 5, 5, 5, 5, 6, 7, 10, 13, 16, 17, 17, 17, + 17, 17, 15, 15, 14, 13, 12, 11, 10, 9, 9, 7, 6, 6, 5, 3, 2, 2, 3, 4, 5, 5, 5, 6, 6, 9, 11, 14, 16, 18, 18, 18, + 17, 17, 16, 15, 15, 14, 13, 12, 11, 10, 10, 9, 7, 7, 5, 4, 3, 2, 3, 4, 5, 5, 5, 6, 9, 10, 13, 15, 18, 18, 18, 18, + 18, 17, 17, 16, 15, 15, 14, 12, 11, 11, 11, 9, 8, 7, 6, 5, 4, 3, 3, 4, 5, 5, 6, 6, 9, 11, 13, 17, 18, 18, 18, 18, + 20, 19, 18, 18, 16, 16, 14, 13, 12, 11, 11, 10, 9, 8, 7, 5, 5, 4, 4, 5, 6, 6, 6, 7, 10, 11, 15, 18, 18, 18, 18, 18, + 20, 20, 19, 18, 18, 17, 15, 14, 13, 13, 12, 11, 10, 9, 8, 6, 6, 5, 5, 6, 6, 7, 7, 8, 11, 12, 16, 18, 18, 18, 18, 18, + 22, 21, 20, 19, 18, 17, 17, 17, 15, 15, 14, 13, 11, 10, 8, 7, 6, 6, 6, 6, 7, 7, 8, 8, 11, 14, 17, 18, 18, 18, 18, 17, + 22, 22, 21, 20, 19, 18, 17, 17, 17, 16, 16, 15, 14, 12, 10, 8, 7, 6, 6, 7, 7, 8, 8, 10, 13, 16, 18, 18, 18, 18, 18, 16, + 22, 22, 22, 21, 20, 19, 18, 17, 17, 17, 16, 16, 15, 15, 12, 10, 7, 6, 6, 7, 8, 8, 9, 11, 14, 16, 18, 18, 18, 18, 17, 16, + ] + b: [ + 13, 13, 13, 11, 10, 9, 9, 8, 7, 7, 7, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 9, 9, 4, 4, 7, 9, 9, 9, 10, 10, + 13, 13, 12, 11, 10, 9, 9, 8, 7, 7, 7, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 3, 3, 7, 9, 9, 10, 10, 10, + 13, 13, 12, 11, 10, 10, 9, 9, 7, 7, 6, 6, 6, 5, 5, 5, 5, 6, 6, 6, 5, 5, 5, 4, 3, 3, 6, 9, 10, 11, 11, 11, + 13, 13, 12, 11, 10, 10, 9, 9, 8, 7, 6, 6, 5, 5, 4, 4, 4, 4, 5, 5, 5, 5, 5, 3, 2, 3, 5, 9, 10, 11, 11, 13, + 13, 13, 12, 11, 11, 10, 9, 9, 8, 7, 6, 5, 5, 4, 4, 3, 3, 4, 4, 5, 5, 5, 3, 2, 2, 3, 5, 8, 11, 11, 12, 13, + 13, 12, 12, 11, 11, 10, 9, 8, 8, 7, 5, 5, 4, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 3, 7, 11, 11, 11, 11, + 13, 13, 12, 11, 11, 10, 9, 8, 8, 7, 5, 4, 4, 3, 3, 3, 3, 3, 3, 2, 2, 3, 2, 2, 2, 2, 3, 6, 11, 11, 11, 11, + 13, 13, 12, 11, 11, 11, 9, 8, 7, 6, 5, 4, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 2, 2, 2, 5, 10, 12, 12, 11, + 13, 13, 13, 11, 11, 11, 10, 8, 7, 6, 5, 4, 3, 3, 3, 2, 2, 3, 3, 2, 2, 2, 2, 2, 2, 2, 2, 4, 9, 12, 12, 12, + 14, 13, 13, 12, 11, 11, 10, 8, 7, 6, 4, 4, 3, 2, 2, 2, 2, 3, 3, 2, 2, 2, 2, 2, 2, 2, 2, 4, 9, 12, 13, 14, + 13, 13, 12, 12, 11, 11, 10, 8, 7, 6, 4, 3, 2, 2, 2, 1, 2, 3, 3, 2, 2, 1, 2, 2, 2, 2, 2, 4, 8, 12, 14, 14, + 13, 13, 12, 12, 11, 11, 11, 8, 7, 6, 5, 3, 2, 2, 1, 1, 2, 3, 3, 2, 2, 2, 2, 2, 2, 2, 3, 4, 8, 12, 14, 14, + 13, 13, 12, 12, 12, 11, 11, 8, 7, 6, 5, 3, 2, 1, 1, 1, 1, 2, 3, 3, 2, 2, 2, 2, 2, 3, 3, 4, 8, 12, 14, 14, + 13, 13, 13, 12, 12, 11, 11, 8, 6, 6, 5, 3, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 4, 7, 12, 13, 14, + 13, 13, 13, 12, 12, 11, 10, 8, 6, 5, 4, 2, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 3, 3, 4, 5, 7, 12, 14, 14, + 14, 14, 13, 13, 13, 11, 10, 8, 7, 5, 4, 2, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 4, 4, 5, 8, 14, 16, 16, + 14, 14, 13, 13, 13, 12, 11, 9, 7, 5, 4, 2, 1, 0, 0, 0, 1, 1, 2, 2, 2, 2, 3, 3, 4, 4, 4, 6, 10, 15, 16, 16, + 15, 14, 14, 14, 15, 15, 11, 9, 8, 5, 4, 2, 1, 0, 0, 0, 1, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 7, 11, 15, 16, 17, + 15, 15, 14, 14, 15, 15, 12, 10, 9, 6, 4, 2, 1, 0, 0, 0, 1, 1, 1, 2, 3, 3, 3, 4, 5, 5, 6, 7, 11, 15, 16, 16, + 15, 15, 15, 15, 15, 15, 12, 10, 9, 6, 5, 2, 1, 0, 0, 0, 1, 1, 1, 3, 3, 3, 4, 4, 5, 6, 6, 7, 11, 15, 16, 17, + 16, 16, 15, 16, 15, 15, 12, 11, 10, 7, 5, 3, 2, 1, 0, 0, 1, 1, 2, 3, 4, 4, 4, 5, 6, 6, 7, 9, 12, 16, 16, 16, + 16, 16, 16, 16, 16, 15, 13, 12, 10, 9, 6, 5, 3, 2, 1, 1, 1, 2, 3, 4, 4, 4, 5, 6, 6, 7, 8, 9, 14, 16, 17, 16, + 16, 16, 16, 16, 16, 15, 14, 12, 11, 10, 8, 6, 5, 4, 2, 1, 1, 2, 3, 4, 5, 5, 6, 6, 6, 7, 9, 12, 15, 16, 16, 16, + 17, 17, 18, 17, 16, 16, 14, 13, 12, 11, 9, 8, 6, 5, 4, 2, 2, 2, 3, 4, 6, 6, 6, 6, 7, 8, 11, 14, 16, 17, 16, 16, + 17, 17, 18, 18, 17, 16, 16, 14, 13, 12, 11, 9, 8, 7, 5, 4, 2, 3, 4, 6, 6, 6, 6, 7, 8, 10, 12, 15, 17, 17, 17, 16, + 18, 18, 18, 18, 18, 17, 16, 15, 14, 13, 12, 11, 9, 8, 6, 5, 4, 4, 4, 5, 6, 6, 7, 7, 10, 11, 14, 16, 17, 17, 17, 17, + 18, 18, 19, 19, 19, 18, 17, 16, 15, 14, 14, 11, 10, 9, 7, 6, 6, 5, 5, 5, 6, 7, 7, 7, 10, 12, 14, 17, 17, 17, 17, 17, + 20, 20, 20, 20, 20, 19, 18, 17, 16, 15, 14, 13, 11, 10, 9, 7, 6, 6, 6, 6, 7, 7, 7, 8, 11, 12, 15, 18, 18, 17, 17, 16, + 22, 21, 21, 21, 21, 21, 20, 19, 17, 16, 16, 14, 13, 11, 10, 8, 7, 7, 7, 7, 8, 8, 8, 9, 11, 13, 17, 18, 18, 17, 16, 15, + 23, 22, 22, 22, 22, 21, 21, 20, 19, 19, 18, 16, 14, 13, 11, 9, 8, 8, 8, 8, 8, 8, 9, 10, 12, 15, 18, 18, 18, 17, 16, 14, + 23, 24, 24, 23, 23, 22, 22, 21, 21, 20, 20, 19, 18, 15, 13, 11, 9, 8, 8, 8, 9, 9, 10, 11, 13, 16, 18, 18, 17, 17, 15, 14, + 24, 24, 24, 24, 24, 23, 22, 22, 22, 22, 20, 20, 19, 18, 15, 13, 10, 9, 9, 9, 9, 10, 11, 12, 15, 17, 18, 18, 17, 16, 15, 13, + ] +... diff --git a/src/ipa/mali-c55/data/meson.build b/src/ipa/mali-c55/data/meson.build index 88109d17..8a5fdd36 100644 --- a/src/ipa/mali-c55/data/meson.build +++ b/src/ipa/mali-c55/data/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 conf_files = files([ + 'imx415.yaml', 'uncalibrated.yaml' ])