From patchwork Thu Jun 13 13:26:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20296 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 778D7C32CF for ; Thu, 13 Jun 2024 13:26:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E03EC654A8; Thu, 13 Jun 2024 15:26:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="MqH4aDgr"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7D2D0654A1 for ; Thu, 13 Jun 2024 15:26:31 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 44D46114D; Thu, 13 Jun 2024 15:26:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1718285177; bh=SW/Bc71D7wO94TAos07eWlZ7nOGhHqcoh8lJ+reYuS0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MqH4aDgrMjK42ZdAeGLzCKiWK5v+sC1KVwNsAtLgd7ooRjAkpLZqrbTTfphiwk6f0 cdbrOwFxHvKRQNdq3y0NEJFcK+gCS3oMCL6gUL5wmiRsl2fIIm3L220eHAqPGW/ewC yl1HPQk64PEN6Mg2sYdF9X+2X+JH73LKxqpx8Ak4= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: dan.scally@ideasonboard.com, nayden.kanchev@arm.com, jacopo.mondi@ideasonboard.com Subject: [PATCH 09/10] ipa: mali-c55: Add Lens Shading Correction algorithm Date: Thu, 13 Jun 2024 14:26:01 +0100 Message-Id: <20240613132602.1021721-10-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240613132602.1021721-1-dan.scally@ideasonboard.com> References: <20240613132602.1021721-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a lens shading correction algorithm to the mali-c55 IPA. This algorithm parses tables from Yaml in a easy to follow format before munging them into Arm's interleaved mesh to be copied to the ISP. A colour temperature estimate from the AGC statistics is used to select the appropriate table to apply; this can be some interpolation of two tables, in which case the colour temperature estimate is also used to derive the coefficient that does the blending. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- src/ipa/mali-c55/algorithms/lsc.cpp | 218 ++++++++++++++++++++++++ src/ipa/mali-c55/algorithms/lsc.h | 45 +++++ src/ipa/mali-c55/algorithms/meson.build | 1 + 3 files changed, 264 insertions(+) create mode 100644 src/ipa/mali-c55/algorithms/lsc.cpp create mode 100644 src/ipa/mali-c55/algorithms/lsc.h diff --git a/src/ipa/mali-c55/algorithms/lsc.cpp b/src/ipa/mali-c55/algorithms/lsc.cpp new file mode 100644 index 00000000..4b4a4c6e --- /dev/null +++ b/src/ipa/mali-c55/algorithms/lsc.cpp @@ -0,0 +1,218 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas On Board Oy + * + * lsc.cpp - Mali-C55 Lens shading correction algorithm + */ + +#include "lsc.h" + +#include "libcamera/internal/yaml_parser.h" + +namespace libcamera { + +namespace ipa::mali_c55::algorithms { + +LOG_DEFINE_CATEGORY(MaliC55Lsc) + +Lsc::Lsc() +{ +} + +int Lsc::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) +{ + if (!tuningData.contains("meshScale")) { + LOG(MaliC55Lsc, Error) << "meshScale missing from tuningData"; + return -EINVAL; + } + + meshScale_ = tuningData["meshScale"].get(0); + + const YamlObject &yamlSets = tuningData["sets"]; + if (!yamlSets.isList()) { + LOG(MaliC55Lsc, Error) << "LSC tables missing or invalid"; + return -EINVAL; + } + + size_t tableSize = 0; + const auto &sets = yamlSets.asList(); + for (const auto &yamlSet : sets) { + uint32_t ct = yamlSet["ct"].get(0); + + if (!ct) { + LOG(MaliC55Lsc, Error) << "Invalid colour temperature"; + return -EINVAL; + } + + if (std::count(colourTemperatures_.begin(), + colourTemperatures_.end(), ct)) { + LOG(MaliC55Lsc, Error) + << "Multiple sets found for colour temperature"; + return -EINVAL; + } + + std::vector rTable = + yamlSet["r"].getList().value_or(std::vector{}); + std::vector gTable = + yamlSet["g"].getList().value_or(std::vector{}); + std::vector bTable = + yamlSet["b"].getList().value_or(std::vector{}); + + /* + * Some validation to do; only 16x16 and 32x32 tables of + * coefficients are acceptable, and all tables across all of the + * sets must be the same size. The first time we encounter a + * table we check that it is an acceptable size and if so make + * sure all other tables are of equal size. + */ + if (!tableSize) { + if (rTable.size() != 256 && rTable.size() != 1024) { + LOG(MaliC55Lsc, Error) + << "Invalid table size for colour temperature " << ct; + return -EINVAL; + } + tableSize = rTable.size(); + } + + if (rTable.size() != tableSize || + gTable.size() != tableSize || + bTable.size() != tableSize) { + LOG(MaliC55Lsc, Error) + << "Invalid or mismatched table size for colour temperature " << ct; + return -EINVAL; + } + + if (colourTemperatures_.size() >= 3) { + LOG(MaliC55Lsc, Error) + << "A maximum of 3 colour temperatures are supported"; + return -EINVAL; + } + + for (unsigned int i = 0; i < tableSize; i++) { + mesh_[kRedOffset + i] |= + (rTable[i] << (colourTemperatures_.size() * 8)); + mesh_[kGreenOffset + i] |= + (gTable[i] << (colourTemperatures_.size() * 8)); + mesh_[kBlueOffset + i] |= + (bTable[i] << (colourTemperatures_.size() * 8)); + } + + colourTemperatures_.push_back(ct); + } + + if (tableSize == 256) + meshSize_ = 15; + else + meshSize_ = 31; + + return 0; +} + +void Lsc::fillConfigParamsBlock(mali_c55_params_block_header *block) const +{ + struct mali_c55_params_mesh_shading_config *config = + reinterpret_cast(block); + + config->header.type = MALI_C55_PARAM_MESH_SHADING_CONFIG; + config->header.enabled = true; + config->header.size = sizeof(struct mali_c55_params_mesh_shading_config); + + config->mesh_show = false; + config->mesh_scale = meshScale_; + config->mesh_page_r = 0; + config->mesh_page_g = 1; + config->mesh_page_b = 2; + config->mesh_width = meshSize_; + config->mesh_height = meshSize_; + + std::copy(mesh_.begin(), mesh_.end(), config->mesh); +} + +void Lsc::fillSelectionParamsBlock(mali_c55_params_block_header *block, + uint8_t bank, uint8_t alpha) const +{ + struct mali_c55_params_mesh_shading_selection *config = + reinterpret_cast(block); + + config->header.type = MALI_C55_PARAM_MESH_SHADING_SELECTION; + config->header.enabled = true; + config->header.size = sizeof(struct mali_c55_params_mesh_shading_selection); + + config->mesh_alpha_bank_r = bank; + config->mesh_alpha_bank_g = bank; + config->mesh_alpha_bank_b = bank; + config->mesh_alpha_r = alpha; + config->mesh_alpha_g = alpha; + config->mesh_alpha_b = alpha; + config->mesh_strength = 0x1000; /* Otherwise known as 1.0 */ +} + +std::tuple Lsc::findBankAndAlpha(uint32_t ct) const +{ + unsigned int i; + + ct = std::clamp(ct, colourTemperatures_.front(), + colourTemperatures_.back()); + + for (i = 0; i < colourTemperatures_.size() - 1; i++) { + if (ct >= colourTemperatures_[i] && + ct <= colourTemperatures_[i + 1]) + break; + } + + /* + * With the clamping, we're guaranteed an index into colourTemperatures_ + * that's <= colourTemperatures_.size() - 1. + */ + uint8_t alpha = (255 * (ct - colourTemperatures_[i])) / + (colourTemperatures_[i + 1] - colourTemperatures_[i]); + + return { i, alpha }; +} + +void Lsc::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + mali_c55_params_block_header *block) +{ + /* + * For each frame we assess the colour temperature of the **last** frame + * and then select an appropriately blended table of coefficients based + * on that ct. As a bit of a shortcut, if we've only a single table the + * handling is somewhat simpler; if it's the first frame we just select + * that table and if we're past the first frame then we can just do + * nothing - the config will never change. + */ + uint32_t temperatureK = context.activeState.agc.temperatureK; + uint8_t bank, alpha; + + if (colourTemperatures_.size() == 1) { + if (frame > 0) + return; + + bank = 0; + alpha = 0; + } else { + std::tie(bank, alpha) = findBankAndAlpha(temperatureK); + } + + fillSelectionParamsBlock(block, bank, alpha); + + if (frame > 0) + return; + + char *params = reinterpret_cast(block); + block = reinterpret_cast + (params + sizeof(struct mali_c55_params_mesh_shading_selection)); + + /* + * If this is the first frame, we need to load the parsed coefficient + * tables from tuning data to the ISP. + */ + fillConfigParamsBlock(block); +} + +REGISTER_IPA_ALGORITHM(Lsc, "Lsc") + +} /* namespace ipa::mali_c55::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/mali-c55/algorithms/lsc.h b/src/ipa/mali-c55/algorithms/lsc.h new file mode 100644 index 00000000..9a47c5ab --- /dev/null +++ b/src/ipa/mali-c55/algorithms/lsc.h @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas On Board Oy + * + * lsc.h - Mali-C55 Lens shading correction algorithm + */ + +#include +#include + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::mali_c55::algorithms { + +class Lsc : public Algorithm +{ +public: + Lsc(); + ~Lsc() = default; + + int init(IPAContext &context, const YamlObject &tuningData) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + mali_c55_params_block_header *block) override; +private: + static constexpr unsigned int kRedOffset = 0; + static constexpr unsigned int kGreenOffset = 1024; + static constexpr unsigned int kBlueOffset = 2048; + + void fillConfigParamsBlock(mali_c55_params_block_header *block) const; + void fillSelectionParamsBlock(mali_c55_params_block_header *block, + uint8_t bank, uint8_t alpha) const; + std::tuple findBankAndAlpha(uint32_t ct) const; + + std::vector mesh_ = std::vector(3072); + std::vector colourTemperatures_; + uint32_t meshScale_; + uint32_t meshSize_; +}; + +} /* namespace ipa::mali_c55::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/mali-c55/algorithms/meson.build b/src/ipa/mali-c55/algorithms/meson.build index f11791aa..1665da07 100644 --- a/src/ipa/mali-c55/algorithms/meson.build +++ b/src/ipa/mali-c55/algorithms/meson.build @@ -4,4 +4,5 @@ mali_c55_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', 'blc.cpp', + 'lsc.cpp', ])