[{"id":37985,"web_url":"https://patchwork.libcamera.org/comment/37985/","msgid":"<20260128111247.GC2558360@killaraus>","date":"2026-01-28T11:12:47","subject":"Re: [PATCH v5 13/15] ipa: rkisp1: lsc: Resample polynomial lens\n\tshading tables at configure time","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Tue, Jan 20, 2026 at 01:26:18PM +0100, Stefan Klug wrote:\n> The lens shading correction is always applied based on the sensor crop\n> bounds. This leads to incorrect lens shading correction for analog crops\n> that do not cover the whole sensor.\n> \n> To fix that, we need to adapt the lens shading table for the selected\n> analog crop at configure time. Introduce an abstract ShadingDescriptor\n> class that holds the lens shading information that can then be sampled\n> at configure time for a specific crop rectangle.\n> \n> Resampling for a specific crop is only implemented for polynomial lsc\n> data. For tabular data, a warning is logged and the unmodified table is\n> returned. This matches the current functionality for tabular data and is\n> a huge improvement for polynomial data.\n> \n> Signed-off-by: Stefan Klug \n> Reviewed-by: Kieran Bingham \n> Reviewed-by: Barnabás Pőcze \n> \n> ---\n> \n> Changes in v4:\n> - Removed trailing dot from commit message\n> - Added missing 'override'\n> - Collected tags\n> \n> Changes in v2:\n> - Replaced \"auto const\" by \"const auto\"\n> - Replaces some vector parameters by Spans\n> - Changed sampleForCrop to return the components\n> - Replaced min/max by clamp\n> - Replaced map.swap() by move assignment\n> ---\n> src/ipa/rkisp1/algorithms/lsc.cpp | 233 ++++++++++++++++++++++++--------------\n> src/ipa/rkisp1/algorithms/lsc.h | 13 +++\n> 2 files changed, 159 insertions(+), 87 deletions(-)\n> \n> diff --git a/src/ipa/rkisp1/algorithms/lsc.cpp b/src/ipa/rkisp1/algorithms/lsc.cpp\n> index f51c690c2020c8a046fbab8f1da31b9ba4a32ac0..8589243ea04f5a712fb2ddd95677a197566f390f 100644\n> --- a/src/ipa/rkisp1/algorithms/lsc.cpp\n> +++ b/src/ipa/rkisp1/algorithms/lsc.cpp\n> @@ -72,38 +72,127 @@ namespace {\n> \n> constexpr int kColourTemperatureQuantization = 10;\n> \n> -class LscPolynomialLoader\n> +class LscPolynomialShadingDescriptor : public LensShadingCorrection::ShadingDescriptor\n> {\n> public:\n> -\tLscPolynomialLoader(const Size &sensorSize,\n> -\t\t\t const Rectangle &cropRectangle,\n> -\t\t\t const std::vector &xSizes,\n> -\t\t\t const std::vector &ySizes)\n> -\t\t: sensorSize_(sensorSize),\n> -\t\t cropRectangle_(cropRectangle),\n> -\t\t xSizes_(xSizes),\n> -\t\t ySizes_(ySizes)\n> +\tLscPolynomialShadingDescriptor(const LscPolynomial &pr, const LscPolynomial &pgr,\n> +\t\t\t\t const LscPolynomial &pgb, const LscPolynomial &pb)\n> +\t\t: pr_(pr), pgr_(pgr), pgb_(pgb), pb_(pb)\n> \t{\n> \t}\n> \n> -\tint parseLscData(const YamlObject &yamlSets,\n> -\t\t\t std::map &lscData);\n> +\tLensShadingCorrection::Components sampleForCrop(const Rectangle &cropRectangle,\n> +\t\t\t\t\t\t\tSpan xSizes,\n> +\t\t\t\t\t\t\tSpan ySizes) override;\n> \n> private:\n> -\tstd::vector sizesListToPositions(const std::vector &sizes);\n> \tstd::vector samplePolynomial(const LscPolynomial &poly,\n> \t\t\t\t\t Span xPositions,\n> \t\t\t\t\t Span yPositions,\n> \t\t\t\t\t const Rectangle &cropRectangle);\n> \n> +\tstd::vector sizesListToPositions(Span sizes);\n> +\n> +\tLscPolynomial pr_;\n> +\tLscPolynomial pgr_;\n> +\tLscPolynomial pgb_;\n> +\tLscPolynomial pb_;\n> +};\n> +\n> +LensShadingCorrection::Components\n> +LscPolynomialShadingDescriptor::sampleForCrop(const Rectangle &cropRectangle,\n> +\t\t\t\t\t Span xSizes,\n> +\t\t\t\t\t Span ySizes)\n> +{\n> +\tstd::vector xPos = sizesListToPositions(xSizes);\n> +\tstd::vector yPos = sizesListToPositions(ySizes);\n> +\n> +\treturn {\n> +\t\t.r = samplePolynomial(pr_, xPos, yPos, cropRectangle),\n> +\t\t.gr = samplePolynomial(pgr_, xPos, yPos, cropRectangle),\n> +\t\t.gb = samplePolynomial(pgb_, xPos, yPos, cropRectangle),\n> +\t\t.b = samplePolynomial(pb_, xPos, yPos, cropRectangle)\n> +\t};\n> +}\n> +\n> +std::vector\n> +LscPolynomialShadingDescriptor::samplePolynomial(const LscPolynomial &poly,\n> +\t\t\t\t\t\t Span xPositions,\n> +\t\t\t\t\t\t Span yPositions,\n> +\t\t\t\t\t\t const Rectangle &cropRectangle)\n> +{\n> +\tdouble m = poly.getM();\n> +\tdouble x0 = cropRectangle.x / m;\n> +\tdouble y0 = cropRectangle.y / m;\n> +\tdouble w = cropRectangle.width / m;\n> +\tdouble h = cropRectangle.height / m;\n> +\tstd::vector samples;\n> +\n> +\tsamples.reserve(xPositions.size() * yPositions.size());\n> +\n> +\tfor (double y : yPositions) {\n> +\t\tfor (double x : xPositions) {\n> +\t\t\tdouble xp = x0 + x * w;\n> +\t\t\tdouble yp = y0 + y * h;\n> +\t\t\t/*\n> +\t\t\t * The hardware uses 2.10 fixed point format and limits\n> +\t\t\t * the legal values to [1..3.999]. Scale and clamp the\n> +\t\t\t * sampled value accordingly.\n> +\t\t\t */\n> +\t\t\tint v = static_cast(\n> +\t\t\t\tpoly.sampleAtNormalizedPixelPos(xp, yp) *\n> +\t\t\t\t1024);\n> +\t\t\tv = std::clamp(v, 1024, 4095);\n> +\t\t\tsamples.push_back(v);\n> +\t\t}\n> +\t}\n> +\treturn samples;\n> +}\n> +\n> +/*\n> + * The rkisp1 LSC grid spacing is defined by the cell sizes on the first half of\n> + * the grid. This is then mirrored in hardware to the other half. See\n> + * parseSizes() for further details. For easier handling, this function converts\n\nSame comment here as in patch 04/15.\n\n> + * the cell sizes of half the grid to a list of position of the whole grid (on\n> + * one axis). Example:\n> + *\n> + * input: | 0.2 | 0.3 |\n> + * output: 0.0 0.2 0.5 0.8 1.0\n> + */\n> +std::vector\n> +LscPolynomialShadingDescriptor::sizesListToPositions(Span sizes)\n> +{\n> +\tconst int half = sizes.size();\n> +\tstd::vector positions(half * 2 + 1);\n> +\tdouble x = 0.0;\n> +\n> +\tpositions[half] = 0.5;\n> +\tfor (int i = 1; i <= half; i++) {\n> +\t\tx += sizes[half - i];\n> +\t\tpositions[half - i] = 0.5 - x;\n> +\t\tpositions[half + i] = 0.5 + x;\n> +\t}\n> +\n> +\treturn positions;\n> +}\n> +\n> +class LscPolynomialLoader\n> +{\n> +public:\n> +\tLscPolynomialLoader(const Size &sensorSize)\n> +\t\t: sensorSize_(sensorSize)\n> +\t{\n> +\t}\n> +\n> +\tint parseLscData(const YamlObject &yamlSets,\n> +\t\t\t LensShadingCorrection::ShadingDescriptorMap &lscData);\n> +\n> +private:\n> \tSize sensorSize_;\n> -\tRectangle cropRectangle_;\n> -\tconst std::vector &xSizes_;\n> -\tconst std::vector &ySizes_;\n> };\n> \n> int LscPolynomialLoader::parseLscData(const YamlObject &yamlSets,\n> -\t\t\t\t std::map &lscData)\n> +\t\t\t\t LensShadingCorrection::ShadingDescriptorMap &lscData)\n> {\n> \tconst auto &sets = yamlSets.asList();\n> \tfor (const auto &yamlSet : sets) {\n> @@ -117,7 +206,6 @@ int LscPolynomialLoader::parseLscData(const YamlObject &yamlSets,\n> \t\t\treturn -EINVAL;\n> \t\t}\n> \n> -\t\tLensShadingCorrection::Components &set = lscData[ct];\n> \t\tpr = yamlSet[\"r\"].get();\n> \t\tpgr = yamlSet[\"gr\"].get();\n> \t\tpgb = yamlSet[\"gb\"].get();\n> @@ -135,12 +223,9 @@ int LscPolynomialLoader::parseLscData(const YamlObject &yamlSets,\n> \t\tpgb->setReferenceImageSize(sensorSize_);\n> \t\tpb->setReferenceImageSize(sensorSize_);\n> \n> -\t\tstd::vector xPos = sizesListToPositions(xSizes_);\n> -\t\tstd::vector yPos = sizesListToPositions(ySizes_);\n> -\t\tset.r = samplePolynomial(*pr, xPos, yPos, cropRectangle_);\n> -\t\tset.gr = samplePolynomial(*pgr, xPos, yPos, cropRectangle_);\n> -\t\tset.gb = samplePolynomial(*pgb, xPos, yPos, cropRectangle_);\n> -\t\tset.b = samplePolynomial(*pb, xPos, yPos, cropRectangle_);\n> +\t\tlscData.emplace(\n> +\t\t\tct, std::make_unique(\n> +\t\t\t\t *pr, *pgr, *pgb, *pb));\n> \t}\n> \n> \tif (lscData.empty()) {\n> @@ -151,70 +236,33 @@ int LscPolynomialLoader::parseLscData(const YamlObject &yamlSets,\n> \treturn 0;\n> }\n> \n> -/*\n> - * The rkisp1 LSC grid spacing is defined by the cell sizes on the first half of\n> - * the grid. This is then mirrored in hardware to the other half. See\n> - * parseSizes() for further details. For easier handling, this function converts\n> - * the cell sizes of half the grid to a list of position of the whole grid (on\n> - * one axis). Example:\n> - *\n> - * input: | 0.2 | 0.3 |\n> - * output: 0.0 0.2 0.5 0.8 1.0\n> - */\n> -std::vector LscPolynomialLoader::sizesListToPositions(const std::vector &sizes)\n> +class LscTableShadingDescriptor : public LensShadingCorrection::ShadingDescriptor\n> {\n> -\tconst int half = sizes.size();\n> -\tstd::vector positions(half * 2 + 1);\n> -\tdouble x = 0.0;\n> -\n> -\tpositions[half] = 0.5;\n> -\tfor (int i = 1; i <= half; i++) {\n> -\t\tx += sizes[half - i];\n> -\t\tpositions[half - i] = 0.5 - x;\n> -\t\tpositions[half + i] = 0.5 + x;\n> +public:\n> +\tLscTableShadingDescriptor(LensShadingCorrection::Components components)\n> +\t\t: lscData_(std::move(components))\n> +\t{\n> \t}\n> \n> -\treturn positions;\n> -}\n> -\n> -std::vector LscPolynomialLoader::samplePolynomial(const LscPolynomial &poly,\n> -\t\t\t\t\t\t\t Span xPositions,\n> -\t\t\t\t\t\t\t Span yPositions,\n> -\t\t\t\t\t\t\t const Rectangle &cropRectangle)\n> -{\n> -\tdouble m = poly.getM();\n> -\tdouble x0 = cropRectangle.x / m;\n> -\tdouble y0 = cropRectangle.y / m;\n> -\tdouble w = cropRectangle.width / m;\n> -\tdouble h = cropRectangle.height / m;\n> -\tstd::vector samples;\n> -\n> -\tsamples.reserve(xPositions.size() * yPositions.size());\n> -\n> -\tfor (double y : yPositions) {\n> -\t\tfor (double x : xPositions) {\n> -\t\t\tdouble xp = x0 + x * w;\n> -\t\t\tdouble yp = y0 + y * h;\n> -\t\t\t/*\n> -\t\t\t * The hardware uses 2.10 fixed point format and limits\n> -\t\t\t * the legal values to [1..3.999]. Scale and clamp the\n> -\t\t\t * sampled value accordingly.\n> -\t\t\t */\n> -\t\t\tint v = static_cast(\n> -\t\t\t\tpoly.sampleAtNormalizedPixelPos(xp, yp) *\n> -\t\t\t\t1024);\n> -\t\t\tv = std::min(std::max(v, 1024), 4095);\n> -\t\t\tsamples.push_back(v);\n> -\t\t}\n> +\tLensShadingCorrection::Components\n> +\tsampleForCrop([[maybe_unused]] const Rectangle &cropRectangle,\n> +\t\t [[maybe_unused]] Span xSizes,\n> +\t\t [[maybe_unused]] Span ySizes) override\n> +\t{\n> +\t\tLOG(RkISP1Lsc, Warning)\n> +\t\t\t<< \"Tabular LSC data doesn't support resampling\";\n> +\t\treturn lscData_;\n> \t}\n> -\treturn samples;\n> -}\n> +\n> +private:\n> +\tLensShadingCorrection::Components lscData_;\n> +};\n> \n> class LscTableLoader\n> {\n> public:\n> \tint parseLscData(const YamlObject &yamlSets,\n> -\t\t\t std::map &lscData);\n> +\t\t\t LensShadingCorrection::ShadingDescriptorMap &lscData);\n> \n> private:\n> \tstd::vector parseTable(const YamlObject &tuningData,\n> @@ -222,7 +270,7 @@ private:\n> };\n> \n> int LscTableLoader::parseLscData(const YamlObject &yamlSets,\n> -\t\t\t\t std::map &lscData)\n> +\t\t\t\t LensShadingCorrection::ShadingDescriptorMap &lscData)\n> {\n> \tconst auto &sets = yamlSets.asList();\n> \n> @@ -236,8 +284,7 @@ int LscTableLoader::parseLscData(const YamlObject &yamlSets,\n> \t\t\treturn -EINVAL;\n> \t\t}\n> \n> -\t\tLensShadingCorrection::Components &set = lscData[ct];\n> -\n> +\t\tLensShadingCorrection::Components set;\n> \t\tset.r = parseTable(yamlSet, \"r\");\n> \t\tset.gr = parseTable(yamlSet, \"gr\");\n> \t\tset.gb = parseTable(yamlSet, \"gb\");\n> @@ -250,6 +297,9 @@ int LscTableLoader::parseLscData(const YamlObject &yamlSets,\n> \t\t\t\t<< \" is missing tables\";\n> \t\t\treturn -EINVAL;\n> \t\t}\n> +\n> +\t\tlscData.emplace(\n> +\t\t\tct, std::make_unique(std::move(set)));\n> \t}\n> \n> \tif (lscData.empty()) {\n> @@ -341,7 +391,7 @@ int LensShadingCorrection::init([[maybe_unused]] IPAContext &context,\n> \t\treturn -EINVAL;\n> \t}\n> \n> -\tstd::map lscData;\n> +\tShadingDescriptorMap lscData;\n> \tint ret = 0;\n> \n> \tstd::string type = tuningData[\"type\"].get(\"table\");\n> @@ -351,10 +401,11 @@ int LensShadingCorrection::init([[maybe_unused]] IPAContext &context,\n> \t\tret = loader.parseLscData(yamlSets, lscData);\n> \t} else if (type == \"polynomial\") {\n> \t\tLOG(RkISP1Lsc, Debug) << \"Loading polynomial LSC data.\";\n> -\t\tauto loader = LscPolynomialLoader(context.sensorInfo.activeAreaSize,\n> -\t\t\t\t\t\t context.sensorInfo.analogCrop,\n> -\t\t\t\t\t\t xSize_,\n> -\t\t\t\t\t\t ySize_);\n> +\t\t/*\n> +\t\t * \\todo: Most likely the reference frame should be native_size.\n> +\t\t * Let's wait how the internal discussions progress.\n> +\t\t */\n> +\t\tauto loader = LscPolynomialLoader(context.sensorInfo.activeAreaSize);\n> \t\tret = loader.parseLscData(yamlSets, lscData);\n> \t} else {\n> \t\tLOG(RkISP1Lsc, Error) << \"Unsupported LSC data type '\"\n> @@ -365,7 +416,7 @@ int LensShadingCorrection::init([[maybe_unused]] IPAContext &context,\n> \tif (ret)\n> \t\treturn ret;\n> \n> -\tsets_.setData(std::move(lscData));\n> +\tshadingDescriptors_ = std::move(lscData);\n> \n> \treturn 0;\n> }\n> @@ -401,6 +452,14 @@ int LensShadingCorrection::configure(IPAContext &context,\n> \t\tyGrad_[i] = std::round(32768 / ySizes_[i]);\n> \t}\n> \n> +\tLOG(RkISP1Lsc, Debug) << \"Sample LSC data for \" << configInfo.analogCrop;\n> +\tstd::map shadingData;\n> +\tfor (const auto &[t, descriptor] : shadingDescriptors_)\n> +\t\tshadingData[t] = descriptor->sampleForCrop(configInfo.analogCrop,\n> +\t\t\t\t\t\t\t xSize_, ySize_);\n> +\n> +\tsets_.setData(std::move(shadingData));\n> +\n> \tcontext.configuration.lsc.enabled = true;\n> \treturn 0;\n> }\n> diff --git a/src/ipa/rkisp1/algorithms/lsc.h b/src/ipa/rkisp1/algorithms/lsc.h\n> index 7b68dda1a0d4257c345e0f1b77aa498f8183c92f..3097740a6cb2ce9063a4ba087856987a489b0ab6 100644\n> --- a/src/ipa/rkisp1/algorithms/lsc.h\n> +++ b/src/ipa/rkisp1/algorithms/lsc.h\n> @@ -8,6 +8,7 @@\n> #pragma once\n> \n> #include \n> +#include \n> \n> #include \"libipa/interpolator.h\"\n> \n> @@ -36,10 +37,22 @@ public:\n> \t\tstd::vector b;\n> \t};\n> \n> +\tclass ShadingDescriptor\n> +\t{\n> +\tpublic:\n> +\t\tvirtual ~ShadingDescriptor() = default;\n> +\t\tvirtual Components sampleForCrop(const Rectangle &cropRectangle,\n> +\t\t\t\t\t\t Span xSizes,\n> +\t\t\t\t\t\t Span ySizes) = 0;\n> +\t};\n> +\n> +\tusing ShadingDescriptorMap = std::map>;\n> +\n> private:\n> \tvoid setParameters(rkisp1_cif_isp_lsc_config &config);\n> \tvoid copyTable(rkisp1_cif_isp_lsc_config &config, const Components &set0);\n> \n> +\tShadingDescriptorMap shadingDescriptors_;\n> \tipa::Interpolator sets_;\n> \tstd::vector xSize_;\n> \tstd::vector ySize_;","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5FDEFC3226\n\tfor ;\n\tWed, 28 Jan 2026 11:12:52 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 4ED5E61FCB;\n\tWed, 28 Jan 2026 12:12:51 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B9C4B61FC5\n\tfor ;\n\tWed, 28 Jan 2026 12:12:49 +0100 (CET)","from pendragon.ideasonboard.com\n\t(2001-14ba-703d-e500--2a1.rev.dnainternet.fi\n\t[IPv6:2001:14ba:703d:e500::2a1])\n\tby perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 7057673B;\n\tWed, 28 Jan 2026 12:12:12 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"igy86VHe\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1769598732;\n\tbh=7gTCkBQv7Ah3N2oQ92fP7z4AaXOv5lcb4qtycLM6GPc=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=igy86VHe9jW21VhZsVpGZXVdWpQUoal4f22np3Ta/fujc0GzA76UryuRYQheAv6vZ\n\tZ4lSuVrqAfOd3K4PLDZCoQinE945NyS7wFhBTlPWJc7mLFUOYelNrmcmi8B0FM5L68\n\t6qa3e27uTEtCkP5yHR1Eu9tYUijdjBKt50aIL/6U=","Date":"Wed, 28 Jan 2026 13:12:47 +0200","From":"Laurent Pinchart ","To":"Stefan Klug ","Cc":"libcamera-devel@lists.libcamera.org, Kieran Bingham\n\t, =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?=\n\t","Subject":"Re: [PATCH v5 13/15] ipa: rkisp1: lsc: Resample polynomial lens\n\tshading tables at configure time","Message-ID":"<20260128111247.GC2558360@killaraus>","References":"<20260120-sklug-lsc-resampling-v2-dev-v5-0-ef5cec7b299f@ideasonboard.com>\n\t<20260120-sklug-lsc-resampling-v2-dev-v5-13-ef5cec7b299f@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20260120-sklug-lsc-resampling-v2-dev-v5-13-ef5cec7b299f@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]