[{"id":33190,"web_url":"https://patchwork.libcamera.org/comment/33190/","msgid":"<20250127015253.GD17899@pendragon.ideasonboard.com>","date":"2025-01-27T01:52:53","subject":"Re: [PATCH v4 9/9] libcamera: software_isp: Apply CCM in debayering","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Milan,\n\nThank you for the patch.\n\nOn Mon, Jan 13, 2025 at 02:51:06PM +0100, Milan Zamazal wrote:\n> This patch applies color correction matrix (CCM) in debayering if the\n> CCM is specified.  Not using CCM must still be supported for performance\n> reasons.\n> \n> The CCM is applied as follows:\n> \n>             [r1 r2 r3]\n>   [r g b] * [g1 g2 g3]\n>             [b1 b2 b3]\n> \n> The CCM matrix (the right side of the multiplication) is constant during\n> single frame processing, while the input pixel (the left side) changes.\n> Because each of the color channels is only 8-bit in software ISP, we can\n> make 9 lookup tables with 256 input values for multiplications of each\n> of the r_i, g_i, b_i values.  This way we don't have to multiply each\n> pixel, we can use table lookups and additions instead.  Gamma (which is\n> non-linear and thus cannot be a part of the 9 lookup tables values) is\n> applied on the final values rounded to integers using another lookup\n> table.\n\nWhat's the speedup compared to using fixed-point integer multiplications\n?\n\n> We use int16_t to store the precomputed multiplications.  This seems to\n> be noticeably (>10%) faster than `float' for the price of slightly less\n> accuracy and it covers the range of values that sane CCMs produce.  The\n> selection and structure of data is performance critical, for example\n> using bytes would add significant (>10%) speedup but would be too short\n> to cover the value range.\n> \n> The color lookup tables are changed to unions to be able to serve as\n> both simple lookup tables or CCM lookup tables.  This is arguable but it\n> keeps the code easier to understand if nothing else.  It is important to\n> make the unions on the whole lookup tables rather than their values\n> because the latter might cause a noticeable slowdown on simple lookup\n> due to the sheer fact that the table is not compact.  Using unions makes\n> the initialization of the tables dubious because it is done before the\n> decision about using CCM is made but the initial values are dubious\n> anyway.\n\nI have a hard time feeling sympathy towards that argument. It sounds\nlike \"it's not great already, so let's make it worse\".\n\n> Using std::variant instead of unions would be even more\n> difficult, consider e.g. mmap allocation and the followup type-casting\n> of the debayer params.\n> \n> The tables are copied (as before), which is not elegant but also not a\n> big problem.  There are patches posted that use shared buffers for\n> parameters passing in software ISP (see software ISP TODO #5) and they\n> can be adjusted for the new parameter format.\n> \n> Color gains from white balance are supposed not to be a part of the\n> specified CCM.  They are applied on it using matrix multiplication,\n> which is simple and in correspondence with future additions in the form\n> of matrix multiplication, like saturation adjustment.\n> \n> With this patch, the reported per-frame slowdown when applying CCM is\n> about 45% on Debix Model A and about 75% on TI AM69 SK.\n> \n> Using std::clamp in debayering adds some performance penalty (a few\n> percent).  The clamping is necessary to eliminate out of range values\n> possibly produced by the CCM.  If it could be avoided by adjusting the\n> precomputed tables some way then performance could be improved a bit.\n> \n> Signed-off-by: Milan Zamazal <mzamazal@redhat.com>\n> ---\n>  .../internal/software_isp/debayer_params.h    | 20 +++++-\n>  src/ipa/simple/algorithms/lut.cpp             | 63 ++++++++++++++-----\n>  src/ipa/simple/algorithms/lut.h               |  1 +\n>  src/libcamera/software_isp/debayer.cpp        | 50 +++++++++++++++\n>  src/libcamera/software_isp/debayer_cpu.cpp    | 34 +++++++---\n>  src/libcamera/software_isp/debayer_cpu.h      |  7 ++-\n>  src/libcamera/software_isp/software_isp.cpp   |  6 +-\n>  7 files changed, 147 insertions(+), 34 deletions(-)\n> \n> diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h\n> index 7d8fdd48..f8731eae 100644\n> --- a/include/libcamera/internal/software_isp/debayer_params.h\n> +++ b/include/libcamera/internal/software_isp/debayer_params.h\n> @@ -18,11 +18,25 @@ namespace libcamera {\n>  struct DebayerParams {\n>  \tstatic constexpr unsigned int kRGBLookupSize = 256;\n>  \n> +\tstruct CcmColumn {\n> +\t\tint16_t r;\n> +\t\tint16_t g;\n> +\t\tint16_t b;\n> +\t};\n> +\n>  \tusing ColorLookupTable = std::array<uint8_t, kRGBLookupSize>;\n> +\tusing CcmLookupTable = std::array<CcmColumn, kRGBLookupSize>;\n> +\tusing GammaLookupTable = std::array<uint8_t, kRGBLookupSize>;\n> +\n> +\tunion LookupTable {\n> +\t\tColorLookupTable simple;\n> +\t\tCcmLookupTable ccm;\n> +\t};\n>  \n> -\tColorLookupTable red;\n> -\tColorLookupTable green;\n> -\tColorLookupTable blue;\n> +\tLookupTable red;\n> +\tLookupTable green;\n> +\tLookupTable blue;\n> +\tGammaLookupTable gammaLut;\n\nThis is difficult to understand, especially without comments. If I'm not\nmistaken, the idea is that the CPU implementation of the software ISP\nwill use either <color>.simple, or <color>.ccm + gammaLut. The former is\nused when CCM is disabled, and combines colour gains and gamma in a\nsingle LUT. The latter is used when CCM is enabled, and combines colour\ngains and CCM in the CCM LUT, with the gamma LUT applying gamma only.\n\nI think it would be simpler to understand if you specified all LUTs for\nall colour components, unconditionally.\n\n\tstruct ColorLookupTable {\n\t\tCcmLookupTable ccm;\n\t\tGammaLookupTable gamma;\n\t}\n\n\tColorLookupTable red;\n\tColorLookupTable green;\n\tColorLookupTable blue;\n\n>  };\n>  \n>  } /* namespace libcamera */\n> diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp\n> index 243f0818..ddb2adf1 100644\n> --- a/src/ipa/simple/algorithms/lut.cpp\n> +++ b/src/ipa/simple/algorithms/lut.cpp\n> @@ -80,6 +80,11 @@ void Lut::updateGammaTable(IPAContext &context)\n>  \tcontext.activeState.gamma.contrast = contrast;\n>  }\n>  \n> +int16_t Lut::ccmValue(unsigned int i, float ccm) const\n> +{\n> +\treturn std::round(i * ccm);\n> +}\n> +\n>  void Lut::prepare(IPAContext &context,\n>  \t\t  [[maybe_unused]] const uint32_t frame,\n>  \t\t  [[maybe_unused]] IPAFrameContext &frameContext,\n> @@ -91,28 +96,52 @@ void Lut::prepare(IPAContext &context,\n>  \t * observed, it's not permanently prone to minor fluctuations or\n>  \t * rounding errors.\n>  \t */\n> -\tif (context.activeState.gamma.blackLevel != context.activeState.blc.level ||\n> -\t    context.activeState.gamma.contrast != context.activeState.knobs.contrast)\n> +\tconst bool gammaUpdateNeeded =\n> +\t\tcontext.activeState.gamma.blackLevel != context.activeState.blc.level ||\n> +\t\tcontext.activeState.gamma.contrast != context.activeState.knobs.contrast;\n> +\tif (gammaUpdateNeeded)\n>  \t\tupdateGammaTable(context);\n>  \n>  \tauto &gains = context.activeState.awb.gains;\n>  \tauto &gammaTable = context.activeState.gamma.gammaTable;\n>  \tconst unsigned int gammaTableSize = gammaTable.size();\n> -\n> -\tfor (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) {\n> -\t\tconst double div = static_cast<double>(DebayerParams::kRGBLookupSize) /\n> -\t\t\t\t   gammaTableSize;\n> -\t\t/* Apply gamma after gain! */\n> -\t\tunsigned int idx;\n> -\t\tidx = std::min({ static_cast<unsigned int>(i * gains.red / div),\n> -\t\t\t\t gammaTableSize - 1 });\n> -\t\tparams->red[i] = gammaTable[idx];\n> -\t\tidx = std::min({ static_cast<unsigned int>(i * gains.green / div),\n> -\t\t\t\t gammaTableSize - 1 });\n> -\t\tparams->green[i] = gammaTable[idx];\n> -\t\tidx = std::min({ static_cast<unsigned int>(i * gains.blue / div),\n> -\t\t\t\t gammaTableSize - 1 });\n> -\t\tparams->blue[i] = gammaTable[idx];\n> +\tconst double div = static_cast<double>(DebayerParams::kRGBLookupSize) /\n> +\t\t\t   gammaTableSize;\n> +\n> +\tif (!context.activeState.ccm.enabled) {\n> +\t\tfor (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) {\n> +\t\t\t/* Apply gamma after gain! */\n> +\t\t\tunsigned int idx;\n> +\t\t\tidx = std::min({ static_cast<unsigned int>(i * gains.red / div),\n> +\t\t\t\t\t gammaTableSize - 1 });\n> +\t\t\tparams->red.simple[i] = gammaTable[idx];\n> +\t\t\tidx = std::min({ static_cast<unsigned int>(i * gains.green / div),\n> +\t\t\t\t\t gammaTableSize - 1 });\n> +\t\t\tparams->green.simple[i] = gammaTable[idx];\n> +\t\t\tidx = std::min({ static_cast<unsigned int>(i * gains.blue / div),\n> +\t\t\t\t\t gammaTableSize - 1 });\n> +\t\t\tparams->blue.simple[i] = gammaTable[idx];\n> +\t\t}\n> +\t} else if (context.activeState.ccm.changed || gammaUpdateNeeded) {\n> +\t\tMatrix<double, 3, 3> gainCcm = { { gains.red, 0, 0,\n> +\t\t\t\t\t\t   0, gains.green, 0,\n> +\t\t\t\t\t\t   0, 0, gains.blue } };\n> +\t\tauto ccm = gainCcm * context.activeState.ccm.ccm;\n> +\t\tauto &red = params->red.ccm;\n> +\t\tauto &green = params->green.ccm;\n> +\t\tauto &blue = params->blue.ccm;\n> +\t\tfor (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) {\n> +\t\t\tred[i].r = ccmValue(i, ccm[0][0]);\n> +\t\t\tred[i].g = ccmValue(i, ccm[1][0]);\n> +\t\t\tred[i].b = ccmValue(i, ccm[2][0]);\n> +\t\t\tgreen[i].r = ccmValue(i, ccm[0][1]);\n> +\t\t\tgreen[i].g = ccmValue(i, ccm[1][1]);\n> +\t\t\tgreen[i].b = ccmValue(i, ccm[2][1]);\n> +\t\t\tblue[i].r = ccmValue(i, ccm[0][2]);\n> +\t\t\tblue[i].g = ccmValue(i, ccm[1][2]);\n> +\t\t\tblue[i].b = ccmValue(i, ccm[2][2]);\n> +\t\t\tparams->gammaLut[i] = gammaTable[i / div];\n> +\t\t}\n>  \t}\n>  }\n>  \n> diff --git a/src/ipa/simple/algorithms/lut.h b/src/ipa/simple/algorithms/lut.h\n> index 889f864b..77324800 100644\n> --- a/src/ipa/simple/algorithms/lut.h\n> +++ b/src/ipa/simple/algorithms/lut.h\n> @@ -33,6 +33,7 @@ public:\n>  \n>  private:\n>  \tvoid updateGammaTable(IPAContext &context);\n> +\tint16_t ccmValue(unsigned int i, float ccm) const;\n>  };\n>  \n>  } /* namespace ipa::soft::algorithms */\n> diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp\n> index 45fe6960..559de3e6 100644\n> --- a/src/libcamera/software_isp/debayer.cpp\n> +++ b/src/libcamera/software_isp/debayer.cpp\n> @@ -23,11 +23,56 @@ namespace libcamera {\n>   * \\brief Size of a color lookup table\n>   */\n>  \n> +/**\n> + * \\struct DebayerParams::CcmColumn\n> + * \\brief Type of a single column of a color correction matrix\n> + */\n> +\n> +/**\n> + * \\var DebayerParams::CcmColumn::r\n> + * \\brief Red (first) component of a CCM column\n> + */\n> +\n> +/**\n> + * \\var DebayerParams::CcmColumn::g\n> + * \\brief Green (second) component of a CCM column\n> + */\n> +\n> +/**\n> + * \\var DebayerParams::CcmColumn::b\n> + * \\brief Blue (third) component of a CCM column\n> + */\n> +\n>  /**\n>   * \\typedef DebayerParams::ColorLookupTable\n> + * \\brief Type of the simple lookup tables for red, green, blue values\n> + */\n> +\n> +/**\n> + * \\typedef DebayerParams::CcmLookupTable\n> + * \\brief Type of the CCM lookup tables for red, green, blue values\n> + */\n> +\n> +/**\n> + * \\typedef DebayerParams::GammaLookupTable\n> + * \\brief Type of the gamma lookup tables for CCM\n> + */\n> +\n> +/**\n> + * \\union DebayerParams::LookupTable\n>   * \\brief Type of the lookup tables for red, green, blue values\n>   */\n>  \n> +/**\n> + * \\var DebayerParams::LookupTable::simple\n> + * \\brief Simple lookup table for red, green, blue values\n> + */\n> +\n> +/**\n> + * \\var DebayerParams::LookupTable::ccm\n> + * \\brief CCM lookup table for red, green, blue values\n> + */\n> +\n>  /**\n>   * \\var DebayerParams::red\n>   * \\brief Lookup table for red color, mapping input values to output values\n> @@ -43,6 +88,11 @@ namespace libcamera {\n>   * \\brief Lookup table for blue color, mapping input values to output values\n>   */\n>  \n> +/**\n> + * \\var DebayerParams::gammaLut\n> + * \\brief Gamma lookup table used with color correction matrix\n> + */\n> +\n>  /**\n>   * \\class Debayer\n>   * \\brief Base debayering class\n> diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp\n> index 3c6597f9..38715b72 100644\n> --- a/src/libcamera/software_isp/debayer_cpu.cpp\n> +++ b/src/libcamera/software_isp/debayer_cpu.cpp\n> @@ -11,6 +11,7 @@\n>  \n>  #include \"debayer_cpu.h\"\n>  \n> +#include <algorithm>\n>  #include <stdlib.h>\n>  #include <sys/ioctl.h>\n>  #include <time.h>\n> @@ -51,8 +52,12 @@ DebayerCpu::DebayerCpu(std::unique_ptr<SwStatsCpu> stats)\n>  \tenableInputMemcpy_ = true;\n>  \n>  \t/* Initialize color lookup tables */\n> -\tfor (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++)\n> -\t\tred_[i] = green_[i] = blue_[i] = i;\n> +\tfor (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) {\n> +\t\tred_.simple[i] = green_.simple[i] = blue_.simple[i] = i;\n> +\t\tred_.ccm[i].r = red_.ccm[i].g = red_.ccm[i].b = 0;\n> +\t\tgreen_.ccm[i].r = green_.ccm[i].g = green_.ccm[i].b = 0;\n> +\t\tblue_.ccm[i].r = blue_.ccm[i].g = blue_.ccm[i].b = 0;\n> +\t}\n>  }\n>  \n>  DebayerCpu::~DebayerCpu() = default;\n> @@ -62,12 +67,24 @@ DebayerCpu::~DebayerCpu() = default;\n>  \tconst pixel_t *curr = (const pixel_t *)src[1] + xShift_; \\\n>  \tconst pixel_t *next = (const pixel_t *)src[2] + xShift_;\n>  \n> -#define STORE_PIXEL(b, g, r)        \\\n> -\t*dst++ = blue_[b];          \\\n> -\t*dst++ = green_[g];         \\\n> -\t*dst++ = red_[r];           \\\n> -\tif constexpr (addAlphaByte) \\\n> -\t\t*dst++ = 255;       \\\n> +#define GAMMA(value) \\\n> +\t*dst++ = gammaLut_[std::clamp(value, 0, static_cast<int>(gammaLut_.size()) - 1)]\n\nWhat's the point in specifying the CCM lookup table values as 16-bit\nintegers if you clamp to 8 bits anyway ? Is that because you need signed\nvalues ?\n\n> +\n> +#define STORE_PIXEL(b_, g_, r_)                                   \\\n> +\tif constexpr (ccmEnabled) {                               \\\n> +\t\tDebayerParams::CcmColumn &blue = blue_.ccm[b_];   \\\n> +\t\tDebayerParams::CcmColumn &green = green_.ccm[g_]; \\\n> +\t\tDebayerParams::CcmColumn &red = red_.ccm[r_];     \\\n\nconst\n\n> +\t\tGAMMA(blue.r + blue.g + blue.b);                  \\\n> +\t\tGAMMA(green.r + green.g + green.b);               \\\n> +\t\tGAMMA(red.r + red.g + red.b);                     \\\n> +\t} else {                                                  \\\n> +\t\t*dst++ = blue_.simple[b_];                        \\\n> +\t\t*dst++ = green_.simple[g_];                       \\\n> +\t\t*dst++ = red_.simple[r_];                         \\\n> +\t}                                                         \\\n> +\tif constexpr (addAlphaByte)                               \\\n> +\t\t*dst++ = 255;                                     \\\n>  \tx++;\n>  \n>  /*\n> @@ -752,6 +769,7 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output\n>  \tgreen_ = params.green;\n>  \tred_ = swapRedBlueGains_ ? params.blue : params.red;\n>  \tblue_ = swapRedBlueGains_ ? params.red : params.blue;\n> +\tgammaLut_ = params.gammaLut;\n>  \n>  \t/* Copy metadata from the input buffer */\n>  \tFrameMetadata &metadata = output->_d()->metadata();\n> diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h\n> index b2ec8f1b..35c51e4f 100644\n> --- a/src/libcamera/software_isp/debayer_cpu.h\n> +++ b/src/libcamera/software_isp/debayer_cpu.h\n> @@ -137,9 +137,10 @@ private:\n>  \t/* Max. supported Bayer pattern height is 4, debayering this requires 5 lines */\n>  \tstatic constexpr unsigned int kMaxLineBuffers = 5;\n>  \n> -\tDebayerParams::ColorLookupTable red_;\n> -\tDebayerParams::ColorLookupTable green_;\n> -\tDebayerParams::ColorLookupTable blue_;\n> +\tDebayerParams::LookupTable red_;\n> +\tDebayerParams::LookupTable green_;\n> +\tDebayerParams::LookupTable blue_;\n> +\tDebayerParams::GammaLookupTable gammaLut_;\n>  \tdebayerFn debayer0_;\n>  \tdebayerFn debayer1_;\n>  \tdebayerFn debayer2_;\n> diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp\n> index 6a07de85..40e11b9e 100644\n> --- a/src/libcamera/software_isp/software_isp.cpp\n> +++ b/src/libcamera/software_isp/software_isp.cpp\n> @@ -82,9 +82,9 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor,\n>  \tfor (unsigned int i = 0; i < 256; i++)\n>  \t\tgammaTable[i] = UINT8_MAX * std::pow(i / 256.0, 0.5);\n>  \tfor (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) {\n> -\t\tdebayerParams_.red[i] = gammaTable[i];\n> -\t\tdebayerParams_.green[i] = gammaTable[i];\n> -\t\tdebayerParams_.blue[i] = gammaTable[i];\n> +\t\tdebayerParams_.red.simple[i] = gammaTable[i];\n> +\t\tdebayerParams_.green.simple[i] = gammaTable[i];\n> +\t\tdebayerParams_.blue.simple[i] = gammaTable[i];\n>  \t}\n>  \n>  \tif (!dmaHeap_.isValid()) {","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 1E92DBDCBF\n\tfor <parsemail@patchwork.libcamera.org>;\n\tMon, 27 Jan 2025 01:53:08 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 504436855D;\n\tMon, 27 Jan 2025 02:53:07 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 55EB861875\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon, 27 Jan 2025 02:53:05 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id E2FCC352;\n\tMon, 27 Jan 2025 02:51:58 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"bFgiFLgQ\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1737942719;\n\tbh=c5+eWlteg+TROUNjCI3MDFhKQ4teJJaFGkyTF3aBMTk=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=bFgiFLgQgVpM4HPKYNiQ41RC1zBz+5hniL1u4vUd2LU3unoGY5mtB0+FctKi7iEbb\n\tlM0g4liNaQOTTrR63uTRlXY/WVP1vConL3hZi/R45D9fbB5ARNMFGWpPNaKMMeQXjt\n\tulIkAMxcJEiaZGM4J35aj7steM0yyv7n125ueP7k=","Date":"Mon, 27 Jan 2025 03:52:53 +0200","From":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","To":"Milan Zamazal <mzamazal@redhat.com>","Cc":"libcamera-devel@lists.libcamera.org,\n\tRobert Mader <robert.mader@collabora.com>,\n\tHans de Goede <hdegoede@redhat.com>,\n\tKieran Bingham <kieran.bingham@ideasonboard.com>","Subject":"Re: [PATCH v4 9/9] libcamera: software_isp: Apply CCM in debayering","Message-ID":"<20250127015253.GD17899@pendragon.ideasonboard.com>","References":"<20250113135108.13924-1-mzamazal@redhat.com>\n\t<20250113135108.13924-10-mzamazal@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20250113135108.13924-10-mzamazal@redhat.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":33221,"web_url":"https://patchwork.libcamera.org/comment/33221/","msgid":"<85a5bbdk6a.fsf@mzamazal-thinkpadp1gen3.tpbc.csb>","date":"2025-01-28T15:37:01","subject":"Re: [PATCH v4 9/9] libcamera: software_isp: Apply CCM in debayering","submitter":{"id":177,"url":"https://patchwork.libcamera.org/api/people/177/","name":"Milan Zamazal","email":"mzamazal@redhat.com"},"content":"Hi Laurent,\n\nthank you for review.\n\nLaurent Pinchart <laurent.pinchart@ideasonboard.com> writes:\n\n> Hi Milan,\n>\n> Thank you for the patch.\n>\n> On Mon, Jan 13, 2025 at 02:51:06PM +0100, Milan Zamazal wrote:\n>> This patch applies color correction matrix (CCM) in debayering if the\n>> CCM is specified.  Not using CCM must still be supported for performance\n>> reasons.\n>> \n>> The CCM is applied as follows:\n>> \n>>             [r1 r2 r3]\n>>   [r g b] * [g1 g2 g3]\n>>             [b1 b2 b3]\n>> \n>> The CCM matrix (the right side of the multiplication) is constant during\n>> single frame processing, while the input pixel (the left side) changes.\n>> Because each of the color channels is only 8-bit in software ISP, we can\n>> make 9 lookup tables with 256 input values for multiplications of each\n>> of the r_i, g_i, b_i values.  This way we don't have to multiply each\n>> pixel, we can use table lookups and additions instead.  Gamma (which is\n>> non-linear and thus cannot be a part of the 9 lookup tables values) is\n>> applied on the final values rounded to integers using another lookup\n>> table.\n>\n> What's the speedup compared to using fixed-point integer multiplications\n> ?\n\nJust a quick test by replacing the table lookups with multiplications by\narbitrary constant integers + right shift on the final result increases\nthe measured time per frame by ~25% in my environment.  (When on it, I\ntried using floats, without the final shift, and the slowdown was close\nto 70%.)\n\n>> We use int16_t to store the precomputed multiplications.  This seems to\n>> be noticeably (>10%) faster than `float' for the price of slightly less\n>> accuracy and it covers the range of values that sane CCMs produce.  The\n>> selection and structure of data is performance critical, for example\n>> using bytes would add significant (>10%) speedup but would be too short\n>> to cover the value range.\n>> \n>> The color lookup tables are changed to unions to be able to serve as\n>> both simple lookup tables or CCM lookup tables.  This is arguable but it\n>> keeps the code easier to understand if nothing else.  It is important to\n>> make the unions on the whole lookup tables rather than their values\n>> because the latter might cause a noticeable slowdown on simple lookup\n>> due to the sheer fact that the table is not compact.  Using unions makes\n>> the initialization of the tables dubious because it is done before the\n>> decision about using CCM is made but the initial values are dubious\n>> anyway.\n>\n> I have a hard time feeling sympathy towards that argument. It sounds\n> like \"it's not great already, so let's make it worse\".\n\nOK, OK, ...  It'll be resolved by not using the union.\n\n>> Using std::variant instead of unions would be even more\n>> difficult, consider e.g. mmap allocation and the followup type-casting\n>> of the debayer params.\n>> \n>> The tables are copied (as before), which is not elegant but also not a\n>> big problem.  There are patches posted that use shared buffers for\n>> parameters passing in software ISP (see software ISP TODO #5) and they\n>> can be adjusted for the new parameter format.\n>> \n>> Color gains from white balance are supposed not to be a part of the\n>> specified CCM.  They are applied on it using matrix multiplication,\n>> which is simple and in correspondence with future additions in the form\n>> of matrix multiplication, like saturation adjustment.\n>> \n>> With this patch, the reported per-frame slowdown when applying CCM is\n>> about 45% on Debix Model A and about 75% on TI AM69 SK.\n>> \n>> Using std::clamp in debayering adds some performance penalty (a few\n>> percent).  The clamping is necessary to eliminate out of range values\n>> possibly produced by the CCM.  If it could be avoided by adjusting the\n>> precomputed tables some way then performance could be improved a bit.\n>> \n>> Signed-off-by: Milan Zamazal <mzamazal@redhat.com>\n>> ---\n>>  .../internal/software_isp/debayer_params.h    | 20 +++++-\n>>  src/ipa/simple/algorithms/lut.cpp             | 63 ++++++++++++++-----\n>>  src/ipa/simple/algorithms/lut.h               |  1 +\n>>  src/libcamera/software_isp/debayer.cpp        | 50 +++++++++++++++\n>>  src/libcamera/software_isp/debayer_cpu.cpp    | 34 +++++++---\n>>  src/libcamera/software_isp/debayer_cpu.h      |  7 ++-\n>>  src/libcamera/software_isp/software_isp.cpp   |  6 +-\n>>  7 files changed, 147 insertions(+), 34 deletions(-)\n>> \n>> diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h\n>> index 7d8fdd48..f8731eae 100644\n>> --- a/include/libcamera/internal/software_isp/debayer_params.h\n>> +++ b/include/libcamera/internal/software_isp/debayer_params.h\n>> @@ -18,11 +18,25 @@ namespace libcamera {\n>>  struct DebayerParams {\n>>  \tstatic constexpr unsigned int kRGBLookupSize = 256;\n>>  \n>> +\tstruct CcmColumn {\n>> +\t\tint16_t r;\n>> +\t\tint16_t g;\n>> +\t\tint16_t b;\n>> +\t};\n>> +\n>>  \tusing ColorLookupTable = std::array<uint8_t, kRGBLookupSize>;\n>> +\tusing CcmLookupTable = std::array<CcmColumn, kRGBLookupSize>;\n>> +\tusing GammaLookupTable = std::array<uint8_t, kRGBLookupSize>;\n>> +\n>> +\tunion LookupTable {\n>> +\t\tColorLookupTable simple;\n>> +\t\tCcmLookupTable ccm;\n>> +\t};\n>>  \n>> -\tColorLookupTable red;\n>> -\tColorLookupTable green;\n>> -\tColorLookupTable blue;\n>> +\tLookupTable red;\n>> +\tLookupTable green;\n>> +\tLookupTable blue;\n>> +\tGammaLookupTable gammaLut;\n>\n> This is difficult to understand, especially without comments. If I'm not\n> mistaken, the idea is that the CPU implementation of the software ISP\n> will use either <color>.simple, or <color>.ccm + gammaLut. The former is\n> used when CCM is disabled, and combines colour gains and gamma in a\n> single LUT. The latter is used when CCM is enabled, and combines colour\n> gains and CCM in the CCM LUT, with the gamma LUT applying gamma only.\n\nYes.\n\n> I think it would be simpler to understand if you specified all LUTs for\n> all colour components, unconditionally.\n>\n> \tstruct ColorLookupTable {\n> \t\tCcmLookupTable ccm;\n> \t\tGammaLookupTable gamma;\n> \t}\n>\n> \tColorLookupTable red;\n> \tColorLookupTable green;\n> \tColorLookupTable blue;\n\nSomething like this was the other option I originally considered, so I'm\nfine with such a change.  Gamma is applied only on the final result\n(with CCM), so it must be kept separately.  And non-CCM and CCM lookup\ntables should be separated.  I'll do it in v5. \n\n>>  };\n>>  \n>>  } /* namespace libcamera */\n>> diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp\n>> index 243f0818..ddb2adf1 100644\n>> --- a/src/ipa/simple/algorithms/lut.cpp\n>> +++ b/src/ipa/simple/algorithms/lut.cpp\n>> @@ -80,6 +80,11 @@ void Lut::updateGammaTable(IPAContext &context)\n>>  \tcontext.activeState.gamma.contrast = contrast;\n>>  }\n>>  \n>> +int16_t Lut::ccmValue(unsigned int i, float ccm) const\n>> +{\n>> +\treturn std::round(i * ccm);\n>> +}\n>> +\n>>  void Lut::prepare(IPAContext &context,\n>>  \t\t  [[maybe_unused]] const uint32_t frame,\n>>  \t\t  [[maybe_unused]] IPAFrameContext &frameContext,\n>> @@ -91,28 +96,52 @@ void Lut::prepare(IPAContext &context,\n>>  \t * observed, it's not permanently prone to minor fluctuations or\n>>  \t * rounding errors.\n>>  \t */\n>> -\tif (context.activeState.gamma.blackLevel != context.activeState.blc.level ||\n>> -\t    context.activeState.gamma.contrast != context.activeState.knobs.contrast)\n>> +\tconst bool gammaUpdateNeeded =\n>> +\t\tcontext.activeState.gamma.blackLevel != context.activeState.blc.level ||\n>> +\t\tcontext.activeState.gamma.contrast != context.activeState.knobs.contrast;\n>> +\tif (gammaUpdateNeeded)\n>>  \t\tupdateGammaTable(context);\n>>  \n>>  \tauto &gains = context.activeState.awb.gains;\n>>  \tauto &gammaTable = context.activeState.gamma.gammaTable;\n>>  \tconst unsigned int gammaTableSize = gammaTable.size();\n>> -\n>> -\tfor (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) {\n>> -\t\tconst double div = static_cast<double>(DebayerParams::kRGBLookupSize) /\n>> -\t\t\t\t   gammaTableSize;\n>> -\t\t/* Apply gamma after gain! */\n>> -\t\tunsigned int idx;\n>> -\t\tidx = std::min({ static_cast<unsigned int>(i * gains.red / div),\n>> -\t\t\t\t gammaTableSize - 1 });\n>> -\t\tparams->red[i] = gammaTable[idx];\n>> -\t\tidx = std::min({ static_cast<unsigned int>(i * gains.green / div),\n>> -\t\t\t\t gammaTableSize - 1 });\n>> -\t\tparams->green[i] = gammaTable[idx];\n>> -\t\tidx = std::min({ static_cast<unsigned int>(i * gains.blue / div),\n>> -\t\t\t\t gammaTableSize - 1 });\n>> -\t\tparams->blue[i] = gammaTable[idx];\n>> +\tconst double div = static_cast<double>(DebayerParams::kRGBLookupSize) /\n>> +\t\t\t   gammaTableSize;\n>> +\n>> +\tif (!context.activeState.ccm.enabled) {\n>> +\t\tfor (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) {\n>> +\t\t\t/* Apply gamma after gain! */\n>> +\t\t\tunsigned int idx;\n>> +\t\t\tidx = std::min({ static_cast<unsigned int>(i * gains.red / div),\n>> +\t\t\t\t\t gammaTableSize - 1 });\n>> +\t\t\tparams->red.simple[i] = gammaTable[idx];\n>> +\t\t\tidx = std::min({ static_cast<unsigned int>(i * gains.green / div),\n>> +\t\t\t\t\t gammaTableSize - 1 });\n>> +\t\t\tparams->green.simple[i] = gammaTable[idx];\n>> +\t\t\tidx = std::min({ static_cast<unsigned int>(i * gains.blue / div),\n>> +\t\t\t\t\t gammaTableSize - 1 });\n>> +\t\t\tparams->blue.simple[i] = gammaTable[idx];\n>> +\t\t}\n>> +\t} else if (context.activeState.ccm.changed || gammaUpdateNeeded) {\n>> +\t\tMatrix<double, 3, 3> gainCcm = { { gains.red, 0, 0,\n>> +\t\t\t\t\t\t   0, gains.green, 0,\n>> +\t\t\t\t\t\t   0, 0, gains.blue } };\n>> +\t\tauto ccm = gainCcm * context.activeState.ccm.ccm;\n>> +\t\tauto &red = params->red.ccm;\n>> +\t\tauto &green = params->green.ccm;\n>> +\t\tauto &blue = params->blue.ccm;\n>> +\t\tfor (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) {\n>> +\t\t\tred[i].r = ccmValue(i, ccm[0][0]);\n>> +\t\t\tred[i].g = ccmValue(i, ccm[1][0]);\n>> +\t\t\tred[i].b = ccmValue(i, ccm[2][0]);\n>> +\t\t\tgreen[i].r = ccmValue(i, ccm[0][1]);\n>> +\t\t\tgreen[i].g = ccmValue(i, ccm[1][1]);\n>> +\t\t\tgreen[i].b = ccmValue(i, ccm[2][1]);\n>> +\t\t\tblue[i].r = ccmValue(i, ccm[0][2]);\n>> +\t\t\tblue[i].g = ccmValue(i, ccm[1][2]);\n>> +\t\t\tblue[i].b = ccmValue(i, ccm[2][2]);\n>> +\t\t\tparams->gammaLut[i] = gammaTable[i / div];\n>> +\t\t}\n>>  \t}\n>>  }\n>>  \n>> diff --git a/src/ipa/simple/algorithms/lut.h b/src/ipa/simple/algorithms/lut.h\n>> index 889f864b..77324800 100644\n>> --- a/src/ipa/simple/algorithms/lut.h\n>> +++ b/src/ipa/simple/algorithms/lut.h\n>> @@ -33,6 +33,7 @@ public:\n>>  \n>>  private:\n>>  \tvoid updateGammaTable(IPAContext &context);\n>> +\tint16_t ccmValue(unsigned int i, float ccm) const;\n>>  };\n>>  \n>>  } /* namespace ipa::soft::algorithms */\n>> diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp\n>> index 45fe6960..559de3e6 100644\n>> --- a/src/libcamera/software_isp/debayer.cpp\n>> +++ b/src/libcamera/software_isp/debayer.cpp\n>> @@ -23,11 +23,56 @@ namespace libcamera {\n>>   * \\brief Size of a color lookup table\n>>   */\n>>  \n>> +/**\n>> + * \\struct DebayerParams::CcmColumn\n>> + * \\brief Type of a single column of a color correction matrix\n>> + */\n>> +\n>> +/**\n>> + * \\var DebayerParams::CcmColumn::r\n>> + * \\brief Red (first) component of a CCM column\n>> + */\n>> +\n>> +/**\n>> + * \\var DebayerParams::CcmColumn::g\n>> + * \\brief Green (second) component of a CCM column\n>> + */\n>> +\n>> +/**\n>> + * \\var DebayerParams::CcmColumn::b\n>> + * \\brief Blue (third) component of a CCM column\n>> + */\n>> +\n>>  /**\n>>   * \\typedef DebayerParams::ColorLookupTable\n>> + * \\brief Type of the simple lookup tables for red, green, blue values\n>> + */\n>> +\n>> +/**\n>> + * \\typedef DebayerParams::CcmLookupTable\n>> + * \\brief Type of the CCM lookup tables for red, green, blue values\n>> + */\n>> +\n>> +/**\n>> + * \\typedef DebayerParams::GammaLookupTable\n>> + * \\brief Type of the gamma lookup tables for CCM\n>> + */\n>> +\n>> +/**\n>> + * \\union DebayerParams::LookupTable\n>>   * \\brief Type of the lookup tables for red, green, blue values\n>>   */\n>>  \n>> +/**\n>> + * \\var DebayerParams::LookupTable::simple\n>> + * \\brief Simple lookup table for red, green, blue values\n>> + */\n>> +\n>> +/**\n>> + * \\var DebayerParams::LookupTable::ccm\n>> + * \\brief CCM lookup table for red, green, blue values\n>> + */\n>> +\n>>  /**\n>>   * \\var DebayerParams::red\n>>   * \\brief Lookup table for red color, mapping input values to output values\n>> @@ -43,6 +88,11 @@ namespace libcamera {\n>>   * \\brief Lookup table for blue color, mapping input values to output values\n>>   */\n>>  \n>> +/**\n>> + * \\var DebayerParams::gammaLut\n>> + * \\brief Gamma lookup table used with color correction matrix\n>> + */\n>> +\n>>  /**\n>>   * \\class Debayer\n>>   * \\brief Base debayering class\n>> diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp\n>> index 3c6597f9..38715b72 100644\n>> --- a/src/libcamera/software_isp/debayer_cpu.cpp\n>> +++ b/src/libcamera/software_isp/debayer_cpu.cpp\n>> @@ -11,6 +11,7 @@\n>>  \n>>  #include \"debayer_cpu.h\"\n>>  \n>> +#include <algorithm>\n>>  #include <stdlib.h>\n>>  #include <sys/ioctl.h>\n>>  #include <time.h>\n>> @@ -51,8 +52,12 @@ DebayerCpu::DebayerCpu(std::unique_ptr<SwStatsCpu> stats)\n>>  \tenableInputMemcpy_ = true;\n>>  \n>>  \t/* Initialize color lookup tables */\n>> -\tfor (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++)\n>> -\t\tred_[i] = green_[i] = blue_[i] = i;\n>> +\tfor (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) {\n>> +\t\tred_.simple[i] = green_.simple[i] = blue_.simple[i] = i;\n>> +\t\tred_.ccm[i].r = red_.ccm[i].g = red_.ccm[i].b = 0;\n>> +\t\tgreen_.ccm[i].r = green_.ccm[i].g = green_.ccm[i].b = 0;\n>> +\t\tblue_.ccm[i].r = blue_.ccm[i].g = blue_.ccm[i].b = 0;\n>> +\t}\n>>  }\n>>  \n>>  DebayerCpu::~DebayerCpu() = default;\n>> @@ -62,12 +67,24 @@ DebayerCpu::~DebayerCpu() = default;\n>>  \tconst pixel_t *curr = (const pixel_t *)src[1] + xShift_; \\\n>>  \tconst pixel_t *next = (const pixel_t *)src[2] + xShift_;\n>>  \n>> -#define STORE_PIXEL(b, g, r)        \\\n>> -\t*dst++ = blue_[b];          \\\n>> -\t*dst++ = green_[g];         \\\n>> -\t*dst++ = red_[r];           \\\n>> -\tif constexpr (addAlphaByte) \\\n>> -\t\t*dst++ = 255;       \\\n>> +#define GAMMA(value) \\\n>> +\t*dst++ = gammaLut_[std::clamp(value, 0, static_cast<int>(gammaLut_.size()) - 1)]\n>\n> What's the point in specifying the CCM lookup table values as 16-bit\n> integers if you clamp to 8 bits anyway ? Is that because you need signed\n> values ?\n\nThe table values are uint8_t values multiplied by matrix elements, which\nmay be arbitrary numbers.  Each multiplication result can be within\nuint8_t range or negative or bigger.  Three multiplication results are\nsummed so they may return back to uint8_t range and only then clamping\ncan be applied.\n\n>> +\n>> +#define STORE_PIXEL(b_, g_, r_)                                   \\\n>> +\tif constexpr (ccmEnabled) {                               \\\n>> +\t\tDebayerParams::CcmColumn &blue = blue_.ccm[b_];   \\\n>> +\t\tDebayerParams::CcmColumn &green = green_.ccm[g_]; \\\n>> +\t\tDebayerParams::CcmColumn &red = red_.ccm[r_];     \\\n>\n> const\n\nAck.\n\n>> +\t\tGAMMA(blue.r + blue.g + blue.b);                  \\\n>> +\t\tGAMMA(green.r + green.g + green.b);               \\\n>> +\t\tGAMMA(red.r + red.g + red.b);                     \\\n>> +\t} else {                                                  \\\n>> +\t\t*dst++ = blue_.simple[b_];                        \\\n>> +\t\t*dst++ = green_.simple[g_];                       \\\n>> +\t\t*dst++ = red_.simple[r_];                         \\\n>> +\t}                                                         \\\n>> +\tif constexpr (addAlphaByte)                               \\\n>> +\t\t*dst++ = 255;                                     \\\n>>  \tx++;\n>>  \n>>  /*\n>> @@ -752,6 +769,7 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output\n>>  \tgreen_ = params.green;\n>>  \tred_ = swapRedBlueGains_ ? params.blue : params.red;\n>>  \tblue_ = swapRedBlueGains_ ? params.red : params.blue;\n>> +\tgammaLut_ = params.gammaLut;\n>>  \n>>  \t/* Copy metadata from the input buffer */\n>>  \tFrameMetadata &metadata = output->_d()->metadata();\n>> diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h\n>> index b2ec8f1b..35c51e4f 100644\n>> --- a/src/libcamera/software_isp/debayer_cpu.h\n>> +++ b/src/libcamera/software_isp/debayer_cpu.h\n>> @@ -137,9 +137,10 @@ private:\n>>  \t/* Max. supported Bayer pattern height is 4, debayering this requires 5 lines */\n>>  \tstatic constexpr unsigned int kMaxLineBuffers = 5;\n>>  \n>> -\tDebayerParams::ColorLookupTable red_;\n>> -\tDebayerParams::ColorLookupTable green_;\n>> -\tDebayerParams::ColorLookupTable blue_;\n>> +\tDebayerParams::LookupTable red_;\n>> +\tDebayerParams::LookupTable green_;\n>> +\tDebayerParams::LookupTable blue_;\n>> +\tDebayerParams::GammaLookupTable gammaLut_;\n>>  \tdebayerFn debayer0_;\n>>  \tdebayerFn debayer1_;\n>>  \tdebayerFn debayer2_;\n>> diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp\n>> index 6a07de85..40e11b9e 100644\n>> --- a/src/libcamera/software_isp/software_isp.cpp\n>> +++ b/src/libcamera/software_isp/software_isp.cpp\n>> @@ -82,9 +82,9 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor,\n>>  \tfor (unsigned int i = 0; i < 256; i++)\n>>  \t\tgammaTable[i] = UINT8_MAX * std::pow(i / 256.0, 0.5);\n>>  \tfor (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) {\n>> -\t\tdebayerParams_.red[i] = gammaTable[i];\n>> -\t\tdebayerParams_.green[i] = gammaTable[i];\n>> -\t\tdebayerParams_.blue[i] = gammaTable[i];\n>> +\t\tdebayerParams_.red.simple[i] = gammaTable[i];\n>> +\t\tdebayerParams_.green.simple[i] = gammaTable[i];\n>> +\t\tdebayerParams_.blue.simple[i] = gammaTable[i];\n>>  \t}\n>>  \n>>  \tif (!dmaHeap_.isValid()) {","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 53CA7BD87C\n\tfor <parsemail@patchwork.libcamera.org>;\n\tTue, 28 Jan 2025 15:37:13 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 207B46855D;\n\tTue, 28 Jan 2025 16:37:12 +0100 (CET)","from us-smtp-delivery-124.mimecast.com\n\t(us-smtp-delivery-124.mimecast.com [170.10.129.124])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id AA47168546\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tTue, 28 Jan 2025 16:37:09 +0100 (CET)","from mail-wm1-f72.google.com (mail-wm1-f72.google.com\n\t[209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS\n\t(version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n\tus-mta-577-BgvhB4voPzmJqvtrCK9jNw-1; Tue, 28 Jan 2025 10:37:06 -0500","by mail-wm1-f72.google.com with SMTP id\n\t5b1f17b1804b1-436723db6c4so40391605e9.3\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tTue, 28 Jan 2025 07:37:06 -0800 (PST)","from mzamazal-thinkpadp1gen3.tpbc.csb (nat-pool-brq-t.redhat.com.\n\t[213.175.37.10]) by smtp.gmail.com with ESMTPSA id\n\t5b1f17b1804b1-438bd4857cesm170176345e9.13.2025.01.28.07.37.02\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tTue, 28 Jan 2025 07:37:02 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=redhat.com header.i=@redhat.com\n\theader.b=\"PK4nb6QY\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n\ts=mimecast20190719; t=1738078628;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tin-reply-to:in-reply-to:references:references;\n\tbh=uFTs5mFMvP25M0ZqGdHBFJQ8AulQDU8e5T0FlIZ2rfE=;\n\tb=PK4nb6QYAOF3ZHVOUiqgwZkFr/+BxSqX8iAryCLs4EJ6GJPMri2Vh+kbdMxeXD875LbTd4\n\tiSO2w8TTmPtDHDtrtsGzY3GsMbByHyFxs5r47zQqUK9Zv1zbVMlUn/+PvcjLrznbuYEgY8\n\t5QB6Ync4YVaJ2V78+WkhA0LlD3In1cM=","X-MC-Unique":"BgvhB4voPzmJqvtrCK9jNw-1","X-Mimecast-MFC-AGG-ID":"BgvhB4voPzmJqvtrCK9jNw","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1738078625; x=1738683425;\n\th=mime-version:user-agent:message-id:date:references:in-reply-to\n\t:subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=uFTs5mFMvP25M0ZqGdHBFJQ8AulQDU8e5T0FlIZ2rfE=;\n\tb=qE0v/eMW5cqhxRB3Tbr58YLL8BeDhiNUj1GSWaw+RbEb7HKeHoyPv2fJbNZk9J0awT\n\tvCmi36Gd7mie37x6vpjuNVY4Z5SlfMzcp672rq9jfF8+UeCLxTYFrzbG1abP+0o80LvO\n\tKr7CzqAxMjVCEYN1+xJxhwDrTWHMGAb8qGfCQYKBv2UZFiIz3QA6rqQmrNoBk4/e+z7A\n\tbdaG7gpoQXarVSaKmw+QZMG2Fljzsk8zcAAquDugAxCq2X0oKMGIFAW+tu0O7BtW7pwz\n\tfzw2Kbooi3RT/3x96KEfW9SybppeJP098WT5oxOoPf/vGd8TrUR2eWcpc0W0LBxVZVJZ\n\t0bfw==","X-Gm-Message-State":"AOJu0YyYxX3uDOIcVM05khXjAX6k45dKpsTpKsHfXMANijVgXnfUT9hP\n\tvaNrYpcv259/IxgFPRA7Pv3I1ziH3Co5f2GbaB29zCbgp6nb3M87wfgK8ieuMOFa5tcYDTQxdDa\n\t333bzjiujXTF/gVfpgzt8XzkySVRcJSqhFNDVDyoLCgE8/y9gG8gCbhyFtGokVh9Hc4Z9qO0=","X-Gm-Gg":"ASbGncvdwMaNbVRbohn8GCwstm6Rn3G1IgTuFy0ct4V575qg5/bYsyXaJZdEbJwQnlz\n\tzCq7Q7OCoYk//nUDqpcgjdD5XZFMGIs3NPYrzGGRam6F2J8owG2AeX9rjJa+9lNCCQL9srbB7Ll\n\tNBlxmYWwEPEvrdmM5qC/Xfqr9sdLDEhB5+BhINlQs1wugdB6Hxw2iMhGdVdB1D3KC1jAAjd5dYg\n\tX5OXM69F965I3qMbp3rm7eeHHzC+gxgDvIALTlO5iswQn8ZEvSiU6gyaNUif91aLuijuQDV2n2U\n\tIt/O1FTwmD3VxPGpiekMa5BqQEK/mrW2MUR+6559klcnog6NvEZ6q8y/","X-Received":["by 2002:a05:600c:5486:b0:434:f586:753c with SMTP id\n\t5b1f17b1804b1-438913ca694mr417188525e9.7.1738078625251; \n\tTue, 28 Jan 2025 07:37:05 -0800 (PST)","by 2002:a05:600c:5486:b0:434:f586:753c with SMTP id\n\t5b1f17b1804b1-438913ca694mr417187135e9.7.1738078623262; \n\tTue, 28 Jan 2025 07:37:03 -0800 (PST)"],"X-Google-Smtp-Source":"AGHT+IFGJq14KLm1U60IeWgT3PJI3GKixmbloEQ9l/4OQ71C/ZKVKhXz7yHDnlcl4bPoqZzLUB4JIA==","From":"Milan Zamazal <mzamazal@redhat.com>","To":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","Cc":"libcamera-devel@lists.libcamera.org,  Robert Mader\n\t<robert.mader@collabora.com>,  Hans de Goede <hdegoede@redhat.com>,\n\tKieran Bingham <kieran.bingham@ideasonboard.com>","Subject":"Re: [PATCH v4 9/9] libcamera: software_isp: Apply CCM in debayering","In-Reply-To":"<20250127015253.GD17899@pendragon.ideasonboard.com> (Laurent\n\tPinchart's message of \"Mon, 27 Jan 2025 03:52:53 +0200\")","References":"<20250113135108.13924-1-mzamazal@redhat.com>\n\t<20250113135108.13924-10-mzamazal@redhat.com>\n\t<20250127015253.GD17899@pendragon.ideasonboard.com>","Date":"Tue, 28 Jan 2025 16:37:01 +0100","Message-ID":"<85a5bbdk6a.fsf@mzamazal-thinkpadp1gen3.tpbc.csb>","User-Agent":"Gnus/5.13 (Gnus v5.13)","MIME-Version":"1.0","X-Mimecast-Spam-Score":"0","X-Mimecast-MFC-PROC-ID":"oEzsqoDrRH3tqiUvfAQ4u-G-fp4PtmhEd_P-AjFU1Rs_1738078625","X-Mimecast-Originator":"redhat.com","Content-Type":"text/plain","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}}]