Message ID | 20250326090849.15494-11-mzamazal@redhat.com |
---|---|
State | Accepted |
Headers | show |
Series |
|
Related | show |
Quoting Milan Zamazal (2025-03-26 09:08:47) > This patch applies color correction matrix (CCM) in debayering if the > CCM is specified. Not using CCM must still be supported for performance > reasons. > > The CCM is applied as follows: > > [r1 g1 b1] [r] > [r2 g2 b2] * [g] > [r3 g3 b3] [b] > > The CCM matrix (the left side of the multiplication) is constant during > single frame processing, while the input pixel (the right side) changes. > Because each of the color channels is only 8-bit in software ISP, we can > make 9 lookup tables with 256 input values for multiplications of each > of the r_i, g_i, b_i values. This way we don't have to multiply each > pixel, we can use table lookups and additions instead. Gamma (which is > non-linear and thus cannot be a part of the 9 lookup tables values) is > applied on the final values rounded to integers using another lookup > table. > > Because the changing part is the pixel value with three color elements, > only three dynamic table lookups are needed. We use three lookup tables > to represent the multiplied matrix values, each of the tables > corresponding to the given matrix column and pixel color. > > We use int16_t to store the precomputed multiplications. This seems to > be noticeably (>10%) faster than `float' for the price of slightly less > accuracy and it covers the range of values that sane CCMs produce. The > selection and structure of data is performance critical, for example > using bytes would add significant (>10%) speedup but would be too short > to cover the value range. > > The color lookup tables can be represented either as unions, > accommodating tables for both the CCM and non-CCM cases, or as separate > tables for each of the cases, leaving the tables for the other case > unused. The latter is selected as a matter of preference. > > The tables are copied (as before), which is not elegant but also not a > big problem. There are patches posted that use shared buffers for > parameters passing in software ISP (see software ISP TODO #5) and they > can be adjusted for the new parameter format. > > Color gains from white balance are supposed not to be a part of the > specified CCM. They are applied on it using matrix multiplication, > which is simple and in correspondence with future additions in the form > of matrix multiplication, like saturation adjustment. > > With this patch, the reported per-frame slowdown when applying CCM is > about 45% on Debix Model A and about 75% on TI AM69 SK. > > Using std::clamp in debayering adds some performance penalty (a few > percent). The clamping is necessary to eliminate out of range values > possibly produced by the CCM. If it could be avoided by adjusting the > precomputed tables some way then performance could be improved a bit. > > Signed-off-by: Milan Zamazal <mzamazal@redhat.com> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> > --- > .../internal/software_isp/debayer_params.h | 36 ++++++++++-- > src/ipa/simple/algorithms/lut.cpp | 53 +++++++++++++---- > src/ipa/simple/algorithms/lut.h | 1 + > src/libcamera/software_isp/debayer.cpp | 57 ++++++++++++++++++- > src/libcamera/software_isp/debayer_cpu.cpp | 53 +++++++++++++---- > src/libcamera/software_isp/debayer_cpu.h | 10 +++- > 6 files changed, 177 insertions(+), 33 deletions(-) > > diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h > index 7d8fdd48..217cd5d9 100644 > --- a/include/libcamera/internal/software_isp/debayer_params.h > +++ b/include/libcamera/internal/software_isp/debayer_params.h > @@ -1,6 +1,6 @@ > /* SPDX-License-Identifier: LGPL-2.1-or-later */ > /* > - * Copyright (C) 2023, 2024 Red Hat Inc. > + * Copyright (C) 2023-2025 Red Hat Inc. > * > * Authors: > * Hans de Goede <hdegoede@redhat.com> > @@ -18,11 +18,37 @@ namespace libcamera { > struct DebayerParams { > static constexpr unsigned int kRGBLookupSize = 256; > > - using ColorLookupTable = std::array<uint8_t, kRGBLookupSize>; > + struct CcmColumn { > + int16_t r; > + int16_t g; > + int16_t b; > + }; > > - ColorLookupTable red; > - ColorLookupTable green; > - ColorLookupTable blue; > + using LookupTable = std::array<uint8_t, kRGBLookupSize>; > + using CcmLookupTable = std::array<CcmColumn, kRGBLookupSize>; > + > + /* > + * Color lookup tables when CCM is not used. > + * > + * Each color of a debayered pixel is amended by the corresponding > + * value in the given table. > + */ > + LookupTable red; > + LookupTable green; > + LookupTable blue; > + > + /* > + * Color and gamma lookup tables when CCM is used. > + * > + * Each of the CcmLookupTable's corresponds to a CCM column; together they > + * make a complete 3x3 CCM lookup table. The CCM is applied on debayered > + * pixels and then the gamma lookup table is used to set the resulting > + * values of all the three colors. > + */ > + CcmLookupTable redCcm; > + CcmLookupTable greenCcm; > + CcmLookupTable blueCcm; > + LookupTable gammaLut; > }; > > } /* namespace libcamera */ > diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp > index 3a3daed7..a06cdeba 100644 > --- a/src/ipa/simple/algorithms/lut.cpp > +++ b/src/ipa/simple/algorithms/lut.cpp > @@ -1,6 +1,6 @@ > /* SPDX-License-Identifier: LGPL-2.1-or-later */ > /* > - * Copyright (C) 2024, Red Hat Inc. > + * Copyright (C) 2024-2025, Red Hat Inc. > * > * Color lookup tables construction > */ > @@ -80,6 +80,11 @@ void Lut::updateGammaTable(IPAContext &context) > context.activeState.gamma.contrast = contrast; > } > > +int16_t Lut::ccmValue(unsigned int i, float ccm) const > +{ > + return std::round(i * ccm); > +} > + > void Lut::prepare(IPAContext &context, > [[maybe_unused]] const uint32_t frame, > [[maybe_unused]] IPAFrameContext &frameContext, > @@ -91,22 +96,46 @@ void Lut::prepare(IPAContext &context, > * observed, it's not permanently prone to minor fluctuations or > * rounding errors. > */ > - if (context.activeState.gamma.blackLevel != context.activeState.blc.level || > - context.activeState.gamma.contrast != context.activeState.knobs.contrast) > + const bool gammaUpdateNeeded = > + context.activeState.gamma.blackLevel != context.activeState.blc.level || > + context.activeState.gamma.contrast != context.activeState.knobs.contrast; > + if (gammaUpdateNeeded) > updateGammaTable(context); > > auto &gains = context.activeState.awb.gains; > auto &gammaTable = context.activeState.gamma.gammaTable; > const unsigned int gammaTableSize = gammaTable.size(); > - > - for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { > - const double div = static_cast<double>(DebayerParams::kRGBLookupSize) / > - gammaTableSize; > - /* Apply gamma after gain! */ > - const RGB<float> lutGains = (gains * i / div).min(gammaTableSize - 1); > - params->red[i] = gammaTable[static_cast<unsigned int>(lutGains.r())]; > - params->green[i] = gammaTable[static_cast<unsigned int>(lutGains.g())]; > - params->blue[i] = gammaTable[static_cast<unsigned int>(lutGains.b())]; > + const double div = static_cast<double>(DebayerParams::kRGBLookupSize) / > + gammaTableSize; > + > + if (!context.ccmEnabled) { > + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { > + /* Apply gamma after gain! */ > + const RGB<float> lutGains = (gains * i / div).min(gammaTableSize - 1); > + params->red[i] = gammaTable[static_cast<unsigned int>(lutGains.r())]; > + params->green[i] = gammaTable[static_cast<unsigned int>(lutGains.g())]; > + params->blue[i] = gammaTable[static_cast<unsigned int>(lutGains.b())]; > + } > + } else if (context.activeState.ccm.changed || gammaUpdateNeeded) { > + Matrix<float, 3, 3> gainCcm = { { gains.r(), 0, 0, > + 0, gains.g(), 0, > + 0, 0, gains.b() } }; > + auto ccm = gainCcm * context.activeState.ccm.ccm; > + auto &red = params->redCcm; > + auto &green = params->greenCcm; > + auto &blue = params->blueCcm; > + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { > + red[i].r = ccmValue(i, ccm[0][0]); > + red[i].g = ccmValue(i, ccm[1][0]); > + red[i].b = ccmValue(i, ccm[2][0]); > + green[i].r = ccmValue(i, ccm[0][1]); > + green[i].g = ccmValue(i, ccm[1][1]); > + green[i].b = ccmValue(i, ccm[2][1]); > + blue[i].r = ccmValue(i, ccm[0][2]); > + blue[i].g = ccmValue(i, ccm[1][2]); > + blue[i].b = ccmValue(i, ccm[2][2]); > + params->gammaLut[i] = gammaTable[i / div]; > + } > } > } > > diff --git a/src/ipa/simple/algorithms/lut.h b/src/ipa/simple/algorithms/lut.h > index 889f864b..77324800 100644 > --- a/src/ipa/simple/algorithms/lut.h > +++ b/src/ipa/simple/algorithms/lut.h > @@ -33,6 +33,7 @@ public: > > private: > void updateGammaTable(IPAContext &context); > + int16_t ccmValue(unsigned int i, float ccm) const; > }; > > } /* namespace ipa::soft::algorithms */ > diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp > index 34e42201..e9e18c48 100644 > --- a/src/libcamera/software_isp/debayer.cpp > +++ b/src/libcamera/software_isp/debayer.cpp > @@ -1,7 +1,7 @@ > /* SPDX-License-Identifier: LGPL-2.1-or-later */ > /* > * Copyright (C) 2023, Linaro Ltd > - * Copyright (C) 2023, 2024 Red Hat Inc. > + * Copyright (C) 2023-2025 Red Hat Inc. > * > * Authors: > * Hans de Goede <hdegoede@redhat.com> > @@ -24,8 +24,39 @@ namespace libcamera { > */ > > /** > - * \typedef DebayerParams::ColorLookupTable > - * \brief Type of the lookup tables for red, green, blue values > + * \struct DebayerParams::CcmColumn > + * \brief Type of a single column of a color correction matrix (CCM) > + * > + * When multiplying an input pixel, columns in the CCM correspond to the red, > + * green or blue component of input pixel values, while rows correspond to the > + * red, green or blue components of the output pixel values. The members of the > + * CcmColumn structure are named after the colour components of the output pixel > + * values they correspond to. > + */ > + > +/** > + * \var DebayerParams::CcmColumn::r > + * \brief Red (first) component of a CCM column > + */ > + > +/** > + * \var DebayerParams::CcmColumn::g > + * \brief Green (second) component of a CCM column > + */ > + > +/** > + * \var DebayerParams::CcmColumn::b > + * \brief Blue (third) component of a CCM column > + */ > + > +/** > + * \typedef DebayerParams::LookupTable > + * \brief Type of the lookup tables for single lookup values > + */ > + > +/** > + * \typedef DebayerParams::CcmLookupTable > + * \brief Type of the CCM lookup tables for red, green, blue values > */ > > /** > @@ -43,6 +74,26 @@ namespace libcamera { > * \brief Lookup table for blue color, mapping input values to output values > */ > > +/** > + * \var DebayerParams::redCcm > + * \brief Lookup table for the CCM red column, mapping input values to output values > + */ > + > +/** > + * \var DebayerParams::greenCcm > + * \brief Lookup table for the CCM green column, mapping input values to output values > + */ > + > +/** > + * \var DebayerParams::blueCcm > + * \brief Lookup table for the CCM blue column, mapping input values to output values > + */ > + > +/** > + * \var DebayerParams::gammaLut > + * \brief Gamma lookup table used with color correction matrix > + */ > + > /** > * \class Debayer > * \brief Base debayering class > diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp > index 0cd03a8f..66f6038c 100644 > --- a/src/libcamera/software_isp/debayer_cpu.cpp > +++ b/src/libcamera/software_isp/debayer_cpu.cpp > @@ -1,7 +1,7 @@ > /* SPDX-License-Identifier: LGPL-2.1-or-later */ > /* > * Copyright (C) 2023, Linaro Ltd > - * Copyright (C) 2023, Red Hat Inc. > + * Copyright (C) 2023-2025 Red Hat Inc. > * > * Authors: > * Hans de Goede <hdegoede@redhat.com> > @@ -11,9 +11,11 @@ > > #include "debayer_cpu.h" > > +#include <algorithm> > #include <stdlib.h> > #include <sys/ioctl.h> > #include <time.h> > +#include <utility> > > #include <linux/dma-buf.h> > > @@ -51,8 +53,12 @@ DebayerCpu::DebayerCpu(std::unique_ptr<SwStatsCpu> stats) > enableInputMemcpy_ = true; > > /* Initialize color lookup tables */ > - for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) > + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { > red_[i] = green_[i] = blue_[i] = i; > + redCcm_[i] = { static_cast<int16_t>(i), 0, 0 }; > + greenCcm_[i] = { 0, static_cast<int16_t>(i), 0 }; > + blueCcm_[i] = { 0, 0, static_cast<int16_t>(i) }; > + } > } > > DebayerCpu::~DebayerCpu() = default; > @@ -62,12 +68,24 @@ DebayerCpu::~DebayerCpu() = default; > const pixel_t *curr = (const pixel_t *)src[1] + xShift_; \ > const pixel_t *next = (const pixel_t *)src[2] + xShift_; > > -#define STORE_PIXEL(b, g, r) \ > - *dst++ = blue_[b]; \ > - *dst++ = green_[g]; \ > - *dst++ = red_[r]; \ > - if constexpr (addAlphaByte) \ > - *dst++ = 255; \ > +#define GAMMA(value) \ > + *dst++ = gammaLut_[std::clamp(value, 0, static_cast<int>(gammaLut_.size()) - 1)] > + > +#define STORE_PIXEL(b_, g_, r_) \ > + if constexpr (ccmEnabled) { \ > + const DebayerParams::CcmColumn &blue = blueCcm_[b_]; \ > + const DebayerParams::CcmColumn &green = greenCcm_[g_]; \ > + const DebayerParams::CcmColumn &red = redCcm_[r_]; \ > + GAMMA(blue.b + green.b + red.b); \ > + GAMMA(blue.g + green.g + red.g); \ > + GAMMA(blue.r + green.r + red.r); \ > + } else { \ > + *dst++ = blue_[b_]; \ > + *dst++ = green_[g_]; \ > + *dst++ = red_[r_]; \ > + } \ > + if constexpr (addAlphaByte) \ > + *dst++ = 255; \ > x++; > > /* > @@ -755,8 +773,23 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output > dmaSyncers.emplace_back(plane.fd, DmaSyncer::SyncType::Write); > > green_ = params.green; > - red_ = swapRedBlueGains_ ? params.blue : params.red; > - blue_ = swapRedBlueGains_ ? params.red : params.blue; > + greenCcm_ = params.greenCcm; > + if (swapRedBlueGains_) { > + red_ = params.blue; > + blue_ = params.red; > + redCcm_ = params.blueCcm; > + blueCcm_ = params.redCcm; > + for (unsigned int i = 0; i < 256; i++) { > + std::swap(redCcm_[i].r, redCcm_[i].b); > + std::swap(blueCcm_[i].r, blueCcm_[i].b); > + } > + } else { > + red_ = params.red; > + blue_ = params.blue; > + redCcm_ = params.redCcm; > + blueCcm_ = params.blueCcm; > + } > + gammaLut_ = params.gammaLut; > > /* Copy metadata from the input buffer */ > FrameMetadata &metadata = output->_d()->metadata(); > diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h > index 21c08a2d..926195e9 100644 > --- a/src/libcamera/software_isp/debayer_cpu.h > +++ b/src/libcamera/software_isp/debayer_cpu.h > @@ -138,9 +138,13 @@ private: > /* Max. supported Bayer pattern height is 4, debayering this requires 5 lines */ > static constexpr unsigned int kMaxLineBuffers = 5; > > - DebayerParams::ColorLookupTable red_; > - DebayerParams::ColorLookupTable green_; > - DebayerParams::ColorLookupTable blue_; > + DebayerParams::LookupTable red_; > + DebayerParams::LookupTable green_; > + DebayerParams::LookupTable blue_; > + DebayerParams::CcmLookupTable redCcm_; > + DebayerParams::CcmLookupTable greenCcm_; > + DebayerParams::CcmLookupTable blueCcm_; > + DebayerParams::LookupTable gammaLut_; > debayerFn debayer0_; > debayerFn debayer1_; > debayerFn debayer2_; > -- > 2.49.0 >
diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index 7d8fdd48..217cd5d9 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2023, 2024 Red Hat Inc. + * Copyright (C) 2023-2025 Red Hat Inc. * * Authors: * Hans de Goede <hdegoede@redhat.com> @@ -18,11 +18,37 @@ namespace libcamera { struct DebayerParams { static constexpr unsigned int kRGBLookupSize = 256; - using ColorLookupTable = std::array<uint8_t, kRGBLookupSize>; + struct CcmColumn { + int16_t r; + int16_t g; + int16_t b; + }; - ColorLookupTable red; - ColorLookupTable green; - ColorLookupTable blue; + using LookupTable = std::array<uint8_t, kRGBLookupSize>; + using CcmLookupTable = std::array<CcmColumn, kRGBLookupSize>; + + /* + * Color lookup tables when CCM is not used. + * + * Each color of a debayered pixel is amended by the corresponding + * value in the given table. + */ + LookupTable red; + LookupTable green; + LookupTable blue; + + /* + * Color and gamma lookup tables when CCM is used. + * + * Each of the CcmLookupTable's corresponds to a CCM column; together they + * make a complete 3x3 CCM lookup table. The CCM is applied on debayered + * pixels and then the gamma lookup table is used to set the resulting + * values of all the three colors. + */ + CcmLookupTable redCcm; + CcmLookupTable greenCcm; + CcmLookupTable blueCcm; + LookupTable gammaLut; }; } /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp index 3a3daed7..a06cdeba 100644 --- a/src/ipa/simple/algorithms/lut.cpp +++ b/src/ipa/simple/algorithms/lut.cpp @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2024, Red Hat Inc. + * Copyright (C) 2024-2025, Red Hat Inc. * * Color lookup tables construction */ @@ -80,6 +80,11 @@ void Lut::updateGammaTable(IPAContext &context) context.activeState.gamma.contrast = contrast; } +int16_t Lut::ccmValue(unsigned int i, float ccm) const +{ + return std::round(i * ccm); +} + void Lut::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameContext, @@ -91,22 +96,46 @@ void Lut::prepare(IPAContext &context, * observed, it's not permanently prone to minor fluctuations or * rounding errors. */ - if (context.activeState.gamma.blackLevel != context.activeState.blc.level || - context.activeState.gamma.contrast != context.activeState.knobs.contrast) + const bool gammaUpdateNeeded = + context.activeState.gamma.blackLevel != context.activeState.blc.level || + context.activeState.gamma.contrast != context.activeState.knobs.contrast; + if (gammaUpdateNeeded) updateGammaTable(context); auto &gains = context.activeState.awb.gains; auto &gammaTable = context.activeState.gamma.gammaTable; const unsigned int gammaTableSize = gammaTable.size(); - - for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { - const double div = static_cast<double>(DebayerParams::kRGBLookupSize) / - gammaTableSize; - /* Apply gamma after gain! */ - const RGB<float> lutGains = (gains * i / div).min(gammaTableSize - 1); - params->red[i] = gammaTable[static_cast<unsigned int>(lutGains.r())]; - params->green[i] = gammaTable[static_cast<unsigned int>(lutGains.g())]; - params->blue[i] = gammaTable[static_cast<unsigned int>(lutGains.b())]; + const double div = static_cast<double>(DebayerParams::kRGBLookupSize) / + gammaTableSize; + + if (!context.ccmEnabled) { + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { + /* Apply gamma after gain! */ + const RGB<float> lutGains = (gains * i / div).min(gammaTableSize - 1); + params->red[i] = gammaTable[static_cast<unsigned int>(lutGains.r())]; + params->green[i] = gammaTable[static_cast<unsigned int>(lutGains.g())]; + params->blue[i] = gammaTable[static_cast<unsigned int>(lutGains.b())]; + } + } else if (context.activeState.ccm.changed || gammaUpdateNeeded) { + Matrix<float, 3, 3> gainCcm = { { gains.r(), 0, 0, + 0, gains.g(), 0, + 0, 0, gains.b() } }; + auto ccm = gainCcm * context.activeState.ccm.ccm; + auto &red = params->redCcm; + auto &green = params->greenCcm; + auto &blue = params->blueCcm; + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { + red[i].r = ccmValue(i, ccm[0][0]); + red[i].g = ccmValue(i, ccm[1][0]); + red[i].b = ccmValue(i, ccm[2][0]); + green[i].r = ccmValue(i, ccm[0][1]); + green[i].g = ccmValue(i, ccm[1][1]); + green[i].b = ccmValue(i, ccm[2][1]); + blue[i].r = ccmValue(i, ccm[0][2]); + blue[i].g = ccmValue(i, ccm[1][2]); + blue[i].b = ccmValue(i, ccm[2][2]); + params->gammaLut[i] = gammaTable[i / div]; + } } } diff --git a/src/ipa/simple/algorithms/lut.h b/src/ipa/simple/algorithms/lut.h index 889f864b..77324800 100644 --- a/src/ipa/simple/algorithms/lut.h +++ b/src/ipa/simple/algorithms/lut.h @@ -33,6 +33,7 @@ public: private: void updateGammaTable(IPAContext &context); + int16_t ccmValue(unsigned int i, float ccm) const; }; } /* namespace ipa::soft::algorithms */ diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index 34e42201..e9e18c48 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* * Copyright (C) 2023, Linaro Ltd - * Copyright (C) 2023, 2024 Red Hat Inc. + * Copyright (C) 2023-2025 Red Hat Inc. * * Authors: * Hans de Goede <hdegoede@redhat.com> @@ -24,8 +24,39 @@ namespace libcamera { */ /** - * \typedef DebayerParams::ColorLookupTable - * \brief Type of the lookup tables for red, green, blue values + * \struct DebayerParams::CcmColumn + * \brief Type of a single column of a color correction matrix (CCM) + * + * When multiplying an input pixel, columns in the CCM correspond to the red, + * green or blue component of input pixel values, while rows correspond to the + * red, green or blue components of the output pixel values. The members of the + * CcmColumn structure are named after the colour components of the output pixel + * values they correspond to. + */ + +/** + * \var DebayerParams::CcmColumn::r + * \brief Red (first) component of a CCM column + */ + +/** + * \var DebayerParams::CcmColumn::g + * \brief Green (second) component of a CCM column + */ + +/** + * \var DebayerParams::CcmColumn::b + * \brief Blue (third) component of a CCM column + */ + +/** + * \typedef DebayerParams::LookupTable + * \brief Type of the lookup tables for single lookup values + */ + +/** + * \typedef DebayerParams::CcmLookupTable + * \brief Type of the CCM lookup tables for red, green, blue values */ /** @@ -43,6 +74,26 @@ namespace libcamera { * \brief Lookup table for blue color, mapping input values to output values */ +/** + * \var DebayerParams::redCcm + * \brief Lookup table for the CCM red column, mapping input values to output values + */ + +/** + * \var DebayerParams::greenCcm + * \brief Lookup table for the CCM green column, mapping input values to output values + */ + +/** + * \var DebayerParams::blueCcm + * \brief Lookup table for the CCM blue column, mapping input values to output values + */ + +/** + * \var DebayerParams::gammaLut + * \brief Gamma lookup table used with color correction matrix + */ + /** * \class Debayer * \brief Base debayering class diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 0cd03a8f..66f6038c 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* * Copyright (C) 2023, Linaro Ltd - * Copyright (C) 2023, Red Hat Inc. + * Copyright (C) 2023-2025 Red Hat Inc. * * Authors: * Hans de Goede <hdegoede@redhat.com> @@ -11,9 +11,11 @@ #include "debayer_cpu.h" +#include <algorithm> #include <stdlib.h> #include <sys/ioctl.h> #include <time.h> +#include <utility> #include <linux/dma-buf.h> @@ -51,8 +53,12 @@ DebayerCpu::DebayerCpu(std::unique_ptr<SwStatsCpu> stats) enableInputMemcpy_ = true; /* Initialize color lookup tables */ - for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { red_[i] = green_[i] = blue_[i] = i; + redCcm_[i] = { static_cast<int16_t>(i), 0, 0 }; + greenCcm_[i] = { 0, static_cast<int16_t>(i), 0 }; + blueCcm_[i] = { 0, 0, static_cast<int16_t>(i) }; + } } DebayerCpu::~DebayerCpu() = default; @@ -62,12 +68,24 @@ DebayerCpu::~DebayerCpu() = default; const pixel_t *curr = (const pixel_t *)src[1] + xShift_; \ const pixel_t *next = (const pixel_t *)src[2] + xShift_; -#define STORE_PIXEL(b, g, r) \ - *dst++ = blue_[b]; \ - *dst++ = green_[g]; \ - *dst++ = red_[r]; \ - if constexpr (addAlphaByte) \ - *dst++ = 255; \ +#define GAMMA(value) \ + *dst++ = gammaLut_[std::clamp(value, 0, static_cast<int>(gammaLut_.size()) - 1)] + +#define STORE_PIXEL(b_, g_, r_) \ + if constexpr (ccmEnabled) { \ + const DebayerParams::CcmColumn &blue = blueCcm_[b_]; \ + const DebayerParams::CcmColumn &green = greenCcm_[g_]; \ + const DebayerParams::CcmColumn &red = redCcm_[r_]; \ + GAMMA(blue.b + green.b + red.b); \ + GAMMA(blue.g + green.g + red.g); \ + GAMMA(blue.r + green.r + red.r); \ + } else { \ + *dst++ = blue_[b_]; \ + *dst++ = green_[g_]; \ + *dst++ = red_[r_]; \ + } \ + if constexpr (addAlphaByte) \ + *dst++ = 255; \ x++; /* @@ -755,8 +773,23 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output dmaSyncers.emplace_back(plane.fd, DmaSyncer::SyncType::Write); green_ = params.green; - red_ = swapRedBlueGains_ ? params.blue : params.red; - blue_ = swapRedBlueGains_ ? params.red : params.blue; + greenCcm_ = params.greenCcm; + if (swapRedBlueGains_) { + red_ = params.blue; + blue_ = params.red; + redCcm_ = params.blueCcm; + blueCcm_ = params.redCcm; + for (unsigned int i = 0; i < 256; i++) { + std::swap(redCcm_[i].r, redCcm_[i].b); + std::swap(blueCcm_[i].r, blueCcm_[i].b); + } + } else { + red_ = params.red; + blue_ = params.blue; + redCcm_ = params.redCcm; + blueCcm_ = params.blueCcm; + } + gammaLut_ = params.gammaLut; /* Copy metadata from the input buffer */ FrameMetadata &metadata = output->_d()->metadata(); diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 21c08a2d..926195e9 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -138,9 +138,13 @@ private: /* Max. supported Bayer pattern height is 4, debayering this requires 5 lines */ static constexpr unsigned int kMaxLineBuffers = 5; - DebayerParams::ColorLookupTable red_; - DebayerParams::ColorLookupTable green_; - DebayerParams::ColorLookupTable blue_; + DebayerParams::LookupTable red_; + DebayerParams::LookupTable green_; + DebayerParams::LookupTable blue_; + DebayerParams::CcmLookupTable redCcm_; + DebayerParams::CcmLookupTable greenCcm_; + DebayerParams::CcmLookupTable blueCcm_; + DebayerParams::LookupTable gammaLut_; debayerFn debayer0_; debayerFn debayer1_; debayerFn debayer2_;