| Message ID | 20260114113016.25162-7-mzamazal@redhat.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series |
|
| Related | show |
2026. 01. 14. 12:30 keltezéssel, Milan Zamazal írta: > The combined matrix must be reset to the initial value before each frame > is prepared. This must be done outside algorithms because any of the > algorithms may be disabled while the matrix must be always initialised. > > Let's initialise the combined matrix to the identity matrix (which keeps > the pixel values unchanged) in software ISP just before calling > `prepare' on the algorithms. > > Matrix updates can no longer be skipped in ccm.cpp, otherwise the CCM > won't be applied if there is no temperature or saturation change. We > explicitly track whether the CCM has been set up completely rather than > relying on the frame number, to avoid missing the initialisation in case > the first frame is skipped due to some error. > > Signed-off-by: Milan Zamazal <mzamazal@redhat.com> > --- > src/ipa/simple/algorithms/ccm.cpp | 31 +++++++++++++++---------------- > src/ipa/simple/algorithms/ccm.h | 2 ++ > src/ipa/simple/soft_simple.cpp | 2 ++ > 3 files changed, 19 insertions(+), 16 deletions(-) > > [...] > diff --git a/src/ipa/simple/algorithms/ccm.h b/src/ipa/simple/algorithms/ccm.h > index 8279a3d59..19bf283af 100644 > --- a/src/ipa/simple/algorithms/ccm.h > +++ b/src/ipa/simple/algorithms/ccm.h > @@ -47,6 +47,8 @@ private: > unsigned int lastCt_; > std::optional<float> lastSaturation_; > Interpolator<Matrix<float, 3, 3>> ccm_; > + Matrix<float, 3, 3> currentCcm_; > + bool ccmAssigned_ = false; Could std::optional<Matrix<...>> be used here? > }; > > } /* namespace ipa::soft::algorithms */ > diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp > index 57836c73c..732e82510 100644 > --- a/src/ipa/simple/soft_simple.cpp > +++ b/src/ipa/simple/soft_simple.cpp > @@ -282,6 +282,8 @@ void IPASoftSimple::queueRequest(const uint32_t frame, const ControlList &contro > > void IPASoftSimple::computeParams(const uint32_t frame) > { > + context_.activeState.combinedMatrix = Matrix<float, 3, 3>::identity(); > + > IPAFrameContext &frameContext = context_.frameContexts.get(frame); > for (auto const &algo : algorithms()) > algo->prepare(context_, frame, frameContext, params_);
2026. 01. 14. 12:30 keltezéssel, Milan Zamazal írta: > The combined matrix must be reset to the initial value before each frame > is prepared. This must be done outside algorithms because any of the > algorithms may be disabled while the matrix must be always initialised. > > Let's initialise the combined matrix to the identity matrix (which keeps > the pixel values unchanged) in software ISP just before calling > `prepare' on the algorithms. > > Matrix updates can no longer be skipped in ccm.cpp, otherwise the CCM > won't be applied if there is no temperature or saturation change. We > explicitly track whether the CCM has been set up completely rather than > relying on the frame number, to avoid missing the initialisation in case > the first frame is skipped due to some error. > > Signed-off-by: Milan Zamazal <mzamazal@redhat.com> > --- > src/ipa/simple/algorithms/ccm.cpp | 31 +++++++++++++++---------------- > src/ipa/simple/algorithms/ccm.h | 2 ++ > src/ipa/simple/soft_simple.cpp | 2 ++ > 3 files changed, 19 insertions(+), 16 deletions(-) > > diff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp > index a3e8cd6c4..8992a8ee1 100644 > --- a/src/ipa/simple/algorithms/ccm.cpp > +++ b/src/ipa/simple/algorithms/ccm.cpp > @@ -83,7 +83,7 @@ void Ccm::applySaturation(Matrix<float, 3, 3> &ccm, float saturation) > ccm = ycbcr2rgb * saturationMatrix * rgb2ycbcr * ccm; > } > > -void Ccm::prepare(IPAContext &context, const uint32_t frame, > +void Ccm::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, > IPAFrameContext &frameContext, [[maybe_unused]] DebayerParams *params) > { > auto &saturation = context.activeState.knobs.saturation; > @@ -91,24 +91,23 @@ void Ccm::prepare(IPAContext &context, const uint32_t frame, > const unsigned int ct = context.activeState.awb.temperatureK; > > /* Change CCM only on saturation or bigger temperature changes. */ > - if (frame > 0 && > - utils::abs_diff(ct, lastCt_) < kTemperatureThreshold && > - saturation == lastSaturation_) { > - frameContext.ccm = context.activeState.ccm; > - return; > + if (!ccmAssigned_ || > + utils::abs_diff(ct, lastCt_) >= kTemperatureThreshold || > + saturation != lastSaturation_) { > + currentCcm_ = ccm_.getInterpolated(ct); > + ccmAssigned_ = true; > + if (saturation) > + applySaturation(currentCcm_, saturation.value()); > + lastCt_ = ct; > + lastSaturation_ = saturation; > + context.activeState.matrixChanged = true; > } > > - lastCt_ = ct; > - lastSaturation_ = saturation; > - Matrix<float, 3, 3> ccm = ccm_.getInterpolated(ct); > - if (saturation) > - applySaturation(ccm, saturation.value()); > - > - context.activeState.combinedMatrix = ccm; > - context.activeState.ccm = ccm; > + context.activeState.combinedMatrix = > + currentCcm_ * context.activeState.combinedMatrix; > + context.activeState.ccm = currentCcm_; It seems to me that after these changes `context.activeState.ccm` becomes essentially unused, can it be removed? Or I suppose one of `context.activeState.ccm` and `currentCcm_` appears to be unnecessary in the end. > frameContext.saturation = saturation; > - context.activeState.matrixChanged = true; > - frameContext.ccm = ccm; > + frameContext.ccm = currentCcm_; > } > [...]
Barnabás Pőcze <barnabas.pocze@ideasonboard.com> writes: > 2026. 01. 14. 12:30 keltezéssel, Milan Zamazal írta: >> The combined matrix must be reset to the initial value before each frame >> is prepared. This must be done outside algorithms because any of the >> algorithms may be disabled while the matrix must be always initialised. >> Let's initialise the combined matrix to the identity matrix (which keeps >> the pixel values unchanged) in software ISP just before calling >> `prepare' on the algorithms. >> Matrix updates can no longer be skipped in ccm.cpp, otherwise the CCM >> won't be applied if there is no temperature or saturation change. We >> explicitly track whether the CCM has been set up completely rather than >> relying on the frame number, to avoid missing the initialisation in case >> the first frame is skipped due to some error. >> Signed-off-by: Milan Zamazal <mzamazal@redhat.com> >> --- >> src/ipa/simple/algorithms/ccm.cpp | 31 +++++++++++++++---------------- >> src/ipa/simple/algorithms/ccm.h | 2 ++ >> src/ipa/simple/soft_simple.cpp | 2 ++ >> 3 files changed, 19 insertions(+), 16 deletions(-) >> [...] >> diff --git a/src/ipa/simple/algorithms/ccm.h b/src/ipa/simple/algorithms/ccm.h >> index 8279a3d59..19bf283af 100644 >> --- a/src/ipa/simple/algorithms/ccm.h >> +++ b/src/ipa/simple/algorithms/ccm.h >> @@ -47,6 +47,8 @@ private: >> unsigned int lastCt_; >> std::optional<float> lastSaturation_; >> Interpolator<Matrix<float, 3, 3>> ccm_; >> + Matrix<float, 3, 3> currentCcm_; >> + bool ccmAssigned_ = false; > > > Could std::optional<Matrix<...>> be used here? Yes, that's better, I'll change it. >> }; >> } /* namespace ipa::soft::algorithms */ >> diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp >> index 57836c73c..732e82510 100644 >> --- a/src/ipa/simple/soft_simple.cpp >> +++ b/src/ipa/simple/soft_simple.cpp >> @@ -282,6 +282,8 @@ void IPASoftSimple::queueRequest(const uint32_t frame, const ControlList &contro >> void IPASoftSimple::computeParams(const uint32_t frame) >> { >> + context_.activeState.combinedMatrix = Matrix<float, 3, 3>::identity(); >> + >> IPAFrameContext &frameContext = context_.frameContexts.get(frame); >> for (auto const &algo : algorithms()) >> algo->prepare(context_, frame, frameContext, params_);
Barnabás Pőcze <barnabas.pocze@ideasonboard.com> writes: > 2026. 01. 14. 12:30 keltezéssel, Milan Zamazal írta: >> The combined matrix must be reset to the initial value before each frame >> is prepared. This must be done outside algorithms because any of the >> algorithms may be disabled while the matrix must be always initialised. >> Let's initialise the combined matrix to the identity matrix (which keeps >> the pixel values unchanged) in software ISP just before calling >> `prepare' on the algorithms. >> Matrix updates can no longer be skipped in ccm.cpp, otherwise the CCM >> won't be applied if there is no temperature or saturation change. We >> explicitly track whether the CCM has been set up completely rather than >> relying on the frame number, to avoid missing the initialisation in case >> the first frame is skipped due to some error. >> Signed-off-by: Milan Zamazal <mzamazal@redhat.com> >> --- >> src/ipa/simple/algorithms/ccm.cpp | 31 +++++++++++++++---------------- >> src/ipa/simple/algorithms/ccm.h | 2 ++ >> src/ipa/simple/soft_simple.cpp | 2 ++ >> 3 files changed, 19 insertions(+), 16 deletions(-) >> diff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp >> index a3e8cd6c4..8992a8ee1 100644 >> --- a/src/ipa/simple/algorithms/ccm.cpp >> +++ b/src/ipa/simple/algorithms/ccm.cpp >> @@ -83,7 +83,7 @@ void Ccm::applySaturation(Matrix<float, 3, 3> &ccm, float saturation) >> ccm = ycbcr2rgb * saturationMatrix * rgb2ycbcr * ccm; >> } >> -void Ccm::prepare(IPAContext &context, const uint32_t frame, >> +void Ccm::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, >> IPAFrameContext &frameContext, [[maybe_unused]] DebayerParams *params) >> { >> auto &saturation = context.activeState.knobs.saturation; >> @@ -91,24 +91,23 @@ void Ccm::prepare(IPAContext &context, const uint32_t frame, >> const unsigned int ct = context.activeState.awb.temperatureK; >> /* Change CCM only on saturation or bigger temperature changes. */ >> - if (frame > 0 && >> - utils::abs_diff(ct, lastCt_) < kTemperatureThreshold && >> - saturation == lastSaturation_) { >> - frameContext.ccm = context.activeState.ccm; >> - return; >> + if (!ccmAssigned_ || >> + utils::abs_diff(ct, lastCt_) >= kTemperatureThreshold || >> + saturation != lastSaturation_) { >> + currentCcm_ = ccm_.getInterpolated(ct); >> + ccmAssigned_ = true; >> + if (saturation) >> + applySaturation(currentCcm_, saturation.value()); >> + lastCt_ = ct; >> + lastSaturation_ = saturation; >> + context.activeState.matrixChanged = true; >> } >> - lastCt_ = ct; >> - lastSaturation_ = saturation; >> - Matrix<float, 3, 3> ccm = ccm_.getInterpolated(ct); >> - if (saturation) >> - applySaturation(ccm, saturation.value()); >> - >> - context.activeState.combinedMatrix = ccm; >> - context.activeState.ccm = ccm; >> + context.activeState.combinedMatrix = >> + currentCcm_ * context.activeState.combinedMatrix; >> + context.activeState.ccm = currentCcm_; > > It seems to me that after these changes `context.activeState.ccm` becomes > essentially unused, can it be removed? Or I suppose one of `context.activeState.ccm` > and `currentCcm_` appears to be unnecessary in the end. Yes, I'll remove context.activeState.ccm. >> frameContext.saturation = saturation; >> - context.activeState.matrixChanged = true; >> - frameContext.ccm = ccm; >> + frameContext.ccm = currentCcm_; >> } >> [...]
diff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp index a3e8cd6c4..8992a8ee1 100644 --- a/src/ipa/simple/algorithms/ccm.cpp +++ b/src/ipa/simple/algorithms/ccm.cpp @@ -83,7 +83,7 @@ void Ccm::applySaturation(Matrix<float, 3, 3> &ccm, float saturation) ccm = ycbcr2rgb * saturationMatrix * rgb2ycbcr * ccm; } -void Ccm::prepare(IPAContext &context, const uint32_t frame, +void Ccm::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, IPAFrameContext &frameContext, [[maybe_unused]] DebayerParams *params) { auto &saturation = context.activeState.knobs.saturation; @@ -91,24 +91,23 @@ void Ccm::prepare(IPAContext &context, const uint32_t frame, const unsigned int ct = context.activeState.awb.temperatureK; /* Change CCM only on saturation or bigger temperature changes. */ - if (frame > 0 && - utils::abs_diff(ct, lastCt_) < kTemperatureThreshold && - saturation == lastSaturation_) { - frameContext.ccm = context.activeState.ccm; - return; + if (!ccmAssigned_ || + utils::abs_diff(ct, lastCt_) >= kTemperatureThreshold || + saturation != lastSaturation_) { + currentCcm_ = ccm_.getInterpolated(ct); + ccmAssigned_ = true; + if (saturation) + applySaturation(currentCcm_, saturation.value()); + lastCt_ = ct; + lastSaturation_ = saturation; + context.activeState.matrixChanged = true; } - lastCt_ = ct; - lastSaturation_ = saturation; - Matrix<float, 3, 3> ccm = ccm_.getInterpolated(ct); - if (saturation) - applySaturation(ccm, saturation.value()); - - context.activeState.combinedMatrix = ccm; - context.activeState.ccm = ccm; + context.activeState.combinedMatrix = + currentCcm_ * context.activeState.combinedMatrix; + context.activeState.ccm = currentCcm_; frameContext.saturation = saturation; - context.activeState.matrixChanged = true; - frameContext.ccm = ccm; + frameContext.ccm = currentCcm_; } void Ccm::process([[maybe_unused]] IPAContext &context, diff --git a/src/ipa/simple/algorithms/ccm.h b/src/ipa/simple/algorithms/ccm.h index 8279a3d59..19bf283af 100644 --- a/src/ipa/simple/algorithms/ccm.h +++ b/src/ipa/simple/algorithms/ccm.h @@ -47,6 +47,8 @@ private: unsigned int lastCt_; std::optional<float> lastSaturation_; Interpolator<Matrix<float, 3, 3>> ccm_; + Matrix<float, 3, 3> currentCcm_; + bool ccmAssigned_ = false; }; } /* namespace ipa::soft::algorithms */ diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 57836c73c..732e82510 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -282,6 +282,8 @@ void IPASoftSimple::queueRequest(const uint32_t frame, const ControlList &contro void IPASoftSimple::computeParams(const uint32_t frame) { + context_.activeState.combinedMatrix = Matrix<float, 3, 3>::identity(); + IPAFrameContext &frameContext = context_.frameContexts.get(frame); for (auto const &algo : algorithms()) algo->prepare(context_, frame, frameContext, params_);
The combined matrix must be reset to the initial value before each frame is prepared. This must be done outside algorithms because any of the algorithms may be disabled while the matrix must be always initialised. Let's initialise the combined matrix to the identity matrix (which keeps the pixel values unchanged) in software ISP just before calling `prepare' on the algorithms. Matrix updates can no longer be skipped in ccm.cpp, otherwise the CCM won't be applied if there is no temperature or saturation change. We explicitly track whether the CCM has been set up completely rather than relying on the frame number, to avoid missing the initialisation in case the first frame is skipped due to some error. Signed-off-by: Milan Zamazal <mzamazal@redhat.com> --- src/ipa/simple/algorithms/ccm.cpp | 31 +++++++++++++++---------------- src/ipa/simple/algorithms/ccm.h | 2 ++ src/ipa/simple/soft_simple.cpp | 2 ++ 3 files changed, 19 insertions(+), 16 deletions(-)