[v5,06/15] libcamera: ipa: simple: Initialise the general correction matrix

26000 diff mbox series

The combined matrix must be reset to the initial value before each frame
is prepared.  This must be done outside algorithms because any of the
algorithms may be disabled while the matrix must be always initialised.

Let's initialise the combined matrix to the identity matrix (which keeps
the pixel values unchanged) in software ISP just before calling
`prepare' on the algorithms.

Matrix updates can no longer be skipped in ccm.cpp, otherwise the CCM
won't be applied if there is no temperature or saturation change.  We
explicitly track whether the CCM has been set up completely rather than
relying on the frame number, to avoid missing the initialisation in case
the first frame is skipped due to some error.

Reviewed-by: Robert Mader <robert.mader@collabora.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
---
 src/ipa/simple/algorithms/ccm.cpp | 29 +++++++++++++----------------
 src/ipa/simple/algorithms/ccm.h   |  3 ++-
 src/ipa/simple/ipa_context.h      |  1 -
 src/ipa/simple/soft_simple.cpp    |  2 ++
 4 files changed, 17 insertions(+), 18 deletions(-)

Quoting Milan Zamazal (2026-01-28 11:43:53)
> The combined matrix must be reset to the initial value before each frame
> is prepared.  This must be done outside algorithms because any of the
> algorithms may be disabled while the matrix must be always initialised.
> 
> Let's initialise the combined matrix to the identity matrix (which keeps
> the pixel values unchanged) in software ISP just before calling
> `prepare' on the algorithms.
> 
> Matrix updates can no longer be skipped in ccm.cpp, otherwise the CCM
> won't be applied if there is no temperature or saturation change.  We
> explicitly track whether the CCM has been set up completely rather than
> relying on the frame number, to avoid missing the initialisation in case
> the first frame is skipped due to some error.
> 
> Reviewed-by: Robert Mader <robert.mader@collabora.com>
> Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
> ---
>  src/ipa/simple/algorithms/ccm.cpp | 29 +++++++++++++----------------
>  src/ipa/simple/algorithms/ccm.h   |  3 ++-
>  src/ipa/simple/ipa_context.h      |  1 -
>  src/ipa/simple/soft_simple.cpp    |  2 ++
>  4 files changed, 17 insertions(+), 18 deletions(-)
> 
> diff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp
> index a3e8cd6c4..85643645b 100644
> --- a/src/ipa/simple/algorithms/ccm.cpp
> +++ b/src/ipa/simple/algorithms/ccm.cpp
> @@ -83,7 +83,7 @@ void Ccm::applySaturation(Matrix<float, 3, 3> &ccm, float saturation)
>         ccm = ycbcr2rgb * saturationMatrix * rgb2ycbcr * ccm;
>  }
>  
> -void Ccm::prepare(IPAContext &context, const uint32_t frame,
> +void Ccm::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame,
>                   IPAFrameContext &frameContext, [[maybe_unused]] DebayerParams *params)
>  {
>         auto &saturation = context.activeState.knobs.saturation;
> @@ -91,24 +91,21 @@ void Ccm::prepare(IPAContext &context, const uint32_t frame,
>         const unsigned int ct = context.activeState.awb.temperatureK;
>  
>         /* Change CCM only on saturation or bigger temperature changes. */
> -       if (frame > 0 &&
> -           utils::abs_diff(ct, lastCt_) < kTemperatureThreshold &&
> -           saturation == lastSaturation_) {
> -               frameContext.ccm = context.activeState.ccm;
> -               return;
> +       if (!currentCcm_ ||
> +           utils::abs_diff(ct, lastCt_) >= kTemperatureThreshold ||
> +           saturation != lastSaturation_) {
> +               currentCcm_ = ccm_.getInterpolated(ct);
> +               if (saturation)
> +                       applySaturation(currentCcm_.value(), saturation.value());
> +               lastCt_ = ct;
> +               lastSaturation_ = saturation;
> +               context.activeState.matrixChanged = true;
>         }
>  
> -       lastCt_ = ct;
> -       lastSaturation_ = saturation;
> -       Matrix<float, 3, 3> ccm = ccm_.getInterpolated(ct);
> -       if (saturation)
> -               applySaturation(ccm, saturation.value());
> -
> -       context.activeState.combinedMatrix = ccm;
> -       context.activeState.ccm = ccm;
> +       context.activeState.combinedMatrix =
> +               currentCcm_.value() * context.activeState.combinedMatrix;
>         frameContext.saturation = saturation;
> -       context.activeState.matrixChanged = true;
> -       frameContext.ccm = ccm;
> +       frameContext.ccm = currentCcm_.value();
>  }
>  
>  void Ccm::process([[maybe_unused]] IPAContext &context,
> diff --git a/src/ipa/simple/algorithms/ccm.h b/src/ipa/simple/algorithms/ccm.h
> index 8279a3d59..867a680c3 100644
> --- a/src/ipa/simple/algorithms/ccm.h
> +++ b/src/ipa/simple/algorithms/ccm.h
> @@ -1,6 +1,6 @@
>  /* SPDX-License-Identifier: LGPL-2.1-or-later */
>  /*
> - * Copyright (C) 2024-2025, Red Hat Inc.
> + * Copyright (C) 2024-2026, Red Hat Inc.
>   *
>   * Color correction matrix
>   */
> @@ -47,6 +47,7 @@ private:
>         unsigned int lastCt_;
>         std::optional<float> lastSaturation_;
>         Interpolator<Matrix<float, 3, 3>> ccm_;
> +       std::optional<Matrix<float, 3, 3>> currentCcm_;
>  };
>  
>  } /* namespace ipa::soft::algorithms */
> diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h
> index 58dcad290..a3ff3d038 100644
> --- a/src/ipa/simple/ipa_context.h
> +++ b/src/ipa/simple/ipa_context.h
> @@ -62,7 +62,6 @@ struct IPAActiveState {
>                 double contrastExp;
>         } gamma;
>  
> -       Matrix<float, 3, 3> ccm;
>         Matrix<float, 3, 3> combinedMatrix;
>         bool matrixChanged = false;
>  
> diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp
> index 57836c73c..732e82510 100644
> --- a/src/ipa/simple/soft_simple.cpp
> +++ b/src/ipa/simple/soft_simple.cpp
> @@ -282,6 +282,8 @@ void IPASoftSimple::queueRequest(const uint32_t frame, const ControlList &contro
>  
>  void IPASoftSimple::computeParams(const uint32_t frame)
>  {
> +       context_.activeState.combinedMatrix = Matrix<float, 3, 3>::identity();

this is an interesting/different use case of activeState that we haven't
seen yet as it's reset for each context.

Something which requires being reset for each frame ... sounds to me
like something that should go in the FrameContext... but there might
still be nuances there too.

So I'm tempted to say this is fine for now - but might be something to
revisit later...

Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>

> +
>         IPAFrameContext &frameContext = context_.frameContexts.get(frame);
>         for (auto const &algo : algorithms())
>                 algo->prepare(context_, frame, frameContext, params_);
> -- 
> 2.52.0
>