[v5,00/36] ipa: libipa: Introduce libipa algorithms
mbox series

Message ID 20260708-libipa-algorithms-v5-0-0759d0359f52@ideasonboard.com
Headers show
Series
  • ipa: libipa: Introduce libipa algorithms
Related show

Message

Jacopo Mondi July 8, 2026, 3:50 p.m. UTC
This series introduces the first algorithms in libipa: Awb, Lsc and Ccm.

As most of the implementations of the common algorithms derive from
RkISP1 it wasn't possible to separate the algorithm introduction with
the RkISP1 IPA modifications to maintain bisectability.

This series contains:

- Awb (Grey and Bayes)
 - RkISP1
 - Simple
 - Mali-C55

- Ccm
 - RkISP1
 - Simple
 - Mali-C55

- Lsc (grid and polynomial)
 - RkISP1
 - Mali-C55

Tested on RkISP1 with imx219 (Grey world awb and grid-based LSC) and on
Mali-C55 with imx708 (Bayes AWB and grid-based LSC).

More test on Simple and with polynomial LSC would be helpful :)

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
Changes in v5:
- Add "ipa: libipa: lsc_polynomial: Fix polynomial parsing error"
  to fix and exiting potential issue with Polynomial parsing
- Fix Simple, which I broke in v4, thanks Milan
- Link to v4: https://lore.kernel.org/r/20260708-libipa-algorithms-v4-0-f0e99f035294@ideasonboard.com

Changes in v4:
- s/awb/AWB in comments everywhere, same for CCM and LSC
- Awb: changelog per patch
  - Make AwbStats pure virtual and move stats parsing logic to the IPA
    modules
  - Reduce comments/renames to reduce patch size and comments
- Collect tags

Changes in v3:
- Address comments on Awb from Stefan:
  - Introduce AwbImplementation::Result
  - Move AwbStats implementation to IPAs
  - Drop AwbStats::rg()/bg() and re-implement AwbGray accordingly
- Fix Saturation bug in Simple Ccm algorithm
- Fix Simple Ccm gain limits using Q<2, 8>
- Update Mali CCM and Gamma to latest uAPI patch version
- Fix Mali CCM gain masking as reported by Dan
- Break down libIPA Lsc implementation as requested by Stefan

Changes in v2:
- Introduce awb::Context to simplify ActiveState and FrameContext
  definition
- Add mali uAPI header for CCM and Gamma
- Do not rename Mali functions for hardware configuration in awb and lsc
- Spelling and documentation

---
Jacopo Mondi (33):
      ipa: libipa: awb: Reimplement AwbAlgorithm
      ipa: mali-c55: awb: Port to use libipa AwbAlgorithm
      ipa: libipa: Add CcmAlgorithm to libipa
      ipa: rkisp1: ccm: Port to use CcmAlgorithm
      include: linux: mali-c55: Update to support CCM and Gamma
      ipa: mali-c55: Implement Ccm algorithm
      ipa: rkisp1: lsc: Re-architecture Lsc support
      ipa: libipa: Introduce lsc_base.h
      ipa: libipa: lsc_polynomial: Fix polynomial parsing error
      ipa: libipa: lsc_polynomial: Rename LscPolynomial
      ipa: libipa: lsc_polynomial: Do not inline functions
      ipa: libipa: lsc: Move Interpolator<lsc::Components> to lsc_base
      ipa: rkisp1: lsc: Remove rogue empty line
      ipa: libipa: lsc_polynomial: Move LscPolynomial from RkISP1
      ipa: libipa: lsc_table: Move LscTable from RkISP1
      ipa: libipa: lsc_base: Add sampleForCrop documentation
      ipa: rksip1: lsc: Make parseSizes() a class function
      ipa: libipa: lsc: Introduce lsc::ActiveState
      ipa: libipa: lsc: Introduce lsc::FrameContext
      ipa: libipa: lsc: Introduce LscAlgorithm
      ipa: libipa: lsc: Move sizesListToPositions() back to rkisp1
      ipa: rkisp1: lsc: Port to use LscAlgorithm
      ipa: libipa: lsc: Make Components a map<>
      ipa: libipa: lsc: Document Components parsing
      ipa: libipa: lsc_polynomial: Remove sensorSize_
      ipa: libipa: lsc: Document LscDescriptor::sensorSize usage
      ipa: libipa: lsc: Template LscAlgorithm with register format
      ipa: libipa: lsc_base: Describe the function of lsc_base
      ipa: libipa: lsc: Template lsc::Components
      ipa: libipa: lsc: Rename sampleForCrop to resampleLscData
      ipa: libipa: lsc: Specialize Interpolator for uint8_t
      ipa: mali-c55: Add sensorInfo to IPAContext
      ipa: mali-c55: Port to use LscAlgorithm

Kieran Bingham (2):
      ipa: simple: awb: Port to use libipa AwbAlgorithm
      ipa: simple: Use libipa CcmAlgorithm

Laurent Pinchart (1):
      ipa: libipa: lsc_polynomial: Don't inline ValueNode accessor

 .../internal/software_isp/debayer_params.h         |   4 +-
 include/linux/media/arm/mali-c55-config.h          |  86 +++-
 src/ipa/libipa/awb.cpp                             | 418 +++++++++++++++++---
 src/ipa/libipa/awb.h                               | 100 ++++-
 src/ipa/libipa/awb_bayes.cpp                       |  52 +--
 src/ipa/libipa/awb_bayes.h                         |  12 +-
 src/ipa/libipa/awb_grey.cpp                        |  16 +-
 src/ipa/libipa/awb_grey.h                          |   6 +-
 src/ipa/libipa/ccm.cpp                             | 245 ++++++++++++
 src/ipa/libipa/ccm.h                               |  77 ++++
 src/ipa/libipa/lsc.cpp                             | 311 +++++++++++++++
 src/ipa/libipa/lsc.h                               | 137 +++++++
 src/ipa/libipa/lsc_base.cpp                        | 178 +++++++++
 src/ipa/libipa/lsc_base.h                          |  90 +++++
 src/ipa/libipa/lsc_polynomial.cpp                  | 177 ++++++++-
 src/ipa/libipa/lsc_polynomial.h                    | 152 +++++---
 src/ipa/libipa/lsc_table.cpp                       |  40 ++
 src/ipa/libipa/lsc_table.h                         | 118 ++++++
 src/ipa/libipa/meson.build                         |   8 +
 src/ipa/mali-c55/algorithms/awb.cpp                | 223 ++++++-----
 src/ipa/mali-c55/algorithms/awb.h                  |  28 +-
 src/ipa/mali-c55/algorithms/ccm.cpp                | 173 +++++++++
 src/ipa/mali-c55/algorithms/ccm.h                  |  66 ++++
 src/ipa/mali-c55/algorithms/lsc.cpp                | 156 +++++---
 src/ipa/mali-c55/algorithms/lsc.h                  |  32 +-
 src/ipa/mali-c55/algorithms/meson.build            |   1 +
 src/ipa/mali-c55/ipa_context.cpp                   |   3 +
 src/ipa/mali-c55/ipa_context.h                     |  20 +-
 src/ipa/mali-c55/mali-c55.cpp                      |   2 +
 src/ipa/mali-c55/params.h                          |   2 +
 src/ipa/rkisp1/algorithms/awb.cpp                  | 191 ++-------
 src/ipa/rkisp1/algorithms/awb.h                    |  18 +-
 src/ipa/rkisp1/algorithms/ccm.cpp                  |  97 +----
 src/ipa/rkisp1/algorithms/ccm.h                    |  17 +-
 src/ipa/rkisp1/algorithms/lsc.cpp                  | 432 +++------------------
 src/ipa/rkisp1/algorithms/lsc.h                    |  41 +-
 src/ipa/rkisp1/ipa_context.h                       |  49 +--
 src/ipa/simple/algorithms/awb.cpp                  | 141 +++++--
 src/ipa/simple/algorithms/awb.h                    |  29 ++
 src/ipa/simple/algorithms/blc.cpp                  |   2 +-
 src/ipa/simple/algorithms/ccm.cpp                  |  72 ++--
 src/ipa/simple/algorithms/ccm.h                    |  23 +-
 src/ipa/simple/ipa_context.h                       |  15 +-
 src/libcamera/software_isp/debayer_cpu.cpp         |  12 +-
 44 files changed, 2945 insertions(+), 1127 deletions(-)
---
base-commit: 709ad59a8d90af3570be1c8d02dee0ec9e011954
change-id: 20260615-libipa-algorithms-59666bca0a2f

Best regards,

Comments

Milan Zamazal July 8, 2026, 8:13 p.m. UTC | #1
Hi Jacopo,

thank you for the update.

Jacopo Mondi <jacopo.mondi@ideasonboard.com> writes:

> This series introduces the first algorithms in libipa: Awb, Lsc and Ccm.
>
> As most of the implementations of the common algorithms derive from
> RkISP1 it wasn't possible to separate the algorithm introduction with
> the RkISP1 IPA modifications to maintain bisectability.
>
> This series contains:
>
> - Awb (Grey and Bayes)
>  - RkISP1
>  - Simple
>  - Mali-C55
>
> - Ccm
>  - RkISP1
>  - Simple
>  - Mali-C55
>
> - Lsc (grid and polynomial)
>  - RkISP1
>  - Mali-C55
>
> Tested on RkISP1 with imx219 (Grey world awb and grid-based LSC) and on
> Mali-C55 with imx708 (Bayes AWB and grid-based LSC).
>
> More test on Simple and with polynomial LSC would be helpful :)

Simple seems to work for me.  As for polynomial LSC, it does something
(with the softISP LSC patches), probably meaningful, but not sure how to
check correctness easily.

> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
> Changes in v5:
> - Add "ipa: libipa: lsc_polynomial: Fix polynomial parsing error"
>   to fix and exiting potential issue with Polynomial parsing

Thanks, it fails gracefully now.

> - Fix Simple, which I broke in v4, thanks Milan
> - Link to v4: https://lore.kernel.org/r/20260708-libipa-algorithms-v4-0-f0e99f035294@ideasonboard.com
>
> Changes in v4:
> - s/awb/AWB in comments everywhere, same for CCM and LSC
> - Awb: changelog per patch
>   - Make AwbStats pure virtual and move stats parsing logic to the IPA
>     modules
>   - Reduce comments/renames to reduce patch size and comments
> - Collect tags
>
> Changes in v3:
> - Address comments on Awb from Stefan:
>   - Introduce AwbImplementation::Result
>   - Move AwbStats implementation to IPAs
>   - Drop AwbStats::rg()/bg() and re-implement AwbGray accordingly
> - Fix Saturation bug in Simple Ccm algorithm
> - Fix Simple Ccm gain limits using Q<2, 8>
> - Update Mali CCM and Gamma to latest uAPI patch version
> - Fix Mali CCM gain masking as reported by Dan
> - Break down libIPA Lsc implementation as requested by Stefan
>
> Changes in v2:
> - Introduce awb::Context to simplify ActiveState and FrameContext
>   definition
> - Add mali uAPI header for CCM and Gamma
> - Do not rename Mali functions for hardware configuration in awb and lsc
> - Spelling and documentation
>
> ---
> Jacopo Mondi (33):
>       ipa: libipa: awb: Reimplement AwbAlgorithm
>       ipa: mali-c55: awb: Port to use libipa AwbAlgorithm
>       ipa: libipa: Add CcmAlgorithm to libipa
>       ipa: rkisp1: ccm: Port to use CcmAlgorithm
>       include: linux: mali-c55: Update to support CCM and Gamma
>       ipa: mali-c55: Implement Ccm algorithm
>       ipa: rkisp1: lsc: Re-architecture Lsc support
>       ipa: libipa: Introduce lsc_base.h
>       ipa: libipa: lsc_polynomial: Fix polynomial parsing error
>       ipa: libipa: lsc_polynomial: Rename LscPolynomial
>       ipa: libipa: lsc_polynomial: Do not inline functions
>       ipa: libipa: lsc: Move Interpolator<lsc::Components> to lsc_base
>       ipa: rkisp1: lsc: Remove rogue empty line
>       ipa: libipa: lsc_polynomial: Move LscPolynomial from RkISP1
>       ipa: libipa: lsc_table: Move LscTable from RkISP1
>       ipa: libipa: lsc_base: Add sampleForCrop documentation
>       ipa: rksip1: lsc: Make parseSizes() a class function
>       ipa: libipa: lsc: Introduce lsc::ActiveState
>       ipa: libipa: lsc: Introduce lsc::FrameContext
>       ipa: libipa: lsc: Introduce LscAlgorithm
>       ipa: libipa: lsc: Move sizesListToPositions() back to rkisp1
>       ipa: rkisp1: lsc: Port to use LscAlgorithm
>       ipa: libipa: lsc: Make Components a map<>
>       ipa: libipa: lsc: Document Components parsing
>       ipa: libipa: lsc_polynomial: Remove sensorSize_
>       ipa: libipa: lsc: Document LscDescriptor::sensorSize usage
>       ipa: libipa: lsc: Template LscAlgorithm with register format
>       ipa: libipa: lsc_base: Describe the function of lsc_base
>       ipa: libipa: lsc: Template lsc::Components
>       ipa: libipa: lsc: Rename sampleForCrop to resampleLscData
>       ipa: libipa: lsc: Specialize Interpolator for uint8_t
>       ipa: mali-c55: Add sensorInfo to IPAContext
>       ipa: mali-c55: Port to use LscAlgorithm
>
> Kieran Bingham (2):
>       ipa: simple: awb: Port to use libipa AwbAlgorithm
>       ipa: simple: Use libipa CcmAlgorithm
>
> Laurent Pinchart (1):
>       ipa: libipa: lsc_polynomial: Don't inline ValueNode accessor
>
>  .../internal/software_isp/debayer_params.h         |   4 +-
>  include/linux/media/arm/mali-c55-config.h          |  86 +++-
>  src/ipa/libipa/awb.cpp                             | 418 +++++++++++++++++---
>  src/ipa/libipa/awb.h                               | 100 ++++-
>  src/ipa/libipa/awb_bayes.cpp                       |  52 +--
>  src/ipa/libipa/awb_bayes.h                         |  12 +-
>  src/ipa/libipa/awb_grey.cpp                        |  16 +-
>  src/ipa/libipa/awb_grey.h                          |   6 +-
>  src/ipa/libipa/ccm.cpp                             | 245 ++++++++++++
>  src/ipa/libipa/ccm.h                               |  77 ++++
>  src/ipa/libipa/lsc.cpp                             | 311 +++++++++++++++
>  src/ipa/libipa/lsc.h                               | 137 +++++++
>  src/ipa/libipa/lsc_base.cpp                        | 178 +++++++++
>  src/ipa/libipa/lsc_base.h                          |  90 +++++
>  src/ipa/libipa/lsc_polynomial.cpp                  | 177 ++++++++-
>  src/ipa/libipa/lsc_polynomial.h                    | 152 +++++---
>  src/ipa/libipa/lsc_table.cpp                       |  40 ++
>  src/ipa/libipa/lsc_table.h                         | 118 ++++++
>  src/ipa/libipa/meson.build                         |   8 +
>  src/ipa/mali-c55/algorithms/awb.cpp                | 223 ++++++-----
>  src/ipa/mali-c55/algorithms/awb.h                  |  28 +-
>  src/ipa/mali-c55/algorithms/ccm.cpp                | 173 +++++++++
>  src/ipa/mali-c55/algorithms/ccm.h                  |  66 ++++
>  src/ipa/mali-c55/algorithms/lsc.cpp                | 156 +++++---
>  src/ipa/mali-c55/algorithms/lsc.h                  |  32 +-
>  src/ipa/mali-c55/algorithms/meson.build            |   1 +
>  src/ipa/mali-c55/ipa_context.cpp                   |   3 +
>  src/ipa/mali-c55/ipa_context.h                     |  20 +-
>  src/ipa/mali-c55/mali-c55.cpp                      |   2 +
>  src/ipa/mali-c55/params.h                          |   2 +
>  src/ipa/rkisp1/algorithms/awb.cpp                  | 191 ++-------
>  src/ipa/rkisp1/algorithms/awb.h                    |  18 +-
>  src/ipa/rkisp1/algorithms/ccm.cpp                  |  97 +----
>  src/ipa/rkisp1/algorithms/ccm.h                    |  17 +-
>  src/ipa/rkisp1/algorithms/lsc.cpp                  | 432 +++------------------
>  src/ipa/rkisp1/algorithms/lsc.h                    |  41 +-
>  src/ipa/rkisp1/ipa_context.h                       |  49 +--
>  src/ipa/simple/algorithms/awb.cpp                  | 141 +++++--
>  src/ipa/simple/algorithms/awb.h                    |  29 ++
>  src/ipa/simple/algorithms/blc.cpp                  |   2 +-
>  src/ipa/simple/algorithms/ccm.cpp                  |  72 ++--
>  src/ipa/simple/algorithms/ccm.h                    |  23 +-
>  src/ipa/simple/ipa_context.h                       |  15 +-
>  src/libcamera/software_isp/debayer_cpu.cpp         |  12 +-
>  44 files changed, 2945 insertions(+), 1127 deletions(-)
> ---
> base-commit: 709ad59a8d90af3570be1c8d02dee0ec9e011954
> change-id: 20260615-libipa-algorithms-59666bca0a2f
>
> Best regards,