[libcamera-devel,v8] ipa: ipu3: af: Auto focus for dw9719 Surface Go2 VCM

15363 diff mbox series

Since VCM for surface Go 2 (dw9719) had been successfully
driven, this Af module can be used to control the VCM and
determine the focus value based on the IPU3 AF state.

Based on the values from the IPU3 AF buffer, the variance
of each focus step is determined and a greedy approach is
used to find the maximum variance of the AF state and an
appropriate focus value.

The grid configuration is implemented as a context. Also,
the grid parameter- AF_MIN_BLOCK_WIDTH is set to 4 (default
is 3) since if the default value is used, x_start
(x_start > 640) will be at an incorrect location of the
image (rightmost of the sensor).

Signed-off-by: Kate Hsuan <hpa@redhat.com>
Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
---
Changes in v8:
1. Revised and improved the comments.
2. Simplified the algorithm.
---
 src/ipa/ipu3/algorithms/af.cpp      | 435 ++++++++++++++++++++++++++++
 src/ipa/ipu3/algorithms/af.h        |  79 +++++
 src/ipa/ipu3/algorithms/meson.build |   1 +
 src/ipa/ipu3/ipa_context.cpp        |  23 ++
 src/ipa/ipu3/ipa_context.h          |  10 +
 src/ipa/ipu3/ipu3.cpp               |   2 +
 6 files changed, 550 insertions(+)
 create mode 100644 src/ipa/ipu3/algorithms/af.cpp
 create mode 100644 src/ipa/ipu3/algorithms/af.h

Hi Kieran,



On Mon, Feb 14, 2022 at 5:51 PM Kate Hsuan <hpa@redhat.com> wrote:
>
> Since VCM for surface Go 2 (dw9719) had been successfully
> driven, this Af module can be used to control the VCM and
> determine the focus value based on the IPU3 AF state.
>
> Based on the values from the IPU3 AF buffer, the variance
> of each focus step is determined and a greedy approach is
> used to find the maximum variance of the AF state and an
> appropriate focus value.
>
> The grid configuration is implemented as a context. Also,
> the grid parameter- AF_MIN_BLOCK_WIDTH is set to 4 (default
> is 3) since if the default value is used, x_start
> (x_start > 640) will be at an incorrect location of the
> image (rightmost of the sensor).
>
> Signed-off-by: Kate Hsuan <hpa@redhat.com>
> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> ---
> Changes in v8:
> 1. Revised and improved the comments.
> 2. Simplified the algorithm.
> ---
>  src/ipa/ipu3/algorithms/af.cpp      | 435 ++++++++++++++++++++++++++++
>  src/ipa/ipu3/algorithms/af.h        |  79 +++++
>  src/ipa/ipu3/algorithms/meson.build |   1 +
>  src/ipa/ipu3/ipa_context.cpp        |  23 ++
>  src/ipa/ipu3/ipa_context.h          |  10 +
>  src/ipa/ipu3/ipu3.cpp               |   2 +
>  6 files changed, 550 insertions(+)
>  create mode 100644 src/ipa/ipu3/algorithms/af.cpp
>  create mode 100644 src/ipa/ipu3/algorithms/af.h
>
> diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp
> new file mode 100644
> index 00000000..17055c04
> --- /dev/null
> +++ b/src/ipa/ipu3/algorithms/af.cpp
> @@ -0,0 +1,435 @@
> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> +/*
> + * Copyright (C) 2021, Red Hat
> + *
> + * af.cpp - IPU3 auto focus algorithm
> + */
> +
> +#include "af.h"
> +
> +#include <algorithm>
> +#include <chrono>
> +#include <cmath>
> +#include <fcntl.h>
> +#include <numeric>
> +#include <sys/ioctl.h>
> +#include <sys/stat.h>
> +#include <sys/types.h>
> +#include <unistd.h>
> +
> +#include <linux/videodev2.h>
> +
> +#include <libcamera/base/log.h>
> +
> +#include <libcamera/ipa/core_ipa_interface.h>
> +
> +#include "libipa/histogram.h"
> +
> +/**
> + * \file af.h
> + */
> +
> +/**
> + * \var kAfMinGridWidth
> + * \brief the minimum width of AF grid.
> + * The minimum grid horizontal dimensions.
> +*/
> +
> +/**
> + * \var kAfMinGridHeight
> + * \brief the minimum height of AF grid.
> + * The minimum grid vertical dimensions.
> +*/
> +
> +/**
> + * \var kAfMaxGridWidth
> + * \brief the maximum width of AF grid.
> + * The maximum grid horizontal dimensions.
> +*/
> +
> +/**
> + * \var kAfMaxGridHeight
> + * \brief The maximum height of AF grid.
> + * The maximum grid vertical dimensions.
> +*/
> +
> +/**
> + * \var kAfMinGridBlockWidth
> + * \brief The minimum block size of the width.
> + * The minimum value of Log2 of the width of the grid cell.
> + */
> +
> +/**
> + * \var kAfMinGridBlockHeight
> + * \brief The minimum block size of the height.
> + * The minimum value of Log2 of the height of the grid cell.
> + */
> +
> +/**
> + * \def kAfMaxGridBlockWidth
> + * \brief The maximum block size of the width.
> + * The maximum value of Log2 of the width of the grid cell.
> + */
> +
> +/**
> + * \var kAfMaxGridBlockHeight
> + * \brief The maximum block size of the height.
> + * The maximum value of Log2 of the height of the grid cell.
> + */
> +
> +/**
> + * \var kAfDefaultHeightPerSlice
> + * \brief The default number of blocks in vertical axis per slice.
> + * The number of blocks in vertical axis per slice.
> + */
> +
> +namespace libcamera {
> +
> +using namespace std::literals::chrono_literals;
> +
> +namespace ipa::ipu3::algorithms {
> +
> +LOG_DEFINE_CATEGORY(IPU3Af)
> +
> +/**
> + * Maximum focus steps of the VCM control
> + * \todo should be obtained from the VCM driver
> + */
> +static constexpr uint32_t kMaxFocusSteps = 1023;
> +
> +/* Minimum focus step for searching appropriate focus */
> +static constexpr uint32_t kCoarseSearchStep = 30;
> +static constexpr uint32_t kFineSearchStep = 1;
> +
> +/* Max ratio of variance change, 0.0 < kMaxChange < 1.0 */
> +static constexpr double kMaxChange = 0.5;
> +
> +/* The numbers of frame to be ignored, before performing focus scan. */
> +static constexpr uint32_t kIgnoreFrame = 10;
> +
> +/* Fine scan range 0 < kFineRange < 1 */
> +static constexpr double kFineRange = 0.05;
> +
> +/* Settings for IPU3 AF filter */
> +static struct ipu3_uapi_af_filter_config afFilterConfigDefault = {
> +       .y1_coeff_0 = { 0, 1, 3, 7 },
> +       .y1_coeff_1 = { 11, 13, 1, 2 },
> +       .y1_coeff_2 = { 8, 19, 34, 242 },
> +       .y1_sign_vec = 0x7fdffbfe,
> +       .y2_coeff_0 = { 0, 1, 6, 6 },
> +       .y2_coeff_1 = { 13, 25, 3, 0 },
> +       .y2_coeff_2 = { 25, 3, 177, 254 },
> +       .y2_sign_vec = 0x4e53ca72,
> +       .y_calc = { 8, 8, 8, 8 },
> +       .nf = { 0, 9, 0, 9, 0 },
> +};
> +
> +/**
> + * \class Af
> + * \brief An auto-focus algorithm based on IPU3 statistics
> + * This algorithm is used to determine the position of the lens to make a
> + * focused image. The IPU3 AF processing block computes the statistics that
> + * are composed by two types of filtered value and stores in a AF buffer.
> + * Typically, for a clear image, it has a relatively higher contrast than a
> + * blurred one. Therefore, if an image with the highest contrast can be
> + * found through the scan, the position of the len indicates to a clearest
> + * image.
> + */
> +Af::Af()
> +       : focus_(0), bestFocus_(0), currentVariance_(0.0), previousVariance_(0.0),
> +         coarseCompleted_(false), fineCompleted_(false)
> +{
> +}
> +
> +/**
> + * \copydoc libcamera::ipa::Algorithm::prepare
> + */
> +void Af::prepare(IPAContext &context, ipu3_uapi_params *params)
> +{
> +       const struct ipu3_uapi_grid_config &grid = context.configuration.af.afGrid;
> +       params->acc_param.af.grid_cfg = grid;
> +       params->acc_param.af.filter_config = afFilterConfigDefault;
> +
> +       /* Enable AF processing block */
> +       params->use.acc_af = 1;
> +}
> +
> +/**
> + * \brief Configure the Af given a configInfo
> + * \param[in] context The shared IPA context
> + * \param[in] configInfo The IPA configuration data
> + * \return 0
> + */
> +int Af::configure(IPAContext &context, const IPAConfigInfo &configInfo)
> +{
> +       struct ipu3_uapi_grid_config &grid = context.configuration.af.afGrid;
> +       grid.width = kAfMinGridWidth;
> +       grid.height = kAfMinGridHeight;
> +       grid.block_width_log2 = kAfMinGridBlockWidth;
> +       grid.block_height_log2 = kAfMinGridBlockHeight;
> +       grid.height_per_slice = kAfDefaultHeightPerSlice;
> +
> +       /* x_start and y start are default to BDS center */
> +       grid.x_start = (configInfo.bdsOutputSize.width / 2) -
> +                      (((grid.width << grid.block_width_log2) / 2));
> +       grid.y_start = (configInfo.bdsOutputSize.height / 2) -
> +                      (((grid.height << grid.block_height_log2) / 2));
> +
> +       /* x_start and y_start should be even */
> +       grid.x_start = (grid.x_start / 2) * 2;
> +       grid.y_start = (grid.y_start / 2) * 2;
> +       grid.y_start = grid.y_start | IPU3_UAPI_GRID_Y_START_EN;
> +
> +       /* Initial max focus step */
> +       maxStep_ = kMaxFocusSteps;
> +
> +       /* Initial focus value */
> +       context.frameContext.af.focus = 0;
> +       /* Maximum variance of the AF statistics */
> +       context.frameContext.af.maxVariance = 0;
> +       /* The stable AF value flag. if it is true, the AF should be in a stable state. */
> +       context.frameContext.af.stable = false;
> +
> +       return 0;
> +}
> +
> +/**
> + * \brief AF coarse scan
> + * Find a near focused image using a coarse step. The step is determined by coarseSearchStep.
> + * \param[in] context The shared IPA context
> + */
> +void Af::afCoarseScan(IPAContext &context)
> +{
> +       if (coarseCompleted_)
> +               return;
> +
> +       if (afNeedIgnoreFrame())
> +               return;
> +
> +       if (afScan(context, kCoarseSearchStep)) {
> +               coarseCompleted_ = true;
> +               context.frameContext.af.maxVariance = 0;
> +               focus_ = context.frameContext.af.focus -
> +                        (context.frameContext.af.focus * kFineRange);
> +               context.frameContext.af.focus = focus_;
> +               previousVariance_ = 0;
> +               maxStep_ = std::clamp(focus_ + static_cast<uint32_t>((focus_ * kFineRange)),
> +                                     0U, kMaxFocusSteps);
> +       }
> +}
> +
> +/**
> + * \brief AF fine scan
> + * Find an optimum lens position with moving 1 step for each search.
> + * \param[in] context The shared IPA context
> + */
> +void Af::afFineScan(IPAContext &context)
> +{
> +       if (!coarseCompleted_)
> +               return;
> +
> +       if (afNeedIgnoreFrame())
> +               return;
> +
> +       if (afScan(context, kFineSearchStep)) {
> +               context.frameContext.af.stable = true;
> +               fineCompleted_ = true;
> +       }
> +}
> +
> +/**
> + * \brief AF reset
> + * Reset all the parameters to start over the AF process.
> + * \param[in] context The shared IPA context
> + */
> +void Af::afReset(IPAContext &context)
> +{
> +       if (afNeedIgnoreFrame())
> +               return;
> +
> +       context.frameContext.af.maxVariance = 0;
> +       context.frameContext.af.focus = 0;
> +       focus_ = 0;
> +       context.frameContext.af.stable = false;
> +       ignoreCounter_ = kIgnoreFrame;
> +       previousVariance_ = 0.0;
> +       coarseCompleted_ = false;
> +       fineCompleted_ = false;
> +       maxStep_ = kMaxFocusSteps;
> +}
> +
> +/**
> + * \brief AF variance comparison.
> + * It always picks the largest variance to replace the previous one. The image
> + * with a larger variance also indicates it is a clearer image than previous
> + * one. If it finds the negative sign of derivative, it returns immediately.
> + * \param[in] context The IPA context
> + * \param min_step The VCM movement step.
> + * \return True, if it finds a AF value.
> + */
> +bool Af::afScan(IPAContext &context, int min_step)
> +{
> +       if (focus_ > maxStep_) {
> +               /* If reach the max step, move lens to the position. */
> +               context.frameContext.af.focus = bestFocus_;
> +               return true;
> +       } else {
> +               /*
> +                * Find the maximum of the variance by estimating its
> +                * derivative. If the direction changes, it means we have
> +                * passed a maximum one step before.
> +               */
> +               if ((currentVariance_ - context.frameContext.af.maxVariance) >=
> +                   -(context.frameContext.af.maxVariance * 0.1)) {
> +                       /*
> +                        * Positive and zero derivative:
> +                        * The variance is still increasing. The focus could be
> +                        * increased for the next comparison. Also, the max variance
> +                        * and previous focus value are updated.
> +                        */
> +                       bestFocus_ = focus_;
> +                       focus_ += min_step;
> +                       context.frameContext.af.focus = focus_;
> +                       context.frameContext.af.maxVariance = currentVariance_;
> +               } else {
> +                       /*
> +                        * Negative derivative:
> +                        * The variance starts to decrease which means the maximum
> +                        * variance is found. Set focus step to previous good one
> +                        * then return immediately.
> +                        */
> +                       context.frameContext.af.focus = bestFocus_;
> +                       return true;
> +               }
> +       }
> +
> +       previousVariance_ = currentVariance_;
> +       LOG(IPU3Af, Debug) << " Previous step is "
> +                          << bestFocus_
> +                          << " Current step is "
> +                          << focus_;
> +       return false;
> +}
> +
> +/**
> + * \brief Determine the frame to be ignored.
> + * \return Return true the frame is ignored.
> + * \return Return false the frame should be processed.
> + */
> +bool Af::afNeedIgnoreFrame()
> +{
> +       if (ignoreCounter_ == 0)
> +               return false;
> +       else
> +               ignoreCounter_--;
> +       return true;
> +}
> +
> +/**
> + * \brief Reset frame ignore counter.
> + */
> +void Af::afIgnoreFrameReset()
> +{
> +       ignoreCounter_ = kIgnoreFrame;
> +}
> +
> +/**
> + * \brief Estemate variance
> + */
> +double Af::afEstemateVariance(y_table_item_t *y_item, uint32_t len,
> +                             bool isY1)
> +{
> +       uint32_t z = 0;
> +       uint32_t total = 0;
> +       double mean;
> +       double var_sum = 0;
> +
> +       for (z = 0; z < len; z++) {
> +               if (isY1)
> +                       total += y_item[z].y1_avg;
> +               else
> +                       total += y_item[z].y2_avg;
> +       }
> +       mean = total / len;
> +       for (z = 0; z < len; z++) {
> +               if (isY1)
> +                       var_sum += pow((y_item[z].y1_avg - mean), 2);
> +               else
> +                       var_sum += pow((y_item[z].y2_avg - mean), 2);
> +       }
> +
> +       return var_sum / static_cast<double>(len);
> +}
> +
> +/**
> + * \brief Determine out-of-focus situation.
> + * Out-of-focus means that the variance change rate for a focused and a new
> + * variance is greater than a threshold.
> + * \param context The IPA context.
> + * \return If it is out-of-focus, return true.
> + * \return If is is focused, return false.
> + */
> +bool Af::afIsOutOfFocus(IPAContext context)
> +{
> +       const uint32_t diff_var = std::abs(currentVariance_ -
> +                                          context.frameContext.af.maxVariance);
> +       const double var_ratio = diff_var / context.frameContext.af.maxVariance;
> +       LOG(IPU3Af, Debug) << "Variance change rate: "
> +                          << var_ratio
> +                          << " Current VCM step: "
> +                          << context.frameContext.af.focus;
> +       if (var_ratio > kMaxChange)
> +               return true;
> +       else
> +               return false;
> +}
> +
> +/**
> + * \brief Determine the max contrast image and lens position.
> + * Ideally, a clear image also has a raletively higher contrast. So, every
> + * images for each focus step should be tested to find a optimal focus step.
> + * The Hill Climbing Algorithm[1] is used to find the maximum variance of the
> + * AF statistic which is the AF output of IPU3. The focus step is increased
> + * then the variance of the AF statistic is estimated. If it finds the negative
> + * derivative which means we just passed the peak, the best focus is found.
> + *
> + * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing
> + * \param[in] context The IPA context.
> + * \param[in] stats The statistic buffer of IPU3.
> + */
> +void Af::process(IPAContext &context, const ipu3_uapi_stats_3a *stats)
> +{
> +       y_table_item_t y_item[IPU3_UAPI_AF_Y_TABLE_MAX_SIZE / sizeof(y_table_item_t)];
> +       uint32_t afRawBufferLen;
> +
> +       /* Evaluate the AF buffer length */
> +       afRawBufferLen = context.configuration.af.afGrid.width *
> +                        context.configuration.af.afGrid.height;
> +
> +       memcpy(y_item, stats->af_raw_buffer.y_table,
> +              afRawBufferLen * sizeof(y_table_item_t));
> +
> +       /*
> +        * Calculate the mean and the variance of AF statistics for a given grid.
> +        * For coarse: y1 are used.
> +        * For fine: y2 results are used.
> +        */
> +       if (coarseCompleted_)
> +               currentVariance_ = afEstemateVariance(y_item, afRawBufferLen, false);
> +       else
> +               currentVariance_ = afEstemateVariance(y_item, afRawBufferLen, true);
> +
> +       if (!context.frameContext.af.stable) {
> +               afCoarseScan(context);
> +               afFineScan(context);
> +       } else {
> +               if (afIsOutOfFocus(context))
> +                       afReset(context);
> +               else
> +                       afIgnoreFrameReset();
> +       }
> +}
> +
> +} /* namespace ipa::ipu3::algorithms */
> +
> +} /* namespace libcamera */
> diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h
> new file mode 100644
> index 00000000..2ca78c84
> --- /dev/null
> +++ b/src/ipa/ipu3/algorithms/af.h
> @@ -0,0 +1,79 @@
> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> +/*
> + * Copyright (C) 2021, Red Hat
> + *
> + * af.h - IPU3 Af algorithm
> + */
> +
> +#pragma once
> +
> +#include <linux/intel-ipu3.h>
> +
> +#include <libcamera/base/utils.h>
> +
> +#include <libcamera/geometry.h>
> +
> +#include "algorithm.h"
> +
> +/* Static variables from repo of chromium */
> +static constexpr uint8_t kAfMinGridWidth = 16;
> +static constexpr uint8_t kAfMinGridHeight = 16;
> +static constexpr uint8_t kAfMaxGridWidth = 32;
> +static constexpr uint8_t kAfMaxGridHeight = 24;
> +static constexpr uint16_t kAfMinGridBlockWidth = 4;
> +static constexpr uint16_t kAfMinGridBlockHeight = 3;
> +static constexpr uint16_t kAfMaxGridBlockWidth = 6;
> +static constexpr uint16_t kAfMaxGridBlockHeight = 6;
> +static constexpr uint16_t kAfDefaultHeightPerSlice = 2;
> +
> +namespace libcamera {
> +
> +namespace ipa::ipu3::algorithms {
> +
> +class Af : public Algorithm
> +{
> +       /* The format of y_table. From ipu3-ipa repo */
> +       typedef struct __attribute__((packed)) y_table_item {
> +               uint16_t y1_avg;
> +               uint16_t y2_avg;
> +       } y_table_item_t;
> +public:
> +       Af();
> +       ~Af() = default;
> +
> +       void prepare(IPAContext &context, ipu3_uapi_params *params) override;
> +       int configure(IPAContext &context, const IPAConfigInfo &configInfo) override;
> +       void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override;
> +
> +private:
> +       void afCoarseScan(IPAContext &context);
> +       void afFineScan(IPAContext &context);
> +       bool afScan(IPAContext &context, int min_step);
> +       void afReset(IPAContext &context);
> +       bool afNeedIgnoreFrame();
> +       void afIgnoreFrameReset();
> +       double afEstemateVariance(y_table_item_t *y_item, uint32_t len,
> +                                 bool isY1);
> +       bool afIsOutOfFocus(IPAContext context);
> +
> +       /* VCM step configuration. It is the current setting of the VCM step. */
> +       uint32_t focus_;
> +       /* The best VCM step. It is a local optimum VCM step during scanning. */
> +       uint32_t bestFocus_;
> +       /* Current AF statistic variance. */
> +       double currentVariance_;
> +       /* The frames are ignore before starting measuring. */
> +       uint32_t ignoreCounter_;
> +       /* It is used to determine the derivative during scanning */
> +       double previousVariance_;
> +       /* The designated maximum range of focus scanning. */
> +       uint32_t maxStep_;
> +       /* If the coarse scan completes, it is set to true. */
> +       bool coarseCompleted_;
> +       /* If the fine scan completes, it is set to true. */
> +       bool fineCompleted_;
> +};
> +
> +} /* namespace ipa::ipu3::algorithms */
> +
> +} /* namespace libcamera */
> diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build
> index 4db6ae1d..b70a551c 100644
> --- a/src/ipa/ipu3/algorithms/meson.build
> +++ b/src/ipa/ipu3/algorithms/meson.build
> @@ -1,6 +1,7 @@
>  # SPDX-License-Identifier: CC0-1.0
>
>  ipu3_ipa_algorithms = files([
> +    'af.cpp',
>      'agc.cpp',
>      'awb.cpp',
>      'blc.cpp',
> diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp
> index 86794ac1..e8f7367c 100644
> --- a/src/ipa/ipu3/ipa_context.cpp
> +++ b/src/ipa/ipu3/ipa_context.cpp
> @@ -69,6 +69,29 @@ namespace libcamera::ipa::ipu3 {
>   * \brief Number of cells on one line including the ImgU padding
>   */
>
> +/**
> + * \var IPASessionConfiguration::af
> + * \brief AF grid configuration of the IPA
> + *
> + * \var IPASessionConfiguration::af.afGrid
> + * \brief AF scene grid configuration.
> + */
> +
> +/**
> + * \var IPAFrameContext::af
> + * \brief Context for the Automatic Focus algorithm
> + *
> + * \struct  IPAFrameContext::af
> + * \var IPAFrameContext::af.focus
> + * \brief Current position of the lens
> + *
> + * \var IPAFrameContext::af.maxVariance
> + * \brief The maximum variance of the current image.
> + *
> + * \var IPAFrameContext::af.stable
> + * \brief It is set to true, if the best focus is found.
> + */
> +
>  /**
>   * \var IPASessionConfiguration::agc
>   * \brief AGC parameters configuration of the IPA
> diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h
> index c6dc0814..60ad3194 100644
> --- a/src/ipa/ipu3/ipa_context.h
> +++ b/src/ipa/ipu3/ipa_context.h
> @@ -25,6 +25,10 @@ struct IPASessionConfiguration {
>                 uint32_t stride;
>         } grid;
>
> +       struct {
> +               ipu3_uapi_grid_config afGrid;
> +       } af;
> +
>         struct {
>                 utils::Duration minShutterSpeed;
>                 utils::Duration maxShutterSpeed;
> @@ -34,6 +38,12 @@ struct IPASessionConfiguration {
>  };
>
>  struct IPAFrameContext {
> +       struct {
> +               uint32_t focus;
> +               double maxVariance;
> +               bool stable;
> +       } af;
> +
>         struct {
>                 uint32_t exposure;
>                 double gain;
> diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp
> index e44a31bb..417e0562 100644
> --- a/src/ipa/ipu3/ipu3.cpp
> +++ b/src/ipa/ipu3/ipu3.cpp
> @@ -30,6 +30,7 @@
>
>  #include "libcamera/internal/mapped_framebuffer.h"
>
> +#include "algorithms/af.h"
>  #include "algorithms/agc.h"
>  #include "algorithms/algorithm.h"
>  #include "algorithms/awb.h"
> @@ -295,6 +296,7 @@ int IPAIPU3::init(const IPASettings &settings,
>         }
>
>         /* Construct our Algorithms */
> +       algorithms_.push_back(std::make_unique<algorithms::Af>());
>         algorithms_.push_back(std::make_unique<algorithms::Agc>());
>         algorithms_.push_back(std::make_unique<algorithms::Awb>());
>         algorithms_.push_back(std::make_unique<algorithms::BlackLevelCorrection>());
> --
> 2.33.1
>

Sorry about that. I forget to loop you in.
I resend the mail again.

Thank you.

Hi Kate,

Thanks for the patch !

On 14/02/2022 10:51, Kate Hsuan wrote:
> Since VCM for surface Go 2 (dw9719) had been successfully
> driven, this Af module can be used to control the VCM and
> determine the focus value based on the IPU3 AF state.
> 
> Based on the values from the IPU3 AF buffer, the variance
> of each focus step is determined and a greedy approach is
> used to find the maximum variance of the AF state and an
> appropriate focus value.
> 
> The grid configuration is implemented as a context. Also,
> the grid parameter- AF_MIN_BLOCK_WIDTH is set to 4 (default
> is 3) since if the default value is used, x_start
> (x_start > 640) will be at an incorrect location of the
> image (rightmost of the sensor).
> 
> Signed-off-by: Kate Hsuan <hpa@redhat.com>
> Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> ---
> Changes in v8:
> 1. Revised and improved the comments.
> 2. Simplified the algorithm.
> ---
>   src/ipa/ipu3/algorithms/af.cpp      | 435 ++++++++++++++++++++++++++++
>   src/ipa/ipu3/algorithms/af.h        |  79 +++++
>   src/ipa/ipu3/algorithms/meson.build |   1 +
>   src/ipa/ipu3/ipa_context.cpp        |  23 ++
>   src/ipa/ipu3/ipa_context.h          |  10 +
>   src/ipa/ipu3/ipu3.cpp               |   2 +
>   6 files changed, 550 insertions(+)
>   create mode 100644 src/ipa/ipu3/algorithms/af.cpp
>   create mode 100644 src/ipa/ipu3/algorithms/af.h
> 
> diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp
> new file mode 100644
> index 00000000..17055c04
> --- /dev/null
> +++ b/src/ipa/ipu3/algorithms/af.cpp
> @@ -0,0 +1,435 @@
> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> +/*
> + * Copyright (C) 2021, Red Hat
> + *
> + * af.cpp - IPU3 auto focus algorithm
> + */
> +
> +#include "af.h"
> +
> +#include <algorithm>
> +#include <chrono>
> +#include <cmath>
> +#include <fcntl.h>
> +#include <numeric>
> +#include <sys/ioctl.h>
> +#include <sys/stat.h>
> +#include <sys/types.h>
> +#include <unistd.h>
> +
> +#include <linux/videodev2.h>
> +
> +#include <libcamera/base/log.h>
> +
> +#include <libcamera/ipa/core_ipa_interface.h>
> +
> +#include "libipa/histogram.h"
> +
> +/**
> + * \file af.h
> + */
> +
> +/**
> + * \var kAfMinGridWidth
> + * \brief the minimum width of AF grid.
> + * The minimum grid horizontal dimensions.
> +*/
> +
> +/**
> + * \var kAfMinGridHeight
> + * \brief the minimum height of AF grid.
> + * The minimum grid vertical dimensions.
> +*/
> +
> +/**
> + * \var kAfMaxGridWidth
> + * \brief the maximum width of AF grid.
> + * The maximum grid horizontal dimensions.
> +*/
> +
> +/**
> + * \var kAfMaxGridHeight
> + * \brief The maximum height of AF grid.
> + * The maximum grid vertical dimensions.
> +*/
> +
> +/**
> + * \var kAfMinGridBlockWidth
> + * \brief The minimum block size of the width.
> + * The minimum value of Log2 of the width of the grid cell.
> + */
> +
> +/**
> + * \var kAfMinGridBlockHeight
> + * \brief The minimum block size of the height.
> + * The minimum value of Log2 of the height of the grid cell.
> + */
> +
> +/**
> + * \def kAfMaxGridBlockWidth
> + * \brief The maximum block size of the width.
> + * The maximum value of Log2 of the width of the grid cell.
> + */
> +
> +/**
> + * \var kAfMaxGridBlockHeight
> + * \brief The maximum block size of the height.
> + * The maximum value of Log2 of the height of the grid cell.
> + */
> +
> +/**
> + * \var kAfDefaultHeightPerSlice
> + * \brief The default number of blocks in vertical axis per slice.
> + * The number of blocks in vertical axis per slice.
> + */
> +
> +namespace libcamera {
> +
> +using namespace std::literals::chrono_literals;
> +
> +namespace ipa::ipu3::algorithms {
> +
> +LOG_DEFINE_CATEGORY(IPU3Af)
> +
> +/**
> + * Maximum focus steps of the VCM control
> + * \todo should be obtained from the VCM driver
> + */
> +static constexpr uint32_t kMaxFocusSteps = 1023;
> +
> +/* Minimum focus step for searching appropriate focus */
> +static constexpr uint32_t kCoarseSearchStep = 30;
> +static constexpr uint32_t kFineSearchStep = 1;
> +
> +/* Max ratio of variance change, 0.0 < kMaxChange < 1.0 */
> +static constexpr double kMaxChange = 0.5;
> +
> +/* The numbers of frame to be ignored, before performing focus scan. */
> +static constexpr uint32_t kIgnoreFrame = 10;
> +
> +/* Fine scan range 0 < kFineRange < 1 */
> +static constexpr double kFineRange = 0.05;
> +
> +/* Settings for IPU3 AF filter */
> +static struct ipu3_uapi_af_filter_config afFilterConfigDefault = {
> +	.y1_coeff_0 = { 0, 1, 3, 7 },
> +	.y1_coeff_1 = { 11, 13, 1, 2 },
> +	.y1_coeff_2 = { 8, 19, 34, 242 },
> +	.y1_sign_vec = 0x7fdffbfe,
> +	.y2_coeff_0 = { 0, 1, 6, 6 },
> +	.y2_coeff_1 = { 13, 25, 3, 0 },
> +	.y2_coeff_2 = { 25, 3, 177, 254 },
> +	.y2_sign_vec = 0x4e53ca72,
> +	.y_calc = { 8, 8, 8, 8 },
> +	.nf = { 0, 9, 0, 9, 0 },
> +};
> +
> +/**
> + * \class Af
> + * \brief An auto-focus algorithm based on IPU3 statistics
> + * This algorithm is used to determine the position of the lens to make a
> + * focused image. The IPU3 AF processing block computes the statistics that
> + * are composed by two types of filtered value and stores in a AF buffer.
> + * Typically, for a clear image, it has a relatively higher contrast than a
> + * blurred one. Therefore, if an image with the highest contrast can be
> + * found through the scan, the position of the len indicates to a clearest
> + * image.
> + */
> +Af::Af()
> +	: focus_(0), bestFocus_(0), currentVariance_(0.0), previousVariance_(0.0),
> +	  coarseCompleted_(false), fineCompleted_(false)
> +{
> +}
> +
> +/**
> + * \copydoc libcamera::ipa::Algorithm::prepare
> + */
> +void Af::prepare(IPAContext &context, ipu3_uapi_params *params)
> +{
> +	const struct ipu3_uapi_grid_config &grid = context.configuration.af.afGrid;
> +	params->acc_param.af.grid_cfg = grid;
> +	params->acc_param.af.filter_config = afFilterConfigDefault;
> +
> +	/* Enable AF processing block */
> +	params->use.acc_af = 1;
> +}
> +
> +/**
> + * \brief Configure the Af given a configInfo
> + * \param[in] context The shared IPA context
> + * \param[in] configInfo The IPA configuration data
> + * \return 0
> + */
> +int Af::configure(IPAContext &context, const IPAConfigInfo &configInfo)
> +{
> +	struct ipu3_uapi_grid_config &grid = context.configuration.af.afGrid;
> +	grid.width = kAfMinGridWidth;
> +	grid.height = kAfMinGridHeight;
> +	grid.block_width_log2 = kAfMinGridBlockWidth;
> +	grid.block_height_log2 = kAfMinGridBlockHeight;
> +	grid.height_per_slice = kAfDefaultHeightPerSlice;
> +
> +	/* x_start and y start are default to BDS center */
> +	grid.x_start = (configInfo.bdsOutputSize.width / 2) -
> +		       (((grid.width << grid.block_width_log2) / 2));
> +	grid.y_start = (configInfo.bdsOutputSize.height / 2) -
> +		       (((grid.height << grid.block_height_log2) / 2));
> +
> +	/* x_start and y_start should be even */
> +	grid.x_start = (grid.x_start / 2) * 2;
> +	grid.y_start = (grid.y_start / 2) * 2;
> +	grid.y_start = grid.y_start | IPU3_UAPI_GRID_Y_START_EN;
> +
> +	/* Initial max focus step */
> +	maxStep_ = kMaxFocusSteps;
> +
> +	/* Initial focus value */
> +	context.frameContext.af.focus = 0;
> +	/* Maximum variance of the AF statistics */
> +	context.frameContext.af.maxVariance = 0;
> +	/* The stable AF value flag. if it is true, the AF should be in a stable state. */
> +	context.frameContext.af.stable = false;
> +
> +	return 0;
> +}
> +
> +/**
> + * \brief AF coarse scan
> + * Find a near focused image using a coarse step. The step is determined by coarseSearchStep.
> + * \param[in] context The shared IPA context
> + */
> +void Af::afCoarseScan(IPAContext &context)
> +{
> +	if (coarseCompleted_)
> +		return;
> +
> +	if (afNeedIgnoreFrame())
> +		return;
> +
> +	if (afScan(context, kCoarseSearchStep)) {
> +		coarseCompleted_ = true;
> +		context.frameContext.af.maxVariance = 0;
> +		focus_ = context.frameContext.af.focus -
> +			 (context.frameContext.af.focus * kFineRange);
> +		context.frameContext.af.focus = focus_;
> +		previousVariance_ = 0;
> +		maxStep_ = std::clamp(focus_ + static_cast<uint32_t>((focus_ * kFineRange)),
> +				      0U, kMaxFocusSteps);
> +	}
> +}
> +
> +/**
> + * \brief AF fine scan
> + * Find an optimum lens position with moving 1 step for each search.
> + * \param[in] context The shared IPA context
> + */
> +void Af::afFineScan(IPAContext &context)
> +{
> +	if (!coarseCompleted_)
> +		return;
> +
> +	if (afNeedIgnoreFrame())
> +		return;
> +
> +	if (afScan(context, kFineSearchStep)) {
> +		context.frameContext.af.stable = true;
> +		fineCompleted_ = true;
> +	}
> +}
> +
> +/**
> + * \brief AF reset
> + * Reset all the parameters to start over the AF process.
> + * \param[in] context The shared IPA context
> + */
> +void Af::afReset(IPAContext &context)
> +{
> +	if (afNeedIgnoreFrame())
> +		return;
> +
> +	context.frameContext.af.maxVariance = 0;
> +	context.frameContext.af.focus = 0;
> +	focus_ = 0;
> +	context.frameContext.af.stable = false;
> +	ignoreCounter_ = kIgnoreFrame;
> +	previousVariance_ = 0.0;
> +	coarseCompleted_ = false;
> +	fineCompleted_ = false;
> +	maxStep_ = kMaxFocusSteps;
> +}
> +
> +/**
> + * \brief AF variance comparison.
> + * It always picks the largest variance to replace the previous one. The image
> + * with a larger variance also indicates it is a clearer image than previous
> + * one. If it finds the negative sign of derivative, it returns immediately.
> + * \param[in] context The IPA context
> + * \param min_step The VCM movement step.
> + * \return True, if it finds a AF value.
> + */
> +bool Af::afScan(IPAContext &context, int min_step)
> +{
> +	if (focus_ > maxStep_) {
> +		/* If reach the max step, move lens to the position. */
> +		context.frameContext.af.focus = bestFocus_;
> +		return true;
> +	} else {
> +		/*
> +		 * Find the maximum of the variance by estimating its
> +		 * derivative. If the direction changes, it means we have
> +		 * passed a maximum one step before.
> +		*/
> +		if ((currentVariance_ - context.frameContext.af.maxVariance) >=
> +		    -(context.frameContext.af.maxVariance * 0.1)) {
> +			/*
> +			 * Positive and zero derivative:
> +			 * The variance is still increasing. The focus could be
> +			 * increased for the next comparison. Also, the max variance
> +			 * and previous focus value are updated.
> +			 */
> +			bestFocus_ = focus_;
> +			focus_ += min_step;
> +			context.frameContext.af.focus = focus_;
> +			context.frameContext.af.maxVariance = currentVariance_;
> +		} else {
> +			/*
> +			 * Negative derivative:
> +			 * The variance starts to decrease which means the maximum
> +			 * variance is found. Set focus step to previous good one
> +			 * then return immediately.
> +			 */
> +			context.frameContext.af.focus = bestFocus_;
> +			return true;
> +		}
> +	}
> +
> +	previousVariance_ = currentVariance_;
> +	LOG(IPU3Af, Debug) << " Previous step is "
> +			   << bestFocus_
> +			   << " Current step is "
> +			   << focus_;
> +	return false;
> +}
> +
> +/**
> + * \brief Determine the frame to be ignored.
> + * \return Return true the frame is ignored.
> + * \return Return false the frame should be processed.
> + */
> +bool Af::afNeedIgnoreFrame()
> +{
> +	if (ignoreCounter_ == 0)
> +		return false;
> +	else
> +		ignoreCounter_--;
> +	return true;
> +}
> +
> +/**
> + * \brief Reset frame ignore counter.
> + */
> +void Af::afIgnoreFrameReset()
> +{
> +	ignoreCounter_ = kIgnoreFrame;
> +}
> +
> +/**
> + * \brief Estemate variance
> + */
> +double Af::afEstemateVariance(y_table_item_t *y_item, uint32_t len,
> +			      bool isY1)
> +{
> +	uint32_t z = 0;
> +	uint32_t total = 0;
> +	double mean;
> +	double var_sum = 0;
> +
> +	for (z = 0; z < len; z++) {
> +		if (isY1)
> +			total += y_item[z].y1_avg;
> +		else
> +			total += y_item[z].y2_avg;
> +	}
> +	mean = total / len;
> +	for (z = 0; z < len; z++) {
> +		if (isY1)
> +			var_sum += pow((y_item[z].y1_avg - mean), 2);
> +		else
> +			var_sum += pow((y_item[z].y2_avg - mean), 2);
> +	}
> +
> +	return var_sum / static_cast<double>(len);
> +}
> +
> +/**
> + * \brief Determine out-of-focus situation.
> + * Out-of-focus means that the variance change rate for a focused and a new
> + * variance is greater than a threshold.
> + * \param context The IPA context.
> + * \return If it is out-of-focus, return true.
> + * \return If is is focused, return false.
> + */
> +bool Af::afIsOutOfFocus(IPAContext context)
> +{
> +	const uint32_t diff_var = std::abs(currentVariance_ -
> +					   context.frameContext.af.maxVariance);
> +	const double var_ratio = diff_var / context.frameContext.af.maxVariance;
> +	LOG(IPU3Af, Debug) << "Variance change rate: "
> +			   << var_ratio
> +			   << " Current VCM step: "
> +			   << context.frameContext.af.focus;
> +	if (var_ratio > kMaxChange)
> +		return true;
> +	else
> +		return false;
> +}
> +
> +/**
> + * \brief Determine the max contrast image and lens position.
> + * Ideally, a clear image also has a raletively higher contrast. So, every
> + * images for each focus step should be tested to find a optimal focus step.
> + * The Hill Climbing Algorithm[1] is used to find the maximum variance of the
> + * AF statistic which is the AF output of IPU3. The focus step is increased
> + * then the variance of the AF statistic is estimated. If it finds the negative
> + * derivative which means we just passed the peak, the best focus is found.
> + *
> + * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing

Thanks for this reference ;-).
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>

> + * \param[in] context The IPA context.
> + * \param[in] stats The statistic buffer of IPU3.
> + */
> +void Af::process(IPAContext &context, const ipu3_uapi_stats_3a *stats)
> +{
> +	y_table_item_t y_item[IPU3_UAPI_AF_Y_TABLE_MAX_SIZE / sizeof(y_table_item_t)];
> +	uint32_t afRawBufferLen;
> +
> +	/* Evaluate the AF buffer length */
> +	afRawBufferLen = context.configuration.af.afGrid.width *
> +			 context.configuration.af.afGrid.height;
> +
> +	memcpy(y_item, stats->af_raw_buffer.y_table,
> +	       afRawBufferLen * sizeof(y_table_item_t));
> +
> +	/*
> +	 * Calculate the mean and the variance of AF statistics for a given grid.
> +	 * For coarse: y1 are used.
> +	 * For fine: y2 results are used.
> +	 */
> +	if (coarseCompleted_)
> +		currentVariance_ = afEstemateVariance(y_item, afRawBufferLen, false);
> +	else
> +		currentVariance_ = afEstemateVariance(y_item, afRawBufferLen, true);
> +
> +	if (!context.frameContext.af.stable) {
> +		afCoarseScan(context);
> +		afFineScan(context);
> +	} else {
> +		if (afIsOutOfFocus(context))
> +			afReset(context);
> +		else
> +			afIgnoreFrameReset();
> +	}
> +}
> +
> +} /* namespace ipa::ipu3::algorithms */
> +
> +} /* namespace libcamera */
> diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h
> new file mode 100644
> index 00000000..2ca78c84
> --- /dev/null
> +++ b/src/ipa/ipu3/algorithms/af.h
> @@ -0,0 +1,79 @@
> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> +/*
> + * Copyright (C) 2021, Red Hat
> + *
> + * af.h - IPU3 Af algorithm
> + */
> +
> +#pragma once
> +
> +#include <linux/intel-ipu3.h>
> +
> +#include <libcamera/base/utils.h>
> +
> +#include <libcamera/geometry.h>
> +
> +#include "algorithm.h"
> +
> +/* Static variables from repo of chromium */
> +static constexpr uint8_t kAfMinGridWidth = 16;
> +static constexpr uint8_t kAfMinGridHeight = 16;
> +static constexpr uint8_t kAfMaxGridWidth = 32;
> +static constexpr uint8_t kAfMaxGridHeight = 24;
> +static constexpr uint16_t kAfMinGridBlockWidth = 4;
> +static constexpr uint16_t kAfMinGridBlockHeight = 3;
> +static constexpr uint16_t kAfMaxGridBlockWidth = 6;
> +static constexpr uint16_t kAfMaxGridBlockHeight = 6;
> +static constexpr uint16_t kAfDefaultHeightPerSlice = 2;
> +
> +namespace libcamera {
> +
> +namespace ipa::ipu3::algorithms {
> +
> +class Af : public Algorithm
> +{
> +	/* The format of y_table. From ipu3-ipa repo */
> +	typedef struct __attribute__((packed)) y_table_item {
> +		uint16_t y1_avg;
> +		uint16_t y2_avg;
> +	} y_table_item_t;
> +public:
> +	Af();
> +	~Af() = default;
> +
> +	void prepare(IPAContext &context, ipu3_uapi_params *params) override;
> +	int configure(IPAContext &context, const IPAConfigInfo &configInfo) override;
> +	void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override;
> +
> +private:
> +	void afCoarseScan(IPAContext &context);
> +	void afFineScan(IPAContext &context);
> +	bool afScan(IPAContext &context, int min_step);
> +	void afReset(IPAContext &context);
> +	bool afNeedIgnoreFrame();
> +	void afIgnoreFrameReset();
> +	double afEstemateVariance(y_table_item_t *y_item, uint32_t len,
> +				  bool isY1);
> +	bool afIsOutOfFocus(IPAContext context);
> +
> +	/* VCM step configuration. It is the current setting of the VCM step. */
> +	uint32_t focus_;
> +	/* The best VCM step. It is a local optimum VCM step during scanning. */
> +	uint32_t bestFocus_;
> +	/* Current AF statistic variance. */
> +	double currentVariance_;
> +	/* The frames are ignore before starting measuring. */
> +	uint32_t ignoreCounter_;
> +	/* It is used to determine the derivative during scanning */
> +	double previousVariance_;
> +	/* The designated maximum range of focus scanning. */
> +	uint32_t maxStep_;
> +	/* If the coarse scan completes, it is set to true. */
> +	bool coarseCompleted_;
> +	/* If the fine scan completes, it is set to true. */
> +	bool fineCompleted_;
> +};
> +
> +} /* namespace ipa::ipu3::algorithms */
> +
> +} /* namespace libcamera */
> diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build
> index 4db6ae1d..b70a551c 100644
> --- a/src/ipa/ipu3/algorithms/meson.build
> +++ b/src/ipa/ipu3/algorithms/meson.build
> @@ -1,6 +1,7 @@
>   # SPDX-License-Identifier: CC0-1.0
>   
>   ipu3_ipa_algorithms = files([
> +    'af.cpp',
>       'agc.cpp',
>       'awb.cpp',
>       'blc.cpp',
> diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp
> index 86794ac1..e8f7367c 100644
> --- a/src/ipa/ipu3/ipa_context.cpp
> +++ b/src/ipa/ipu3/ipa_context.cpp
> @@ -69,6 +69,29 @@ namespace libcamera::ipa::ipu3 {
>    * \brief Number of cells on one line including the ImgU padding
>    */
>   
> +/**
> + * \var IPASessionConfiguration::af
> + * \brief AF grid configuration of the IPA
> + *
> + * \var IPASessionConfiguration::af.afGrid
> + * \brief AF scene grid configuration.
> + */
> +
> +/**
> + * \var IPAFrameContext::af
> + * \brief Context for the Automatic Focus algorithm
> + *
> + * \struct  IPAFrameContext::af
> + * \var IPAFrameContext::af.focus
> + * \brief Current position of the lens
> + *
> + * \var IPAFrameContext::af.maxVariance
> + * \brief The maximum variance of the current image.
> + *
> + * \var IPAFrameContext::af.stable
> + * \brief It is set to true, if the best focus is found.
> + */
> +
>   /**
>    * \var IPASessionConfiguration::agc
>    * \brief AGC parameters configuration of the IPA
> diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h
> index c6dc0814..60ad3194 100644
> --- a/src/ipa/ipu3/ipa_context.h
> +++ b/src/ipa/ipu3/ipa_context.h
> @@ -25,6 +25,10 @@ struct IPASessionConfiguration {
>   		uint32_t stride;
>   	} grid;
>   
> +	struct {
> +		ipu3_uapi_grid_config afGrid;
> +	} af;
> +
>   	struct {
>   		utils::Duration minShutterSpeed;
>   		utils::Duration maxShutterSpeed;
> @@ -34,6 +38,12 @@ struct IPASessionConfiguration {
>   };
>   
>   struct IPAFrameContext {
> +	struct {
> +		uint32_t focus;
> +		double maxVariance;
> +		bool stable;
> +	} af;
> +
>   	struct {
>   		uint32_t exposure;
>   		double gain;
> diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp
> index e44a31bb..417e0562 100644
> --- a/src/ipa/ipu3/ipu3.cpp
> +++ b/src/ipa/ipu3/ipu3.cpp
> @@ -30,6 +30,7 @@
>   
>   #include "libcamera/internal/mapped_framebuffer.h"
>   
> +#include "algorithms/af.h"
>   #include "algorithms/agc.h"
>   #include "algorithms/algorithm.h"
>   #include "algorithms/awb.h"
> @@ -295,6 +296,7 @@ int IPAIPU3::init(const IPASettings &settings,
>   	}
>   
>   	/* Construct our Algorithms */
> +	algorithms_.push_back(std::make_unique<algorithms::Af>());
>   	algorithms_.push_back(std::make_unique<algorithms::Agc>());
>   	algorithms_.push_back(std::make_unique<algorithms::Awb>());
>   	algorithms_.push_back(std::make_unique<algorithms::BlackLevelCorrection>());