[{"id":25146,"web_url":"https://patchwork.libcamera.org/comment/25146/","msgid":"<20220928070656.l5c5wjxcvctsskgw@uno.localdomain>","date":"2022-09-28T07:06:56","subject":"Re: [libcamera-devel] [PATCH 3/3] ipa: ipu3: af: AfMode and\n\tLensPosition control implementation","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Kate,\n\nOn Fri, Sep 16, 2022 at 06:37:13PM +0800, Kate Hsuan via libcamera-devel wrote:\n> AfMode controls include manual, auto, and continuous modes. For auto\n> and continuous modes, both of the algorithms try to keep the image\n> focused. The difference between them is that continuous mode has higher\n> sensitivity to the change of image variance than auto mode. So, the AF\n> is frequently triggered in continuous mode. The user-defined lens\n> position can only be set in manual mode.\n\nWe discussed in the past with David and Naush from RPi if contrast-based\nCAF without PDAF statistics from the sensor is a good idea or not. I\ncannot tell how it performs yet, and if we want it or not, but for\nsake of simplicity can we split this patch in two, one that adds\nmanual mode support and another one that adds CAF support ?\n\nThe introduction of Manual mode alone would also help with aligning\nthis design with the other algorithms that has been ported to use the\nFrameContext queue in:\nhttps://patchwork.libcamera.org/project/libcamera/list/?series=3506\non which this series would be rebased.\n\nI would:\n1) Make use of the frame context and active state for auto mode (if\n necessary)\n2) Make use of frame context for manual mode like it's done in that\n series for AWB on RkISP1 to see if the concepts apply to AF as\n well. It won't be straight-forward as the active state and frame\n context are still evolvng and each algorithms might have different\n requirements, but to validate them we should try to port all\n algorithms to use them\n3) Eventually implement CAF\n\nA question remains: how to express lens movement ranges. This requires\na larger discussion and I think you're free to do what's best for your\nsetup for this first implementation, including using directly the range\nof the lens used in your setup.\n\n>\n> Signed-off-by: Kate Hsuan \n> ---\n> src/ipa/ipu3/algorithms/af.cpp | 106 ++++++++++++++++++++++++++++-----\n> src/ipa/ipu3/algorithms/af.h | 15 +++++\n> 2 files changed, 105 insertions(+), 16 deletions(-)\n>\n> diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp\n> index 4835a034..c57d3e18 100644\n> --- a/src/ipa/ipu3/algorithms/af.cpp\n> +++ b/src/ipa/ipu3/algorithms/af.cpp\n> @@ -21,6 +21,8 @@\n>\n> #include \n>\n> +#include \n> +\n> #include \n>\n> #include \"libipa/histogram.h\"\n> @@ -109,7 +111,8 @@ static struct ipu3_uapi_af_filter_config afFilterConfigDefault = {\n> */\n> Af::Af()\n> \t: focus_(0), bestFocus_(0), currentVariance_(0.0), previousVariance_(0.0),\n> -\t coarseCompleted_(false), fineCompleted_(false)\n> +\t coarseCompleted_(false), fineCompleted_(false), maxChange_(kMaxChange),\n> +\t afMode_(controls::AfModeAuto)\n> {\n> }\n>\n> @@ -194,6 +197,69 @@ int Af::configure(IPAContext &context, const IPAConfigInfo &configInfo)\n> \treturn 0;\n> }\n>\n> +/**\n> + * \\brief AF controls handler\n> + *\n> + * Put the control parameter to the corresponding variables when receiving the controls\n> + * from the user.\n\nDocumentation comes after the parameters documentation.\nAlso you can use\n\n * \\copydoc libcamera::ipa::Algorithm::queueRequest\n\n> + * \\param[in] context The shared IPA context\n> + * \\param[in] frame Frame number\n> + * \\param[in] controls control list of the request\n> + */\n> +void Af::queueRequest([[maybe_unused]] IPAContext &context,\n> +\t\t [[maybe_unused]] const uint32_t frame,\n> +\t\t const ControlList &controls)\n> +{\n> +\tfor (auto const &ctrl : controls) {\n\nYou can\n const auto &afMode = controls.get(controls::AfMode;\n if (afMode && *afMode == controls::afModeAuto...\n\nif it helps you\n\n> +\t\tunsigned int ctrlEnum = ctrl.first;\n> +\t\tconst ControlValue &ctrlValue = ctrl.second;\n> +\n> +\t\tswitch (ctrlEnum) {\n> +\t\tcase controls::AF_MODE:\n> +\t\t\tafModeSet(ctrlValue.get());\n> +\t\t\tbreak;\n> +\t\tcase controls::LENS_POSITION:\n> +\t\t\tlensPosition_ = ctrlValue.get();\n\nWe need a bit more of logic here.\n\nIf AfModeAuto then controls::lensPosition should be ignored.\n\nIn AfModeAuto auto-focus scans are driven by the AfTrigger control.\nIt is specified in control_ids.yaml as:\n\n - name: AfModeAuto\n value: 1\n description: |\n The AF algorithm is in auto mode. This means that the algorithm\n will never move the lens or change state unless the AfTrigger\n control is used. The AfTrigger control can be used to initiate a\n focus scan, the results of which will be reported by AfState.\n\n If the autofocus algorithm is moved from AfModeAuto to another\n mode while a scan is in progress, the scan is cancelled\n immediately, without waiting for the scan to finish.\n\n When first entering this mode the AfState will report\n AfStateIdle. When a trigger control is sent, AfState will\n report AfStateScanning for a period before spontaneously\n changing to AfStateFocused or AfStateFailed, depending on\n the outcome of the scan. It will remain in this state until\n another scan is initiated by the AfTrigger control. If a scan is\n cancelled (without changing to another mode), AfState will return\n to AfStateIdle.\n\nYou also need to populate AfState which if I'm not mistaken is\ncurrently not reported.\n\nWhat do you think about implementing AfTrigger and AfState support on\nthe existing Auto mode implementation (using the\nFrameContext/ActiveState) before adding Manual/CAF modes ?\n\n> +\t\t}\n> +\t}\n> +}\n> +\n> +/**\n> + * \\brief AF Mode set\n> + *\n> + * Set AF mode, including manual, auto, and continuous.\n> + *\n> + * \\param[in] AF operation mode 0, 1, 2 are manual, auto, and continuous, respectively.\n\nSame comments, parameters documentation first\n\n> + */\n> +\n> +void Af::afModeSet(uint32_t mode)\n> +{\n> +\tswitch (mode) {\n> +\tcase controls::AfModeManual:\n> +\tcase controls::AfModeAuto:\n> +\t\tafMode_ = mode;\n> +\t\tbreak;\n> +\tcase controls::AfModeContinuous:\n> +\t\tafMode_ = mode;\n> +\t\tmaxChange_ = 0.05;\n> +\t\tbreak;\n> +\tdefault:\n> +\t\tafMode_ = controls::AfModeAuto;\n\nThe current afMode (auto or manual or CAF) should in example be part\nof the active state probably\n\n> +\t}\n> +\n> +\tLOG(IPU3Af, Debug) << \"AfMode set \" << mode;\n> +}\n> +\n> +/**\n> + * \\brief Set lens position\n> + *\n> + * Set user-defined lens position to the focus steps.\n> + */\n> +void Af::afLensPositionSet(IPAContext &context)\n> +{\n> +\tcontext.activeState.af.focus = kMaxFocusSteps * lensPosition_;\n> +}\n> +\n> /**\n> * \\brief AF coarse scan\n> *\n> @@ -397,7 +463,7 @@ bool Af::afIsOutOfFocus(IPAContext context)\n> \t\t\t << \" Current VCM step: \"\n> \t\t\t << context.activeState.af.focus;\n>\n> -\tif (var_ratio > kMaxChange)\n> +\tif (var_ratio > maxChange_)\n> \t\treturn true;\n> \telse\n> \t\treturn false;\n> @@ -432,21 +498,29 @@ void Af::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameCon\n> \tSpan y_items(reinterpret_cast(&stats->af_raw_buffer.y_table),\n> \t\t\t\t\t afRawBufferLen);\n>\n> -\t/*\n> -\t * Calculate the mean and the variance of AF statistics for a given grid.\n> -\t * For coarse: y1 are used.\n> -\t * For fine: y2 results are used.\n> -\t */\n> -\tcurrentVariance_ = afEstimateVariance(y_items, !coarseCompleted_);\n> +\tswitch (afMode_) {\n> +\tcase controls::AfModeManual:\n> +\t\tafLensPositionSet(context);\n> +\t\tbreak;\n> +\tcase controls::AfModeContinuous:\n> +\tcase controls::AfModeAuto:\n> +\tdefault:\n> +\t\t/*\n> +\t\t * Calculate the mean and the variance of AF statistics for a given grid.\n> +\t\t * For coarse: y1 are used.\n> +\t\t * For fine: y2 results are used.\n> +\t\t */\n> +\t\tcurrentVariance_ = afEstimateVariance(y_items, !coarseCompleted_);\n>\n> -\tif (!context.activeState.af.stable) {\n> -\t\tafCoarseScan(context);\n> -\t\tafFineScan(context);\n> -\t} else {\n> -\t\tif (afIsOutOfFocus(context))\n> -\t\t\tafReset(context);\n> -\t\telse\n> -\t\t\tafIgnoreFrameReset();\n> +\t\tif (!context.activeState.af.stable) {\n> +\t\t\tafCoarseScan(context);\n> +\t\t\tafFineScan(context);\n> +\t\t} else {\n> +\t\t\tif (afIsOutOfFocus(context))\n> +\t\t\t\tafReset(context);\n> +\t\t\telse\n> +\t\t\t\tafIgnoreFrameReset();\n> +\t\t}\n> \t}\n> }\n>\n> diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h\n> index ccf015f3..77bab2a7 100644\n> --- a/src/ipa/ipu3/algorithms/af.h\n> +++ b/src/ipa/ipu3/algorithms/af.h\n> @@ -26,6 +26,7 @@ class Af : public Algorithm\n> \t\tuint16_t y1_avg;\n> \t\tuint16_t y2_avg;\n> \t} y_table_item_t;\n> +\n> public:\n> \tAf();\n> \t~Af() = default;\n> @@ -34,6 +35,9 @@ public:\n> \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n> \tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n> \t\t const ipu3_uapi_stats_3a *stats) override;\n> +\tvoid queueRequest([[maybe_unused]] IPAContext &context,\n> +\t\t\t [[maybe_unused]] const uint32_t frame,\n> +\t\t\t [[maybe_unused]] const ControlList &controls) override;\n>\n> private:\n> \tvoid afCoarseScan(IPAContext &context);\n> @@ -46,6 +50,11 @@ private:\n>\n> \tbool afIsOutOfFocus(IPAContext context);\n>\n> +\tvoid afModeSet(uint32_t mode);\n> +\tvoid afModeGet();\n> +\n> +\tvoid afLensPositionSet(IPAContext &context);\n> +\n> \t/* VCM step configuration. It is the current setting of the VCM step. */\n> \tuint32_t focus_;\n> \t/* The best VCM step. It is a local optimum VCM step during scanning. */\n> @@ -62,6 +71,12 @@ private:\n> \tbool coarseCompleted_;\n> \t/* If the fine scan completes, it is set to true. */\n> \tbool fineCompleted_;\n> +\t/* Max focus change ratio to determine */\n> +\tdouble maxChange_;\n> +\t/* Relative lens position in percentage. */\n> +\tdouble lensPosition_;\n> +\t/* Af operation mode. */\n> +\tuint32_t afMode_;\n> };\n>\n> } /* namespace ipa::ipu3::algorithms */\n> --\n> 2.37.3\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 4EF54BD16B\n\tfor ;\n\tWed, 28 Sep 2022 07:07:02 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 95B5A62292;\n\tWed, 28 Sep 2022 09:07:01 +0200 (CEST)","from relay11.mail.gandi.net (relay11.mail.gandi.net\n\t[IPv6:2001:4b98:dc4:8::231])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id BCC7561F7A\n\tfor ;\n\tWed, 28 Sep 2022 09:06:59 +0200 (CEST)","(Authenticated sender: jacopo@jmondi.org)\n\tby mail.gandi.net (Postfix) with ESMTPSA id A2797100004;\n\tWed, 28 Sep 2022 07:06:58 +0000 (UTC)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1664348821;\n\tbh=7Cm8y3HVhCK5RzH7nTKbTEtdpPtNhmBU+NM1fhzT/HE=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=ik778FPKhzOLBMIL78VwT6qn3cCksRZkZFNztv1N8+QRgqtyadyRvbIKAS3Ybvms0\n\t9FMoqDTHTI7Mlwf4C75RjwNIAofyvI/tr27i5TUvdcm+z8nCGEqR47Vuj+6jwT+N6m\n\tiBfr/+bClv45iSqaIwOwh2XiLGIv7uejw1dymH7UBzGXXAu6QgOoP0CYIz2ZjI9I/w\n\tc7+vUVDXv/WgSc6B4QiAwCgHsXGuD+3cKcBCtTVR91dhxi5aj39NaNB9kgcGdPasMT\n\t1iggrbegIgVmHcY1CgKNfnXyys+0mh2U8nzGm+aQQPyNzhhatOZWcnp0SPpR4NskP0\n\ttP/gJe2xt5FqQ==","Date":"Wed, 28 Sep 2022 09:06:56 +0200","To":"Kate Hsuan ","Message-ID":"<20220928070656.l5c5wjxcvctsskgw@uno.localdomain>","References":"<20220916103713.21132-1-hpa@redhat.com>\n\t<20220916103713.21132-4-hpa@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20220916103713.21132-4-hpa@redhat.com>","Subject":"Re: [libcamera-devel] [PATCH 3/3] ipa: ipu3: af: AfMode and\n\tLensPosition control implementation","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Jacopo Mondi via libcamera-devel ","Reply-To":"Jacopo Mondi ","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":25165,"web_url":"https://patchwork.libcamera.org/comment/25165/","msgid":"","date":"2022-09-29T06:46:37","subject":"Re: [libcamera-devel] [PATCH 3/3] ipa: ipu3: af: AfMode and\n\tLensPosition control implementation","submitter":{"id":105,"url":"https://patchwork.libcamera.org/api/people/105/","name":"Kate Hsuan","email":"hpa@redhat.com"},"content":"On Wed, Sep 28, 2022 at 3:07 PM Jacopo Mondi wrote:\n>\n> Hi Kate,\n>\n> On Fri, Sep 16, 2022 at 06:37:13PM +0800, Kate Hsuan via libcamera-devel wrote:\n> > AfMode controls include manual, auto, and continuous modes. For auto\n> > and continuous modes, both of the algorithms try to keep the image\n> > focused. The difference between them is that continuous mode has higher\n> > sensitivity to the change of image variance than auto mode. So, the AF\n> > is frequently triggered in continuous mode. The user-defined lens\n> > position can only be set in manual mode.\n>\n> We discussed in the past with David and Naush from RPi if contrast-based\n> CAF without PDAF statistics from the sensor is a good idea or not. I\n> cannot tell how it performs yet, and if we want it or not, but for\n> sake of simplicity can we split this patch in two, one that adds\n> manual mode support and another one that adds CAF support ?\n\nOkay. No problem.\n\n>\n> The introduction of Manual mode alone would also help with aligning\n> this design with the other algorithms that has been ported to use the\n> FrameContext queue in:\n> https://patchwork.libcamera.org/project/libcamera/list/?series=3506\n> on which this series would be rebased.\n>\n> I would:\n> 1) Make use of the frame context and active state for auto mode (if\n> necessary)\n> 2) Make use of frame context for manual mode like it's done in that\n> series for AWB on RkISP1 to see if the concepts apply to AF as\n> well. It won't be straight-forward as the active state and frame\n> context are still evolvng and each algorithms might have different\n> requirements, but to validate them we should try to port all\n> algorithms to use them\n\nYou mean all the controls should use framecontext. Okay, I would base\non this series\nto propose my v2 patch.\n\n> 3) Eventually implement CAF\n>\n> A question remains: how to express lens movement ranges. This requires\n> a larger discussion and I think you're free to do what's best for your\n> setup for this first implementation, including using directly the range\n> of the lens used in your setup.\n\nThis is a good question for deciding whether to use the percentage or\nthe range of the lens.\nA normalized value such as a floating point based percentage or 0-1000\ninteger (basically, it is 1000 times of percentage) is good for the\napplication implementation.\nSo, here, I would like to use normalized values to pass the value\nbetween the application and the algorithm.\n\n>\n> >\n> > Signed-off-by: Kate Hsuan \n> > ---\n> > src/ipa/ipu3/algorithms/af.cpp | 106 ++++++++++++++++++++++++++++-----\n> > src/ipa/ipu3/algorithms/af.h | 15 +++++\n> > 2 files changed, 105 insertions(+), 16 deletions(-)\n> >\n> > diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp\n> > index 4835a034..c57d3e18 100644\n> > --- a/src/ipa/ipu3/algorithms/af.cpp\n> > +++ b/src/ipa/ipu3/algorithms/af.cpp\n> > @@ -21,6 +21,8 @@\n> >\n> > #include \n> >\n> > +#include \n> > +\n> > #include \n> >\n> > #include \"libipa/histogram.h\"\n> > @@ -109,7 +111,8 @@ static struct ipu3_uapi_af_filter_config afFilterConfigDefault = {\n> > */\n> > Af::Af()\n> > : focus_(0), bestFocus_(0), currentVariance_(0.0), previousVariance_(0.0),\n> > - coarseCompleted_(false), fineCompleted_(false)\n> > + coarseCompleted_(false), fineCompleted_(false), maxChange_(kMaxChange),\n> > + afMode_(controls::AfModeAuto)\n> > {\n> > }\n> >\n> > @@ -194,6 +197,69 @@ int Af::configure(IPAContext &context, const IPAConfigInfo &configInfo)\n> > return 0;\n> > }\n> >\n> > +/**\n> > + * \\brief AF controls handler\n> > + *\n> > + * Put the control parameter to the corresponding variables when receiving the controls\n> > + * from the user.\n>\n> Documentation comes after the parameters documentation.\n> Also you can use\n\nOkay\n\n>\n> * \\copydoc libcamera::ipa::Algorithm::queueRequest\n>\n> > + * \\param[in] context The shared IPA context\n> > + * \\param[in] frame Frame number\n> > + * \\param[in] controls control list of the request\n> > + */\n> > +void Af::queueRequest([[maybe_unused]] IPAContext &context,\n> > + [[maybe_unused]] const uint32_t frame,\n> > + const ControlList &controls)\n> > +{\n> > + for (auto const &ctrl : controls) {\n>\n> You can\n> const auto &afMode = controls.get(controls::AfMode;\n> if (afMode && *afMode == controls::afModeAuto...\n>\n> if it helps you\n>\n> > + unsigned int ctrlEnum = ctrl.first;\n> > + const ControlValue &ctrlValue = ctrl.second;\n> > +\n> > + switch (ctrlEnum) {\n> > + case controls::AF_MODE:\n> > + afModeSet(ctrlValue.get());\n> > + break;\n> > + case controls::LENS_POSITION:\n> > + lensPosition_ = ctrlValue.get();\n>\n> We need a bit more of logic here.\n>\n> If AfModeAuto then controls::lensPosition should be ignored.\n>\n> In AfModeAuto auto-focus scans are driven by the AfTrigger control.\n> It is specified in control_ids.yaml as:\n>\n> - name: AfModeAuto\n> value: 1\n> description: |\n> The AF algorithm is in auto mode. This means that the algorithm\n> will never move the lens or change state unless the AfTrigger\n> control is used. The AfTrigger control can be used to initiate a\n> focus scan, the results of which will be reported by AfState.\n>\n> If the autofocus algorithm is moved from AfModeAuto to another\n> mode while a scan is in progress, the scan is cancelled\n> immediately, without waiting for the scan to finish.\n>\n> When first entering this mode the AfState will report\n> AfStateIdle. When a trigger control is sent, AfState will\n> report AfStateScanning for a period before spontaneously\n> changing to AfStateFocused or AfStateFailed, depending on\n> the outcome of the scan. It will remain in this state until\n> another scan is initiated by the AfTrigger control. If a scan is\n> cancelled (without changing to another mode), AfState will return\n> to AfStateIdle.\n>\n> You also need to populate AfState which if I'm not mistaken is\n> currently not reported.\n>\n> What do you think about implementing AfTrigger and AfState support on\n> the existing Auto mode implementation (using the\n> FrameContext/ActiveState) before adding Manual/CAF modes ?\n\nOkay. I'll try implementing them in AutoMode first.\n\n\n>\n> > + }\n> > + }\n> > +}\n> > +\n> > +/**\n> > + * \\brief AF Mode set\n> > + *\n> > + * Set AF mode, including manual, auto, and continuous.\n> > + *\n> > + * \\param[in] AF operation mode 0, 1, 2 are manual, auto, and continuous, respectively.\n>\n> Same comments, parameters documentation first\n>\n> > + */\n> > +\n> > +void Af::afModeSet(uint32_t mode)\n> > +{\n> > + switch (mode) {\n> > + case controls::AfModeManual:\n> > + case controls::AfModeAuto:\n> > + afMode_ = mode;\n> > + break;\n> > + case controls::AfModeContinuous:\n> > + afMode_ = mode;\n> > + maxChange_ = 0.05;\n> > + break;\n> > + default:\n> > + afMode_ = controls::AfModeAuto;\n>\n> The current afMode (auto or manual or CAF) should in example be part\n> of the active state probably\n\nOkay.\n\n\n>\n> > + }\n> > +\n> > + LOG(IPU3Af, Debug) << \"AfMode set \" << mode;\n> > +}\n> > +\n> > +/**\n> > + * \\brief Set lens position\n> > + *\n> > + * Set user-defined lens position to the focus steps.\n> > + */\n> > +void Af::afLensPositionSet(IPAContext &context)\n> > +{\n> > + context.activeState.af.focus = kMaxFocusSteps * lensPosition_;\n> > +}\n> > +\n> > /**\n> > * \\brief AF coarse scan\n> > *\n> > @@ -397,7 +463,7 @@ bool Af::afIsOutOfFocus(IPAContext context)\n> > << \" Current VCM step: \"\n> > << context.activeState.af.focus;\n> >\n> > - if (var_ratio > kMaxChange)\n> > + if (var_ratio > maxChange_)\n> > return true;\n> > else\n> > return false;\n> > @@ -432,21 +498,29 @@ void Af::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameCon\n> > Span y_items(reinterpret_cast(&stats->af_raw_buffer.y_table),\n> > afRawBufferLen);\n> >\n> > - /*\n> > - * Calculate the mean and the variance of AF statistics for a given grid.\n> > - * For coarse: y1 are used.\n> > - * For fine: y2 results are used.\n> > - */\n> > - currentVariance_ = afEstimateVariance(y_items, !coarseCompleted_);\n> > + switch (afMode_) {\n> > + case controls::AfModeManual:\n> > + afLensPositionSet(context);\n> > + break;\n> > + case controls::AfModeContinuous:\n> > + case controls::AfModeAuto:\n> > + default:\n> > + /*\n> > + * Calculate the mean and the variance of AF statistics for a given grid.\n> > + * For coarse: y1 are used.\n> > + * For fine: y2 results are used.\n> > + */\n> > + currentVariance_ = afEstimateVariance(y_items, !coarseCompleted_);\n> >\n> > - if (!context.activeState.af.stable) {\n> > - afCoarseScan(context);\n> > - afFineScan(context);\n> > - } else {\n> > - if (afIsOutOfFocus(context))\n> > - afReset(context);\n> > - else\n> > - afIgnoreFrameReset();\n> > + if (!context.activeState.af.stable) {\n> > + afCoarseScan(context);\n> > + afFineScan(context);\n> > + } else {\n> > + if (afIsOutOfFocus(context))\n> > + afReset(context);\n> > + else\n> > + afIgnoreFrameReset();\n> > + }\n> > }\n> > }\n> >\n> > diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h\n> > index ccf015f3..77bab2a7 100644\n> > --- a/src/ipa/ipu3/algorithms/af.h\n> > +++ b/src/ipa/ipu3/algorithms/af.h\n> > @@ -26,6 +26,7 @@ class Af : public Algorithm\n> > uint16_t y1_avg;\n> > uint16_t y2_avg;\n> > } y_table_item_t;\n> > +\n> > public:\n> > Af();\n> > ~Af() = default;\n> > @@ -34,6 +35,9 @@ public:\n> > int configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n> > void process(IPAContext &context, IPAFrameContext *frameContext,\n> > const ipu3_uapi_stats_3a *stats) override;\n> > + void queueRequest([[maybe_unused]] IPAContext &context,\n> > + [[maybe_unused]] const uint32_t frame,\n> > + [[maybe_unused]] const ControlList &controls) override;\n> >\n> > private:\n> > void afCoarseScan(IPAContext &context);\n> > @@ -46,6 +50,11 @@ private:\n> >\n> > bool afIsOutOfFocus(IPAContext context);\n> >\n> > + void afModeSet(uint32_t mode);\n> > + void afModeGet();\n> > +\n> > + void afLensPositionSet(IPAContext &context);\n> > +\n> > /* VCM step configuration. It is the current setting of the VCM step. */\n> > uint32_t focus_;\n> > /* The best VCM step. It is a local optimum VCM step during scanning. */\n> > @@ -62,6 +71,12 @@ private:\n> > bool coarseCompleted_;\n> > /* If the fine scan completes, it is set to true. */\n> > bool fineCompleted_;\n> > + /* Max focus change ratio to determine */\n> > + double maxChange_;\n> > + /* Relative lens position in percentage. */\n> > + double lensPosition_;\n> > + /* Af operation mode. */\n> > + uint32_t afMode_;\n> > };\n> >\n> > } /* namespace ipa::ipu3::algorithms */\n> > --\n> > 2.37.3\n> >\n>\n\n\n--\nBR,\nKate","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 13C31C0DA4\n\tfor ;\n\tThu, 29 Sep 2022 06:46:56 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 582246237D;\n\tThu, 29 Sep 2022 08:46:55 +0200 (CEST)","from us-smtp-delivery-124.mimecast.com\n\t(us-smtp-delivery-124.mimecast.com [170.10.129.124])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E238A61F77\n\tfor ;\n\tThu, 29 Sep 2022 08:46:52 +0200 (CEST)","from mail-yb1-f197.google.com (mail-yb1-f197.google.com\n\t[209.85.219.197]) by relay.mimecast.com with ESMTP with STARTTLS\n\t(version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id\n\tus-mta-589-E-KRR5k-NVCQAnuRdkdclg-1; Thu, 29 Sep 2022 02:46:50 -0400","by mail-yb1-f197.google.com with SMTP id\n\tt6-20020a25b706000000b006b38040b6f7so365636ybj.6\n\tfor ;\n\tWed, 28 Sep 2022 23:46:50 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1664434015;\n\tbh=kk0JQWbtLQuFUTY98nVMtwcVcxQurTFEEzTnvynZi6g=;\n\th=References:In-Reply-To:Date:To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=fR5op6j6H9OoNeQgAtFyDllQHwvUtCmeRAcBZeuESMe8Z9J+pBSfKVykvqAm5qJAY\n\taYqC9PRB4LtoueVmx09mMuoKv+gSUK7clYwSfmKSaH6+hz0auUkiW0IGbXee9KlvCs\n\tedvI1TL7gC1+L2HMTPuAuj6vt1gE4CyCNFLyzhUyicwdD11A4+GJYL16Mx4FT/PEZB\n\tMRnnWe7RGsye6XiRHLqB3b+6TpeLwL8bFg0sO1vqkelCDNQ3oGNKbpP7xI17rNzwMU\n\tdwiNynA0+MFYy7bxDV8pXPPZhXJgd5nF4E7cWQpSS7uJCnwI7XksybHxFPZnNZ1DSN\n\tX0oiAkoxIZavQ==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n\ts=mimecast20190719; t=1664434011;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tin-reply-to:in-reply-to:references:references;\n\tbh=MhkWJ75aylfprwu0hwbgARQUogWYygrTJxvCb5BrgNU=;\n\tb=QjMSv1Sd0D4AhdQ6rXvkwjGtL5RKm5hs/yU46HC16nZmUIz/CLKtYQV/RJGj88lWEFS3Un\n\tcm+xIRihPDorPAqWthppdaXkFclVQS8+TzSo7jK8ziBwvz/loTvDpNE9oda72CXDFuYwTZ\n\tJDeMMiijPpirAeQ/Y4HJv7BxQk8Bb1Y="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=redhat.com\n\theader.i=@redhat.com header.b=\"QjMSv1Sd\"; \n\tdkim-atps=neutral","X-MC-Unique":"E-KRR5k-NVCQAnuRdkdclg-1","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:x-gm-message-state:from:to:cc:subject:date;\n\tbh=MhkWJ75aylfprwu0hwbgARQUogWYygrTJxvCb5BrgNU=;\n\tb=cGwhnzQmVXotGvb5YKGy6SmwKTCwIUwokyP8jt25H9oje09WgkLgmbD/CP1wZNbnqB\n\tCjhBZtLgBr2EqE84Z/kYr+FMyoRiu/kKJpsmM4RJfiinwkgdQwwOtscv4+TCGl/JclOw\n\tA2EXng68h7BvFTTG3Mso67F3fqUhn30JJqc6QW/4LzmxVz+x472f9Ymnh+np/veSITaB\n\tJr2ckpJYaINhPCSlbUEfgZHmdXjJww+X7kuMw1k0zCydixMKleWf2I7sBeRnwPX7WH8c\n\tn21Vx0tY+i36TqyKHnyIrGpWOxLGEG54al03+2BJCDN0qyLg74fXZotD9ikIgwtuVGnl\n\tkSjw==","X-Gm-Message-State":"ACrzQf0CigwfddaIWBuGlRP8Z1t9qIqDzZDIsqWSITzLvRkavyxWv5UG\n\tLTSBXdhja5GlIOS/EJAuZ239lWrN4637H6ccwskZ74Qh2Qd2brwLuwTlvBDboOu/ni1MLilt7fh\n\tfT3vcDlsRJ4UkGnMBGvXNj/IEbZ5rtj9b+oO2Kq3TXd/FOhn66A==","X-Received":["by 2002:a0d:cb8f:0:b0:34d:22db:14ef with SMTP id\n\tn137-20020a0dcb8f000000b0034d22db14efmr1686141ywd.470.1664434009850; \n\tWed, 28 Sep 2022 23:46:49 -0700 (PDT)","by 2002:a0d:cb8f:0:b0:34d:22db:14ef with SMTP id\n\tn137-20020a0dcb8f000000b0034d22db14efmr1686100ywd.470.1664434008978;\n\tWed, 28 Sep 2022 23:46:48 -0700 (PDT)"],"X-Google-Smtp-Source":"AMsMyM7r88XL1WdaduW6rfEALljGCo4sfJmeOSPChqBmuB9sd+UBjryqFoFo+E7GyjFLDHgtQjo4X1F0PctApahPtjE=","MIME-Version":"1.0","References":"<20220916103713.21132-1-hpa@redhat.com>\n\t<20220916103713.21132-4-hpa@redhat.com>\n\t<20220928070656.l5c5wjxcvctsskgw@uno.localdomain>","In-Reply-To":"<20220928070656.l5c5wjxcvctsskgw@uno.localdomain>","Date":"Thu, 29 Sep 2022 14:46:37 +0800","Message-ID":"","To":"Jacopo Mondi ","X-Mimecast-Spam-Score":"0","X-Mimecast-Originator":"redhat.com","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [PATCH 3/3] ipa: ipu3: af: AfMode and\n\tLensPosition control implementation","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Kate Hsuan via libcamera-devel ","Reply-To":"Kate Hsuan ","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]