From patchwork Sat Oct 31 18:40:33 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28726 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EA97BC3261 for ; Sat, 31 Oct 2026 18:41:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9DC9E69140; Sat, 31 Oct 2026 19:41:00 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="DZqUjkAd"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 33E6A69130 for ; Sat, 31 Oct 2026 19:40:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1793472057; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NiZXTuTzKUDpnANpya0GOmUDA5/AX1+cJOMkf0+Rtmw=; b=DZqUjkAdb7n0J3IU73o2TWmeVFfu5akmtXJ6/OtrYz2VN77t/O0eI0a/zAG/y1P5OSUt/W LlOeTwzIVRW86ecLSJAQe9nQ34neww5GEno1ro3XDuWcfTmzDQZ3yCCXwddOYK2COQQ+bn cQmoV8cqOjyAbrsjn8xBLPnK4GQZZPc= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-548--tfMB4LXPuCQ8AoWORpSpA-1; Sat, 31 Oct 2026 14:40:53 -0400 X-MC-Unique: -tfMB4LXPuCQ8AoWORpSpA-1 X-Mimecast-MFC-AGG-ID: -tfMB4LXPuCQ8AoWORpSpA_1793472052 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 38C981955EA7; Sat, 31 Oct 2026 18:40:52 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7F4ED1800480; Sat, 31 Oct 2026 18:40:50 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader Subject: [PATCH v2 1/8] libcamera: software_isp: Let a single point decide about computing stats Date: Sat, 31 Oct 2026 19:40:33 +0100 Message-ID: <20261031184040.20139-2-mzamazal@redhat.com> In-Reply-To: <20261031184040.20139-1-mzamazal@redhat.com> References: <20261031184040.20139-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: -cHel530xVXurb6otUECKCe6T0gsf9fIpYh0ApvtE2A_1793472052 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Currently, the decision whether to compute stats for a given frame is implemented in SwStatsCpu in several places and duplicated in debayer_egl.cpp. Let's decide it solely in SwStatsCpu. This will also help if a more flexible or complex update frequency is introduced, e.g. to support stats for autofocus. Signed-off-by: Milan Zamazal --- .../internal/software_isp/swstats_cpu.h | 21 ++++++++++--------- src/libcamera/software_isp/debayer_egl.cpp | 6 +++--- src/libcamera/software_isp/swstats_cpu.cpp | 16 +++++++++++--- 3 files changed, 27 insertions(+), 16 deletions(-) diff --git a/include/libcamera/internal/software_isp/swstats_cpu.h b/include/libcamera/internal/software_isp/swstats_cpu.h index 551870921..615049084 100644 --- a/include/libcamera/internal/software_isp/swstats_cpu.h +++ b/include/libcamera/internal/software_isp/swstats_cpu.h @@ -38,14 +38,6 @@ public: SwStatsCpu(const CameraManager &cm); ~SwStatsCpu() = default; - /* - * The combination of pipeline + sensor delays means that - * exposure changes can take up to 3 frames to get applied, - * Run stats once every 4 frames to ensure any previous - * exposure changes have been applied. - */ - static constexpr uint32_t kStatPerNumFrames = 4; - bool isValid() const { return sharedStats_.fd().isValid(); } const SharedFD &getStatsFD() { return sharedStats_.fd(); } @@ -54,13 +46,14 @@ public: int configure(const StreamConfiguration &inputCfg, unsigned int statsBufferCount = 1); void setWindow(const Rectangle &window); + bool updating(uint32_t frame); void startFrame(uint32_t frame); void finishFrame(uint32_t frame, uint32_t bufferId); void processFrame(uint32_t frame, uint32_t bufferId, MappedFrameBuffer &input); void processLine0(uint32_t frame, unsigned int y, const uint8_t *src[], unsigned int statsBufferIndex = 0) { - if (frame % kStatPerNumFrames) + if (!updating(frame)) return; if ((y & ySkipMask_) || y < static_cast(window_.y) || @@ -72,7 +65,7 @@ public: void processLine2(uint32_t frame, unsigned int y, const uint8_t *src[], unsigned int statsBufferIndex = 0) { - if (frame % kStatPerNumFrames) + if (!updating(frame)) return; if ((y & ySkipMask_) || y < static_cast(window_.y) || @@ -88,6 +81,14 @@ private: using statsProcessFn = void (SwStatsCpu::*)(const uint8_t *src[], SwIspStats &stats); using processFrameFn = void (SwStatsCpu::*)(MappedFrameBuffer &in); + /* + * The combination of pipeline + sensor delays means that + * exposure changes can take up to 3 frames to get applied, + * Run stats once every 4 frames to ensure any previous + * exposure changes have been applied. + */ + static constexpr uint32_t kStatPerNumFrames = 4; + int setupStandardBayerOrder(BayerFormat::Order order); /* Bayer 8 bpp unpacked */ void statsBGGR8Line0(const uint8_t *src[], SwIspStats &stats); diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index 33933b11d..af5eaab93 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -646,9 +646,7 @@ void DebayerEGL::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output metadata.planes()[0].bytesused = output->planes()[0].length; /* Calculate stats for the whole frame */ - if (frame % SwStatsCpu::kStatPerNumFrames) { - stats_->finishFrame(frame, 0); - } else { + if (stats_->updating(frame)) { if (!inMapped) { /* * The buffer was directly imported into EGL and thus @@ -660,6 +658,8 @@ void DebayerEGL::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output inMapped.emplace(input, MappedFrameBuffer::MapFlag::Read); } stats_->processFrame(frame, 0, inMapped.value()); + } else { + stats_->finishFrame(frame, 0); } inDmaSyncer.reset(); diff --git a/src/libcamera/software_isp/swstats_cpu.cpp b/src/libcamera/software_isp/swstats_cpu.cpp index 74633815e..0aa5597a9 100644 --- a/src/libcamera/software_isp/swstats_cpu.cpp +++ b/src/libcamera/software_isp/swstats_cpu.cpp @@ -398,7 +398,7 @@ void SwStatsCpu::statsGBRG12PLine0(const uint8_t *src[], SwIspStats &stats) */ void SwStatsCpu::startFrame(uint32_t frame) { - if (frame % kStatPerNumFrames) + if (!updating(frame)) return; if (window_.width == 0) @@ -420,7 +420,7 @@ void SwStatsCpu::startFrame(uint32_t frame) */ void SwStatsCpu::finishFrame(uint32_t frame, uint32_t bufferId) { - bool valid = frame % kStatPerNumFrames == 0; + bool valid = updating(frame); if (valid) { sharedStats_->sum_ = RGB({ 0, 0, 0 }); @@ -622,7 +622,7 @@ void SwStatsCpu::processBayerFrame2(MappedFrameBuffer &in) */ void SwStatsCpu::processFrame(uint32_t frame, uint32_t bufferId, MappedFrameBuffer &input) { - if (frame % kStatPerNumFrames) { + if (!updating(frame)) { finishFrame(frame, bufferId); return; } @@ -634,4 +634,14 @@ void SwStatsCpu::processFrame(uint32_t frame, uint32_t bufferId, MappedFrameBuff bench_.finishFrame(); } +/** + * \brief Determine whether stats update is required for the given frame + * \param[in] frame The frame number + * \return Whether the stats are updated for the given frame + */ +bool SwStatsCpu::updating(uint32_t frame) +{ + return frame % kStatPerNumFrames == 0; +} + } /* namespace libcamera */ From patchwork Sat Oct 31 18:40:34 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28728 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 42830C3261 for ; Sat, 31 Oct 2026 18:41:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AD37F69149; Sat, 31 Oct 2026 19:41:04 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="jCtL5+2M"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5705069132 for ; Sat, 31 Oct 2026 19:41:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1793472059; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jPWwtr5v60XlMlufzFYuhv6IR/Dpe5IErQFFlfNV1X8=; b=jCtL5+2MZJTTuLXJymPLytQqf5SlDwHrvu1D5B81wvAI63rhH0Yf82DJONhZrXs7dDufUR 7+RAvV5CdEjXoXaRXJd02mf54mghSYTSmlGKhsMbAb8Um1KjpMOPrfgfeAmfxqajz7Po8V YW6Bd5f88IGYcjxjEedFC/FLwdD0nTI= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-556-siVCBQTmN5GHnViJ0_eGtQ-1; Sat, 31 Oct 2026 14:40:56 -0400 X-MC-Unique: siVCBQTmN5GHnViJ0_eGtQ-1 X-Mimecast-MFC-AGG-ID: siVCBQTmN5GHnViJ0_eGtQ_1793472055 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E498E1955DE9; Sat, 31 Oct 2026 18:40:54 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 97E921800480; Sat, 31 Oct 2026 18:40:52 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader Subject: [PATCH v2 2/8] ipa: ipu3: Move non-specific parts of Af to libipa Date: Sat, 31 Oct 2026 19:40:34 +0100 Message-ID: <20261031184040.20139-3-mzamazal@redhat.com> In-Reply-To: <20261031184040.20139-1-mzamazal@redhat.com> References: <20261031184040.20139-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: -mIxIV97mNngZ7WC9ngtwg0AsJ2ZKTK1C8wYKinvct0_1793472055 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Some parts of the ipu3 autofocus algorithm could be reused by other pipelines, most notably softisp. Let's move the parts of the algorithm that are not specific to the IPU3 hardware to libipa and use them from there in ipu3. Signed-off-by: Milan Zamazal --- src/ipa/ipu3/algorithms/af.cpp | 219 +----------------------- src/ipa/ipu3/algorithms/af.h | 27 +-- src/ipa/ipu3/ipa_context.cpp | 17 +- src/ipa/ipu3/ipa_context.h | 8 +- src/ipa/libipa/af.cpp | 299 +++++++++++++++++++++++++++++++++ src/ipa/libipa/af.h | 69 ++++++++ src/ipa/libipa/meson.build | 2 + 7 files changed, 382 insertions(+), 259 deletions(-) create mode 100644 src/ipa/libipa/af.cpp create mode 100644 src/ipa/libipa/af.h diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index e796b06a2..bb412e165 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -59,25 +59,6 @@ namespace ipa::ipu3::algorithms { LOG_DEFINE_CATEGORY(IPU3Af) -/** - * Maximum focus steps of the VCM control - * \todo should be obtained from the VCM driver - */ -static constexpr uint32_t kMaxFocusSteps = 1023; - -/* Minimum focus step for searching appropriate focus */ -static constexpr uint32_t kCoarseSearchStep = 30; -static constexpr uint32_t kFineSearchStep = 1; - -/* Max ratio of variance change, 0.0 < kMaxChange < 1.0 */ -static constexpr double kMaxChange = 0.5; - -/* The numbers of frame to be ignored, before performing focus scan. */ -static constexpr uint32_t kIgnoreFrame = 10; - -/* Fine scan range 0 < kFineRange < 1 */ -static constexpr double kFineRange = 0.05; - /* Settings for IPU3 AF filter */ static struct ipu3_uapi_af_filter_config afFilterConfigDefault = { .y1_coeff_0 = { 0, 1, 3, 7 }, @@ -105,8 +86,6 @@ static struct ipu3_uapi_af_filter_config afFilterConfigDefault = { * image. */ Af::Af() - : focus_(0), bestFocus_(0), currentVariance_(0.0), previousVariance_(0.0), - coarseCompleted_(false), fineCompleted_(false) { } @@ -162,18 +141,7 @@ int Af::configure(IPAContext &context, const IPAConfigInfo &configInfo) grid.y_start = utils::alignDown(start.y, 2); grid.y_start |= IPU3_UAPI_GRID_Y_START_EN; - /* Initial max focus step */ - maxStep_ = kMaxFocusSteps; - - /* Initial frame ignore counter */ - afIgnoreFrameReset(); - - /* Initial focus value */ - context.activeState.af.focus = 0; - /* Maximum variance of the AF statistics */ - context.activeState.af.maxVariance = 0; - /* The stable AF value flag. if it is true, the AF should be in a stable state. */ - context.activeState.af.stable = false; + afAlgo_.configure(context.activeState.af); return 0; } @@ -194,151 +162,6 @@ void Af::prepare(IPAContext &context, params->use.acc_af = 1; } -/** - * \brief AF coarse scan - * \param[in] context The shared IPA context - * - * Find a near focused image using a coarse step. The step is determined by - * kCoarseSearchStep. - */ -void Af::afCoarseScan(IPAContext &context) -{ - if (coarseCompleted_) - return; - - if (afNeedIgnoreFrame()) - return; - - if (afScan(context, kCoarseSearchStep)) { - coarseCompleted_ = true; - context.activeState.af.maxVariance = 0; - focus_ = context.activeState.af.focus - - (context.activeState.af.focus * kFineRange); - context.activeState.af.focus = focus_; - previousVariance_ = 0; - maxStep_ = std::clamp(focus_ + static_cast((focus_ * kFineRange)), - 0U, kMaxFocusSteps); - } -} - -/** - * \brief AF fine scan - * \param[in] context The shared IPA context - * - * Find an optimum lens position with moving 1 step for each search. - */ -void Af::afFineScan(IPAContext &context) -{ - if (!coarseCompleted_) - return; - - if (afNeedIgnoreFrame()) - return; - - if (afScan(context, kFineSearchStep)) { - context.activeState.af.stable = true; - fineCompleted_ = true; - } -} - -/** - * \brief AF reset - * \param[in] context The shared IPA context - * - * Reset all the parameters to start over the AF process. - */ -void Af::afReset(IPAContext &context) -{ - if (afNeedIgnoreFrame()) - return; - - context.activeState.af.maxVariance = 0; - context.activeState.af.focus = 0; - focus_ = 0; - context.activeState.af.stable = false; - ignoreCounter_ = kIgnoreFrame; - previousVariance_ = 0.0; - coarseCompleted_ = false; - fineCompleted_ = false; - maxStep_ = kMaxFocusSteps; -} - -/** - * \brief AF variance comparison - * \param[in] context The IPA context - * \param[in] min_step The VCM movement step - * - * We always pick the largest variance to replace the previous one. The image - * with a larger variance also indicates it is a clearer image than previous - * one. If we find a negative derivative, we return immediately. - * - * \return True, if it finds a AF value. - */ -bool Af::afScan(IPAContext &context, int min_step) -{ - if (focus_ > maxStep_) { - /* If reach the max step, move lens to the position. */ - context.activeState.af.focus = bestFocus_; - return true; - } else { - /* - * Find the maximum of the variance by estimating its - * derivative. If the direction changes, it means we have - * passed a maximum one step before. - */ - if ((currentVariance_ - context.activeState.af.maxVariance) >= - -(context.activeState.af.maxVariance * 0.1)) { - /* - * Positive and zero derivative: - * The variance is still increasing. The focus could be - * increased for the next comparison. Also, the max variance - * and previous focus value are updated. - */ - bestFocus_ = focus_; - focus_ += min_step; - context.activeState.af.focus = focus_; - context.activeState.af.maxVariance = currentVariance_; - } else { - /* - * Negative derivative: - * The variance starts to decrease which means the maximum - * variance is found. Set focus step to previous good one - * then return immediately. - */ - context.activeState.af.focus = bestFocus_; - return true; - } - } - - previousVariance_ = currentVariance_; - LOG(IPU3Af, Debug) << " Previous step is " - << bestFocus_ - << " Current step is " - << focus_; - return false; -} - -/** - * \brief Determine the frame to be ignored - * \return Return True if the frame should be ignored, false otherwise - */ -bool Af::afNeedIgnoreFrame() -{ - if (ignoreCounter_ == 0) - return false; - else - ignoreCounter_--; - return true; -} - -/** - * \brief Reset frame ignore counter - */ -void Af::afIgnoreFrameReset() -{ - ignoreCounter_ = kIgnoreFrame; -} - /** * \brief Estimate variance * \param[in] y_items The AF filter data set from the IPU3 statistics buffer @@ -373,33 +196,6 @@ double Af::afEstimateVariance(std::span y_items, bool isY1 return var_sum / y_items.size(); } -/** - * \brief Determine out-of-focus situation - * \param[in] context The IPA context - * - * Out-of-focus means that the variance change rate for a focused and a new - * variance is greater than a threshold. - * - * \return True if the variance threshold is crossed indicating lost focus, - * false otherwise - */ -bool Af::afIsOutOfFocus(IPAContext &context) -{ - const uint32_t diff_var = std::abs(currentVariance_ - - context.activeState.af.maxVariance); - const double var_ratio = diff_var / context.activeState.af.maxVariance; - - LOG(IPU3Af, Debug) << "Variance change rate: " - << var_ratio - << " Current VCM step: " - << context.activeState.af.focus; - - if (var_ratio > kMaxChange) - return true; - else - return false; -} - /** * \brief Determine the max contrast image and lens position * \param[in] context The IPA context @@ -438,17 +234,8 @@ void Af::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, * For coarse: y1 are used. * For fine: y2 results are used. */ - currentVariance_ = afEstimateVariance(y_items, !coarseCompleted_); - - if (!context.activeState.af.stable) { - afCoarseScan(context); - afFineScan(context); - } else { - if (afIsOutOfFocus(context)) - afReset(context); - else - afIgnoreFrameReset(); - } + double variance = afEstimateVariance(y_items, !afAlgo_.coarseCompleted()); + afAlgo_.process(context.activeState.af, variance); } REGISTER_IPA_ALGORITHM(Af, "Af") diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index 320b67496..fcf1c9cf6 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -13,6 +13,8 @@ #include +#include "libipa/af.h" + #include "algorithm.h" namespace libcamera { @@ -40,32 +42,9 @@ public: ControlList &metadata) override; private: - void afCoarseScan(IPAContext &context); - void afFineScan(IPAContext &context); - bool afScan(IPAContext &context, int min_step); - void afReset(IPAContext &context); - bool afNeedIgnoreFrame(); - void afIgnoreFrameReset(); double afEstimateVariance(std::span y_items, bool isY1); - bool afIsOutOfFocus(IPAContext &context); - - /* VCM step configuration. It is the current setting of the VCM step. */ - uint32_t focus_; - /* The best VCM step. It is a local optimum VCM step during scanning. */ - uint32_t bestFocus_; - /* Current AF statistic variance. */ - double currentVariance_; - /* The frames are ignore before starting measuring. */ - uint32_t ignoreCounter_; - /* It is used to determine the derivative during scanning */ - double previousVariance_; - /* The designated maximum range of focus scanning. */ - uint32_t maxStep_; - /* If the coarse scan completes, it is set to true. */ - bool coarseCompleted_; - /* If the fine scan completes, it is set to true. */ - bool fineCompleted_; + AfAlgorithm afAlgo_; }; } /* namespace ipa::ipu3::algorithms */ diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 6632c6f45..9f628cb45 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -88,22 +88,13 @@ namespace libcamera::ipa::ipu3 { */ /** - * \var IPAActiveState::af - * \brief Context for the Automatic Focus algorithm - * - * \var IPAActiveState::af.focus - * \brief Current position of the lens - * - * \var IPAActiveState::af.maxVariance - * \brief The maximum variance of the current image - * - * \var IPAActiveState::af.stable - * \brief It is set to true, if the best focus is found + * \var IPASessionConfiguration::agc + * \brief AGC parameters configuration of the IPA */ /** - * \var IPASessionConfiguration::agc - * \brief AGC parameters configuration of the IPA + * \var IPAActiveState::af + * \brief Autofocus state */ /** diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index fde628c22..ca3a9fc9d 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -24,6 +24,7 @@ #include #include #include +#include "libipa/af.h" namespace libcamera { @@ -44,12 +45,7 @@ struct IPASessionConfiguration { }; struct IPAActiveState { - struct { - uint32_t focus; - double maxVariance; - bool stable; - } af; - + ipa::af::ActiveState af; agc::ActiveState agc; ipa::awb::ActiveState awb; ipa::ccm::ActiveState ccm; diff --git a/src/ipa/libipa/af.cpp b/src/ipa/libipa/af.cpp new file mode 100644 index 000000000..492d65d1e --- /dev/null +++ b/src/ipa/libipa/af.cpp @@ -0,0 +1,299 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2026 Red Hat + * + * Autofocus algorithm + */ + +#include "af.h" + +namespace libcamera { + +namespace ipa { + +LOG_DEFINE_CATEGORY(Af) + +namespace af { + +/** + * \struct ActiveState + * \brief Autofocus algorithm state + * + * \var ActiveState::focus + * \brief The focus value for the V4L2 control + * + * \var ActiveState::maxVariance + * \brief Maximum variance of the AF statistics + * + * \var ActiveState::stable + * \brief Flag determining whether focus is in a stable state + */ + +} /* namespace af */ + +/** + * \class AfAlgorithm + * \brief Implementation of a common autofocus algorithm + * + * The algorithm is provided with a single variance value in + * \a AfAlgorithm::process and adjusts \a ActiveState::focus accordingly. + */ + +/** + * Maximum focus steps of the VCM control + * \todo should be obtained from the VCM driver + */ +static constexpr uint32_t kMaxFocusSteps = 1023; + +/* Minimum focus step for searching appropriate focus */ +static constexpr uint32_t kCoarseSearchStep = 30; +static constexpr uint32_t kFineSearchStep = 1; + +/* Max ratio of variance change, 0.0 < kMaxChange < 1.0 */ +static constexpr double kMaxChange = 0.5; + +/* The numbers of frame to be ignored, before performing focus scan. */ +static constexpr uint32_t kIgnoreFrame = 10; + +/* Fine scan range 0 < kFineRange < 1 */ +static constexpr double kFineRange = 0.05; + +AfAlgorithm::AfAlgorithm() + : focus_(0), bestFocus_(0), currentVariance_(0.0), previousVariance_(0.0), + coarseCompleted_(false), fineCompleted_(false) +{ +} + +/** + * \fn AfAlgorithm::coarseCompleted() + * \brief Return whether the coarse autofocus scan is completed + */ + +/** + * \brief AF coarse scan + * \param[in] state The autofocus state + * + * Find a near focused image using a coarse step. The step is determined by + * kCoarseSearchStep. + */ +void AfAlgorithm::afCoarseScan(af::ActiveState &state) +{ + if (coarseCompleted_) + return; + + if (afNeedIgnoreFrame()) + return; + + if (afScan(state, kCoarseSearchStep)) { + coarseCompleted_ = true; + state.maxVariance = 0; + focus_ = state.focus - + (state.focus * kFineRange); + state.focus = focus_; + previousVariance_ = 0; + maxStep_ = std::clamp(focus_ + static_cast((focus_ * kFineRange)), + 0U, kMaxFocusSteps); + } +} + +/** + * \brief AF fine scan + * \param[in] state The autofocus state + * + * Find an optimum lens position with moving 1 step for each search. + */ +void AfAlgorithm::afFineScan(af::ActiveState &state) +{ + if (!coarseCompleted_) + return; + + if (afNeedIgnoreFrame()) + return; + + if (afScan(state, kFineSearchStep)) { + state.stable = true; + fineCompleted_ = true; + } +} + +/** + * \brief AF reset + * \param[in] state The autofocus state + * + * Reset all the parameters to start over the AF process. + */ +void AfAlgorithm::afReset(af::ActiveState &state) +{ + if (afNeedIgnoreFrame()) + return; + + state.maxVariance = 0; + state.focus = 0; + focus_ = 0; + state.stable = false; + ignoreCounter_ = kIgnoreFrame; + previousVariance_ = 0.0; + coarseCompleted_ = false; + fineCompleted_ = false; + maxStep_ = kMaxFocusSteps; +} + +/** + * \brief AF variance comparison + * \param[in] state The autofocus state + * \param[in] min_step The VCM movement step + * + * We always pick the largest variance to replace the previous one. The image + * with a larger variance also indicates it is a clearer image than previous + * one. If we find a negative derivative, we return immediately. + * + * \return True, if it finds a AF value + */ +bool AfAlgorithm::afScan(af::ActiveState &state, int min_step) +{ + if (focus_ > maxStep_) { + /* If reach the max step, move lens to the position. */ + state.focus = bestFocus_; + return true; + } else { + /* + * Find the maximum of the variance by estimating its + * derivative. If the direction changes, it means we have + * passed a maximum one step before. + */ + if ((currentVariance_ - state.maxVariance) >= + -(state.maxVariance * 0.1)) { + /* + * Positive and zero derivative: + * The variance is still increasing. The focus could be + * increased for the next comparison. Also, the max variance + * and previous focus value are updated. + */ + bestFocus_ = focus_; + focus_ += min_step; + state.focus = focus_; + state.maxVariance = currentVariance_; + } else { + /* + * Negative derivative: + * The variance starts to decrease which means the maximum + * variance is found. Set focus step to previous good one + * then return immediately. + */ + state.focus = bestFocus_; + return true; + } + } + + previousVariance_ = currentVariance_; + LOG(Af, Debug) << " Previous step is " + << bestFocus_ + << " Current step is " + << focus_; + return false; +} + +/** + * \brief Determine the frame to be ignored + * \return Return True if the frame should be ignored, false otherwise + */ +bool AfAlgorithm::afNeedIgnoreFrame() +{ + if (ignoreCounter_ == 0) + return false; + else + ignoreCounter_--; + return true; +} + +/** + * \brief Reset frame ignore counter + */ +void AfAlgorithm::afIgnoreFrameReset() +{ + ignoreCounter_ = kIgnoreFrame; +} + +/** + * \brief Determine out-of-focus situation + * \param[in] state The autofocus state + * + * Out-of-focus means that the variance change rate for a focused and a new + * variance is greater than a threshold. + * + * \return True if the variance threshold is crossed indicating lost focus, + * false otherwise + */ +bool AfAlgorithm::afIsOutOfFocus(af::ActiveState &state) +{ + const uint32_t diff_var = std::abs(currentVariance_ - + state.maxVariance); + const double var_ratio = diff_var / state.maxVariance; + + LOG(Af, Debug) << "Variance change rate: " + << var_ratio + << " Current VCM step: " + << state.focus; + + if (var_ratio > kMaxChange) + return true; + else + return false; +} + +/** + * \brief Configure the algorithm and \a state + * \param[in] state The autofocus state + * + * The method just sets some initial variable values rather than performing real + * configuration but it is supposed to be called from IPA's `configure`. + */ +void AfAlgorithm::configure(af::ActiveState &state) +{ + /* Initial max focus step */ + maxStep_ = kMaxFocusSteps; + + /* Initial frame ignore counter */ + afIgnoreFrameReset(); + + /* Initial focus value */ + state.focus = 0; + /* Maximum variance of the AF statistics */ + state.maxVariance = 0; + /* The stable AF value flag. if it is true, the AF should be in a stable state. */ + state.stable = false; +} + +/** + * \brief Determine the max contrast image and lens position + * \param[in] state The autofocus state + * \param[in] variance The variance of autofocus statistics + * + * Ideally, a clear image also has a relatively higher contrast. So, every + * image for each focus step should be tested to find an optimal focus step. + * + * The Hill Climbing Algorithm[1] is used to find the maximum variance of the AF + * statistics. The focus step is increased then the variance of the AF + * statistics are estimated. If it finds the negative derivative we have just + * passed the peak, and we infer that the best focus is found. + * + * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing + */ +void AfAlgorithm::process(af::ActiveState &state, double variance) +{ + currentVariance_ = variance; + + if (!state.stable) { + afCoarseScan(state); + afFineScan(state); + } else { + if (afIsOutOfFocus(state)) + afReset(state); + else + afIgnoreFrameReset(); + } +} + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/af.h b/src/ipa/libipa/af.h new file mode 100644 index 000000000..c902b434c --- /dev/null +++ b/src/ipa/libipa/af.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2026 Red Hat + * + * Autofocus algorithm + */ + +#pragma once + +#include + +#include + +namespace libcamera { + +namespace ipa { + +LOG_DECLARE_CATEGORY(Af) + +namespace af { + +struct ActiveState { + uint32_t focus; + double maxVariance; + bool stable; +}; + +} /* namespace af */ + +class AfAlgorithm +{ +public: + AfAlgorithm(); + + void configure(af::ActiveState &state); + void process(af::ActiveState &state, double variance); + bool coarseCompleted() { return coarseCompleted_; } + +private: + void afCoarseScan(af::ActiveState &af); + void afFineScan(af::ActiveState &af); + bool afScan(af::ActiveState &af, int min_step); + void afReset(af::ActiveState &af); + bool afNeedIgnoreFrame(); + void afIgnoreFrameReset(); + + bool afIsOutOfFocus(af::ActiveState &af); + + /* VCM step configuration. It is the current setting of the VCM step. */ + uint32_t focus_; + /* The best VCM step. It is a local optimum VCM step during scanning. */ + uint32_t bestFocus_; + /* Current AF statistic variance. */ + double currentVariance_; + /* The frames are ignore before starting measuring. */ + uint32_t ignoreCounter_; + /* It is used to determine the derivative during scanning */ + double previousVariance_; + /* The designated maximum range of focus scanning. */ + uint32_t maxStep_; + /* If the coarse scan completes, it is set to true. */ + bool coarseCompleted_; + /* If the fine scan completes, it is set to true. */ + bool fineCompleted_; +}; + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index a38d5d528..b80c3b017 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 libipa_headers = files([ + 'af.h', 'agc.h', 'agc_mean_luminance.h', 'agc_msv.h', @@ -30,6 +31,7 @@ libipa_headers = files([ ]) libipa_sources = files([ + 'af.cpp', 'agc.cpp', 'agc_mean_luminance.cpp', 'agc_msv.cpp', From patchwork Sat Oct 31 18:40:35 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28727 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B1817C3261 for ; Sat, 31 Oct 2026 18:41:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 59CE869146; Sat, 31 Oct 2026 19:41:03 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="TI9i7tV8"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 43A2469133 for ; Sat, 31 Oct 2026 19:41:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1793472059; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gChfWa4EdlBSIqxRmKY+JuojOPsJ5d+HWnEOeTrfmLk=; b=TI9i7tV8c0amrY1ohLEey/H1clf0rlbJE7upZDWVWwgFx1RUeXEBD6TofmYy0Q3b7e6lZp FQBfTb8F3/SZ4eZg/WcdhZGI7kmzkY6V2XvXKS7M73wvmeB3zvgTmzd/WwpfqiagukN8TZ BXi2VSLhTIGRzoPXUHNVcJtXkjEMHsw= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-257-s-2ofiYOPm-QLSCHxdDqRA-1; Sat, 31 Oct 2026 14:40:58 -0400 X-MC-Unique: s-2ofiYOPm-QLSCHxdDqRA-1 X-Mimecast-MFC-AGG-ID: s-2ofiYOPm-QLSCHxdDqRA_1793472057 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E548819560A7; Sat, 31 Oct 2026 18:40:56 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 780AF1800480; Sat, 31 Oct 2026 18:40:55 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader Subject: [PATCH v2 3/8] ipa: libipa: Add af::prepareControls() Date: Sat, 31 Oct 2026 19:40:35 +0100 Message-ID: <20261031184040.20139-4-mzamazal@redhat.com> In-Reply-To: <20261031184040.20139-1-mzamazal@redhat.com> References: <20261031184040.20139-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: fgFyWsQ-ND006FYlP0Ij_PEAZgL_9fcFRhH_wjMrf10_1793472057 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Similarly to what other algorithms do, let's add a control initialisation function to the libipa autofocus algorithm and use it in ipu3 IPA. Signed-off-by: Milan Zamazal --- src/ipa/ipu3/ipu3.cpp | 3 +-- src/ipa/libipa/af.h | 7 +++++++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 30d52cfcf..599817061 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -555,8 +555,7 @@ void IPAIPU3::setControls(unsigned int frame) frameContext.agc.exposure, frameContext.agc.gain); ControlList lensCtrls(lensCtrls_); - lensCtrls.set(V4L2_CID_FOCUS_ABSOLUTE, - static_cast(context_.activeState.af.focus)); + af::prepareControls(lensCtrls, context_.activeState.af.focus); setSensorControls.emit(frame, ctrls, lensCtrls); } diff --git a/src/ipa/libipa/af.h b/src/ipa/libipa/af.h index c902b434c..de9f7e0ba 100644 --- a/src/ipa/libipa/af.h +++ b/src/ipa/libipa/af.h @@ -7,6 +7,8 @@ #pragma once +#include + #include #include @@ -25,6 +27,11 @@ struct ActiveState { bool stable; }; +inline void prepareControls(ControlList &controls, int32_t focus) +{ + controls.set(V4L2_CID_FOCUS_ABSOLUTE, focus); +} + } /* namespace af */ class AfAlgorithm From patchwork Sat Oct 31 18:40:36 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28729 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 492ECC3261 for ; Sat, 31 Oct 2026 18:41:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E7ADA69140; Sat, 31 Oct 2026 19:41:06 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="f9T4/cyB"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E183C6914A for ; Sat, 31 Oct 2026 19:41:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1793472063; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xDkCCH66h+ojerlkvVlVvfTl1Ph606P+Tjou2ihK1j8=; b=f9T4/cyBA6WXCQul++r/VE+Dg4dX6zmedWru+kOMFQGQnWjooE31JfgzKr0HmYMzzto6Ld VPoIL1EMKFONolTnn/xmLoxlfAjB0HmvA7T8WaRUDhj6Sl2QJb/Z2lv2lBxESBOWKFnKXm HRYs65xlbY1mx+vaV7UL3L5+2ick74c= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-327-jThSOE_CPoyLds4RqQZJdw-1; Sat, 31 Oct 2026 14:41:00 -0400 X-MC-Unique: jThSOE_CPoyLds4RqQZJdw-1 X-Mimecast-MFC-AGG-ID: jThSOE_CPoyLds4RqQZJdw_1793472059 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BD5CD1802154; Sat, 31 Oct 2026 18:40:58 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4F9661800480; Sat, 31 Oct 2026 18:40:57 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader Subject: [PATCH v2 4/8] libcamera: software_isp: Add variance to stats Date: Sat, 31 Oct 2026 19:40:36 +0100 Message-ID: <20261031184040.20139-5-mzamazal@redhat.com> In-Reply-To: <20261031184040.20139-1-mzamazal@redhat.com> References: <20261031184040.20139-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: uM1jNgzqFNuYG_E3CcZNGKQNVmnQHaatHZTo1u0SFdA_1793472059 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The libipa Af algorithm requests variance value as its input. Let's add the corresponding member to SwIspStats and initialise it. This commit is preparatory and doesn't implement computation of the variance nor introduces other changes to the stats like when to update them for autofocus. Signed-off-by: Milan Zamazal --- include/libcamera/internal/software_isp/swisp_stats.h | 8 +++++++- src/libcamera/software_isp/swstats_cpu.cpp | 3 +++ 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/software_isp/swisp_stats.h b/include/libcamera/internal/software_isp/swisp_stats.h index 65d826f6c..a0f6f4a79 100644 --- a/include/libcamera/internal/software_isp/swisp_stats.h +++ b/include/libcamera/internal/software_isp/swisp_stats.h @@ -17,7 +17,7 @@ namespace libcamera { /** * \brief Statistics for the Software ISP * - * The software ISP computes two types of statistics: + * The software ISP computes the following types of statistics: * * - The accumulator counts the number of non-saturated pixels and sums their * R, G and B channels individually. Pixels that have any saturated channel @@ -27,6 +27,8 @@ namespace libcamera { * - The histogram groups pixels in bins based on their luminance and counts * the number of pixels in each bin. Saturated pixels are included. * + * - The variance value for autofocus. + * * The value types are large enough to not overflow. Should they still overflow * for some reason, no check is performed and they wrap around. */ @@ -56,6 +58,10 @@ struct SwIspStats { * \brief A histogram of luminance values of all the sampled pixels */ Histogram yHistogram; + /** + * \brief A variance value for the autofocus algorithm + */ + double variance; }; } /* namespace libcamera */ diff --git a/src/libcamera/software_isp/swstats_cpu.cpp b/src/libcamera/software_isp/swstats_cpu.cpp index 0aa5597a9..ed1aa83e4 100644 --- a/src/libcamera/software_isp/swstats_cpu.cpp +++ b/src/libcamera/software_isp/swstats_cpu.cpp @@ -407,6 +407,7 @@ void SwStatsCpu::startFrame(uint32_t frame) for (auto &s : stats_) { s.sum_ = RGB({ 0, 0, 0 }); s.numPixels_ = 0; + s.variance = 0.0; s.yHistogram.fill(0); } } @@ -425,10 +426,12 @@ void SwStatsCpu::finishFrame(uint32_t frame, uint32_t bufferId) if (valid) { sharedStats_->sum_ = RGB({ 0, 0, 0 }); sharedStats_->numPixels_ = 0; + sharedStats_->variance = 0.0; sharedStats_->yHistogram.fill(0); for (const auto &s : stats_) { sharedStats_->sum_ += s.sum_; sharedStats_->numPixels_ += s.numPixels_; + sharedStats_->variance += s.variance; for (unsigned int j = 0; j < SwIspStats::kYHistogramSize; j++) sharedStats_->yHistogram[j] += s.yHistogram[j]; } From patchwork Sat Oct 31 18:40:37 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28730 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 19247C3261 for ; Sat, 31 Oct 2026 18:41:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AC0C769155; Sat, 31 Oct 2026 19:41:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="KI25zZ+L"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7074569136 for ; Sat, 31 Oct 2026 19:41:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1793472065; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HTtzdjunsdTrRXATGXlN1YEdOFc93A15QA0q1eEwaZQ=; b=KI25zZ+Lne0y7m/6QhF2TIuKdqof/l5l6YSOsg73EkDgeRaRwNhffLkOSBLLXT2ZIrsCU3 Hsh0SW9FD1Mx4JNj/MTCEKedkIBbKABlNkpryX/8dGAoLW6wE+osm8P0sdWk2JV8Nkxkah T0OaqwpT226va1CfDyuF+7vvjuK3Zlc= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-584-B90Un3ioMSa10MWQx0uk-g-1; Sat, 31 Oct 2026 14:41:01 -0400 X-MC-Unique: B90Un3ioMSa10MWQx0uk-g-1 X-Mimecast-MFC-AGG-ID: B90Un3ioMSa10MWQx0uk-g_1793472060 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9D9621955BCB; Sat, 31 Oct 2026 18:41:00 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 276771800480; Sat, 31 Oct 2026 18:40:58 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader Subject: [PATCH v2 5/8] ipa: softisp: Remove unused imports from ipa_context.h Date: Sat, 31 Oct 2026 19:40:37 +0100 Message-ID: <20261031184040.20139-6-mzamazal@redhat.com> In-Reply-To: <20261031184040.20139-1-mzamazal@redhat.com> References: <20261031184040.20139-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: AYa6XAq4YQkEqM3vT86kHNnGqPmMuvWta-xd3Ym3Hos_1793472060 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham Reviewed-by: Barnabás Pőcze --- src/ipa/softisp/ipa_context.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/ipa/softisp/ipa_context.h b/src/ipa/softisp/ipa_context.h index 1c5db17e3..9d8e9f4fb 100644 --- a/src/ipa/softisp/ipa_context.h +++ b/src/ipa/softisp/ipa_context.h @@ -7,7 +7,6 @@ #pragma once -#include #include #include #include @@ -15,7 +14,6 @@ #include #include "libcamera/internal/matrix.h" -#include "libcamera/internal/vector.h" #include #include From patchwork Sat Oct 31 18:40:38 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28731 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6CBB6C3302 for ; Sat, 31 Oct 2026 18:41:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D65DE69159; Sat, 31 Oct 2026 19:41:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="fyYEiG/6"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D850E69136 for ; Sat, 31 Oct 2026 19:41:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1793472067; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=j4pZkBBkzbCBJCp0Jb5XQoQmBbTKaK3mm6zI9E+V8aA=; b=fyYEiG/6lkxnvCOTGeG8tZ+fznIcFSoLNLYXbO8mViZ84hK/HB1GHPKAVG3DBpLd6nDe0I GBdGZWHj3dAYedOX5OUsxCyJfEDh9Ehwe4ElFtAkgibwJy5MHwQs3awlFN9KViwnulNplA Mxd2lXMRIu8bRXxfU543pe07UB1DYN4= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-596-yULyiYtXNT6DtAr-PXbiJw-1; Sat, 31 Oct 2026 14:41:03 -0400 X-MC-Unique: yULyiYtXNT6DtAr-PXbiJw-1 X-Mimecast-MFC-AGG-ID: yULyiYtXNT6DtAr-PXbiJw_1793472062 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 799A81802154; Sat, 31 Oct 2026 18:41:02 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 07FE918005BC; Sat, 31 Oct 2026 18:41:00 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader Subject: [PATCH v2 6/8] ipa: softisp: Add Af algorithm Date: Sat, 31 Oct 2026 19:40:38 +0100 Message-ID: <20261031184040.20139-7-mzamazal@redhat.com> In-Reply-To: <20261031184040.20139-1-mzamazal@redhat.com> References: <20261031184040.20139-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 64d9OegYdok9aQc_tP5f5V86ow2ibFJpIV2SSF5gMDM_1793472062 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Let's add Af algorithm, which is mostly a proxy to the libipa Af. The only special feature is setting afEnabled flag to prevent attempting lens movements when the algorithm is not enabled. The algorithm is not enabled yet because there are no autofocus stats computed. Signed-off-by: Milan Zamazal --- src/ipa/softisp/algorithms/af.cpp | 53 ++++++++++++++++++++++++++ src/ipa/softisp/algorithms/af.h | 39 +++++++++++++++++++ src/ipa/softisp/algorithms/meson.build | 1 + src/ipa/softisp/data/uncalibrated.yaml | 1 + src/ipa/softisp/ipa_context.h | 3 ++ 5 files changed, 97 insertions(+) create mode 100644 src/ipa/softisp/algorithms/af.cpp create mode 100644 src/ipa/softisp/algorithms/af.h diff --git a/src/ipa/softisp/algorithms/af.cpp b/src/ipa/softisp/algorithms/af.cpp new file mode 100644 index 000000000..557cca004 --- /dev/null +++ b/src/ipa/softisp/algorithms/af.cpp @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Red Hat Inc. + * + * Autofocus + */ + +#include "af.h" + +#include + +#include + +#include "libcamera/internal/software_isp/swisp_stats.h" + +namespace libcamera { + +namespace ipa::softisp::algorithms { + +LOG_DEFINE_CATEGORY(IPASoftIspAf) + +int Af::init(IPAContext &context, [[maybe_unused]] const ValueNode &tuningData) +{ + /* Informs that lens controls can get focus data */ + context.afEnabled = true; + + return 0; +} + +int Af::configure(IPAContext &context, + [[maybe_unused]] const IPAConfigInfo &configInfo) +{ + afAlgo_.configure(context.activeState.af); + + return 0; +} + +void Af::process(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + const SwIspStats *stats, + [[maybe_unused]] ControlList &metadata) +{ + const double variance = stats->variance; + LOG(IPASoftIspAf, Debug) << "Focus stats: " << variance; + afAlgo_.process(context.activeState.af, variance); +} + +REGISTER_IPA_ALGORITHM(Af, "Af") + +} /* namespace ipa::softisp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/softisp/algorithms/af.h b/src/ipa/softisp/algorithms/af.h new file mode 100644 index 000000000..881cb4475 --- /dev/null +++ b/src/ipa/softisp/algorithms/af.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Red Hat Inc. + * + * Autofocus + */ + +#pragma once + +#include "libipa/af.h" + +#include "algorithm.h" +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::softisp::algorithms { + +class Af : public Algorithm +{ +public: + Af() = default; + ~Af() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPAConfigInfo &configInfo) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const SwIspStats *stats, + ControlList &metadata) override; + +private: + AfAlgorithm afAlgo_; +}; + +} /* namespace ipa::softisp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/softisp/algorithms/meson.build b/src/ipa/softisp/algorithms/meson.build index d98f4b1f4..7d1077117 100644 --- a/src/ipa/softisp/algorithms/meson.build +++ b/src/ipa/softisp/algorithms/meson.build @@ -2,6 +2,7 @@ softisp_ipa_algorithms = files([ 'adjust.cpp', + 'af.cpp', 'awb.cpp', 'agc.cpp', 'blc.cpp', diff --git a/src/ipa/softisp/data/uncalibrated.yaml b/src/ipa/softisp/data/uncalibrated.yaml index fc90ca526..53d3792a2 100644 --- a/src/ipa/softisp/data/uncalibrated.yaml +++ b/src/ipa/softisp/data/uncalibrated.yaml @@ -4,6 +4,7 @@ version: 1 algorithms: - BlackLevel: + # - Af: - Awb: # Color correction matrices can be defined here. The CCM algorithm # has a significant performance impact, and should only be enabled diff --git a/src/ipa/softisp/ipa_context.h b/src/ipa/softisp/ipa_context.h index 9d8e9f4fb..39823261c 100644 --- a/src/ipa/softisp/ipa_context.h +++ b/src/ipa/softisp/ipa_context.h @@ -20,6 +20,7 @@ #include #include #include +#include "libipa/af.h" #include "libipa/lsc.h" #include "core_ipa_interface.h" @@ -39,6 +40,7 @@ struct IPAActiveState { ipa::awb::ActiveState awb; ipa::ccm::ActiveState ccm; ipa::agc::ActiveState agc; + ipa::af::ActiveState af; struct { uint8_t level; @@ -88,6 +90,7 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; bool ccmEnabled = false; + bool afEnabled = false; bool lscEnabled = false; ipa::lsc::ActiveState lsc; }; From patchwork Sat Oct 31 18:40:39 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28732 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E8070C3261 for ; Sat, 31 Oct 2026 18:41:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5D3F469157; Sat, 31 Oct 2026 19:41:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="ReWFu4Ue"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9E8856914F for ; Sat, 31 Oct 2026 19:41:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1793472067; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iVh8OxiP2DB2YgTJTwmaET0iiaDwQhUHqsegUm5kr98=; b=ReWFu4Ue094Mnu9HlyMXggkEZa20rG1g7n/ZojVRg/P17Hb1JuvmyPa7sGdOvVoPqi3fKi HyIN3DfMEvGfwejIJk0lOMBv0BY3KnddpyarbMcIPfZu8MwnHTAlQjVWovuewhp53awpTE Er4aaG/9aMJsJVN0t8wrSYL6Okk6DKI= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-85-6UpkkboPPR2yWzOOE8nwMQ-1; Sat, 31 Oct 2026 14:41:05 -0400 X-MC-Unique: 6UpkkboPPR2yWzOOE8nwMQ-1 X-Mimecast-MFC-AGG-ID: 6UpkkboPPR2yWzOOE8nwMQ_1793472064 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7375C1802157; Sat, 31 Oct 2026 18:41:04 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D973618005BA; Sat, 31 Oct 2026 18:41:02 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader Subject: [PATCH v2 7/8] libcamera: software_isp: Add autofocus controls Date: Sat, 31 Oct 2026 19:40:39 +0100 Message-ID: <20261031184040.20139-8-mzamazal@redhat.com> In-Reply-To: <20261031184040.20139-1-mzamazal@redhat.com> References: <20261031184040.20139-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: u7NRCsyXb4lFm6sw4gyiLNdLbiyu0cwrNpzxlYzktXM_1793472064 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to support focusing operations, lens controls are set up and passed around, similarly to the sensor controls. Currently, the only control used, and only if the sensor supports autofocus, is V4L2_CID_FOCUS_ABSOLUTE. Instead of serializing the whole lens ControList in setSensorControls, we could pass around just the value. But using the ControlList is more consistent with what we do with the sensor controls and what is done in ipu3 pipeline. Additionally, it's ready for possibly amending the controls in future; we might end up having to determine between V4L2_CID_FOCUS_ABSOLUTE or V4L2_CID_FOCUS_RELATIVE, or V4L2_CID_FOCUS_AUTO settings. Signed-off-by: Milan Zamazal --- .../libcamera/internal/software_isp/software_isp.h | 4 ++-- include/libcamera/ipa/softisp.mojom | 3 ++- src/ipa/softisp/ipa_context.h | 1 + src/ipa/softisp/softisp.cpp | 7 ++++++- src/libcamera/pipeline/simple/simple.cpp | 13 +++++++++++-- src/libcamera/software_isp/software_isp.cpp | 4 ++-- 6 files changed, 24 insertions(+), 8 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index b64910637..be90a229a 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -87,11 +87,11 @@ public: Signal outputBufferReady; Signal ispStatsReady; Signal metadataReady; - Signal setSensorControls; + Signal setSensorControls; private: void saveIspParams(uint32_t frame); - void setSensorCtrls(const ControlList &sensorControls); + void setSensorCtrls(const ControlList &sensorControls, const ControlList &lensControls); void statsReady(uint32_t frame, uint32_t bufferId); void inputReady(FrameBuffer *input); void outputReady(FrameBuffer *output); diff --git a/include/libcamera/ipa/softisp.mojom b/include/libcamera/ipa/softisp.mojom index e42742ed7..833f4440b 100644 --- a/include/libcamera/ipa/softisp.mojom +++ b/include/libcamera/ipa/softisp.mojom @@ -10,6 +10,7 @@ import "include/libcamera/ipa/core.mojom"; struct IPAConfigInfo { libcamera.ControlInfoMap sensorControls; + libcamera.ControlInfoMap lensControls; }; interface IPASoftIspInterface { @@ -34,7 +35,7 @@ interface IPASoftIspInterface { }; interface IPASoftIspEventInterface { - setSensorControls(libcamera.ControlList sensorControls); + setSensorControls(libcamera.ControlList sensorControls, libcamera.ControlList lensControls); paramsComputed(uint32 frame); metadataReady(uint32 frame, libcamera.ControlList metadata); }; diff --git a/src/ipa/softisp/ipa_context.h b/src/ipa/softisp/ipa_context.h index 39823261c..73b145cab 100644 --- a/src/ipa/softisp/ipa_context.h +++ b/src/ipa/softisp/ipa_context.h @@ -84,6 +84,7 @@ struct IPAContext { IPACameraSensorInfo sensorInfo; ControlInfoMap sensorControls; + ControlInfoMap lensControls; std::unique_ptr camHelper; IPASessionConfiguration configuration; IPAActiveState activeState; diff --git a/src/ipa/softisp/softisp.cpp b/src/ipa/softisp/softisp.cpp index 00249a300..e96a8d596 100644 --- a/src/ipa/softisp/softisp.cpp +++ b/src/ipa/softisp/softisp.cpp @@ -27,6 +27,7 @@ #include "libcamera/internal/yaml_parser.h" #include "algorithms/adjust.h" +#include "libipa/af.h" #include "libipa/agc.h" #include "libipa/camera_sensor_helper.h" @@ -192,6 +193,7 @@ int IPASoftIsp::init(const IPASettings &settings, int IPASoftIsp::configure(const IPAConfigInfo &configInfo, ControlInfoMap *ipaControls) { context_.sensorControls = configInfo.sensorControls; + context_.lensControls = configInfo.lensControls; /* Clear the IPA context before the streaming session. */ context_.configuration = {}; @@ -256,7 +258,10 @@ void IPASoftIsp::processStats(const uint32_t frame, ControlList ctrls(context_.sensorControls); agc::prepareControls(ctrls, context_.camHelper.get(), frameContext.agc.exposure, frameContext.agc.gain); - setSensorControls.emit(ctrls); + ControlList lensCtrls(context_.lensControls); + if (context_.afEnabled) + af::prepareControls(lensCtrls, context_.activeState.af.focus); + setSensorControls.emit(ctrls, lensCtrls); } std::string IPASoftIsp::logPrefix() const diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 35e066499..4eaccaf01 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -34,6 +34,7 @@ #include #include "libcamera/internal/camera.h" +#include "libcamera/internal/camera_lens.h" #include "libcamera/internal/camera_manager.h" #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/camera_sensor_properties.h" @@ -374,7 +375,7 @@ private: void ispStatsReady(uint32_t frame, uint32_t bufferId); void metadataReady(uint32_t frame, const ControlList &metadata); - void setSensorControls(const ControlList &sensorControls); + void setSensorControls(const ControlList &sensorControls, const ControlList &lensControls); }; class SimpleCameraConfiguration : public CameraConfiguration @@ -1046,7 +1047,7 @@ void SimpleCameraData::metadataReady(uint32_t frame, const ControlList &metadata tryCompleteRequest(info->request); } -void SimpleCameraData::setSensorControls(const ControlList &sensorControls) +void SimpleCameraData::setSensorControls(const ControlList &sensorControls, const ControlList &lensControls) { delayedCtrls_->push(sensorControls); /* @@ -1060,6 +1061,12 @@ void SimpleCameraData::setSensorControls(const ControlList &sensorControls) if (!frameStartEmitter_) { ControlList ctrls(sensorControls); sensor_->setControls(&ctrls); + + CameraLens *lens = sensor_->focusLens(); + if (lens && lensControls.contains(V4L2_CID_FOCUS_ABSOLUTE)) { + ControlValue const &focusValue = lensControls.get(V4L2_CID_FOCUS_ABSOLUTE); + lens->setFocusPosition(focusValue.get()); + } } } @@ -1652,6 +1659,8 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) } else { ipa::softisp::IPAConfigInfo configInfo; configInfo.sensorControls = data->sensor_->controls(); + if (data->sensor_->focusLens()) + configInfo.lensControls = data->sensor_->focusLens()->controls(); return data->swIsp_->configure(inputCfg, outputCfgs, configInfo, &data->controlInfo_); } } diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 627625b4f..53ae3d76f 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -443,9 +443,9 @@ void SoftwareIsp::saveIspParams([[maybe_unused]] uint32_t frame) debayerParams_ = *sharedParams_; } -void SoftwareIsp::setSensorCtrls(const ControlList &sensorControls) +void SoftwareIsp::setSensorCtrls(const ControlList &sensorControls, const ControlList &lensControls) { - setSensorControls.emit(sensorControls); + setSensorControls.emit(sensorControls, lensControls); } void SoftwareIsp::statsReady(uint32_t frame, uint32_t bufferId) From patchwork Sat Oct 31 18:40:40 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28733 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 06E22C3261 for ; Sat, 31 Oct 2026 18:41:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ACF2869153; Sat, 31 Oct 2026 19:41:39 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Mn2MyoOt"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3DC6C69133 for ; Sat, 31 Oct 2026 19:41:38 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1793472097; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J82exmQ69WvTMReomIWL2uEXYnRRDdml0mEacDiGZLY=; b=Mn2MyoOtgFzEGvZTGAmi6kOjMkoSE1TtnWoJVeax2l9Jb9HXsizOnrPYDgFG7Hepj6LVf4 MlQi/ZK2Gtf+vw9Hg70tDHg9bX7Mwv3nAa+fYE/EPdkQ4ykCcSP4FdUeSfyN1Cix/b5or/ +Qw7+7hoPHdRzwJaq8y+7A8kpTaMHe8= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-543-DJ-k9egCMRGR139RH7P-Mg-1; Sat, 31 Oct 2026 14:41:07 -0400 X-MC-Unique: DJ-k9egCMRGR139RH7P-Mg-1 X-Mimecast-MFC-AGG-ID: DJ-k9egCMRGR139RH7P-Mg_1793472066 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6301E1955BC1; Sat, 31 Oct 2026 18:41:06 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id EAC991800480; Sat, 31 Oct 2026 18:41:04 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader Subject: [PATCH v2 8/8] ipa: libipa: Use autofocus limits if available Date: Sat, 31 Oct 2026 19:40:40 +0100 Message-ID: <20261031184040.20139-9-mzamazal@redhat.com> In-Reply-To: <20261031184040.20139-1-mzamazal@redhat.com> References: <20261031184040.20139-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 1TpR03UiptzYiz60l4Jq0o4bb1GB-tKSBgdx9UHQtyk_1793472066 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Let's try to set the maximum and the initial focusing positions rather than using hard-wired values. Signed-off-by: Milan Zamazal --- src/ipa/ipu3/algorithms/af.cpp | 2 +- src/ipa/libipa/af.cpp | 26 +++++++++++++++++++++----- src/ipa/libipa/af.h | 5 ++++- src/ipa/softisp/algorithms/af.cpp | 5 ++--- 4 files changed, 28 insertions(+), 10 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index bb412e165..f79ef4b99 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -141,7 +141,7 @@ int Af::configure(IPAContext &context, const IPAConfigInfo &configInfo) grid.y_start = utils::alignDown(start.y, 2); grid.y_start |= IPU3_UAPI_GRID_Y_START_EN; - afAlgo_.configure(context.activeState.af); + afAlgo_.configure(context.activeState.af, configInfo.lensControls); return 0; } diff --git a/src/ipa/libipa/af.cpp b/src/ipa/libipa/af.cpp index 492d65d1e..fc71bcd7b 100644 --- a/src/ipa/libipa/af.cpp +++ b/src/ipa/libipa/af.cpp @@ -7,6 +7,8 @@ #include "af.h" +#include + namespace libcamera { namespace ipa { @@ -40,8 +42,7 @@ namespace af { */ /** - * Maximum focus steps of the VCM control - * \todo should be obtained from the VCM driver + * Default maximum focus steps of the VCM control */ static constexpr uint32_t kMaxFocusSteps = 1023; @@ -135,7 +136,7 @@ void AfAlgorithm::afReset(af::ActiveState &state) previousVariance_ = 0.0; coarseCompleted_ = false; fineCompleted_ = false; - maxStep_ = kMaxFocusSteps; + maxStep_ = initMaxStep_; } /** @@ -244,14 +245,15 @@ bool AfAlgorithm::afIsOutOfFocus(af::ActiveState &state) /** * \brief Configure the algorithm and \a state * \param[in] state The autofocus state + * \param[in] lensControls The lens controls * * The method just sets some initial variable values rather than performing real * configuration but it is supposed to be called from IPA's `configure`. */ -void AfAlgorithm::configure(af::ActiveState &state) +void AfAlgorithm::configure(af::ActiveState &state, const ControlInfoMap &lensControls) { /* Initial max focus step */ - maxStep_ = kMaxFocusSteps; + initMaxStep_ = kMaxFocusSteps; /* Initial frame ignore counter */ afIgnoreFrameReset(); @@ -262,6 +264,20 @@ void AfAlgorithm::configure(af::ActiveState &state) state.maxVariance = 0; /* The stable AF value flag. if it is true, the AF should be in a stable state. */ state.stable = false; + + const auto it = lensControls.find(V4L2_CID_FOCUS_ABSOLUTE); + if (it != lensControls.end()) { + const ControlInfo &focusInfo = it->second; + if (!focusInfo.max().isNone()) + initMaxStep_ = static_cast(focusInfo.max().get()); + if (!focusInfo.def().isNone()) + state.focus = static_cast(focusInfo.def().get()); + } + + maxStep_ = initMaxStep_; + + LOG(Af, Debug) << "Configured focus parameters: max=" << initMaxStep_ + << " focus=" << state.focus; } /** diff --git a/src/ipa/libipa/af.h b/src/ipa/libipa/af.h index de9f7e0ba..464fc0f7f 100644 --- a/src/ipa/libipa/af.h +++ b/src/ipa/libipa/af.h @@ -12,6 +12,7 @@ #include #include +#include namespace libcamera { @@ -39,7 +40,7 @@ class AfAlgorithm public: AfAlgorithm(); - void configure(af::ActiveState &state); + void configure(af::ActiveState &state, const ControlInfoMap &lensControls); void process(af::ActiveState &state, double variance); bool coarseCompleted() { return coarseCompleted_; } @@ -65,6 +66,8 @@ private: double previousVariance_; /* The designated maximum range of focus scanning. */ uint32_t maxStep_; + /* Initial value of maxStep_ */ + uint32_t initMaxStep_; /* If the coarse scan completes, it is set to true. */ bool coarseCompleted_; /* If the fine scan completes, it is set to true. */ diff --git a/src/ipa/softisp/algorithms/af.cpp b/src/ipa/softisp/algorithms/af.cpp index 557cca004..4fb834f8c 100644 --- a/src/ipa/softisp/algorithms/af.cpp +++ b/src/ipa/softisp/algorithms/af.cpp @@ -27,10 +27,9 @@ int Af::init(IPAContext &context, [[maybe_unused]] const ValueNode &tuningData) return 0; } -int Af::configure(IPAContext &context, - [[maybe_unused]] const IPAConfigInfo &configInfo) +int Af::configure(IPAContext &context, const IPAConfigInfo &configInfo) { - afAlgo_.configure(context.activeState.af); + afAlgo_.configure(context.activeState.af, configInfo.lensControls); return 0; }