From patchwork Thu Nov 19 20:46:04 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 29049 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2F5F5C3355 for ; Thu, 19 Nov 2026 20:46:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 04C916A930; Thu, 19 Nov 2026 21:46:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="CNUROKBN"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5B88C6A861 for ; Thu, 19 Nov 2026 21:46:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1795121191; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qe5yAfJ/EL8YaMJRU7xGXd1699bgyk57AwdKb4wQotA=; b=CNUROKBNt/iQTyQ09cP7dzA/qs4N/GDXh7sHWPnjXBcd1vdZOVQeJ7ZZhnhunfdTICb1sj /2+un5Z4ie405I3OXkp/QI/87MzbWj7i07ehOtsW0tu/wCNeroEvrI87yYDPJnjBQPYnSn vvvG8swajactHCfv8ezOA7lSakU5LiQ= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-563-P7bxuob5OpaU9baFbagLyA-1; Thu, 19 Nov 2026 20:46:25 +0000 X-MC-Unique: P7bxuob5OpaU9baFbagLyA-1 X-Mimecast-MFC-AGG-ID: P7bxuob5OpaU9baFbagLyA_1795121184 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 55A8A180047F; Thu, 19 Nov 2026 20:46:23 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2C2C4195604C; Thu, 19 Nov 2026 20:46:20 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader , Jayden Milne Subject: [PATCH v3 1/8] libcamera: software_isp: Let a single point decide about computing stats Date: Thu, 19 Nov 2026 21:46:04 +0100 Message-ID: <20261119204611.278218-2-mzamazal@redhat.com> In-Reply-To: <20261119204611.278218-1-mzamazal@redhat.com> References: <20261119204611.278218-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: DX0HJrkjqt1tHxGlFT5DbSRNNo9qTe2kwX3Dn47Xg8M_1795121184 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Currently, the decision whether to compute stats for a given frame is implemented in SwStatsCpu in several places and duplicated in debayer_egl.cpp. Let's decide it solely in SwStatsCpu. This will also help if a more flexible or complex update frequency is introduced, e.g. to support stats for autofocus. Signed-off-by: Milan Zamazal --- .../internal/software_isp/swstats_cpu.h | 21 ++++++++++--------- src/libcamera/software_isp/debayer_egl.cpp | 6 +++--- src/libcamera/software_isp/swstats_cpu.cpp | 16 +++++++++++--- 3 files changed, 27 insertions(+), 16 deletions(-) diff --git a/include/libcamera/internal/software_isp/swstats_cpu.h b/include/libcamera/internal/software_isp/swstats_cpu.h index 551870921..615049084 100644 --- a/include/libcamera/internal/software_isp/swstats_cpu.h +++ b/include/libcamera/internal/software_isp/swstats_cpu.h @@ -38,14 +38,6 @@ public: SwStatsCpu(const CameraManager &cm); ~SwStatsCpu() = default; - /* - * The combination of pipeline + sensor delays means that - * exposure changes can take up to 3 frames to get applied, - * Run stats once every 4 frames to ensure any previous - * exposure changes have been applied. - */ - static constexpr uint32_t kStatPerNumFrames = 4; - bool isValid() const { return sharedStats_.fd().isValid(); } const SharedFD &getStatsFD() { return sharedStats_.fd(); } @@ -54,13 +46,14 @@ public: int configure(const StreamConfiguration &inputCfg, unsigned int statsBufferCount = 1); void setWindow(const Rectangle &window); + bool updating(uint32_t frame); void startFrame(uint32_t frame); void finishFrame(uint32_t frame, uint32_t bufferId); void processFrame(uint32_t frame, uint32_t bufferId, MappedFrameBuffer &input); void processLine0(uint32_t frame, unsigned int y, const uint8_t *src[], unsigned int statsBufferIndex = 0) { - if (frame % kStatPerNumFrames) + if (!updating(frame)) return; if ((y & ySkipMask_) || y < static_cast(window_.y) || @@ -72,7 +65,7 @@ public: void processLine2(uint32_t frame, unsigned int y, const uint8_t *src[], unsigned int statsBufferIndex = 0) { - if (frame % kStatPerNumFrames) + if (!updating(frame)) return; if ((y & ySkipMask_) || y < static_cast(window_.y) || @@ -88,6 +81,14 @@ private: using statsProcessFn = void (SwStatsCpu::*)(const uint8_t *src[], SwIspStats &stats); using processFrameFn = void (SwStatsCpu::*)(MappedFrameBuffer &in); + /* + * The combination of pipeline + sensor delays means that + * exposure changes can take up to 3 frames to get applied, + * Run stats once every 4 frames to ensure any previous + * exposure changes have been applied. + */ + static constexpr uint32_t kStatPerNumFrames = 4; + int setupStandardBayerOrder(BayerFormat::Order order); /* Bayer 8 bpp unpacked */ void statsBGGR8Line0(const uint8_t *src[], SwIspStats &stats); diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index 33933b11d..af5eaab93 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -646,9 +646,7 @@ void DebayerEGL::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output metadata.planes()[0].bytesused = output->planes()[0].length; /* Calculate stats for the whole frame */ - if (frame % SwStatsCpu::kStatPerNumFrames) { - stats_->finishFrame(frame, 0); - } else { + if (stats_->updating(frame)) { if (!inMapped) { /* * The buffer was directly imported into EGL and thus @@ -660,6 +658,8 @@ void DebayerEGL::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output inMapped.emplace(input, MappedFrameBuffer::MapFlag::Read); } stats_->processFrame(frame, 0, inMapped.value()); + } else { + stats_->finishFrame(frame, 0); } inDmaSyncer.reset(); diff --git a/src/libcamera/software_isp/swstats_cpu.cpp b/src/libcamera/software_isp/swstats_cpu.cpp index 74633815e..0aa5597a9 100644 --- a/src/libcamera/software_isp/swstats_cpu.cpp +++ b/src/libcamera/software_isp/swstats_cpu.cpp @@ -398,7 +398,7 @@ void SwStatsCpu::statsGBRG12PLine0(const uint8_t *src[], SwIspStats &stats) */ void SwStatsCpu::startFrame(uint32_t frame) { - if (frame % kStatPerNumFrames) + if (!updating(frame)) return; if (window_.width == 0) @@ -420,7 +420,7 @@ void SwStatsCpu::startFrame(uint32_t frame) */ void SwStatsCpu::finishFrame(uint32_t frame, uint32_t bufferId) { - bool valid = frame % kStatPerNumFrames == 0; + bool valid = updating(frame); if (valid) { sharedStats_->sum_ = RGB({ 0, 0, 0 }); @@ -622,7 +622,7 @@ void SwStatsCpu::processBayerFrame2(MappedFrameBuffer &in) */ void SwStatsCpu::processFrame(uint32_t frame, uint32_t bufferId, MappedFrameBuffer &input) { - if (frame % kStatPerNumFrames) { + if (!updating(frame)) { finishFrame(frame, bufferId); return; } @@ -634,4 +634,14 @@ void SwStatsCpu::processFrame(uint32_t frame, uint32_t bufferId, MappedFrameBuff bench_.finishFrame(); } +/** + * \brief Determine whether stats update is required for the given frame + * \param[in] frame The frame number + * \return Whether the stats are updated for the given frame + */ +bool SwStatsCpu::updating(uint32_t frame) +{ + return frame % kStatPerNumFrames == 0; +} + } /* namespace libcamera */ From patchwork Thu Nov 19 20:46:05 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 29050 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 82DF6C3355 for ; Thu, 19 Nov 2026 20:46:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8A1596A939; Thu, 19 Nov 2026 21:46:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="HK0hPZvX"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 93CE96A928 for ; Thu, 19 Nov 2026 21:46:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1795121191; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=G0Ek+EmXuyrxKFpT0U+4PQU+aA0HBdfkCRPZlPzszis=; b=HK0hPZvXYUu1LVTAK6Cs6oeI+Gme2KDOh0JsA6h3viYMHGcaO78xvgfQM7wBWr3Za8T9Me pkURLIeoTpm0TVCmzGcZLSADc6JB3reqCqs0SibUZAHMVX1OQ3VwlunGZ2bJTsRXOoRGWG iRw4VuizVxMYIEJ1eQSV84lRfNPoADg= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-621-vAyhFXUyNNSfxFrvd3xY8w-1; Thu, 19 Nov 2026 20:46:27 +0000 X-MC-Unique: vAyhFXUyNNSfxFrvd3xY8w-1 X-Mimecast-MFC-AGG-ID: vAyhFXUyNNSfxFrvd3xY8w_1795121186 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C9575195607A; Thu, 19 Nov 2026 20:46:25 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B5A0D195604E; Thu, 19 Nov 2026 20:46:23 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader , Jayden Milne Subject: [PATCH v3 2/8] ipa: ipu3: Move non-specific parts of Af to libipa Date: Thu, 19 Nov 2026 21:46:05 +0100 Message-ID: <20261119204611.278218-3-mzamazal@redhat.com> In-Reply-To: <20261119204611.278218-1-mzamazal@redhat.com> References: <20261119204611.278218-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: IA19FaUiqCVcsdxqKQfrkWMshEtHsE2tD1gROWcoe7o_1795121186 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Some parts of the ipu3 autofocus algorithm could be reused by other pipelines, most notably softisp. Let's move the parts of the algorithm that are not specific to the IPU3 hardware to libipa and use them from there in ipu3. Signed-off-by: Milan Zamazal --- src/ipa/ipu3/algorithms/af.cpp | 219 +----------------------- src/ipa/ipu3/algorithms/af.h | 27 +-- src/ipa/ipu3/ipa_context.cpp | 17 +- src/ipa/ipu3/ipa_context.h | 8 +- src/ipa/libipa/af.cpp | 299 +++++++++++++++++++++++++++++++++ src/ipa/libipa/af.h | 69 ++++++++ src/ipa/libipa/meson.build | 2 + 7 files changed, 382 insertions(+), 259 deletions(-) create mode 100644 src/ipa/libipa/af.cpp create mode 100644 src/ipa/libipa/af.h diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index e796b06a2..bb412e165 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -59,25 +59,6 @@ namespace ipa::ipu3::algorithms { LOG_DEFINE_CATEGORY(IPU3Af) -/** - * Maximum focus steps of the VCM control - * \todo should be obtained from the VCM driver - */ -static constexpr uint32_t kMaxFocusSteps = 1023; - -/* Minimum focus step for searching appropriate focus */ -static constexpr uint32_t kCoarseSearchStep = 30; -static constexpr uint32_t kFineSearchStep = 1; - -/* Max ratio of variance change, 0.0 < kMaxChange < 1.0 */ -static constexpr double kMaxChange = 0.5; - -/* The numbers of frame to be ignored, before performing focus scan. */ -static constexpr uint32_t kIgnoreFrame = 10; - -/* Fine scan range 0 < kFineRange < 1 */ -static constexpr double kFineRange = 0.05; - /* Settings for IPU3 AF filter */ static struct ipu3_uapi_af_filter_config afFilterConfigDefault = { .y1_coeff_0 = { 0, 1, 3, 7 }, @@ -105,8 +86,6 @@ static struct ipu3_uapi_af_filter_config afFilterConfigDefault = { * image. */ Af::Af() - : focus_(0), bestFocus_(0), currentVariance_(0.0), previousVariance_(0.0), - coarseCompleted_(false), fineCompleted_(false) { } @@ -162,18 +141,7 @@ int Af::configure(IPAContext &context, const IPAConfigInfo &configInfo) grid.y_start = utils::alignDown(start.y, 2); grid.y_start |= IPU3_UAPI_GRID_Y_START_EN; - /* Initial max focus step */ - maxStep_ = kMaxFocusSteps; - - /* Initial frame ignore counter */ - afIgnoreFrameReset(); - - /* Initial focus value */ - context.activeState.af.focus = 0; - /* Maximum variance of the AF statistics */ - context.activeState.af.maxVariance = 0; - /* The stable AF value flag. if it is true, the AF should be in a stable state. */ - context.activeState.af.stable = false; + afAlgo_.configure(context.activeState.af); return 0; } @@ -194,151 +162,6 @@ void Af::prepare(IPAContext &context, params->use.acc_af = 1; } -/** - * \brief AF coarse scan - * \param[in] context The shared IPA context - * - * Find a near focused image using a coarse step. The step is determined by - * kCoarseSearchStep. - */ -void Af::afCoarseScan(IPAContext &context) -{ - if (coarseCompleted_) - return; - - if (afNeedIgnoreFrame()) - return; - - if (afScan(context, kCoarseSearchStep)) { - coarseCompleted_ = true; - context.activeState.af.maxVariance = 0; - focus_ = context.activeState.af.focus - - (context.activeState.af.focus * kFineRange); - context.activeState.af.focus = focus_; - previousVariance_ = 0; - maxStep_ = std::clamp(focus_ + static_cast((focus_ * kFineRange)), - 0U, kMaxFocusSteps); - } -} - -/** - * \brief AF fine scan - * \param[in] context The shared IPA context - * - * Find an optimum lens position with moving 1 step for each search. - */ -void Af::afFineScan(IPAContext &context) -{ - if (!coarseCompleted_) - return; - - if (afNeedIgnoreFrame()) - return; - - if (afScan(context, kFineSearchStep)) { - context.activeState.af.stable = true; - fineCompleted_ = true; - } -} - -/** - * \brief AF reset - * \param[in] context The shared IPA context - * - * Reset all the parameters to start over the AF process. - */ -void Af::afReset(IPAContext &context) -{ - if (afNeedIgnoreFrame()) - return; - - context.activeState.af.maxVariance = 0; - context.activeState.af.focus = 0; - focus_ = 0; - context.activeState.af.stable = false; - ignoreCounter_ = kIgnoreFrame; - previousVariance_ = 0.0; - coarseCompleted_ = false; - fineCompleted_ = false; - maxStep_ = kMaxFocusSteps; -} - -/** - * \brief AF variance comparison - * \param[in] context The IPA context - * \param[in] min_step The VCM movement step - * - * We always pick the largest variance to replace the previous one. The image - * with a larger variance also indicates it is a clearer image than previous - * one. If we find a negative derivative, we return immediately. - * - * \return True, if it finds a AF value. - */ -bool Af::afScan(IPAContext &context, int min_step) -{ - if (focus_ > maxStep_) { - /* If reach the max step, move lens to the position. */ - context.activeState.af.focus = bestFocus_; - return true; - } else { - /* - * Find the maximum of the variance by estimating its - * derivative. If the direction changes, it means we have - * passed a maximum one step before. - */ - if ((currentVariance_ - context.activeState.af.maxVariance) >= - -(context.activeState.af.maxVariance * 0.1)) { - /* - * Positive and zero derivative: - * The variance is still increasing. The focus could be - * increased for the next comparison. Also, the max variance - * and previous focus value are updated. - */ - bestFocus_ = focus_; - focus_ += min_step; - context.activeState.af.focus = focus_; - context.activeState.af.maxVariance = currentVariance_; - } else { - /* - * Negative derivative: - * The variance starts to decrease which means the maximum - * variance is found. Set focus step to previous good one - * then return immediately. - */ - context.activeState.af.focus = bestFocus_; - return true; - } - } - - previousVariance_ = currentVariance_; - LOG(IPU3Af, Debug) << " Previous step is " - << bestFocus_ - << " Current step is " - << focus_; - return false; -} - -/** - * \brief Determine the frame to be ignored - * \return Return True if the frame should be ignored, false otherwise - */ -bool Af::afNeedIgnoreFrame() -{ - if (ignoreCounter_ == 0) - return false; - else - ignoreCounter_--; - return true; -} - -/** - * \brief Reset frame ignore counter - */ -void Af::afIgnoreFrameReset() -{ - ignoreCounter_ = kIgnoreFrame; -} - /** * \brief Estimate variance * \param[in] y_items The AF filter data set from the IPU3 statistics buffer @@ -373,33 +196,6 @@ double Af::afEstimateVariance(std::span y_items, bool isY1 return var_sum / y_items.size(); } -/** - * \brief Determine out-of-focus situation - * \param[in] context The IPA context - * - * Out-of-focus means that the variance change rate for a focused and a new - * variance is greater than a threshold. - * - * \return True if the variance threshold is crossed indicating lost focus, - * false otherwise - */ -bool Af::afIsOutOfFocus(IPAContext &context) -{ - const uint32_t diff_var = std::abs(currentVariance_ - - context.activeState.af.maxVariance); - const double var_ratio = diff_var / context.activeState.af.maxVariance; - - LOG(IPU3Af, Debug) << "Variance change rate: " - << var_ratio - << " Current VCM step: " - << context.activeState.af.focus; - - if (var_ratio > kMaxChange) - return true; - else - return false; -} - /** * \brief Determine the max contrast image and lens position * \param[in] context The IPA context @@ -438,17 +234,8 @@ void Af::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, * For coarse: y1 are used. * For fine: y2 results are used. */ - currentVariance_ = afEstimateVariance(y_items, !coarseCompleted_); - - if (!context.activeState.af.stable) { - afCoarseScan(context); - afFineScan(context); - } else { - if (afIsOutOfFocus(context)) - afReset(context); - else - afIgnoreFrameReset(); - } + double variance = afEstimateVariance(y_items, !afAlgo_.coarseCompleted()); + afAlgo_.process(context.activeState.af, variance); } REGISTER_IPA_ALGORITHM(Af, "Af") diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index 320b67496..fcf1c9cf6 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -13,6 +13,8 @@ #include +#include "libipa/af.h" + #include "algorithm.h" namespace libcamera { @@ -40,32 +42,9 @@ public: ControlList &metadata) override; private: - void afCoarseScan(IPAContext &context); - void afFineScan(IPAContext &context); - bool afScan(IPAContext &context, int min_step); - void afReset(IPAContext &context); - bool afNeedIgnoreFrame(); - void afIgnoreFrameReset(); double afEstimateVariance(std::span y_items, bool isY1); - bool afIsOutOfFocus(IPAContext &context); - - /* VCM step configuration. It is the current setting of the VCM step. */ - uint32_t focus_; - /* The best VCM step. It is a local optimum VCM step during scanning. */ - uint32_t bestFocus_; - /* Current AF statistic variance. */ - double currentVariance_; - /* The frames are ignore before starting measuring. */ - uint32_t ignoreCounter_; - /* It is used to determine the derivative during scanning */ - double previousVariance_; - /* The designated maximum range of focus scanning. */ - uint32_t maxStep_; - /* If the coarse scan completes, it is set to true. */ - bool coarseCompleted_; - /* If the fine scan completes, it is set to true. */ - bool fineCompleted_; + AfAlgorithm afAlgo_; }; } /* namespace ipa::ipu3::algorithms */ diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 6632c6f45..9f628cb45 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -88,22 +88,13 @@ namespace libcamera::ipa::ipu3 { */ /** - * \var IPAActiveState::af - * \brief Context for the Automatic Focus algorithm - * - * \var IPAActiveState::af.focus - * \brief Current position of the lens - * - * \var IPAActiveState::af.maxVariance - * \brief The maximum variance of the current image - * - * \var IPAActiveState::af.stable - * \brief It is set to true, if the best focus is found + * \var IPASessionConfiguration::agc + * \brief AGC parameters configuration of the IPA */ /** - * \var IPASessionConfiguration::agc - * \brief AGC parameters configuration of the IPA + * \var IPAActiveState::af + * \brief Autofocus state */ /** diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index fde628c22..ca3a9fc9d 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -24,6 +24,7 @@ #include #include #include +#include "libipa/af.h" namespace libcamera { @@ -44,12 +45,7 @@ struct IPASessionConfiguration { }; struct IPAActiveState { - struct { - uint32_t focus; - double maxVariance; - bool stable; - } af; - + ipa::af::ActiveState af; agc::ActiveState agc; ipa::awb::ActiveState awb; ipa::ccm::ActiveState ccm; diff --git a/src/ipa/libipa/af.cpp b/src/ipa/libipa/af.cpp new file mode 100644 index 000000000..492d65d1e --- /dev/null +++ b/src/ipa/libipa/af.cpp @@ -0,0 +1,299 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2026 Red Hat + * + * Autofocus algorithm + */ + +#include "af.h" + +namespace libcamera { + +namespace ipa { + +LOG_DEFINE_CATEGORY(Af) + +namespace af { + +/** + * \struct ActiveState + * \brief Autofocus algorithm state + * + * \var ActiveState::focus + * \brief The focus value for the V4L2 control + * + * \var ActiveState::maxVariance + * \brief Maximum variance of the AF statistics + * + * \var ActiveState::stable + * \brief Flag determining whether focus is in a stable state + */ + +} /* namespace af */ + +/** + * \class AfAlgorithm + * \brief Implementation of a common autofocus algorithm + * + * The algorithm is provided with a single variance value in + * \a AfAlgorithm::process and adjusts \a ActiveState::focus accordingly. + */ + +/** + * Maximum focus steps of the VCM control + * \todo should be obtained from the VCM driver + */ +static constexpr uint32_t kMaxFocusSteps = 1023; + +/* Minimum focus step for searching appropriate focus */ +static constexpr uint32_t kCoarseSearchStep = 30; +static constexpr uint32_t kFineSearchStep = 1; + +/* Max ratio of variance change, 0.0 < kMaxChange < 1.0 */ +static constexpr double kMaxChange = 0.5; + +/* The numbers of frame to be ignored, before performing focus scan. */ +static constexpr uint32_t kIgnoreFrame = 10; + +/* Fine scan range 0 < kFineRange < 1 */ +static constexpr double kFineRange = 0.05; + +AfAlgorithm::AfAlgorithm() + : focus_(0), bestFocus_(0), currentVariance_(0.0), previousVariance_(0.0), + coarseCompleted_(false), fineCompleted_(false) +{ +} + +/** + * \fn AfAlgorithm::coarseCompleted() + * \brief Return whether the coarse autofocus scan is completed + */ + +/** + * \brief AF coarse scan + * \param[in] state The autofocus state + * + * Find a near focused image using a coarse step. The step is determined by + * kCoarseSearchStep. + */ +void AfAlgorithm::afCoarseScan(af::ActiveState &state) +{ + if (coarseCompleted_) + return; + + if (afNeedIgnoreFrame()) + return; + + if (afScan(state, kCoarseSearchStep)) { + coarseCompleted_ = true; + state.maxVariance = 0; + focus_ = state.focus - + (state.focus * kFineRange); + state.focus = focus_; + previousVariance_ = 0; + maxStep_ = std::clamp(focus_ + static_cast((focus_ * kFineRange)), + 0U, kMaxFocusSteps); + } +} + +/** + * \brief AF fine scan + * \param[in] state The autofocus state + * + * Find an optimum lens position with moving 1 step for each search. + */ +void AfAlgorithm::afFineScan(af::ActiveState &state) +{ + if (!coarseCompleted_) + return; + + if (afNeedIgnoreFrame()) + return; + + if (afScan(state, kFineSearchStep)) { + state.stable = true; + fineCompleted_ = true; + } +} + +/** + * \brief AF reset + * \param[in] state The autofocus state + * + * Reset all the parameters to start over the AF process. + */ +void AfAlgorithm::afReset(af::ActiveState &state) +{ + if (afNeedIgnoreFrame()) + return; + + state.maxVariance = 0; + state.focus = 0; + focus_ = 0; + state.stable = false; + ignoreCounter_ = kIgnoreFrame; + previousVariance_ = 0.0; + coarseCompleted_ = false; + fineCompleted_ = false; + maxStep_ = kMaxFocusSteps; +} + +/** + * \brief AF variance comparison + * \param[in] state The autofocus state + * \param[in] min_step The VCM movement step + * + * We always pick the largest variance to replace the previous one. The image + * with a larger variance also indicates it is a clearer image than previous + * one. If we find a negative derivative, we return immediately. + * + * \return True, if it finds a AF value + */ +bool AfAlgorithm::afScan(af::ActiveState &state, int min_step) +{ + if (focus_ > maxStep_) { + /* If reach the max step, move lens to the position. */ + state.focus = bestFocus_; + return true; + } else { + /* + * Find the maximum of the variance by estimating its + * derivative. If the direction changes, it means we have + * passed a maximum one step before. + */ + if ((currentVariance_ - state.maxVariance) >= + -(state.maxVariance * 0.1)) { + /* + * Positive and zero derivative: + * The variance is still increasing. The focus could be + * increased for the next comparison. Also, the max variance + * and previous focus value are updated. + */ + bestFocus_ = focus_; + focus_ += min_step; + state.focus = focus_; + state.maxVariance = currentVariance_; + } else { + /* + * Negative derivative: + * The variance starts to decrease which means the maximum + * variance is found. Set focus step to previous good one + * then return immediately. + */ + state.focus = bestFocus_; + return true; + } + } + + previousVariance_ = currentVariance_; + LOG(Af, Debug) << " Previous step is " + << bestFocus_ + << " Current step is " + << focus_; + return false; +} + +/** + * \brief Determine the frame to be ignored + * \return Return True if the frame should be ignored, false otherwise + */ +bool AfAlgorithm::afNeedIgnoreFrame() +{ + if (ignoreCounter_ == 0) + return false; + else + ignoreCounter_--; + return true; +} + +/** + * \brief Reset frame ignore counter + */ +void AfAlgorithm::afIgnoreFrameReset() +{ + ignoreCounter_ = kIgnoreFrame; +} + +/** + * \brief Determine out-of-focus situation + * \param[in] state The autofocus state + * + * Out-of-focus means that the variance change rate for a focused and a new + * variance is greater than a threshold. + * + * \return True if the variance threshold is crossed indicating lost focus, + * false otherwise + */ +bool AfAlgorithm::afIsOutOfFocus(af::ActiveState &state) +{ + const uint32_t diff_var = std::abs(currentVariance_ - + state.maxVariance); + const double var_ratio = diff_var / state.maxVariance; + + LOG(Af, Debug) << "Variance change rate: " + << var_ratio + << " Current VCM step: " + << state.focus; + + if (var_ratio > kMaxChange) + return true; + else + return false; +} + +/** + * \brief Configure the algorithm and \a state + * \param[in] state The autofocus state + * + * The method just sets some initial variable values rather than performing real + * configuration but it is supposed to be called from IPA's `configure`. + */ +void AfAlgorithm::configure(af::ActiveState &state) +{ + /* Initial max focus step */ + maxStep_ = kMaxFocusSteps; + + /* Initial frame ignore counter */ + afIgnoreFrameReset(); + + /* Initial focus value */ + state.focus = 0; + /* Maximum variance of the AF statistics */ + state.maxVariance = 0; + /* The stable AF value flag. if it is true, the AF should be in a stable state. */ + state.stable = false; +} + +/** + * \brief Determine the max contrast image and lens position + * \param[in] state The autofocus state + * \param[in] variance The variance of autofocus statistics + * + * Ideally, a clear image also has a relatively higher contrast. So, every + * image for each focus step should be tested to find an optimal focus step. + * + * The Hill Climbing Algorithm[1] is used to find the maximum variance of the AF + * statistics. The focus step is increased then the variance of the AF + * statistics are estimated. If it finds the negative derivative we have just + * passed the peak, and we infer that the best focus is found. + * + * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing + */ +void AfAlgorithm::process(af::ActiveState &state, double variance) +{ + currentVariance_ = variance; + + if (!state.stable) { + afCoarseScan(state); + afFineScan(state); + } else { + if (afIsOutOfFocus(state)) + afReset(state); + else + afIgnoreFrameReset(); + } +} + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/af.h b/src/ipa/libipa/af.h new file mode 100644 index 000000000..c902b434c --- /dev/null +++ b/src/ipa/libipa/af.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2026 Red Hat + * + * Autofocus algorithm + */ + +#pragma once + +#include + +#include + +namespace libcamera { + +namespace ipa { + +LOG_DECLARE_CATEGORY(Af) + +namespace af { + +struct ActiveState { + uint32_t focus; + double maxVariance; + bool stable; +}; + +} /* namespace af */ + +class AfAlgorithm +{ +public: + AfAlgorithm(); + + void configure(af::ActiveState &state); + void process(af::ActiveState &state, double variance); + bool coarseCompleted() { return coarseCompleted_; } + +private: + void afCoarseScan(af::ActiveState &af); + void afFineScan(af::ActiveState &af); + bool afScan(af::ActiveState &af, int min_step); + void afReset(af::ActiveState &af); + bool afNeedIgnoreFrame(); + void afIgnoreFrameReset(); + + bool afIsOutOfFocus(af::ActiveState &af); + + /* VCM step configuration. It is the current setting of the VCM step. */ + uint32_t focus_; + /* The best VCM step. It is a local optimum VCM step during scanning. */ + uint32_t bestFocus_; + /* Current AF statistic variance. */ + double currentVariance_; + /* The frames are ignore before starting measuring. */ + uint32_t ignoreCounter_; + /* It is used to determine the derivative during scanning */ + double previousVariance_; + /* The designated maximum range of focus scanning. */ + uint32_t maxStep_; + /* If the coarse scan completes, it is set to true. */ + bool coarseCompleted_; + /* If the fine scan completes, it is set to true. */ + bool fineCompleted_; +}; + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index a38d5d528..b80c3b017 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 libipa_headers = files([ + 'af.h', 'agc.h', 'agc_mean_luminance.h', 'agc_msv.h', @@ -30,6 +31,7 @@ libipa_headers = files([ ]) libipa_sources = files([ + 'af.cpp', 'agc.cpp', 'agc_mean_luminance.cpp', 'agc_msv.cpp', From patchwork Thu Nov 19 20:46:06 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 29051 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 697E4C3366 for ; Thu, 19 Nov 2026 20:46:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CA1126A935; Thu, 19 Nov 2026 21:46:37 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="WuKHga8Y"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C78296A92D for ; Thu, 19 Nov 2026 21:46:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1795121194; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XN9V97KmHUO/OwLqfsBCCQZj4ZrVS2Hsdrq1z90iaDQ=; b=WuKHga8YLF4YocapcvALDgI8aCa/+2NxFTQ1pDBSD8wGrAf/ytLeqP8zum1UwIxzgIG66k 2o3WEFb4G6WykyVEtTzgedjYnZP25XIgxqLBrJkfeza/6UtAsYOtUhMQke2DtjOyMkC2ac FwjUPHK4S/7BSUvxoSn3WUfI3tAQtBg= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-30-RRrK1yrpP3OcrfcI-mxcSQ-1; Thu, 19 Nov 2026 20:46:29 +0000 X-MC-Unique: RRrK1yrpP3OcrfcI-mxcSQ-1 X-Mimecast-MFC-AGG-ID: RRrK1yrpP3OcrfcI-mxcSQ_1795121188 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 814EC180065F; Thu, 19 Nov 2026 20:46:28 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 773C3195604E; Thu, 19 Nov 2026 20:46:26 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader , Jayden Milne Subject: [PATCH v3 3/8] ipa: libipa: Add af::prepareControls() Date: Thu, 19 Nov 2026 21:46:06 +0100 Message-ID: <20261119204611.278218-4-mzamazal@redhat.com> In-Reply-To: <20261119204611.278218-1-mzamazal@redhat.com> References: <20261119204611.278218-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: wTHxJiQ1N_trOG3YZR_nGcUGXBk6LC3t8pWgwMCJwdg_1795121188 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Similarly to what other algorithms do, let's add a control initialisation function to the libipa autofocus algorithm and use it in ipu3 IPA. Signed-off-by: Milan Zamazal --- src/ipa/ipu3/ipu3.cpp | 3 +-- src/ipa/libipa/af.h | 7 +++++++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index e4333539a..605a57eea 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -554,8 +554,7 @@ void IPAIPU3::setControls(unsigned int frame) agc::prepareControls(ctrls, context_.camHelper.get(), frameContext.agc); ControlList lensCtrls(lensCtrls_); - lensCtrls.set(V4L2_CID_FOCUS_ABSOLUTE, - static_cast(context_.activeState.af.focus)); + af::prepareControls(lensCtrls, context_.activeState.af.focus); setSensorControls.emit(frame, ctrls, lensCtrls); } diff --git a/src/ipa/libipa/af.h b/src/ipa/libipa/af.h index c902b434c..de9f7e0ba 100644 --- a/src/ipa/libipa/af.h +++ b/src/ipa/libipa/af.h @@ -7,6 +7,8 @@ #pragma once +#include + #include #include @@ -25,6 +27,11 @@ struct ActiveState { bool stable; }; +inline void prepareControls(ControlList &controls, int32_t focus) +{ + controls.set(V4L2_CID_FOCUS_ABSOLUTE, focus); +} + } /* namespace af */ class AfAlgorithm From patchwork Thu Nov 19 20:46:07 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 29052 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BDA6CC3355 for ; Thu, 19 Nov 2026 20:46:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B2A206A93E; Thu, 19 Nov 2026 21:46:39 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="KTEPDTPS"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9742C6A92B for ; Thu, 19 Nov 2026 21:46:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1795121195; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CnZ9nLs7HSTUQOQbH6OpA94OG3IJfJcqNyBU+6o8wwc=; b=KTEPDTPS/RfmyoSTcaIArpCgHNbvkPVmuzaOVez+BTqk36oalihnugwfk/6Aw81rQRf/an 9NFV7XQC3Za+2ZazvO+4yxqGyZtq/jNSVzNuygGeAtrxo+WjRpc5HTQKX45EK6S9SsPr0e S9rWtlW4xW2uxEDKtnkJxStTEqPrToo= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-142-qyBO7QvzPc2hv-FLgYcU_A-1; Thu, 19 Nov 2026 20:46:31 +0000 X-MC-Unique: qyBO7QvzPc2hv-FLgYcU_A-1 X-Mimecast-MFC-AGG-ID: qyBO7QvzPc2hv-FLgYcU_A_1795121190 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9A7741800630; Thu, 19 Nov 2026 20:46:30 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E215B195604E; Thu, 19 Nov 2026 20:46:28 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader , Jayden Milne Subject: [PATCH v3 4/8] libcamera: software_isp: Add variance to stats Date: Thu, 19 Nov 2026 21:46:07 +0100 Message-ID: <20261119204611.278218-5-mzamazal@redhat.com> In-Reply-To: <20261119204611.278218-1-mzamazal@redhat.com> References: <20261119204611.278218-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: epWDGBy0rpvlXhVxJ-F8LHwQVlE6jLA_ePIEuGuiLi8_1795121190 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The libipa Af algorithm requests variance value as its input. Let's add the corresponding member to SwIspStats and initialise it. This commit is preparatory and doesn't implement computation of the variance nor introduces other changes to the stats like when to update them for autofocus. Signed-off-by: Milan Zamazal --- include/libcamera/internal/software_isp/swisp_stats.h | 8 +++++++- src/libcamera/software_isp/swstats_cpu.cpp | 3 +++ 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/software_isp/swisp_stats.h b/include/libcamera/internal/software_isp/swisp_stats.h index 65d826f6c..a0f6f4a79 100644 --- a/include/libcamera/internal/software_isp/swisp_stats.h +++ b/include/libcamera/internal/software_isp/swisp_stats.h @@ -17,7 +17,7 @@ namespace libcamera { /** * \brief Statistics for the Software ISP * - * The software ISP computes two types of statistics: + * The software ISP computes the following types of statistics: * * - The accumulator counts the number of non-saturated pixels and sums their * R, G and B channels individually. Pixels that have any saturated channel @@ -27,6 +27,8 @@ namespace libcamera { * - The histogram groups pixels in bins based on their luminance and counts * the number of pixels in each bin. Saturated pixels are included. * + * - The variance value for autofocus. + * * The value types are large enough to not overflow. Should they still overflow * for some reason, no check is performed and they wrap around. */ @@ -56,6 +58,10 @@ struct SwIspStats { * \brief A histogram of luminance values of all the sampled pixels */ Histogram yHistogram; + /** + * \brief A variance value for the autofocus algorithm + */ + double variance; }; } /* namespace libcamera */ diff --git a/src/libcamera/software_isp/swstats_cpu.cpp b/src/libcamera/software_isp/swstats_cpu.cpp index 0aa5597a9..ed1aa83e4 100644 --- a/src/libcamera/software_isp/swstats_cpu.cpp +++ b/src/libcamera/software_isp/swstats_cpu.cpp @@ -407,6 +407,7 @@ void SwStatsCpu::startFrame(uint32_t frame) for (auto &s : stats_) { s.sum_ = RGB({ 0, 0, 0 }); s.numPixels_ = 0; + s.variance = 0.0; s.yHistogram.fill(0); } } @@ -425,10 +426,12 @@ void SwStatsCpu::finishFrame(uint32_t frame, uint32_t bufferId) if (valid) { sharedStats_->sum_ = RGB({ 0, 0, 0 }); sharedStats_->numPixels_ = 0; + sharedStats_->variance = 0.0; sharedStats_->yHistogram.fill(0); for (const auto &s : stats_) { sharedStats_->sum_ += s.sum_; sharedStats_->numPixels_ += s.numPixels_; + sharedStats_->variance += s.variance; for (unsigned int j = 0; j < SwIspStats::kYHistogramSize; j++) sharedStats_->yHistogram[j] += s.yHistogram[j]; } From patchwork Thu Nov 19 20:46:08 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 29053 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 14461C3361 for ; Thu, 19 Nov 2026 20:46:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C789A6A92C; Thu, 19 Nov 2026 21:46:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="ide5xzwj"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B90296A934 for ; Thu, 19 Nov 2026 21:46:37 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1795121196; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wSG2bZCCuqPAsjX21RfT7YHgwLxiru6tHtBEyMb4BNs=; b=ide5xzwjyCnxDpxm8+NqirAhniAvtlgU926Cm8legNXVi7yTbssLCBo0EB6x4vMiI+R1I4 VN5Zt9hYP3AOHPgQiO01zBN9zo2mzaMRWG5Gwb/w2kWStP2kA7QsiN1K3I/mt21jTk7IW5 JlCwgselkR85iZM/qd6I+qGseJHiIvo= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-223-WRdJFjmTO4yOUqiSMHl3Nw-1; Thu, 19 Nov 2026 20:46:34 +0000 X-MC-Unique: WRdJFjmTO4yOUqiSMHl3Nw-1 X-Mimecast-MFC-AGG-ID: WRdJFjmTO4yOUqiSMHl3Nw_1795121193 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 074DC180061F; Thu, 19 Nov 2026 20:46:33 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 44E50195604C; Thu, 19 Nov 2026 20:46:30 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader , Jayden Milne Subject: [PATCH v3 5/8] ipa: softisp: Remove unused imports from ipa_context.h Date: Thu, 19 Nov 2026 21:46:08 +0100 Message-ID: <20261119204611.278218-6-mzamazal@redhat.com> In-Reply-To: <20261119204611.278218-1-mzamazal@redhat.com> References: <20261119204611.278218-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: pXVNSp4rxuQYblUjHlcc4C6FdQIgEuq7KFBeML0difw_1795121193 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham Reviewed-by: Barnabás Pőcze --- src/ipa/softisp/ipa_context.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/ipa/softisp/ipa_context.h b/src/ipa/softisp/ipa_context.h index 62d1ae84b..ccb3d7244 100644 --- a/src/ipa/softisp/ipa_context.h +++ b/src/ipa/softisp/ipa_context.h @@ -7,7 +7,6 @@ #pragma once -#include #include #include #include @@ -15,7 +14,6 @@ #include #include "libcamera/internal/matrix.h" -#include "libcamera/internal/vector.h" #include #include From patchwork Thu Nov 19 20:46:09 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 29054 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 25535C3355 for ; Thu, 19 Nov 2026 20:46:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C930C6A939; Thu, 19 Nov 2026 21:46:43 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="IXfxhNAX"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E1B706A940 for ; Thu, 19 Nov 2026 21:46:39 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1795121199; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+duszOYZ6h1uosir/kboDnD2BHHT0koJg0fte349sgs=; b=IXfxhNAXxpjeyJXkv1urtkrn35KT3nyRvbOeiMLvoS+FCIHhhahIV1BE/5SlBWdHWGh7eb uno4JvOfPBnBLEGNIdV0sfs0PCYziBxPoeJxIBbotP5lQLpYAOs8NVobenevJD56ezEyae jEAWFJ6NH7KKYnH0vbTXFmzw90qGDZ0= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-527-C6msd7_HM5CUqnz9XzbKRw-1; Thu, 19 Nov 2026 20:46:36 +0000 X-MC-Unique: C6msd7_HM5CUqnz9XzbKRw-1 X-Mimecast-MFC-AGG-ID: C6msd7_HM5CUqnz9XzbKRw_1795121195 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 18A561955F06; Thu, 19 Nov 2026 20:46:35 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6739D195604F; Thu, 19 Nov 2026 20:46:33 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader , Jayden Milne Subject: [PATCH v3 6/8] ipa: softisp: Add Af algorithm Date: Thu, 19 Nov 2026 21:46:09 +0100 Message-ID: <20261119204611.278218-7-mzamazal@redhat.com> In-Reply-To: <20261119204611.278218-1-mzamazal@redhat.com> References: <20261119204611.278218-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: q9iqJkHloWtl1m3EJnDUDrCQDDkmlMCLvt6XsI0qve4_1795121195 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Let's add Af algorithm, which is mostly a proxy to the libipa Af. The only special feature is setting afEnabled flag to prevent attempting lens movements when the algorithm is not enabled. The algorithm is not enabled yet because there are no autofocus stats computed. Signed-off-by: Milan Zamazal --- src/ipa/softisp/algorithms/af.cpp | 56 ++++++++++++++++++++++++++ src/ipa/softisp/algorithms/af.h | 39 ++++++++++++++++++ src/ipa/softisp/algorithms/meson.build | 1 + src/ipa/softisp/data/uncalibrated.yaml | 1 + src/ipa/softisp/ipa_context.h | 3 ++ 5 files changed, 100 insertions(+) create mode 100644 src/ipa/softisp/algorithms/af.cpp create mode 100644 src/ipa/softisp/algorithms/af.h diff --git a/src/ipa/softisp/algorithms/af.cpp b/src/ipa/softisp/algorithms/af.cpp new file mode 100644 index 000000000..154fec03e --- /dev/null +++ b/src/ipa/softisp/algorithms/af.cpp @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Red Hat Inc. + * + * Autofocus + */ + +#include "af.h" + +#include + +#include + +#include "libcamera/internal/software_isp/swisp_stats.h" + +namespace libcamera { + +namespace ipa::softisp::algorithms { + +LOG_DEFINE_CATEGORY(IPASoftIspAf) + +int Af::init(IPAContext &context, [[maybe_unused]] const ValueNode &tuningData) +{ + /* Informs that lens controls can get focus data */ + context.afEnabled = true; + + return 0; +} + +int Af::configure(IPAContext &context, + [[maybe_unused]] const IPAConfigInfo &configInfo) +{ + afAlgo_.configure(context.activeState.af); + + return 0; +} + +void Af::process(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + const SwIspStats *stats, + [[maybe_unused]] ControlList &metadata) +{ + if (!stats->valid) + return; + + const double variance = stats->variance; + LOG(IPASoftIspAf, Debug) << "Focus stats: " << variance; + afAlgo_.process(context.activeState.af, variance); +} + +REGISTER_IPA_ALGORITHM(Af, "Af") + +} /* namespace ipa::softisp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/softisp/algorithms/af.h b/src/ipa/softisp/algorithms/af.h new file mode 100644 index 000000000..881cb4475 --- /dev/null +++ b/src/ipa/softisp/algorithms/af.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Red Hat Inc. + * + * Autofocus + */ + +#pragma once + +#include "libipa/af.h" + +#include "algorithm.h" +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::softisp::algorithms { + +class Af : public Algorithm +{ +public: + Af() = default; + ~Af() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPAConfigInfo &configInfo) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const SwIspStats *stats, + ControlList &metadata) override; + +private: + AfAlgorithm afAlgo_; +}; + +} /* namespace ipa::softisp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/softisp/algorithms/meson.build b/src/ipa/softisp/algorithms/meson.build index d98f4b1f4..7d1077117 100644 --- a/src/ipa/softisp/algorithms/meson.build +++ b/src/ipa/softisp/algorithms/meson.build @@ -2,6 +2,7 @@ softisp_ipa_algorithms = files([ 'adjust.cpp', + 'af.cpp', 'awb.cpp', 'agc.cpp', 'blc.cpp', diff --git a/src/ipa/softisp/data/uncalibrated.yaml b/src/ipa/softisp/data/uncalibrated.yaml index fc90ca526..53d3792a2 100644 --- a/src/ipa/softisp/data/uncalibrated.yaml +++ b/src/ipa/softisp/data/uncalibrated.yaml @@ -4,6 +4,7 @@ version: 1 algorithms: - BlackLevel: + # - Af: - Awb: # Color correction matrices can be defined here. The CCM algorithm # has a significant performance impact, and should only be enabled diff --git a/src/ipa/softisp/ipa_context.h b/src/ipa/softisp/ipa_context.h index ccb3d7244..e602de31f 100644 --- a/src/ipa/softisp/ipa_context.h +++ b/src/ipa/softisp/ipa_context.h @@ -20,6 +20,7 @@ #include #include #include +#include "libipa/af.h" #include "libipa/lsc.h" #include "core_ipa_interface.h" @@ -39,6 +40,7 @@ struct IPAActiveState { ipa::awb::ActiveState awb; ipa::ccm::ActiveState ccm; ipa::agc::ActiveState agc; + ipa::af::ActiveState af; struct { uint8_t level; @@ -88,6 +90,7 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; bool ccmEnabled = false; + bool afEnabled = false; bool lscEnabled = false; }; From patchwork Thu Nov 19 20:46:10 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 29055 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5C118C3361 for ; Thu, 19 Nov 2026 20:46:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D93416A94D; Thu, 19 Nov 2026 21:46:44 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="QnyFpTvD"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D68C96A940 for ; Thu, 19 Nov 2026 21:46:41 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1795121200; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n+f7UzKI7UusM5CfdTYfW1QynUCvW+e/VjWDMdlF8D4=; b=QnyFpTvDYAbzBFe+jsjaFggKbFKbaCvREgZ1mSny/MzpjVglzTjOTT57+H4jPjxQLBpcUb FIIzGtH2v5RgWZwpAFT8lS5I2Yrxp39yHtkjRuAH53MJptG/mN3C2hzRsm2GOJUrT97fuS uKllGjZ1deWYsV0T6bwKGLMBM8TmpOw= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-679-WM3UM2IWNxKXj4QHKVz60A-1; Thu, 19 Nov 2026 20:46:38 +0000 X-MC-Unique: WM3UM2IWNxKXj4QHKVz60A-1 X-Mimecast-MFC-AGG-ID: WM3UM2IWNxKXj4QHKVz60A_1795121197 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 30DB01954B3B; Thu, 19 Nov 2026 20:46:37 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 79348195604C; Thu, 19 Nov 2026 20:46:35 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader , Jayden Milne Subject: [PATCH v3 7/8] libcamera: software_isp: Add autofocus controls Date: Thu, 19 Nov 2026 21:46:10 +0100 Message-ID: <20261119204611.278218-8-mzamazal@redhat.com> In-Reply-To: <20261119204611.278218-1-mzamazal@redhat.com> References: <20261119204611.278218-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: hu10kHScoUpAX55kqPCZoffN_ZLrEeHcJk1xJ0I3qqA_1795121197 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to support focusing operations, lens controls are set up and passed around, similarly to the sensor controls. Currently, the only control used, and only if the sensor supports autofocus, is V4L2_CID_FOCUS_ABSOLUTE. Instead of serializing the whole lens ControList in setSensorControls, we could pass around just the value. But using the ControlList is more consistent with what we do with the sensor controls and what is done in ipu3 pipeline. Additionally, it's ready for possibly amending the controls in future; we might end up having to determine between V4L2_CID_FOCUS_ABSOLUTE or V4L2_CID_FOCUS_RELATIVE, or V4L2_CID_FOCUS_AUTO settings. Signed-off-by: Milan Zamazal --- .../libcamera/internal/software_isp/software_isp.h | 4 ++-- include/libcamera/ipa/softisp.mojom | 3 ++- src/ipa/softisp/ipa_context.h | 1 + src/ipa/softisp/softisp.cpp | 7 ++++++- src/libcamera/pipeline/simple/simple.cpp | 13 +++++++++++-- src/libcamera/software_isp/software_isp.cpp | 4 ++-- 6 files changed, 24 insertions(+), 8 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index b64910637..be90a229a 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -87,11 +87,11 @@ public: Signal outputBufferReady; Signal ispStatsReady; Signal metadataReady; - Signal setSensorControls; + Signal setSensorControls; private: void saveIspParams(uint32_t frame); - void setSensorCtrls(const ControlList &sensorControls); + void setSensorCtrls(const ControlList &sensorControls, const ControlList &lensControls); void statsReady(uint32_t frame, uint32_t bufferId); void inputReady(FrameBuffer *input); void outputReady(FrameBuffer *output); diff --git a/include/libcamera/ipa/softisp.mojom b/include/libcamera/ipa/softisp.mojom index e42742ed7..833f4440b 100644 --- a/include/libcamera/ipa/softisp.mojom +++ b/include/libcamera/ipa/softisp.mojom @@ -10,6 +10,7 @@ import "include/libcamera/ipa/core.mojom"; struct IPAConfigInfo { libcamera.ControlInfoMap sensorControls; + libcamera.ControlInfoMap lensControls; }; interface IPASoftIspInterface { @@ -34,7 +35,7 @@ interface IPASoftIspInterface { }; interface IPASoftIspEventInterface { - setSensorControls(libcamera.ControlList sensorControls); + setSensorControls(libcamera.ControlList sensorControls, libcamera.ControlList lensControls); paramsComputed(uint32 frame); metadataReady(uint32 frame, libcamera.ControlList metadata); }; diff --git a/src/ipa/softisp/ipa_context.h b/src/ipa/softisp/ipa_context.h index e602de31f..308a7a6f1 100644 --- a/src/ipa/softisp/ipa_context.h +++ b/src/ipa/softisp/ipa_context.h @@ -84,6 +84,7 @@ struct IPAContext { IPACameraSensorInfo sensorInfo; ControlInfoMap sensorControls; + ControlInfoMap lensControls; std::unique_ptr camHelper; IPASessionConfiguration configuration; IPAActiveState activeState; diff --git a/src/ipa/softisp/softisp.cpp b/src/ipa/softisp/softisp.cpp index e35f07737..50e39c63e 100644 --- a/src/ipa/softisp/softisp.cpp +++ b/src/ipa/softisp/softisp.cpp @@ -27,6 +27,7 @@ #include "libcamera/internal/yaml_parser.h" #include "algorithms/adjust.h" +#include "libipa/af.h" #include "libipa/agc.h" #include "libipa/camera_sensor_helper.h" @@ -192,6 +193,7 @@ int IPASoftIsp::init(const IPASettings &settings, int IPASoftIsp::configure(const IPAConfigInfo &configInfo, ControlInfoMap *ipaControls) { context_.sensorControls = configInfo.sensorControls; + context_.lensControls = configInfo.lensControls; /* Clear the IPA context before the streaming session. */ context_.configuration = {}; @@ -255,7 +257,10 @@ void IPASoftIsp::processStats(const uint32_t frame, ControlList ctrls(context_.sensorControls); agc::prepareControls(ctrls, context_.camHelper.get(), frameContext.agc); - setSensorControls.emit(ctrls); + ControlList lensCtrls(context_.lensControls); + if (context_.afEnabled) + af::prepareControls(lensCtrls, context_.activeState.af.focus); + setSensorControls.emit(ctrls, lensCtrls); } std::string IPASoftIsp::logPrefix() const diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 2f7f4c893..8f419de89 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -34,6 +34,7 @@ #include #include "libcamera/internal/camera.h" +#include "libcamera/internal/camera_lens.h" #include "libcamera/internal/camera_manager.h" #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/camera_sensor_properties.h" @@ -374,7 +375,7 @@ private: void ispStatsReady(uint32_t frame, uint32_t bufferId); void metadataReady(uint32_t frame, const ControlList &metadata); - void setSensorControls(const ControlList &sensorControls); + void setSensorControls(const ControlList &sensorControls, const ControlList &lensControls); }; class SimpleCameraConfiguration : public CameraConfiguration @@ -1047,7 +1048,7 @@ void SimpleCameraData::metadataReady(uint32_t frame, const ControlList &metadata tryCompleteRequest(info->request); } -void SimpleCameraData::setSensorControls(const ControlList &sensorControls) +void SimpleCameraData::setSensorControls(const ControlList &sensorControls, const ControlList &lensControls) { delayedCtrls_->push(sensorControls); /* @@ -1061,6 +1062,12 @@ void SimpleCameraData::setSensorControls(const ControlList &sensorControls) if (!frameStartEmitter_) { ControlList ctrls(sensorControls); sensor_->setControls(&ctrls); + + CameraLens *lens = sensor_->focusLens(); + if (lens && lensControls.contains(V4L2_CID_FOCUS_ABSOLUTE)) { + ControlValue const &focusValue = lensControls.get(V4L2_CID_FOCUS_ABSOLUTE); + lens->setFocusPosition(focusValue.get()); + } } } @@ -1653,6 +1660,8 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) } else { ipa::softisp::IPAConfigInfo configInfo; configInfo.sensorControls = data->sensor_->controls(); + if (data->sensor_->focusLens()) + configInfo.lensControls = data->sensor_->focusLens()->controls(); return data->swIsp_->configure(inputCfg, outputCfgs, configInfo, &data->controlInfo_); } } diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 627625b4f..53ae3d76f 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -443,9 +443,9 @@ void SoftwareIsp::saveIspParams([[maybe_unused]] uint32_t frame) debayerParams_ = *sharedParams_; } -void SoftwareIsp::setSensorCtrls(const ControlList &sensorControls) +void SoftwareIsp::setSensorCtrls(const ControlList &sensorControls, const ControlList &lensControls) { - setSensorControls.emit(sensorControls); + setSensorControls.emit(sensorControls, lensControls); } void SoftwareIsp::statsReady(uint32_t frame, uint32_t bufferId) From patchwork Thu Nov 19 20:46:11 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 29056 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 691D9C3355 for ; Thu, 19 Nov 2026 20:46:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 19D556A94B; Thu, 19 Nov 2026 21:46:49 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="NwEyi4y7"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2B4676A92B for ; Thu, 19 Nov 2026 21:46:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1795121206; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8yf9RyxL/eGRvWf3oqPtkwtwef0ceWMrHtdcmXgDqnI=; b=NwEyi4y7O5hCgBCaFRVSlKZxmhVjO2G9zgTHfUk7ozUSv2t1hnamoD25yqVrovpk4XwiV/ l4RLP+2iagZrljhV/8zxGh1Rw7uv3IgBT9GLKxTs0P4bABINteoZlx8dIbZVa/D1uNGxcg 7TL0eWdBEq81ueYE0F00R9LRBJIvrD4= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-314-JAlr_41XP5eV6SgcEtsK4g-1; Thu, 19 Nov 2026 20:46:40 +0000 X-MC-Unique: JAlr_41XP5eV6SgcEtsK4g-1 X-Mimecast-MFC-AGG-ID: JAlr_41XP5eV6SgcEtsK4g_1795121199 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 432101955DC3; Thu, 19 Nov 2026 20:46:39 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.csb (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 91B15195604C; Thu, 19 Nov 2026 20:46:37 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= , Kieran Bingham , Robert Mader , Jayden Milne Subject: [PATCH v3 8/8] ipa: libipa: Use autofocus limits if available Date: Thu, 19 Nov 2026 21:46:11 +0100 Message-ID: <20261119204611.278218-9-mzamazal@redhat.com> In-Reply-To: <20261119204611.278218-1-mzamazal@redhat.com> References: <20261119204611.278218-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 3uoYtCoOPjDycvBUcNF6gz-gFx2Z1O1MNBIbP9-WwSM_1795121199 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Let's try to set the maximum and the initial focusing positions rather than using hard-wired values. Signed-off-by: Milan Zamazal --- src/ipa/ipu3/algorithms/af.cpp | 2 +- src/ipa/libipa/af.cpp | 26 +++++++++++++++++++++----- src/ipa/libipa/af.h | 5 ++++- src/ipa/softisp/algorithms/af.cpp | 5 ++--- 4 files changed, 28 insertions(+), 10 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index bb412e165..f79ef4b99 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -141,7 +141,7 @@ int Af::configure(IPAContext &context, const IPAConfigInfo &configInfo) grid.y_start = utils::alignDown(start.y, 2); grid.y_start |= IPU3_UAPI_GRID_Y_START_EN; - afAlgo_.configure(context.activeState.af); + afAlgo_.configure(context.activeState.af, configInfo.lensControls); return 0; } diff --git a/src/ipa/libipa/af.cpp b/src/ipa/libipa/af.cpp index 492d65d1e..fc71bcd7b 100644 --- a/src/ipa/libipa/af.cpp +++ b/src/ipa/libipa/af.cpp @@ -7,6 +7,8 @@ #include "af.h" +#include + namespace libcamera { namespace ipa { @@ -40,8 +42,7 @@ namespace af { */ /** - * Maximum focus steps of the VCM control - * \todo should be obtained from the VCM driver + * Default maximum focus steps of the VCM control */ static constexpr uint32_t kMaxFocusSteps = 1023; @@ -135,7 +136,7 @@ void AfAlgorithm::afReset(af::ActiveState &state) previousVariance_ = 0.0; coarseCompleted_ = false; fineCompleted_ = false; - maxStep_ = kMaxFocusSteps; + maxStep_ = initMaxStep_; } /** @@ -244,14 +245,15 @@ bool AfAlgorithm::afIsOutOfFocus(af::ActiveState &state) /** * \brief Configure the algorithm and \a state * \param[in] state The autofocus state + * \param[in] lensControls The lens controls * * The method just sets some initial variable values rather than performing real * configuration but it is supposed to be called from IPA's `configure`. */ -void AfAlgorithm::configure(af::ActiveState &state) +void AfAlgorithm::configure(af::ActiveState &state, const ControlInfoMap &lensControls) { /* Initial max focus step */ - maxStep_ = kMaxFocusSteps; + initMaxStep_ = kMaxFocusSteps; /* Initial frame ignore counter */ afIgnoreFrameReset(); @@ -262,6 +264,20 @@ void AfAlgorithm::configure(af::ActiveState &state) state.maxVariance = 0; /* The stable AF value flag. if it is true, the AF should be in a stable state. */ state.stable = false; + + const auto it = lensControls.find(V4L2_CID_FOCUS_ABSOLUTE); + if (it != lensControls.end()) { + const ControlInfo &focusInfo = it->second; + if (!focusInfo.max().isNone()) + initMaxStep_ = static_cast(focusInfo.max().get()); + if (!focusInfo.def().isNone()) + state.focus = static_cast(focusInfo.def().get()); + } + + maxStep_ = initMaxStep_; + + LOG(Af, Debug) << "Configured focus parameters: max=" << initMaxStep_ + << " focus=" << state.focus; } /** diff --git a/src/ipa/libipa/af.h b/src/ipa/libipa/af.h index de9f7e0ba..464fc0f7f 100644 --- a/src/ipa/libipa/af.h +++ b/src/ipa/libipa/af.h @@ -12,6 +12,7 @@ #include #include +#include namespace libcamera { @@ -39,7 +40,7 @@ class AfAlgorithm public: AfAlgorithm(); - void configure(af::ActiveState &state); + void configure(af::ActiveState &state, const ControlInfoMap &lensControls); void process(af::ActiveState &state, double variance); bool coarseCompleted() { return coarseCompleted_; } @@ -65,6 +66,8 @@ private: double previousVariance_; /* The designated maximum range of focus scanning. */ uint32_t maxStep_; + /* Initial value of maxStep_ */ + uint32_t initMaxStep_; /* If the coarse scan completes, it is set to true. */ bool coarseCompleted_; /* If the fine scan completes, it is set to true. */ diff --git a/src/ipa/softisp/algorithms/af.cpp b/src/ipa/softisp/algorithms/af.cpp index 154fec03e..8d14e3142 100644 --- a/src/ipa/softisp/algorithms/af.cpp +++ b/src/ipa/softisp/algorithms/af.cpp @@ -27,10 +27,9 @@ int Af::init(IPAContext &context, [[maybe_unused]] const ValueNode &tuningData) return 0; } -int Af::configure(IPAContext &context, - [[maybe_unused]] const IPAConfigInfo &configInfo) +int Af::configure(IPAContext &context, const IPAConfigInfo &configInfo) { - afAlgo_.configure(context.activeState.af); + afAlgo_.configure(context.activeState.af, configInfo.lensControls); return 0; }