From patchwork Wed May 7 07:52:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23342 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A54A5C3226 for ; Wed, 7 May 2025 08:28:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1B1FB68B2C; Wed, 7 May 2025 10:28:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="gisJUzeO"; dkim-atps=neutral Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E9D7C68AD3 for ; Wed, 7 May 2025 10:27:57 +0200 (CEST) Received: by mail-wm1-x334.google.com with SMTP id 5b1f17b1804b1-43cf825f46bso5806365e9.3 for ; Wed, 07 May 2025 01:27:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1746606477; x=1747211277; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Zd5df1quEaR166Fq4cu5Y3EUJiRzpvBQBh9y5xYqZPs=; b=gisJUzeOnjFP/YsqXgkkGQd8hRAgIwhsVaO66VyZf5FA6M/I9hqhJE05QuaENvIPXT LxgRlwjwlgn0d6pSiBp5Zkh0KCWXyUt8swjOkYBCRXcCNRNHMN7c3PPwJ89UiK6cvM7H K6Y0HN6SGHzLdyc/hiujSEMPRtmdH+R2RikcWUnk4mqaoYSove3qjOz5yn6YVDlEN9IL gnOm1y+NLyWcvpUjFJxOYMtZrq2BOay434vYtC6BTBCkbeeEraxTupLQQbJWbXzfD8sM RI0wmP5sb9+ntQn95LJ8mDUsqD/+BFEVCJwbbAuASM9YcdeM2mdbQ/5hjHitiPR4yF3L vHVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746606477; x=1747211277; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Zd5df1quEaR166Fq4cu5Y3EUJiRzpvBQBh9y5xYqZPs=; b=AR9rJcNDYvTG/dW5DgTcU3nR0k2eU1z/oM14K3OSWsjGCD4WPEg4vKdSG+VYc+IPXS CUMkq5m2EKzwmzBpQ0S+d7DbAP6OeI+P9WHIZHCjxRLsklQt0arjrPeeLP04zJ/huNj6 jiUgpC1enCYyVs8Z/QVbh7SdoDT4pgNuXMQwc2A9cdO4TjzjD3jYimMVwWCA8RXiZ3r1 n+s4bCwJ9ZQJvWn0cav+Xujg4Y8jCfdXuU3df86bRr8EBm0r3cjdQHevzXqI8pwReNum NcIHXbqLz5491HZf3h90XfohBuFGq9gdN+Boo75kO6QQmqbccY7M7MuwfXBVokvbYom3 qGLw== X-Gm-Message-State: AOJu0YzDt09SvfdXuO46p0RMyKHs8kc2SE+Q2LLGmN8Rit7W3FWrqlWI nqySWPVxzaZOnbJM/1zpn9057SSXvTf4LhyNTd2DN3RUExD2ji0niv558W12P+Xrpa9v7nBaEs+ I X-Gm-Gg: ASbGncsRGHBWnBJ0Z0zwd//A5k6Tq+/PPnQhbv5Yg5k07wtxGGi4Wwt19sh0mD0VBVM 3pcrl98gbXkB/uhEuoKKHNLVwFXIlTt0lAWH3/cjJ1ylGInB9Z73E2Vj0JoOc4T4jnOKnGgwsc4 8XswWxQwMpChwKW8HVJtc/S7ZgZVCP0pWpV0mIzETapw8YVt9UXvhFcqzz2gPpwOw1WmNFnCf4g dVDFog52QsOYfPuDLylFnFpozahODN0evNVsNFRpIS+GMRiwo1P0FqwDAlDm96Z0Dm+4kbLl83e fO3a5htW31qR/ZF81sqdwX0RpHCA4Fjo04I7ktUyYByBE9fEd9JyZNJqxOeBmaTdc7uuBTM= X-Google-Smtp-Source: AGHT+IGu+sIxewoMnakQwFrd+WtpFgPW8oYU700bgvVtqjA/1ds+luR965SVMGpBuRSl7mKdcytpiA== X-Received: by 2002:a05:600c:3554:b0:441:c5ee:cdb4 with SMTP id 5b1f17b1804b1-441d44dd442mr6653345e9.4.1746606477201; Wed, 07 May 2025 01:27:57 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-441d440fff6sm22182575e9.25.2025.05.07.01.27.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 May 2025 01:27:56 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [RFC PATCH 1/6] libcamera: framebuffer: Add FrameMetadata::Status::FrameStartup Date: Wed, 7 May 2025 08:52:08 +0100 Message-ID: <20250507082753.2306743-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250507082753.2306743-1-naush@raspberrypi.com> References: <20250507082753.2306743-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new status enum, FrameStartup, used to denote that even though the frame has been successfully captured, the IQ parameters set by the IPA will cause the frame to be unusable and applications are advised to not consume this frame. An example of this would be on a cold-start of the 3A algorithms, and there will be large oscillations to converge to a stable state quickly. Additional, update the definition of the FrameError state to include its usage when the sensor is known to produce a number of invalid/error frames after stream-on. Signed-off-by: Naushir Patuck --- include/libcamera/framebuffer.h | 1 + src/libcamera/framebuffer.cpp | 11 +++++++++-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/include/libcamera/framebuffer.h b/include/libcamera/framebuffer.h index ff83924300ac..e83825b466aa 100644 --- a/include/libcamera/framebuffer.h +++ b/include/libcamera/framebuffer.h @@ -26,6 +26,7 @@ struct FrameMetadata { FrameSuccess, FrameError, FrameCancelled, + FrameStartup, }; struct Plane { diff --git a/src/libcamera/framebuffer.cpp b/src/libcamera/framebuffer.cpp index 826848f75a56..36e56d593bc3 100644 --- a/src/libcamera/framebuffer.cpp +++ b/src/libcamera/framebuffer.cpp @@ -44,11 +44,18 @@ LOG_DEFINE_CATEGORY(Buffer) * of the FrameMetadata structure are valid. * \var FrameMetadata::FrameError * An error occurred during capture of the frame. The frame data may be partly - * or fully invalid. The sequence and timestamp fields of the FrameMetadata - * structure is valid, the other fields may be invalid. + * or fully invalid. This status may also indicate an invalid frame produced by + * the sensor during its startup or restart phase. The sequence and timestamp + * fields of the FrameMetadata structure is valid, the other fields may be + * invalid. * \var FrameMetadata::FrameCancelled * Capture stopped before the frame completed. The frame data is not valid. All * fields of the FrameMetadata structure but the status field are invalid. + * \var FrameMetadata::FrameStartup The frame has been successfully captured. + * However, the IPA is in a cold-start or reset phase and will result in image + * quality parameters producing unusable images. Applications are recommended to + * not consume these frames. All fields of the FrameMetadata structure are + * valid. */ /** From patchwork Wed May 7 07:52:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23343 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 92DEAC3226 for ; Wed, 7 May 2025 08:28:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BDBB668B30; Wed, 7 May 2025 10:28:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="gLOn5J1f"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1573268B2F for ; Wed, 7 May 2025 10:27:59 +0200 (CEST) Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-43ceeb85ab2so6124365e9.0 for ; Wed, 07 May 2025 01:27:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1746606478; x=1747211278; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fnaMAqT31Sn1HyYeUSz/LRhNsF9NQEXQj8bpQHhMNn0=; b=gLOn5J1fthRI2lpvsaWKenLBHcmUtBoLoMkAAp61xt95xnt45YVorb/M+5j6LJhmtO +HHPVZxtuJFPJQy+bYMFb4+SPGs9TGUSiRvpZu1n6aRhyNljXhGIssN4621hSodjikDF BuTQSYgMs2tlL+R3igOGPjD21XNQa/6qiwR/8mEYENGg6Q0E1D+iVVK5qpJOM8+NjFHz GZiOSVJlmhoFa/WUJoWkX6azDOMpYhVL6V6DW86ILWgvW7QNmcqy0xDzgXu3nkqPzXbE HB58mZiV2p2oBbT+xxWaavy9HMg5c7Vo8V5EKQ8ks8sV4HghCPBnMH/teaMh6Iv5UshQ qAWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746606478; x=1747211278; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fnaMAqT31Sn1HyYeUSz/LRhNsF9NQEXQj8bpQHhMNn0=; b=vUX/eXctppgSJdjFZYAFz8lipTcWhq8bXXXeHSpp2pGwbVFT1LOwyZjAxm7NuXmoAX T1LTiybAS+LZ87R91qPuO8VbEMH/N5Kbf5a7iQ42cqaAWtC8Koo2RBcHDoh2F6wC1Y8O pZJCxangL9KIeYjqy2ImM4G2n9WR03GuR6FVWRSXa+/0dvKIDp9j9fgCVOnA1vWErwCA 4ZAjbvH9Wq2OuZaIZu6LtPPCrVRc74UgtsQzbCQDFuuaD3wzCgBQnrDWKDPjpXyAn93V TGwplPcl+014PLMY8+TKAJfeGoYfA4uN0DDUEJtXzbtLz8PhtItLFglTnvpeP4KWE9W9 P5lg== X-Gm-Message-State: AOJu0Ywx54ZSM0Z5Y1xfChVEmFXJS+02UfiJGjIlf7HsmL0JHCcoRXVI 0IO0Ki9TRINwibwvoueBdzNu8nA8h+YsVPIKzEmj6qfVHw0mVo6j7UMr/WP3SfCnp1yAVWEi6TY L X-Gm-Gg: ASbGncuRC4qfcyCY3xKDrMyJBajVsq0Z6IBhTZmyefBJeIy/zTFuzEh2eYBWMgHF7mm w7ueyCgtbr5B3hZ7jLL4wgJU/ulfCC75dkGl8PZLQMULH2Cw929kVreHICR3S8QJj5ij5TidBt1 HiT02TqKYIv0ngmO2MVo69t1F51sirQW3lU8Nq6k4OrnNCN5tPRMA0gd2yWI4t5Uyelu7UmAlg1 D42veTj2C5QSpfvAFK8Wgtb9TPaAifLXDGa7D5hlraznlgNh4/pdyFInAthsOUbg0gksgc5i2Ny +S6QfwlyWFtWTvbNC+uwQ80cuGoIJ8Npv3VNdI3MQ67PVaXsnz4zOHYvW2MI X-Google-Smtp-Source: AGHT+IHi1q/l/kinKA98ZH8w+3vOHua7qQcRiDkumRVPaE0luc9b38F00rrDALa47ysEezWZQIyhRA== X-Received: by 2002:a05:600c:1c8c:b0:441:b397:e324 with SMTP id 5b1f17b1804b1-441d44ed543mr5482525e9.9.1746606478460; Wed, 07 May 2025 01:27:58 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-441d440fff6sm22182575e9.25.2025.05.07.01.27.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 May 2025 01:27:58 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [RFC PATCH 2/6] ipa: rpi: Replace dropFrameCount in the IPA -> PH interface Date: Wed, 7 May 2025 08:52:09 +0100 Message-ID: <20250507082753.2306743-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250507082753.2306743-1-naush@raspberrypi.com> References: <20250507082753.2306743-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Replace the dropFrameCount parameter returned from ipa::start() to the pipeline handler by startupFrameCount and invalidFrameCount. The former counts the number of frames required for AWB/AGC to converge, and the latter counts the number of invalid frames produced by the sensor when starting up. In the pipeline handler, use the sum of these 2 values to replicate the existing dropFrameCount behaviour. Signed-off-by: Naushir Patuck --- include/libcamera/ipa/raspberrypi.mojom | 3 ++- src/ipa/rpi/common/ipa_base.cpp | 14 ++++++++------ .../pipeline/rpi/common/pipeline_base.cpp | 4 ++-- 3 files changed, 12 insertions(+), 9 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index e30c70bde08b..12b083e9d04b 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -52,7 +52,8 @@ struct ConfigResult { struct StartResult { libcamera.ControlList controls; - int32 dropFrameCount; + int32 startupFrameCount; + int32 invalidFrameCount; }; struct PrepareParams { diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index e0a93daa9db2..c15f8a7bf71e 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -324,6 +324,7 @@ void IpaBase::start(const ControlList &controls, StartResult *result) * "mistrusted", which depends on whether this is a startup from cold, * or merely a mode switch in a running system. */ + unsigned int agcConvergenceFrames = 0, awbConvergenceFrames = 0; frameCount_ = 0; if (firstStart_) { dropFrameCount_ = helper_->hideFramesStartup(); @@ -336,7 +337,6 @@ void IpaBase::start(const ControlList &controls, StartResult *result) * (mistrustCount_) that they won't see. But if zero (i.e. * no convergence necessary), no frames need to be dropped. */ - unsigned int agcConvergenceFrames = 0; RPiController::AgcAlgorithm *agc = dynamic_cast( controller_.getAlgorithm("agc")); if (agc) { @@ -345,7 +345,6 @@ void IpaBase::start(const ControlList &controls, StartResult *result) agcConvergenceFrames += mistrustCount_; } - unsigned int awbConvergenceFrames = 0; RPiController::AwbAlgorithm *awb = dynamic_cast( controller_.getAlgorithm("awb")); if (awb) { @@ -353,15 +352,18 @@ void IpaBase::start(const ControlList &controls, StartResult *result) if (awbConvergenceFrames) awbConvergenceFrames += mistrustCount_; } - - dropFrameCount_ = std::max({ dropFrameCount_, agcConvergenceFrames, awbConvergenceFrames }); - LOG(IPARPI, Debug) << "Drop " << dropFrameCount_ << " frames on startup"; } else { dropFrameCount_ = helper_->hideFramesModeSwitch(); mistrustCount_ = helper_->mistrustFramesModeSwitch(); } - result->dropFrameCount = dropFrameCount_; + result->startupFrameCount = std::max({ agcConvergenceFrames, awbConvergenceFrames }); + result->invalidFrameCount = dropFrameCount_; + + dropFrameCount_ = std::max({ dropFrameCount_, agcConvergenceFrames, awbConvergenceFrames }); + + LOG(IPARPI, Debug) << "Startup frames: " << result->startupFrameCount + << " Invalid frames: " << result->invalidFrameCount; firstStart_ = false; lastRunTimestamp_ = 0; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 1f13e5230fae..5956485953a2 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -660,8 +660,8 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) data->setSensorControls(result.controls); /* Configure the number of dropped frames required on startup. */ - data->dropFrameCount_ = data->config_.disableStartupFrameDrops - ? 0 : result.dropFrameCount; + data->dropFrameCount_ = data->config_.disableStartupFrameDrops ? + 0 : result.startupFrameCount + result.invalidFrameCount; for (auto const stream : data->streams_) stream->resetBuffers(); From patchwork Wed May 7 07:52:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23344 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4DEA6C3226 for ; Wed, 7 May 2025 08:28:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A270368B36; Wed, 7 May 2025 10:28:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="FObYVrtc"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 40EA968B30 for ; Wed, 7 May 2025 10:28:00 +0200 (CEST) Received: by mail-wr1-x429.google.com with SMTP id ffacd0b85a97d-39d73b97500so374091f8f.3 for ; Wed, 07 May 2025 01:28:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1746606479; x=1747211279; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=85vcpWZxa6mtEJPyiFwMvAsazIU0BVD2UXwGQZLD+8o=; b=FObYVrtcEFP/HnCR36dC2LysZVbcwhR8i5fRp/kpK93UqmEhev42u6fcmLfVDY0gDR k3+anq+kwl9WzhSqn9k4eqqwKkDwuhAxFLL7DSjgi+16FZkeOirO0J0Qjkfpn5B11Pgg VwjmWuSEtL16ITqY9VgN55tNtTeBs95CwE9wi0SkaM7ywzPsHWZpt1ymh+FQ+IpSnK5n M85KIMu+Jo0RzR2mby1EzHO2JumFReNwQaFlB/PoWN+zgIrKotgsUQNQIj5KlcmbUNXp bSVaGvWz0b+qw++K1eGGFGCQ67UaBBVwzWiFeSuk2rGI8pVx893mnkwd9zINZQTMBM01 YRcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746606479; x=1747211279; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=85vcpWZxa6mtEJPyiFwMvAsazIU0BVD2UXwGQZLD+8o=; b=AowHmf2hQUttsB61ohS+CvDhu1bR1yj4GwkTy0VyhXqoN3AcVTFLgD5zu0/4ChUSuP lq7LUKqN1Nxw04rfJiiIaKDspyM98ZI+GcTSJpk/aDlXyH5aiuW90sUeFddtHXPv29mU LSSZtLfy7WOSeRfqd8CPJB5GLn88O03SVecwAOBr7WTjgVTzml9y5TTSpKrSpgu9shQn cV2y+Wo+dyEdIBDJ4yfX38/qIfNxloVpBoRZdwINLW1NJUr73nVJj0a9QbPG9up3FHkC fBFbF+CB/lje3v3nR+xPeM0jpHSRONVq4goLvkrLotcJg751ke5wSY3RKCyYde7hsE5a V/ow== X-Gm-Message-State: AOJu0Yz4hrAvqCvlzC6tUIIhfqJlRBhAQOdDtN0/IAU6w0vnzzKjqSuZ bgHKHlt1ohZHufjO6K7bTC3NSO9NYEc+sGI5/ML/IsbDMhsrYh0ZSb+x8k9OfOordSjYEJ8eHCT S X-Gm-Gg: ASbGncs7qMAk4hwQwrVzeaSbG4maYklDqWM1kdes234EvwNbUGqj/y6ELj8StP8kTPV vKNUsl698CoxkURF7RZoQBNA94XAlsEN1w6wMpghcdQ8SrNa7d8d2FX+qUiXwtiohaWTXpIoLwn q5JSeGo3vZykeKZWWxRej3vOpFh5L2Ra4fJElyKBW6wna8hDpRgJEDzL4q0dbLLhgB47jH6S8kc wCu/YYbseq5seV57+f2N2mCMMMyMNSA6LsrDJkipV+wgbkXxxJ91S9IkdIN7jSEujDPI7jW0J0i 5Wm6BOviUn1n/Mtf6JqcHKN5Jjt1RjJnx6DZJN2OsdlquYVk9/ud/luNvVUb X-Google-Smtp-Source: AGHT+IFkly/sLzhr8DUSEzBSGmqUPAXtZLWb6diApNul9uJAcrq0/XAgZLu/sa3J01RUUj7XCXwxvA== X-Received: by 2002:a05:600c:4ecc:b0:439:9fde:da76 with SMTP id 5b1f17b1804b1-441d448cab2mr6312545e9.0.1746606479162; Wed, 07 May 2025 01:27:59 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-441d440fff6sm22182575e9.25.2025.05.07.01.27.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 May 2025 01:27:58 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [RFC PATCH 3/6] pipeline: ipa: rpi: Split RPiCameraData::dropFrameCount_ Date: Wed, 7 May 2025 08:52:10 +0100 Message-ID: <20250507082753.2306743-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250507082753.2306743-1-naush@raspberrypi.com> References: <20250507082753.2306743-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Split the pipeline handler drop frame tracking into startup frames and invalid frames, as reported by the IPA. Remove the drop buffer handling logic in the pipeline handler. Now all image buffers are returned out with the appropriate FrameStatus set for startup or invalid frames. Signed-off-by: Naushir Patuck --- .../pipeline/rpi/common/pipeline_base.cpp | 88 +++++++------------ .../pipeline/rpi/common/pipeline_base.h | 5 +- 2 files changed, 37 insertions(+), 56 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 5956485953a2..21f2daf5bab5 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -659,9 +659,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) if (!result.controls.empty()) data->setSensorControls(result.controls); - /* Configure the number of dropped frames required on startup. */ - data->dropFrameCount_ = data->config_.disableStartupFrameDrops ? - 0 : result.startupFrameCount + result.invalidFrameCount; + /* Configure the number of startup and invalid frames reported by the IPA. */ + data->startupFrameCount_ = result.startupFrameCount; + data->invalidFrameCount_ = result.invalidFrameCount; for (auto const stream : data->streams_) stream->resetBuffers(); @@ -678,7 +678,6 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) data->buffersAllocated_ = true; } - /* We need to set the dropFrameCount_ before queueing buffers. */ ret = queueAllBuffers(camera); if (ret) { LOG(RPI, Error) << "Failed to queue buffers"; @@ -898,23 +897,6 @@ int PipelineHandlerBase::queueAllBuffers(Camera *camera) ret = stream->queueAllBuffers(); if (ret < 0) return ret; - } else { - /* - * For external streams, we must queue up a set of internal - * buffers to handle the number of drop frames requested by - * the IPA. This is done by passing nullptr in queueBuffer(). - * - * The below queueBuffer() call will do nothing if there - * are not enough internal buffers allocated, but this will - * be handled by queuing the request for buffers in the - * RPiStream object. - */ - unsigned int i; - for (i = 0; i < data->dropFrameCount_; i++) { - ret = stream->queueBuffer(nullptr); - if (ret) - return ret; - } } } @@ -1412,7 +1394,15 @@ void CameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::Stream *stream) * buffer back to the stream. */ Request *request = requestQueue_.empty() ? nullptr : requestQueue_.front(); - if (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) { + if (request && request->findBuffer(stream) == buffer) { + FrameMetadata &md = buffer->_d()->metadata(); + + /* Mark the non-converged and invalid frames in the metadata. */ + if (startupFrameCount_) + md.status = FrameMetadata::Status::FrameStartup; + if (invalidFrameCount_) + md.status = FrameMetadata::Status::FrameError; + /* * Tag the buffer as completed, returning it to the * application. @@ -1458,42 +1448,32 @@ void CameraData::handleState() void CameraData::checkRequestCompleted() { - bool requestCompleted = false; - /* - * If we are dropping this frame, do not touch the request, simply - * change the state to IDLE when ready. - */ - if (!dropFrameCount_) { - Request *request = requestQueue_.front(); - if (request->hasPendingBuffers()) - return; + Request *request = requestQueue_.front(); + if (request->hasPendingBuffers()) + return; - /* Must wait for metadata to be filled in before completing. */ - if (state_ != State::IpaComplete) - return; + /* Must wait for metadata to be filled in before completing. */ + if (state_ != State::IpaComplete) + return; - LOG(RPI, Debug) << "Completing request sequence: " - << request->sequence(); + LOG(RPI, Debug) << "Completing request sequence: " + << request->sequence(); - pipe()->completeRequest(request); - requestQueue_.pop(); - requestCompleted = true; - } + pipe()->completeRequest(request); + requestQueue_.pop(); - /* - * Make sure we have three outputs completed in the case of a dropped - * frame. - */ - if (state_ == State::IpaComplete && - ((ispOutputCount_ == ispOutputTotal_ && dropFrameCount_) || - requestCompleted)) { - LOG(RPI, Debug) << "Going into Idle state"; - state_ = State::Idle; - if (dropFrameCount_) { - dropFrameCount_--; - LOG(RPI, Debug) << "Dropping frame at the request of the IPA (" - << dropFrameCount_ << " left)"; - } + LOG(RPI, Debug) << "Going into Idle state"; + state_ = State::Idle; + + if (startupFrameCount_) { + startupFrameCount_--; + LOG(RPI, Debug) << "Decrementing startup frames to " + << startupFrameCount_; + } + if (invalidFrameCount_) { + invalidFrameCount_--; + LOG(RPI, Debug) << "Decrementing invalid frames to " + << invalidFrameCount_; } } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index aae0c2f35888..6023f9f9d6b3 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -48,7 +48,7 @@ class CameraData : public Camera::Private public: CameraData(PipelineHandler *pipe) : Camera::Private(pipe), state_(State::Stopped), - dropFrameCount_(0), buffersAllocated_(false), + startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false), ispOutputCount_(0), ispOutputTotal_(0) { } @@ -151,7 +151,8 @@ public: /* Mapping of CropParams keyed by the output stream order in CameraConfiguration */ std::map cropParams_; - unsigned int dropFrameCount_; + unsigned int startupFrameCount_; + unsigned int invalidFrameCount_; /* * If set, this stores the value that represets a gain of one for From patchwork Wed May 7 07:52:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23345 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 501B4C3226 for ; Wed, 7 May 2025 08:28:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7262168B37; Wed, 7 May 2025 10:28:12 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="amtIQMeV"; dkim-atps=neutral Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3C91C68B31 for ; Wed, 7 May 2025 10:28:01 +0200 (CEST) Received: by mail-wr1-x434.google.com with SMTP id ffacd0b85a97d-39c2688e2bbso438510f8f.1 for ; Wed, 07 May 2025 01:28:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1746606480; x=1747211280; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TDFSDhxsWtnuegD2eNp0so8bkMmcVepBWj9qp8gGI+4=; b=amtIQMeVvuX80CKpubGGo0fDVcTkWMDGXHPljK0ztot5vjpGY7aZ2PoiUYSvg13OfX I6oBGQ8jVDWUoGLJ8i32ml6osJKK/xlNwzGPHrvZmUW6RKdMRI460kRhtzgWgN9Sfb9D 3OvRvcER/Ib8BLffiLnpX1HNXuh1xiRe/NkwLjrbdlIEmO9z49yuytMeSKtbCjG4Y88Q 5CFxmQ+Z2kxLY91mk2ulTemKvtOS/6T4dqTfQUqpK3+PUitZQFboXMTHFBEGwMw7owwU 9cVDSr4ljKGIs4tROD0WcFnyaXpaXGLOMp5VoMXr2QI/8qLCZ8Yfs3H2gRFByp1MH02w kaMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746606480; x=1747211280; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TDFSDhxsWtnuegD2eNp0so8bkMmcVepBWj9qp8gGI+4=; b=X7bkpQMoyiQspYh1QU7QbBa1565mXsks6kTRAsUKFbfRPhPcqJ7BXD2NT76ysqP7WN AC50stQy/+zC5xdrrgT07cUN5k41I1MHfsGz1FwPFYvU6rD2/SQoiUzA9zJxA1G8zE2m iqVqFu80ims+oV+gDrCBrYI61fRh3zfsb6P02MlUcMn9ssn6UnEI9d99XvVcyuO7Pg/U Yoxec+bebx/qnetpKEo+kEJ0+bA4/pLDDeeb49VTgBpFG8aTsdfWpzdWhIlHkmzjAcrM 1nbwtxIP+WUUk9Tr4LixbJo0EaAerSXO8JcI47oorTj83a2QLL/SqGb1h6Lkrg6ah9E4 0IwQ== X-Gm-Message-State: AOJu0YwUMa5YlIM3mGP5E4xsgwXJgoLPnq/dmwEREVyaWw6fdhz+khoR SZXgEBYZ7AHTjDXK3MDyXBxei9N2NNsnwuzsEv06d1Ek8sLmqLLLwofi5NmeygHdAPapNduhCXj X X-Gm-Gg: ASbGncvQmAr5FWn1+VgZKSciRooAgbK3ByHphZjfAUFNW/BkncsPl8mrj1DYm8731MM x7xGM83LEfiFFDYfKur1uu4ck21XkUuQBXnwtKdxCG6bJ7EQcukU/pWGeTPrDCVEnyT1/WVxoqP 6o/3Bjvf60gCz9NMC3PDDTRoYSFso7dlXL6u95GmSdnccu+edEyLaoLrc4FhXeiU9ky62nTvrsm 2byxuqO9K7ahq3OdJlqVvlkCaIkIfxIQxKF9/A7o+v7wQYQxfydgJLhD3Tu5uCCg4aM6YIGomBj +6FMo1rahRJSpUg0QV4xFioV2qb4bprqSd4cNneM1TCDtls3gO922qq95l+K7DiuwJdix28= X-Google-Smtp-Source: AGHT+IEKiGSxB3ET0jnS5ncxHDzlHfdU4DLEQyyWAmJ9QL0a2sqoBWeIQhyfEWi6BdBy9kUwrm44mw== X-Received: by 2002:a05:6000:1acc:b0:3a0:87f6:5c7a with SMTP id ffacd0b85a97d-3a0b4a17181mr619387f8f.3.1746606479946; Wed, 07 May 2025 01:27:59 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-441d440fff6sm22182575e9.25.2025.05.07.01.27.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 May 2025 01:27:59 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [RFC PATCH 4/6] pipeline: rpi: Remove disable_startup_frame_drops config option Date: Wed, 7 May 2025 08:52:11 +0100 Message-ID: <20250507082753.2306743-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250507082753.2306743-1-naush@raspberrypi.com> References: <20250507082753.2306743-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the previous change to not drop frames in the pipeline handler, the "disable_startup_frame_drops" pipeline config option is not used. Remove it, and throw a warning if the option is present in the YAML config file. Signed-off-by: Naushir Patuck --- src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 7 ++++--- src/libcamera/pipeline/rpi/common/pipeline_base.h | 5 ----- src/libcamera/pipeline/rpi/pisp/data/example.yaml | 5 ----- src/libcamera/pipeline/rpi/vc4/data/example.yaml | 5 ----- 4 files changed, 4 insertions(+), 18 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 21f2daf5bab5..4afdaed5935d 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1077,7 +1077,6 @@ void CameraData::enumerateVideoDevices(MediaLink *link, const std::string &front int CameraData::loadPipelineConfiguration() { config_ = { - .disableStartupFrameDrops = false, .cameraTimeoutValue = 0, }; @@ -1114,8 +1113,10 @@ int CameraData::loadPipelineConfiguration() const YamlObject &phConfig = (*root)["pipeline_handler"]; - config_.disableStartupFrameDrops = - phConfig["disable_startup_frame_drops"].get(config_.disableStartupFrameDrops); + if (phConfig.contains("disable_startup_frame_drops")) + LOG(RPI, Warning) + << "The disable_startup_frame_drops key is now deprecated, " + << "please use FrameMetadata::Status::FrameStartup instead."; config_.cameraTimeoutValue = phConfig["camera_timeout_value_ms"].get(config_.cameraTimeoutValue); diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 6023f9f9d6b3..e27c4f860d1a 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -164,11 +164,6 @@ public: bool buffersAllocated_; struct Config { - /* - * Override any request from the IPA to drop a number of startup - * frames. - */ - bool disableStartupFrameDrops; /* * Override the camera timeout value calculated by the IPA based * on frame durations. diff --git a/src/libcamera/pipeline/rpi/pisp/data/example.yaml b/src/libcamera/pipeline/rpi/pisp/data/example.yaml index d67e654a8b9e..baf03be79bb3 100644 --- a/src/libcamera/pipeline/rpi/pisp/data/example.yaml +++ b/src/libcamera/pipeline/rpi/pisp/data/example.yaml @@ -16,11 +16,6 @@ # # "num_cfe_config_queue": 2, - # Override any request from the IPA to drop a number of startup - # frames. - # - # "disable_startup_frame_drops": false, - # Custom timeout value (in ms) for camera to use. This overrides # the value computed by the pipeline handler based on frame # durations. diff --git a/src/libcamera/pipeline/rpi/vc4/data/example.yaml b/src/libcamera/pipeline/rpi/vc4/data/example.yaml index b8e01adeaf40..27e543488d48 100644 --- a/src/libcamera/pipeline/rpi/vc4/data/example.yaml +++ b/src/libcamera/pipeline/rpi/vc4/data/example.yaml @@ -29,11 +29,6 @@ # # "min_total_unicam_buffers": 4, - # Override any request from the IPA to drop a number of startup - # frames. - # - # "disable_startup_frame_drops": false, - # Custom timeout value (in ms) for camera to use. This overrides # the value computed by the pipeline handler based on frame # durations. From patchwork Wed May 7 07:52:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23346 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6132BC3226 for ; Wed, 7 May 2025 08:28:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4E77768B34; Wed, 7 May 2025 10:28:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="m8bRhiI7"; dkim-atps=neutral Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8B7A768B33 for ; Wed, 7 May 2025 10:28:01 +0200 (CEST) Received: by mail-wr1-x431.google.com with SMTP id ffacd0b85a97d-3a0b0dd3b5bso161521f8f.1 for ; Wed, 07 May 2025 01:28:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1746606481; x=1747211281; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HyZHAEPEvRlhx9HxiH7c+Sa84pjnUrsNuklV5311nY4=; b=m8bRhiI7VHLdkVmYaGltNOAzXLNhftjO5FNcHyDA4DsLNAOG1erw4y7NjRLeXAkFGN hUfkWK5W/kUEpRY0DL/Vxs6cxamXT8dgch7ebMbFzpS/xEypf5mmMacUOIqJPKKVGoUe w0xaYUIHE1QOJNBXfPKAsI9FFGcBk1IGY0Lhug1zAnQsZcaYCZgMgOC4jBSw5qiWATkj creDKIz/m0Kz2IoeLZbp2n4NO2ryO0hAJGYeaHg+K3yRd3qkUt6YjRHR6uKAEHMnp+A6 CfgYRmyUGQJFtaYnz22N3pv17sabPNyp5jD7W4JRaLOQn/4v2YkJrPXbjWfncExulsWw 8Kkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746606481; x=1747211281; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HyZHAEPEvRlhx9HxiH7c+Sa84pjnUrsNuklV5311nY4=; b=A77v3m4JVBQDG3YxkkL8pkeqg0KlF07KS0CxE6I6MSemeQUN/al7EjSMc5t/YRk+WI o7kzZRgQuml8vrQe0LoTPCB7cdSoMMwXMZjP1ZH0cYF/lbJH/5lm4obHvRRBxO7Q9sLk das5Eegp2zn2gEkCCFNIx91WckGdmDDimSlEz38U2v5uyD2vZdO6/ehw6Ql46JnJ2RXC CL/oJkyQgjNgmEK45eGn66zwLtfP8QvbCuMAF2bnOE9Cb/oFbWFgXAM/at6lqidTLYcz DSgjJqFkw+MUrK/gX736zAz3fZ2WA/EglWLSVclbEltpg1V47BPK2xNKO1wrRc12YE5W 48pw== X-Gm-Message-State: AOJu0Yz2YgjNqszihTc+E4zkGwVbTkNWocMk51nT94GPCv5pyL5pKZqv dKFWP2CGaPOicE1Yq42XizK0eEo+4OQg6LrRKnoX9iIZGCaccdBeWjByIOqdaH9LfgE3NHJYw9U I X-Gm-Gg: ASbGncuKEDYHGrY+hh6fNU3QtDJXs51HHYE/I+d2vvC6LHdZtg23j8NLgSvxz+CXstS bNqcA3aeYWhHhQ6+gVXTHpfyL2LD3b6X+SPTkFwZvNFW521heUo82s1sfnpPWTtlVYG0Qbyx2Ur PzDzJ9UUSFmR7mUdfzV8nBIf5Zm9yJ1KUF45FsGRMGpMFE8sl9kyibH2Ge3levmzqZxh+HLX5Ud 2OZDjDdKDz85g57ecgF7qHLPW58jVymkzrWFt/ZkX7xY1+g1qQiMoX5ANwZX4zYgcxv2qRQ6bVI Px7m8cgqgxuhOwpSWKLEKol9hCgB6zxagKqEJK2nhWRUpRCCeez0UvvexH6V X-Google-Smtp-Source: AGHT+IGr21+tSsVRDiZfnP+i3RPITy8FkNZSO/gplnSdPsan+gGNn9X3G630ixx1It6KluUfyt+SZQ== X-Received: by 2002:a05:600c:3d9b:b0:439:9a1f:d73d with SMTP id 5b1f17b1804b1-441d44ecc08mr5683615e9.8.1746606480774; Wed, 07 May 2025 01:28:00 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-441d440fff6sm22182575e9.25.2025.05.07.01.28.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 May 2025 01:28:00 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [RFC PATCH 5/6] pipeline: rpi: Remove ispOutputCount_ and ispOutputTotal_ Date: Wed, 7 May 2025 08:52:12 +0100 Message-ID: <20250507082753.2306743-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250507082753.2306743-1-naush@raspberrypi.com> References: <20250507082753.2306743-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the drop frame logic removed from the pipeline handler, these member variables and not used, so remove them. Signed-off-by: Naushir Patuck --- src/libcamera/pipeline/rpi/common/pipeline_base.h | 6 +----- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 12 ------------ src/libcamera/pipeline/rpi/vc4/vc4.cpp | 15 --------------- 3 files changed, 1 insertion(+), 32 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index e27c4f860d1a..898f31577059 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -48,8 +48,7 @@ class CameraData : public Camera::Private public: CameraData(PipelineHandler *pipe) : Camera::Private(pipe), state_(State::Stopped), - startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false), - ispOutputCount_(0), ispOutputTotal_(0) + startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false) { } @@ -179,9 +178,6 @@ protected: virtual void tryRunPipeline() = 0; - unsigned int ispOutputCount_; - unsigned int ispOutputTotal_; - private: void checkRequestCompleted(); }; diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index 91e7f4c94d96..ccf135c3d8ce 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -1834,12 +1834,6 @@ void PiSPCameraData::beOutputDequeue(FrameBuffer *buffer) dmabufSyncEnd(buffer->planes()[0].fd); handleStreamBuffer(buffer, stream); - - /* - * Increment the number of ISP outputs generated. - * This is needed to track dropped frames. - */ - ispOutputCount_++; handleState(); } @@ -1885,7 +1879,6 @@ void PiSPCameraData::prepareIspComplete(const ipa::RPi::BufferIds &buffers, bool * If there is no need to run the Backend, just signal that the * input buffer is completed and all Backend outputs are ready. */ - ispOutputCount_ = ispOutputTotal_; buffer = cfe_[Cfe::Output0].getBuffers().at(bayerId).buffer; handleStreamBuffer(buffer, &cfe_[Cfe::Output0]); } else @@ -1994,7 +1987,6 @@ int PiSPCameraData::configureBe(const std::optional &yuvColorSpace) global.bayer_enables |= PISP_BE_BAYER_ENABLE_INPUT; global.bayer_order = toPiSPBayerOrder(cfeFormat.fourcc); - ispOutputTotal_ = 1; /* Config buffer */ if (PISP_IMAGE_FORMAT_COMPRESSED(inputFormat.format)) { pisp_decompress_config decompress; decompress.offset = DefaultCompressionOffset; @@ -2025,7 +2017,6 @@ int PiSPCameraData::configureBe(const std::optional &yuvColorSpace) setupOutputClipping(ispFormat0, outputFormat0); be_->SetOutputFormat(0, outputFormat0); - ispOutputTotal_++; } if (global.rgb_enables & PISP_BE_RGB_ENABLE_OUTPUT1) { @@ -2049,7 +2040,6 @@ int PiSPCameraData::configureBe(const std::optional &yuvColorSpace) setupOutputClipping(ispFormat1, outputFormat1); be_->SetOutputFormat(1, outputFormat1); - ispOutputTotal_++; } /* Setup the TDN I/O blocks in case TDN gets turned on later. */ @@ -2256,8 +2246,6 @@ void PiSPCameraData::prepareCfe() void PiSPCameraData::prepareBe(uint32_t bufferId, bool stitchSwapBuffers) { - ispOutputCount_ = 0; - FrameBuffer *buffer = cfe_[Cfe::Output0].getBuffers().at(bufferId).buffer; LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << bufferId diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index fe910bdf2ff9..ac6dab814d35 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -597,8 +597,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi stream->setFlags(StreamFlag::External); } - ispOutputTotal_ = outStreams.size(); - /* * If ISP::Output0 stream has not been configured by the application, * we must allow the hardware to generate an output so that the data @@ -625,8 +623,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi return -EINVAL; } - ispOutputTotal_++; - LOG(RPI, Debug) << "Defaulting ISP Output0 format to " << format; } @@ -662,8 +658,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi << ret; return -EINVAL; } - - ispOutputTotal_++; } /* ISP statistics output format. */ @@ -676,8 +670,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi return ret; } - ispOutputTotal_++; - /* * Configure the Unicam embedded data output format only if the sensor * supports it. @@ -843,12 +835,6 @@ void Vc4CameraData::ispOutputDequeue(FrameBuffer *buffer) handleStreamBuffer(buffer, stream); } - /* - * Increment the number of ISP outputs generated. - * This is needed to track dropped frames. - */ - ispOutputCount_++; - handleState(); } @@ -880,7 +866,6 @@ void Vc4CameraData::prepareIspComplete(const ipa::RPi::BufferIds &buffers, << ", timestamp: " << buffer->metadata().timestamp; isp_[Isp::Input].queueBuffer(buffer); - ispOutputCount_ = 0; if (sensorMetadata_ && embeddedId) { buffer = unicam_[Unicam::Embedded].getBuffers().at(embeddedId & RPi::MaskID).buffer; From patchwork Wed May 7 07:52:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23347 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 701EBC327D for ; Wed, 7 May 2025 08:28:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DD80568B3B; Wed, 7 May 2025 10:28:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="K4ryHFui"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6337668B34 for ; Wed, 7 May 2025 10:28:02 +0200 (CEST) Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-440668acbf3so8483225e9.0 for ; Wed, 07 May 2025 01:28:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1746606482; x=1747211282; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iDJm16zb7hwM/O4mxusEMEmz8cHY3wVG5puaLJGJwFQ=; b=K4ryHFuisiiMZy6EuiUVlNmeSuDhJ1WLekK+Jbs+PQGXOwJhR+2xXYS8H/BL9OYgXa KsUt5TS4BLwGY9PJcV8nYvLrCYN1TVurRD1aUH4DL/rm8OmtkOtxalYBXQq7wpG6gn+A LR95fWeVsy4xB/XlsY/rqEJLYGiLaJrcNBFFM1V7uoHw0T3XwrhC2/BJfHSJUq4CgtOj 3RHanH7cheF7+OBxuVd1ww90fouwv99/V+utsaN2009DpEUq98ayCRsfQcxNnqGnmxgU eorhXf7h/DmOW1Qre421EQSt6QB58E5LfPOGMIv6n8JJ+mr9+Skvn06qNi2BXOjj+qqT xsgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746606482; x=1747211282; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iDJm16zb7hwM/O4mxusEMEmz8cHY3wVG5puaLJGJwFQ=; b=f8JK78kK3X8RKMfc9veTgsHjaf89JEZWEYNy82WM+OorjdlhoeApJ0eBeV1iMPtwgF kxnYiooGbpWkN/UvqlzvPKvWolRoAJxfQFwHe+XL+L7FikvFQMBrFxJwKtajXtMJN0h3 QqwZvBoT4wxlb4KO37woS+4LABAvZZgmGPIqTsAPHG836ZzGasF9zRqleAaUrEzJHxBs fruyK1+SxnURHXXLwcFbyhPjB6nhSainih5JtKh+JXQltuATm8HzPNPqJK1VAP0YIdBk OnI4E03TU9CWWVnGy5bJZmJ1JgvPHFLRqVFZ/abWOMQcORyWqSgpAc+Wq4L/kGl9/cZX bNMw== X-Gm-Message-State: AOJu0YzGzy7YMi0Tfay7k/94/oWg8BE8wOjivSf64xLqp6lcmJs7rUPS OqpMEuWtHpoFcsCrxgsQG6ArBa862s7horcQ+iHjaSkkvHDu9ZzjiVMUDjaCZ7A0Lq2v4rCft6t f X-Gm-Gg: ASbGncuCMKAzjWisjKApaF6O234RJXTFm9kAfyJVhpiErP2RFZ6yVAya4th05vTi9AO iRkxnaP43s+HbmTS0F3UwWT2WqkgSl/O7ApGxKPR86YhWgkkrWpZjQPmymnggFb60wA04+frlBY xMZM2UCWksdOQ22yFrsGJwPXnv3+yPzUXW+SWUsrcQONYpRbDAgCXZI8EdJgN3AnPCMm7Yi5r/q 6J/j7jFaqirq5zsycLNXgXk8ikpk2RnqAsu8CxrqbwxvKmYmzMuZe0SYqPLZzl6E/TI8kqJf7Bz IvZfWHJ0A/NEIijkytRcOFzSpweR2O3OgL9/n6QoACjMRlxZeF4xEr1CBQxRhw6TDdL2SfQ= X-Google-Smtp-Source: AGHT+IGNP/Tfnn44LQ7KO3cu0qjf7TlS2Nhom2UURYYxtDwAQcfLi+9twHZa/ASzeGrz1Ajpbedhpg== X-Received: by 2002:a05:600c:3b9d:b0:43b:c228:1ee0 with SMTP id 5b1f17b1804b1-441d44e0813mr5925655e9.7.1746606481679; Wed, 07 May 2025 01:28:01 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-441d440fff6sm22182575e9.25.2025.05.07.01.28.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 May 2025 01:28:01 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [RFC PATCH 6/6] ipa: rpi: Rename dropFrameCount_ to startupCount_ Date: Wed, 7 May 2025 08:52:13 +0100 Message-ID: <20250507082753.2306743-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250507082753.2306743-1-naush@raspberrypi.com> References: <20250507082753.2306743-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rename dropFrameCount_ to startupCount_ to better reflect its use as frames are no longer dropped by the pipeline handler. Signed-off-by: Naushir Patuck --- src/ipa/rpi/common/ipa_base.cpp | 10 +++++----- src/ipa/rpi/common/ipa_base.h | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index c15f8a7bf71e..8d591faeceaa 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -327,7 +327,7 @@ void IpaBase::start(const ControlList &controls, StartResult *result) unsigned int agcConvergenceFrames = 0, awbConvergenceFrames = 0; frameCount_ = 0; if (firstStart_) { - dropFrameCount_ = helper_->hideFramesStartup(); + startupCount_ = helper_->hideFramesStartup(); mistrustCount_ = helper_->mistrustFramesStartup(); /* @@ -353,14 +353,14 @@ void IpaBase::start(const ControlList &controls, StartResult *result) awbConvergenceFrames += mistrustCount_; } } else { - dropFrameCount_ = helper_->hideFramesModeSwitch(); + startupCount_ = helper_->hideFramesModeSwitch(); mistrustCount_ = helper_->mistrustFramesModeSwitch(); } result->startupFrameCount = std::max({ agcConvergenceFrames, awbConvergenceFrames }); - result->invalidFrameCount = dropFrameCount_; + result->invalidFrameCount = startupCount_; - dropFrameCount_ = std::max({ dropFrameCount_, agcConvergenceFrames, awbConvergenceFrames }); + startupCount_ = std::max({ startupCount_, agcConvergenceFrames, awbConvergenceFrames }); LOG(IPARPI, Debug) << "Startup frames: " << result->startupFrameCount << " Invalid frames: " << result->invalidFrameCount; @@ -443,7 +443,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) /* Allow a 10% margin on the comparison below. */ Duration delta = (frameTimestamp - lastRunTimestamp_) * 1.0ns; - if (lastRunTimestamp_ && frameCount_ > dropFrameCount_ && + if (lastRunTimestamp_ && frameCount_ > startupCount_ && delta < controllerMinFrameDuration * 0.9 && !hdrChange) { /* * Ensure we merge the previous frame's metadata with the current diff --git a/src/ipa/rpi/common/ipa_base.h b/src/ipa/rpi/common/ipa_base.h index 1a811beb31f2..a51afc156a8f 100644 --- a/src/ipa/rpi/common/ipa_base.h +++ b/src/ipa/rpi/common/ipa_base.h @@ -115,8 +115,8 @@ private: /* How many frames we should avoid running control algos on. */ unsigned int mistrustCount_; - /* Number of frames that need to be dropped on startup. */ - unsigned int dropFrameCount_; + /* Number of frames that need to be marked as dropped on startup. */ + unsigned int startupCount_; /* Frame timestamp for the last run of the controller. */ uint64_t lastRunTimestamp_;