From patchwork Fri Jun 6 10:55:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23475 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C1ECBC31E9 for ; Fri, 6 Jun 2025 10:57:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3B7EC68DBD; Fri, 6 Jun 2025 12:56:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="kEvJafhd"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5673868DB3 for ; Fri, 6 Jun 2025 12:56:55 +0200 (CEST) Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-450cfb8b335so1660795e9.2 for ; Fri, 06 Jun 2025 03:56:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1749207415; x=1749812215; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=a2HaDOVW+9Aps8YprLiWRQYgcbe5COfEJ4ggym7ZoOg=; b=kEvJafhdRiQf7FQuchsE+Me+zLR+Or6Vc0Rt+fCiZYRbDVsxKcI5r02xVAI/uSX5vZ U/RE0Hf+qyMCLhcCp+jXXpoiXdrP2ahOd3tdkXy/MSW/rxr39RHNla2vON4KD5NeZ0Hv x1L/1EUvBWEb+GDXnfEalvJlFI7nTnMeTN7Vm7iSGhw9BFWLaR9YuxuZGPetYewQaFlx p4IjRUc1GAL+0Kiauifi/2aolLbqu1OwifJ5CEvHfzRq2M8WKXSFPuITUAaR62X/vt8o nU9tAOwqGzCpMyOuzIcTIL8d9zIw7GmCCbVCVGCfqfatfro+NK5z1so0O8xue9sokcZo bIHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749207415; x=1749812215; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a2HaDOVW+9Aps8YprLiWRQYgcbe5COfEJ4ggym7ZoOg=; b=AkuVuRD2+8rT5EqXM7ABXeDMEVa56kUfTmNFmOBI22vj2Rt0F+YGa3nX3X3oQFG1qU nRJRgDIj7tgTyFFZx5y4OZvIfrCUKXgZhTa6KgbblDVLZeefS5b2TRa8wICZfg9OF/Re NAHyeY8QCE/fMHENvQLJl7N0n7BJiv3uXgeZzySqL+CyTxz38Jxe2eXXNJxSavovqXz5 dwfRwChQFUhCZT4rUM7BjBPHVQVGx9bAbf6tZyeebzMTiBQ1ML4rhuOeJXnaU1J5c1DE C5vrMijV+rvoByT9pm/QfHtvH6oUEA9Z5Ye6qAYKgfJXSiiszQxi1lkSSdpdZPXtUQh5 y1JQ== X-Gm-Message-State: AOJu0YwOZo+drj4QfH0+E9v4Rnc1mOdCnzHLYH8hvX7jyk3+XgiOwKFI KUwhS4fAqfuBHpyxGVRlDaSUdt7FHRV8MC74RIebPR1T7qnDe2WSuSvDGZuL0qX3DiAeOg3vkac szPul X-Gm-Gg: ASbGncuAcOPFgzwoGRiEpNvjf0+Y/gk09w9LORV9sOlCpZtHN7Kw4BuUlbn5StIRe0D oZH+j2jvcaMTXKaxrFWjipxWAoh5HsYmv7QVqZMrhsxfOAv3rTC+NiuGnoDGNXM5qsukEshsog4 8aKPhu2+34a4nzUg0sMaF8kNMMpBhcnDu7ozTa5iaKcU63rYLvVX9pHU6h5ZX0YHcs58z3lvjYD FzX3Z/PV24bAhUYtYMbfxP1AuxjyH8Jj9yVSYKFJ56Ul42ZWKM8EicRRDmKOLAiwt+PjKL1PxN/ R55olZgsqJs5LEB8JllmapvNDCncuXuaVvFSEiNc67iAEu44u6KCOB8xRbur3eQj09CJSJN4+Me X+qp+coHBB3g= X-Google-Smtp-Source: AGHT+IFsZoV8v23pcDTpfzEZ0vwNFgfV+Qih5ZUVTxL+u4p6Zagv2jV8fET5gQjsPlZRr2QTHpKfYA== X-Received: by 2002:a05:6000:2211:b0:3a4:e667:9226 with SMTP id ffacd0b85a97d-3a533191a08mr671186f8f.10.1749207414596; Fri, 06 Jun 2025 03:56:54 -0700 (PDT) Received: from NAUSH-P-DELL.tail9c427.ts.net ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-452137277afsm20092455e9.30.2025.06.06.03.56.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Jun 2025 03:56:54 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman , Jacopo Mondi Subject: [PATCH v3 1/6] libcamera: framebuffer: Add FrameMetadata::Status::FrameStartup Date: Fri, 6 Jun 2025 11:55:19 +0100 Message-ID: <20250606105651.1624640-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250606105651.1624640-1-naush@raspberrypi.com> References: <20250606105651.1624640-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new status enum, FrameStartup, used to denote that even though the frame has been successfully captured, the IQ parameters set by the IPA will cause the frame to be unusable and applications are advised to not consume this frame. An example of this would be on a cold-start of the 3A algorithms, and there will be large oscillations to converge to a stable state quickly. Additional, update the definition of the FrameError state to cover the usage when the sensor is known to produce a number of invalid/error frames after stream-on. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- include/libcamera/framebuffer.h | 1 + src/libcamera/framebuffer.cpp | 13 ++++++++++--- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/include/libcamera/framebuffer.h b/include/libcamera/framebuffer.h index ff83924300ac..e83825b466aa 100644 --- a/include/libcamera/framebuffer.h +++ b/include/libcamera/framebuffer.h @@ -26,6 +26,7 @@ struct FrameMetadata { FrameSuccess, FrameError, FrameCancelled, + FrameStartup, }; struct Plane { diff --git a/src/libcamera/framebuffer.cpp b/src/libcamera/framebuffer.cpp index 826848f75a56..93be497be052 100644 --- a/src/libcamera/framebuffer.cpp +++ b/src/libcamera/framebuffer.cpp @@ -43,12 +43,19 @@ LOG_DEFINE_CATEGORY(Buffer) * The frame has been captured with success and contains valid data. All fields * of the FrameMetadata structure are valid. * \var FrameMetadata::FrameError - * An error occurred during capture of the frame. The frame data may be partly - * or fully invalid. The sequence and timestamp fields of the FrameMetadata - * structure is valid, the other fields may be invalid. + * The frame data is partly or fully corrupted, missing or otherwise invalid. + * This can for instance indicate a hardware transmission error, or invalid data + * produced by the sensor during its startup phase. The sequence and timestamp + * fields of the FrameMetadata structure is valid, all the other fields may be + * invalid. * \var FrameMetadata::FrameCancelled * Capture stopped before the frame completed. The frame data is not valid. All * fields of the FrameMetadata structure but the status field are invalid. + * \var FrameMetadata::FrameStartup The frame has been successfully captured. + * However, the IPA is in a cold-start or reset phase and will result in image + * quality parameters producing unusable images. Applications are recommended to + * not consume these frames. All other fields of the FrameMetadata structure are + * valid. */ /** From patchwork Fri Jun 6 10:55:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23476 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 04090C31E9 for ; Fri, 6 Jun 2025 10:57:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9A76F68DC6; Fri, 6 Jun 2025 12:57:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="fXuU0BU3"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5934E68DBF for ; Fri, 6 Jun 2025 12:56:57 +0200 (CEST) Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-43f106a3591so1790395e9.3 for ; Fri, 06 Jun 2025 03:56:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1749207416; x=1749812216; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9YoGPuC/dIaxXZXl2DZZumzaSc1VCEyQJUBMyUKq20s=; b=fXuU0BU34b9bjPk6fBmKqjIUHTjRxJstf/AUHKhwyj4z/2YAgu0Psdz0y7oSqFw++y +4APilJXFF/3JWgt4MCKHHXh2a+n7tXQlKDctnOquGHsNl84rUzprvP5RZ61Yw/Hd1fw k7EZM0S7UnaDqY///rEl4YmzPZ6wltbp2Vot03BJyxX4u743IkVcdkvO4oj0Pvj/YO22 PzzQ+xn0zF8xIv/WvS83XNIYa4raxlTOMw1fhpc2ERZoFW69jA+MZLV9pMkkiXLeYDqh E5djwLRevIXrUWTUtstjyAgNbVOdB87UZAVbDHG1eR7q0hu6IMU/bK7VvTgoQYyn98R7 mN+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749207416; x=1749812216; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9YoGPuC/dIaxXZXl2DZZumzaSc1VCEyQJUBMyUKq20s=; b=SmpYQd2zbyKHCMdi03DYVAZfsxG7JLDRMp8g8DsvjqvS7g2sIM6dUtZsRypAPuJqm4 O9SgVqQ9aLKH/HlR71FiOwushsGqY4QMRmri/5WhLsK9AbgMBc6OsjIQWITOSBVF1rXn dK0AhLLraZmFmbmhbhz1sJSOXnvVUx0dlI9LfUAJk34rZaPiQea0eUvP+gJlwYsa6T1K 99gkNz1wyuCvkT7WJCkHMwKb4oDsHKOpZsgwzbNGGA9DjiYreTGOE/I1CL4pvxNBzVdp CMT0ndwTkG9/rkuzKBrR1kpSCHHr7WHUnudbu2zwaZQwU/xgQ+HV3whRUfugU8fef2GV HVNg== X-Gm-Message-State: AOJu0Yylm/oPqcgl6HmkXERoYhkx4oVvjE7stmkp2omcRzyNbJPRAlBD fApbD/wBAqRow3yYs38nCOstG3iLcgNUUFu8Je+SlSiGfLSOlMjtyjTXS2jfnXRGTUrYGLja5jl suVzB X-Gm-Gg: ASbGncu6Hu9sANwxwyr26KRzEWvf6QZwPvKKX2swyvTCOOF7SwmDzcqGs41eE8yBOfW aYttRBgmz9xR7ygaXFLRxrr7EVo0ZN0LZSuiKKaAdJtwXwV+VpjbNiTELmwrwj0nKlSb7XZpcvo p5z50gdQgW66HQHqdKVrZPmoCkdQrzpzUNpr7YuJ0+rgffPZhmJRfc3qmeysX2XWkV1luJCCiGJ s35GjJEhgdsrzJLtaRvB6TR470Th3LTIhwFUiKdAb61pcJmVuDbfV//j+5IuNFu2c6mURmFTYj6 twSk68V3p/8cE/caFpF0HuGXOWSMHHfOOgI+Ul9e2AKUmqcm/75l7WF+XPG8IlvJVQweTjh31zx l2R+sFsGwhKk= X-Google-Smtp-Source: AGHT+IEpXLfhuab/ICILNtr15MYjv+xfN20emM1EFQdW9A4464JlkWjthQhnrICZxTvte+MYWbwbwg== X-Received: by 2002:a05:600c:3f16:b0:442:7c40:fd81 with SMTP id 5b1f17b1804b1-4529abc8230mr7846405e9.5.1749207416442; Fri, 06 Jun 2025 03:56:56 -0700 (PDT) Received: from NAUSH-P-DELL.tail9c427.ts.net ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-452137277afsm20092455e9.30.2025.06.06.03.56.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Jun 2025 03:56:55 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman , Jacopo Mondi Subject: [PATCH v3 2/6] ipa: rpi: Replace dropFrameCount in the IPA -> PH interface Date: Fri, 6 Jun 2025 11:55:20 +0100 Message-ID: <20250606105651.1624640-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250606105651.1624640-1-naush@raspberrypi.com> References: <20250606105651.1624640-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Replace the dropFrameCount parameter returned from ipa::start() to the pipeline handler by startupFrameCount and invalidFrameCount. The former counts the number of frames required for AWB/AGC to converge, and the latter counts the number of invalid frames produced by the sensor when starting up. In the pipeline handler, use the sum of these 2 values to replicate the existing dropFrameCount behaviour. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- include/libcamera/ipa/raspberrypi.mojom | 3 ++- src/ipa/rpi/common/ipa_base.cpp | 14 ++++++++------ .../pipeline/rpi/common/pipeline_base.cpp | 4 ++-- 3 files changed, 12 insertions(+), 9 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index e30c70bde08b..12b083e9d04b 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -52,7 +52,8 @@ struct ConfigResult { struct StartResult { libcamera.ControlList controls; - int32 dropFrameCount; + int32 startupFrameCount; + int32 invalidFrameCount; }; struct PrepareParams { diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index e0f8b7e782f4..eee81e064e0d 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -324,6 +324,7 @@ void IpaBase::start(const ControlList &controls, StartResult *result) * "mistrusted", which depends on whether this is a startup from cold, * or merely a mode switch in a running system. */ + unsigned int agcConvergenceFrames = 0, awbConvergenceFrames = 0; frameCount_ = 0; if (firstStart_) { dropFrameCount_ = helper_->hideFramesStartup(); @@ -336,7 +337,6 @@ void IpaBase::start(const ControlList &controls, StartResult *result) * (mistrustCount_) that they won't see. But if zero (i.e. * no convergence necessary), no frames need to be dropped. */ - unsigned int agcConvergenceFrames = 0; RPiController::AgcAlgorithm *agc = dynamic_cast( controller_.getAlgorithm("agc")); if (agc) { @@ -345,7 +345,6 @@ void IpaBase::start(const ControlList &controls, StartResult *result) agcConvergenceFrames += mistrustCount_; } - unsigned int awbConvergenceFrames = 0; RPiController::AwbAlgorithm *awb = dynamic_cast( controller_.getAlgorithm("awb")); if (awb) { @@ -353,15 +352,18 @@ void IpaBase::start(const ControlList &controls, StartResult *result) if (awbConvergenceFrames) awbConvergenceFrames += mistrustCount_; } - - dropFrameCount_ = std::max({ dropFrameCount_, agcConvergenceFrames, awbConvergenceFrames }); - LOG(IPARPI, Debug) << "Drop " << dropFrameCount_ << " frames on startup"; } else { dropFrameCount_ = helper_->hideFramesModeSwitch(); mistrustCount_ = helper_->mistrustFramesModeSwitch(); } - result->dropFrameCount = dropFrameCount_; + result->startupFrameCount = std::max({ agcConvergenceFrames, awbConvergenceFrames }); + result->invalidFrameCount = dropFrameCount_; + + dropFrameCount_ = std::max({ dropFrameCount_, agcConvergenceFrames, awbConvergenceFrames }); + + LOG(IPARPI, Debug) << "Startup frames: " << result->startupFrameCount + << " Invalid frames: " << result->invalidFrameCount; firstStart_ = false; lastRunTimestamp_ = 0; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 1f13e5230fae..5956485953a2 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -660,8 +660,8 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) data->setSensorControls(result.controls); /* Configure the number of dropped frames required on startup. */ - data->dropFrameCount_ = data->config_.disableStartupFrameDrops - ? 0 : result.dropFrameCount; + data->dropFrameCount_ = data->config_.disableStartupFrameDrops ? + 0 : result.startupFrameCount + result.invalidFrameCount; for (auto const stream : data->streams_) stream->resetBuffers(); From patchwork Fri Jun 6 10:55:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23477 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ADC82C3292 for ; Fri, 6 Jun 2025 10:57:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2B31A68DCA; Fri, 6 Jun 2025 12:57:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="a5VGpBlt"; dkim-atps=neutral Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3206D68DB0 for ; Fri, 6 Jun 2025 12:56:59 +0200 (CEST) Received: by mail-wr1-x42b.google.com with SMTP id ffacd0b85a97d-3a4ee391e6fso176689f8f.3 for ; Fri, 06 Jun 2025 03:56:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1749207418; x=1749812218; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AWiiOo6NhxVE0fDk/z9rUm9ZZS3TZ/840uwenlIidpI=; b=a5VGpBlt+mOXEal1wtdaErniHPttoqAw5SRK0V5JS2Sv6fj45I7kGMecUajF7dwYmu uI6oY11O/0drf53VpupzvXeuTFKRHfyoxhU2PaTNrMZ9phd3S9bSlV8BfZPyTU0DHNx7 JUl9oJM+zfWmfL7v0YwYmGfLAhsoFM86si5+k4RdjTYjw6/Grw3XZG2mwsnMO2in/uJi ilkzpzZV55z4+GBFQDafXz+4jhfGgG6jG51P/xy4QvIwU5WokCTa3MQ1WfEzPVlHDPU4 E27CGV19WbAEolcxQ+6+hD6GcyzqPFoueeHBpUdpeEQAPF4mNFkaRqVGTcei9tqb8G8D HVXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749207418; x=1749812218; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AWiiOo6NhxVE0fDk/z9rUm9ZZS3TZ/840uwenlIidpI=; b=EUO3Be9vXhks1FX0o63nU88SDhRRE3sCkJcFU2v6yB0kXVOnFtpMhkMB4wMwS8K9W/ 0eooIu430MhHfQaX4qLBkMdljKp4DwAL/iU23vDDQb+ioJzhc5VXcagpDWm5ePf+usJ1 4QcJ4r4MfR2SoU8OJMyI3dVphDFDwgopjHswh3pRqvVNU3KrgPLm5gmCh+CcX1wTCUsJ q+j1ZFa43H6+zOMoKkEinvoSBjPXWNpkOWVxWFH7OgnNQMhOybRZwNnuGLdR0AjZ1+kg r/Y6du8XCgwzq571bXFIR5SnhsoeZ2ycesMReZJ7C5CFj65I1XmKcuISsjOjjDlX2iM0 O4Kg== X-Gm-Message-State: AOJu0YzYY42uwCN3YoYLsVrvpXBkWOyt/81eAqqo5ZLNXGLAnuy1SN40 hlKRUqHrZ5LY6R5OmC30ousn/dunLmA87pxJjeL/ES/AsSw80pMjsV8iW8WPPqs1b8K3K7rzYQq dzQdK X-Gm-Gg: ASbGncvVHGzN9hYv2KP7ZiYg2Z3/HnQnYP2vC9vQOqiQ2vlstroLI+g33W16oVJYT86 erc36aQhbZO+FBagDt5bsHD8jNHakbiEAJ/ylhYSkd4D1Toq6QWjrSc6isatxH+SoBRg0DwZlmy hoW9zIeG+cyAb9p9d2Rt9sfpMNUi9pFN3Cmd64pPKhG38xl62BWvWrPZOxe23CqA9CUtBg5KL6q xv/g88VxVAqJZIi2WANR3LGYWBXl/x/qksDyoUzxdw07lWsW6Yx8ZshHbfK9fzcBLN+fBYUQC28 D5h3KaJnN7LGNvoVEodySSn/A5J3lfWjhVspt2t0jNeV2eLhFkU93bDcOaMB/v7ZPMp08h7A65t MDu3DzXoWSA4= X-Google-Smtp-Source: AGHT+IFSNBIaMZVCMznBifiyAi49yzhyLjungw2y27LjwIVdocrnpsCpuQ5onWq9sdrgWDkyytoHAA== X-Received: by 2002:a05:6000:400e:b0:3a3:63d3:368e with SMTP id ffacd0b85a97d-3a532f7e27cmr590673f8f.0.1749207418427; Fri, 06 Jun 2025 03:56:58 -0700 (PDT) Received: from NAUSH-P-DELL.tail9c427.ts.net ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-452137277afsm20092455e9.30.2025.06.06.03.56.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Jun 2025 03:56:57 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman , Jacopo Mondi Subject: [PATCH v3 3/6] pipeline: ipa: rpi: Split RPiCameraData::dropFrameCount_ Date: Fri, 6 Jun 2025 11:55:21 +0100 Message-ID: <20250606105651.1624640-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250606105651.1624640-1-naush@raspberrypi.com> References: <20250606105651.1624640-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Split the pipeline handler drop frame tracking into startup frames and invalid frames, as reported by the IPA. Remove the drop buffer handling logic in the pipeline handler. Now all image buffers are returned out with the appropriate FrameStatus set for startup or invalid frames. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- .../pipeline/rpi/common/pipeline_base.cpp | 98 ++++++++----------- .../pipeline/rpi/common/pipeline_base.h | 5 +- 2 files changed, 42 insertions(+), 61 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 5956485953a2..3f0b7abdc59a 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -659,9 +659,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) if (!result.controls.empty()) data->setSensorControls(result.controls); - /* Configure the number of dropped frames required on startup. */ - data->dropFrameCount_ = data->config_.disableStartupFrameDrops ? - 0 : result.startupFrameCount + result.invalidFrameCount; + /* Configure the number of startup and invalid frames reported by the IPA. */ + data->startupFrameCount_ = result.startupFrameCount; + data->invalidFrameCount_ = result.invalidFrameCount; for (auto const stream : data->streams_) stream->resetBuffers(); @@ -678,7 +678,6 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) data->buffersAllocated_ = true; } - /* We need to set the dropFrameCount_ before queueing buffers. */ ret = queueAllBuffers(camera); if (ret) { LOG(RPI, Error) << "Failed to queue buffers"; @@ -894,28 +893,12 @@ int PipelineHandlerBase::queueAllBuffers(Camera *camera) int ret; for (auto const stream : data->streams_) { - if (!(stream->getFlags() & StreamFlag::External)) { - ret = stream->queueAllBuffers(); - if (ret < 0) - return ret; - } else { - /* - * For external streams, we must queue up a set of internal - * buffers to handle the number of drop frames requested by - * the IPA. This is done by passing nullptr in queueBuffer(). - * - * The below queueBuffer() call will do nothing if there - * are not enough internal buffers allocated, but this will - * be handled by queuing the request for buffers in the - * RPiStream object. - */ - unsigned int i; - for (i = 0; i < data->dropFrameCount_; i++) { - ret = stream->queueBuffer(nullptr); - if (ret) - return ret; - } - } + if (stream->getFlags() & StreamFlag::External) + continue; + + ret = stream->queueAllBuffers(); + if (ret < 0) + return ret; } return 0; @@ -1412,7 +1395,15 @@ void CameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::Stream *stream) * buffer back to the stream. */ Request *request = requestQueue_.empty() ? nullptr : requestQueue_.front(); - if (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) { + if (request && request->findBuffer(stream) == buffer) { + FrameMetadata &md = buffer->_d()->metadata(); + + /* Mark the non-converged and invalid frames in the metadata. */ + if (invalidFrameCount_) + md.status = FrameMetadata::Status::FrameError; + else if (startupFrameCount_) + md.status = FrameMetadata::Status::FrameStartup; + /* * Tag the buffer as completed, returning it to the * application. @@ -1458,42 +1449,31 @@ void CameraData::handleState() void CameraData::checkRequestCompleted() { - bool requestCompleted = false; - /* - * If we are dropping this frame, do not touch the request, simply - * change the state to IDLE when ready. - */ - if (!dropFrameCount_) { - Request *request = requestQueue_.front(); - if (request->hasPendingBuffers()) - return; + Request *request = requestQueue_.front(); + if (request->hasPendingBuffers()) + return; - /* Must wait for metadata to be filled in before completing. */ - if (state_ != State::IpaComplete) - return; + /* Must wait for metadata to be filled in before completing. */ + if (state_ != State::IpaComplete) + return; - LOG(RPI, Debug) << "Completing request sequence: " - << request->sequence(); + LOG(RPI, Debug) << "Completing request sequence: " + << request->sequence(); - pipe()->completeRequest(request); - requestQueue_.pop(); - requestCompleted = true; - } + pipe()->completeRequest(request); + requestQueue_.pop(); - /* - * Make sure we have three outputs completed in the case of a dropped - * frame. - */ - if (state_ == State::IpaComplete && - ((ispOutputCount_ == ispOutputTotal_ && dropFrameCount_) || - requestCompleted)) { - LOG(RPI, Debug) << "Going into Idle state"; - state_ = State::Idle; - if (dropFrameCount_) { - dropFrameCount_--; - LOG(RPI, Debug) << "Dropping frame at the request of the IPA (" - << dropFrameCount_ << " left)"; - } + LOG(RPI, Debug) << "Going into Idle state"; + state_ = State::Idle; + + if (invalidFrameCount_) { + invalidFrameCount_--; + LOG(RPI, Debug) << "Decrementing invalid frames to " + << invalidFrameCount_; + } else if (startupFrameCount_) { + startupFrameCount_--; + LOG(RPI, Debug) << "Decrementing startup frames to " + << startupFrameCount_; } } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index aae0c2f35888..6023f9f9d6b3 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -48,7 +48,7 @@ class CameraData : public Camera::Private public: CameraData(PipelineHandler *pipe) : Camera::Private(pipe), state_(State::Stopped), - dropFrameCount_(0), buffersAllocated_(false), + startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false), ispOutputCount_(0), ispOutputTotal_(0) { } @@ -151,7 +151,8 @@ public: /* Mapping of CropParams keyed by the output stream order in CameraConfiguration */ std::map cropParams_; - unsigned int dropFrameCount_; + unsigned int startupFrameCount_; + unsigned int invalidFrameCount_; /* * If set, this stores the value that represets a gain of one for From patchwork Fri Jun 6 10:55:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23478 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1F048C31E9 for ; Fri, 6 Jun 2025 10:57:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1311768DBF; Fri, 6 Jun 2025 12:57:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="RnkhOZ5F"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0E7AB68DC3 for ; Fri, 6 Jun 2025 12:57:00 +0200 (CEST) Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-44ff4081975so470095e9.1 for ; Fri, 06 Jun 2025 03:57:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1749207419; x=1749812219; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8NJR7QRxV2lxaWVQp05RjZAoEuju/VJwaB09A6PDeMc=; b=RnkhOZ5FaHiL9v1pMggHTPZvRfekHQ0bVt8hMF/brjSr/rTEXeXookPkJzPA+/mjeh kZ0iCvIgdQ8JB2NoBveGuraeFk2WOlqwSDDZE1Htm3JUXjRTsDrYKvmH3JrAxMbHs8F4 nmWL7zLzz7+QUwMf0c4eL+n0j8Tcc0Ah/7dbSE6qGjBk37nY3n3akrvDRfWeBntfBG4O hsYkF+GK7HMvHxKhN0sR7lCOfbGDcnsZDq8WtY1jN+qvkQ28bKxCg+jYW9xrK2DWuIYH IpgBGGtqspzX5yJ21nvVZY69uEh1Kqp16oV/V/JXBjqZ9lORoww9OypZV19j45Ts7fbf RXXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749207419; x=1749812219; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8NJR7QRxV2lxaWVQp05RjZAoEuju/VJwaB09A6PDeMc=; b=EPoCi3PsMy4SsoG0Z8Hn0ff+lCbeW9aJL3oA8JCoKJY8BY2j6k5EBqKoFgFAOXyGJw Oyfn0mUFBut+V8UbY+VffnvO6ml8MD3GKoRQNxtBuNm+JuETQyvx6yQteoz+/2spbc3g KgZM7cjIio1qcTCE7fv1IifIDKBkKEJl5cHPG7Pz9a1mwYI2IpDT+yolvVYEYsLvXIwG 2GgGEj4CyBlA1lpPcDNkbSf0fHCr9ocBrIb9da6Iqdy41QHDIy+pFyBgmStYsVX9kjZ6 rPfruyxyNBo2NmYni9n3uqnglFBEb/Dzz0QH/g4tcKFrTLM4v0U/L4GJIECudMqZpDEM WnKQ== X-Gm-Message-State: AOJu0YzRq3Ad7NlXmgXCQ4Ea/zvzQ9VlCnpon2MjsB8m7Fcdjxe80kk9 H5dLU+UBQ6b8epHXIit9Rf8WNwyYECZZ3fJ8b4qMuSOKRSky1R3GzFqiAqlgagbPzzmfn7C9gHD kVppv X-Gm-Gg: ASbGncuKdw7KCSWn12K8eRtAqLMAoPf9iqCJ0rwKGWl4+V/lWOrli+ala2l1VecDjDa 1cHSH8BEAVBCOGEE42NjZRDbQ3XKNQu79Hsw3/neDYORGvOiZZM+ozLvQfg6cRXM0X5y9cfibGp HvnUruVd4qs8EEoC8EGsNnSWlelg/pdXfXVWU5E2RSkzomLcAEACcBgx4J3k6qEpJvzo9HqgTHy +ZKZYKBFLH+MfBbe2vFLS7xGQ6EyQERNxqFQlKZfpbR9VR3FDAt6Wii0yfV8SMSf0uQn6TMtRHe CbXF+ZZdY18eHH3oEt5gFfWigUchPRLFmPBxXGwbuZtnLLoB/pBseXZC8DKM7mMKyRavzVHaUOZ F X-Google-Smtp-Source: AGHT+IHptrA5q0hNbIuu9PZBQQGJaQSXmW5e0o+MhrAne8aJqTfEoL0gJBCc51h8+3500PRutFx82w== X-Received: by 2002:a05:600c:4f0f:b0:442:fff5:5185 with SMTP id 5b1f17b1804b1-4529abdfba3mr7809445e9.6.1749207418999; Fri, 06 Jun 2025 03:56:58 -0700 (PDT) Received: from NAUSH-P-DELL.tail9c427.ts.net ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-452137277afsm20092455e9.30.2025.06.06.03.56.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Jun 2025 03:56:58 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman , Jacopo Mondi Subject: [PATCH v3 4/6] pipeline: rpi: Remove disable_startup_frame_drops config option Date: Fri, 6 Jun 2025 11:55:22 +0100 Message-ID: <20250606105651.1624640-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250606105651.1624640-1-naush@raspberrypi.com> References: <20250606105651.1624640-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the previous change to not drop frames in the pipeline handler, the "disable_startup_frame_drops" pipeline config option is not used. Remove it, and throw a warning if the option is present in the YAML config file. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 7 ++++--- src/libcamera/pipeline/rpi/common/pipeline_base.h | 5 ----- src/libcamera/pipeline/rpi/pisp/data/example.yaml | 5 ----- src/libcamera/pipeline/rpi/vc4/data/example.yaml | 5 ----- 4 files changed, 4 insertions(+), 18 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 3f0b7abdc59a..d8c7ca9352de 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1078,7 +1078,6 @@ void CameraData::enumerateVideoDevices(MediaLink *link, const std::string &front int CameraData::loadPipelineConfiguration() { config_ = { - .disableStartupFrameDrops = false, .cameraTimeoutValue = 0, }; @@ -1115,8 +1114,10 @@ int CameraData::loadPipelineConfiguration() const YamlObject &phConfig = (*root)["pipeline_handler"]; - config_.disableStartupFrameDrops = - phConfig["disable_startup_frame_drops"].get(config_.disableStartupFrameDrops); + if (phConfig.contains("disable_startup_frame_drops")) + LOG(RPI, Warning) + << "The disable_startup_frame_drops key is now deprecated, " + << "startup frames are now identified by the FrameMetadata::Status::FrameStartup flag"; config_.cameraTimeoutValue = phConfig["camera_timeout_value_ms"].get(config_.cameraTimeoutValue); diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 6023f9f9d6b3..e27c4f860d1a 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -164,11 +164,6 @@ public: bool buffersAllocated_; struct Config { - /* - * Override any request from the IPA to drop a number of startup - * frames. - */ - bool disableStartupFrameDrops; /* * Override the camera timeout value calculated by the IPA based * on frame durations. diff --git a/src/libcamera/pipeline/rpi/pisp/data/example.yaml b/src/libcamera/pipeline/rpi/pisp/data/example.yaml index d67e654a8b9e..baf03be79bb3 100644 --- a/src/libcamera/pipeline/rpi/pisp/data/example.yaml +++ b/src/libcamera/pipeline/rpi/pisp/data/example.yaml @@ -16,11 +16,6 @@ # # "num_cfe_config_queue": 2, - # Override any request from the IPA to drop a number of startup - # frames. - # - # "disable_startup_frame_drops": false, - # Custom timeout value (in ms) for camera to use. This overrides # the value computed by the pipeline handler based on frame # durations. diff --git a/src/libcamera/pipeline/rpi/vc4/data/example.yaml b/src/libcamera/pipeline/rpi/vc4/data/example.yaml index b8e01adeaf40..27e543488d48 100644 --- a/src/libcamera/pipeline/rpi/vc4/data/example.yaml +++ b/src/libcamera/pipeline/rpi/vc4/data/example.yaml @@ -29,11 +29,6 @@ # # "min_total_unicam_buffers": 4, - # Override any request from the IPA to drop a number of startup - # frames. - # - # "disable_startup_frame_drops": false, - # Custom timeout value (in ms) for camera to use. This overrides # the value computed by the pipeline handler based on frame # durations. From patchwork Fri Jun 6 10:55:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23479 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D07EFC3324 for ; Fri, 6 Jun 2025 10:57:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CA89A68DC6; Fri, 6 Jun 2025 12:57:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="ad2U3I9z"; dkim-atps=neutral Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8DC3668DBE for ; Fri, 6 Jun 2025 12:57:00 +0200 (CEST) Received: by mail-wm1-x331.google.com with SMTP id 5b1f17b1804b1-450ddb35583so1798975e9.1 for ; Fri, 06 Jun 2025 03:57:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1749207420; x=1749812220; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UjhoSFLjQT8XeMDrUTrn+s1ViaVwS7kMkgdRcH6/WUg=; b=ad2U3I9zpLAhCS17UruZrqXLptJUe8aFCQf4/ZjuAtCMf/8PQbBf307kWIyDiVpFtj +OiHq0qM/LgYlE9zRZs8RlRooyzifeegDHwZL/t6qzYk3n8DWI1XmccVpRhqxWc4LkGp wdK+Rra3A1AKuRvFN1+9OnjR/9usoIpO3lthh1O/lMaWqsmGj51mQQEhsCZMN2iL1cxV V4zsGV9qwyGdDJ6SQEYJKZqwPphY1G5aF8/H5KfSWqbz1299uHIwZDMX4G8tPFObq9mH Txa0y6qp9kaX85royTt/1PqIVNOPWSDPaP8ORpFGw4PWdk0CtOqsAFdaCxzz57gKWKS5 alVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749207420; x=1749812220; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UjhoSFLjQT8XeMDrUTrn+s1ViaVwS7kMkgdRcH6/WUg=; b=nK4ba1uUap2yDjb5IsdlLgSkQKJSpaaaWAxpgljbU3ujdOnGRzc6RhYFkBLPaOADb/ vAs6w9PodbqPtfCPhZo4zg3gO0XpebPW/jrWrQ5xrBI5tQ4U/PzHJpmMpd4vTxmrmTBA ozupxd+6et6hxHDArbQrNUfbcHUzhWwScuRj2NaR15wQi93QBHrquKMSKfKiLWNo12l0 eI+QEIFNfzN8+rnlwfwrC8eO18xBXWu5eVYUcg7sVbLyM6GVD936wewAL0I2/RDhqaZn GAqsqZAf7/5AHXqQWcxsrYtNM2L/5SSuZsBOmpA6FtQbTwkrjekRM3f87d9q7bbf2VnE syMQ== X-Gm-Message-State: AOJu0Yyh5VVtBLDChPX9D3z560SZrWVvXzMSGrvacV8dVnLKmZb3Mv0z Zy2XIhQCTZ9fWRqVsufiLSxrMTxYHe+73wTTkmz0cT01mhOYZHmoLzETzHEJ39+iiwVtYqMYy0d 0dM9Y X-Gm-Gg: ASbGncsAWZPf//Qxhzpm73dZpzdoZBICORFM5/7Y82ST9uELsjjWVcTc1ukYHZVQbgG NzJd7dFbKGxNsIJMM6T5GimD7JE9Kj7tBZZvLfE1ic0t/IsLaGjfaoYvpG81+7dCYTTnq7ad5t+ LJBtMnB1m0XbXpUPA72OS142VpD7bQdRglRHtTT8SMUcmwxUxF8aCCJE+bgVb4WPEkdn4tPO1as HuHwBp52XzFSdWPSuL7Xu2MSmoC7t3GDYzUifOi3OLYHHXKnlsOpyeqsd6j1w/Cmzqrz2FGd0lQ Yt5HSsvpzva4/r26xnDedc4uQr1SelldqHlLKmpEmIJbZ8JUCDAwT1vj4/F1pcvzr0ZCK6LXvwo 5o9+ZgjmcOOU= X-Google-Smtp-Source: AGHT+IFv0jDjECPcCbx/BNJrcRuXsTVfA7mx6ZBPiR5Cr5Qd8uEOKs82+tFJ7xIIyNgdz0thgKhOsw== X-Received: by 2002:a05:600c:6297:b0:450:d5ed:3c20 with SMTP id 5b1f17b1804b1-4529abd72bfmr7101445e9.6.1749207419576; Fri, 06 Jun 2025 03:56:59 -0700 (PDT) Received: from NAUSH-P-DELL.tail9c427.ts.net ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-452137277afsm20092455e9.30.2025.06.06.03.56.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Jun 2025 03:56:59 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman , Jacopo Mondi Subject: [PATCH v3 5/6] pipeline: rpi: Remove ispOutputCount_ and ispOutputTotal_ Date: Fri, 6 Jun 2025 11:55:23 +0100 Message-ID: <20250606105651.1624640-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250606105651.1624640-1-naush@raspberrypi.com> References: <20250606105651.1624640-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the drop frame logic removed from the pipeline handler, these member variables and not used, so remove them. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- src/libcamera/pipeline/rpi/common/pipeline_base.h | 6 +----- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 12 ------------ src/libcamera/pipeline/rpi/vc4/vc4.cpp | 15 --------------- 3 files changed, 1 insertion(+), 32 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index e27c4f860d1a..898f31577059 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -48,8 +48,7 @@ class CameraData : public Camera::Private public: CameraData(PipelineHandler *pipe) : Camera::Private(pipe), state_(State::Stopped), - startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false), - ispOutputCount_(0), ispOutputTotal_(0) + startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false) { } @@ -179,9 +178,6 @@ protected: virtual void tryRunPipeline() = 0; - unsigned int ispOutputCount_; - unsigned int ispOutputTotal_; - private: void checkRequestCompleted(); }; diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index 91e7f4c94d96..ccf135c3d8ce 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -1834,12 +1834,6 @@ void PiSPCameraData::beOutputDequeue(FrameBuffer *buffer) dmabufSyncEnd(buffer->planes()[0].fd); handleStreamBuffer(buffer, stream); - - /* - * Increment the number of ISP outputs generated. - * This is needed to track dropped frames. - */ - ispOutputCount_++; handleState(); } @@ -1885,7 +1879,6 @@ void PiSPCameraData::prepareIspComplete(const ipa::RPi::BufferIds &buffers, bool * If there is no need to run the Backend, just signal that the * input buffer is completed and all Backend outputs are ready. */ - ispOutputCount_ = ispOutputTotal_; buffer = cfe_[Cfe::Output0].getBuffers().at(bayerId).buffer; handleStreamBuffer(buffer, &cfe_[Cfe::Output0]); } else @@ -1994,7 +1987,6 @@ int PiSPCameraData::configureBe(const std::optional &yuvColorSpace) global.bayer_enables |= PISP_BE_BAYER_ENABLE_INPUT; global.bayer_order = toPiSPBayerOrder(cfeFormat.fourcc); - ispOutputTotal_ = 1; /* Config buffer */ if (PISP_IMAGE_FORMAT_COMPRESSED(inputFormat.format)) { pisp_decompress_config decompress; decompress.offset = DefaultCompressionOffset; @@ -2025,7 +2017,6 @@ int PiSPCameraData::configureBe(const std::optional &yuvColorSpace) setupOutputClipping(ispFormat0, outputFormat0); be_->SetOutputFormat(0, outputFormat0); - ispOutputTotal_++; } if (global.rgb_enables & PISP_BE_RGB_ENABLE_OUTPUT1) { @@ -2049,7 +2040,6 @@ int PiSPCameraData::configureBe(const std::optional &yuvColorSpace) setupOutputClipping(ispFormat1, outputFormat1); be_->SetOutputFormat(1, outputFormat1); - ispOutputTotal_++; } /* Setup the TDN I/O blocks in case TDN gets turned on later. */ @@ -2256,8 +2246,6 @@ void PiSPCameraData::prepareCfe() void PiSPCameraData::prepareBe(uint32_t bufferId, bool stitchSwapBuffers) { - ispOutputCount_ = 0; - FrameBuffer *buffer = cfe_[Cfe::Output0].getBuffers().at(bufferId).buffer; LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << bufferId diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index fe910bdf2ff9..ac6dab814d35 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -597,8 +597,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi stream->setFlags(StreamFlag::External); } - ispOutputTotal_ = outStreams.size(); - /* * If ISP::Output0 stream has not been configured by the application, * we must allow the hardware to generate an output so that the data @@ -625,8 +623,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi return -EINVAL; } - ispOutputTotal_++; - LOG(RPI, Debug) << "Defaulting ISP Output0 format to " << format; } @@ -662,8 +658,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi << ret; return -EINVAL; } - - ispOutputTotal_++; } /* ISP statistics output format. */ @@ -676,8 +670,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi return ret; } - ispOutputTotal_++; - /* * Configure the Unicam embedded data output format only if the sensor * supports it. @@ -843,12 +835,6 @@ void Vc4CameraData::ispOutputDequeue(FrameBuffer *buffer) handleStreamBuffer(buffer, stream); } - /* - * Increment the number of ISP outputs generated. - * This is needed to track dropped frames. - */ - ispOutputCount_++; - handleState(); } @@ -880,7 +866,6 @@ void Vc4CameraData::prepareIspComplete(const ipa::RPi::BufferIds &buffers, << ", timestamp: " << buffer->metadata().timestamp; isp_[Isp::Input].queueBuffer(buffer); - ispOutputCount_ = 0; if (sensorMetadata_ && embeddedId) { buffer = unicam_[Unicam::Embedded].getBuffers().at(embeddedId & RPi::MaskID).buffer; From patchwork Fri Jun 6 10:55:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23480 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3225EC3325 for ; Fri, 6 Jun 2025 10:57:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 29E7A68DC4; Fri, 6 Jun 2025 12:57:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="kqXQOqsn"; dkim-atps=neutral Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E1CD268DBF for ; Fri, 6 Jun 2025 12:57:00 +0200 (CEST) Received: by mail-wr1-x42c.google.com with SMTP id ffacd0b85a97d-3a4e57d018cso318295f8f.1 for ; Fri, 06 Jun 2025 03:57:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1749207420; x=1749812220; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+1kR8yrdGRXTW45504PuBDRwP7cmAEbOv3Y11k50OaI=; b=kqXQOqsnp5TkRG2o8FNLin+JQUc5DvIliGX/32y6MMMrTmhejdMv9u1tB+qonUmDD8 LJ8DK56QyDgaf+4EmwJ5SfdqyYvGlmVr1MNH9yyRg/+4kQy9J0uemK6igi5BMWzYoRBW /cyAEj2cABeywuSBCTAoPxjt+T5+KBXRnGNhFK/qnceZ2EPem2MDvwEsDyfCDgbEV031 Eir6wMmdtHSaLZYKMScDlPWSIdlNBdgpVHzqnxozJhJsuvlq8AfIF9oKXIPOWMt2ydBT DNouWh05Wc5RtN09cm2BSgOoHy+nsYczrheUI+HSaOXEjuoguKkF8M5dWxW4m3tiGBc4 ek1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749207420; x=1749812220; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+1kR8yrdGRXTW45504PuBDRwP7cmAEbOv3Y11k50OaI=; b=FiTFWUMTwFkECYZ2XW5efzNLM9CNvGGRpTfEs7mt2tU91lLBuwCP+EOifWiuyJ3BwK f1EK7yG/12MZlvT2urecBvjCH6HDUsmU0oB50kWlxknjVIWuEXZ96+pyqejEJec/RgVf 3nA5jBxrUk1jODWj+soy+77JYNyFNbbVXy47MGROxfj7EIhiyxrtQDhdN5X4u6h7+uJ+ huB3Nhb/cR3t6qm6RfWjwQLGooZN4nTBl+vBBb70PrwM2y3MojYnRFcDg7zbSQjLTLC2 OX1mu3aX/T8seTBgp6n1OW0gjqJcJUv0uhGFJqK2fZpezJNgvVyLmW2unMIx3XpDYplV sWVw== X-Gm-Message-State: AOJu0YzzRAz+Nld4iRqKOtnAms3ic4Y371KcNn9MsVTMP/Sj8wcmOK71 NNGjxQRGcH9iSBXNnPFPSFACkYqAMSE2OD7VZtTyNJbtDA5kNaRzZ3A2FdnXHKBYoQK8xzK+Q0W 6H1L2 X-Gm-Gg: ASbGncvA4OlAFFtFGPSB/3WbC16egD7mV4aU7VL5T9cjLXtViokaMgqP5n5ElrzUCQr 6d8B1px67+3bTbrtC3/1C6dSX3eCyVlm8F8xh+XAfSjZ1NUj1ebtbkDXgI5JWCMkTDPKz7Z8ued dj51f7BdxGNCaHVV0UebWAwa9Z8c6rBm5AbNmMEM5l+hKkVStaIA2YcAvCoxUxp0Adw0IvrRL1K IXhIcFIpBydDSYzeiy+erd3quW5vohqkMDXYiE5uJj8u00UNy6fVDzz5r01gqE1D/K6lDvTRpie BTZp0PUMk5n1S7fFeqRdIRXjyokq62osV27MK5IprkSCh8EKCL8naQN/w4rvvCDbzo1U/jegvVJ U X-Google-Smtp-Source: AGHT+IECrpuT7Vsm0ybGrsT1SDG6pyRE7VNVe4Qu3vQihNe6veKOCtK/vZlZQ2gC+CntrBAqB4LTog== X-Received: by 2002:a05:600c:6206:b0:43d:2318:ed7f with SMTP id 5b1f17b1804b1-4528a05cf37mr8223605e9.0.1749207420177; Fri, 06 Jun 2025 03:57:00 -0700 (PDT) Received: from NAUSH-P-DELL.tail9c427.ts.net ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-452137277afsm20092455e9.30.2025.06.06.03.56.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Jun 2025 03:56:59 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman , Jacopo Mondi Subject: [PATCH v3 6/6] ipa: rpi: Rename dropFrameCount_ to invalidCount_ Date: Fri, 6 Jun 2025 11:55:24 +0100 Message-ID: <20250606105651.1624640-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250606105651.1624640-1-naush@raspberrypi.com> References: <20250606105651.1624640-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rename dropFrameCount_ to invalidCount_ to better reflect its use as frames are no longer dropped by the pipeline handler. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- src/ipa/rpi/common/ipa_base.cpp | 10 +++++----- src/ipa/rpi/common/ipa_base.h | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index eee81e064e0d..50e25f05765d 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -327,7 +327,7 @@ void IpaBase::start(const ControlList &controls, StartResult *result) unsigned int agcConvergenceFrames = 0, awbConvergenceFrames = 0; frameCount_ = 0; if (firstStart_) { - dropFrameCount_ = helper_->hideFramesStartup(); + invalidCount_ = helper_->hideFramesStartup(); mistrustCount_ = helper_->mistrustFramesStartup(); /* @@ -353,14 +353,14 @@ void IpaBase::start(const ControlList &controls, StartResult *result) awbConvergenceFrames += mistrustCount_; } } else { - dropFrameCount_ = helper_->hideFramesModeSwitch(); + invalidCount_ = helper_->hideFramesModeSwitch(); mistrustCount_ = helper_->mistrustFramesModeSwitch(); } result->startupFrameCount = std::max({ agcConvergenceFrames, awbConvergenceFrames }); - result->invalidFrameCount = dropFrameCount_; + result->invalidFrameCount = invalidCount_; - dropFrameCount_ = std::max({ dropFrameCount_, agcConvergenceFrames, awbConvergenceFrames }); + invalidCount_ = std::max({ invalidCount_, agcConvergenceFrames, awbConvergenceFrames }); LOG(IPARPI, Debug) << "Startup frames: " << result->startupFrameCount << " Invalid frames: " << result->invalidFrameCount; @@ -443,7 +443,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) /* Allow a 10% margin on the comparison below. */ Duration delta = (frameTimestamp - lastRunTimestamp_) * 1.0ns; - if (lastRunTimestamp_ && frameCount_ > dropFrameCount_ && + if (lastRunTimestamp_ && frameCount_ > invalidCount_ && delta < controllerMinFrameDuration * 0.9 && !hdrChange) { /* * Ensure we merge the previous frame's metadata with the current diff --git a/src/ipa/rpi/common/ipa_base.h b/src/ipa/rpi/common/ipa_base.h index 1a811beb31f2..e818104ba633 100644 --- a/src/ipa/rpi/common/ipa_base.h +++ b/src/ipa/rpi/common/ipa_base.h @@ -115,8 +115,8 @@ private: /* How many frames we should avoid running control algos on. */ unsigned int mistrustCount_; - /* Number of frames that need to be dropped on startup. */ - unsigned int dropFrameCount_; + /* Number of frames that need to be marked as dropped on startup. */ + unsigned int invalidCount_; /* Frame timestamp for the last run of the controller. */ uint64_t lastRunTimestamp_;