From patchwork Thu May 22 07:48:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23416 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D8EC7C31E9 for ; Thu, 22 May 2025 07:52:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7846868D9C; Thu, 22 May 2025 09:52:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="erC//0aG"; dkim-atps=neutral Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A2D0568D85 for ; Thu, 22 May 2025 09:52:48 +0200 (CEST) Received: by mail-wr1-x432.google.com with SMTP id ffacd0b85a97d-3a36977f261so759925f8f.1 for ; Thu, 22 May 2025 00:52:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1747900368; x=1748505168; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7kayUoIMXpwu00j/brvCdM7ksfHri9lnN79748W/YaY=; b=erC//0aGiif6kN17Aiw2ynzh5NE2xV6fu0xCuoOUntOdULuqSY4Wn1m/XU6DiRg1qJ h4Jo4WcVQUyJL3110IHMbDVYYWP0qO2HkSwjiWfjHxqzPO+yy0blYjBC3d4v4aEDTLtc ZxEZLQw9+1jpNjenMZXzGyWTE+U9VMKgkk1HbOa3bm/198+Bn3EwOcKid1iiDewEpH3r 5JaMBkOfMuBHtWmHaw1AqWL8R2uePokeY3iJoqeRMacpZmmBThPcCh8RGMc9toWvloMt fMK/q1lU4v9K2qzOE+BEzvZEkNka4CRZfboIMQBk7Kdap65qfQ85xgx3aLJ6FaX/PLif rBAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747900368; x=1748505168; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7kayUoIMXpwu00j/brvCdM7ksfHri9lnN79748W/YaY=; b=ZlGmi70g/YSjs/+Nkimxvg/g+50JBCpQtrI158S8Cg5qAAwUhN9f3mbh16S3rrKQhm vqDozvohroqBdEg3rCIHGHQiTLMloo/0N5q/OyVHdt4VYtgQnFkV/4B5FrUpt6fEvKc4 hNXx39+BoG0YqkBRItMOgqSOkY1t4WUj8ZGVuDh9RNC7w91L4zHIUJi/A7rb4n+8g54o PnTLjdCQRO3s+SHodXtQlvT0YDCjtpWvaJOUcExnfqXRTlhEWhsx4B3cPIP+1P+AkPLM QRsL1r3reFpk5a6CXBXE5HIGqWlSyJ5QoThIZU8U2JQZbS+K0DFJBy3wRHaYRlcd5Tqe KKsA== X-Gm-Message-State: AOJu0YyK86s8voWWhb99JEVmtCJ+vtXhWCVGYf1NAchDWB0k95ZJ58vn /frvqegH4n5EfRFaFx6Mse+7SR0zn1pGz5IVtAmWFbeTHq9ofkcW/2Y3zUIi+ApayIWsu0UvneJ YKod+ X-Gm-Gg: ASbGncsxPNrGrtbmY5AjuzZ7o4Wj1zR/aO+pFdgpKeg84SAORJAv76MtDL60yZVDMFZ 3sIaXpbrZHyzNjTOxeUuyRgK+47inE6L0V/dtf+tDdMFkiyNxiDSqRbsid+5zbQJgMdJJXzChZv gZSD0VBZBjuQv5FJoebeIaaQo7Dx8mjO2o5KJ/rCLiRw9AorMNbdJg68xHvHz+n7ZNbsZavszEy VMJ00flglZypzTzYGtdexKy15Nfy4VXJthkqLiMzwrWRB3feiYbCM9lqZstNELwSCjMkbE3px2d fmOJOgVcNfljmbtSTG5mdub07KFroBCcmD63N9rINZctRd6Ee+XIHN1Z2P6/6+VM5JefdjL4+Ag zIf0cBw== X-Google-Smtp-Source: AGHT+IHM7X6ypTp7FpdsQW8WofCwrmZj5vVnUYpkWX/0xUqzhS4XXJLTmEsIn4i6h73/Tn6S9p4EbQ== X-Received: by 2002:a05:600c:a07:b0:442:f97a:fee7 with SMTP id 5b1f17b1804b1-442fd6770e0mr85986315e9.6.1747900367926; Thu, 22 May 2025 00:52:47 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a3ca066eb7sm4191516f8f.2.2025.05.22.00.52.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 May 2025 00:52:47 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman Subject: [PATCH v2 1/6] libcamera: framebuffer: Add FrameMetadata::Status::FrameStartup Date: Thu, 22 May 2025 08:48:17 +0100 Message-ID: <20250522075244.1198110-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250522075244.1198110-1-naush@raspberrypi.com> References: <20250522075244.1198110-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new status enum, FrameStartup, used to denote that even though the frame has been successfully captured, the IQ parameters set by the IPA will cause the frame to be unusable and applications are advised to not consume this frame. An example of this would be on a cold-start of the 3A algorithms, and there will be large oscillations to converge to a stable state quickly. Additional, update the definition of the FrameError state to cover the usage when the sensor is known to produce a number of invalid/error frames after stream-on. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- include/libcamera/framebuffer.h | 1 + src/libcamera/framebuffer.cpp | 13 ++++++++++--- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/include/libcamera/framebuffer.h b/include/libcamera/framebuffer.h index ff83924300ac..e83825b466aa 100644 --- a/include/libcamera/framebuffer.h +++ b/include/libcamera/framebuffer.h @@ -26,6 +26,7 @@ struct FrameMetadata { FrameSuccess, FrameError, FrameCancelled, + FrameStartup, }; struct Plane { diff --git a/src/libcamera/framebuffer.cpp b/src/libcamera/framebuffer.cpp index 826848f75a56..93be497be052 100644 --- a/src/libcamera/framebuffer.cpp +++ b/src/libcamera/framebuffer.cpp @@ -43,12 +43,19 @@ LOG_DEFINE_CATEGORY(Buffer) * The frame has been captured with success and contains valid data. All fields * of the FrameMetadata structure are valid. * \var FrameMetadata::FrameError - * An error occurred during capture of the frame. The frame data may be partly - * or fully invalid. The sequence and timestamp fields of the FrameMetadata - * structure is valid, the other fields may be invalid. + * The frame data is partly or fully corrupted, missing or otherwise invalid. + * This can for instance indicate a hardware transmission error, or invalid data + * produced by the sensor during its startup phase. The sequence and timestamp + * fields of the FrameMetadata structure is valid, all the other fields may be + * invalid. * \var FrameMetadata::FrameCancelled * Capture stopped before the frame completed. The frame data is not valid. All * fields of the FrameMetadata structure but the status field are invalid. + * \var FrameMetadata::FrameStartup The frame has been successfully captured. + * However, the IPA is in a cold-start or reset phase and will result in image + * quality parameters producing unusable images. Applications are recommended to + * not consume these frames. All other fields of the FrameMetadata structure are + * valid. */ /** From patchwork Thu May 22 07:48:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23417 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6A660C31E9 for ; Thu, 22 May 2025 07:53:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1D89668D93; Thu, 22 May 2025 09:52:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="d0/+eHyb"; dkim-atps=neutral Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 43BF168D8E for ; Thu, 22 May 2025 09:52:49 +0200 (CEST) Received: by mail-wr1-x435.google.com with SMTP id ffacd0b85a97d-3a368259589so801171f8f.3 for ; Thu, 22 May 2025 00:52:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1747900368; x=1748505168; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5BhPQUO12kezx2EizAFy/txyAFMDEYzTcgAZJffzSAU=; b=d0/+eHyb7NYm4btTlpE0PDKnc6RLDFuQ/cSaq3JeZeiobkwY+OHNNO647+CktKvHxA 4lWV0SWEN/gGTdZD/vcW6KNYAYHNvTq/IP6+HjpDVrWnEPCu29mCncg08YS/vrsj89Ja gOPM0BnLe5TyXNz2tcS9MMzPfOOqB0+HlpEgIIYbrNbkPF13YyqkSKScYQZoaEAYWwJ4 FCruqQDZe4PKxP1K6NiXKUiZ4FIA7DzsAZFwP27St/ryQj+iMAybTRddYKc0DJLm8RtY Uixd1KIa5vz2mJwU6dcFHh+VWvbnBvqTJgjObE3LF5O3OgheZVJTZfOSTeoPpoJ0a0an u+7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747900368; x=1748505168; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5BhPQUO12kezx2EizAFy/txyAFMDEYzTcgAZJffzSAU=; b=HfY3y7UzAUuAQ5imlcJcSIYQwpuxxiQtuIVYnuy3IbopVxL2nn1+yruPxbS7S1fYNi hzaPn3cGY6Pbm+1Qw/XQio3Z0NXmVyM0GmFFaUmyue893uh+Aql0/pSqxLUDO29BkDYR F3fn0kLbum+bbH/jzHGtItRZSR7CDWwTwhR8NE1SYarbM1qTKiy1z5Zg2iFHp19SsBPn f8KR0bnZE0h3aOMdy/lNo4GRQv/eHMrl+CVPb91ROCvZhjvRsVLgZ3Lb09qdSU+5xjkT E1KenSFktWuPNH7wXQPOWiOotJm1RP8g3V5E2EC+B6QpCIKyer5bZgfoW2Cj7Pqb8v0S nmbQ== X-Gm-Message-State: AOJu0YyLQ4sbkcXXUiL5NXZroQ3skRylWVJUAkCrmq2rjSWhXbop+V6h AkoIUKx/Tc4kzaj3HmRTFUyll0zvix/fI/qu1YNFfVQIudA6ivi49XBiYD50e0XfTfXpblb9Vdg U9n4f X-Gm-Gg: ASbGncsJCRfcAi/8ZSlbMp5RE74jdT25ocAuZlRhkxJpLUmrfRNV2lAGe47YrQanqgk bGn4xHKn3yFgOE+L4WzHxf47S0+N5HvJ69nHY6uPQOIgzy8ffYkMLSsRI3Hdkeo2B3DXrMb06vh lO/R6xKLM+mJMmWmSxzmSTEHNkNBF2Zdt9bvAYCrYLp/aCxfLuDiFiSCz6N8LJ/MahXtw5YafO2 jmgq2UbrpnTDSjh1GpvOo+a+0rcUZN8zCjxZY50bHFFgNC1jwutb9VTC6o8Yrc0R0iSTSkkCqCZ edYcub1V/gAQjnhpMoOppP09kOLNno0RcGmXFxGkPd8aVjN2c4AZ3bRPmhRcNFGtHDwWwT0= X-Google-Smtp-Source: AGHT+IE7j5Gt+ezuR+wam29k5eGH3PX6+R4nD2usHMrtfU0D4LVqTd/c/7vGrQ5Uv9TSPNzP35xWhw== X-Received: by 2002:adf:f10e:0:b0:3a3:5c8e:2d8d with SMTP id ffacd0b85a97d-3a35c8e2da0mr7072466f8f.0.1747900368488; Thu, 22 May 2025 00:52:48 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a3ca066eb7sm4191516f8f.2.2025.05.22.00.52.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 May 2025 00:52:48 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman Subject: [PATCH v2 2/6] ipa: rpi: Replace dropFrameCount in the IPA -> PH interface Date: Thu, 22 May 2025 08:48:18 +0100 Message-ID: <20250522075244.1198110-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250522075244.1198110-1-naush@raspberrypi.com> References: <20250522075244.1198110-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Replace the dropFrameCount parameter returned from ipa::start() to the pipeline handler by startupFrameCount and invalidFrameCount. The former counts the number of frames required for AWB/AGC to converge, and the latter counts the number of invalid frames produced by the sensor when starting up. In the pipeline handler, use the sum of these 2 values to replicate the existing dropFrameCount behaviour. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- include/libcamera/ipa/raspberrypi.mojom | 3 ++- src/ipa/rpi/common/ipa_base.cpp | 14 ++++++++------ .../pipeline/rpi/common/pipeline_base.cpp | 4 ++-- 3 files changed, 12 insertions(+), 9 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index e30c70bde08b..12b083e9d04b 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -52,7 +52,8 @@ struct ConfigResult { struct StartResult { libcamera.ControlList controls; - int32 dropFrameCount; + int32 startupFrameCount; + int32 invalidFrameCount; }; struct PrepareParams { diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index e0a93daa9db2..c15f8a7bf71e 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -324,6 +324,7 @@ void IpaBase::start(const ControlList &controls, StartResult *result) * "mistrusted", which depends on whether this is a startup from cold, * or merely a mode switch in a running system. */ + unsigned int agcConvergenceFrames = 0, awbConvergenceFrames = 0; frameCount_ = 0; if (firstStart_) { dropFrameCount_ = helper_->hideFramesStartup(); @@ -336,7 +337,6 @@ void IpaBase::start(const ControlList &controls, StartResult *result) * (mistrustCount_) that they won't see. But if zero (i.e. * no convergence necessary), no frames need to be dropped. */ - unsigned int agcConvergenceFrames = 0; RPiController::AgcAlgorithm *agc = dynamic_cast( controller_.getAlgorithm("agc")); if (agc) { @@ -345,7 +345,6 @@ void IpaBase::start(const ControlList &controls, StartResult *result) agcConvergenceFrames += mistrustCount_; } - unsigned int awbConvergenceFrames = 0; RPiController::AwbAlgorithm *awb = dynamic_cast( controller_.getAlgorithm("awb")); if (awb) { @@ -353,15 +352,18 @@ void IpaBase::start(const ControlList &controls, StartResult *result) if (awbConvergenceFrames) awbConvergenceFrames += mistrustCount_; } - - dropFrameCount_ = std::max({ dropFrameCount_, agcConvergenceFrames, awbConvergenceFrames }); - LOG(IPARPI, Debug) << "Drop " << dropFrameCount_ << " frames on startup"; } else { dropFrameCount_ = helper_->hideFramesModeSwitch(); mistrustCount_ = helper_->mistrustFramesModeSwitch(); } - result->dropFrameCount = dropFrameCount_; + result->startupFrameCount = std::max({ agcConvergenceFrames, awbConvergenceFrames }); + result->invalidFrameCount = dropFrameCount_; + + dropFrameCount_ = std::max({ dropFrameCount_, agcConvergenceFrames, awbConvergenceFrames }); + + LOG(IPARPI, Debug) << "Startup frames: " << result->startupFrameCount + << " Invalid frames: " << result->invalidFrameCount; firstStart_ = false; lastRunTimestamp_ = 0; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 1f13e5230fae..5956485953a2 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -660,8 +660,8 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) data->setSensorControls(result.controls); /* Configure the number of dropped frames required on startup. */ - data->dropFrameCount_ = data->config_.disableStartupFrameDrops - ? 0 : result.dropFrameCount; + data->dropFrameCount_ = data->config_.disableStartupFrameDrops ? + 0 : result.startupFrameCount + result.invalidFrameCount; for (auto const stream : data->streams_) stream->resetBuffers(); From patchwork Thu May 22 07:48:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23418 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 525F1C31E9 for ; Thu, 22 May 2025 07:53:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D8B5C68D97; Thu, 22 May 2025 09:52:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="PiMlbtvq"; dkim-atps=neutral Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C671168D7A for ; Thu, 22 May 2025 09:52:49 +0200 (CEST) Received: by mail-wr1-x431.google.com with SMTP id ffacd0b85a97d-3a36561d25cso752401f8f.1 for ; Thu, 22 May 2025 00:52:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1747900369; x=1748505169; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/vbhBGL4NIRBV5xQKNScWF7oI/HN2DraYPV4SxmxcO8=; b=PiMlbtvqkg/1Wv6ntzA8V1svabSmECQH4KOm8/6BV+7+PQdTSiNoi3F3kyzxWaGR3x 6XyY2kDot7dPxg/HZKl1f5HcNeTgrltqRvfLA2nSJwCiPEEBLZiI7krDEoa0mkCoELMj r054kUIbi6jEeDzZnbrCUxCxPU61NoXvpQJufOLRym6M2KcC/KsCwdgSydokeYUmkDWw Z6qYbSP7E73BWbZ0kwLuEl5Bg4C3yB+XYIxRnor/6B9VNapwKfu5w00dCBfq+gWOQUdi fWVcp/WaLCYJ5diFFTTCUrTfa6TuYsDEkpDU4fE8adAqHAMZW7/Uwc4NcPo6x1oUnheo S+UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747900369; x=1748505169; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/vbhBGL4NIRBV5xQKNScWF7oI/HN2DraYPV4SxmxcO8=; b=ugaFPAIiqq/4fGHS8hW1UNNbzbEzglTap86B+zjbTleHVrUf1ewUE1ByqDf/rGTSu8 Jv9LEjhvU4IyIshgwLnCNqsji6y/+v4MtMPldrPEaDJitYJW/oFWBFT8QmEI8XVYXA7O dm+rIG+X0irDJP081Yhwo04XIRTAz6EbtQpqqjxk6KEF2Gpg8nOazb0t6y2o6fRtWLyv ykhkn0bQrliCov0xXGNC08fzsT5cpOAFQP3AMo+0v5h4mYh/bk8zqpOx8t6szVPoziG3 JZeHItdvGNiv+B2DiOPNVlwfgOU09V5HvDSwuL1cxeYDOnHkle/yo96gItQPv//vRP+R W2VQ== X-Gm-Message-State: AOJu0Yzwsd3MwecZKtTAtClzcaDzdoo86Bs72heXToy1Ied2UqWXyv45 NLXFJ/0c0rQNFjjCLJ+k8l9cLuidNAZ6hyNETRplAR1043wFE2mlPzx+YGSm4H3tjubVWGNUL0v N+Q7W X-Gm-Gg: ASbGncuwBuaIKadg0f2tip58pPzsVqelTWFAXmiN0zw+OcJ7vLcAQVofj48ESZB17g3 jbvRcLNvzcKj16w/EcQ1VMjGqEy2DC7lCkmRbXmDtdxiWtKqd3KBkNfZZsXb86lkXr0Nc1UKTAs Dek881CbOFNzg0EJ0lUXhfmWaR7OQjeI88LnZ22JKqk/exUwflIUH/3AIjQEdtMcIk5uyEEqmAD 7Gnb3BTm0by/wtXKlNO6AqfZPWCWhvbjVNEf3DHBHILzLgETtvGBESc4+7i8UIgHWtXIiN2x9Le hg3nvnVUklkyG0Jo5dCeBySx+XHDnxVPsIwLbHtF2R6CbkmyvOkIILjkF6qowadXm7xoguksK5n dXwtUyw== X-Google-Smtp-Source: AGHT+IGCXxvi9bzIUXA/5e1rAFnAG4dfFNNBVaFQkP0zM0DLWkEErbWevx9f7bGbW85VAxFkzLjGNg== X-Received: by 2002:a05:6000:2307:b0:3a3:7be3:d0f9 with SMTP id ffacd0b85a97d-3a37be3d36dmr3374113f8f.1.1747900369089; Thu, 22 May 2025 00:52:49 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a3ca066eb7sm4191516f8f.2.2025.05.22.00.52.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 May 2025 00:52:48 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman Subject: [PATCH v2 3/6] pipeline: ipa: rpi: Split RPiCameraData::dropFrameCount_ Date: Thu, 22 May 2025 08:48:19 +0100 Message-ID: <20250522075244.1198110-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250522075244.1198110-1-naush@raspberrypi.com> References: <20250522075244.1198110-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Split the pipeline handler drop frame tracking into startup frames and invalid frames, as reported by the IPA. Remove the drop buffer handling logic in the pipeline handler. Now all image buffers are returned out with the appropriate FrameStatus set for startup or invalid frames. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- .../pipeline/rpi/common/pipeline_base.cpp | 98 ++++++++----------- .../pipeline/rpi/common/pipeline_base.h | 5 +- 2 files changed, 42 insertions(+), 61 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 5956485953a2..3f0b7abdc59a 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -659,9 +659,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) if (!result.controls.empty()) data->setSensorControls(result.controls); - /* Configure the number of dropped frames required on startup. */ - data->dropFrameCount_ = data->config_.disableStartupFrameDrops ? - 0 : result.startupFrameCount + result.invalidFrameCount; + /* Configure the number of startup and invalid frames reported by the IPA. */ + data->startupFrameCount_ = result.startupFrameCount; + data->invalidFrameCount_ = result.invalidFrameCount; for (auto const stream : data->streams_) stream->resetBuffers(); @@ -678,7 +678,6 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) data->buffersAllocated_ = true; } - /* We need to set the dropFrameCount_ before queueing buffers. */ ret = queueAllBuffers(camera); if (ret) { LOG(RPI, Error) << "Failed to queue buffers"; @@ -894,28 +893,12 @@ int PipelineHandlerBase::queueAllBuffers(Camera *camera) int ret; for (auto const stream : data->streams_) { - if (!(stream->getFlags() & StreamFlag::External)) { - ret = stream->queueAllBuffers(); - if (ret < 0) - return ret; - } else { - /* - * For external streams, we must queue up a set of internal - * buffers to handle the number of drop frames requested by - * the IPA. This is done by passing nullptr in queueBuffer(). - * - * The below queueBuffer() call will do nothing if there - * are not enough internal buffers allocated, but this will - * be handled by queuing the request for buffers in the - * RPiStream object. - */ - unsigned int i; - for (i = 0; i < data->dropFrameCount_; i++) { - ret = stream->queueBuffer(nullptr); - if (ret) - return ret; - } - } + if (stream->getFlags() & StreamFlag::External) + continue; + + ret = stream->queueAllBuffers(); + if (ret < 0) + return ret; } return 0; @@ -1412,7 +1395,15 @@ void CameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::Stream *stream) * buffer back to the stream. */ Request *request = requestQueue_.empty() ? nullptr : requestQueue_.front(); - if (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) { + if (request && request->findBuffer(stream) == buffer) { + FrameMetadata &md = buffer->_d()->metadata(); + + /* Mark the non-converged and invalid frames in the metadata. */ + if (invalidFrameCount_) + md.status = FrameMetadata::Status::FrameError; + else if (startupFrameCount_) + md.status = FrameMetadata::Status::FrameStartup; + /* * Tag the buffer as completed, returning it to the * application. @@ -1458,42 +1449,31 @@ void CameraData::handleState() void CameraData::checkRequestCompleted() { - bool requestCompleted = false; - /* - * If we are dropping this frame, do not touch the request, simply - * change the state to IDLE when ready. - */ - if (!dropFrameCount_) { - Request *request = requestQueue_.front(); - if (request->hasPendingBuffers()) - return; + Request *request = requestQueue_.front(); + if (request->hasPendingBuffers()) + return; - /* Must wait for metadata to be filled in before completing. */ - if (state_ != State::IpaComplete) - return; + /* Must wait for metadata to be filled in before completing. */ + if (state_ != State::IpaComplete) + return; - LOG(RPI, Debug) << "Completing request sequence: " - << request->sequence(); + LOG(RPI, Debug) << "Completing request sequence: " + << request->sequence(); - pipe()->completeRequest(request); - requestQueue_.pop(); - requestCompleted = true; - } + pipe()->completeRequest(request); + requestQueue_.pop(); - /* - * Make sure we have three outputs completed in the case of a dropped - * frame. - */ - if (state_ == State::IpaComplete && - ((ispOutputCount_ == ispOutputTotal_ && dropFrameCount_) || - requestCompleted)) { - LOG(RPI, Debug) << "Going into Idle state"; - state_ = State::Idle; - if (dropFrameCount_) { - dropFrameCount_--; - LOG(RPI, Debug) << "Dropping frame at the request of the IPA (" - << dropFrameCount_ << " left)"; - } + LOG(RPI, Debug) << "Going into Idle state"; + state_ = State::Idle; + + if (invalidFrameCount_) { + invalidFrameCount_--; + LOG(RPI, Debug) << "Decrementing invalid frames to " + << invalidFrameCount_; + } else if (startupFrameCount_) { + startupFrameCount_--; + LOG(RPI, Debug) << "Decrementing startup frames to " + << startupFrameCount_; } } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index aae0c2f35888..6023f9f9d6b3 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -48,7 +48,7 @@ class CameraData : public Camera::Private public: CameraData(PipelineHandler *pipe) : Camera::Private(pipe), state_(State::Stopped), - dropFrameCount_(0), buffersAllocated_(false), + startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false), ispOutputCount_(0), ispOutputTotal_(0) { } @@ -151,7 +151,8 @@ public: /* Mapping of CropParams keyed by the output stream order in CameraConfiguration */ std::map cropParams_; - unsigned int dropFrameCount_; + unsigned int startupFrameCount_; + unsigned int invalidFrameCount_; /* * If set, this stores the value that represets a gain of one for From patchwork Thu May 22 07:48:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23419 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 32D07C331D for ; Thu, 22 May 2025 07:53:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4998068D9B; Thu, 22 May 2025 09:52:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="cEh9Ag+g"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 620EB68D95 for ; Thu, 22 May 2025 09:52:50 +0200 (CEST) Received: by mail-wr1-x429.google.com with SMTP id ffacd0b85a97d-3a361c8a830so955264f8f.0 for ; Thu, 22 May 2025 00:52:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1747900370; x=1748505170; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PQbXpP6ym6kMA9r51U3/3ckVgPiXTqaSa9FB8oZ25C4=; b=cEh9Ag+gN+GBUDvW2KcIkLuMDc6NeksBz7vDpVgSStzUA8hRrpSwxxxkX7moGri2AB 2iW+T6aHOP+qOOUs1uf/gqKecZJMpZ9sbTLm4Lh3uIhuq67imUE4qIAcwyWNXu20e5UW NPZrPqYCMIbwmxJ0Re/inMCWkeX4PluQXN7X6UMf7G8Oc2rQmmWL4Gm4JtEFlkJMVRe9 81c6/J1FcGaS3MCXQ1wVcqbkaj6aIs/aY78BuXidNOiOaAkpMYtVAZ2PIrlPCLwIjkfG rBKsTA4K9Tc8lUHz59JHD2mhiA3Yfx52HTAt9O/WUCWlFNN3TE1jNp46Ika10JJgSEIL z/Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747900370; x=1748505170; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PQbXpP6ym6kMA9r51U3/3ckVgPiXTqaSa9FB8oZ25C4=; b=dpm/u4V/xpFl5CeSd6UQe9/jHiMyVojrmAknQD8moBc5CIT8UcGEp/obpq1yHL3VGf 8NV/a7snLERneZFkRl2EEZD29uvPJx68uK3c9dFzwUuHSZYQhi3Da4W2ovjTW6rRrbCF n68FQ9NVoIm9WJsJfbJm0dt6wnVBJ0yUKoGc5M0zrddKK0dGsDfxgicQ8D3soX+ieN92 cZK9UWOJu1Qqo05dl4alLg8rcSZ8MipiDg+qzAn/xLWC+inG33bMPsEr0Gyk5oN9Teom ROaj+jeP/dDdsqtCVYFm79gCZvZKaDdAiFxbj7S7IrPUmvzzZMxA+BofbwnOjd/vj4VS 3hRw== X-Gm-Message-State: AOJu0Yw5EW+ljaOwnE4L63KDIcRgQkmRLdQCrSKimcEzdF27uZIeOSAH drn6iulfBtFF3szfQqnyAJw+/ko0YVFh+/L/hGwmPNeEVhRlh2hOt7HYMAyLcn+bjNA9LTfhmQj nyCqL X-Gm-Gg: ASbGnctP9dkhVtJxPX2faJnd9ds70u1kYVWGC8E5d5762Cc5dhyUu9A+OS9/YUKlM3d WlWHDdSZHTuSgl2eT8AX6BLfWFVjxkJE8mIYaiIuDxNS8cI31JOGPcRonMSDTdcOnktHC+W8vDY 226MhCJZ3RaG7h2prUCpS0XnqxxRIOJIITcbWKDHuLLM45i8So6L9Hsbts4oTAiPfc/aIG4B1XZ Va1yw+oaqfhr542uM76fFm7zyI/SVTMBQ5bFEGFpB9upm3fAKRMefD0oSX3Vf7GNBX1QXxvIIOc 9lKd0/rPwCQwVZ4uK8Iv5jEiGF/hRxOjbynT3lOaKZWfIeDX5cdiywhyUj4QWWEgtjmLXOQ= X-Google-Smtp-Source: AGHT+IEgzSjIQ4mF2VDv5rkzgsdusSIdr381GPwG6V3jXa8OuBlLObH+1Wq4Azq9gdcP+Lng3OhJmA== X-Received: by 2002:a05:6000:1884:b0:3a3:73dd:92b4 with SMTP id ffacd0b85a97d-3a373dd9431mr4591291f8f.4.1747900369639; Thu, 22 May 2025 00:52:49 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a3ca066eb7sm4191516f8f.2.2025.05.22.00.52.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 May 2025 00:52:49 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman Subject: [PATCH v2 4/6] pipeline: rpi: Remove disable_startup_frame_drops config option Date: Thu, 22 May 2025 08:48:20 +0100 Message-ID: <20250522075244.1198110-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250522075244.1198110-1-naush@raspberrypi.com> References: <20250522075244.1198110-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the previous change to not drop frames in the pipeline handler, the "disable_startup_frame_drops" pipeline config option is not used. Remove it, and throw a warning if the option is present in the YAML config file. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 7 ++++--- src/libcamera/pipeline/rpi/common/pipeline_base.h | 5 ----- src/libcamera/pipeline/rpi/pisp/data/example.yaml | 5 ----- src/libcamera/pipeline/rpi/vc4/data/example.yaml | 5 ----- 4 files changed, 4 insertions(+), 18 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 3f0b7abdc59a..bef057a70353 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1078,7 +1078,6 @@ void CameraData::enumerateVideoDevices(MediaLink *link, const std::string &front int CameraData::loadPipelineConfiguration() { config_ = { - .disableStartupFrameDrops = false, .cameraTimeoutValue = 0, }; @@ -1115,8 +1114,10 @@ int CameraData::loadPipelineConfiguration() const YamlObject &phConfig = (*root)["pipeline_handler"]; - config_.disableStartupFrameDrops = - phConfig["disable_startup_frame_drops"].get(config_.disableStartupFrameDrops); + if (phConfig.contains("disable_startup_frame_drops")) + LOG(RPI, Warning) + << "The disable_startup_frame_drops key is now deprecated, " + << "please use FrameMetadata::Status::FrameStartup instead."; config_.cameraTimeoutValue = phConfig["camera_timeout_value_ms"].get(config_.cameraTimeoutValue); diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 6023f9f9d6b3..e27c4f860d1a 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -164,11 +164,6 @@ public: bool buffersAllocated_; struct Config { - /* - * Override any request from the IPA to drop a number of startup - * frames. - */ - bool disableStartupFrameDrops; /* * Override the camera timeout value calculated by the IPA based * on frame durations. diff --git a/src/libcamera/pipeline/rpi/pisp/data/example.yaml b/src/libcamera/pipeline/rpi/pisp/data/example.yaml index d67e654a8b9e..baf03be79bb3 100644 --- a/src/libcamera/pipeline/rpi/pisp/data/example.yaml +++ b/src/libcamera/pipeline/rpi/pisp/data/example.yaml @@ -16,11 +16,6 @@ # # "num_cfe_config_queue": 2, - # Override any request from the IPA to drop a number of startup - # frames. - # - # "disable_startup_frame_drops": false, - # Custom timeout value (in ms) for camera to use. This overrides # the value computed by the pipeline handler based on frame # durations. diff --git a/src/libcamera/pipeline/rpi/vc4/data/example.yaml b/src/libcamera/pipeline/rpi/vc4/data/example.yaml index b8e01adeaf40..27e543488d48 100644 --- a/src/libcamera/pipeline/rpi/vc4/data/example.yaml +++ b/src/libcamera/pipeline/rpi/vc4/data/example.yaml @@ -29,11 +29,6 @@ # # "min_total_unicam_buffers": 4, - # Override any request from the IPA to drop a number of startup - # frames. - # - # "disable_startup_frame_drops": false, - # Custom timeout value (in ms) for camera to use. This overrides # the value computed by the pipeline handler based on frame # durations. From patchwork Thu May 22 07:48:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23420 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8AD2EC31E9 for ; Thu, 22 May 2025 07:53:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 828D768DA2; Thu, 22 May 2025 09:52:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="BahoVvGZ"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EEA8968D7A for ; Thu, 22 May 2025 09:52:50 +0200 (CEST) Received: by mail-wr1-x429.google.com with SMTP id ffacd0b85a97d-3a35c339e95so939798f8f.3 for ; Thu, 22 May 2025 00:52:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1747900370; x=1748505170; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oOvFZJc72E4DVqgV4Ge8lUdO/k1QYK9oXX2vR1nJ3A0=; b=BahoVvGZtf23QvJA7T7cG182Urr3LG/yaMfp8yQkUrKinMfkSpVSapL326TEYv+vIp 5hYYkWATG5vwQB2pkRw870VQNLQL9sMGQMTdmuInQdLkN8GzWjkfd/vgoihAjG7BUiTM r+UaVB/rUEq4QmJuuBkmtxKVSSbjPF+TLfqGFIWqYnKTAb1vt7cV7sX30y4r0l3IH0Xv wHlnZIvaFmFV5x1xUffnCgr/Ozj/EBZ79zdEC+7ylAfiQTWZXEGf2pT/CvAMwiQcehoN mwyfE6IM8zIZfm2AQJP0s0Xs+2/YA+vUf2ZnGWEqrRewbulcW3Jg/nz5vfyOG+gmRQtN POPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747900370; x=1748505170; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oOvFZJc72E4DVqgV4Ge8lUdO/k1QYK9oXX2vR1nJ3A0=; b=hd8qqJYa1Lnkc/tY3vKl3PU9UlT93D8nFN4dw4iwKkaNUotHyQw1asW0Oaz/4Asxfq pTZctXfqGfn5V34XScf7FynDtcg6d6I3bDIaK0LgKf6wVxZMeSxvAah4tq9oEQ0denNw jASwXrtx6Jhxs93M4i1R6n0oyNjFHnQPEssCavIYvC1HWZXcF950XuZhULciypdt04Or gdDn41IvjDXZR+x7HKJfF6k5+MIgZtZgUxYU6wPGpRyOWX/131JU/bMNycNQRaFah4vr FBfVQeAawwplma58fvie2IwmX1fIS0TjyO8e3JFqOe6k7Kaeafr2rFQ2PIFPys4cjMCR LxdA== X-Gm-Message-State: AOJu0YxenhNJA7qtlaUnqZfdE/kOOg8IWIWcx+ekEnMuYMrrLvljCjHs Nysu1fKRoF7PSYcDFOIOS7L7K9Qeshpxp87wJkn18+ldkyN7Ksani8t0dGjDI7mBiv/VHzvaRiD Shtlf X-Gm-Gg: ASbGnctmf5jMiNmMunoNgUUfC4o5Tjtq0W8iY3MYztEGf55PtAqf+Fbw+GiJ37lw+Hs 0x4lgxNQKN2FpXDVY7gRm9MAZQFZrdEumPvsgyUEezMregeca4DZVxdCNorAl+vhQ9JvZlPAGEE KnlGni2fmnedKisJmkdCVIXk9jmz4DPJvdzkWq65EqncndTiqvsS1OCuq3/5skeCaBW/YHPWuVw /WUx7ZV/Iy8Cr0ikD5RkQfcz8KGmPEsGWILRSoFLtBA4D9flNRAjXrvEmdm5wGqB2U1bvAsbIMf fl8GL+tx8VecJkONJcyPfUHopEWE9NIawvTujxhbEwO6UP+Ep08niH3615jHYQ085u6t+PI= X-Google-Smtp-Source: AGHT+IFejCRPQPwZ22+pMSwxkSO4e98Jdm3FUi0aU/gIYWFTFYMvyyASpXPr+V2s6ykTek974606WQ== X-Received: by 2002:a5d:5848:0:b0:3a0:99a4:e53f with SMTP id ffacd0b85a97d-3a35c843e2cmr6919565f8f.8.1747900370246; Thu, 22 May 2025 00:52:50 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a3ca066eb7sm4191516f8f.2.2025.05.22.00.52.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 May 2025 00:52:49 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman Subject: [PATCH v2 5/6] pipeline: rpi: Remove ispOutputCount_ and ispOutputTotal_ Date: Thu, 22 May 2025 08:48:21 +0100 Message-ID: <20250522075244.1198110-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250522075244.1198110-1-naush@raspberrypi.com> References: <20250522075244.1198110-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the drop frame logic removed from the pipeline handler, these member variables and not used, so remove them. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/libcamera/pipeline/rpi/common/pipeline_base.h | 6 +----- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 12 ------------ src/libcamera/pipeline/rpi/vc4/vc4.cpp | 15 --------------- 3 files changed, 1 insertion(+), 32 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index e27c4f860d1a..898f31577059 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -48,8 +48,7 @@ class CameraData : public Camera::Private public: CameraData(PipelineHandler *pipe) : Camera::Private(pipe), state_(State::Stopped), - startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false), - ispOutputCount_(0), ispOutputTotal_(0) + startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false) { } @@ -179,9 +178,6 @@ protected: virtual void tryRunPipeline() = 0; - unsigned int ispOutputCount_; - unsigned int ispOutputTotal_; - private: void checkRequestCompleted(); }; diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index 91e7f4c94d96..ccf135c3d8ce 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -1834,12 +1834,6 @@ void PiSPCameraData::beOutputDequeue(FrameBuffer *buffer) dmabufSyncEnd(buffer->planes()[0].fd); handleStreamBuffer(buffer, stream); - - /* - * Increment the number of ISP outputs generated. - * This is needed to track dropped frames. - */ - ispOutputCount_++; handleState(); } @@ -1885,7 +1879,6 @@ void PiSPCameraData::prepareIspComplete(const ipa::RPi::BufferIds &buffers, bool * If there is no need to run the Backend, just signal that the * input buffer is completed and all Backend outputs are ready. */ - ispOutputCount_ = ispOutputTotal_; buffer = cfe_[Cfe::Output0].getBuffers().at(bayerId).buffer; handleStreamBuffer(buffer, &cfe_[Cfe::Output0]); } else @@ -1994,7 +1987,6 @@ int PiSPCameraData::configureBe(const std::optional &yuvColorSpace) global.bayer_enables |= PISP_BE_BAYER_ENABLE_INPUT; global.bayer_order = toPiSPBayerOrder(cfeFormat.fourcc); - ispOutputTotal_ = 1; /* Config buffer */ if (PISP_IMAGE_FORMAT_COMPRESSED(inputFormat.format)) { pisp_decompress_config decompress; decompress.offset = DefaultCompressionOffset; @@ -2025,7 +2017,6 @@ int PiSPCameraData::configureBe(const std::optional &yuvColorSpace) setupOutputClipping(ispFormat0, outputFormat0); be_->SetOutputFormat(0, outputFormat0); - ispOutputTotal_++; } if (global.rgb_enables & PISP_BE_RGB_ENABLE_OUTPUT1) { @@ -2049,7 +2040,6 @@ int PiSPCameraData::configureBe(const std::optional &yuvColorSpace) setupOutputClipping(ispFormat1, outputFormat1); be_->SetOutputFormat(1, outputFormat1); - ispOutputTotal_++; } /* Setup the TDN I/O blocks in case TDN gets turned on later. */ @@ -2256,8 +2246,6 @@ void PiSPCameraData::prepareCfe() void PiSPCameraData::prepareBe(uint32_t bufferId, bool stitchSwapBuffers) { - ispOutputCount_ = 0; - FrameBuffer *buffer = cfe_[Cfe::Output0].getBuffers().at(bufferId).buffer; LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << bufferId diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index fe910bdf2ff9..ac6dab814d35 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -597,8 +597,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi stream->setFlags(StreamFlag::External); } - ispOutputTotal_ = outStreams.size(); - /* * If ISP::Output0 stream has not been configured by the application, * we must allow the hardware to generate an output so that the data @@ -625,8 +623,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi return -EINVAL; } - ispOutputTotal_++; - LOG(RPI, Debug) << "Defaulting ISP Output0 format to " << format; } @@ -662,8 +658,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi << ret; return -EINVAL; } - - ispOutputTotal_++; } /* ISP statistics output format. */ @@ -676,8 +670,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi return ret; } - ispOutputTotal_++; - /* * Configure the Unicam embedded data output format only if the sensor * supports it. @@ -843,12 +835,6 @@ void Vc4CameraData::ispOutputDequeue(FrameBuffer *buffer) handleStreamBuffer(buffer, stream); } - /* - * Increment the number of ISP outputs generated. - * This is needed to track dropped frames. - */ - ispOutputCount_++; - handleState(); } @@ -880,7 +866,6 @@ void Vc4CameraData::prepareIspComplete(const ipa::RPi::BufferIds &buffers, << ", timestamp: " << buffer->metadata().timestamp; isp_[Isp::Input].queueBuffer(buffer); - ispOutputCount_ = 0; if (sensorMetadata_ && embeddedId) { buffer = unicam_[Unicam::Embedded].getBuffers().at(embeddedId & RPi::MaskID).buffer; From patchwork Thu May 22 07:48:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23421 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C7097C331D for ; Thu, 22 May 2025 07:53:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 50FAC68D9C; Thu, 22 May 2025 09:53:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="q7KqahLx"; dkim-atps=neutral Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F37D668D9A for ; Thu, 22 May 2025 09:52:51 +0200 (CEST) Received: by mail-wr1-x431.google.com with SMTP id ffacd0b85a97d-3a365bc0af8so559999f8f.3 for ; Thu, 22 May 2025 00:52:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1747900371; x=1748505171; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LZqiQlR9owO1wtt/qZunnLktQ2lr/L7FkNTRGEyHuSo=; b=q7KqahLxI3A8LNi8VO7RTPpIPHyKNnJMpigQjsRIo/WyZ95s63wn+UA0AJxGkJW4Nw 07/d6PKBapnxGef34RHyghOUNEw4eTti80LPhpwwmXYakLgcM4IeRCpooc4ifXqdhpbM 7utKGtx3K4JjfBou+1R/IUGiCNiWTfgHviYSZwyQdavW4duhR7uHo1C4rjBuu9oFuZlj A9JauiZjySax2VmohJGOAznk9z6IQktp9AJCtlrceOlPAK/vja5CITx51lEkov20jkjP qLQE8GgDD4B+7ZhbTex9Si43Mj1TbT64WLnsrHKCtf2QFr6kvTAN7ExX5maqg+peytjQ b+4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747900371; x=1748505171; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LZqiQlR9owO1wtt/qZunnLktQ2lr/L7FkNTRGEyHuSo=; b=D/2DvNefCAUlxRLQH9QllcrXQ9ATN9rnhp/QDYwN3tT6SW0t83WsUCI+N87ZPBx0/2 SN0+TV40VRCjntv/wF6yoDLwXzqqRfaVHkKnr9yEXQkdgLDb8G4goLNuSpyjemRaCNIb em5cOSZdwS3Ffe6malzNxkWApCM2e6i7dNGGYScwP0HDYA67FODufFPNukWBx661D23N Azb4ZH8SbcsJX0O36zXq8W8OMpeutkcg2BUdLvD4IvkZFmprKy5400BxiVlPNvGnRIHS WCvzK8qkGkoeqCsEy0RFnfJKiVj+ik+Ix2mKed42XsU5sAoQC4fSMkrAWpgf5gKaLEJS 1DuA== X-Gm-Message-State: AOJu0YxUTrfgNslSlbMX50Uu2Ppqgk5KGSWuuk6tujgriib2M60NR32w 4EfwdNX4N3m97kGKbxfSjIF99qYzfHeH9z6Hd7JFd4hmvcDTCwOetZ5nU7+sYcL1tkStoAHivDH YKV0B X-Gm-Gg: ASbGnct+JhTUQr23rjZbC5IGK+IwYM8y+wJtdzkv4jJXhf/X3BKrPZslGJiaUgKQUqH 1WhN7+WCViIyKtPtdlwsv0dZG9WNufV6vaEWHgJzoUTSg+n1u8EvHuHpHP00n7M8GmbEct+zvha xg3Px/QDLxLUumBI6tMkwYSkk29Mhmd1SdabZMWNWJPwIZyizM7xB+Oz2dFRkagioKuATWXx8Md 9ClTmt/ZsPcxuVz6A6oOb2e+TvCOgU/dItp+asN1CYyXnq/Z+Za6kecsxV4DkdAT2CLzYTeQsjG 62TQTQsuxFZ/JuYD3iij+im3X8Xly7v56bLExnpIfd6aZ8F7z5lphwvfP5GZS+3VqQ2s8lE= X-Google-Smtp-Source: AGHT+IF8EHRy4DXetsyS78VNd/9WTw78b9zq3+ReMWJtFSYv66fPAamWEBrISf45bENRs4+jkw+STg== X-Received: by 2002:a05:6000:2284:b0:3a3:6e0e:a09d with SMTP id ffacd0b85a97d-3a36e0ea15dmr5782700f8f.6.1747900370791; Thu, 22 May 2025 00:52:50 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a3ca066eb7sm4191516f8f.2.2025.05.22.00.52.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 May 2025 00:52:50 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman Subject: [PATCH v2 6/6] ipa: rpi: Rename dropFrameCount_ to startupCount_ Date: Thu, 22 May 2025 08:48:22 +0100 Message-ID: <20250522075244.1198110-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250522075244.1198110-1-naush@raspberrypi.com> References: <20250522075244.1198110-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rename dropFrameCount_ to startupCount_ to better reflect its use as frames are no longer dropped by the pipeline handler. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/rpi/common/ipa_base.cpp | 10 +++++----- src/ipa/rpi/common/ipa_base.h | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index c15f8a7bf71e..8d591faeceaa 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -327,7 +327,7 @@ void IpaBase::start(const ControlList &controls, StartResult *result) unsigned int agcConvergenceFrames = 0, awbConvergenceFrames = 0; frameCount_ = 0; if (firstStart_) { - dropFrameCount_ = helper_->hideFramesStartup(); + startupCount_ = helper_->hideFramesStartup(); mistrustCount_ = helper_->mistrustFramesStartup(); /* @@ -353,14 +353,14 @@ void IpaBase::start(const ControlList &controls, StartResult *result) awbConvergenceFrames += mistrustCount_; } } else { - dropFrameCount_ = helper_->hideFramesModeSwitch(); + startupCount_ = helper_->hideFramesModeSwitch(); mistrustCount_ = helper_->mistrustFramesModeSwitch(); } result->startupFrameCount = std::max({ agcConvergenceFrames, awbConvergenceFrames }); - result->invalidFrameCount = dropFrameCount_; + result->invalidFrameCount = startupCount_; - dropFrameCount_ = std::max({ dropFrameCount_, agcConvergenceFrames, awbConvergenceFrames }); + startupCount_ = std::max({ startupCount_, agcConvergenceFrames, awbConvergenceFrames }); LOG(IPARPI, Debug) << "Startup frames: " << result->startupFrameCount << " Invalid frames: " << result->invalidFrameCount; @@ -443,7 +443,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) /* Allow a 10% margin on the comparison below. */ Duration delta = (frameTimestamp - lastRunTimestamp_) * 1.0ns; - if (lastRunTimestamp_ && frameCount_ > dropFrameCount_ && + if (lastRunTimestamp_ && frameCount_ > startupCount_ && delta < controllerMinFrameDuration * 0.9 && !hdrChange) { /* * Ensure we merge the previous frame's metadata with the current diff --git a/src/ipa/rpi/common/ipa_base.h b/src/ipa/rpi/common/ipa_base.h index 1a811beb31f2..a51afc156a8f 100644 --- a/src/ipa/rpi/common/ipa_base.h +++ b/src/ipa/rpi/common/ipa_base.h @@ -115,8 +115,8 @@ private: /* How many frames we should avoid running control algos on. */ unsigned int mistrustCount_; - /* Number of frames that need to be dropped on startup. */ - unsigned int dropFrameCount_; + /* Number of frames that need to be marked as dropped on startup. */ + unsigned int startupCount_; /* Frame timestamp for the last run of the controller. */ uint64_t lastRunTimestamp_;