From patchwork Mon May 19 09:20:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23386 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2B90BBD78E for ; Mon, 19 May 2025 09:22:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0330A68D83; Mon, 19 May 2025 11:22:54 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="h/5VoRLR"; dkim-atps=neutral Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 32ABC68B66 for ; Mon, 19 May 2025 11:22:52 +0200 (CEST) Received: by mail-wr1-x435.google.com with SMTP id ffacd0b85a97d-3a374754e03so15388f8f.1 for ; Mon, 19 May 2025 02:22:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1747646571; x=1748251371; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Zd5df1quEaR166Fq4cu5Y3EUJiRzpvBQBh9y5xYqZPs=; b=h/5VoRLRFSyQe+1Ort3JrtAbNQnUlOyef82a2IkmsXsjtIzJ52eCX7oE4hxYSUjJxI 2svevYPO07B4/D2KvgnI2Nmc5c2p/qOs+qgkoUQWxbBxjxGpmdJbtirungP+Ta7kcvms Lkgle/++izaP4s1KeI12NZi5F9JevQ4pAHI18XjD3m5XwhK7afB1hvYLmdNjuh4okAEg hZNbCsVAiYUn1/40tM1JhJfubWLNQBFo4vWuCSxDy9Foi357mK7svB13scFko7bcNP81 u94TrGQKlmo3wm0Di3MxDIcLqH0KHIcwhkSIZlUgYQ11AS6qS4WAXvKfG8WtZiruC10W zLIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747646571; x=1748251371; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Zd5df1quEaR166Fq4cu5Y3EUJiRzpvBQBh9y5xYqZPs=; b=t3HN0x1M2AsiB/m295HA/+PGyo2sh624noUHis5fiDUQkUa4ezqzT773MYqMfy3ggV GljQ8DKHLhO9K51jvNier3bImGt8IMgrTlvVYrkdcBzx7rfmBtvhmf8ne8e5twocAz6Z 7rfewB4/ZQd/sOjuvWd+yXU1qT5B1pZ46GZw8+KiNgBroya2bVI2ASxb8zFULz7TnzWG LfnejfiCOjWSLMzot11/J7F1BnCFtPiaUIWYTPonvUkrEAROTGjgk906s9hqfO4B6juW 3yjBpvX4VC5xRfi4JkMMaj5E6l3eXs36oZ4+2XWaTaIwVCoX3d7LBcwrFeiAUt+XazKW b7VA== X-Gm-Message-State: AOJu0YyKuNewrd1kVTWIR4gztt2OhXF1lok//kvqbGYSbEZO2xlIlTzA /xiigD3fUtUipGnp1zOOA3V+J4bHRKVE76Oiacaz2k4CUAFWgr3z9dEEKbErrK0jA+2VFWaby/Z aAD5e X-Gm-Gg: ASbGncuXmUOgL8UvyPPHkLSDKkKmI7BomGs4MH+bHzLrzCvD05BASxYvIpJt1uGOD4r I9L5XNHv3L2IAjxDgSonJC4cuuB17igb2kGdZZDkMtwv+cjgZzH8ILAvixCGwMj7MglcPOU0kIH YJdeRIm0k/jJJ65ai+G6flL6Rti48gKwSrrqqg9FUjU5W5hGPP1H4GhHu2VEBdSG3QHd/YdJJFJ 7seyWO9dgs2AonYbVmyhWJ3vxVuo1JUK1+Ls4Iq3kKN1r0I6bbxkJKCasScKT2Q6xfu13j4S6jQ a1dStCB8E7dhqnvou88uoie0LDyuf3/q27+vXssLT+UqNfeZFoGhu3sHPvSu96mae+HTBfE= X-Google-Smtp-Source: AGHT+IGH0jjMMjxZatKIOHlGIhGeB6XfaUzBtNyhsIC2uUcITvRjiMdiddJBL6Qf8nKvs4aKk2JufQ== X-Received: by 2002:a05:6000:3112:b0:39f:728:4324 with SMTP id ffacd0b85a97d-3a35c825f55mr3565664f8f.9.1747646571527; Mon, 19 May 2025 02:22:51 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a366e08747sm8645961f8f.95.2025.05.19.02.22.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 May 2025 02:22:51 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [PATCH v1 1/6] libcamera: framebuffer: Add FrameMetadata::Status::FrameStartup Date: Mon, 19 May 2025 10:20:49 +0100 Message-ID: <20250519092245.269048-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250519092245.269048-1-naush@raspberrypi.com> References: <20250519092245.269048-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new status enum, FrameStartup, used to denote that even though the frame has been successfully captured, the IQ parameters set by the IPA will cause the frame to be unusable and applications are advised to not consume this frame. An example of this would be on a cold-start of the 3A algorithms, and there will be large oscillations to converge to a stable state quickly. Additional, update the definition of the FrameError state to include its usage when the sensor is known to produce a number of invalid/error frames after stream-on. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- include/libcamera/framebuffer.h | 1 + src/libcamera/framebuffer.cpp | 11 +++++++++-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/include/libcamera/framebuffer.h b/include/libcamera/framebuffer.h index ff83924300ac..e83825b466aa 100644 --- a/include/libcamera/framebuffer.h +++ b/include/libcamera/framebuffer.h @@ -26,6 +26,7 @@ struct FrameMetadata { FrameSuccess, FrameError, FrameCancelled, + FrameStartup, }; struct Plane { diff --git a/src/libcamera/framebuffer.cpp b/src/libcamera/framebuffer.cpp index 826848f75a56..36e56d593bc3 100644 --- a/src/libcamera/framebuffer.cpp +++ b/src/libcamera/framebuffer.cpp @@ -44,11 +44,18 @@ LOG_DEFINE_CATEGORY(Buffer) * of the FrameMetadata structure are valid. * \var FrameMetadata::FrameError * An error occurred during capture of the frame. The frame data may be partly - * or fully invalid. The sequence and timestamp fields of the FrameMetadata - * structure is valid, the other fields may be invalid. + * or fully invalid. This status may also indicate an invalid frame produced by + * the sensor during its startup or restart phase. The sequence and timestamp + * fields of the FrameMetadata structure is valid, the other fields may be + * invalid. * \var FrameMetadata::FrameCancelled * Capture stopped before the frame completed. The frame data is not valid. All * fields of the FrameMetadata structure but the status field are invalid. + * \var FrameMetadata::FrameStartup The frame has been successfully captured. + * However, the IPA is in a cold-start or reset phase and will result in image + * quality parameters producing unusable images. Applications are recommended to + * not consume these frames. All fields of the FrameMetadata structure are + * valid. */ /** From patchwork Mon May 19 09:20:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23387 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0DE47BD78E for ; Mon, 19 May 2025 09:23:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3C95768D7D; Mon, 19 May 2025 11:22:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="UY41eqT0"; dkim-atps=neutral Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DA40968B66 for ; Mon, 19 May 2025 11:22:52 +0200 (CEST) Received: by mail-wr1-x42c.google.com with SMTP id ffacd0b85a97d-3a362e099cfso400356f8f.2 for ; Mon, 19 May 2025 02:22:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1747646572; x=1748251372; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fnaMAqT31Sn1HyYeUSz/LRhNsF9NQEXQj8bpQHhMNn0=; b=UY41eqT0smcRWRSd3LFzmd89npTGGMe6Q1NjR5O62G9oYKYpP1zEv+HCDNM3pMKMbg 5YnB/uGy/bZQcCx7TkD97Yz3CJjID6+/hpe65VXDeZ0bMA50y3NknEwoqw01UFKZ13fb PWim7kuzem5yU8vHpBY2l2KOsu3bWwUWSmiTPpqcymSH3UFVtAGjacwulIt5GpAwx3qV YWaIQ0qc+APajzuGWnI5VP6x0LTOIpbREAKa3ZtHzEVV9s+PPhkQLyh016+kP6HlYAjW F8cVZTZQTycukYdL93f7q6Lj8UVJ9+Tx6So9WObrh98EP+0zor1PHIdiQDyCbx/5K2cz qrLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747646572; x=1748251372; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fnaMAqT31Sn1HyYeUSz/LRhNsF9NQEXQj8bpQHhMNn0=; b=gCI2DPXGepkiOexk4drZeA+0/TbZ+IWY9gJT9uHHPYdI7M0PpfTyXxxys0fePogj05 eR0iYesnubYAQm2lDg30BsSeGSfP2eM71R4xVjKQR0ibGf2mzAF+4EH7OyHF6uzMeIVE HDKjkW/lX2kpY9UF2W3ykPr3MYWM+JokHw3hPvKrFs+MjfODL77chSKQEGHrqWzNqLL9 5Vn2DRXALYykMcrZaRZCeEUGG7P251KVX4sc73jslFy1VgIzIDyVbcGnpg6JKouh6jhS F1YPkNL+dG1lvQJdPZURTg2kZeEFnitPsjEwoXbpzeC3Smx16qI4E47VbMbN4HdkIPtS fsCQ== X-Gm-Message-State: AOJu0YzFsivEAE6mly7lpYsn24/CqSU9lrg1fh2dWOipwg1bBB1UowbP vXV+S6YgV9KCTajnEOrbOPlDigKxOjb4n3C0cYXLMwf3j7P8rWPrd/hBOT7pjx8AJXkKeNCQH/y Hcpum X-Gm-Gg: ASbGncsVReOcYAiBfqckBRCMkE1XnhOw+8wu1WNWwOffuuaU6ywKR7vHxwkPLzz8vOw f+evlifehzd0ya0IUFowx9RwtpSDxIkiCHzl7foOHV8Z//lsh17d0T30vBe5U54KVtXuzCHK1Ba LCGm+anPLC8hTQqTpwD9Gytek2gULOI8ZOxV60sDKjUZO3xBn2KyVEcYHx7HLIIQyOjUC0IEd4d CGuqbe3qAA9pZRExiTbq9tP7H/lP0LgEByOtnYw/8nfHzp6fYKTTo6tYVy+DteV4R9QVblO47r8 TcrzHHHnwRvFrGF/+RkCwu35olSsF/xJrhWkVBfck85p8DeVWKqkQwjvIdrt7mIHPTerLOw= X-Google-Smtp-Source: AGHT+IFNSWXSO7SVzY8UXBRz30mcxBjKr8b4W3bugqa6AxWLA4JYnt75K+Ag4HRSzYyHwvF+C9qkqQ== X-Received: by 2002:a5d:64c5:0:b0:3a3:6964:57c8 with SMTP id ffacd0b85a97d-3a369645a0fmr2005956f8f.5.1747646572107; Mon, 19 May 2025 02:22:52 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a366e08747sm8645961f8f.95.2025.05.19.02.22.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 May 2025 02:22:51 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [PATCH v1 2/6] ipa: rpi: Replace dropFrameCount in the IPA -> PH interface Date: Mon, 19 May 2025 10:20:50 +0100 Message-ID: <20250519092245.269048-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250519092245.269048-1-naush@raspberrypi.com> References: <20250519092245.269048-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Replace the dropFrameCount parameter returned from ipa::start() to the pipeline handler by startupFrameCount and invalidFrameCount. The former counts the number of frames required for AWB/AGC to converge, and the latter counts the number of invalid frames produced by the sensor when starting up. In the pipeline handler, use the sum of these 2 values to replicate the existing dropFrameCount behaviour. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- include/libcamera/ipa/raspberrypi.mojom | 3 ++- src/ipa/rpi/common/ipa_base.cpp | 14 ++++++++------ .../pipeline/rpi/common/pipeline_base.cpp | 4 ++-- 3 files changed, 12 insertions(+), 9 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index e30c70bde08b..12b083e9d04b 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -52,7 +52,8 @@ struct ConfigResult { struct StartResult { libcamera.ControlList controls; - int32 dropFrameCount; + int32 startupFrameCount; + int32 invalidFrameCount; }; struct PrepareParams { diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index e0a93daa9db2..c15f8a7bf71e 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -324,6 +324,7 @@ void IpaBase::start(const ControlList &controls, StartResult *result) * "mistrusted", which depends on whether this is a startup from cold, * or merely a mode switch in a running system. */ + unsigned int agcConvergenceFrames = 0, awbConvergenceFrames = 0; frameCount_ = 0; if (firstStart_) { dropFrameCount_ = helper_->hideFramesStartup(); @@ -336,7 +337,6 @@ void IpaBase::start(const ControlList &controls, StartResult *result) * (mistrustCount_) that they won't see. But if zero (i.e. * no convergence necessary), no frames need to be dropped. */ - unsigned int agcConvergenceFrames = 0; RPiController::AgcAlgorithm *agc = dynamic_cast( controller_.getAlgorithm("agc")); if (agc) { @@ -345,7 +345,6 @@ void IpaBase::start(const ControlList &controls, StartResult *result) agcConvergenceFrames += mistrustCount_; } - unsigned int awbConvergenceFrames = 0; RPiController::AwbAlgorithm *awb = dynamic_cast( controller_.getAlgorithm("awb")); if (awb) { @@ -353,15 +352,18 @@ void IpaBase::start(const ControlList &controls, StartResult *result) if (awbConvergenceFrames) awbConvergenceFrames += mistrustCount_; } - - dropFrameCount_ = std::max({ dropFrameCount_, agcConvergenceFrames, awbConvergenceFrames }); - LOG(IPARPI, Debug) << "Drop " << dropFrameCount_ << " frames on startup"; } else { dropFrameCount_ = helper_->hideFramesModeSwitch(); mistrustCount_ = helper_->mistrustFramesModeSwitch(); } - result->dropFrameCount = dropFrameCount_; + result->startupFrameCount = std::max({ agcConvergenceFrames, awbConvergenceFrames }); + result->invalidFrameCount = dropFrameCount_; + + dropFrameCount_ = std::max({ dropFrameCount_, agcConvergenceFrames, awbConvergenceFrames }); + + LOG(IPARPI, Debug) << "Startup frames: " << result->startupFrameCount + << " Invalid frames: " << result->invalidFrameCount; firstStart_ = false; lastRunTimestamp_ = 0; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 1f13e5230fae..5956485953a2 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -660,8 +660,8 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) data->setSensorControls(result.controls); /* Configure the number of dropped frames required on startup. */ - data->dropFrameCount_ = data->config_.disableStartupFrameDrops - ? 0 : result.dropFrameCount; + data->dropFrameCount_ = data->config_.disableStartupFrameDrops ? + 0 : result.startupFrameCount + result.invalidFrameCount; for (auto const stream : data->streams_) stream->resetBuffers(); From patchwork Mon May 19 09:20:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23388 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D6222C322E for ; Mon, 19 May 2025 09:23:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E72FA68C91; Mon, 19 May 2025 11:22:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="enwqg//h"; dkim-atps=neutral Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8263368D80 for ; Mon, 19 May 2025 11:22:53 +0200 (CEST) Received: by mail-wr1-x436.google.com with SMTP id ffacd0b85a97d-3a365bc0af8so259441f8f.3 for ; Mon, 19 May 2025 02:22:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1747646573; x=1748251373; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=85vcpWZxa6mtEJPyiFwMvAsazIU0BVD2UXwGQZLD+8o=; b=enwqg//h/yq1Sdo/JO8Jf0P53JnQgWVrndpVeGElYcSJoSzB41XgksNcvaPDB3daI8 kQvx0VghWxabWlJngfzuRxoW3y4o6lUgSZGAXVWcYLK4TPuu+wD594raWweQeMU5GtCV kFPXbVe1bs8HiYwsgZUFT4hRYhZI3URg0Y11d409UZQJ0JiW57vMQ8j+JtrqYvEB9+LL gBv6YRiB78la2U41wVlCDXNAidQzpw8o14OQ4ljWSo6a1sM7TPWiCFi5BfRNklmbNjGs cNHtsFp4p2zoSwcPYITs3VCqcRcxXzZVrpap8BZ04E0oozB0HzwUKGHMp2+fhf6tfyi4 VbLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747646573; x=1748251373; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=85vcpWZxa6mtEJPyiFwMvAsazIU0BVD2UXwGQZLD+8o=; b=mxrPcPewB773TKPaLcQwSmkIasWZTApKQymwRAlxq74frGuD44LVI233TYDyNoPuE0 rzeaMVb7/e8TfFGvc3silNnnZH94MrG82bhei8UXdGJzSRKYN9N3P+7j5JL765r3gYYg xsJkb5PpiNZ28qcsfBvICDrF2vq3pim6wWyIXtAYvFU7/na02m5ynfX+x6Y38R1bwX+O iOGUVpBcdLXoJxooLepdbkdgQZaH9pNamldYkofT6airt0WlXFkmNwojxnaSZKwQcpW8 Kf51nSkjTc7T+C32cqre8ZUOSC/5FQlpR2blSY/QAnEWiv0FdPANexQv0kCmcSb0Y/vF lcNw== X-Gm-Message-State: AOJu0Yy3eyp/5GNNY8iaJUOGUoA2ChY1G6BFRYZwzh1P4YDuzEYi5j2j GlLDuhyPEhJZeBvJSMzrJ9xsgCRtkoGxpkzbkygqogrRPiGVelUyv7VN6eQQuMZ8jiD9QH019Q4 oskdz X-Gm-Gg: ASbGncsAQ3Xa14iK6MZCcl5xhu0Llhmk3tCAnns+7dcPl3P4rciLLvhguc0fPD5qivn YolhiyIfMiFls5j2rBKSOzkxHWgJ1aJhyjBvgncwGhoMnP6WkwhDzJM0JL3YzSPYleSRxHy+6JR IX0mD1Yxw8YiXVuJd95GTdwfSJ+xqBP9fgNBANO2RD/xRWBFSNASif0cqFhkLtDoowRoFehTdrZ BnRhgfxUb3UrOkMlircJluOFXg/QRbwK4xAtPmb4nzVFuiDh/Yjg1Crh3ADtiHh7gofXQabK6UC 8ce+j3EX5ob4srReLoBcsxLwRurw132PsP7NV187vseZCQF72wfTi3KSvD0ibSSYzsh97jY= X-Google-Smtp-Source: AGHT+IEUeIzVYBaXlphr4CKcpPM31oDxnW7w5WZF9G6M/+VeDpegRRh6rCKi34TVnOj+4bIlPz06xg== X-Received: by 2002:a5d:404a:0:b0:3a3:62e1:a7dc with SMTP id ffacd0b85a97d-3a362e1ab39mr2706273f8f.1.1747646572691; Mon, 19 May 2025 02:22:52 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a366e08747sm8645961f8f.95.2025.05.19.02.22.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 May 2025 02:22:52 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [PATCH v1 3/6] pipeline: ipa: rpi: Split RPiCameraData::dropFrameCount_ Date: Mon, 19 May 2025 10:20:51 +0100 Message-ID: <20250519092245.269048-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250519092245.269048-1-naush@raspberrypi.com> References: <20250519092245.269048-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Split the pipeline handler drop frame tracking into startup frames and invalid frames, as reported by the IPA. Remove the drop buffer handling logic in the pipeline handler. Now all image buffers are returned out with the appropriate FrameStatus set for startup or invalid frames. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- .../pipeline/rpi/common/pipeline_base.cpp | 88 +++++++------------ .../pipeline/rpi/common/pipeline_base.h | 5 +- 2 files changed, 37 insertions(+), 56 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 5956485953a2..21f2daf5bab5 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -659,9 +659,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) if (!result.controls.empty()) data->setSensorControls(result.controls); - /* Configure the number of dropped frames required on startup. */ - data->dropFrameCount_ = data->config_.disableStartupFrameDrops ? - 0 : result.startupFrameCount + result.invalidFrameCount; + /* Configure the number of startup and invalid frames reported by the IPA. */ + data->startupFrameCount_ = result.startupFrameCount; + data->invalidFrameCount_ = result.invalidFrameCount; for (auto const stream : data->streams_) stream->resetBuffers(); @@ -678,7 +678,6 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) data->buffersAllocated_ = true; } - /* We need to set the dropFrameCount_ before queueing buffers. */ ret = queueAllBuffers(camera); if (ret) { LOG(RPI, Error) << "Failed to queue buffers"; @@ -898,23 +897,6 @@ int PipelineHandlerBase::queueAllBuffers(Camera *camera) ret = stream->queueAllBuffers(); if (ret < 0) return ret; - } else { - /* - * For external streams, we must queue up a set of internal - * buffers to handle the number of drop frames requested by - * the IPA. This is done by passing nullptr in queueBuffer(). - * - * The below queueBuffer() call will do nothing if there - * are not enough internal buffers allocated, but this will - * be handled by queuing the request for buffers in the - * RPiStream object. - */ - unsigned int i; - for (i = 0; i < data->dropFrameCount_; i++) { - ret = stream->queueBuffer(nullptr); - if (ret) - return ret; - } } } @@ -1412,7 +1394,15 @@ void CameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::Stream *stream) * buffer back to the stream. */ Request *request = requestQueue_.empty() ? nullptr : requestQueue_.front(); - if (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) { + if (request && request->findBuffer(stream) == buffer) { + FrameMetadata &md = buffer->_d()->metadata(); + + /* Mark the non-converged and invalid frames in the metadata. */ + if (startupFrameCount_) + md.status = FrameMetadata::Status::FrameStartup; + if (invalidFrameCount_) + md.status = FrameMetadata::Status::FrameError; + /* * Tag the buffer as completed, returning it to the * application. @@ -1458,42 +1448,32 @@ void CameraData::handleState() void CameraData::checkRequestCompleted() { - bool requestCompleted = false; - /* - * If we are dropping this frame, do not touch the request, simply - * change the state to IDLE when ready. - */ - if (!dropFrameCount_) { - Request *request = requestQueue_.front(); - if (request->hasPendingBuffers()) - return; + Request *request = requestQueue_.front(); + if (request->hasPendingBuffers()) + return; - /* Must wait for metadata to be filled in before completing. */ - if (state_ != State::IpaComplete) - return; + /* Must wait for metadata to be filled in before completing. */ + if (state_ != State::IpaComplete) + return; - LOG(RPI, Debug) << "Completing request sequence: " - << request->sequence(); + LOG(RPI, Debug) << "Completing request sequence: " + << request->sequence(); - pipe()->completeRequest(request); - requestQueue_.pop(); - requestCompleted = true; - } + pipe()->completeRequest(request); + requestQueue_.pop(); - /* - * Make sure we have three outputs completed in the case of a dropped - * frame. - */ - if (state_ == State::IpaComplete && - ((ispOutputCount_ == ispOutputTotal_ && dropFrameCount_) || - requestCompleted)) { - LOG(RPI, Debug) << "Going into Idle state"; - state_ = State::Idle; - if (dropFrameCount_) { - dropFrameCount_--; - LOG(RPI, Debug) << "Dropping frame at the request of the IPA (" - << dropFrameCount_ << " left)"; - } + LOG(RPI, Debug) << "Going into Idle state"; + state_ = State::Idle; + + if (startupFrameCount_) { + startupFrameCount_--; + LOG(RPI, Debug) << "Decrementing startup frames to " + << startupFrameCount_; + } + if (invalidFrameCount_) { + invalidFrameCount_--; + LOG(RPI, Debug) << "Decrementing invalid frames to " + << invalidFrameCount_; } } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index aae0c2f35888..6023f9f9d6b3 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -48,7 +48,7 @@ class CameraData : public Camera::Private public: CameraData(PipelineHandler *pipe) : Camera::Private(pipe), state_(State::Stopped), - dropFrameCount_(0), buffersAllocated_(false), + startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false), ispOutputCount_(0), ispOutputTotal_(0) { } @@ -151,7 +151,8 @@ public: /* Mapping of CropParams keyed by the output stream order in CameraConfiguration */ std::map cropParams_; - unsigned int dropFrameCount_; + unsigned int startupFrameCount_; + unsigned int invalidFrameCount_; /* * If set, this stores the value that represets a gain of one for From patchwork Mon May 19 09:20:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23389 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7AE68BD78E for ; Mon, 19 May 2025 09:23:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9173D68D7C; Mon, 19 May 2025 11:23:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="cGBBtjuR"; dkim-atps=neutral Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ED80B68D82 for ; Mon, 19 May 2025 11:22:53 +0200 (CEST) Received: by mail-wr1-x432.google.com with SMTP id ffacd0b85a97d-3a361c8a830so413028f8f.0 for ; Mon, 19 May 2025 02:22:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1747646573; x=1748251373; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TDFSDhxsWtnuegD2eNp0so8bkMmcVepBWj9qp8gGI+4=; b=cGBBtjuRCD5MwpEsu+RNJkvvDvGVDUG3HgAUdXt89YtlAMgw2pjvuGcvfkRYVmGTIf FOzLOK8wWo4OPetB7Dzn8vRlWXCPEaMhUZxDhgoIFn/Un31VoJJ/mtSJSXa/1+5vKuUG JGZvc/n14m0xfKgjzzwPnlYJsplZq4nQZrRqnoPPt8cHVIteavEHZjmPsetkMSyXC3GH vzQMa8FsjcM/MSmHQYq50CuuL3FXPI47yftSv7eFZ40sQ8MfdZ/jai0a4ptCk4zjKc05 NbPRmW4UlhnzLWGj1oy4Ir0mITS6/xOA9/YzDVCMu0eonIEeR0mZfLUXxe8mogSjOMDh PBIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747646573; x=1748251373; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TDFSDhxsWtnuegD2eNp0so8bkMmcVepBWj9qp8gGI+4=; b=nMeZ6nluGqQGSzty4+eW4vzCMjbr0stDcO8RIAbVOIUa7Py8mYHJnKBPpaxV7i1/qJ 0Z7ITmx8n12qj1ZM0rVYIkBSRRm1C1+rPaiD5WNH1J0LsxgEFlPj6l7udeAgsG9u9fco 93UwxyiLcNb0wDmIvnWwiil5XeWZ608aWAh/zXSJiiL0VOjAgAVvT3FoxprEnxsClheh M47gaZtxqd8KMhe5iSk0hicpap0BerGP4PU4NIsgjgxCcbCPfUImfEQRvsBnOjKZ0IqX vPCam/4xftDQeIBHMURC+YHN2B8ql/IeYdnIDEqPuDOuQxT80iRsjyDju5JJQGg4xmVc M9rQ== X-Gm-Message-State: AOJu0YwQG57MqqpmLSfRM/sKhVBKr35UXOoYX8N8fv4oDHv86OF8ktPu lXz7597CTYTydUiZJMPYD0ohP7dNjHrUPDOs1N4OMT8kgGBXvezA1PeFoBJtPlwwg4M/tOzHGKp fuFQK X-Gm-Gg: ASbGncsozmqAuquM/uZEc9eQw9j5uiP3Cqr2VOBuL2EVwU7TBXhyMxt3JPGnV/goUAS QYGsxWxdQV8Tql59Gqd//ncwovRbzDcRmtbsZmPiVQbZ/v2OzNDSvw1VFLiXfYvcXsDR9qAwXZU bV6uljfgYGF639fSeEAHcEZQ9D4upQkjST3LjV1xndpoOABQJYBN9d2vWzSSDyxNkPN5GcaPNlb I9teY9g1uVpNq8t2CK0hNdx1KJzfXo5eUFoxGGFMGbGnOYbzsXw7Gz5K84JDHPaXYJv6Nkgm47o N0EVtc+onF16GpmGv4upUjE0lO3/lMOLg6P21KPcVnlwPq2FQ+wkWAH5vxWgdXMuQG0YbIg= X-Google-Smtp-Source: AGHT+IF2tysLNeEBGECrTG+UjzZ3WrZk7mSIXTGYp3ZjmlsJXuaBKtg+zKMHsEswEqmLr/kkbEfJNQ== X-Received: by 2002:a05:6000:438a:b0:3a3:75ea:f9d3 with SMTP id ffacd0b85a97d-3a375eafce5mr315084f8f.11.1747646573249; Mon, 19 May 2025 02:22:53 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a366e08747sm8645961f8f.95.2025.05.19.02.22.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 May 2025 02:22:52 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [PATCH v1 4/6] pipeline: rpi: Remove disable_startup_frame_drops config option Date: Mon, 19 May 2025 10:20:52 +0100 Message-ID: <20250519092245.269048-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250519092245.269048-1-naush@raspberrypi.com> References: <20250519092245.269048-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the previous change to not drop frames in the pipeline handler, the "disable_startup_frame_drops" pipeline config option is not used. Remove it, and throw a warning if the option is present in the YAML config file. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman (config_.disableStartupFrameDrops); + if (phConfig.contains("disable_startup_frame_drops")) + LOG(RPI, Warning) + << "The disable_startup_frame_drops key is now deprecated, " + << "please use FrameMetadata::Status::FrameStartup instead."; config_.cameraTimeoutValue = phConfig["camera_timeout_value_ms"].get(config_.cameraTimeoutValue); diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 6023f9f9d6b3..e27c4f860d1a 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -164,11 +164,6 @@ public: bool buffersAllocated_; struct Config { - /* - * Override any request from the IPA to drop a number of startup - * frames. - */ - bool disableStartupFrameDrops; /* * Override the camera timeout value calculated by the IPA based * on frame durations. diff --git a/src/libcamera/pipeline/rpi/pisp/data/example.yaml b/src/libcamera/pipeline/rpi/pisp/data/example.yaml index d67e654a8b9e..baf03be79bb3 100644 --- a/src/libcamera/pipeline/rpi/pisp/data/example.yaml +++ b/src/libcamera/pipeline/rpi/pisp/data/example.yaml @@ -16,11 +16,6 @@ # # "num_cfe_config_queue": 2, - # Override any request from the IPA to drop a number of startup - # frames. - # - # "disable_startup_frame_drops": false, - # Custom timeout value (in ms) for camera to use. This overrides # the value computed by the pipeline handler based on frame # durations. diff --git a/src/libcamera/pipeline/rpi/vc4/data/example.yaml b/src/libcamera/pipeline/rpi/vc4/data/example.yaml index b8e01adeaf40..27e543488d48 100644 --- a/src/libcamera/pipeline/rpi/vc4/data/example.yaml +++ b/src/libcamera/pipeline/rpi/vc4/data/example.yaml @@ -29,11 +29,6 @@ # # "min_total_unicam_buffers": 4, - # Override any request from the IPA to drop a number of startup - # frames. - # - # "disable_startup_frame_drops": false, - # Custom timeout value (in ms) for camera to use. This overrides # the value computed by the pipeline handler based on frame # durations. From patchwork Mon May 19 09:20:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23390 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 050F2C331E for ; Mon, 19 May 2025 09:23:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C957768D7A; Mon, 19 May 2025 11:23:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="nnnrMIs3"; dkim-atps=neutral Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8A0D268D7A for ; Mon, 19 May 2025 11:22:54 +0200 (CEST) Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-43cfe99f2a7so4145935e9.2 for ; Mon, 19 May 2025 02:22:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1747646574; x=1748251374; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HyZHAEPEvRlhx9HxiH7c+Sa84pjnUrsNuklV5311nY4=; b=nnnrMIs3iphU3CGkXZZcEhkBGAeJj5olIpVvhPpyJ8zyGYYSbDmwKCw3EjdLMeARGs OoO7G+U2gAB5yE8c89SPWhXstHTv6uKZ7JfWvv3cickd8jR72BOH0ljkTJNE1Lf4D9c3 v/a1BG0uVIBVse0otbRxjrLmzU4eXvv0ntw9TD8qPG5KjyGopfcI3HBcKdZjPeYoHcoc CTvZU/vSG0CI6yt2zpkdkfWtihcN21GU5B0UCDvzCtuCmNopVdgg1tBCNCxqXPANroSp jbaZK0o0qSCntTetb89eug0ct8f8jnMCXs/Vu0dYIIjOF/Lvgxi3dJ7Bn559xBcrkUvC uLcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747646574; x=1748251374; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HyZHAEPEvRlhx9HxiH7c+Sa84pjnUrsNuklV5311nY4=; b=eqRDyLHBUsW2lfWCRgTAYeGZFdf4/LUVV2vx9ZDRdWHrCxWydKiNRgdhrL/tnodBST 24TMY087A5jVRUvf0sA11/nyZMVTHpt5pAyKuX2kCzbpjATHrTRNNktb8GhxKI9TrIjB pxUyWA5goxaOUT+QruVwwap2iaC0lvccpYOQBxf/fS8hg3ihoRVabThuj4xOE9dqGR5w dSJvH24AXsMh+xEEwZpWjks+IU9qbWL2eNDgz9q2c+zbOOWtaBq/QkMTUR93LiFohrZv suJl8AI3iuf+7XJC8v0wDiDIvsILwmNuEEpvYzpef/atA9g7P+Xo7tb9oH977dNz41Fi w+6A== X-Gm-Message-State: AOJu0Yw9fhSoaIceRRg1fFfA/9eT2DepakWRAOCRdRIn7ybZibFQa6+h IUpCsK9AdiVGDuuaPctmXULo+z1BFbE5hKKP9D4A1oWUxaRXm8PPk/Hm03rhsKY9dWAsumF7AoU GXHi3 X-Gm-Gg: ASbGncv1WwNcNFP5MU3F1ASuL0lsDSnLX5OGrJ9t5bTvt53wL8ASPrtPK0U3YGs9fZI zX1cVoRQ+ZtYS+IISCtvvto2FIyMODDmw/JFtnhELkSvdgC9q1fPFTwqjOvCe6IwS9BAu0DAUYO QIbdpmaFSIG/Jl86GLHWrEUwfpV/lgOa6FgV4U9YgS/kjGJ/meMKX+fk4t8yk+N9CqPlVzZ7uXH JR7ao9jYjlFWIAfVXVEJcu7SC+CxN/EFQEcal0qoGt14VTTj1JIBudBq+VaVwVfJV/Ii+gwapey QKB38Lv5z8NbAqRFnX8qw3B61sT0XnIAg1JiJ0Ej3CVS3WqwtUtoF70CRm+H7IJ/q8nlMWTIssF l7yBmPg== X-Google-Smtp-Source: AGHT+IHKPj7U53tgMhR2lnKh+fg6qYusXxNLkRdDDXG0VWXDG5P+fTTEsONIbP1RoZYb/4yalhxSsQ== X-Received: by 2002:a05:600d:108:10b0:43b:c0fa:f9bf with SMTP id 5b1f17b1804b1-442fd7165b7mr21711295e9.3.1747646573789; Mon, 19 May 2025 02:22:53 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a366e08747sm8645961f8f.95.2025.05.19.02.22.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 May 2025 02:22:53 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [PATCH v1 5/6] pipeline: rpi: Remove ispOutputCount_ and ispOutputTotal_ Date: Mon, 19 May 2025 10:20:53 +0100 Message-ID: <20250519092245.269048-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250519092245.269048-1-naush@raspberrypi.com> References: <20250519092245.269048-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the drop frame logic removed from the pipeline handler, these member variables and not used, so remove them. Signed-off-by: Naushir Patuck --- src/libcamera/pipeline/rpi/common/pipeline_base.h | 6 +----- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 12 ------------ src/libcamera/pipeline/rpi/vc4/vc4.cpp | 15 --------------- 3 files changed, 1 insertion(+), 32 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index e27c4f860d1a..898f31577059 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -48,8 +48,7 @@ class CameraData : public Camera::Private public: CameraData(PipelineHandler *pipe) : Camera::Private(pipe), state_(State::Stopped), - startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false), - ispOutputCount_(0), ispOutputTotal_(0) + startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false) { } @@ -179,9 +178,6 @@ protected: virtual void tryRunPipeline() = 0; - unsigned int ispOutputCount_; - unsigned int ispOutputTotal_; - private: void checkRequestCompleted(); }; diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index 91e7f4c94d96..ccf135c3d8ce 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -1834,12 +1834,6 @@ void PiSPCameraData::beOutputDequeue(FrameBuffer *buffer) dmabufSyncEnd(buffer->planes()[0].fd); handleStreamBuffer(buffer, stream); - - /* - * Increment the number of ISP outputs generated. - * This is needed to track dropped frames. - */ - ispOutputCount_++; handleState(); } @@ -1885,7 +1879,6 @@ void PiSPCameraData::prepareIspComplete(const ipa::RPi::BufferIds &buffers, bool * If there is no need to run the Backend, just signal that the * input buffer is completed and all Backend outputs are ready. */ - ispOutputCount_ = ispOutputTotal_; buffer = cfe_[Cfe::Output0].getBuffers().at(bayerId).buffer; handleStreamBuffer(buffer, &cfe_[Cfe::Output0]); } else @@ -1994,7 +1987,6 @@ int PiSPCameraData::configureBe(const std::optional &yuvColorSpace) global.bayer_enables |= PISP_BE_BAYER_ENABLE_INPUT; global.bayer_order = toPiSPBayerOrder(cfeFormat.fourcc); - ispOutputTotal_ = 1; /* Config buffer */ if (PISP_IMAGE_FORMAT_COMPRESSED(inputFormat.format)) { pisp_decompress_config decompress; decompress.offset = DefaultCompressionOffset; @@ -2025,7 +2017,6 @@ int PiSPCameraData::configureBe(const std::optional &yuvColorSpace) setupOutputClipping(ispFormat0, outputFormat0); be_->SetOutputFormat(0, outputFormat0); - ispOutputTotal_++; } if (global.rgb_enables & PISP_BE_RGB_ENABLE_OUTPUT1) { @@ -2049,7 +2040,6 @@ int PiSPCameraData::configureBe(const std::optional &yuvColorSpace) setupOutputClipping(ispFormat1, outputFormat1); be_->SetOutputFormat(1, outputFormat1); - ispOutputTotal_++; } /* Setup the TDN I/O blocks in case TDN gets turned on later. */ @@ -2256,8 +2246,6 @@ void PiSPCameraData::prepareCfe() void PiSPCameraData::prepareBe(uint32_t bufferId, bool stitchSwapBuffers) { - ispOutputCount_ = 0; - FrameBuffer *buffer = cfe_[Cfe::Output0].getBuffers().at(bufferId).buffer; LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << bufferId diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index fe910bdf2ff9..ac6dab814d35 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -597,8 +597,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi stream->setFlags(StreamFlag::External); } - ispOutputTotal_ = outStreams.size(); - /* * If ISP::Output0 stream has not been configured by the application, * we must allow the hardware to generate an output so that the data @@ -625,8 +623,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi return -EINVAL; } - ispOutputTotal_++; - LOG(RPI, Debug) << "Defaulting ISP Output0 format to " << format; } @@ -662,8 +658,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi << ret; return -EINVAL; } - - ispOutputTotal_++; } /* ISP statistics output format. */ @@ -676,8 +670,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi return ret; } - ispOutputTotal_++; - /* * Configure the Unicam embedded data output format only if the sensor * supports it. @@ -843,12 +835,6 @@ void Vc4CameraData::ispOutputDequeue(FrameBuffer *buffer) handleStreamBuffer(buffer, stream); } - /* - * Increment the number of ISP outputs generated. - * This is needed to track dropped frames. - */ - ispOutputCount_++; - handleState(); } @@ -880,7 +866,6 @@ void Vc4CameraData::prepareIspComplete(const ipa::RPi::BufferIds &buffers, << ", timestamp: " << buffer->metadata().timestamp; isp_[Isp::Input].queueBuffer(buffer); - ispOutputCount_ = 0; if (sensorMetadata_ && embeddedId) { buffer = unicam_[Unicam::Embedded].getBuffers().at(embeddedId & RPi::MaskID).buffer; From patchwork Mon May 19 09:20:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23391 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1B9C1C322E for ; Mon, 19 May 2025 09:23:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D2DF168D85; Mon, 19 May 2025 11:23:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="PAXxyTPp"; dkim-atps=neutral Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1DBEB68C91 for ; Mon, 19 May 2025 11:22:55 +0200 (CEST) Received: by mail-wr1-x42b.google.com with SMTP id ffacd0b85a97d-3a35c339e95so446780f8f.3 for ; Mon, 19 May 2025 02:22:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1747646574; x=1748251374; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iDJm16zb7hwM/O4mxusEMEmz8cHY3wVG5puaLJGJwFQ=; b=PAXxyTPpoZ6ATP/IzuAYo53fg//LUciYZ0eaiLfl+jYec6gARyp7qiADLcND5qHWif jS/5F5NEHCrC6ypWNBiyfX/AMhJvyxHSeDFoOqyq5Sbe9yM4x9D24LDVTVIi8t6ONhDW CZvMF36x9IXswXY9dUMInhhznZqH7aw95M94aMgCDv/4lHLa0+/0H7rcdq8cbg2qYrAN uvqiRvp74Xy60hWx4HvISjVhoWvmVVz2pTVfI3sxRqrBNWQHGtsCDyWhWD0XguGjjxSR s7S+lcRG+zxpoArUNA0CM2BTuoJy2iTiA7V2BoWB9x6Snmq1eRYOmF4opUQobKJMcz7h Q7iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747646574; x=1748251374; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iDJm16zb7hwM/O4mxusEMEmz8cHY3wVG5puaLJGJwFQ=; b=oy329vJg6yDn9RvgNlCDip91EKYVqoly8vukFJA3vA1U3n0hREAZC+E83p9tmnzTKR decqrUBJIQKBY3JwRk1nUdwabmnzT5NVLMTVtfAehJ7bllfeTJUV32mFcvPJBSPh31wo 3/Qx8DtkpdSbs3pCLUvxKz0/L+KTFxvKsTMbUIW5zcEumUHMCikQzt+1CFOL03glEH0M ryBfbZ28mhOp+O2r/HTrFCqDZPwlZi7hEtAStaN11UvgAL6XuLxXpE5SFe6D8n/cFmET 5sPVlBpTf7utmhUduQyeCCn7L+KOBNSfM+Et42xQKKKAw6s+gWzcH0QSEuagUsB4mTGI NFWg== X-Gm-Message-State: AOJu0YzvPa/VAyZWIqdtp/sVU0e34OHinMfoPl+qKuDo8yqfAF8vKn6z dg5g9YcWhDk5PX65DN8NqerDA0+iXHpgBCiSPe0VmLSykwPl9CLmE2ypXKsA4ZVRbsCQG2TKeaI VXB9I X-Gm-Gg: ASbGncvPX20lgBzfdLR6ioDZfi5DZHyVObsphV3XA78OV8GJbE4TTmGugQ4s8PkUAZc VaBUpYIxWP1tfYhFdp4Njns7OFwP+lXsT4iGvCVxqTYBetbtBI6IF8f2JyIIsN2wbf1dFO11jl8 Fl5rNnETRuL7OTdq3JQBrTNY8LT9nAflTfgSeHptMWq+8NXvmBIZWrxLs9f3FsmZnJUWqr3PvW5 JJ4Y4AL6vdwbMR4896iRQbONPHBPu2ZEYrL1ancKDvHCFZA3IIhvgAxy4SXBiWXSaDG4XX5IGwF ZNqxbDNmiyKNN/YWHO8M+yRdooPb98CWASMbXEvP53WPM76jk+681rPOKqUE1WPOXGtQpRM= X-Google-Smtp-Source: AGHT+IGjOTq95UtX2EB5DWIuy2WVmx2OoCIrmpIMFx85c+3/6l4+ikKdajIARCnR5jo2crSuR0wTBA== X-Received: by 2002:a5d:5848:0:b0:3a0:99a4:e53f with SMTP id ffacd0b85a97d-3a35c843e2cmr3406507f8f.8.1747646574387; Mon, 19 May 2025 02:22:54 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a366e08747sm8645961f8f.95.2025.05.19.02.22.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 May 2025 02:22:54 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [PATCH v1 6/6] ipa: rpi: Rename dropFrameCount_ to startupCount_ Date: Mon, 19 May 2025 10:20:54 +0100 Message-ID: <20250519092245.269048-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250519092245.269048-1-naush@raspberrypi.com> References: <20250519092245.269048-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rename dropFrameCount_ to startupCount_ to better reflect its use as frames are no longer dropped by the pipeline handler. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/rpi/common/ipa_base.cpp | 10 +++++----- src/ipa/rpi/common/ipa_base.h | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index c15f8a7bf71e..8d591faeceaa 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -327,7 +327,7 @@ void IpaBase::start(const ControlList &controls, StartResult *result) unsigned int agcConvergenceFrames = 0, awbConvergenceFrames = 0; frameCount_ = 0; if (firstStart_) { - dropFrameCount_ = helper_->hideFramesStartup(); + startupCount_ = helper_->hideFramesStartup(); mistrustCount_ = helper_->mistrustFramesStartup(); /* @@ -353,14 +353,14 @@ void IpaBase::start(const ControlList &controls, StartResult *result) awbConvergenceFrames += mistrustCount_; } } else { - dropFrameCount_ = helper_->hideFramesModeSwitch(); + startupCount_ = helper_->hideFramesModeSwitch(); mistrustCount_ = helper_->mistrustFramesModeSwitch(); } result->startupFrameCount = std::max({ agcConvergenceFrames, awbConvergenceFrames }); - result->invalidFrameCount = dropFrameCount_; + result->invalidFrameCount = startupCount_; - dropFrameCount_ = std::max({ dropFrameCount_, agcConvergenceFrames, awbConvergenceFrames }); + startupCount_ = std::max({ startupCount_, agcConvergenceFrames, awbConvergenceFrames }); LOG(IPARPI, Debug) << "Startup frames: " << result->startupFrameCount << " Invalid frames: " << result->invalidFrameCount; @@ -443,7 +443,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) /* Allow a 10% margin on the comparison below. */ Duration delta = (frameTimestamp - lastRunTimestamp_) * 1.0ns; - if (lastRunTimestamp_ && frameCount_ > dropFrameCount_ && + if (lastRunTimestamp_ && frameCount_ > startupCount_ && delta < controllerMinFrameDuration * 0.9 && !hdrChange) { /* * Ensure we merge the previous frame's metadata with the current diff --git a/src/ipa/rpi/common/ipa_base.h b/src/ipa/rpi/common/ipa_base.h index 1a811beb31f2..a51afc156a8f 100644 --- a/src/ipa/rpi/common/ipa_base.h +++ b/src/ipa/rpi/common/ipa_base.h @@ -115,8 +115,8 @@ private: /* How many frames we should avoid running control algos on. */ unsigned int mistrustCount_; - /* Number of frames that need to be dropped on startup. */ - unsigned int dropFrameCount_; + /* Number of frames that need to be marked as dropped on startup. */ + unsigned int startupCount_; /* Frame timestamp for the last run of the controller. */ uint64_t lastRunTimestamp_;