{"id":23388,"url":"https://patchwork.libcamera.org/api/patches/23388/?format=json","web_url":"https://patchwork.libcamera.org/patch/23388/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20250519092245.269048-4-naush@raspberrypi.com>","date":"2025-05-19T09:20:51","name":"[v1,3/6] pipeline: ipa: rpi: Split RPiCameraData::dropFrameCount_","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"d2a0959af87fbdfc473cf30a4e1aeaf288cf168c","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/?format=json","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/23388/mbox/","series":[{"id":5182,"url":"https://patchwork.libcamera.org/api/series/5182/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5182","date":"2025-05-19T09:20:48","name":"Eliminating startup frames","version":1,"mbox":"https://patchwork.libcamera.org/series/5182/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/23388/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/23388/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id D6222C322E\n\tfor ;\n\tMon, 19 May 2025 09:23:01 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id E72FA68C91;\n\tMon, 19 May 2025 11:22:59 +0200 (CEST)","from mail-wr1-x436.google.com (mail-wr1-x436.google.com\n\t[IPv6:2a00:1450:4864:20::436])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 8263368D80\n\tfor ;\n\tMon, 19 May 2025 11:22:53 +0200 (CEST)","by mail-wr1-x436.google.com with SMTP id\n\tffacd0b85a97d-3a365bc0af8so259441f8f.3\n\tfor ;\n\tMon, 19 May 2025 02:22:53 -0700 (PDT)","from NAUSH-P-DELL.pitowers.org ([93.93.133.154])\n\tby smtp.gmail.com with ESMTPSA id\n\tffacd0b85a97d-3a366e08747sm8645961f8f.95.2025.05.19.02.22.52\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tMon, 19 May 2025 02:22:52 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"enwqg//h\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google; t=1747646573; x=1748251373;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=85vcpWZxa6mtEJPyiFwMvAsazIU0BVD2UXwGQZLD+8o=;\n\tb=enwqg//h/yq1Sdo/JO8Jf0P53JnQgWVrndpVeGElYcSJoSzB41XgksNcvaPDB3daI8\n\tkQvx0VghWxabWlJngfzuRxoW3y4o6lUgSZGAXVWcYLK4TPuu+wD594raWweQeMU5GtCV\n\tkFPXbVe1bs8HiYwsgZUFT4hRYhZI3URg0Y11d409UZQJ0JiW57vMQ8j+JtrqYvEB9+LL\n\tgBv6YRiB78la2U41wVlCDXNAidQzpw8o14OQ4ljWSo6a1sM7TPWiCFi5BfRNklmbNjGs\n\tcNHtsFp4p2zoSwcPYITs3VCqcRcxXzZVrpap8BZ04E0oozB0HzwUKGHMp2+fhf6tfyi4\n\tVbLQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1747646573; x=1748251373;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc\n\t:subject:date:message-id:reply-to;\n\tbh=85vcpWZxa6mtEJPyiFwMvAsazIU0BVD2UXwGQZLD+8o=;\n\tb=mxrPcPewB773TKPaLcQwSmkIasWZTApKQymwRAlxq74frGuD44LVI233TYDyNoPuE0\n\trzeaMVb7/e8TfFGvc3silNnnZH94MrG82bhei8UXdGJzSRKYN9N3P+7j5JL765r3gYYg\n\txsJkb5PpiNZ28qcsfBvICDrF2vq3pim6wWyIXtAYvFU7/na02m5ynfX+x6Y38R1bwX+O\n\tiOGUVpBcdLXoJxooLepdbkdgQZaH9pNamldYkofT6airt0WlXFkmNwojxnaSZKwQcpW8\n\tKf51nSkjTc7T+C32cqre8ZUOSC/5FQlpR2blSY/QAnEWiv0FdPANexQv0kCmcSb0Y/vF\n\tlcNw==","X-Gm-Message-State":"AOJu0Yy3eyp/5GNNY8iaJUOGUoA2ChY1G6BFRYZwzh1P4YDuzEYi5j2j\n\tGlLDuhyPEhJZeBvJSMzrJ9xsgCRtkoGxpkzbkygqogrRPiGVelUyv7VN6eQQuMZ8jiD9QH019Q4\n\toskdz","X-Gm-Gg":"ASbGncsAQ3Xa14iK6MZCcl5xhu0Llhmk3tCAnns+7dcPl3P4rciLLvhguc0fPD5qivn\n\tYolhiyIfMiFls5j2rBKSOzkxHWgJ1aJhyjBvgncwGhoMnP6WkwhDzJM0JL3YzSPYleSRxHy+6JR\n\tIX0mD1Yxw8YiXVuJd95GTdwfSJ+xqBP9fgNBANO2RD/xRWBFSNASif0cqFhkLtDoowRoFehTdrZ\n\tBnRhgfxUb3UrOkMlircJluOFXg/QRbwK4xAtPmb4nzVFuiDh/Yjg1Crh3ADtiHh7gofXQabK6UC\n\t8ce+j3EX5ob4srReLoBcsxLwRurw132PsP7NV187vseZCQF72wfTi3KSvD0ibSSYzsh97jY=","X-Google-Smtp-Source":"AGHT+IEUeIzVYBaXlphr4CKcpPM31oDxnW7w5WZF9G6M/+VeDpegRRh6rCKi34TVnOj+4bIlPz06xg==","X-Received":"by 2002:a5d:404a:0:b0:3a3:62e1:a7dc with SMTP id\n\tffacd0b85a97d-3a362e1ab39mr2706273f8f.1.1747646572691; \n\tMon, 19 May 2025 02:22:52 -0700 (PDT)","From":"Naushir Patuck ","To":"libcamera-devel@lists.libcamera.org","Cc":"Naushir Patuck ","Subject":"[PATCH v1 3/6] pipeline: ipa: rpi: Split\n\tRPiCameraData::dropFrameCount_","Date":"Mon, 19 May 2025 10:20:51 +0100","Message-ID":"<20250519092245.269048-4-naush@raspberrypi.com>","X-Mailer":"git-send-email 2.43.0","In-Reply-To":"<20250519092245.269048-1-naush@raspberrypi.com>","References":"<20250519092245.269048-1-naush@raspberrypi.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Split the pipeline handler drop frame tracking into startup frames and\ninvalid frames, as reported by the IPA.\n\nRemove the drop buffer handling logic in the pipeline handler. Now all\nimage buffers are returned out with the appropriate FrameStatus set\nfor startup or invalid frames.\n\nSigned-off-by: Naushir Patuck \n---\n .../pipeline/rpi/common/pipeline_base.cpp | 88 +++++++------------\n .../pipeline/rpi/common/pipeline_base.h | 5 +-\n 2 files changed, 37 insertions(+), 56 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\nindex 5956485953a2..21f2daf5bab5 100644\n--- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n+++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n@@ -659,9 +659,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls)\n \tif (!result.controls.empty())\n \t\tdata->setSensorControls(result.controls);\n \n-\t/* Configure the number of dropped frames required on startup. */\n-\tdata->dropFrameCount_ = data->config_.disableStartupFrameDrops ?\n-\t\t\t0 : result.startupFrameCount + result.invalidFrameCount;\n+\t/* Configure the number of startup and invalid frames reported by the IPA. */\n+\tdata->startupFrameCount_ = result.startupFrameCount;\n+\tdata->invalidFrameCount_ = result.invalidFrameCount;\n \n \tfor (auto const stream : data->streams_)\n \t\tstream->resetBuffers();\n@@ -678,7 +678,6 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls)\n \t\tdata->buffersAllocated_ = true;\n \t}\n \n-\t/* We need to set the dropFrameCount_ before queueing buffers. */\n \tret = queueAllBuffers(camera);\n \tif (ret) {\n \t\tLOG(RPI, Error) << \"Failed to queue buffers\";\n@@ -898,23 +897,6 @@ int PipelineHandlerBase::queueAllBuffers(Camera *camera)\n \t\t\tret = stream->queueAllBuffers();\n \t\t\tif (ret < 0)\n \t\t\t\treturn ret;\n-\t\t} else {\n-\t\t\t/*\n-\t\t\t * For external streams, we must queue up a set of internal\n-\t\t\t * buffers to handle the number of drop frames requested by\n-\t\t\t * the IPA. This is done by passing nullptr in queueBuffer().\n-\t\t\t *\n-\t\t\t * The below queueBuffer() call will do nothing if there\n-\t\t\t * are not enough internal buffers allocated, but this will\n-\t\t\t * be handled by queuing the request for buffers in the\n-\t\t\t * RPiStream object.\n-\t\t\t */\n-\t\t\tunsigned int i;\n-\t\t\tfor (i = 0; i < data->dropFrameCount_; i++) {\n-\t\t\t\tret = stream->queueBuffer(nullptr);\n-\t\t\t\tif (ret)\n-\t\t\t\t\treturn ret;\n-\t\t\t}\n \t\t}\n \t}\n \n@@ -1412,7 +1394,15 @@ void CameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::Stream *stream)\n \t * buffer back to the stream.\n \t */\n \tRequest *request = requestQueue_.empty() ? nullptr : requestQueue_.front();\n-\tif (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) {\n+\tif (request && request->findBuffer(stream) == buffer) {\n+\t\tFrameMetadata &md = buffer->_d()->metadata();\n+\n+\t\t/* Mark the non-converged and invalid frames in the metadata. */\n+\t\tif (startupFrameCount_)\n+\t\t\tmd.status = FrameMetadata::Status::FrameStartup;\n+\t\tif (invalidFrameCount_)\n+\t\t\tmd.status = FrameMetadata::Status::FrameError;\n+\n \t\t/*\n \t\t * Tag the buffer as completed, returning it to the\n \t\t * application.\n@@ -1458,42 +1448,32 @@ void CameraData::handleState()\n \n void CameraData::checkRequestCompleted()\n {\n-\tbool requestCompleted = false;\n-\t/*\n-\t * If we are dropping this frame, do not touch the request, simply\n-\t * change the state to IDLE when ready.\n-\t */\n-\tif (!dropFrameCount_) {\n-\t\tRequest *request = requestQueue_.front();\n-\t\tif (request->hasPendingBuffers())\n-\t\t\treturn;\n+\tRequest *request = requestQueue_.front();\n+\tif (request->hasPendingBuffers())\n+\t\treturn;\n \n-\t\t/* Must wait for metadata to be filled in before completing. */\n-\t\tif (state_ != State::IpaComplete)\n-\t\t\treturn;\n+\t/* Must wait for metadata to be filled in before completing. */\n+\tif (state_ != State::IpaComplete)\n+\t\treturn;\n \n-\t\tLOG(RPI, Debug) << \"Completing request sequence: \"\n-\t\t\t\t<< request->sequence();\n+\tLOG(RPI, Debug) << \"Completing request sequence: \"\n+\t\t\t<< request->sequence();\n \n-\t\tpipe()->completeRequest(request);\n-\t\trequestQueue_.pop();\n-\t\trequestCompleted = true;\n-\t}\n+\tpipe()->completeRequest(request);\n+\trequestQueue_.pop();\n \n-\t/*\n-\t * Make sure we have three outputs completed in the case of a dropped\n-\t * frame.\n-\t */\n-\tif (state_ == State::IpaComplete &&\n-\t ((ispOutputCount_ == ispOutputTotal_ && dropFrameCount_) ||\n-\t requestCompleted)) {\n-\t\tLOG(RPI, Debug) << \"Going into Idle state\";\n-\t\tstate_ = State::Idle;\n-\t\tif (dropFrameCount_) {\n-\t\t\tdropFrameCount_--;\n-\t\t\tLOG(RPI, Debug) << \"Dropping frame at the request of the IPA (\"\n-\t\t\t\t\t<< dropFrameCount_ << \" left)\";\n-\t\t}\n+\tLOG(RPI, Debug) << \"Going into Idle state\";\n+\tstate_ = State::Idle;\n+\n+\tif (startupFrameCount_) {\n+\t\tstartupFrameCount_--;\n+\t\tLOG(RPI, Debug) << \"Decrementing startup frames to \"\n+\t\t\t\t<< startupFrameCount_;\n+\t}\n+\tif (invalidFrameCount_) {\n+\t\tinvalidFrameCount_--;\n+\t\tLOG(RPI, Debug) << \"Decrementing invalid frames to \"\n+\t\t\t\t<< invalidFrameCount_;\n \t}\n }\n \ndiff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h\nindex aae0c2f35888..6023f9f9d6b3 100644\n--- a/src/libcamera/pipeline/rpi/common/pipeline_base.h\n+++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h\n@@ -48,7 +48,7 @@ class CameraData : public Camera::Private\n public:\n \tCameraData(PipelineHandler *pipe)\n \t\t: Camera::Private(pipe), state_(State::Stopped),\n-\t\t dropFrameCount_(0), buffersAllocated_(false),\n+\t\t startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false),\n \t\t ispOutputCount_(0), ispOutputTotal_(0)\n \t{\n \t}\n@@ -151,7 +151,8 @@ public:\n \t/* Mapping of CropParams keyed by the output stream order in CameraConfiguration */\n \tstd::map cropParams_;\n \n-\tunsigned int dropFrameCount_;\n+\tunsigned int startupFrameCount_;\n+\tunsigned int invalidFrameCount_;\n \n \t/*\n \t * If set, this stores the value that represets a gain of one for\n","prefixes":["v1","3/6"]}