{"id":23418,"url":"https://patchwork.libcamera.org/api/patches/23418/?format=json","web_url":"https://patchwork.libcamera.org/patch/23418/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20250522075244.1198110-4-naush@raspberrypi.com>","date":"2025-05-22T07:48:19","name":"[v2,3/6] pipeline: ipa: rpi: Split RPiCameraData::dropFrameCount_","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"de85597814b5ec8f9729d9cdfcdb14f4d9346cbb","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/?format=json","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/23418/mbox/","series":[{"id":5188,"url":"https://patchwork.libcamera.org/api/series/5188/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5188","date":"2025-05-22T07:48:16","name":"Eliminating startup frames","version":2,"mbox":"https://patchwork.libcamera.org/series/5188/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/23418/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/23418/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 525F1C31E9\n\tfor ;\n\tThu, 22 May 2025 07:53:02 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D8B5C68D97;\n\tThu, 22 May 2025 09:52:56 +0200 (CEST)","from mail-wr1-x431.google.com (mail-wr1-x431.google.com\n\t[IPv6:2a00:1450:4864:20::431])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C671168D7A\n\tfor ;\n\tThu, 22 May 2025 09:52:49 +0200 (CEST)","by mail-wr1-x431.google.com with SMTP id\n\tffacd0b85a97d-3a36561d25cso752401f8f.1\n\tfor ;\n\tThu, 22 May 2025 00:52:49 -0700 (PDT)","from NAUSH-P-DELL.pitowers.org ([93.93.133.154])\n\tby smtp.gmail.com with ESMTPSA id\n\tffacd0b85a97d-3a3ca066eb7sm4191516f8f.2.2025.05.22.00.52.48\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tThu, 22 May 2025 00:52:48 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"PiMlbtvq\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google; t=1747900369; x=1748505169;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=/vbhBGL4NIRBV5xQKNScWF7oI/HN2DraYPV4SxmxcO8=;\n\tb=PiMlbtvqkg/1Wv6ntzA8V1svabSmECQH4KOm8/6BV+7+PQdTSiNoi3F3kyzxWaGR3x\n\t6XyY2kDot7dPxg/HZKl1f5HcNeTgrltqRvfLA2nSJwCiPEEBLZiI7krDEoa0mkCoELMj\n\tr054kUIbi6jEeDzZnbrCUxCxPU61NoXvpQJufOLRym6M2KcC/KsCwdgSydokeYUmkDWw\n\tZ6qYbSP7E73BWbZ0kwLuEl5Bg4C3yB+XYIxRnor/6B9VNapwKfu5w00dCBfq+gWOQUdi\n\tfWVcp/WaLCYJ5diFFTTCUrTfa6TuYsDEkpDU4fE8adAqHAMZW7/Uwc4NcPo6x1oUnheo\n\tS+UA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1747900369; x=1748505169;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc\n\t:subject:date:message-id:reply-to;\n\tbh=/vbhBGL4NIRBV5xQKNScWF7oI/HN2DraYPV4SxmxcO8=;\n\tb=ugaFPAIiqq/4fGHS8hW1UNNbzbEzglTap86B+zjbTleHVrUf1ewUE1ByqDf/rGTSu8\n\tJv9LEjhvU4IyIshgwLnCNqsji6y/+v4MtMPldrPEaDJitYJW/oFWBFT8QmEI8XVYXA7O\n\tdm+rIG+X0irDJP081Yhwo04XIRTAz6EbtQpqqjxk6KEF2Gpg8nOazb0t6y2o6fRtWLyv\n\tykhkn0bQrliCov0xXGNC08fzsT5cpOAFQP3AMo+0v5h4mYh/bk8zqpOx8t6szVPoziG3\n\tJZeHItdvGNiv+B2DiOPNVlwfgOU09V5HvDSwuL1cxeYDOnHkle/yo96gItQPv//vRP+R\n\tW2VQ==","X-Gm-Message-State":"AOJu0Yzwsd3MwecZKtTAtClzcaDzdoo86Bs72heXToy1Ied2UqWXyv45\n\tNLXFJ/0c0rQNFjjCLJ+k8l9cLuidNAZ6hyNETRplAR1043wFE2mlPzx+YGSm4H3tjubVWGNUL0v\n\tN+Q7W","X-Gm-Gg":"ASbGncuwBuaIKadg0f2tip58pPzsVqelTWFAXmiN0zw+OcJ7vLcAQVofj48ESZB17g3\n\tjbvRcLNvzcKj16w/EcQ1VMjGqEy2DC7lCkmRbXmDtdxiWtKqd3KBkNfZZsXb86lkXr0Nc1UKTAs\n\tDek881CbOFNzg0EJ0lUXhfmWaR7OQjeI88LnZ22JKqk/exUwflIUH/3AIjQEdtMcIk5uyEEqmAD\n\t7Gnb3BTm0by/wtXKlNO6AqfZPWCWhvbjVNEf3DHBHILzLgETtvGBESc4+7i8UIgHWtXIiN2x9Le\n\thg3nvnVUklkyG0Jo5dCeBySx+XHDnxVPsIwLbHtF2R6CbkmyvOkIILjkF6qowadXm7xoguksK5n\n\tdXwtUyw==","X-Google-Smtp-Source":"AGHT+IGCXxvi9bzIUXA/5e1rAFnAG4dfFNNBVaFQkP0zM0DLWkEErbWevx9f7bGbW85VAxFkzLjGNg==","X-Received":"by 2002:a05:6000:2307:b0:3a3:7be3:d0f9 with SMTP id\n\tffacd0b85a97d-3a37be3d36dmr3374113f8f.1.1747900369089; \n\tThu, 22 May 2025 00:52:49 -0700 (PDT)","From":"Naushir Patuck ","To":"libcamera-devel@lists.libcamera.org","Cc":"Naushir Patuck ,\n\tDavid Plowman ","Subject":"[PATCH v2 3/6] pipeline: ipa: rpi: Split\n\tRPiCameraData::dropFrameCount_","Date":"Thu, 22 May 2025 08:48:19 +0100","Message-ID":"<20250522075244.1198110-4-naush@raspberrypi.com>","X-Mailer":"git-send-email 2.43.0","In-Reply-To":"<20250522075244.1198110-1-naush@raspberrypi.com>","References":"<20250522075244.1198110-1-naush@raspberrypi.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Split the pipeline handler drop frame tracking into startup frames and\ninvalid frames, as reported by the IPA.\n\nRemove the drop buffer handling logic in the pipeline handler. Now all\nimage buffers are returned out with the appropriate FrameStatus set\nfor startup or invalid frames.\n\nSigned-off-by: Naushir Patuck \nReviewed-by: David Plowman \n---\n .../pipeline/rpi/common/pipeline_base.cpp | 98 ++++++++-----------\n .../pipeline/rpi/common/pipeline_base.h | 5 +-\n 2 files changed, 42 insertions(+), 61 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\nindex 5956485953a2..3f0b7abdc59a 100644\n--- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n+++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n@@ -659,9 +659,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls)\n \tif (!result.controls.empty())\n \t\tdata->setSensorControls(result.controls);\n \n-\t/* Configure the number of dropped frames required on startup. */\n-\tdata->dropFrameCount_ = data->config_.disableStartupFrameDrops ?\n-\t\t\t0 : result.startupFrameCount + result.invalidFrameCount;\n+\t/* Configure the number of startup and invalid frames reported by the IPA. */\n+\tdata->startupFrameCount_ = result.startupFrameCount;\n+\tdata->invalidFrameCount_ = result.invalidFrameCount;\n \n \tfor (auto const stream : data->streams_)\n \t\tstream->resetBuffers();\n@@ -678,7 +678,6 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls)\n \t\tdata->buffersAllocated_ = true;\n \t}\n \n-\t/* We need to set the dropFrameCount_ before queueing buffers. */\n \tret = queueAllBuffers(camera);\n \tif (ret) {\n \t\tLOG(RPI, Error) << \"Failed to queue buffers\";\n@@ -894,28 +893,12 @@ int PipelineHandlerBase::queueAllBuffers(Camera *camera)\n \tint ret;\n \n \tfor (auto const stream : data->streams_) {\n-\t\tif (!(stream->getFlags() & StreamFlag::External)) {\n-\t\t\tret = stream->queueAllBuffers();\n-\t\t\tif (ret < 0)\n-\t\t\t\treturn ret;\n-\t\t} else {\n-\t\t\t/*\n-\t\t\t * For external streams, we must queue up a set of internal\n-\t\t\t * buffers to handle the number of drop frames requested by\n-\t\t\t * the IPA. This is done by passing nullptr in queueBuffer().\n-\t\t\t *\n-\t\t\t * The below queueBuffer() call will do nothing if there\n-\t\t\t * are not enough internal buffers allocated, but this will\n-\t\t\t * be handled by queuing the request for buffers in the\n-\t\t\t * RPiStream object.\n-\t\t\t */\n-\t\t\tunsigned int i;\n-\t\t\tfor (i = 0; i < data->dropFrameCount_; i++) {\n-\t\t\t\tret = stream->queueBuffer(nullptr);\n-\t\t\t\tif (ret)\n-\t\t\t\t\treturn ret;\n-\t\t\t}\n-\t\t}\n+\t\tif (stream->getFlags() & StreamFlag::External)\n+\t\t\tcontinue;\n+\n+\t\tret = stream->queueAllBuffers();\n+\t\tif (ret < 0)\n+\t\t\treturn ret;\n \t}\n \n \treturn 0;\n@@ -1412,7 +1395,15 @@ void CameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::Stream *stream)\n \t * buffer back to the stream.\n \t */\n \tRequest *request = requestQueue_.empty() ? nullptr : requestQueue_.front();\n-\tif (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) {\n+\tif (request && request->findBuffer(stream) == buffer) {\n+\t\tFrameMetadata &md = buffer->_d()->metadata();\n+\n+\t\t/* Mark the non-converged and invalid frames in the metadata. */\n+\t\tif (invalidFrameCount_)\n+\t\t\tmd.status = FrameMetadata::Status::FrameError;\n+\t\telse if (startupFrameCount_)\n+\t\t\tmd.status = FrameMetadata::Status::FrameStartup;\n+\n \t\t/*\n \t\t * Tag the buffer as completed, returning it to the\n \t\t * application.\n@@ -1458,42 +1449,31 @@ void CameraData::handleState()\n \n void CameraData::checkRequestCompleted()\n {\n-\tbool requestCompleted = false;\n-\t/*\n-\t * If we are dropping this frame, do not touch the request, simply\n-\t * change the state to IDLE when ready.\n-\t */\n-\tif (!dropFrameCount_) {\n-\t\tRequest *request = requestQueue_.front();\n-\t\tif (request->hasPendingBuffers())\n-\t\t\treturn;\n+\tRequest *request = requestQueue_.front();\n+\tif (request->hasPendingBuffers())\n+\t\treturn;\n \n-\t\t/* Must wait for metadata to be filled in before completing. */\n-\t\tif (state_ != State::IpaComplete)\n-\t\t\treturn;\n+\t/* Must wait for metadata to be filled in before completing. */\n+\tif (state_ != State::IpaComplete)\n+\t\treturn;\n \n-\t\tLOG(RPI, Debug) << \"Completing request sequence: \"\n-\t\t\t\t<< request->sequence();\n+\tLOG(RPI, Debug) << \"Completing request sequence: \"\n+\t\t\t<< request->sequence();\n \n-\t\tpipe()->completeRequest(request);\n-\t\trequestQueue_.pop();\n-\t\trequestCompleted = true;\n-\t}\n+\tpipe()->completeRequest(request);\n+\trequestQueue_.pop();\n \n-\t/*\n-\t * Make sure we have three outputs completed in the case of a dropped\n-\t * frame.\n-\t */\n-\tif (state_ == State::IpaComplete &&\n-\t ((ispOutputCount_ == ispOutputTotal_ && dropFrameCount_) ||\n-\t requestCompleted)) {\n-\t\tLOG(RPI, Debug) << \"Going into Idle state\";\n-\t\tstate_ = State::Idle;\n-\t\tif (dropFrameCount_) {\n-\t\t\tdropFrameCount_--;\n-\t\t\tLOG(RPI, Debug) << \"Dropping frame at the request of the IPA (\"\n-\t\t\t\t\t<< dropFrameCount_ << \" left)\";\n-\t\t}\n+\tLOG(RPI, Debug) << \"Going into Idle state\";\n+\tstate_ = State::Idle;\n+\n+\tif (invalidFrameCount_) {\n+\t\tinvalidFrameCount_--;\n+\t\tLOG(RPI, Debug) << \"Decrementing invalid frames to \"\n+\t\t\t\t<< invalidFrameCount_;\n+\t} else if (startupFrameCount_) {\n+\t\tstartupFrameCount_--;\n+\t\tLOG(RPI, Debug) << \"Decrementing startup frames to \"\n+\t\t\t\t<< startupFrameCount_;\n \t}\n }\n \ndiff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h\nindex aae0c2f35888..6023f9f9d6b3 100644\n--- a/src/libcamera/pipeline/rpi/common/pipeline_base.h\n+++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h\n@@ -48,7 +48,7 @@ class CameraData : public Camera::Private\n public:\n \tCameraData(PipelineHandler *pipe)\n \t\t: Camera::Private(pipe), state_(State::Stopped),\n-\t\t dropFrameCount_(0), buffersAllocated_(false),\n+\t\t startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false),\n \t\t ispOutputCount_(0), ispOutputTotal_(0)\n \t{\n \t}\n@@ -151,7 +151,8 @@ public:\n \t/* Mapping of CropParams keyed by the output stream order in CameraConfiguration */\n \tstd::map cropParams_;\n \n-\tunsigned int dropFrameCount_;\n+\tunsigned int startupFrameCount_;\n+\tunsigned int invalidFrameCount_;\n \n \t/*\n \t * If set, this stores the value that represets a gain of one for\n","prefixes":["v2","3/6"]}