{"id":23477,"url":"https://patchwork.libcamera.org/api/patches/23477/?format=json","web_url":"https://patchwork.libcamera.org/patch/23477/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20250606105651.1624640-4-naush@raspberrypi.com>","date":"2025-06-06T10:55:21","name":"[v3,3/6] pipeline: ipa: rpi: Split RPiCameraData::dropFrameCount_","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"de85597814b5ec8f9729d9cdfcdb14f4d9346cbb","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/?format=json","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/23477/mbox/","series":[{"id":5209,"url":"https://patchwork.libcamera.org/api/series/5209/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5209","date":"2025-06-06T10:55:18","name":"Eliminating startup frames","version":3,"mbox":"https://patchwork.libcamera.org/series/5209/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/23477/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/23477/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id ADC82C3292\n\tfor ;\n\tFri, 6 Jun 2025 10:57:07 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 2B31A68DCA;\n\tFri, 6 Jun 2025 12:57:06 +0200 (CEST)","from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com\n\t[IPv6:2a00:1450:4864:20::42b])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3206D68DB0\n\tfor ;\n\tFri, 6 Jun 2025 12:56:59 +0200 (CEST)","by mail-wr1-x42b.google.com with SMTP id\n\tffacd0b85a97d-3a4ee391e6fso176689f8f.3\n\tfor ;\n\tFri, 06 Jun 2025 03:56:59 -0700 (PDT)","from NAUSH-P-DELL.tail9c427.ts.net ([93.93.133.154])\n\tby smtp.gmail.com with ESMTPSA id\n\t5b1f17b1804b1-452137277afsm20092455e9.30.2025.06.06.03.56.56\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tFri, 06 Jun 2025 03:56:57 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"a5VGpBlt\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google; t=1749207418; x=1749812218;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=AWiiOo6NhxVE0fDk/z9rUm9ZZS3TZ/840uwenlIidpI=;\n\tb=a5VGpBlt+mOXEal1wtdaErniHPttoqAw5SRK0V5JS2Sv6fj45I7kGMecUajF7dwYmu\n\tuI6oY11O/0drf53VpupzvXeuTFKRHfyoxhU2PaTNrMZ9phd3S9bSlV8BfZPyTU0DHNx7\n\tJUl9oJM+zfWmfL7v0YwYmGfLAhsoFM86si5+k4RdjTYjw6/Grw3XZG2mwsnMO2in/uJi\n\tilkzpzZV55z4+GBFQDafXz+4jhfGgG6jG51P/xy4QvIwU5WokCTa3MQ1WfEzPVlHDPU4\n\tE27CGV19WbAEolcxQ+6+hD6GcyzqPFoueeHBpUdpeEQAPF4mNFkaRqVGTcei9tqb8G8D\n\tHVXA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1749207418; x=1749812218;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc\n\t:subject:date:message-id:reply-to;\n\tbh=AWiiOo6NhxVE0fDk/z9rUm9ZZS3TZ/840uwenlIidpI=;\n\tb=EUO3Be9vXhks1FX0o63nU88SDhRRE3sCkJcFU2v6yB0kXVOnFtpMhkMB4wMwS8K9W/\n\t0eooIu430MhHfQaX4qLBkMdljKp4DwAL/iU23vDDQb+ioJzhc5VXcagpDWm5ePf+usJ1\n\t4QcJ4r4MfR2SoU8OJMyI3dVphDFDwgopjHswh3pRqvVNU3KrgPLm5gmCh+CcX1wTCUsJ\n\tq+j1ZFa43H6+zOMoKkEinvoSBjPXWNpkOWVxWFH7OgnNQMhOybRZwNnuGLdR0AjZ1+kg\n\tr/Y6du8XCgwzq571bXFIR5SnhsoeZ2ycesMReZJ7C5CFj65I1XmKcuISsjOjjDlX2iM0\n\tO4Kg==","X-Gm-Message-State":"AOJu0YzYY42uwCN3YoYLsVrvpXBkWOyt/81eAqqo5ZLNXGLAnuy1SN40\n\thlKRUqHrZ5LY6R5OmC30ousn/dunLmA87pxJjeL/ES/AsSw80pMjsV8iW8WPPqs1b8K3K7rzYQq\n\tdzQdK","X-Gm-Gg":"ASbGncvVHGzN9hYv2KP7ZiYg2Z3/HnQnYP2vC9vQOqiQ2vlstroLI+g33W16oVJYT86\n\terc36aQhbZO+FBagDt5bsHD8jNHakbiEAJ/ylhYSkd4D1Toq6QWjrSc6isatxH+SoBRg0DwZlmy\n\thoW9zIeG+cyAb9p9d2Rt9sfpMNUi9pFN3Cmd64pPKhG38xl62BWvWrPZOxe23CqA9CUtBg5KL6q\n\txv/g88VxVAqJZIi2WANR3LGYWBXl/x/qksDyoUzxdw07lWsW6Yx8ZshHbfK9fzcBLN+fBYUQC28\n\tD5h3KaJnN7LGNvoVEodySSn/A5J3lfWjhVspt2t0jNeV2eLhFkU93bDcOaMB/v7ZPMp08h7A65t\n\tMDu3DzXoWSA4=","X-Google-Smtp-Source":"AGHT+IFSNBIaMZVCMznBifiyAi49yzhyLjungw2y27LjwIVdocrnpsCpuQ5onWq9sdrgWDkyytoHAA==","X-Received":"by 2002:a05:6000:400e:b0:3a3:63d3:368e with SMTP id\n\tffacd0b85a97d-3a532f7e27cmr590673f8f.0.1749207418427; \n\tFri, 06 Jun 2025 03:56:58 -0700 (PDT)","From":"Naushir Patuck ","To":"libcamera-devel@lists.libcamera.org","Cc":"Naushir Patuck ,\n\tDavid Plowman ,\n\tJacopo Mondi ","Subject":"[PATCH v3 3/6] pipeline: ipa: rpi: Split\n\tRPiCameraData::dropFrameCount_","Date":"Fri, 6 Jun 2025 11:55:21 +0100","Message-ID":"<20250606105651.1624640-4-naush@raspberrypi.com>","X-Mailer":"git-send-email 2.43.0","In-Reply-To":"<20250606105651.1624640-1-naush@raspberrypi.com>","References":"<20250606105651.1624640-1-naush@raspberrypi.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Split the pipeline handler drop frame tracking into startup frames and\ninvalid frames, as reported by the IPA.\n\nRemove the drop buffer handling logic in the pipeline handler. Now all\nimage buffers are returned out with the appropriate FrameStatus set\nfor startup or invalid frames.\n\nSigned-off-by: Naushir Patuck \nReviewed-by: David Plowman \nReviewed-by: Jacopo Mondi \n---\n .../pipeline/rpi/common/pipeline_base.cpp | 98 ++++++++-----------\n .../pipeline/rpi/common/pipeline_base.h | 5 +-\n 2 files changed, 42 insertions(+), 61 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\nindex 5956485953a2..3f0b7abdc59a 100644\n--- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n+++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n@@ -659,9 +659,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls)\n \tif (!result.controls.empty())\n \t\tdata->setSensorControls(result.controls);\n \n-\t/* Configure the number of dropped frames required on startup. */\n-\tdata->dropFrameCount_ = data->config_.disableStartupFrameDrops ?\n-\t\t\t0 : result.startupFrameCount + result.invalidFrameCount;\n+\t/* Configure the number of startup and invalid frames reported by the IPA. */\n+\tdata->startupFrameCount_ = result.startupFrameCount;\n+\tdata->invalidFrameCount_ = result.invalidFrameCount;\n \n \tfor (auto const stream : data->streams_)\n \t\tstream->resetBuffers();\n@@ -678,7 +678,6 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls)\n \t\tdata->buffersAllocated_ = true;\n \t}\n \n-\t/* We need to set the dropFrameCount_ before queueing buffers. */\n \tret = queueAllBuffers(camera);\n \tif (ret) {\n \t\tLOG(RPI, Error) << \"Failed to queue buffers\";\n@@ -894,28 +893,12 @@ int PipelineHandlerBase::queueAllBuffers(Camera *camera)\n \tint ret;\n \n \tfor (auto const stream : data->streams_) {\n-\t\tif (!(stream->getFlags() & StreamFlag::External)) {\n-\t\t\tret = stream->queueAllBuffers();\n-\t\t\tif (ret < 0)\n-\t\t\t\treturn ret;\n-\t\t} else {\n-\t\t\t/*\n-\t\t\t * For external streams, we must queue up a set of internal\n-\t\t\t * buffers to handle the number of drop frames requested by\n-\t\t\t * the IPA. This is done by passing nullptr in queueBuffer().\n-\t\t\t *\n-\t\t\t * The below queueBuffer() call will do nothing if there\n-\t\t\t * are not enough internal buffers allocated, but this will\n-\t\t\t * be handled by queuing the request for buffers in the\n-\t\t\t * RPiStream object.\n-\t\t\t */\n-\t\t\tunsigned int i;\n-\t\t\tfor (i = 0; i < data->dropFrameCount_; i++) {\n-\t\t\t\tret = stream->queueBuffer(nullptr);\n-\t\t\t\tif (ret)\n-\t\t\t\t\treturn ret;\n-\t\t\t}\n-\t\t}\n+\t\tif (stream->getFlags() & StreamFlag::External)\n+\t\t\tcontinue;\n+\n+\t\tret = stream->queueAllBuffers();\n+\t\tif (ret < 0)\n+\t\t\treturn ret;\n \t}\n \n \treturn 0;\n@@ -1412,7 +1395,15 @@ void CameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::Stream *stream)\n \t * buffer back to the stream.\n \t */\n \tRequest *request = requestQueue_.empty() ? nullptr : requestQueue_.front();\n-\tif (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) {\n+\tif (request && request->findBuffer(stream) == buffer) {\n+\t\tFrameMetadata &md = buffer->_d()->metadata();\n+\n+\t\t/* Mark the non-converged and invalid frames in the metadata. */\n+\t\tif (invalidFrameCount_)\n+\t\t\tmd.status = FrameMetadata::Status::FrameError;\n+\t\telse if (startupFrameCount_)\n+\t\t\tmd.status = FrameMetadata::Status::FrameStartup;\n+\n \t\t/*\n \t\t * Tag the buffer as completed, returning it to the\n \t\t * application.\n@@ -1458,42 +1449,31 @@ void CameraData::handleState()\n \n void CameraData::checkRequestCompleted()\n {\n-\tbool requestCompleted = false;\n-\t/*\n-\t * If we are dropping this frame, do not touch the request, simply\n-\t * change the state to IDLE when ready.\n-\t */\n-\tif (!dropFrameCount_) {\n-\t\tRequest *request = requestQueue_.front();\n-\t\tif (request->hasPendingBuffers())\n-\t\t\treturn;\n+\tRequest *request = requestQueue_.front();\n+\tif (request->hasPendingBuffers())\n+\t\treturn;\n \n-\t\t/* Must wait for metadata to be filled in before completing. */\n-\t\tif (state_ != State::IpaComplete)\n-\t\t\treturn;\n+\t/* Must wait for metadata to be filled in before completing. */\n+\tif (state_ != State::IpaComplete)\n+\t\treturn;\n \n-\t\tLOG(RPI, Debug) << \"Completing request sequence: \"\n-\t\t\t\t<< request->sequence();\n+\tLOG(RPI, Debug) << \"Completing request sequence: \"\n+\t\t\t<< request->sequence();\n \n-\t\tpipe()->completeRequest(request);\n-\t\trequestQueue_.pop();\n-\t\trequestCompleted = true;\n-\t}\n+\tpipe()->completeRequest(request);\n+\trequestQueue_.pop();\n \n-\t/*\n-\t * Make sure we have three outputs completed in the case of a dropped\n-\t * frame.\n-\t */\n-\tif (state_ == State::IpaComplete &&\n-\t ((ispOutputCount_ == ispOutputTotal_ && dropFrameCount_) ||\n-\t requestCompleted)) {\n-\t\tLOG(RPI, Debug) << \"Going into Idle state\";\n-\t\tstate_ = State::Idle;\n-\t\tif (dropFrameCount_) {\n-\t\t\tdropFrameCount_--;\n-\t\t\tLOG(RPI, Debug) << \"Dropping frame at the request of the IPA (\"\n-\t\t\t\t\t<< dropFrameCount_ << \" left)\";\n-\t\t}\n+\tLOG(RPI, Debug) << \"Going into Idle state\";\n+\tstate_ = State::Idle;\n+\n+\tif (invalidFrameCount_) {\n+\t\tinvalidFrameCount_--;\n+\t\tLOG(RPI, Debug) << \"Decrementing invalid frames to \"\n+\t\t\t\t<< invalidFrameCount_;\n+\t} else if (startupFrameCount_) {\n+\t\tstartupFrameCount_--;\n+\t\tLOG(RPI, Debug) << \"Decrementing startup frames to \"\n+\t\t\t\t<< startupFrameCount_;\n \t}\n }\n \ndiff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h\nindex aae0c2f35888..6023f9f9d6b3 100644\n--- a/src/libcamera/pipeline/rpi/common/pipeline_base.h\n+++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h\n@@ -48,7 +48,7 @@ class CameraData : public Camera::Private\n public:\n \tCameraData(PipelineHandler *pipe)\n \t\t: Camera::Private(pipe), state_(State::Stopped),\n-\t\t dropFrameCount_(0), buffersAllocated_(false),\n+\t\t startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false),\n \t\t ispOutputCount_(0), ispOutputTotal_(0)\n \t{\n \t}\n@@ -151,7 +151,8 @@ public:\n \t/* Mapping of CropParams keyed by the output stream order in CameraConfiguration */\n \tstd::map cropParams_;\n \n-\tunsigned int dropFrameCount_;\n+\tunsigned int startupFrameCount_;\n+\tunsigned int invalidFrameCount_;\n \n \t/*\n \t * If set, this stores the value that represets a gain of one for\n","prefixes":["v3","3/6"]}