From patchwork Thu May 22 07:48:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23418 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 525F1C31E9 for ; Thu, 22 May 2025 07:53:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D8B5C68D97; Thu, 22 May 2025 09:52:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="PiMlbtvq"; dkim-atps=neutral Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C671168D7A for ; Thu, 22 May 2025 09:52:49 +0200 (CEST) Received: by mail-wr1-x431.google.com with SMTP id ffacd0b85a97d-3a36561d25cso752401f8f.1 for ; Thu, 22 May 2025 00:52:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1747900369; x=1748505169; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/vbhBGL4NIRBV5xQKNScWF7oI/HN2DraYPV4SxmxcO8=; b=PiMlbtvqkg/1Wv6ntzA8V1svabSmECQH4KOm8/6BV+7+PQdTSiNoi3F3kyzxWaGR3x 6XyY2kDot7dPxg/HZKl1f5HcNeTgrltqRvfLA2nSJwCiPEEBLZiI7krDEoa0mkCoELMj r054kUIbi6jEeDzZnbrCUxCxPU61NoXvpQJufOLRym6M2KcC/KsCwdgSydokeYUmkDWw Z6qYbSP7E73BWbZ0kwLuEl5Bg4C3yB+XYIxRnor/6B9VNapwKfu5w00dCBfq+gWOQUdi fWVcp/WaLCYJ5diFFTTCUrTfa6TuYsDEkpDU4fE8adAqHAMZW7/Uwc4NcPo6x1oUnheo S+UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747900369; x=1748505169; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/vbhBGL4NIRBV5xQKNScWF7oI/HN2DraYPV4SxmxcO8=; b=ugaFPAIiqq/4fGHS8hW1UNNbzbEzglTap86B+zjbTleHVrUf1ewUE1ByqDf/rGTSu8 Jv9LEjhvU4IyIshgwLnCNqsji6y/+v4MtMPldrPEaDJitYJW/oFWBFT8QmEI8XVYXA7O dm+rIG+X0irDJP081Yhwo04XIRTAz6EbtQpqqjxk6KEF2Gpg8nOazb0t6y2o6fRtWLyv ykhkn0bQrliCov0xXGNC08fzsT5cpOAFQP3AMo+0v5h4mYh/bk8zqpOx8t6szVPoziG3 JZeHItdvGNiv+B2DiOPNVlwfgOU09V5HvDSwuL1cxeYDOnHkle/yo96gItQPv//vRP+R W2VQ== X-Gm-Message-State: AOJu0Yzwsd3MwecZKtTAtClzcaDzdoo86Bs72heXToy1Ied2UqWXyv45 NLXFJ/0c0rQNFjjCLJ+k8l9cLuidNAZ6hyNETRplAR1043wFE2mlPzx+YGSm4H3tjubVWGNUL0v N+Q7W X-Gm-Gg: ASbGncuwBuaIKadg0f2tip58pPzsVqelTWFAXmiN0zw+OcJ7vLcAQVofj48ESZB17g3 jbvRcLNvzcKj16w/EcQ1VMjGqEy2DC7lCkmRbXmDtdxiWtKqd3KBkNfZZsXb86lkXr0Nc1UKTAs Dek881CbOFNzg0EJ0lUXhfmWaR7OQjeI88LnZ22JKqk/exUwflIUH/3AIjQEdtMcIk5uyEEqmAD 7Gnb3BTm0by/wtXKlNO6AqfZPWCWhvbjVNEf3DHBHILzLgETtvGBESc4+7i8UIgHWtXIiN2x9Le hg3nvnVUklkyG0Jo5dCeBySx+XHDnxVPsIwLbHtF2R6CbkmyvOkIILjkF6qowadXm7xoguksK5n dXwtUyw== X-Google-Smtp-Source: AGHT+IGCXxvi9bzIUXA/5e1rAFnAG4dfFNNBVaFQkP0zM0DLWkEErbWevx9f7bGbW85VAxFkzLjGNg== X-Received: by 2002:a05:6000:2307:b0:3a3:7be3:d0f9 with SMTP id ffacd0b85a97d-3a37be3d36dmr3374113f8f.1.1747900369089; Thu, 22 May 2025 00:52:49 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a3ca066eb7sm4191516f8f.2.2025.05.22.00.52.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 May 2025 00:52:48 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman Subject: [PATCH v2 3/6] pipeline: ipa: rpi: Split RPiCameraData::dropFrameCount_ Date: Thu, 22 May 2025 08:48:19 +0100 Message-ID: <20250522075244.1198110-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250522075244.1198110-1-naush@raspberrypi.com> References: <20250522075244.1198110-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Split the pipeline handler drop frame tracking into startup frames and invalid frames, as reported by the IPA. Remove the drop buffer handling logic in the pipeline handler. Now all image buffers are returned out with the appropriate FrameStatus set for startup or invalid frames. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- .../pipeline/rpi/common/pipeline_base.cpp | 98 ++++++++----------- .../pipeline/rpi/common/pipeline_base.h | 5 +- 2 files changed, 42 insertions(+), 61 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 5956485953a2..3f0b7abdc59a 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -659,9 +659,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) if (!result.controls.empty()) data->setSensorControls(result.controls); - /* Configure the number of dropped frames required on startup. */ - data->dropFrameCount_ = data->config_.disableStartupFrameDrops ? - 0 : result.startupFrameCount + result.invalidFrameCount; + /* Configure the number of startup and invalid frames reported by the IPA. */ + data->startupFrameCount_ = result.startupFrameCount; + data->invalidFrameCount_ = result.invalidFrameCount; for (auto const stream : data->streams_) stream->resetBuffers(); @@ -678,7 +678,6 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) data->buffersAllocated_ = true; } - /* We need to set the dropFrameCount_ before queueing buffers. */ ret = queueAllBuffers(camera); if (ret) { LOG(RPI, Error) << "Failed to queue buffers"; @@ -894,28 +893,12 @@ int PipelineHandlerBase::queueAllBuffers(Camera *camera) int ret; for (auto const stream : data->streams_) { - if (!(stream->getFlags() & StreamFlag::External)) { - ret = stream->queueAllBuffers(); - if (ret < 0) - return ret; - } else { - /* - * For external streams, we must queue up a set of internal - * buffers to handle the number of drop frames requested by - * the IPA. This is done by passing nullptr in queueBuffer(). - * - * The below queueBuffer() call will do nothing if there - * are not enough internal buffers allocated, but this will - * be handled by queuing the request for buffers in the - * RPiStream object. - */ - unsigned int i; - for (i = 0; i < data->dropFrameCount_; i++) { - ret = stream->queueBuffer(nullptr); - if (ret) - return ret; - } - } + if (stream->getFlags() & StreamFlag::External) + continue; + + ret = stream->queueAllBuffers(); + if (ret < 0) + return ret; } return 0; @@ -1412,7 +1395,15 @@ void CameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::Stream *stream) * buffer back to the stream. */ Request *request = requestQueue_.empty() ? nullptr : requestQueue_.front(); - if (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) { + if (request && request->findBuffer(stream) == buffer) { + FrameMetadata &md = buffer->_d()->metadata(); + + /* Mark the non-converged and invalid frames in the metadata. */ + if (invalidFrameCount_) + md.status = FrameMetadata::Status::FrameError; + else if (startupFrameCount_) + md.status = FrameMetadata::Status::FrameStartup; + /* * Tag the buffer as completed, returning it to the * application. @@ -1458,42 +1449,31 @@ void CameraData::handleState() void CameraData::checkRequestCompleted() { - bool requestCompleted = false; - /* - * If we are dropping this frame, do not touch the request, simply - * change the state to IDLE when ready. - */ - if (!dropFrameCount_) { - Request *request = requestQueue_.front(); - if (request->hasPendingBuffers()) - return; + Request *request = requestQueue_.front(); + if (request->hasPendingBuffers()) + return; - /* Must wait for metadata to be filled in before completing. */ - if (state_ != State::IpaComplete) - return; + /* Must wait for metadata to be filled in before completing. */ + if (state_ != State::IpaComplete) + return; - LOG(RPI, Debug) << "Completing request sequence: " - << request->sequence(); + LOG(RPI, Debug) << "Completing request sequence: " + << request->sequence(); - pipe()->completeRequest(request); - requestQueue_.pop(); - requestCompleted = true; - } + pipe()->completeRequest(request); + requestQueue_.pop(); - /* - * Make sure we have three outputs completed in the case of a dropped - * frame. - */ - if (state_ == State::IpaComplete && - ((ispOutputCount_ == ispOutputTotal_ && dropFrameCount_) || - requestCompleted)) { - LOG(RPI, Debug) << "Going into Idle state"; - state_ = State::Idle; - if (dropFrameCount_) { - dropFrameCount_--; - LOG(RPI, Debug) << "Dropping frame at the request of the IPA (" - << dropFrameCount_ << " left)"; - } + LOG(RPI, Debug) << "Going into Idle state"; + state_ = State::Idle; + + if (invalidFrameCount_) { + invalidFrameCount_--; + LOG(RPI, Debug) << "Decrementing invalid frames to " + << invalidFrameCount_; + } else if (startupFrameCount_) { + startupFrameCount_--; + LOG(RPI, Debug) << "Decrementing startup frames to " + << startupFrameCount_; } } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index aae0c2f35888..6023f9f9d6b3 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -48,7 +48,7 @@ class CameraData : public Camera::Private public: CameraData(PipelineHandler *pipe) : Camera::Private(pipe), state_(State::Stopped), - dropFrameCount_(0), buffersAllocated_(false), + startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false), ispOutputCount_(0), ispOutputTotal_(0) { } @@ -151,7 +151,8 @@ public: /* Mapping of CropParams keyed by the output stream order in CameraConfiguration */ std::map cropParams_; - unsigned int dropFrameCount_; + unsigned int startupFrameCount_; + unsigned int invalidFrameCount_; /* * If set, this stores the value that represets a gain of one for