From patchwork Wed May 7 07:52:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23344 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4DEA6C3226 for ; Wed, 7 May 2025 08:28:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A270368B36; Wed, 7 May 2025 10:28:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="FObYVrtc"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 40EA968B30 for ; Wed, 7 May 2025 10:28:00 +0200 (CEST) Received: by mail-wr1-x429.google.com with SMTP id ffacd0b85a97d-39d73b97500so374091f8f.3 for ; Wed, 07 May 2025 01:28:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1746606479; x=1747211279; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=85vcpWZxa6mtEJPyiFwMvAsazIU0BVD2UXwGQZLD+8o=; b=FObYVrtcEFP/HnCR36dC2LysZVbcwhR8i5fRp/kpK93UqmEhev42u6fcmLfVDY0gDR k3+anq+kwl9WzhSqn9k4eqqwKkDwuhAxFLL7DSjgi+16FZkeOirO0J0Qjkfpn5B11Pgg VwjmWuSEtL16ITqY9VgN55tNtTeBs95CwE9wi0SkaM7ywzPsHWZpt1ymh+FQ+IpSnK5n M85KIMu+Jo0RzR2mby1EzHO2JumFReNwQaFlB/PoWN+zgIrKotgsUQNQIj5KlcmbUNXp bSVaGvWz0b+qw++K1eGGFGCQ67UaBBVwzWiFeSuk2rGI8pVx893mnkwd9zINZQTMBM01 YRcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746606479; x=1747211279; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=85vcpWZxa6mtEJPyiFwMvAsazIU0BVD2UXwGQZLD+8o=; b=AowHmf2hQUttsB61ohS+CvDhu1bR1yj4GwkTy0VyhXqoN3AcVTFLgD5zu0/4ChUSuP lq7LUKqN1Nxw04rfJiiIaKDspyM98ZI+GcTSJpk/aDlXyH5aiuW90sUeFddtHXPv29mU LSSZtLfy7WOSeRfqd8CPJB5GLn88O03SVecwAOBr7WTjgVTzml9y5TTSpKrSpgu9shQn cV2y+Wo+dyEdIBDJ4yfX38/qIfNxloVpBoRZdwINLW1NJUr73nVJj0a9QbPG9up3FHkC fBFbF+CB/lje3v3nR+xPeM0jpHSRONVq4goLvkrLotcJg751ke5wSY3RKCyYde7hsE5a V/ow== X-Gm-Message-State: AOJu0Yz4hrAvqCvlzC6tUIIhfqJlRBhAQOdDtN0/IAU6w0vnzzKjqSuZ bgHKHlt1ohZHufjO6K7bTC3NSO9NYEc+sGI5/ML/IsbDMhsrYh0ZSb+x8k9OfOordSjYEJ8eHCT S X-Gm-Gg: ASbGncs7qMAk4hwQwrVzeaSbG4maYklDqWM1kdes234EvwNbUGqj/y6ELj8StP8kTPV vKNUsl698CoxkURF7RZoQBNA94XAlsEN1w6wMpghcdQ8SrNa7d8d2FX+qUiXwtiohaWTXpIoLwn q5JSeGo3vZykeKZWWxRej3vOpFh5L2Ra4fJElyKBW6wna8hDpRgJEDzL4q0dbLLhgB47jH6S8kc wCu/YYbseq5seV57+f2N2mCMMMyMNSA6LsrDJkipV+wgbkXxxJ91S9IkdIN7jSEujDPI7jW0J0i 5Wm6BOviUn1n/Mtf6JqcHKN5Jjt1RjJnx6DZJN2OsdlquYVk9/ud/luNvVUb X-Google-Smtp-Source: AGHT+IFkly/sLzhr8DUSEzBSGmqUPAXtZLWb6diApNul9uJAcrq0/XAgZLu/sa3J01RUUj7XCXwxvA== X-Received: by 2002:a05:600c:4ecc:b0:439:9fde:da76 with SMTP id 5b1f17b1804b1-441d448cab2mr6312545e9.0.1746606479162; Wed, 07 May 2025 01:27:59 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-441d440fff6sm22182575e9.25.2025.05.07.01.27.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 May 2025 01:27:58 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [RFC PATCH 3/6] pipeline: ipa: rpi: Split RPiCameraData::dropFrameCount_ Date: Wed, 7 May 2025 08:52:10 +0100 Message-ID: <20250507082753.2306743-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250507082753.2306743-1-naush@raspberrypi.com> References: <20250507082753.2306743-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Split the pipeline handler drop frame tracking into startup frames and invalid frames, as reported by the IPA. Remove the drop buffer handling logic in the pipeline handler. Now all image buffers are returned out with the appropriate FrameStatus set for startup or invalid frames. Signed-off-by: Naushir Patuck --- .../pipeline/rpi/common/pipeline_base.cpp | 88 +++++++------------ .../pipeline/rpi/common/pipeline_base.h | 5 +- 2 files changed, 37 insertions(+), 56 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 5956485953a2..21f2daf5bab5 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -659,9 +659,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) if (!result.controls.empty()) data->setSensorControls(result.controls); - /* Configure the number of dropped frames required on startup. */ - data->dropFrameCount_ = data->config_.disableStartupFrameDrops ? - 0 : result.startupFrameCount + result.invalidFrameCount; + /* Configure the number of startup and invalid frames reported by the IPA. */ + data->startupFrameCount_ = result.startupFrameCount; + data->invalidFrameCount_ = result.invalidFrameCount; for (auto const stream : data->streams_) stream->resetBuffers(); @@ -678,7 +678,6 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) data->buffersAllocated_ = true; } - /* We need to set the dropFrameCount_ before queueing buffers. */ ret = queueAllBuffers(camera); if (ret) { LOG(RPI, Error) << "Failed to queue buffers"; @@ -898,23 +897,6 @@ int PipelineHandlerBase::queueAllBuffers(Camera *camera) ret = stream->queueAllBuffers(); if (ret < 0) return ret; - } else { - /* - * For external streams, we must queue up a set of internal - * buffers to handle the number of drop frames requested by - * the IPA. This is done by passing nullptr in queueBuffer(). - * - * The below queueBuffer() call will do nothing if there - * are not enough internal buffers allocated, but this will - * be handled by queuing the request for buffers in the - * RPiStream object. - */ - unsigned int i; - for (i = 0; i < data->dropFrameCount_; i++) { - ret = stream->queueBuffer(nullptr); - if (ret) - return ret; - } } } @@ -1412,7 +1394,15 @@ void CameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::Stream *stream) * buffer back to the stream. */ Request *request = requestQueue_.empty() ? nullptr : requestQueue_.front(); - if (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) { + if (request && request->findBuffer(stream) == buffer) { + FrameMetadata &md = buffer->_d()->metadata(); + + /* Mark the non-converged and invalid frames in the metadata. */ + if (startupFrameCount_) + md.status = FrameMetadata::Status::FrameStartup; + if (invalidFrameCount_) + md.status = FrameMetadata::Status::FrameError; + /* * Tag the buffer as completed, returning it to the * application. @@ -1458,42 +1448,32 @@ void CameraData::handleState() void CameraData::checkRequestCompleted() { - bool requestCompleted = false; - /* - * If we are dropping this frame, do not touch the request, simply - * change the state to IDLE when ready. - */ - if (!dropFrameCount_) { - Request *request = requestQueue_.front(); - if (request->hasPendingBuffers()) - return; + Request *request = requestQueue_.front(); + if (request->hasPendingBuffers()) + return; - /* Must wait for metadata to be filled in before completing. */ - if (state_ != State::IpaComplete) - return; + /* Must wait for metadata to be filled in before completing. */ + if (state_ != State::IpaComplete) + return; - LOG(RPI, Debug) << "Completing request sequence: " - << request->sequence(); + LOG(RPI, Debug) << "Completing request sequence: " + << request->sequence(); - pipe()->completeRequest(request); - requestQueue_.pop(); - requestCompleted = true; - } + pipe()->completeRequest(request); + requestQueue_.pop(); - /* - * Make sure we have three outputs completed in the case of a dropped - * frame. - */ - if (state_ == State::IpaComplete && - ((ispOutputCount_ == ispOutputTotal_ && dropFrameCount_) || - requestCompleted)) { - LOG(RPI, Debug) << "Going into Idle state"; - state_ = State::Idle; - if (dropFrameCount_) { - dropFrameCount_--; - LOG(RPI, Debug) << "Dropping frame at the request of the IPA (" - << dropFrameCount_ << " left)"; - } + LOG(RPI, Debug) << "Going into Idle state"; + state_ = State::Idle; + + if (startupFrameCount_) { + startupFrameCount_--; + LOG(RPI, Debug) << "Decrementing startup frames to " + << startupFrameCount_; + } + if (invalidFrameCount_) { + invalidFrameCount_--; + LOG(RPI, Debug) << "Decrementing invalid frames to " + << invalidFrameCount_; } } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index aae0c2f35888..6023f9f9d6b3 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -48,7 +48,7 @@ class CameraData : public Camera::Private public: CameraData(PipelineHandler *pipe) : Camera::Private(pipe), state_(State::Stopped), - dropFrameCount_(0), buffersAllocated_(false), + startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false), ispOutputCount_(0), ispOutputTotal_(0) { } @@ -151,7 +151,8 @@ public: /* Mapping of CropParams keyed by the output stream order in CameraConfiguration */ std::map cropParams_; - unsigned int dropFrameCount_; + unsigned int startupFrameCount_; + unsigned int invalidFrameCount_; /* * If set, this stores the value that represets a gain of one for