From patchwork Mon May 19 09:20:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23388 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D6222C322E for ; Mon, 19 May 2025 09:23:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E72FA68C91; Mon, 19 May 2025 11:22:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="enwqg//h"; dkim-atps=neutral Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8263368D80 for ; Mon, 19 May 2025 11:22:53 +0200 (CEST) Received: by mail-wr1-x436.google.com with SMTP id ffacd0b85a97d-3a365bc0af8so259441f8f.3 for ; Mon, 19 May 2025 02:22:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1747646573; x=1748251373; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=85vcpWZxa6mtEJPyiFwMvAsazIU0BVD2UXwGQZLD+8o=; b=enwqg//h/yq1Sdo/JO8Jf0P53JnQgWVrndpVeGElYcSJoSzB41XgksNcvaPDB3daI8 kQvx0VghWxabWlJngfzuRxoW3y4o6lUgSZGAXVWcYLK4TPuu+wD594raWweQeMU5GtCV kFPXbVe1bs8HiYwsgZUFT4hRYhZI3URg0Y11d409UZQJ0JiW57vMQ8j+JtrqYvEB9+LL gBv6YRiB78la2U41wVlCDXNAidQzpw8o14OQ4ljWSo6a1sM7TPWiCFi5BfRNklmbNjGs cNHtsFp4p2zoSwcPYITs3VCqcRcxXzZVrpap8BZ04E0oozB0HzwUKGHMp2+fhf6tfyi4 VbLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747646573; x=1748251373; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=85vcpWZxa6mtEJPyiFwMvAsazIU0BVD2UXwGQZLD+8o=; b=mxrPcPewB773TKPaLcQwSmkIasWZTApKQymwRAlxq74frGuD44LVI233TYDyNoPuE0 rzeaMVb7/e8TfFGvc3silNnnZH94MrG82bhei8UXdGJzSRKYN9N3P+7j5JL765r3gYYg xsJkb5PpiNZ28qcsfBvICDrF2vq3pim6wWyIXtAYvFU7/na02m5ynfX+x6Y38R1bwX+O iOGUVpBcdLXoJxooLepdbkdgQZaH9pNamldYkofT6airt0WlXFkmNwojxnaSZKwQcpW8 Kf51nSkjTc7T+C32cqre8ZUOSC/5FQlpR2blSY/QAnEWiv0FdPANexQv0kCmcSb0Y/vF lcNw== X-Gm-Message-State: AOJu0Yy3eyp/5GNNY8iaJUOGUoA2ChY1G6BFRYZwzh1P4YDuzEYi5j2j GlLDuhyPEhJZeBvJSMzrJ9xsgCRtkoGxpkzbkygqogrRPiGVelUyv7VN6eQQuMZ8jiD9QH019Q4 oskdz X-Gm-Gg: ASbGncsAQ3Xa14iK6MZCcl5xhu0Llhmk3tCAnns+7dcPl3P4rciLLvhguc0fPD5qivn YolhiyIfMiFls5j2rBKSOzkxHWgJ1aJhyjBvgncwGhoMnP6WkwhDzJM0JL3YzSPYleSRxHy+6JR IX0mD1Yxw8YiXVuJd95GTdwfSJ+xqBP9fgNBANO2RD/xRWBFSNASif0cqFhkLtDoowRoFehTdrZ BnRhgfxUb3UrOkMlircJluOFXg/QRbwK4xAtPmb4nzVFuiDh/Yjg1Crh3ADtiHh7gofXQabK6UC 8ce+j3EX5ob4srReLoBcsxLwRurw132PsP7NV187vseZCQF72wfTi3KSvD0ibSSYzsh97jY= X-Google-Smtp-Source: AGHT+IEUeIzVYBaXlphr4CKcpPM31oDxnW7w5WZF9G6M/+VeDpegRRh6rCKi34TVnOj+4bIlPz06xg== X-Received: by 2002:a5d:404a:0:b0:3a3:62e1:a7dc with SMTP id ffacd0b85a97d-3a362e1ab39mr2706273f8f.1.1747646572691; Mon, 19 May 2025 02:22:52 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a366e08747sm8645961f8f.95.2025.05.19.02.22.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 May 2025 02:22:52 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck Subject: [PATCH v1 3/6] pipeline: ipa: rpi: Split RPiCameraData::dropFrameCount_ Date: Mon, 19 May 2025 10:20:51 +0100 Message-ID: <20250519092245.269048-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250519092245.269048-1-naush@raspberrypi.com> References: <20250519092245.269048-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Split the pipeline handler drop frame tracking into startup frames and invalid frames, as reported by the IPA. Remove the drop buffer handling logic in the pipeline handler. Now all image buffers are returned out with the appropriate FrameStatus set for startup or invalid frames. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- .../pipeline/rpi/common/pipeline_base.cpp | 88 +++++++------------ .../pipeline/rpi/common/pipeline_base.h | 5 +- 2 files changed, 37 insertions(+), 56 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 5956485953a2..21f2daf5bab5 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -659,9 +659,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) if (!result.controls.empty()) data->setSensorControls(result.controls); - /* Configure the number of dropped frames required on startup. */ - data->dropFrameCount_ = data->config_.disableStartupFrameDrops ? - 0 : result.startupFrameCount + result.invalidFrameCount; + /* Configure the number of startup and invalid frames reported by the IPA. */ + data->startupFrameCount_ = result.startupFrameCount; + data->invalidFrameCount_ = result.invalidFrameCount; for (auto const stream : data->streams_) stream->resetBuffers(); @@ -678,7 +678,6 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) data->buffersAllocated_ = true; } - /* We need to set the dropFrameCount_ before queueing buffers. */ ret = queueAllBuffers(camera); if (ret) { LOG(RPI, Error) << "Failed to queue buffers"; @@ -898,23 +897,6 @@ int PipelineHandlerBase::queueAllBuffers(Camera *camera) ret = stream->queueAllBuffers(); if (ret < 0) return ret; - } else { - /* - * For external streams, we must queue up a set of internal - * buffers to handle the number of drop frames requested by - * the IPA. This is done by passing nullptr in queueBuffer(). - * - * The below queueBuffer() call will do nothing if there - * are not enough internal buffers allocated, but this will - * be handled by queuing the request for buffers in the - * RPiStream object. - */ - unsigned int i; - for (i = 0; i < data->dropFrameCount_; i++) { - ret = stream->queueBuffer(nullptr); - if (ret) - return ret; - } } } @@ -1412,7 +1394,15 @@ void CameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::Stream *stream) * buffer back to the stream. */ Request *request = requestQueue_.empty() ? nullptr : requestQueue_.front(); - if (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) { + if (request && request->findBuffer(stream) == buffer) { + FrameMetadata &md = buffer->_d()->metadata(); + + /* Mark the non-converged and invalid frames in the metadata. */ + if (startupFrameCount_) + md.status = FrameMetadata::Status::FrameStartup; + if (invalidFrameCount_) + md.status = FrameMetadata::Status::FrameError; + /* * Tag the buffer as completed, returning it to the * application. @@ -1458,42 +1448,32 @@ void CameraData::handleState() void CameraData::checkRequestCompleted() { - bool requestCompleted = false; - /* - * If we are dropping this frame, do not touch the request, simply - * change the state to IDLE when ready. - */ - if (!dropFrameCount_) { - Request *request = requestQueue_.front(); - if (request->hasPendingBuffers()) - return; + Request *request = requestQueue_.front(); + if (request->hasPendingBuffers()) + return; - /* Must wait for metadata to be filled in before completing. */ - if (state_ != State::IpaComplete) - return; + /* Must wait for metadata to be filled in before completing. */ + if (state_ != State::IpaComplete) + return; - LOG(RPI, Debug) << "Completing request sequence: " - << request->sequence(); + LOG(RPI, Debug) << "Completing request sequence: " + << request->sequence(); - pipe()->completeRequest(request); - requestQueue_.pop(); - requestCompleted = true; - } + pipe()->completeRequest(request); + requestQueue_.pop(); - /* - * Make sure we have three outputs completed in the case of a dropped - * frame. - */ - if (state_ == State::IpaComplete && - ((ispOutputCount_ == ispOutputTotal_ && dropFrameCount_) || - requestCompleted)) { - LOG(RPI, Debug) << "Going into Idle state"; - state_ = State::Idle; - if (dropFrameCount_) { - dropFrameCount_--; - LOG(RPI, Debug) << "Dropping frame at the request of the IPA (" - << dropFrameCount_ << " left)"; - } + LOG(RPI, Debug) << "Going into Idle state"; + state_ = State::Idle; + + if (startupFrameCount_) { + startupFrameCount_--; + LOG(RPI, Debug) << "Decrementing startup frames to " + << startupFrameCount_; + } + if (invalidFrameCount_) { + invalidFrameCount_--; + LOG(RPI, Debug) << "Decrementing invalid frames to " + << invalidFrameCount_; } } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index aae0c2f35888..6023f9f9d6b3 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -48,7 +48,7 @@ class CameraData : public Camera::Private public: CameraData(PipelineHandler *pipe) : Camera::Private(pipe), state_(State::Stopped), - dropFrameCount_(0), buffersAllocated_(false), + startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false), ispOutputCount_(0), ispOutputTotal_(0) { } @@ -151,7 +151,8 @@ public: /* Mapping of CropParams keyed by the output stream order in CameraConfiguration */ std::map cropParams_; - unsigned int dropFrameCount_; + unsigned int startupFrameCount_; + unsigned int invalidFrameCount_; /* * If set, this stores the value that represets a gain of one for