From patchwork Fri Jun 6 10:55:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23477 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ADC82C3292 for ; Fri, 6 Jun 2025 10:57:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2B31A68DCA; Fri, 6 Jun 2025 12:57:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="a5VGpBlt"; dkim-atps=neutral Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3206D68DB0 for ; Fri, 6 Jun 2025 12:56:59 +0200 (CEST) Received: by mail-wr1-x42b.google.com with SMTP id ffacd0b85a97d-3a4ee391e6fso176689f8f.3 for ; Fri, 06 Jun 2025 03:56:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1749207418; x=1749812218; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AWiiOo6NhxVE0fDk/z9rUm9ZZS3TZ/840uwenlIidpI=; b=a5VGpBlt+mOXEal1wtdaErniHPttoqAw5SRK0V5JS2Sv6fj45I7kGMecUajF7dwYmu uI6oY11O/0drf53VpupzvXeuTFKRHfyoxhU2PaTNrMZ9phd3S9bSlV8BfZPyTU0DHNx7 JUl9oJM+zfWmfL7v0YwYmGfLAhsoFM86si5+k4RdjTYjw6/Grw3XZG2mwsnMO2in/uJi ilkzpzZV55z4+GBFQDafXz+4jhfGgG6jG51P/xy4QvIwU5WokCTa3MQ1WfEzPVlHDPU4 E27CGV19WbAEolcxQ+6+hD6GcyzqPFoueeHBpUdpeEQAPF4mNFkaRqVGTcei9tqb8G8D HVXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749207418; x=1749812218; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AWiiOo6NhxVE0fDk/z9rUm9ZZS3TZ/840uwenlIidpI=; b=EUO3Be9vXhks1FX0o63nU88SDhRRE3sCkJcFU2v6yB0kXVOnFtpMhkMB4wMwS8K9W/ 0eooIu430MhHfQaX4qLBkMdljKp4DwAL/iU23vDDQb+ioJzhc5VXcagpDWm5ePf+usJ1 4QcJ4r4MfR2SoU8OJMyI3dVphDFDwgopjHswh3pRqvVNU3KrgPLm5gmCh+CcX1wTCUsJ q+j1ZFa43H6+zOMoKkEinvoSBjPXWNpkOWVxWFH7OgnNQMhOybRZwNnuGLdR0AjZ1+kg r/Y6du8XCgwzq571bXFIR5SnhsoeZ2ycesMReZJ7C5CFj65I1XmKcuISsjOjjDlX2iM0 O4Kg== X-Gm-Message-State: AOJu0YzYY42uwCN3YoYLsVrvpXBkWOyt/81eAqqo5ZLNXGLAnuy1SN40 hlKRUqHrZ5LY6R5OmC30ousn/dunLmA87pxJjeL/ES/AsSw80pMjsV8iW8WPPqs1b8K3K7rzYQq dzQdK X-Gm-Gg: ASbGncvVHGzN9hYv2KP7ZiYg2Z3/HnQnYP2vC9vQOqiQ2vlstroLI+g33W16oVJYT86 erc36aQhbZO+FBagDt5bsHD8jNHakbiEAJ/ylhYSkd4D1Toq6QWjrSc6isatxH+SoBRg0DwZlmy hoW9zIeG+cyAb9p9d2Rt9sfpMNUi9pFN3Cmd64pPKhG38xl62BWvWrPZOxe23CqA9CUtBg5KL6q xv/g88VxVAqJZIi2WANR3LGYWBXl/x/qksDyoUzxdw07lWsW6Yx8ZshHbfK9fzcBLN+fBYUQC28 D5h3KaJnN7LGNvoVEodySSn/A5J3lfWjhVspt2t0jNeV2eLhFkU93bDcOaMB/v7ZPMp08h7A65t MDu3DzXoWSA4= X-Google-Smtp-Source: AGHT+IFSNBIaMZVCMznBifiyAi49yzhyLjungw2y27LjwIVdocrnpsCpuQ5onWq9sdrgWDkyytoHAA== X-Received: by 2002:a05:6000:400e:b0:3a3:63d3:368e with SMTP id ffacd0b85a97d-3a532f7e27cmr590673f8f.0.1749207418427; Fri, 06 Jun 2025 03:56:58 -0700 (PDT) Received: from NAUSH-P-DELL.tail9c427.ts.net ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-452137277afsm20092455e9.30.2025.06.06.03.56.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Jun 2025 03:56:57 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman , Jacopo Mondi Subject: [PATCH v3 3/6] pipeline: ipa: rpi: Split RPiCameraData::dropFrameCount_ Date: Fri, 6 Jun 2025 11:55:21 +0100 Message-ID: <20250606105651.1624640-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250606105651.1624640-1-naush@raspberrypi.com> References: <20250606105651.1624640-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Split the pipeline handler drop frame tracking into startup frames and invalid frames, as reported by the IPA. Remove the drop buffer handling logic in the pipeline handler. Now all image buffers are returned out with the appropriate FrameStatus set for startup or invalid frames. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- .../pipeline/rpi/common/pipeline_base.cpp | 98 ++++++++----------- .../pipeline/rpi/common/pipeline_base.h | 5 +- 2 files changed, 42 insertions(+), 61 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 5956485953a2..3f0b7abdc59a 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -659,9 +659,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) if (!result.controls.empty()) data->setSensorControls(result.controls); - /* Configure the number of dropped frames required on startup. */ - data->dropFrameCount_ = data->config_.disableStartupFrameDrops ? - 0 : result.startupFrameCount + result.invalidFrameCount; + /* Configure the number of startup and invalid frames reported by the IPA. */ + data->startupFrameCount_ = result.startupFrameCount; + data->invalidFrameCount_ = result.invalidFrameCount; for (auto const stream : data->streams_) stream->resetBuffers(); @@ -678,7 +678,6 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) data->buffersAllocated_ = true; } - /* We need to set the dropFrameCount_ before queueing buffers. */ ret = queueAllBuffers(camera); if (ret) { LOG(RPI, Error) << "Failed to queue buffers"; @@ -894,28 +893,12 @@ int PipelineHandlerBase::queueAllBuffers(Camera *camera) int ret; for (auto const stream : data->streams_) { - if (!(stream->getFlags() & StreamFlag::External)) { - ret = stream->queueAllBuffers(); - if (ret < 0) - return ret; - } else { - /* - * For external streams, we must queue up a set of internal - * buffers to handle the number of drop frames requested by - * the IPA. This is done by passing nullptr in queueBuffer(). - * - * The below queueBuffer() call will do nothing if there - * are not enough internal buffers allocated, but this will - * be handled by queuing the request for buffers in the - * RPiStream object. - */ - unsigned int i; - for (i = 0; i < data->dropFrameCount_; i++) { - ret = stream->queueBuffer(nullptr); - if (ret) - return ret; - } - } + if (stream->getFlags() & StreamFlag::External) + continue; + + ret = stream->queueAllBuffers(); + if (ret < 0) + return ret; } return 0; @@ -1412,7 +1395,15 @@ void CameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::Stream *stream) * buffer back to the stream. */ Request *request = requestQueue_.empty() ? nullptr : requestQueue_.front(); - if (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) { + if (request && request->findBuffer(stream) == buffer) { + FrameMetadata &md = buffer->_d()->metadata(); + + /* Mark the non-converged and invalid frames in the metadata. */ + if (invalidFrameCount_) + md.status = FrameMetadata::Status::FrameError; + else if (startupFrameCount_) + md.status = FrameMetadata::Status::FrameStartup; + /* * Tag the buffer as completed, returning it to the * application. @@ -1458,42 +1449,31 @@ void CameraData::handleState() void CameraData::checkRequestCompleted() { - bool requestCompleted = false; - /* - * If we are dropping this frame, do not touch the request, simply - * change the state to IDLE when ready. - */ - if (!dropFrameCount_) { - Request *request = requestQueue_.front(); - if (request->hasPendingBuffers()) - return; + Request *request = requestQueue_.front(); + if (request->hasPendingBuffers()) + return; - /* Must wait for metadata to be filled in before completing. */ - if (state_ != State::IpaComplete) - return; + /* Must wait for metadata to be filled in before completing. */ + if (state_ != State::IpaComplete) + return; - LOG(RPI, Debug) << "Completing request sequence: " - << request->sequence(); + LOG(RPI, Debug) << "Completing request sequence: " + << request->sequence(); - pipe()->completeRequest(request); - requestQueue_.pop(); - requestCompleted = true; - } + pipe()->completeRequest(request); + requestQueue_.pop(); - /* - * Make sure we have three outputs completed in the case of a dropped - * frame. - */ - if (state_ == State::IpaComplete && - ((ispOutputCount_ == ispOutputTotal_ && dropFrameCount_) || - requestCompleted)) { - LOG(RPI, Debug) << "Going into Idle state"; - state_ = State::Idle; - if (dropFrameCount_) { - dropFrameCount_--; - LOG(RPI, Debug) << "Dropping frame at the request of the IPA (" - << dropFrameCount_ << " left)"; - } + LOG(RPI, Debug) << "Going into Idle state"; + state_ = State::Idle; + + if (invalidFrameCount_) { + invalidFrameCount_--; + LOG(RPI, Debug) << "Decrementing invalid frames to " + << invalidFrameCount_; + } else if (startupFrameCount_) { + startupFrameCount_--; + LOG(RPI, Debug) << "Decrementing startup frames to " + << startupFrameCount_; } } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index aae0c2f35888..6023f9f9d6b3 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -48,7 +48,7 @@ class CameraData : public Camera::Private public: CameraData(PipelineHandler *pipe) : Camera::Private(pipe), state_(State::Stopped), - dropFrameCount_(0), buffersAllocated_(false), + startupFrameCount_(0), invalidFrameCount_(0), buffersAllocated_(false), ispOutputCount_(0), ispOutputTotal_(0) { } @@ -151,7 +151,8 @@ public: /* Mapping of CropParams keyed by the output stream order in CameraConfiguration */ std::map cropParams_; - unsigned int dropFrameCount_; + unsigned int startupFrameCount_; + unsigned int invalidFrameCount_; /* * If set, this stores the value that represets a gain of one for