From patchwork Fri Jun 6 16:41:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23499 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A3E11C332E for ; Fri, 6 Jun 2025 16:42:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1E19568DD2; Fri, 6 Jun 2025 18:42:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pp3FBgoL"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 96C1C68DD3 for ; Fri, 6 Jun 2025 18:42:20 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 224156DC for ; Fri, 6 Jun 2025 18:42:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228136; bh=v4haTZL+iX8qFxPedc94HVWa6Y3xwi6lSmxDSlZEqSQ=; h=From:To:Subject:Date:In-Reply-To:References:From; b=pp3FBgoLpfhjhH84HlnN0IIip6dIhlIxbUaOmTyENc70CBLS/iAa23Etk3eIwM1rU /OL5rFqM/yuMTwkteOCUXGgFU9c89O2zLjqo+halTydr+FlEtGchkfNJqu0b1k786m zpYQF9j1roZ0kpWquQJ13VOGolOFYYlsQ0dkJxFY= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 20/23] libcamera: pipeline: rpi: Queue metadata until completion Date: Fri, 6 Jun 2025 18:41:53 +0200 Message-ID: <20250606164156.1442682-21-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The rpi pipeline drops a certain number of initial frames. It uses `ControlList::clear()` to remove the metadata items of a request that was populated while processing a frame that is ultimately dropped. Since the decision to drop a frame only occurs at the very end of the processing, this stop-gap measure is introduced to delay metadata completion until the request is actually completed. Obsoleted by https://patchwork.libcamera.org/cover/23474/. Signed-off-by: Barnabás Pőcze --- .../pipeline/rpi/common/pipeline_base.cpp | 29 +++++++++++-------- .../pipeline/rpi/common/pipeline_base.h | 4 +-- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 8 ++--- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 8 ++--- 4 files changed, 27 insertions(+), 22 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 98507a152..d432cfb51 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -780,7 +780,7 @@ int PipelineHandlerBase::queueRequestDevice(Camera *camera, Request *request) } /* Push the request to the back of the queue. */ - data->requestQueue_.push(request); + data->requestQueue_.push({ request, {} }); data->handleState(); return 0; @@ -1232,8 +1232,8 @@ void CameraData::metadataReady(const ControlList &metadata) /* Add to the Request metadata buffer what the IPA has provided. */ /* Last thing to do is to fill up the request metadata. */ - Request *request = requestQueue_.front(); - request->metadata().merge(metadata); + ControlList &requestMetadata = requestQueue_.front().second; + requestMetadata.merge(metadata); /* * Inform the sensor of the latest colour gains if it has the @@ -1392,7 +1392,7 @@ void CameraData::clearIncompleteRequests() * back to the application. */ while (!requestQueue_.empty()) { - Request *request = requestQueue_.front(); + auto &[request, metadata] = requestQueue_.front(); for (auto &b : request->buffers()) { FrameBuffer *buffer = b.second; @@ -1406,6 +1406,9 @@ void CameraData::clearIncompleteRequests() } } + // TODO: need this when cancelled? + request->metadata().merge(metadata); + pipe()->completeRequest(request); requestQueue_.pop(); } @@ -1418,7 +1421,7 @@ void CameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::Stream *stream) * that we actually have one to action, otherwise we just return * buffer back to the stream. */ - Request *request = requestQueue_.empty() ? nullptr : requestQueue_.front(); + Request *request = requestQueue_.empty() ? nullptr : requestQueue_.front().first; if (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) { /* * Tag the buffer as completed, returning it to the @@ -1471,7 +1474,7 @@ void CameraData::checkRequestCompleted() * change the state to IDLE when ready. */ if (!dropFrameCount_) { - Request *request = requestQueue_.front(); + auto &[request, metadata] = requestQueue_.front(); if (request->hasPendingBuffers()) return; @@ -1482,6 +1485,8 @@ void CameraData::checkRequestCompleted() LOG(RPI, Debug) << "Completing request sequence: " << request->sequence(); + request->metadata().merge(metadata); + pipe()->completeRequest(request); requestQueue_.pop(); requestCompleted = true; @@ -1504,10 +1509,10 @@ void CameraData::checkRequestCompleted() } } -void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request *request) +void CameraData::fillRequestMetadata(const ControlList &bufferControls, ControlList &metadata) { - request->metadata().set(controls::SensorTimestamp, - bufferControls.get(controls::SensorTimestamp).value_or(0)); + metadata.set(controls::SensorTimestamp, + bufferControls.get(controls::SensorTimestamp).value_or(0)); if (cropParams_.size()) { std::vector crops; @@ -1515,10 +1520,10 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request for (auto const &[k, v] : cropParams_) crops.push_back(scaleIspCrop(v.ispCrop)); - request->metadata().set(controls::ScalerCrop, crops[0]); + metadata.set(controls::ScalerCrop, crops[0]); if (crops.size() > 1) { - request->metadata().set(controls::rpi::ScalerCrops, - Span(crops.data(), crops.size())); + metadata.set(controls::rpi::ScalerCrops, + Span(crops.data(), crops.size())); } } } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index aae0c2f35..d3a1bd216 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -129,7 +129,7 @@ public: return state_ != State::Stopped && state_ != State::Error; } - std::queue requestQueue_; + std::queue> requestQueue_; /* For handling digital zoom. */ IPACameraSensorInfo sensorInfo_; @@ -179,7 +179,7 @@ public: protected: void fillRequestMetadata(const ControlList &bufferControls, - Request *request); + ControlList &metadata); virtual void tryRunPipeline() = 0; diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index 91e7f4c94..8436cb4fa 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -2318,7 +2318,7 @@ void PiSPCameraData::tryRunPipeline() CfeJob &job = cfeJobQueue_.front(); /* Take the first request from the queue and action the IPA. */ - Request *request = requestQueue_.front(); + auto &[request, metadata] = requestQueue_.front(); /* See if a new ScalerCrop value needs to be applied. */ applyScalerCrop(request->controls()); @@ -2328,8 +2328,8 @@ void PiSPCameraData::tryRunPipeline() * related controls. We clear it first because the request metadata * may have been populated if we have dropped the previous frame. */ - request->metadata().clear(); - fillRequestMetadata(job.sensorControls, request); + metadata.clear(); + fillRequestMetadata(job.sensorControls, metadata); /* Set our state to say the pipeline is active. */ state_ = State::Busy; @@ -2347,7 +2347,7 @@ void PiSPCameraData::tryRunPipeline() params.buffers.bayer = RPi::MaskBayerData | bayerId; params.buffers.stats = RPi::MaskStats | statsId; params.buffers.embedded = 0; - params.ipaContext = requestQueue_.front()->sequence(); + params.ipaContext = requestQueue_.front().first->sequence(); params.delayContext = job.delayContext; params.sensorControls = std::move(job.sensorControls); params.requestControls = request->controls(); diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index fe910bdf2..a311f43a4 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -836,7 +836,7 @@ void Vc4CameraData::ispOutputDequeue(FrameBuffer *buffer) if (stream == &isp_[Isp::Stats]) { ipa::RPi::ProcessParams params; params.buffers.stats = index | RPi::MaskStats; - params.ipaContext = requestQueue_.front()->sequence(); + params.ipaContext = requestQueue_.front().first->sequence(); ipa_->processStats(params); } else { /* Any other ISP output can be handed back to the application now. */ @@ -935,7 +935,7 @@ void Vc4CameraData::tryRunPipeline() return; /* Take the first request from the queue and action the IPA. */ - Request *request = requestQueue_.front(); + auto &[request, metadata] = requestQueue_.front(); /* See if a new ScalerCrop value needs to be applied. */ applyScalerCrop(request->controls()); @@ -945,8 +945,8 @@ void Vc4CameraData::tryRunPipeline() * related controls. We clear it first because the request metadata * may have been populated if we have dropped the previous frame. */ - request->metadata().clear(); - fillRequestMetadata(bayerFrame.controls, request); + metadata.clear(); + fillRequestMetadata(bayerFrame.controls, metadata); /* Set our state to say the pipeline is active. */ state_ = State::Busy;