From patchwork Mon Feb 7 15:12:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 15341 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E6B6BBF415 for ; Mon, 7 Feb 2022 15:12:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2E9FE61085; Mon, 7 Feb 2022 16:12:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="TLJtr/GO"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ADD396105F for ; Mon, 7 Feb 2022 16:12:19 +0100 (CET) Received: by mail-wm1-x32c.google.com with SMTP id v129so6564154wme.2 for ; Mon, 07 Feb 2022 07:12:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ldwwezxd1MbR/3Abq8SKMJrIDtvxWlFxRAk2j3Obmr4=; b=TLJtr/GOsncYL+3DHUixneHDa+yRxH+lFEEPKM2KZTsGCrYojXjWd8HRn7ev6cETCs 5EaUBpuAowSUDW+W16NsvDM4Oh8GZSlFC+NL9RKObhJx1cdyCOe9DmGnDWNp4BuFhlNx CO9AjymvPEA0Z3CRy6p75DMjla6GueqOZ/i6jwqmsVjlolI/P4iXaqm4tNqhJUtGT5xY 8uurklDTxYUFoZFXTVAxcskCDNpwMFkE5HrgzHZdYgyKeG/Cunk95JpY9kyvoBVneNJQ OSXCnvpvK4XsX8WkEKK0/ug6ctMU0h0+ApDoO6RV093pVD6sgsv9+NN+BHK0aFSdbgZE 6ocA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ldwwezxd1MbR/3Abq8SKMJrIDtvxWlFxRAk2j3Obmr4=; b=foARRTQX2iKMVZRZ5dZRX93Aw4Fa/b4+4gmz7AG5iNnKfjJw6Dh7tZdp6B1gIhoMAn vVQxpyd2IN72Xoa/FT/DBNIoi5VP9VjAFj2SwN3u7qO1EsxTswtUC0b7jF3gCpSfU62g Gjj6tXl3AzOSRCqIG8DYO9oDKplWGRchglrLVlZk2SeQUxo/5csuP5prb2e6r+uEH3D+ s9os73oQLqhOae25ZxctgedhG3Xc7R6d6UqsivaBLMZ/yKe4xILBgM9IEfcsQmqOwIfT 4oa2ytaELOXaBPrqqXKhNmEibeK4aIhZOTWBvYYcEYu6sCqRSc++CmSVlKX+Wxst6Omo skuQ== X-Gm-Message-State: AOAM530UpUAGAmISMIr2ckh6psZXJHS93pVa3ZDM7XmdhOZWiTEdllLt BsaPyYR1iw9duaF66AaAQgJNhcLGmuWrkQ== X-Google-Smtp-Source: ABdhPJxwzU1Rt+onGgYtFfEc0U2r3zU28SDPnIclztj4VeVnglwXrF4fApsog5sFcVOtpWGdIyOtdw== X-Received: by 2002:a1c:f312:: with SMTP id q18mr11788533wmq.13.1644246739142; Mon, 07 Feb 2022 07:12:19 -0800 (PST) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:4a96:1adf:459f:8a7e]) by smtp.gmail.com with ESMTPSA id i19sm21463085wmq.45.2022.02.07.07.12.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Feb 2022 07:12:18 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 7 Feb 2022 15:12:13 +0000 Message-Id: <20220207151214.887140-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220207151214.887140-1-naush@raspberrypi.com> References: <20220207151214.887140-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/3] pipeline: raspberrypi: Simplify image/embedded buffer matching logic X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Simplify the image and embedded buffer matching logic by removing the assumption that we require a buffer match between the two streams. Instead, if an image buffer does not match with an embedded data buffer, simply use the ControlList provided by DelayedControls for the sensor parameters. Signed-off-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- .../pipeline/raspberrypi/raspberrypi.cpp | 141 ++++-------------- 1 file changed, 31 insertions(+), 110 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 0755de84c70c..af234bd18c5b 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -2081,122 +2081,43 @@ void RPiCameraData::tryRunPipeline() bool RPiCameraData::findMatchingBuffers(BayerFrame &bayerFrame, FrameBuffer *&embeddedBuffer) { - unsigned int embeddedRequeueCount = 0, bayerRequeueCount = 0; - - /* Loop until we find a matching bayer and embedded data buffer. */ - while (!bayerQueue_.empty()) { - /* Start with the front of the bayer queue. */ - FrameBuffer *bayerBuffer = bayerQueue_.front().buffer; - - /* - * Find the embedded data buffer with a matching timestamp to pass to - * the IPA. Any embedded buffers with a timestamp lower than the - * current bayer buffer will be removed and re-queued to the driver. - */ - uint64_t ts = bayerBuffer->metadata().timestamp; - embeddedBuffer = nullptr; - while (!embeddedQueue_.empty()) { - FrameBuffer *b = embeddedQueue_.front(); - if (!unicam_[Unicam::Embedded].isExternal() && b->metadata().timestamp < ts) { - embeddedQueue_.pop(); - unicam_[Unicam::Embedded].returnBuffer(b); - embeddedRequeueCount++; - LOG(RPI, Warning) << "Dropping unmatched input frame in stream " - << unicam_[Unicam::Embedded].name(); - } else if (unicam_[Unicam::Embedded].isExternal() || b->metadata().timestamp == ts) { - /* We pop the item from the queue lower down. */ - embeddedBuffer = b; - break; - } else { - break; /* Only higher timestamps from here. */ - } - } - - if (!embeddedBuffer) { - bool flushedBuffers = false; - - LOG(RPI, Debug) << "Could not find matching embedded buffer"; - - if (!sensorMetadata_) { - /* - * If there is no sensor metadata, simply return the - * first bayer frame in the queue. - */ - LOG(RPI, Debug) << "Returning bayer frame without a match"; - bayerFrame = std::move(bayerQueue_.front()); - bayerQueue_.pop(); - embeddedBuffer = nullptr; - return true; - } - - if (!embeddedQueue_.empty()) { - /* - * Not found a matching embedded buffer for the bayer buffer in - * the front of the queue. This buffer is now orphaned, so requeue - * it back to the device. - */ - unicam_[Unicam::Image].returnBuffer(bayerQueue_.front().buffer); - bayerQueue_.pop(); - bayerRequeueCount++; - LOG(RPI, Warning) << "Dropping unmatched input frame in stream " - << unicam_[Unicam::Image].name(); - } - - /* - * If we have requeued all available embedded data buffers in this loop, - * then we are fully out of sync, so might as well requeue all the pending - * bayer buffers. - */ - if (embeddedRequeueCount == unicam_[Unicam::Embedded].getBuffers().size()) { - /* The embedded queue must be empty at this point! */ - ASSERT(embeddedQueue_.empty()); - - LOG(RPI, Warning) << "Flushing bayer stream!"; - while (!bayerQueue_.empty()) { - unicam_[Unicam::Image].returnBuffer(bayerQueue_.front().buffer); - bayerQueue_.pop(); - } - flushedBuffers = true; - } + if (bayerQueue_.empty()) + return false; - /* - * Similar to the above, if we have requeued all available bayer buffers in - * the loop, then we are fully out of sync, so might as well requeue all the - * pending embedded data buffers. - */ - if (bayerRequeueCount == unicam_[Unicam::Image].getBuffers().size()) { - /* The bayer queue must be empty at this point! */ - ASSERT(bayerQueue_.empty()); - - LOG(RPI, Warning) << "Flushing embedded data stream!"; - while (!embeddedQueue_.empty()) { - unicam_[Unicam::Embedded].returnBuffer(embeddedQueue_.front()); - embeddedQueue_.pop(); - } - flushedBuffers = true; - } + /* Start with the front of the bayer queue. */ + bayerFrame = std::move(bayerQueue_.front()); + bayerQueue_.pop(); - /* - * If the embedded queue has become empty, we cannot do any more. - * Similarly, if we have flushed any one of our queues, we cannot do - * any more. Return from here without a buffer pair. - */ - if (embeddedQueue_.empty() || flushedBuffers) - return false; - } else { - /* - * We have found a matching bayer and embedded data buffer, so - * nothing more to do apart from assigning the bayer frame and - * popping the buffers from the queue. - */ - bayerFrame = std::move(bayerQueue_.front()); - bayerQueue_.pop(); + /* + * Find the embedded data buffer with a matching timestamp to pass to + * the IPA. Any embedded buffers with a timestamp lower than the + * current bayer buffer will be removed and re-queued to the driver. + */ + uint64_t ts = bayerFrame.buffer->metadata().timestamp; + embeddedBuffer = nullptr; + while (!embeddedQueue_.empty()) { + FrameBuffer *b = embeddedQueue_.front(); + if (b->metadata().timestamp < ts) { + embeddedQueue_.pop(); + unicam_[Unicam::Embedded].returnBuffer(b); + LOG(RPI, Debug) << "Dropping unmatched input frame in stream " + << unicam_[Unicam::Embedded].name(); + } else if (b->metadata().timestamp == ts) { + /* Found a match! */ + embeddedBuffer = b; embeddedQueue_.pop(); - return true; + break; + } else { + break; /* Only higher timestamps from here. */ } } - return false; + if (!embeddedBuffer && sensorMetadata_) { + /* Log if there is no matching embedded data buffer found. */ + LOG(RPI, Debug) << "Returning bayer frame without a matching embedded buffer."; + } + + return true; } REGISTER_PIPELINE_HANDLER(PipelineHandlerRPi)