From patchwork Wed Jul 13 12:00:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 16621 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AA484BD1F1 for ; Wed, 13 Jul 2022 12:00:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0830B6330E; Wed, 13 Jul 2022 14:00:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1657713640; bh=29Lum28yNMjZv5s2PHjqhC6HGz01SxN8IMWP5t8rAcw=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=qRl/2eH9urF4ELvoZak+zCIR3dcp/94/Nds99hUcxmTP+WfWoCbZyov5ENnCA4x/V bgsHG6VhIZrRWO/KlGBUbZPBluSek+X7OeoU/niSO0Ca/BRMkxGZaEvwOPrJ3n4SPl l5Z4+b/dUp6/ZH7696Yce5QeJtHpspfWlElP0VCTVYj+mMQTqiHUglK5If1U4u4YbG YZEbZq3Y6KfSv+ZVDy3wOK1iHVEVC5fQtHN0x33HAql0EW2G2iAh3CXX7KZYUa0TJt zwfbO3qqdHO4KWu5UsyGWFKa19FGKfsy/69b1SWOrZwvqWD28DQJp36KkMqD2Wy/gZ b8572s14yeFyw== Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 843576274E for ; Wed, 13 Jul 2022 14:00:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="C7aCl4fH"; dkim-atps=neutral Received: by mail-wm1-x32f.google.com with SMTP id p4so6389908wms.0 for ; Wed, 13 Jul 2022 05:00:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=bsfTuucbWw8td0Lnu5XSHjzMDvmkMt1HhNuCPqZmtuc=; b=C7aCl4fHKypAjZJL38viIZ6Qgs5GTHC56vTvtWVhNDv/O2dpaJE8W3ys4WcCSj/FHv wd4npgWyiMirPpgTP5GudSRr4NPMAFtWhnmmybFLYdyb3nZ8Stte/vzATmd2SrkTsVwe Z4/4qVKy4gujlUz/jCUawLp2ZjpSAHs8z6nMSHP6Z5YO+ChzbnXiI9OHWyK94YT5uYfr ajGY+/AvYWSDvspWOGYd0SYAj2ckvVWfImg+kKXBeguTLab0NHk2HEMNwikQENXmCuKZ DfknQCb0cQx1BGGwHCDMnlLXEPYeAmVXo8qwBVPa0xREPXKqsnJcqvSP3cx7PJyk8yMD iurg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=bsfTuucbWw8td0Lnu5XSHjzMDvmkMt1HhNuCPqZmtuc=; b=VgdHpelab4zAb3MKt8nwKahwlzoyPjtZVS1jQ9WhqvJQw9msk518XdesqCCfSQrVE0 oYtBt6nbVWRw/vOkTKCD1FmD+Au9x9lLt65ZuI7quHiOBK0K1EtMkHATd78TBX/GUoH1 yYYHlCf8Itlq3OS9+1jROdlIahb/9B6RNnUbqpojUkZsBEna8Kz8yDVM6tFO8hdMFlSr yWWIB6Af1H56jVuvept6TRNWgMceRD918VebgyVnilIuQgZf4MoKq1T4tN6FjpNXIZuQ 4b+shKDn7LGQth50HCTZsV+SUzqddVNAo2wB5TOTFicDBhmatOb1pXMUfPaA/Z7+BQsJ b6UQ== X-Gm-Message-State: AJIora/1+QkZfGLi0ZKGT3LdScSqE1p04wIVW27Ue8XCQvqCgUqMoE2M YGYTqupXD9jQ1iB6Bz3PUhkHk665SLwHqg== X-Google-Smtp-Source: AGRyM1v8EMmHDj+RVpHOvU1FVvKpCUu1r6paHfEykvcZaPrKc1snkEOtgPAyOXW6KNUSHi1TE+MNMQ== X-Received: by 2002:a05:600c:3b9b:b0:3a2:d6d8:cfa7 with SMTP id n27-20020a05600c3b9b00b003a2d6d8cfa7mr3221644wms.77.1657713636860; Wed, 13 Jul 2022 05:00:36 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id o38-20020a05600c512600b003a2e2e965absm2061713wms.20.2022.07.13.05.00.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:00:36 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 13 Jul 2022 13:00:35 +0100 Message-Id: <20220713120035.15604-1-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH] pipeline: raspberrypi: Improve image/embedded buffer matching logic X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Nick Hollinghurst Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The logic used to match asynchronous image and embedded buffers was being overly aggressive by possibly allowing an unmatched image buffer to be sent to the IPA if the matching embedded buffer had not yet been dequeued. This condition only occurs when the system is heavily loaded and dropping frames. Fix this by holding image buffer in the queue during these conditions until the next embedded buffer dequeue event. Reported-by: Nick Hollinghurst Signed-off-by: Naushir Patuck Reviewed-by: Nick Hollinghurst Tested-by: Nick Hollinghurst Reviewed-by: Laurent Pinchart --- .../pipeline/raspberrypi/raspberrypi.cpp | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 66a84b1dfb97..ef3c2d11d253 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -2159,16 +2159,12 @@ bool RPiCameraData::findMatchingBuffers(BayerFrame &bayerFrame, FrameBuffer *&em if (bayerQueue_.empty()) return false; - /* Start with the front of the bayer queue. */ - bayerFrame = std::move(bayerQueue_.front()); - bayerQueue_.pop(); - /* * Find the embedded data buffer with a matching timestamp to pass to * the IPA. Any embedded buffers with a timestamp lower than the * current bayer buffer will be removed and re-queued to the driver. */ - uint64_t ts = bayerFrame.buffer->metadata().timestamp; + uint64_t ts = bayerQueue_.front().buffer->metadata().timestamp; embeddedBuffer = nullptr; while (!embeddedQueue_.empty()) { FrameBuffer *b = embeddedQueue_.front(); @@ -2188,10 +2184,23 @@ bool RPiCameraData::findMatchingBuffers(BayerFrame &bayerFrame, FrameBuffer *&em } if (!embeddedBuffer && sensorMetadata_) { + if (embeddedQueue_.empty()) { + /* + * If the embedded buffer queue is empty, wait for the next + * buffer to arrive - dequeue ordering may send the image + * buffer first. + */ + LOG(RPI, Debug) << "Waiting for next embedded buffer."; + return false; + } + /* Log if there is no matching embedded data buffer found. */ LOG(RPI, Debug) << "Returning bayer frame without a matching embedded buffer."; } + bayerFrame = std::move(bayerQueue_.front()); + bayerQueue_.pop(); + return true; }