From patchwork Thu Feb 18 17:01:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 11340 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9AB81BD1F1 for ; Thu, 18 Feb 2021 17:01:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 31C28689CA; Thu, 18 Feb 2021 18:01:37 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Ww1jAaqj"; dkim-atps=neutral Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 00D12689C4 for ; Thu, 18 Feb 2021 18:01:35 +0100 (CET) Received: by mail-wm1-x32b.google.com with SMTP id o82so4433751wme.1 for ; Thu, 18 Feb 2021 09:01:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QOvqICc8HaszzrrUhEyZFOx1hnvZtHh8GWAPTiZOFQc=; b=Ww1jAaqjSAkt9u1JNCBFwGs8V8XNoXEa5/ofSqCBwuzWhmZE9OmhCutXjodrXPVRmF 6VIoD6SgPhX6ZVAeT03YuwYTv09FpjZLmYpifvp9XhthsL2aAy+utNoyccljgsAAOA8M YU9mBo6a4T3+z1mKkme86aRtfUGb+dTqA5agQoklLsp3R7tUISyj82DPeDnLuud8a1rL 1D8YkLoB2Y7ZryEdyef6XqrhdtQehF4HNhkuBITwGfTn9SgiPI1kgTBuYzhIopxJFHjN /9qhOJKD/EjCMGqn9bSjqfBE6OcAXOXlYUx2CLo/9m73Qc8jFFsCq60SavamLmIaDb0s 4vIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QOvqICc8HaszzrrUhEyZFOx1hnvZtHh8GWAPTiZOFQc=; b=DDe5QO+UW/53NAUN3FWmvGpqGHdNU+lcgnpStGW/R68ShuxeUmJhju8t8nD1fPzEJ5 SOBYToGKk8f/gfZHeH+Z1GVS3+N3xPFn0pvrFeidAkmKO3hYcDUTZAowaCtodSUyZwhB JTUyy73RGVsfDK/mM9ypqn0jJP95vEfsIj4w+mfAHvlbQcxfEqZxBB3nGmJUHLXcXq7A qYq3e/dB52+TLeKPP3JxNiboEAwFJJwVH4XkbubRtu7Gk6QH7lqVhqjmREbCWLvxru7r nZe87qF2kIo3gVO96MJ/PzCnPiTccAJK8GgR4K5nOOleV7tNtrRX9E6F8SVxyf38sJ7M 5uew== X-Gm-Message-State: AOAM533yv+/pDTWpcgUxqnZW6DtG2s0ulF6mk73Q9RK9EFGVD+hYESQR /AlCzBa5I/QwxoFEtumddnQ0fhEXrPkFLNCT X-Google-Smtp-Source: ABdhPJzRLFwpwcy1dvYftgMOeTD5zlcFNLbHYYJkl3/D9bF6feG1urbJyUV1gqf7BHkK70f+Ivup1g== X-Received: by 2002:a1c:771a:: with SMTP id t26mr4452037wmi.37.1613667694287; Thu, 18 Feb 2021 09:01:34 -0800 (PST) Received: from naush-laptop.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id t30sm10504747wra.56.2021.02.18.09.01.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Feb 2021 09:01:33 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Thu, 18 Feb 2021 17:01:26 +0000 Message-Id: <20210218170126.2060783-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210218170126.2060783-1-naush@raspberrypi.com> References: <20210218170126.2060783-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 4/4] pipeline: raspberrypi: Allow either strict or non-strict buffer matching X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" A new flag, StrictBufferMatching, is used to control the behavior of the embedded and bayer buffer matching routine. If set to true (default), we reject and drop all bayer frames that do not have a matching embedded data buffer and vice-versa. This guarantees the IPA will always have the correct frame exposure and gain values to use. If set to false, we use bayer frames that do not have a matching embedded data buffer. In this case, IPA will use use the ControlList passed to it for gain and exposure values. Additionally, allow external stream buffers to behave as if StrictBufferMatching = false since we are not allowed to drop them. Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 31 ++++++++++++++++--- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index d969c77993eb..7f66d6995176 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -46,6 +46,22 @@ namespace libcamera { LOG_DEFINE_CATEGORY(RPI) +/* + * Set this to true to reject and drop all bayer frames that do not have a + * matching embedded data buffer and vice-versa. This guarantees the IPA will + * always have the correct frame exposure and gain values to use. + * + * Set this to false to use bayer frames that do not have a matching embedded + * data buffer. In this case, IPA will use use our local history for gain and + * exposure values, occasional frame drops may cause these number to be out of + * sync for a short period. + * + * \todo Ideally, this property should be set by the application, but we don't + * have any mechanism to pass generic properties into a pipeline handler at + * present. + */ +static const bool StrictBufferMatching = true; + namespace { bool isRaw(PixelFormat &pixFmt) @@ -1724,13 +1740,13 @@ bool RPiCameraData::findMatchingBuffers(BayerFrame &bayerFrame, FrameBuffer *&em embeddedBuffer = nullptr; while (!embeddedQueue_.empty()) { FrameBuffer *b = embeddedQueue_.front(); - if (!unicam_[Unicam::Embedded].isExternal() && b->metadata().timestamp < ts) { + if (b->metadata().timestamp < ts) { embeddedQueue_.pop(); unicam_[Unicam::Embedded].queueBuffer(b); embeddedRequeueCount++; LOG(RPI, Warning) << "Dropping unmatched input frame in stream " << unicam_[Unicam::Embedded].name(); - } else if (unicam_[Unicam::Embedded].isExternal() || b->metadata().timestamp == ts) { + } else if (b->metadata().timestamp == ts) { /* We pop the item from the queue lower down. */ embeddedBuffer = b; break; @@ -1744,10 +1760,15 @@ bool RPiCameraData::findMatchingBuffers(BayerFrame &bayerFrame, FrameBuffer *&em LOG(RPI, Debug) << "Could not find matching embedded buffer"; - if (!sensorMetadata_) { + if (unicam_[Unicam::Embedded].isExternal() || + !StrictBufferMatching || !sensorMetadata_) { /* - * If there is no sensor metadata, simply return the - * first bayer frame in the queue. + * If any of the follwing is true: + * - This is an external stream buffer (so cannot be dropped). + * - We do not care about strict buffer matching. + * - There is no sensor metadata present. + * we can simply return the first bayer frame in the queue + * without a matching embedded buffer. */ LOG(RPI, Debug) << "Returning bayer frame without a match"; bayerQueue_.pop();