From patchwork Wed Feb 17 12:50:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 11327 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BDCC2BD1F1 for ; Wed, 17 Feb 2021 12:51:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8C6E46831A; Wed, 17 Feb 2021 13:51:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="iKLINvyu"; dkim-atps=neutral Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8ACCC6381E for ; Wed, 17 Feb 2021 13:51:31 +0100 (CET) Received: by mail-wm1-x332.google.com with SMTP id x16so2043252wmk.3 for ; Wed, 17 Feb 2021 04:51:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=A6IO9Hb2rCfpf+glJ0pLcQzpWIUiMwqvq1rQ0jE7FUw=; b=iKLINvyuX1eDkn+0Kdl0PgKGHTmBBy63QQIqyP5g0IV9XY0RjYBzigZI+HH7hZPqE1 br42jRv3nO11xGAH7cVElyOrH59Fchdy1kCDn2kzOm5FSrDVuJk+xnqqezwTbNmC5UTC /LceHXwVjpzqPMVmyas9+hC1NhpUd77Re/sYMie+OMJ3j1WMtG7PBjFm9AifMgUSoc72 kQQxEnau084F9uHMVMoRfNvkekMmMoILWHdY29R2Gugkz90K81UrhMiokDp19iFpwQ7l PhDV9XumHM0P9W3LIkp+aelbRu4lL7geLi8z8uIAgB1aA/V7MBDw8VC09/JV5uzg9day nJ4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=A6IO9Hb2rCfpf+glJ0pLcQzpWIUiMwqvq1rQ0jE7FUw=; b=kHzYSFKVR/3QpWQCfzJVtVOCJSFunRpWjUyjBU62dYSo29dj7jqmPo90e/1f8jHI4V AL1MMP37uZCqou9dlVAJSXv4o3gIiz3DztyLL5maBuaQIZ5PTxhMG3o6x+xKP/zms1Eh TLZ7lkM8n5jPWeJbWxp8QoRth1cjwb6vKv3aN1RLZbVeyuPyF/0pIe24HvE/+4Gn4Es1 /59VMlblE/cgtcWMBWxKYDIyi6hBLLLzgkdrl1AhIlL0dr0DspdQXQkE3uHS5WKaYTPc tMBu0UsQ6q/TMggFkTKAkk5kE1JIwLXf/NHFpT60C5dYgfC0kRcUy3rcc/AIeQ0tqnmq OMYg== X-Gm-Message-State: AOAM530S5pXWqARinyqToUuuRqg/fEPs04R9ELZ93x0rsFztwXD/ZbKv P1HxJhYy/lgudLd6OXyDMMIVCvF76QFo7lwj X-Google-Smtp-Source: ABdhPJww3QKZlow0q+YXLrrs2dvFf7hQg4Py8NY0XZP8uz7PfRY9JCoPbDke2+z42BbB3UzEyhOezg== X-Received: by 2002:a1c:6285:: with SMTP id w127mr7224973wmb.73.1613566265997; Wed, 17 Feb 2021 04:51:05 -0800 (PST) Received: from naush-laptop.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id r12sm2745281wmg.44.2021.02.17.04.51.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Feb 2021 04:51:01 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 17 Feb 2021 12:50:08 +0000 Message-Id: <20210217125008.1588781-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210217125008.1588781-1-naush@raspberrypi.com> References: <20210217125008.1588781-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 4/4] pipeline: raspberrypi: Allow either strict or non-strict buffer matching X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" A new flag, StrictBufferMatching, is used to control the behavior of the embedded and bayer buffer matching routine. If set to true (default), we reject and drop all bayer frames that do not have a matching embedded data buffer and vice-versa. This guarantees the IPA will always have the correct frame exposure and gain values to use. If set to false, we use bayer frames that do not have a matching embedded data buffer. In this case, IPA will use use the ControlList passed to it for gain and exposure values. Additionally, allow external stream buffers to behave as if StrictBufferMatching = false since we are not allowed to drop them. Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 31 ++++++++++++++++--- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index e76ac486855c..4ab47f8746bd 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -46,6 +46,22 @@ namespace libcamera { LOG_DEFINE_CATEGORY(RPI) +/* + * Set this to true to reject and drop all bayer frames that do not have a + * matching embedded data buffer and vice-versa. This guarantees the IPA will + * always have the correct frame exposure and gain values to use. + * + * Set this to false to use bayer frames that do not have a matching embedded + * data buffer. In this case, IPA will use use our local history for gain and + * exposure values, occasional frame drops may cause these number to be out of + * sync for a short period. + * + * \todo Ideally, this property should be set by the application, but we don't + * have any mechanism to pass generic properties into a pipeline handler at + * present. + */ +static const bool StrictBufferMatching = true; + namespace { bool isRaw(PixelFormat &pixFmt) @@ -1724,13 +1740,13 @@ bool RPiCameraData::findMatchingBuffers(BayerFrame &bayerFrame, FrameBuffer *&em embeddedBuffer = nullptr; while (!embeddedQueue_.empty()) { FrameBuffer *b = embeddedQueue_.front(); - if (!unicam_[Unicam::Embedded].isExternal() && b->metadata().timestamp < ts) { + if (b->metadata().timestamp < ts) { embeddedQueue_.pop(); unicam_[Unicam::Embedded].queueBuffer(b); embeddedRequeueCount++; LOG(RPI, Warning) << "Dropping unmatched input frame in stream " << unicam_[Unicam::Embedded].name(); - } else if (unicam_[Unicam::Embedded].isExternal() || b->metadata().timestamp == ts) { + } else if (b->metadata().timestamp == ts) { /* We pop the item from the queue lower down. */ embeddedBuffer = b; break; @@ -1744,10 +1760,15 @@ bool RPiCameraData::findMatchingBuffers(BayerFrame &bayerFrame, FrameBuffer *&em LOG(RPI, Debug) << "Could not find matching embedded buffer"; - if (!sensorMetadata_) { + if (unicam_[Unicam::Embedded].isExternal() || + !StrictBufferMatching || !sensorMetadata_) { /* - * If there is no sensor metadata, simply return the - * first bayer frame in the queue. + * If any of the follwing is true: + * - This is an external stream buffer (so cannot be dropped). + * - We do not care about strict buffer matching. + * - There is no sensor metadata present. + * we can simply return the first bayer frame in the queue + * without a matching embedded buffer. */ LOG(RPI, Debug) << "Returning bayer frame without a match"; bayerQueue_.pop();