From patchwork Tue Feb 16 10:31:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 11310 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 73D5DBD160 for ; Tue, 16 Feb 2021 10:32:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 34CFF637EE; Tue, 16 Feb 2021 11:32:17 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="fsB5dkMx"; dkim-atps=neutral Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 61613637E5 for ; Tue, 16 Feb 2021 11:32:14 +0100 (CET) Received: by mail-wm1-x32f.google.com with SMTP id o24so13903226wmh.5 for ; Tue, 16 Feb 2021 02:32:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LZl4tMiV9T5lpjTuzgxl4XgOZ5IV08IN5LTVWylkY4s=; b=fsB5dkMxsC/1iSwoCo7Nh3xL6vBSyKjF+0juCBHzP66KaBnCWIMWT5vIwXchbavf8w 31yGVzKOkQBBh7AcMnV08DBb2nJtZnC6uHIx63d1+gOpccxSCv42U/nM2wElC6bYkPJa QtrDtz6VsOfdbHgtwKoOtTFR1rlh2Lc1ec53CG0OJPrmniSy5Dqa0Fk1/kFQlg5MViFr OEfd4xZojXsdZIk9qUdli4iTWn7Oqte1IP9mBaUs/xRDOW/TmappRYU6NuiYhCD94c2z TtAZBukVGXDoj1C/kx9EOxzQrCOaRu11Exlpw5MDlx+nT+RwpKbipemE/PBwXMghSYTd aYDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LZl4tMiV9T5lpjTuzgxl4XgOZ5IV08IN5LTVWylkY4s=; b=IgnjHsfCj+5Fvw+0RE8OTYIkvqlinfzg/TFT/uBlYMdthNvPCk4zgSj07kJsed0N+V 7oU3TN4hPeW+MmlHvh0Eu2rlrFqPZjSq0DCLgqS8nyhyh3X8LQ6QIgE8etW0HjyD3REb 0TGKFZdtfXPqiFlBCr/fPZ6H71XPtqGJ4i8oOhzE/Da0VxmvtkfpKzZSOf3JRd2L2l3t E4AaFf9wLZ6cLxFZ2lxc0GZSRWieZFGLHDl/6HsyUjnOmzaUmXVfvoD/gWzaxv6DR5sI uSY1f+487ravS3nW6rmf8fC3w2QvbR/2pe/mHzHbpeSgZucj4lm9C89e+suKiXz4PU5A dh0Q== X-Gm-Message-State: AOAM530NfWxPTvEWMiB723HtKggbgRvTRjZLGihxeFXPDsUQdfGBMjOy f2TuVP7jPoCxLpFzbdibA9dQxPSU90BnA7NT X-Google-Smtp-Source: ABdhPJxGat7M0IOdW7May/GEifsMSZnkvfnsmciMZ447+Sz6R3peWQQQKnh0PDon/sDm4Qb+Bgjk5g== X-Received: by 2002:a05:600c:4656:: with SMTP id n22mr2755579wmo.130.1613471533874; Tue, 16 Feb 2021 02:32:13 -0800 (PST) Received: from naush-laptop.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id a186sm3018054wme.17.2021.02.16.02.32.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Feb 2021 02:32:13 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 16 Feb 2021 10:31:40 +0000 Message-Id: <20210216103140.1077307-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210216103140.1077307-1-naush@raspberrypi.com> References: <20210216103140.1077307-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 4/4] pipeline: raspberrypi: Allow either strict or non-strict buffer matching X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" A new flag, StrictBufferMatching, is used to control the behavior of the embedded and bayer buffer matching routine. If set to true (default), we reject and drop all bayer frames that do not have a matching embedded data buffer and vice-versa. This guarantees the IPA will always have the correct frame exposure and gain values to use. If set to false, we use bayer frames that do not have a matching embedded data buffer. In this case, IPA will use use the ControlList passed to it for gain and exposure values. Additionally, allow external stream buffers to behave as if StrictBufferMatching = false since we are not allowed to drop them. Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 31 ++++++++++++++++--- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index b6ee014b620e..b0bb12ab0f37 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -43,6 +43,22 @@ namespace libcamera { LOG_DEFINE_CATEGORY(RPI) +/* + * Set this to true to reject and drop all bayer frames that do not have a + * matching embedded data buffer and vice-versa. This guarantees the IPA will + * always have the correct frame exposure and gain values to use. + * + * Set this to false to use bayer frames that do not have a matching embedded + * data buffer. In this case, IPA will use use our local history for gain and + * exposure values, occasional frame drops may cause these number to be out of + * sync for a short period. + * + * \todo Ideally, this property should be set by the application, but we don't + * have any mechanism to pass generic properties into a pipeline handler at + * present. + */ +static const bool StrictBufferMatching = true; + namespace { bool isRaw(PixelFormat &pixFmt) @@ -1732,13 +1748,13 @@ bool RPiCameraData::findMatchingBuffers(BayerFrame &bayerFrame, FrameBuffer *&em embeddedBuffer = nullptr; while (!embeddedQueue_.empty()) { FrameBuffer *b = embeddedQueue_.front(); - if (!unicam_[Unicam::Embedded].isExternal() && b->metadata().timestamp < ts) { + if (b->metadata().timestamp < ts) { embeddedQueue_.pop(); unicam_[Unicam::Embedded].queueBuffer(b); embeddedRequeueCount++; LOG(RPI, Warning) << "Dropping unmatched input frame in stream " << unicam_[Unicam::Embedded].name(); - } else if (unicam_[Unicam::Embedded].isExternal() || b->metadata().timestamp == ts) { + } else if (b->metadata().timestamp == ts) { /* We pop the item from the queue lower down. */ embeddedBuffer = b; break; @@ -1752,10 +1768,15 @@ bool RPiCameraData::findMatchingBuffers(BayerFrame &bayerFrame, FrameBuffer *&em LOG(RPI, Debug) << "Could not find matching embedded buffer"; - if (!sensorMetadata_) { + if (unicam_[Unicam::Embedded].isExternal() || + !StrictBufferMatching || !sensorMetadata_) { /* - * If there is no sensor metadata, simply return the - * first bayer frame in the queue. + * If any of the follwing is true: + * - This is an external stream buffer (so cannot be dropped). + * - We do not care about strict buffer matching. + * - There is no sensor metadata present. + * we can simply return the first bayer frame in the queue + * without a matching embedded buffer. */ LOG(RPI, Debug) << "Returning bayer frame without a match"; bayerQueue_.pop();