From patchwork Tue Mar 2 15:11:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 11465 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CF03DBD1F1 for ; Tue, 2 Mar 2021 15:11:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 98C9B68A9A; Tue, 2 Mar 2021 16:11:29 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Q1H2WZdy"; dkim-atps=neutral Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4D61668AA0 for ; Tue, 2 Mar 2021 16:11:28 +0100 (CET) Received: by mail-wm1-x335.google.com with SMTP id m1so3107227wml.2 for ; Tue, 02 Mar 2021 07:11:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VM2nJZYIxUbh0H6j3sg6aGAGAOyrxfuBY7kCfr0KnlE=; b=Q1H2WZdyUvSQCowTXEGmEeUSbHbcSU/QTewABRTdr7P97RHTlIh6r9JA9x01IN5aRd 2S7QU2Xhyv7UX+Eaijj8ZiqQFwGzwFNCt1eDu5f3+sfx8QJotMyqe0URmq+UYou6g+AA bOSSmsoqSUHtIUZTuMehM407609J3yLDyjJMFTnpUn2/9RWpVYSSnpbHXEZcj9RIgVcd nXKbKGofVNASKBp0c3U5T4Cd9pAExnHf5xZCdze9tAdC7CVxqV0MPpS+jMwLGH6mxDu9 T/RM5tCDmSPjSOHwFOj4OqlZ/uqJScdeRdbZzellMnX1d1VhCYHIjGntyfQrUjqPaFhJ wucg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VM2nJZYIxUbh0H6j3sg6aGAGAOyrxfuBY7kCfr0KnlE=; b=ommN8YpG92iuF79cSW2i/Zh9wYdR7guOiXf23ZkgvTwV6PB0TF1AZRycmIIUqYHaq9 j8s6bvsTF4i87tKMlCuH6EeTGDrU4UfduDjaCyb6LBBN+g12X1CmLCI6zFm3FrjtYALO bLe/9gjVV7WdnNIwVAfB9oXqVCGN/ZLfEz2dKYrniMUuXwUFB9fWLXe9yY8UWaXfPMEn HFdBWWp7JhzeSoO3c46slh65aNuYbnAJNADl4Jfdqk6C5r34OdNYP88PmsNod6DTJmn2 S3+Qhl/JhYjlyI7WL7F67JsB++eauIchhmFUWvAmGMsXHZLb13e4woKvHrxCbCOqNVKG 7csA== X-Gm-Message-State: AOAM531CzZounBHyQXcCX1fONIRHkOn8AZ/mmIu2luHGaEMAusVWeRTA UbD/zjZ+BV97W1dwusmqtnzNWgv4NEbRJdZs X-Google-Smtp-Source: ABdhPJzlwBJENnZAvxGNQ730GdwxXZaO5gNRxSJr7gLJEnsbYLSGM/1OLcln+RqUVGzAGuNjpmeHew== X-Received: by 2002:a05:600c:2dda:: with SMTP id e26mr4574260wmh.155.1614697887695; Tue, 02 Mar 2021 07:11:27 -0800 (PST) Received: from naush-laptop.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id w6sm7561062wrl.49.2021.03.02.07.11.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 07:11:26 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Mar 2021 15:11:11 +0000 Message-Id: <20210302151111.212591-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210302151111.212591-1-naush@raspberrypi.com> References: <20210302151111.212591-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 4/4] pipeline: raspberrypi: Allow either strict or non-strict buffer matching X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" A new flag, StrictBufferMatching, is used to control the behavior of the embedded and bayer buffer matching routine. If set to true (default), we reject and drop all bayer frames that do not have a matching embedded data buffer and vice-versa. This guarantees the IPA will always have the correct frame exposure and gain values to use. If set to false, we use bayer frames that do not have a matching embedded data buffer. In this case, IPA will use use the ControlList passed to it for gain and exposure values. Additionally, allow external stream buffers to behave as if StrictBufferMatching = false since we are not allowed to drop them. Signed-off-by: Naushir Patuck Tested-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 31 ++++++++++++++++--- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 5ae2551957f8..549d6147cd60 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -46,6 +46,22 @@ namespace libcamera { LOG_DEFINE_CATEGORY(RPI) +/* + * Set this to true to reject and drop all bayer frames that do not have a + * matching embedded data buffer and vice-versa. This guarantees the IPA will + * always have the correct frame exposure and gain values to use. + * + * Set this to false to use bayer frames that do not have a matching embedded + * data buffer. In this case, IPA will use use our local history for gain and + * exposure values, occasional frame drops may cause these number to be out of + * sync for a short period. + * + * \todo Ideally, this property should be set by the application, but we don't + * have any mechanism to pass generic properties into a pipeline handler at + * present. + */ +static const bool StrictBufferMatching = true; + namespace { bool isRaw(PixelFormat &pixFmt) @@ -1725,13 +1741,13 @@ bool RPiCameraData::findMatchingBuffers(BayerFrame &bayerFrame, FrameBuffer *&em embeddedBuffer = nullptr; while (!embeddedQueue_.empty()) { FrameBuffer *b = embeddedQueue_.front(); - if (!unicam_[Unicam::Embedded].isExternal() && b->metadata().timestamp < ts) { + if (b->metadata().timestamp < ts) { embeddedQueue_.pop(); unicam_[Unicam::Embedded].queueBuffer(b); embeddedRequeueCount++; LOG(RPI, Warning) << "Dropping unmatched input frame in stream " << unicam_[Unicam::Embedded].name(); - } else if (unicam_[Unicam::Embedded].isExternal() || b->metadata().timestamp == ts) { + } else if (b->metadata().timestamp == ts) { /* We pop the item from the queue lower down. */ embeddedBuffer = b; break; @@ -1745,10 +1761,15 @@ bool RPiCameraData::findMatchingBuffers(BayerFrame &bayerFrame, FrameBuffer *&em LOG(RPI, Debug) << "Could not find matching embedded buffer"; - if (!sensorMetadata_) { + if (unicam_[Unicam::Embedded].isExternal() || + !StrictBufferMatching || !sensorMetadata_) { /* - * If there is no sensor metadata, simply return the - * first bayer frame in the queue. + * If any of the following is true: + * - This is an external stream buffer (so cannot be dropped). + * - We do not care about strict buffer matching. + * - There is no sensor metadata present. + * we can simply return the first bayer frame in the queue + * without a matching embedded buffer. */ LOG(RPI, Debug) << "Returning bayer frame without a match"; bayerFrame = std::move(bayerQueue_.front());