From patchwork Mon Apr 19 13:34:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 12002 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 92F54BD814 for ; Mon, 19 Apr 2021 13:35:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 19E2B68839; Mon, 19 Apr 2021 15:35:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="YYHhqV32"; dkim-atps=neutral Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9F901602CD for ; Mon, 19 Apr 2021 15:34:58 +0200 (CEST) Received: by mail-wr1-x435.google.com with SMTP id h4so25037318wrt.12 for ; Mon, 19 Apr 2021 06:34:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Jo8WkXMl8c5cRR6Ta7ynE+2nvwPxIi03CyEKeDa4jWM=; b=YYHhqV32uRwxKd6bnghJbhyAvakXbTqkj2xOGrR0z42bwEtAzmXnhpG023pRX2/yk3 C8eV6rNkC7ml0DzbP9wBkX5VqF6Y7yB0vUFvbQnpW2vgdjz2awlS9NSaHyRTAyXqNvbu +sRU5YnfzP+NOL1IewXiEjP0Kyg2QmIlHazkkjg4iMovWoJ1Q2W7gYR3YOZty5y61Glb VBbAWUfsfUD7hvop9KyX/g8htZCEXBBm4+1uLwgGikrQPYFq3OnpASZdlX3wYzbSHbDF CXQW0sSgpoWdcIGhkRMNmd6WvLAzYi31fVF9ygcD9EXadk4snvTWBd6odYpanqKHkOWn NvXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Jo8WkXMl8c5cRR6Ta7ynE+2nvwPxIi03CyEKeDa4jWM=; b=RWFywN+ZVgnIM55QURj081G1u4Ir/wLMCQ0VWK6/bQbp8U2zS+rJ1XnUauob6ElJUz imC6QMAlj68roaR/UpbByuFoRjF3OspClKU38yI1hxvviA2MICAgm0zcCB/FJuwFtMuA zHgG4WkcEq2BEciBsYV294ImlxSfGEfO9S4cfHHIPTvt2ewC7varhzcb2snNHCiuyg9u YHedUfYKafXjRsf0tvP1CqOcI18crDTBWJchB3wQIyHGik0lti4vnzdnhCksNKy/JMHw dMjKJ7GVXwVivtynPHl3J17P6ejg0TD8m+9vQ0uWgbsIS95ez+jExazs5fRd5KnuMjKq PTFQ== X-Gm-Message-State: AOAM5309gya0JUoImZl5tNbuiCMPZA5xikdUUoS78HhR3yrWMucZeywB Gx7FCxBCRO6DPcQ1oy6+PuF1vb+s5T5ZvA== X-Google-Smtp-Source: ABdhPJxEou7YCBnt8iOCjokRxtLZbeD74oOYpqTDNyrew/ClzhuMh8xRnglin43cWRXsz7kdnHOzGA== X-Received: by 2002:adf:ea84:: with SMTP id s4mr14656396wrm.142.1618839298181; Mon, 19 Apr 2021 06:34:58 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:15:5363:8f56:4f84:754]) by smtp.gmail.com with ESMTPSA id k11sm19952238wrx.88.2021.04.19.06.34.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Apr 2021 06:34:57 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 19 Apr 2021 14:34:47 +0100 Message-Id: <20210419133451.263733-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210419133451.263733-1-naush@raspberrypi.com> References: <20210419133451.263733-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 1/5] ipa: raspberrypi: Store dropped frame count in a member variable X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Store the number of dropped frames on startup in a member variable. This will be used in a subsequent change for rate limiting the controller algorithms. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/raspberrypi.cpp | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index b0f61d351e3a..f6d1ab16a290 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -143,6 +143,9 @@ private: /* How many frames we should avoid running control algos on. */ unsigned int mistrustCount_; + /* Number of frames that need to be dropped on startup. */ + unsigned int dropFrameCount_; + /* LS table allocation passed in from the pipeline handler. */ FileDescriptor lsTableHandle_; void *lsTable_; @@ -220,9 +223,8 @@ void IPARPi::start(const ControlList &controls, ipa::RPi::StartConfig *startConf */ frameCount_ = 0; checkCount_ = 0; - unsigned int dropFrame = 0; if (firstStart_) { - dropFrame = helper_->HideFramesStartup(); + dropFrameCount_ = helper_->HideFramesStartup(); mistrustCount_ = helper_->MistrustFramesStartup(); /* @@ -250,14 +252,14 @@ void IPARPi::start(const ControlList &controls, ipa::RPi::StartConfig *startConf awbConvergenceFrames += mistrustCount_; } - dropFrame = std::max({ dropFrame, agcConvergenceFrames, awbConvergenceFrames }); - LOG(IPARPI, Debug) << "Drop " << dropFrame << " frames on startup"; + dropFrameCount_ = std::max({ dropFrameCount_, agcConvergenceFrames, awbConvergenceFrames }); + LOG(IPARPI, Debug) << "Drop " << dropFrameCount_ << " frames on startup"; } else { - dropFrame = helper_->HideFramesModeSwitch(); + dropFrameCount_ = helper_->HideFramesModeSwitch(); mistrustCount_ = helper_->MistrustFramesModeSwitch(); } - startConfig->dropFrameCount = dropFrame; + startConfig->dropFrameCount = dropFrameCount_; firstStart_ = false; } From patchwork Mon Apr 19 13:34:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 12003 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E07B1BD814 for ; Mon, 19 Apr 2021 13:35:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 30DC26883A; Mon, 19 Apr 2021 15:35:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="HTpYjrXy"; dkim-atps=neutral Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AB3C768839 for ; Mon, 19 Apr 2021 15:34:59 +0200 (CEST) Received: by mail-wr1-x434.google.com with SMTP id x7so34048542wrw.10 for ; Mon, 19 Apr 2021 06:34:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fW/hur3tefpJwBCdBS9ucikdtiTjIrz8a6UkMMoW9n0=; b=HTpYjrXyhOkDbT+U6Ji+AedTG+1SM/axFXnKyJdP/r6CiTmV4Lq7hVct9ymv0fRm7f 6zBWk57jyTmPgUQIfLDLPAVUp8qvq86LQyW7FN9lHSwBUkPM2kaD2THo4vyUfADLxi7D XY7RSmJNyjzqjlpkgPj4kPVOfLHGqFacYrWKINg/7MeQGDSexlN2o630Axjn2QAnbUgk Xg33RxSHlGs+vV7GujtQ51eXyzmoDNY58jwF0WPou9F2KF/YM+liff4cD+Nofk3CYEJ0 m1HrBQrPdWzw+G5mEbbq5C4JSw+2nBICnBCtgENY6sneUG+uazBxqq07cV0++Ocimx2m ZxUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fW/hur3tefpJwBCdBS9ucikdtiTjIrz8a6UkMMoW9n0=; b=J/YiAC81wE5AdgbcSTVjkPjEl8Q9ZwwmOOIYoTlX+4kgzOdhhcwJ6T8yhlw92Zapzx 9eVl8sQE+ZtiPQO76mK3LtSXepxYh1BHQFGZX2mH6IOGDXjEUjxiiyBptzNK1OcghCRT BXfQ/qTYEve96iekWmjlYBtlezC8ceH0puQ1WtLTEDBSaF2R0ZNIbb5wnpE34zVQKHOn dpJxpvsbPrG6RVhsD5J6EHyjCWE6923CND05FdtpuYygIpMeM6SZrbtwlJwprYh+0/iw yBb2T4UqtOzGVAZUHfjtagbOT8r7Rt/KxHsIvPKm6JJrb12IGJoQ6PltvlceBYv2h+HX Yorw== X-Gm-Message-State: AOAM533eRVXnFOAUnTZcoqP6wpU8yG2yW2jndsqEgLtgpEpji6EDcemK awbsqrA18KJO7SRwdkft0b3Uzdwrg2QPnA== X-Google-Smtp-Source: ABdhPJwfiE/OqWMwEZwwOCxP722thno8kdHcAqn3gr8RrxsrlnOA/Kt8nV+7AvecJk9EmusgvwTBdQ== X-Received: by 2002:a05:6000:362:: with SMTP id f2mr14730584wrf.141.1618839299060; Mon, 19 Apr 2021 06:34:59 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:15:5363:8f56:4f84:754]) by smtp.gmail.com with ESMTPSA id k11sm19952238wrx.88.2021.04.19.06.34.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Apr 2021 06:34:58 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 19 Apr 2021 14:34:48 +0100 Message-Id: <20210419133451.263733-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210419133451.263733-1-naush@raspberrypi.com> References: <20210419133451.263733-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 2/5] ipa: raspberrypi: Reformat RPiController::Metadata class header X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rearrange header includes to be in alphabetical order. Add whitespace to class definition to match libcamera coding guidelines. Fix a typo in the comment showing an example of scoped locks. There are no functional changes in this commit. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/metadata.hpp | 23 ++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/src/ipa/raspberrypi/controller/metadata.hpp b/src/ipa/raspberrypi/controller/metadata.hpp index 4f44ffc6771c..07dd28ed9e0a 100644 --- a/src/ipa/raspberrypi/controller/metadata.hpp +++ b/src/ipa/raspberrypi/controller/metadata.hpp @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: BSD-2-Clause */ /* - * Copyright (C) 2019, Raspberry Pi (Trading) Limited + * Copyright (C) 2019-2021, Raspberry Pi (Trading) Limited * * metadata.hpp - general metadata class */ @@ -9,22 +9,25 @@ // A simple class for carrying arbitrary metadata, for example about an image. #include -#include -#include #include #include +#include +#include namespace RPiController { class Metadata { public: - template void Set(std::string const &tag, T const &value) + template + void Set(std::string const &tag, T const &value) { std::lock_guard lock(mutex_); data_[tag] = value; } - template int Get(std::string const &tag, T &value) const + + template + int Get(std::string const &tag, T &value) const { std::lock_guard lock(mutex_); auto it = data_.find(tag); @@ -33,11 +36,13 @@ public: value = std::any_cast(it->second); return 0; } + void Clear() { std::lock_guard lock(mutex_); data_.clear(); } + Metadata &operator=(Metadata const &other) { std::lock_guard lock(mutex_); @@ -45,7 +50,9 @@ public: data_ = other.data_; return *this; } - template T *GetLocked(std::string const &tag) + + template + T *GetLocked(std::string const &tag) { // This allows in-place access to the Metadata contents, // for which you should be holding the lock. @@ -54,15 +61,17 @@ public: return nullptr; return std::any_cast(&it->second); } + template void SetLocked(std::string const &tag, T const &value) { // Use this only if you're holding the lock yourself. data_[tag] = value; } + // Note: use of (lowercase) lock and unlock means you can create scoped // locks with the standard lock classes. - // e.g. std::lock_guard lock(metadata) + // e.g. std::lock_guard lock(metadata) void lock() { mutex_.lock(); } void unlock() { mutex_.unlock(); } From patchwork Mon Apr 19 13:34:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 12004 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1A494BD814 for ; Mon, 19 Apr 2021 13:35:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0D38968838; Mon, 19 Apr 2021 15:35:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="twneBOGw"; dkim-atps=neutral Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 605D568831 for ; Mon, 19 Apr 2021 15:35:00 +0200 (CEST) Received: by mail-wr1-x430.google.com with SMTP id e7so25105049wrs.11 for ; Mon, 19 Apr 2021 06:35:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uule6aP/GbUVKt6rRXc2oaEhZT9hc7570Iq0GDDxCU0=; b=twneBOGwzAW6ATxXHmLh5kVFuslumGQSPeQiZNUZUlAlluH3tIO80UXF+YXtKp1eBc yCat/cic8oUgc+Vih1CLVAFoxVAbLz4PCe8lLx7Azu16rp/7qVeAcj2KFnRPtK1+eAHv exvjLCnTG2Qv8DfYavdhKVhKOmhZzEZZJImFJR1tcEw980mEU+JnFOB2rN9wGbL2fXZY a3elDZ6aYw7js55k/9bimiulsD6VXNOeAfjCndMSSHjgehCTkUtZF+k/z0QR0iPOe5+6 EWUBCsBPPvZQM0XfKApMcEAxcXvYEU0zRQMJYlk2KBaU18mBIPs32ZFGAZyVdPSiWS6b 0Law== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uule6aP/GbUVKt6rRXc2oaEhZT9hc7570Iq0GDDxCU0=; b=DINRsNuKvbduT0uGvf+aQ1w23290fVkaJwW5CIWiVdQoGAJgK8kFh9Kua/GGuUZWsa JlmdzSlQpdWs1kZZLYa4Inx1lh3j5FR6chEOd1pIjqsNdU9dwwESwqRt+KY0JsoWWQDE kNNe8BhxicLzA1BVEtTgIc1DoexLkfUGbgsPDxML2jolpSQISss6hM/ugJ47xxxAPejo Y7AH1WtvaOFRQybX2svvWl3QTsg75u7xIncYeNvLVQX1WB+ZZAsSHyRAgeXUX65yANZm ZLPQZn3eBNOWdJF66LFjYCssg1aLsYLMUfBrjEpN1YgantLg5vUosAvpl2nxi8TUZuwZ dUig== X-Gm-Message-State: AOAM532rG6/a4q8e+ZiK6nAdVnz/yMPZVspuL/CQTuYPLq+469PK2d4q LBVybXxLZWpAxXfX28ogSgHUkt/FrMXDOg== X-Google-Smtp-Source: ABdhPJx1NoO+ncIMBrSHFR8zMHfq2NsCVthKEM7NodhWlvbE1Y4uFg4lV5t0IncBzm1MEiPva3EJZA== X-Received: by 2002:adf:c587:: with SMTP id m7mr14286722wrg.369.1618839299867; Mon, 19 Apr 2021 06:34:59 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:15:5363:8f56:4f84:754]) by smtp.gmail.com with ESMTPSA id k11sm19952238wrx.88.2021.04.19.06.34.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Apr 2021 06:34:59 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 19 Apr 2021 14:34:49 +0100 Message-Id: <20210419133451.263733-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210419133451.263733-1-naush@raspberrypi.com> References: <20210419133451.263733-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 3/5] ipa: raspberrypi: Add move/copy ctors and operators to Metadata class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a default, move and copy constructor as well as a move operator implementation RPiController::Metadata class. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/metadata.hpp | 24 +++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/src/ipa/raspberrypi/controller/metadata.hpp b/src/ipa/raspberrypi/controller/metadata.hpp index 07dd28ed9e0a..319f2320fc70 100644 --- a/src/ipa/raspberrypi/controller/metadata.hpp +++ b/src/ipa/raspberrypi/controller/metadata.hpp @@ -19,6 +19,21 @@ namespace RPiController { class Metadata { public: + Metadata() = default; + + Metadata(Metadata const &other) + { + std::lock_guard other_lock(other.mutex_); + data_ = other.data_; + } + + Metadata(Metadata &&other) + { + std::lock_guard other_lock(other.mutex_); + data_ = std::move(other.data_); + other.data_.clear(); + } + template void Set(std::string const &tag, T const &value) { @@ -51,6 +66,15 @@ public: return *this; } + Metadata &operator=(Metadata &&other) + { + std::lock_guard lock(mutex_); + std::lock_guard other_lock(other.mutex_); + data_ = std::move(other.data_); + other.data_.clear(); + return *this; + } + template T *GetLocked(std::string const &tag) { From patchwork Mon Apr 19 13:34:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 12005 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D6399BD814 for ; Mon, 19 Apr 2021 13:35:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 930AB68847; Mon, 19 Apr 2021 15:35:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="WTpkoMh2"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 45C1668839 for ; Mon, 19 Apr 2021 15:35:01 +0200 (CEST) Received: by mail-wr1-x429.google.com with SMTP id j5so33117808wrn.4 for ; Mon, 19 Apr 2021 06:35:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ByDL9nYy7x4L3Tto/bzy+16JVkmrBlZp1522yVKSnPo=; b=WTpkoMh2H19oXim9dqh8Y0Z2K2s5qNXE9JeGa6+0v0BbaTdYJoqELK5cUX2qDWoRQo YrYWG5HEJ+5sYLL1xNVuNuguteHuQ9UKiHJacumk0sg3OJTrgAoixdxwr2x8n1G5LFaU XMBmhy88EolMarYA6RyFaynSX9QlFAopvdhjtzjZvc17keJinPGxGsr0CB/g94KXynSr pJ/8m6+5TbgZ/6UAhGD39A6pwzHazxQsil8ZxqpJ+chqz2ORkxpkPhKAStsFeUoy6gb5 BddvPH3dWlqtYDknKyDoDrrzzvXr4cHRvSdgghbxT37+1Oc3kbrsBGo77GqMjDN34s+A ZPqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ByDL9nYy7x4L3Tto/bzy+16JVkmrBlZp1522yVKSnPo=; b=CZD5Eq95sdX1fehXf0fpWc2QpVDPTbG0cqJSz7gx5jx7dVpU70UiDyxmFQ5yazK6ja WGe+G6+8o146Ebck8ZPvxpcm4WW+EanPP/AGJbY4N8gjG9DamlXFdQd+rl3FhkRtDlg/ e2A8oJw3E2Fq5PZNHSjzciuCHjQRYng77Bte+n06iZFjUV+4f/bXUVhHqD/7NXYrulxG cEIykdwAEFGEu4HOgpKQXUslhpZ1XZbtt3hqHBjf7j/8aFzCV4MrnFxqfvWQyuYp3Elu lYFYN6dLc3oANO+JBVCtXgnuXo/mseNlXQzVpM8u0Evv590Ml+EtUhKAfY+ReX3jtGss k64w== X-Gm-Message-State: AOAM533oDdL05wscJp4Iph1SpkYgaBP2jdGOi56mKC2iqN86T6Qn0NXU oeFfcnASWDJDsQ1V0ttDnCEtoN9FfU5zrw== X-Google-Smtp-Source: ABdhPJzLg57UhnlUg6WTznD5fuvOjOr1Ej5SAtG1we4br5V2HjHr4SLXZSwQmQkbDkEQTU4MRbB2hA== X-Received: by 2002:a5d:6d0c:: with SMTP id e12mr14206687wrq.321.1618839300796; Mon, 19 Apr 2021 06:35:00 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:15:5363:8f56:4f84:754]) by smtp.gmail.com with ESMTPSA id k11sm19952238wrx.88.2021.04.19.06.34.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Apr 2021 06:35:00 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 19 Apr 2021 14:34:50 +0100 Message-Id: <20210419133451.263733-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210419133451.263733-1-naush@raspberrypi.com> References: <20210419133451.263733-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 4/5] ipa: raspberrypi: Add Merge method to RPiController::Metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new Merge method to the Metadata class. This will move all key/value pairs between a source and destination metadata object. Once complete, the source Metadata object will be empty. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/metadata.hpp | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/ipa/raspberrypi/controller/metadata.hpp b/src/ipa/raspberrypi/controller/metadata.hpp index 319f2320fc70..1d3e941b3e52 100644 --- a/src/ipa/raspberrypi/controller/metadata.hpp +++ b/src/ipa/raspberrypi/controller/metadata.hpp @@ -75,6 +75,18 @@ public: return *this; } + void Merge(Metadata &other) + { + std::lock_guard lock(mutex_); + std::lock_guard other_lock(other.mutex_); + + for (auto const &kv: other.data_) + data_[kv.first] = std::move(kv.second); + + /* Render the other object as empty now! */ + other.data_.clear(); + } + template T *GetLocked(std::string const &tag) { From patchwork Mon Apr 19 13:34:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 12006 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3CCCBBD814 for ; Mon, 19 Apr 2021 13:35:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F24E26883B; Mon, 19 Apr 2021 15:35:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Q9Yzyija"; dkim-atps=neutral Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7191F68842 for ; Mon, 19 Apr 2021 15:35:02 +0200 (CEST) Received: by mail-wm1-x32a.google.com with SMTP id z6so415489wmg.1 for ; Mon, 19 Apr 2021 06:35:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uhakufSYimFzXE2ktJnS7KEDy6j7582V8I7AiynSico=; b=Q9YzyijafwMXJVJ2LcS+e+WZdUQJsiMFVxFTz/gOsaXZHAp8OzHnsXSEjEd+5dBV8I JLXdmf0cHkgqgQnmJcqcJKGyyhM6wlCIorbcDqNFpoNF0nGjxzhwRDtTrx98wlarGh8f VlnScJKeffYs2maLXp64eD9J0tfwPFSu83RxVFhr7uIxt9rwPdSU0Y1BzAe/BJPK1w/s kXblqjIdQDinnMQqbzVAdnrCpoAulyFJHi3cBehRiPBoeuNI8ZfNP/B1rkx493dipuPe 0d5pbWFpG5R9Qqi2DOmlrPYoD4Wl0ImpqwXtpY1gxuWg3h+zKNbUdR0W/Ip42QXY0b1H oksA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uhakufSYimFzXE2ktJnS7KEDy6j7582V8I7AiynSico=; b=a7oQs/SBPIv2KCvKjXIEtBb4syHaniL3tntCQCC6BofDJN2snSaPcwb5v992GqZQ/2 NHomzgcfN1JR8P9gw0xJcv1nO50T5Lr+uChxaoQXEyzVHgIrJdVMmGYAN6TPgbv8sxBf r7/GADKmmX3OcvTO4FnsKeWZUSMc9hwdsciHKqS1vUW/sTc8fuC3LeF4qwUfq6OJJ8y/ fDHQPJdidDorLO0Q9qtpwGoKb9iTN51PJzlgk6+8TMo9mp2DZ6B0Ms+q2rxZWOPIq3R1 Lx7pQ2mv4CETloenmCmULIaI1lhUzSjvS3xBMC8OYRmd7M8uFHHqloPfJ4fDwzRhXSH+ 2obw== X-Gm-Message-State: AOAM533ErFxrkljkpxrqwvdwB/0VBHBRavUNY5EFnF3Uhb7pXDci7mzN rTKilsMwdgAjl11I/90Vsm66wkCMfKft8g== X-Google-Smtp-Source: ABdhPJxvh54U3MHLPDAZPBOqGysq7cg1RtQQBcmMHFbtIZwq6rKtWUeIdLVoFLbw+AZdFYwgjM7lWw== X-Received: by 2002:a1c:9a16:: with SMTP id c22mr21818840wme.7.1618839301729; Mon, 19 Apr 2021 06:35:01 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:15:5363:8f56:4f84:754]) by smtp.gmail.com with ESMTPSA id k11sm19952238wrx.88.2021.04.19.06.35.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Apr 2021 06:35:01 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 19 Apr 2021 14:34:51 +0100 Message-Id: <20210419133451.263733-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210419133451.263733-1-naush@raspberrypi.com> References: <20210419133451.263733-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 5/5] ipa: raspberrypi: Rate-limit the controller algorithms X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The controller algorithms currently run on every frame provided to the IPA by the pipeline handler. This may be undesirable for very fast fps operating modes where it could significantly increase the computation cycles (per unit time) without providing any significant changes to the IQ parameters. The added latencies could also cause dropped frames. Pass the FrameBuffer timestamp to the IPA through the controls. This timestamp will be used to rate-limit the controller algorithms to run with a minimum inter-frame time given by a compile time constant, currently set to 16.66ms. On startup, we don't rate-limit the algorithms until after the number of frames required for convergence. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/raspberrypi.cpp | 50 +++++++++++++++++-- .../pipeline/raspberrypi/raspberrypi.cpp | 5 ++ 2 files changed, 51 insertions(+), 4 deletions(-) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index f6d1ab16a290..eac1cf8cfe44 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -61,6 +61,14 @@ constexpr unsigned int DefaultExposureTime = 20000; constexpr double defaultMinFrameDuration = 1e6 / 30.0; constexpr double defaultMaxFrameDuration = 1e6 / 0.01; +/* + * Determine the minimum allowable inter-frame duration (in us) to run the + * controller algorithms. If the pipeline handler provider frames at a rate + * higher than this, we rate-limit the controller Prepare() and Process() calls + * to lower than or equal to this rate. + */ +constexpr double controllerMinFrameDuration = 1e6 / 60.0; + LOG_DEFINE_CATEGORY(IPARPI) class IPARPi : public ipa::RPi::IPARPiInterface @@ -68,7 +76,7 @@ class IPARPi : public ipa::RPi::IPARPiInterface public: IPARPi() : controller_(), frameCount_(0), checkCount_(0), mistrustCount_(0), - lsTable_(nullptr), firstStart_(true) + lastRunTimestamp_(0), lsTable_(nullptr), firstStart_(true) { } @@ -146,6 +154,12 @@ private: /* Number of frames that need to be dropped on startup. */ unsigned int dropFrameCount_; + /* Frame timestamp for the last run of the controller. */ + uint64_t lastRunTimestamp_; + + /* Do we run a Controller::process() for this frame? */ + bool processPending_; + /* LS table allocation passed in from the pipeline handler. */ FileDescriptor lsTableHandle_; void *lsTable_; @@ -262,6 +276,7 @@ void IPARPi::start(const ControlList &controls, ipa::RPi::StartConfig *startConf startConfig->dropFrameCount = dropFrameCount_; firstStart_ = false; + lastRunTimestamp_ = 0; } void IPARPi::setMode(const CameraSensorInfo &sensorInfo) @@ -406,7 +421,7 @@ void IPARPi::signalStatReady(uint32_t bufferId) { if (++checkCount_ != frameCount_) /* assert here? */ LOG(IPARPI, Error) << "WARNING: Prepare/Process mismatch!!!"; - if (frameCount_ > mistrustCount_) + if (processPending_ && frameCount_ > mistrustCount_) processStats(bufferId); reportMetadata(); @@ -894,10 +909,11 @@ void IPARPi::returnEmbeddedBuffer(unsigned int bufferId) void IPARPi::prepareISP(const ipa::RPi::ISPConfig &data) { + int64_t frameTimestamp = data.controls.get(controls::draft::SensorTimestamp); + RPiController::Metadata lastMetadata; Span embeddedBuffer; - rpiMetadata_.Clear(); - + lastMetadata = std::move(rpiMetadata_); fillDeviceStatus(data.controls); if (data.embeddedBufferPresent) { @@ -920,6 +936,32 @@ void IPARPi::prepareISP(const ipa::RPi::ISPConfig &data) if (data.embeddedBufferPresent) returnEmbeddedBuffer(data.embeddedBufferId); + if (lastRunTimestamp_ && frameCount_ > dropFrameCount_ && + frameTimestamp - lastRunTimestamp_ < controllerMinFrameDuration * 1e3) { + /* + * Ensure we update the controller metadata with the new frame's + * exposure/gain values so that the correct values are returned + * out in libcamera metadata later on. Any other bits of metadata + * that were added in helper_->Prepare() will also be moved across. + * All other metadata values must remain the same as the last frame. + * + * We do this by merging the current frame's metadata into the + * previous frame's metadata object, and then moving the latter + * into the former. + */ + lastMetadata.Merge(rpiMetadata_); + rpiMetadata_ = std::move(lastMetadata); + processPending_ = false; + LOG(IPARPI, Debug) << "Rate-limiting the controller! inter-frame duration: " + << frameTimestamp - lastRunTimestamp_ + << ", min duration: " + << controllerMinFrameDuration * 1e3; + return; + } + + lastRunTimestamp_ = frameTimestamp; + processPending_ = true; + ControlList ctrls(ispCtrls_); controller_.Prepare(&rpiMetadata_); diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 2a917455500f..9cf9c8c6cebd 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1414,6 +1414,11 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) * DelayedControl and queue them along with the frame buffer. */ ControlList ctrl = delayedCtrls_->get(buffer->metadata().sequence); + /* + * Add the frame timestamp to the ControlList for the IPA to use + * as it does not receive the FrameBuffer object. + */ + ctrl.set(controls::draft::SensorTimestamp, buffer->metadata().timestamp); bayerQueue_.push({ buffer, std::move(ctrl) }); } else { embeddedQueue_.push(buffer);