From patchwork Mon Dec 7 18:01:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10598 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7FFDFBDB1F for ; Mon, 7 Dec 2020 18:01:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 49CCA67E6A; Mon, 7 Dec 2020 19:01:57 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="mHrEhWOa"; dkim-atps=neutral Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 58FB0635A0 for ; Mon, 7 Dec 2020 19:01:55 +0100 (CET) Received: by mail-ej1-x632.google.com with SMTP id d17so20761022ejy.9 for ; Mon, 07 Dec 2020 10:01:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DH+SHOeKlT6c2ZRs8mi0dfzgwQzv992L13kp55/VB20=; b=mHrEhWOaW/D6Y6X/RnSZ3ESMMQNgUKNQGYNso0a71bebfmZkhU9YSbexLn+JzpAgMm zCGdCAPBqfEMCnzcH6Iy+mVHPQ4Nzw44vh2EZD7Nq8xaUkntKQhJ+FvvTq0/qNkKk3RS QaTGmoe5l051ZdBw9w5Ax9HSsJ3zyc7h6huNE4rUJJTY/VI5x7iWrmv8o6GoWKaW4O26 MgY6eBCW1+EpkgoEc12h2IBpYUZzspY9iluP6P/ZCMGG0zKNn67JJHUURL71lP6iXFkC RjPG8UQueZijX7JVx908zxKuXpSiD1YJfANEL5iY+EhmF/48uwXgpEeh7HxORTBXeiks z/fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DH+SHOeKlT6c2ZRs8mi0dfzgwQzv992L13kp55/VB20=; b=o71K4eQ7v1GZvWDjZKmwM7Rw/asuRl1/dMv9BQW0s2UM6CkEWV+7nrClWHGBuOfLAw XGAb2u/S1LE4d1X9UnwzDarvVeIvmSCZRH2TECe5bN8TtLMwYzMVrwQqf92xBbm96/Id ZNsQe2R2HXYM+2cQJfrJ8O1QD13jLAXitJzoaurHbgH3vozW5Kxc4yAPIPhdvyac0xoh VyOMZTPwWywukfNBKu5PMkWbIFxV1Y/JhmwnOCCwR6kKKCmQpek8fH2ZcDlVZodmFFKY dxo1IVqiqPji3N3tDF/X6EvCGHzhp8UGBykXPZ2JIxtfeKDw8ey4WcSxqXgQSJ18DL7K HOKg== X-Gm-Message-State: AOAM531kNCOvDjmT5SAeDU3q2c9r5/argThk1FeC8fyzGQh8zu4fvkWP 0UMeaOnmjrF8upxrTKMh9hCW0jAXQ8/AsIJw X-Google-Smtp-Source: ABdhPJxC/iSWV9H3llOx0xgCAxWEtaZ1jb9VHfY6JmSMP5BfaFahj6IARYSn80HlDZWvxh1JMG9O6Q== X-Received: by 2002:a17:906:6091:: with SMTP id t17mr20418739ejj.476.1607364114686; Mon, 07 Dec 2020 10:01:54 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id be6sm14657977edb.29.2020.12.07.10.01.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Dec 2020 10:01:49 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 7 Dec 2020 18:01:16 +0000 Message-Id: <20201207180121.6374-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201207180121.6374-1-david.plowman@raspberrypi.com> References: <20201207180121.6374-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/6] src: raspberrypi: Pass the drop frame count in start, not configure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The number of frames to drop (not display) is passed back now from the start method, not configure. This means applications have a chance to set fixed exposure/gain before starting the camera and this can affect the frame drop count that is returned. Note how we need to be able to tell the very first time we start the camera from subsequent restarts, hence addition of the "firstStart_" flag. Both the IPA implementation file and the pipeline handler need matching modifications. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/raspberrypi.cpp | 45 ++++++++++--------- .../pipeline/raspberrypi/raspberrypi.cpp | 25 ++++++----- 2 files changed, 38 insertions(+), 32 deletions(-) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 2027f1c0..0e89af00 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -67,7 +67,7 @@ public: IPARPi() : lastMode_({}), controller_(), controllerInit_(false), frameCount_(0), checkCount_(0), mistrustCount_(0), - lsTable_(nullptr) + lsTable_(nullptr), firstStart_(true) { } @@ -145,6 +145,9 @@ private: /* LS table allocation passed in from the pipeline handler. */ FileDescriptor lsTableHandle_; void *lsTable_; + + /* Distinguish the first camera start from others. */ + bool firstStart_; }; int IPARPi::init(const IPASettings &settings) @@ -179,6 +182,27 @@ int IPARPi::start(const IPAOperationData &ipaConfig, IPAOperationData *result) result->operation |= RPi::IPA_CONFIG_SENSOR; } + /* + * Initialise frame counts, and decide how many frames must be hidden or + *"mistrusted", which depends on whether this is a startup from cold, + * or merely a mode switch in a running system. + */ + frameCount_ = 0; + checkCount_ = 0; + unsigned int dropFrame = 0; + if (firstStart_) { + dropFrame = helper_->HideFramesStartup(); + mistrustCount_ = helper_->MistrustFramesStartup(); + } else { + dropFrame = helper_->HideFramesModeSwitch(); + mistrustCount_ = helper_->MistrustFramesModeSwitch(); + } + + result->data.push_back(dropFrame); + result->operation |= RPi::IPA_CONFIG_DROP_FRAMES; + + firstStart_ = false; + return 0; } @@ -293,25 +317,6 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, /* Pass the camera mode to the CamHelper to setup algorithms. */ helper_->SetCameraMode(mode_); - /* - * Initialise frame counts, and decide how many frames must be hidden or - *"mistrusted", which depends on whether this is a startup from cold, - * or merely a mode switch in a running system. - */ - frameCount_ = 0; - checkCount_ = 0; - unsigned int dropFrame = 0; - if (controllerInit_) { - dropFrame = helper_->HideFramesModeSwitch(); - mistrustCount_ = helper_->MistrustFramesModeSwitch(); - } else { - dropFrame = helper_->HideFramesStartup(); - mistrustCount_ = helper_->MistrustFramesStartup(); - } - - result->data.push_back(dropFrame); - result->operation |= RPi::IPA_CONFIG_DROP_FRAMES; - if (!controllerInit_) { /* Load the tuning file for this sensor. */ controller_.Read(tuningFile_.c_str()); diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 89a44763..5ae56628 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -745,13 +745,6 @@ int PipelineHandlerRPi::start(Camera *camera, [[maybe_unused]] ControlList *cont return ret; } - ret = queueAllBuffers(camera); - if (ret) { - LOG(RPI, Error) << "Failed to queue buffers"; - stop(camera); - return ret; - } - /* Check if a ScalerCrop control was specified. */ if (controls) data->applyScalerCrop(*controls); @@ -778,6 +771,19 @@ int PipelineHandlerRPi::start(Camera *camera, [[maybe_unused]] ControlList *cont LOG(RPI, Error) << "V4L2 staggered set failed"; } + if (result.operation & RPi::IPA_CONFIG_DROP_FRAMES) { + /* Configure the number of dropped frames required on startup. */ + data->dropFrameCount_ = result.data[0]; + } + + /* We need to set the dropFrameCount_ before queueing buffers. */ + ret = queueAllBuffers(camera); + if (ret) { + LOG(RPI, Error) << "Failed to queue buffers"; + stop(camera); + return ret; + } + /* * IPA configure may have changed the sensor flips - hence the bayer * order. Get the sensor format and set the ISP input now. @@ -1231,11 +1237,6 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) LOG(RPI, Error) << "V4L2 staggered set failed"; } - if (result.operation & RPi::IPA_CONFIG_DROP_FRAMES) { - /* Configure the number of dropped frames required on startup. */ - dropFrameCount_ = result.data[resultIdx++]; - } - /* * Configure the H/V flip controls based on the combination of * the sensor and user transform. From patchwork Mon Dec 7 18:01:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10599 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D6A2BBDB1F for ; Mon, 7 Dec 2020 18:02:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A080667E6E; Mon, 7 Dec 2020 19:02:07 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="ri0Cn6OM"; dkim-atps=neutral Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DA0CB67E6A for ; Mon, 7 Dec 2020 19:02:05 +0100 (CET) Received: by mail-ej1-x629.google.com with SMTP id f23so20811435ejk.2 for ; Mon, 07 Dec 2020 10:02:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wm3NAMcIrBR7j+LiHw5ffTkWgOX8WCAQ0spGz33JFqg=; b=ri0Cn6OM8wxoW3RQGOlducmwzbC8JX20KHmlR2W/BhDLn5pUsvZp7a6buc27IIAzLp 6gl1d/0pze55VjStrohMrW4rA7GHbxQ8hck8SYvHHQ2KD8kGdgX2OW1ri2UhKxaq5B6B UEiSk6gCpxMJdXQ4bs92gEWRrFAso0y49t380e8Dq1JHrV3yFpkoYHfGV6lR3F4JoLuu SX4XyIhi52rCylzSi2P8L1yplpXhT0Y2KD3arzd4kafQ3tBh6qfRukVn1MuZY+AiQYb/ jHiubW6AdTBbInIR+iFc2v3UYW/Ny06g78F2gqMcS869uDeZ5hb93MBpB9lUgo9oJ6iR pXsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wm3NAMcIrBR7j+LiHw5ffTkWgOX8WCAQ0spGz33JFqg=; b=fI/JyN+leKVGVZhilAVrH8u807TJOqDqCsweQhlHcFKTeGCCNTjxk9PZYBxQ62S7sK kARaKLUmwQnL58pJhpJetCCYJaAIT8GfjTsXSR9AkDjvJoR9mV86WWqW0opgBmzzZTX/ S0Rc6LVxsGnGyBJe/Wgo1DQSDSBxBBwwp9/ByD5OLkf/bCY/d/U3SfnwIzkQ+RKx8DCT NcPmo44Hkeef91o5F65QWHkdjAPsxsXfjsgw/g7ElWo+GfqPtO9USSSZ9WEwZszACOJB NLrLmgAGDuXVMKz9tryj1h17JIvYbOxbrw/7pmpCKj4uXiitVlYeU4HRhQYJhR7s7No5 5PGQ== X-Gm-Message-State: AOAM532QQ51W6wZCIwJgDnGRUYCsL0YR0EpmFTsoa19RsXYP+TIBqjIO RLArkolz58ZhVrVRQjAp7t2N7Rkztcih+gNn X-Google-Smtp-Source: ABdhPJyTnN9ubJ/x6guDYY0MoMLdfrZlkqA5EEvoMmx0F3m4fshvjbFBseZn/6C6qgFKZD2mtwgZGQ== X-Received: by 2002:a17:906:c006:: with SMTP id e6mr19882295ejz.374.1607364125290; Mon, 07 Dec 2020 10:02:05 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id be6sm14657977edb.29.2020.12.07.10.01.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Dec 2020 10:01:58 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 7 Dec 2020 18:01:17 +0000 Message-Id: <20201207180121.6374-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201207180121.6374-1-david.plowman@raspberrypi.com> References: <20201207180121.6374-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/6] src: ipa: raspberrypi: agc: Add GetConvergenceFrames method to AGC base class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We add a GetConvergenceFrames method to the AgcAlgorithm class which can be called when the AGC is started from scratch. It suggests how many frames should be dropped before displaying any (while the AGC converges). The Raspberry Pi specific implementation makes this customisable from the tuning file. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/agc_algorithm.hpp | 1 + src/ipa/raspberrypi/controller/rpi/agc.cpp | 13 +++++++++++++ src/ipa/raspberrypi/controller/rpi/agc.hpp | 2 ++ 3 files changed, 16 insertions(+) diff --git a/src/ipa/raspberrypi/controller/agc_algorithm.hpp b/src/ipa/raspberrypi/controller/agc_algorithm.hpp index b4ea54fb..85fc6084 100644 --- a/src/ipa/raspberrypi/controller/agc_algorithm.hpp +++ b/src/ipa/raspberrypi/controller/agc_algorithm.hpp @@ -15,6 +15,7 @@ class AgcAlgorithm : public Algorithm public: AgcAlgorithm(Controller *controller) : Algorithm(controller) {} // An AGC algorithm must provide the following: + virtual unsigned int GetConvergenceFrames(unsigned int mistrust_frames) const = 0; virtual void SetEv(double ev) = 0; virtual void SetFlickerPeriod(double flicker_period) = 0; virtual void SetFixedShutter(double fixed_shutter) = 0; // microseconds diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 9da18c31..787918cc 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -142,6 +142,7 @@ void AgcConfig::Read(boost::property_tree::ptree const ¶ms) Y_target.Read(params.get_child("y_target")); speed = params.get("speed", 0.2); startup_frames = params.get("startup_frames", 10); + convergence_frames = params.get("convergence_frames", 6); fast_reduce_threshold = params.get("fast_reduce_threshold", 0.4); base_ev = params.get("base_ev", 1.0); @@ -206,6 +207,18 @@ void Agc::Resume() fixed_analogue_gain_ = 0; } +unsigned int Agc::GetConvergenceFrames(unsigned int mistrust_frames) const +{ + // If shutter and gain have been explicitly set, there is no + // convergence to happen, so no need to drop any frames - return zero. + // Otherwise, any frames for which we have been told not to trust the + // statistics must be added to our own count. + if (fixed_shutter_ && fixed_analogue_gain_) + return 0; + else + return config_.convergence_frames + mistrust_frames; +} + void Agc::SetEv(double ev) { ev_ = ev; diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index 95db1812..7d41608a 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -52,6 +52,7 @@ struct AgcConfig { Pwl Y_target; double speed; uint16_t startup_frames; + unsigned int convergence_frames; double max_change; double min_change; double fast_reduce_threshold; @@ -74,6 +75,7 @@ public: bool IsPaused() const override; void Pause() override; void Resume() override; + unsigned int GetConvergenceFrames(unsigned int mistrust_frames) const override; void SetEv(double ev) override; void SetFlickerPeriod(double flicker_period) override; void SetFixedShutter(double fixed_shutter) override; // microseconds From patchwork Mon Dec 7 18:01:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10600 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6F3D9BDB1F for ; Mon, 7 Dec 2020 18:02:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 35F1967E6E; Mon, 7 Dec 2020 19:02:15 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="QKKkQbLB"; dkim-atps=neutral Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5A74C67E6A for ; Mon, 7 Dec 2020 19:02:14 +0100 (CET) Received: by mail-ed1-x533.google.com with SMTP id dk8so11818694edb.1 for ; Mon, 07 Dec 2020 10:02:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mknaU4XTrtLYRHZO3lNKtJ3aTj6z6MQXh7SDuOwDMTg=; b=QKKkQbLB4qmjWSRMMi8lNGn1YJRaxfbdMIWRiXR57fn0+o5nhV5c9sBip/rKXlinuI RwiuJCFO3w1N2qaIzhXjwsYaaSpeQPbxJNQxj3/DBRKQSmiX6GKeAmtuKSm/6i64zwwg knkdEBBRq+jeQ8jJ2OnidaS+NtGsa7sIaj37K36SRJTtbuALM7ZierEwGAAh9RKLYxdb R6llxXbKWLfrAuddcp6TrQtwMC5kkt7eEFX0rsV2W4FqKz+N/hD2cn6hTCQWRFkd/+Bx IhyhyeLXsoVJrJh9v6wHN6FDxbj+IsVt5SrFUYjPHQ8ufUyDvIVkB1sXn5BU+Bd+2tJV QyUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mknaU4XTrtLYRHZO3lNKtJ3aTj6z6MQXh7SDuOwDMTg=; b=bwNfwFixjRhD25yvbdwnTX91FjDIdmFO8POiS4+s1xNQj3KamhId7nLb6CoED4E89G LAeXJQh46xD9kYcEx2NnaqK5DMuR9iVb2SmjfSr9sxGYtd2uthOF5k+fHI/17NBu4/IQ +ZfOt8nYtfCooOewAf2pwvYapqGGxVSBkrQhVipnfaZ0i1bDpX3pbRlAa/kMutJ76kdj TqQ61K4zxK79ra1GDRTsQRb+xeQLG5Mx6mJmEdsV7hfKJAv7cIoYOAzaBjbK1dfiZM9e cpeMN7rh2qNgPFsS89CRN8UhAV0sPBargdvJTX2Ks7bcT3sxTlXzVP03PtZKbOHQtbN6 VKig== X-Gm-Message-State: AOAM533xqJx6i/mbZ3Qe0RE+OlvuF49pDm9SlwB5Fixxo63/udZCMmdb wJ1Sew6/zVEI6Lne1vkXSMlC6N1hxhdYrBr9 X-Google-Smtp-Source: ABdhPJxdknZq9AOLh/8q+YvhUwmgvr0vwr2MdB2X8Mu8Hy9b+nUwzyLxAKLMh14k9jwRZi+I9po8Mg== X-Received: by 2002:aa7:cd71:: with SMTP id ca17mr7226916edb.25.1607364133485; Mon, 07 Dec 2020 10:02:13 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id be6sm14657977edb.29.2020.12.07.10.02.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Dec 2020 10:02:08 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 7 Dec 2020 18:01:18 +0000 Message-Id: <20201207180121.6374-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201207180121.6374-1-david.plowman@raspberrypi.com> References: <20201207180121.6374-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/6] src: ipa: raspberrypi: agc: Add GetConvergenceFrames method to AWB base class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We add a GetConvergenceFrames method to the AwbAlgorithm class which can be called when the AWB is started from scratch. It suggests how many frames should be dropped before displaying any (while the AWB converges). The Raspberry Pi specific implementation makes this customisable from the tuning file. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/awb_algorithm.hpp | 1 + src/ipa/raspberrypi/controller/rpi/awb.cpp | 13 +++++++++++++ src/ipa/raspberrypi/controller/rpi/awb.hpp | 2 ++ 3 files changed, 16 insertions(+) diff --git a/src/ipa/raspberrypi/controller/awb_algorithm.hpp b/src/ipa/raspberrypi/controller/awb_algorithm.hpp index 5be0c9f4..84ae313b 100644 --- a/src/ipa/raspberrypi/controller/awb_algorithm.hpp +++ b/src/ipa/raspberrypi/controller/awb_algorithm.hpp @@ -15,6 +15,7 @@ class AwbAlgorithm : public Algorithm public: AwbAlgorithm(Controller *controller) : Algorithm(controller) {} // An AWB algorithm must provide the following: + virtual unsigned int GetConvergenceFrames(unsigned int mistrust_frames) const = 0; virtual void SetMode(std::string const &mode_name) = 0; virtual void SetManualGains(double manual_r, double manual_b) = 0; }; diff --git a/src/ipa/raspberrypi/controller/rpi/awb.cpp b/src/ipa/raspberrypi/controller/rpi/awb.cpp index 020825e3..6b359ac5 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.cpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.cpp @@ -59,6 +59,7 @@ void AwbConfig::Read(boost::property_tree::ptree const ¶ms) bayes = params.get("bayes", 1); frame_period = params.get("frame_period", 10); startup_frames = params.get("startup_frames", 10); + convergence_frames = params.get("convergence_frames", 3); speed = params.get("speed", 0.05); if (params.get_child_optional("ct_curve")) read_ct_curve(ct_r, ct_b, params.get_child("ct_curve")); @@ -165,6 +166,18 @@ void Awb::Initialise() prev_sync_results_ = sync_results_; } +unsigned int Awb::GetConvergenceFrames(unsigned int mistrust_frames) const +{ + // If colour gains have been explicitly set, there is no convergence + // to happen, so no need to drop any frames - return zero. + // Otherwise, any frames for which we have been told not to trust the + // statistics must be added to our own count. + if (manual_r_ && manual_b_) + return 0; + else + return config_.convergence_frames + mistrust_frames; +} + void Awb::SetMode(std::string const &mode_name) { std::unique_lock lock(settings_mutex_); diff --git a/src/ipa/raspberrypi/controller/rpi/awb.hpp b/src/ipa/raspberrypi/controller/rpi/awb.hpp index 545d85a8..d86b9598 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.hpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.hpp @@ -37,6 +37,7 @@ struct AwbConfig { uint16_t frame_period; // number of initial frames for which speed taken as 1.0 (maximum) uint16_t startup_frames; + unsigned int convergence_frames; // approx number of frames to converge double speed; // IIR filter speed applied to algorithm results bool fast; // "fast" mode uses a 16x16 rather than 32x32 grid Pwl ct_r; // function maps CT to r (= R/G) @@ -82,6 +83,7 @@ public: char const *Name() const override; void Initialise() override; void Read(boost::property_tree::ptree const ¶ms) override; + unsigned int GetConvergenceFrames(unsigned int mistrust_frames) const override; void SetMode(std::string const &name) override; void SetManualGains(double manual_r, double manual_b) override; void SwitchMode(CameraMode const &camera_mode, Metadata *metadata) override; From patchwork Mon Dec 7 18:01:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10601 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C512EBDB1F for ; Mon, 7 Dec 2020 18:02:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8F3C367E6B; Mon, 7 Dec 2020 19:02:24 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="ZjDcB23c"; dkim-atps=neutral Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CC2F667E6A for ; Mon, 7 Dec 2020 19:02:22 +0100 (CET) Received: by mail-ed1-x535.google.com with SMTP id u19so14720216edx.2 for ; Mon, 07 Dec 2020 10:02:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QxO4QgtZgd2HaiyXhTe2SCFiVm9KpKHiGeJ8yVR92K4=; b=ZjDcB23cp0M7qGXQvW80SA3lyCjOvPJwaQ8SsN+1WdcEIznHlagGb6+9BiQmJRZywp RvluOF/OB4W302If/7bWA3sNysG4mxeRtj4dGPBNqOThmlqXyOR1VlmN7dS9X408929Z i4OUKuYl3wmmzv4V8R02Ed9dMODRDLXg8G9Extq40i+P+wVG7mtA+rgLUcZ+hFz0uFm1 8j3IEltOvUgeeFhbm0oT5WJgMp89x6lHqAT32wF2QjTPhtlAAB7ZSPQQoKmKacwwu5E1 3wJMhnb6i39k64cT59cI03eE8iRvlBOzBo+Mp0FdrlWbrg9hY0YZHLPwSNBU1pkovP72 BOKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QxO4QgtZgd2HaiyXhTe2SCFiVm9KpKHiGeJ8yVR92K4=; b=uink8xEBgyGG6xGD+IdHjZTH8tjNN/T00kKOUcxUb+LxkCsxGqR48kyvXb1sWO3rTa f7I93ds46WoYvAdCfvR8bUJf2IcbApznpZFLgjyCSDpAoqDv0DclbTFUNArjn+jn4OZy +SRXVITn5qe6ciN5BdnADwZyySgM81l86czALgsMorcrD9LgBsMwEAlAiTRYn13KwE6Q D854FNOlzNsfrVlD5QZ7JGpovjVGBKj1i2cMbvVz0RT3m2FjYf6k+NTq7kdSo8xakdze L9/MA189H6sH6rC2UrBkM0Kn0d+z7igjQ6+2RnsUaXNwzzQhy+nFnQ3SRdZW+vFeNdOs H0uA== X-Gm-Message-State: AOAM531MrRmh6N5WKxTq7NgzzX7yzyWciAWdaAAZ+k1NhQSsiyfXhN6f mweUmmP+4JpMccTUXkr9DJ/vZ++fxsdUf65u X-Google-Smtp-Source: ABdhPJwi7lNf0P3p6b8180GGqt/UWHjYZQcK31xlvqeOMCX23bq/18HCvR55yhloQ3mQZqeJ3I99yg== X-Received: by 2002:a05:6402:21c7:: with SMTP id bi7mr21822788edb.54.1607364142004; Mon, 07 Dec 2020 10:02:22 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id be6sm14657977edb.29.2020.12.07.10.02.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Dec 2020 10:02:16 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 7 Dec 2020 18:01:19 +0000 Message-Id: <20201207180121.6374-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201207180121.6374-1-david.plowman@raspberrypi.com> References: <20201207180121.6374-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 4/6] src: ipa: raspberrypi: Compute inverse of piecewise linear function X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a method to the piecewise linear function (Pwl) class to compute the inverse of a given Pwl. If the input function is non-monotonic we can only produce a best effort "pseudo" inverse, and we signal this to the caller. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/pwl.cpp | 27 ++++++++++++++++++++++++++ src/ipa/raspberrypi/controller/pwl.hpp | 3 +++ 2 files changed, 30 insertions(+) diff --git a/src/ipa/raspberrypi/controller/pwl.cpp b/src/ipa/raspberrypi/controller/pwl.cpp index aa134a1f..70206418 100644 --- a/src/ipa/raspberrypi/controller/pwl.cpp +++ b/src/ipa/raspberrypi/controller/pwl.cpp @@ -114,6 +114,33 @@ Pwl::PerpType Pwl::Invert(Point const &xy, Point &perp, int &span, return PerpType::None; } +Pwl Pwl::Inverse(bool *true_inverse, const double eps) const +{ + bool appended = false, prepended = false, neither = false; + Pwl inverse; + + for (Point const &p : points_) { + if (inverse.Empty()) + inverse.Append(p.y, p.x, eps); + else if (p.y > inverse.points_.back().x - eps) { + inverse.Append(p.y, p.x, eps); + appended = true; + } else if (p.y < inverse.points_.front().x + eps) { + inverse.Prepend(p.y, p.x, eps); + prepended = true; + } else + neither = true; + } + + // This is not a proper inverse if we found ourselves putting points + // onto both ends of the inverse, or if there were points that couldn't + // go on either. + if (true_inverse) + *true_inverse = !(neither || (appended && prepended)); + + return inverse; +} + Pwl Pwl::Compose(Pwl const &other, const double eps) const { double this_x = points_[0].x, this_y = points_[0].y; diff --git a/src/ipa/raspberrypi/controller/pwl.hpp b/src/ipa/raspberrypi/controller/pwl.hpp index 4f168551..484672f6 100644 --- a/src/ipa/raspberrypi/controller/pwl.hpp +++ b/src/ipa/raspberrypi/controller/pwl.hpp @@ -80,6 +80,9 @@ public: }; PerpType Invert(Point const &xy, Point &perp, int &span, const double eps = 1e-6) const; + // Compute the inverse function. Indicate if it is a proper (true) + // inverse, or only a best effort (e.g. input was non-monotonic). + Pwl Inverse(bool *true_inverse = nullptr, const double eps = 1e-6) const; // Compose two Pwls together, doing "this" first and "other" after. Pwl Compose(Pwl const &other, const double eps = 1e-6) const; // Apply function to (x,y) values at every control point. From patchwork Mon Dec 7 18:01:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10602 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 27E43BDB1F for ; Mon, 7 Dec 2020 18:02:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E46C767E6E; Mon, 7 Dec 2020 19:02:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="KRr9JXfA"; dkim-atps=neutral Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DE5B067E6A for ; Mon, 7 Dec 2020 19:02:29 +0100 (CET) Received: by mail-ej1-x634.google.com with SMTP id qw4so20755662ejb.12 for ; Mon, 07 Dec 2020 10:02:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=G9+1znq3c+n3y5z8pFtYQXOK6+AKuOXoD46Dro7MZM4=; b=KRr9JXfA3C64ODwJG1OsKNn+ZZn6M1N7mN37MuYy+CNCNNcePeZHVpd7XEmzn4Fdkw 2S7fKANjyCMzQg6428XLJml2laVXghh1R6/J4BQWpVqWjMzoLjvtVIZubv8+pB1WN2Dd NiLpm6XXfGU5QtqMZd6oi7mEtBIZk3hEztEQsRJf73GYuM9H0tUPTv8bOiskWdIpkIQE jgwHuMVmKk/q8NXxCJCCI892UyiuTKK7VTSo4V5MBXEf5EMB+nN8c9YnobQwTwExrIVp DJP/l4A6giMkQomoT+7IjyEvHQ5aWjMNQo5Ow7Lh/aFu7iSVDBFvMw7PKtlsyNmaPBkZ PNdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=G9+1znq3c+n3y5z8pFtYQXOK6+AKuOXoD46Dro7MZM4=; b=hbDMYuiuPSEWQnwqS3+mAh9MgklayWLUKDjrdlB2j6VKnIy5kcI3wMTM46dtX4zUrw txDK9AcDcuIaYjX0JlbR+WCTO3MXJF1YPJmVMi6T4Knut8Wjw+uffTvcaBAm4yBMdpkC YcTtH9rfq0rt9ob0bUHnj+6FpZppEmD5C9cIUR5ZNDg9mUCX4nwJktAQ33BFkp5exVp/ XttMzt4XyhZcwEtd36IP4n6BM211sfnCBITswLnhItQrtuXGklj0Qy+AhCFP9C+eD3E6 D5tCaR9OHjwIwtKitDRTTHy2XFXsyJg2lw64mPR0CN8V7lxnOhD93uJIzpmSWWcwCewP jVbw== X-Gm-Message-State: AOAM531NXfY5eO9IvgJ0+uUGDvoJ5jkVfLxbasobDhmcO7v1vpnj0hqj fo+vNlGCePEayOo0AZm0kGUSdyK4PbjFfuQf X-Google-Smtp-Source: ABdhPJwKZyTcfZk46eP/Sf4LotnxOG1chpgWOgfKBQKnN6lG3IIoXT1uO9OO0BA74go+J743d2ofLQ== X-Received: by 2002:a17:906:7b82:: with SMTP id s2mr19718554ejo.435.1607364149228; Mon, 07 Dec 2020 10:02:29 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id be6sm14657977edb.29.2020.12.07.10.02.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Dec 2020 10:02:25 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 7 Dec 2020 18:01:20 +0000 Message-Id: <20201207180121.6374-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201207180121.6374-1-david.plowman@raspberrypi.com> References: <20201207180121.6374-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 5/6] src: ipa: raspberrypi: Estimate the colour temerature if starting with fixed colour gains X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the AWB is started from "cold" with fixed colour gains, we try to estimate the colour temperature this corresponds to (if a calibrated CT curve was supplied). When fixed colour gains are set after the AWB has been running, we leave the CT estimate alone, as the one we have is probably sensible. This estimated colour is passed out in the metadata for other algorithms - notably ALSC - to use. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 6 +++++- src/ipa/raspberrypi/controller/rpi/awb.cpp | 14 ++++++++++++++ src/ipa/raspberrypi/controller/rpi/awb.hpp | 1 + 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index 183a0c95..c354c985 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -146,6 +146,7 @@ void Alsc::Read(boost::property_tree::ptree const ¶ms) config_.threshold = params.get("threshold", 1e-3); } +static double get_ct(Metadata *metadata, double default_ct); static void get_cal_table(double ct, std::vector const &calibrations, double cal_table[XY]); @@ -210,6 +211,9 @@ void Alsc::SwitchMode(CameraMode const &camera_mode, // change. bool reset_tables = first_time_ || compare_modes(camera_mode_, camera_mode); + // Believe the colour temperature from the AWB, if there is one. + ct_ = get_ct(metadata, ct_); + // Ensure the other thread isn't running while we do this. waitForAysncThread(); @@ -254,7 +258,7 @@ void Alsc::fetchAsyncResults() memcpy(sync_results_, async_results_, sizeof(sync_results_)); } -static double get_ct(Metadata *metadata, double default_ct) +double get_ct(Metadata *metadata, double default_ct) { AwbStatus awb_status; awb_status.temperature_K = default_ct; // in case nothing found diff --git a/src/ipa/raspberrypi/controller/rpi/awb.cpp b/src/ipa/raspberrypi/controller/rpi/awb.cpp index 6b359ac5..2266c07b 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.cpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.cpp @@ -19,6 +19,9 @@ using namespace RPiController; const double Awb::RGB::INVALID = -1.0; +// todo - the locking in this algorithm needs some tidying up as has been done +// elsewhere (ALSC and AGC). + void AwbMode::Read(boost::property_tree::ptree const ¶ms) { ct_lo = params.get("lo"); @@ -121,6 +124,7 @@ Awb::Awb(Controller *controller) async_abort_ = async_start_ = async_started_ = async_finished_ = false; mode_ = nullptr; manual_r_ = manual_b_ = 0.0; + first_switch_mode_ = true; async_thread_ = std::thread(std::bind(&Awb::asyncFunc, this)); } @@ -201,9 +205,19 @@ void Awb::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, prev_sync_results_.gain_r = manual_r_; prev_sync_results_.gain_g = 1.0; prev_sync_results_.gain_b = manual_b_; + // If we're starting up for the first time, try and + // "dead reckon" the corresponding colour temperature. + if (first_switch_mode_ && config_.bayes) { + Pwl ct_r_inverse = std::move(config_.ct_r.Inverse()); + Pwl ct_b_inverse = std::move(config_.ct_b.Inverse()); + double ct_r = ct_r_inverse.Eval(ct_r_inverse.Domain().Clip(1 / manual_r_)); + double ct_b = ct_b_inverse.Eval(ct_b_inverse.Domain().Clip(1 / manual_b_)); + prev_sync_results_.temperature_K = (ct_r + ct_b) / 2; + } sync_results_ = prev_sync_results_; } metadata->Set("awb.status", prev_sync_results_); + first_switch_mode_ = false; } void Awb::fetchAsyncResults() diff --git a/src/ipa/raspberrypi/controller/rpi/awb.hpp b/src/ipa/raspberrypi/controller/rpi/awb.hpp index d86b9598..8525af32 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.hpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.hpp @@ -159,6 +159,7 @@ private: double manual_r_; // manual b setting double manual_b_; + bool first_switch_mode_; // is this the first call to SwitchMode? }; static inline Awb::RGB operator+(Awb::RGB const &a, Awb::RGB const &b) From patchwork Mon Dec 7 18:01:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10603 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 822D0BDB1F for ; Mon, 7 Dec 2020 18:02:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4A64967E6E; Mon, 7 Dec 2020 19:02:39 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qVGZKTh1"; dkim-atps=neutral Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A755267E6A for ; Mon, 7 Dec 2020 19:02:37 +0100 (CET) Received: by mail-ej1-x62d.google.com with SMTP id x16so20769098ejj.7 for ; Mon, 07 Dec 2020 10:02:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GUZ3CHBFUgzFpk0zl/qO1zEra+JQmulzfklWh6O0kWA=; b=qVGZKTh1bHWtkAskB52Urx6XmpZVMzyIQRMzXEutZof/FohfyBkFGJjNcjNFCnCyTV p9COUlaggoru19ZVYUobSsI7xw+2KtODv5BUQEw7Yiz6V/6r9plcJWjpf09Qz2szhVsF yQbpmS8yLMFufggcuWF2BekcZrZGdANzDmv+0BYBTRMiyxEOxSbhZ524U+VYym15Bigo 1WhgccuWXu9/2BKKyC8eX4eccRhXs+pR+WebXxuKPVqr637ljEGHuMoZ2u7AeUrUVphL LlR+jkX9kuFGnQINpspaehRvoET3Gy/fHIge2MPHwGjMqYi6n8IEfu8NUPzkR5UP3nzI CtXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GUZ3CHBFUgzFpk0zl/qO1zEra+JQmulzfklWh6O0kWA=; b=Aedy00ZUfnqLOn63CCMj3i3MZ2x94E5c5IR7GQZfjYv7dwKi36QuH9pT1MnshXKs/D r1J4231KeNsKs/58R+0LlIzjr8f7DMRsViABObYUBcXhqP2tXfwOVVFHD+JiLvyWZa4+ sd9gPYiCFTQwaq0+6u4FXfxBJqWFj/ceISZTZuXN3pJaatAu12cuqcnJXHty+HUR/7ey ajTu86LESoCVaQLcGDV0QdwOrPgy/aN7v48PTQbdMIkoXqx3e9hsOyiEe3kyCI4q9e9p N5ktsL+APfhB1A6PRivUaQ+smSEZaaL+F/8cyMNCkFYqqwrv8M/MXjPe0RoWX5uX5qNP EKAA== X-Gm-Message-State: AOAM5330ViDEsUDC1x7f9mhsBCGxoOl5tndGkAVgJlvqLHZ+rnOwkfn5 agxk+mZQbaQ4S5XJ3jBT9r9L0B84HVeia2Ow X-Google-Smtp-Source: ABdhPJzoomxM7azf4Zk4JbyXW9YHCosPP+nQ/QyKFnZirw9pxmUIrJTxWRmVEpJD4pYNm57e7BTpkQ== X-Received: by 2002:a17:906:c20f:: with SMTP id d15mr19828842ejz.341.1607364156876; Mon, 07 Dec 2020 10:02:36 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id be6sm14657977edb.29.2020.12.07.10.02.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Dec 2020 10:02:33 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 7 Dec 2020 18:01:21 +0000 Message-Id: <20201207180121.6374-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201207180121.6374-1-david.plowman@raspberrypi.com> References: <20201207180121.6374-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 6/6] src: ipa: raspberrypi: Move initial frame drop decision to AGC/AWB X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Previously the CamHelper was returning the number of frames to drop (on account of AGC/AWB converging). This wasn't really appropriate, it's better for the algorithms to do it as they know how many frames they might need. The CamHelper::HideFramesStartup method should now just be returning the number of frames to hide because they're bad/invalid in some way, not worrying about the AGC/AWB. For many sensors, the correct value for this is zero. But the ov5647 needs updating as it must return 2. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/cam_helper.cpp | 6 +++--- src/ipa/raspberrypi/cam_helper_ov5647.cpp | 10 ++++++++++ src/ipa/raspberrypi/raspberrypi.cpp | 16 ++++++++++++++++ 3 files changed, 29 insertions(+), 3 deletions(-) diff --git a/src/ipa/raspberrypi/cam_helper.cpp b/src/ipa/raspberrypi/cam_helper.cpp index c8ac3232..6efa0d7f 100644 --- a/src/ipa/raspberrypi/cam_helper.cpp +++ b/src/ipa/raspberrypi/cam_helper.cpp @@ -82,10 +82,10 @@ bool CamHelper::SensorEmbeddedDataPresent() const unsigned int CamHelper::HideFramesStartup() const { /* - * By default, hide 6 frames completely at start-up while AGC etc. sort - * themselves out (converge). + * The number of frames when a camera first starts that shouldn't be + * displayed as they are invalid in some way. */ - return 6; + return 0; } unsigned int CamHelper::HideFramesModeSwitch() const diff --git a/src/ipa/raspberrypi/cam_helper_ov5647.cpp b/src/ipa/raspberrypi/cam_helper_ov5647.cpp index dc5d8275..0b841cd1 100644 --- a/src/ipa/raspberrypi/cam_helper_ov5647.cpp +++ b/src/ipa/raspberrypi/cam_helper_ov5647.cpp @@ -19,6 +19,7 @@ public: uint32_t GainCode(double gain) const override; double Gain(uint32_t gain_code) const override; void GetDelays(int &exposure_delay, int &gain_delay) const override; + unsigned int HideFramesStartup() const override; unsigned int HideFramesModeSwitch() const override; unsigned int MistrustFramesStartup() const override; unsigned int MistrustFramesModeSwitch() const override; @@ -54,6 +55,15 @@ void CamHelperOv5647::GetDelays(int &exposure_delay, int &gain_delay) const gain_delay = 2; } +unsigned int CamHelperOv5647::HideFramesStartup() const +{ + /* + * On startup, we get a couple of under-exposed frames which + * we don't want shown. + */ + return 2; +} + unsigned int CamHelperOv5647::HideFramesModeSwitch() const { /* diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 0e89af00..da92e492 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -193,6 +193,22 @@ int IPARPi::start(const IPAOperationData &ipaConfig, IPAOperationData *result) if (firstStart_) { dropFrame = helper_->HideFramesStartup(); mistrustCount_ = helper_->MistrustFramesStartup(); + + /* The AGC and/or AWB algorithms may want us to drop more frames. */ + unsigned int convergence_frames = 0; + RPiController::AgcAlgorithm *agc = dynamic_cast( + controller_.GetAlgorithm("agc")); + if (agc) + convergence_frames = agc->GetConvergenceFrames(mistrustCount_); + + RPiController::AwbAlgorithm *awb = dynamic_cast( + controller_.GetAlgorithm("awb")); + if (awb) + convergence_frames = std::max(convergence_frames, + awb->GetConvergenceFrames(mistrustCount_)); + + dropFrame = std::max(dropFrame, convergence_frames); + LOG(IPARPI, Debug) << "Drop " << dropFrame << " frames on startup"; } else { dropFrame = helper_->HideFramesModeSwitch(); mistrustCount_ = helper_->MistrustFramesModeSwitch();