From patchwork Tue Dec 8 20:44:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10613 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 60833BDB20 for ; Tue, 8 Dec 2020 20:44:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 76F1467F0A; Tue, 8 Dec 2020 21:44:46 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="hNckknXQ"; dkim-atps=neutral Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7DAFA67E4D for ; Tue, 8 Dec 2020 21:44:45 +0100 (CET) Received: by mail-ed1-x52e.google.com with SMTP id k4so19057498edl.0 for ; Tue, 08 Dec 2020 12:44:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jR9ohWFy4goGadcZ87qnt51zd0sDVZUtX2ovYrNp/ek=; b=hNckknXQO2Vt3nZkdqhID1qxKQVsYTtskKh/PseIpFxUT8YwEmrf0JGgljo/DyITiC t1L0rgP8pbil70UmFGyIH9SPszTTsWjXcdEDkWXrm3iuCGpT8vDLUtbnLzxmUnkFEo88 lDVYJJ0hQWuU7E0xErcEK53A6jvPtyBiQVk8UgKfTQDke/UF53pSlSiNJiXZ4isnMoki IwB5mlGCiO4zsn58409ljX/L45xOcU8czS8pOha7L9eJ9hotDj1oGuHgqTf9FLd4r+Xi ET6xCuyolBWxIlXB6oSqiDbfw6dRQNMeNEIaqVHaauKs+rxRHD46b7BBuV/7xG//5OPP hc6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jR9ohWFy4goGadcZ87qnt51zd0sDVZUtX2ovYrNp/ek=; b=Qdy2ncLI6/DDFlHfrTjIKWJQUtebARfVxmPm7PxxFzD0HPlZZAfy9xEi6Uaf16/6ZS eDdSo2USzL5YAz7SYCJ3dnzARdg1qvXwB7PLYrtnWWn34bilCtF0r2VvWsEQupnCIibh DKqBvJV5xHiQ0Zo6aJKkKvF8wUvYmV9gubebNpQe24TNg+WiIVbiNxMmuoH64RZ2LmCr N3bEzZ4YwKXt1fQ6HCzvls9OtTqi1z6831X1457ze8DZKhFc4Rf3qDmzNw3UuH7QPJkr FTeMwx+F7Y/AguEUngEB+yIIHIKrYZkLFq+t91m70EX1+/kBfGcYbFno1Q1VTaaaWKir vWrg== X-Gm-Message-State: AOAM531f3orKzyhhL/eBJxnciSx4P6Hx86KRhV4BwDEdJgaGLf3mk/wO wSqLY6aylt0hToBO0MlQHYdV2Lm5AQSOHadk X-Google-Smtp-Source: ABdhPJwV1eh7fLIhvhYY8j+w+f7CnQQ98TrXK/XkoNi0dKlTkFAEnJT3CQMxfyHfboqmTM01MqSDEw== X-Received: by 2002:a50:f089:: with SMTP id v9mr26797692edl.353.1607460284809; Tue, 08 Dec 2020 12:44:44 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id be6sm18441471edb.29.2020.12.08.12.44.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Dec 2020 12:44:44 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Tue, 8 Dec 2020 20:44:36 +0000 Message-Id: <20201208204441.9356-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201208204441.9356-1-david.plowman@raspberrypi.com> References: <20201208204441.9356-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/6] src: raspberrypi: Pass the drop frame count in start, not configure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The number of frames to drop (not display) is passed back now from the start method, not configure. This means applications have a chance to set fixed exposure/gain before starting the camera and this can affect the frame drop count that is returned. Note how we need to be able to tell the very first time we start the camera from subsequent restarts, hence addition of the "firstStart_" flag. Both the IPA implementation file and the pipeline handler need matching modifications. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/raspberrypi.cpp | 45 ++++++++++--------- .../pipeline/raspberrypi/raspberrypi.cpp | 25 ++++++----- 2 files changed, 38 insertions(+), 32 deletions(-) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index ebe3013a..ba63480e 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -67,7 +67,7 @@ public: IPARPi() : lastMode_({}), controller_(), controllerInit_(false), frameCount_(0), checkCount_(0), mistrustCount_(0), - lsTable_(nullptr) + lsTable_(nullptr), firstStart_(true) { } @@ -145,6 +145,9 @@ private: /* LS table allocation passed in from the pipeline handler. */ FileDescriptor lsTableHandle_; void *lsTable_; + + /* Distinguish the first camera start from others. */ + bool firstStart_; }; int IPARPi::init(const IPASettings &settings) @@ -180,6 +183,27 @@ int IPARPi::start(const IPAOperationData &data, IPAOperationData *result) result->operation |= RPi::IPA_CONFIG_SENSOR; } + /* + * Initialise frame counts, and decide how many frames must be hidden or + * "mistrusted", which depends on whether this is a startup from cold, + * or merely a mode switch in a running system. + */ + frameCount_ = 0; + checkCount_ = 0; + unsigned int dropFrame = 0; + if (firstStart_) { + dropFrame = helper_->HideFramesStartup(); + mistrustCount_ = helper_->MistrustFramesStartup(); + } else { + dropFrame = helper_->HideFramesModeSwitch(); + mistrustCount_ = helper_->MistrustFramesModeSwitch(); + } + + result->data.push_back(dropFrame); + result->operation |= RPi::IPA_CONFIG_DROP_FRAMES; + + firstStart_ = false; + return 0; } @@ -305,25 +329,6 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, /* Pass the camera mode to the CamHelper to setup algorithms. */ helper_->SetCameraMode(mode_); - /* - * Initialise frame counts, and decide how many frames must be hidden or - *"mistrusted", which depends on whether this is a startup from cold, - * or merely a mode switch in a running system. - */ - frameCount_ = 0; - checkCount_ = 0; - unsigned int dropFrame = 0; - if (controllerInit_) { - dropFrame = helper_->HideFramesModeSwitch(); - mistrustCount_ = helper_->MistrustFramesModeSwitch(); - } else { - dropFrame = helper_->HideFramesStartup(); - mistrustCount_ = helper_->MistrustFramesStartup(); - } - - result->data.push_back(dropFrame); - result->operation |= RPi::IPA_CONFIG_DROP_FRAMES; - if (!controllerInit_) { /* Load the tuning file for this sensor. */ controller_.Read(tuningFile_.c_str()); diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 593fd7ac..439c21ce 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -745,13 +745,6 @@ int PipelineHandlerRPi::start(Camera *camera, [[maybe_unused]] ControlList *cont return ret; } - ret = queueAllBuffers(camera); - if (ret) { - LOG(RPI, Error) << "Failed to queue buffers"; - stop(camera); - return ret; - } - /* Check if a ScalerCrop control was specified. */ if (controls) data->applyScalerCrop(*controls); @@ -779,6 +772,19 @@ int PipelineHandlerRPi::start(Camera *camera, [[maybe_unused]] ControlList *cont LOG(RPI, Error) << "V4L2 staggered set failed"; } + if (result.operation & RPi::IPA_CONFIG_DROP_FRAMES) { + /* Configure the number of dropped frames required on startup. */ + data->dropFrameCount_ = result.data[0]; + } + + /* We need to set the dropFrameCount_ before queueing buffers. */ + ret = queueAllBuffers(camera); + if (ret) { + LOG(RPI, Error) << "Failed to queue buffers"; + stop(camera); + return ret; + } + /* * IPA configure may have changed the sensor flips - hence the bayer * order. Get the sensor format and set the ISP input now. @@ -1237,11 +1243,6 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) LOG(RPI, Error) << "V4L2 staggered set failed"; } - if (result.operation & RPi::IPA_CONFIG_DROP_FRAMES) { - /* Configure the number of dropped frames required on startup. */ - dropFrameCount_ = result.data[resultIdx++]; - } - /* * Configure the H/V flip controls based on the combination of * the sensor and user transform. From patchwork Tue Dec 8 20:44:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10614 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 109E1BDB20 for ; Tue, 8 Dec 2020 20:44:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D156167E4D; Tue, 8 Dec 2020 21:44:48 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="r4251brt"; dkim-atps=neutral Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8575567F0B for ; Tue, 8 Dec 2020 21:44:46 +0100 (CET) Received: by mail-ed1-x52a.google.com with SMTP id u19so19035263edx.2 for ; Tue, 08 Dec 2020 12:44:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UYqtHEtBUNAP4L0qxPTMgG5zq+NcHWYZHLEVzCW38KQ=; b=r4251brtCqji/ieJdgO/tgiZcludjXOD2fc3+XnbXGbTVFrIsD4dcaKFk3P0fKXEzn nNpNk/oOnb66oJb23FloBAxmLTQTpX7mdksOUkLS9FV3FRco+a7V8KNwED4qd1Enyvrg HpH8E2SxKsrp3Ix9A8auXvYT+VpN6voj9eJlYXg30woXvXm+PJ8Vrudz9Q8FUinXvVPl C3pO2mEtEZSVA+emYcTM37EOFWkNyRPOYMuB8sUeG+b/sCrbn8pClAImAkP+4tdMPLSC 0qvNb7AYhtp4FTUoTlHQjSBWu3H5/QUINDVLmn4YMRyHyYaRmoyO5T3a+t1D6d3l9Nl3 0FMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UYqtHEtBUNAP4L0qxPTMgG5zq+NcHWYZHLEVzCW38KQ=; b=mWshqtZb8uEOy3QW521ej47e1KaMpNwR+FXONX1H9HIJj4sEJ235/QGpL3BunX/KOZ B+W87BsJZBcana1l7skawyMdamj+2QSSAi6yTfrFTbP59mUNOZQ4k/n9dso26UDycFLN c111CCEBpk9z1VEyrRDWTpHx+iN3hg2wYwsiFQUyUzbPqbRgG38HoPBRYz1HfIEwcWeB pq3Ln5zgEznnTv8zIxijBPHQWaq/K/7yAjreC7wBJdYx8xrtmnpmYTu4Qx7DBGIg1zKc NvWkbTKhosPPgwUvcN4FmU/h6ltFV8rvfmnipdJkN+BDNT9ULEdEsd/qKzN5e7yPAnKD 3jIA== X-Gm-Message-State: AOAM532TkDDdWyYR80/97G7GeiIDyHHYZZIIqCuzxk2SzMz2Ounm7Ket 5jPBrRCP0BcUsn191v8U9xvtGskcUAVZNrZK X-Google-Smtp-Source: ABdhPJyJpUtXmPka4yBw5n5b6HM+ds+t/rVNam4KpGckBJmJcrGbhKjPLxPg1YAq6zhIuBtz2o1piQ== X-Received: by 2002:aa7:cf8f:: with SMTP id z15mr22970078edx.17.1607460285847; Tue, 08 Dec 2020 12:44:45 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id be6sm18441471edb.29.2020.12.08.12.44.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Dec 2020 12:44:45 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Tue, 8 Dec 2020 20:44:37 +0000 Message-Id: <20201208204441.9356-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201208204441.9356-1-david.plowman@raspberrypi.com> References: <20201208204441.9356-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/6] src: ipa: raspberrypi: agc: Add GetConvergenceFrames method to AGC base class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We add a GetConvergenceFrames method to the AgcAlgorithm class which can be called when the AGC is started from scratch. It suggests how many frames should be dropped before displaying any (while the AGC converges). The Raspberry Pi specific implementation makes this customisable from the tuning file. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/agc_algorithm.hpp | 1 + src/ipa/raspberrypi/controller/rpi/agc.cpp | 11 +++++++++++ src/ipa/raspberrypi/controller/rpi/agc.hpp | 2 ++ 3 files changed, 14 insertions(+) diff --git a/src/ipa/raspberrypi/controller/agc_algorithm.hpp b/src/ipa/raspberrypi/controller/agc_algorithm.hpp index b4ea54fb..981f1de2 100644 --- a/src/ipa/raspberrypi/controller/agc_algorithm.hpp +++ b/src/ipa/raspberrypi/controller/agc_algorithm.hpp @@ -15,6 +15,7 @@ class AgcAlgorithm : public Algorithm public: AgcAlgorithm(Controller *controller) : Algorithm(controller) {} // An AGC algorithm must provide the following: + virtual unsigned int GetConvergenceFrames() const = 0; virtual void SetEv(double ev) = 0; virtual void SetFlickerPeriod(double flicker_period) = 0; virtual void SetFixedShutter(double fixed_shutter) = 0; // microseconds diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 9da18c31..eddd1684 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -142,6 +142,7 @@ void AgcConfig::Read(boost::property_tree::ptree const ¶ms) Y_target.Read(params.get_child("y_target")); speed = params.get("speed", 0.2); startup_frames = params.get("startup_frames", 10); + convergence_frames = params.get("convergence_frames", 6); fast_reduce_threshold = params.get("fast_reduce_threshold", 0.4); base_ev = params.get("base_ev", 1.0); @@ -206,6 +207,16 @@ void Agc::Resume() fixed_analogue_gain_ = 0; } +unsigned int Agc::GetConvergenceFrames() const +{ + // If shutter and gain have been explicitly set, there is no + // convergence to happen, so no need to drop any frames - return zero. + if (fixed_shutter_ && fixed_analogue_gain_) + return 0; + else + return config_.convergence_frames; +} + void Agc::SetEv(double ev) { ev_ = ev; diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index 95db1812..05c334e4 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -52,6 +52,7 @@ struct AgcConfig { Pwl Y_target; double speed; uint16_t startup_frames; + unsigned int convergence_frames; double max_change; double min_change; double fast_reduce_threshold; @@ -74,6 +75,7 @@ public: bool IsPaused() const override; void Pause() override; void Resume() override; + unsigned int GetConvergenceFrames() const override; void SetEv(double ev) override; void SetFlickerPeriod(double flicker_period) override; void SetFixedShutter(double fixed_shutter) override; // microseconds From patchwork Tue Dec 8 20:44:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10615 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 85F12BDB20 for ; Tue, 8 Dec 2020 20:44:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 41C1367E6F; Tue, 8 Dec 2020 21:44:49 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="hySDJF/3"; dkim-atps=neutral Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6D2BF67E4D for ; Tue, 8 Dec 2020 21:44:47 +0100 (CET) Received: by mail-ed1-x531.google.com with SMTP id i24so11129006edj.8 for ; Tue, 08 Dec 2020 12:44:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pvVockX49tEo+XYb/GBsyLuvcr9bEh20/7VlEgiMPRI=; b=hySDJF/3xxvInG0V0Gjgo0MM1BRwfYP8BU87H/IzIkwYzJ5E6/x+/nsiClgVTyED7Z 1M4k2Pnz+ymdzt1kUTJ60ItJxfsbuzVYz6I930TYyhW4U5Q0hyvnm2BUw9fJfbr7gBdI ztaOLaqycjY9asqeBUqitnj80TbW1KEBNMOzwQoyzgtqnXCABhYl4vliUvpb0uRBg27g nhQRYT5K5a+VZewEG7yqUzhgAV4cXYDMqtqAcadKlTyInfHHJCpfpyS4u3wrm5Ul0zF2 ZihVO/DR/opJFtUokwgTrpvn8szjiZZ7QCJXV8COMcbVraDvLv9YlXZoW21232HSSZLU G7aA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pvVockX49tEo+XYb/GBsyLuvcr9bEh20/7VlEgiMPRI=; b=PdbZPm9suXA45IJCUS1/b6A4nDwyPN6NXr6KlLksGPJ+a9rqus8Ty5XDJL9q+yu4GT FLzXpniJlNoEEF/io/ebbsR0KMNAnMjbil142/2TlkqL+eCWxH92ogeWx31rAJ8dFDS1 WXJZBsEFC69RRWNLHHsbJgQgSvs0+Fhz/e7SCa1rGnEE2Y/grkZyge2dWHYScq2WVAk5 qtidWPQmmklV7t03yg3fIGTOHkjmSD5fRCTWGtdv2s8k7fJX/cXBCKzomUlzIxDrAHTv xVCnjENIKKJV3Tq920EGQ7S/6DZma94DlLaCcpZapXupBvkb0N4GIavAFD4BAT7FC036 rWqQ== X-Gm-Message-State: AOAM5305/O7tNoVGQQLMTKempUpdnRO1CQ+kk12wpx1Ur85H8egZ9BBB 0kZC84Hxb9yW1qSHHd3Cp/RvNk2OUMVUeNWj X-Google-Smtp-Source: ABdhPJz5RRjZujl3SuxuJAMeZoJblkc7uEYC/6+3YYBA6v/4CO8SJ7e+F/iHMt/cbPHYQT1u5C/7zg== X-Received: by 2002:a50:ab47:: with SMTP id t7mr26771256edc.289.1607460286723; Tue, 08 Dec 2020 12:44:46 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id be6sm18441471edb.29.2020.12.08.12.44.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Dec 2020 12:44:46 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Tue, 8 Dec 2020 20:44:38 +0000 Message-Id: <20201208204441.9356-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201208204441.9356-1-david.plowman@raspberrypi.com> References: <20201208204441.9356-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 3/6] src: ipa: raspberrypi: awb: Add GetConvergenceFrames method to AWB base class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We add a GetConvergenceFrames method to the AwbAlgorithm class which can be called when the AWB is started from scratch. It suggests how many frames should be dropped before displaying any (while the AWB converges). The Raspberry Pi specific implementation makes this customisable from the tuning file. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/awb_algorithm.hpp | 1 + src/ipa/raspberrypi/controller/rpi/awb.cpp | 11 +++++++++++ src/ipa/raspberrypi/controller/rpi/awb.hpp | 2 ++ 3 files changed, 14 insertions(+) diff --git a/src/ipa/raspberrypi/controller/awb_algorithm.hpp b/src/ipa/raspberrypi/controller/awb_algorithm.hpp index 5be0c9f4..96f88afc 100644 --- a/src/ipa/raspberrypi/controller/awb_algorithm.hpp +++ b/src/ipa/raspberrypi/controller/awb_algorithm.hpp @@ -15,6 +15,7 @@ class AwbAlgorithm : public Algorithm public: AwbAlgorithm(Controller *controller) : Algorithm(controller) {} // An AWB algorithm must provide the following: + virtual unsigned int GetConvergenceFrames() const = 0; virtual void SetMode(std::string const &mode_name) = 0; virtual void SetManualGains(double manual_r, double manual_b) = 0; }; diff --git a/src/ipa/raspberrypi/controller/rpi/awb.cpp b/src/ipa/raspberrypi/controller/rpi/awb.cpp index 020825e3..10600f75 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.cpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.cpp @@ -59,6 +59,7 @@ void AwbConfig::Read(boost::property_tree::ptree const ¶ms) bayes = params.get("bayes", 1); frame_period = params.get("frame_period", 10); startup_frames = params.get("startup_frames", 10); + convergence_frames = params.get("convergence_frames", 3); speed = params.get("speed", 0.05); if (params.get_child_optional("ct_curve")) read_ct_curve(ct_r, ct_b, params.get_child("ct_curve")); @@ -165,6 +166,16 @@ void Awb::Initialise() prev_sync_results_ = sync_results_; } +unsigned int Awb::GetConvergenceFrames() const +{ + // If colour gains have been explicitly set, there is no convergence + // to happen, so no need to drop any frames - return zero. + if (manual_r_ && manual_b_) + return 0; + else + return config_.convergence_frames; +} + void Awb::SetMode(std::string const &mode_name) { std::unique_lock lock(settings_mutex_); diff --git a/src/ipa/raspberrypi/controller/rpi/awb.hpp b/src/ipa/raspberrypi/controller/rpi/awb.hpp index 545d85a8..6fc045ce 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.hpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.hpp @@ -37,6 +37,7 @@ struct AwbConfig { uint16_t frame_period; // number of initial frames for which speed taken as 1.0 (maximum) uint16_t startup_frames; + unsigned int convergence_frames; // approx number of frames to converge double speed; // IIR filter speed applied to algorithm results bool fast; // "fast" mode uses a 16x16 rather than 32x32 grid Pwl ct_r; // function maps CT to r (= R/G) @@ -82,6 +83,7 @@ public: char const *Name() const override; void Initialise() override; void Read(boost::property_tree::ptree const ¶ms) override; + unsigned int GetConvergenceFrames() const override; void SetMode(std::string const &name) override; void SetManualGains(double manual_r, double manual_b) override; void SwitchMode(CameraMode const &camera_mode, Metadata *metadata) override; From patchwork Tue Dec 8 20:44:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10616 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F401DBDB20 for ; Tue, 8 Dec 2020 20:44:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A198667F12; Tue, 8 Dec 2020 21:44:49 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="NlUkXIQ1"; dkim-atps=neutral Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [IPv6:2a00:1450:4864:20::532]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 651CE67E6F for ; Tue, 8 Dec 2020 21:44:48 +0100 (CET) Received: by mail-ed1-x532.google.com with SMTP id cm17so19040349edb.4 for ; Tue, 08 Dec 2020 12:44:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Zy5kG8Dx19vajqkeNd3AVwmfxp8E+gdkVwFyJzoeJFo=; b=NlUkXIQ1f1A74pK7VGzEdKAUWdf/tS4vkfEe93EyRoRG/WDC13Hid+cJGiOdMpWxau N8eAPDd9DHGkFNC6atKLAgbAkidSsVHDiO5SQ3rTjAE2gBcDZACTH+4AJcoVWGmcXlOv IswuGbOd8b0gQYK61+0K4b9xpAFknNEygHhBMWsCgrA1U6bHcwDFSoxu03EE7Z9APr/6 XTqvEQ9SxkIn8hHqEjVFoChy8NdXBQDu7pXko1JVF6rvQZFuHtq6hvjabNr/bcmIenb8 4eY8a+gLyW5zYCvpNLhOfA6GPxWc31crhAEnG4Q0Vqi3P1n5JzpaP1ItXjMOds/QMUuR OfRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Zy5kG8Dx19vajqkeNd3AVwmfxp8E+gdkVwFyJzoeJFo=; b=SH0xSWAQfSL1n7MjxoZ4jYFAiBOgXQ/UFsN0iT4zp/tX+QB0kBJ01lWDuP/jNfWNls kcQ/R02i8/E2pmnpebd+TENKypK0J6C7T9cqdU/eNKuZOFdJLklKP+5I8lHepJ7WL2Aq 2oxPQR914YdJavfVazneBax8qoHHKbjERX87OrRfdkr9PW7Fz4kkyV7BQYwvzVzGiLZw x/WspTS7N7YIEEHCptcN9stgkATq3vp/sgVpGWIJgdNzwVP7vunLcHjjsBBWbRLufm96 DNnStLMHe+wAl8Irwf/ZOnC25zPeWNX3mc0m0MYEOYXkkf322jDcgHcIHwP2BMJmLbo0 YI1w== X-Gm-Message-State: AOAM531oe2vFVtTGiwMNhD9czA8838HdTJNRSnGQ/xXPx31ZiLBvPEHn KWBIC5UK0lXRLpXh4Q4ayOLwuz9Lm6dwdaSu X-Google-Smtp-Source: ABdhPJy57bJJIYrfggRqJwv17gF4JEUiyrBUXEMkMw36r++eQo3Fk4S0belR9VRLctck1Kih0ChQfA== X-Received: by 2002:a05:6402:1caa:: with SMTP id cz10mr14565766edb.345.1607460287756; Tue, 08 Dec 2020 12:44:47 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id be6sm18441471edb.29.2020.12.08.12.44.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Dec 2020 12:44:47 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Tue, 8 Dec 2020 20:44:39 +0000 Message-Id: <20201208204441.9356-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201208204441.9356-1-david.plowman@raspberrypi.com> References: <20201208204441.9356-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 4/6] src: ipa: raspberrypi: Compute inverse of piecewise linear function X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a method to the piecewise linear function (Pwl) class to compute the inverse of a given Pwl. If the input function is non-monotonic we can only produce a best effort "pseudo" inverse, and we signal this to the caller. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/pwl.cpp | 30 ++++++++++++++++++++++++++ src/ipa/raspberrypi/controller/pwl.hpp | 3 +++ 2 files changed, 33 insertions(+) diff --git a/src/ipa/raspberrypi/controller/pwl.cpp b/src/ipa/raspberrypi/controller/pwl.cpp index aa134a1f..130c820b 100644 --- a/src/ipa/raspberrypi/controller/pwl.cpp +++ b/src/ipa/raspberrypi/controller/pwl.cpp @@ -114,6 +114,36 @@ Pwl::PerpType Pwl::Invert(Point const &xy, Point &perp, int &span, return PerpType::None; } +Pwl Pwl::Inverse(bool *true_inverse, const double eps) const +{ + bool appended = false, prepended = false, neither = false; + Pwl inverse; + + for (Point const &p : points_) { + if (inverse.Empty()) + inverse.Append(p.y, p.x, eps); + else if (std::abs(inverse.points_.back().x - p.y) <= eps || + std::abs(inverse.points_.front().x - p.y) <= eps) + /* do nothing */; + else if (p.y > inverse.points_.back().x) { + inverse.Append(p.y, p.x, eps); + appended = true; + } else if (p.y < inverse.points_.front().x) { + inverse.Prepend(p.y, p.x, eps); + prepended = true; + } else + neither = true; + } + + // This is not a proper inverse if we found ourselves putting points + // onto both ends of the inverse, or if there were points that couldn't + // go on either. + if (true_inverse) + *true_inverse = !(neither || (appended && prepended)); + + return inverse; +} + Pwl Pwl::Compose(Pwl const &other, const double eps) const { double this_x = points_[0].x, this_y = points_[0].y; diff --git a/src/ipa/raspberrypi/controller/pwl.hpp b/src/ipa/raspberrypi/controller/pwl.hpp index 4f168551..484672f6 100644 --- a/src/ipa/raspberrypi/controller/pwl.hpp +++ b/src/ipa/raspberrypi/controller/pwl.hpp @@ -80,6 +80,9 @@ public: }; PerpType Invert(Point const &xy, Point &perp, int &span, const double eps = 1e-6) const; + // Compute the inverse function. Indicate if it is a proper (true) + // inverse, or only a best effort (e.g. input was non-monotonic). + Pwl Inverse(bool *true_inverse = nullptr, const double eps = 1e-6) const; // Compose two Pwls together, doing "this" first and "other" after. Pwl Compose(Pwl const &other, const double eps = 1e-6) const; // Apply function to (x,y) values at every control point. From patchwork Tue Dec 8 20:44:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10617 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 658C9BDB20 for ; Tue, 8 Dec 2020 20:44:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 26B3F67F18; Tue, 8 Dec 2020 21:44:52 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="KSnWEE1n"; dkim-atps=neutral Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B101367F09 for ; Tue, 8 Dec 2020 21:44:49 +0100 (CET) Received: by mail-ed1-x531.google.com with SMTP id cw27so18999405edb.5 for ; Tue, 08 Dec 2020 12:44:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vUfawtU5Ys8cWncdGkHdhCkjx5+l5nzjqt7tZ+fvOL0=; b=KSnWEE1nLYgJgXDXed7TxUVwN3BadurdXwU1hiyQIcrXpnBUUxlZwGEtDALC5P91mw HW5cF/PXKZxD4twku71c+NvxJlAKbqCf5kKr4Q9In91UMjCIV+pz+Jdishl5xWjm2EqP jYOmZXX4ovbZCKbI6/huCQUVs8dNs4oJSJzeMuY14NfKjXjwUHefE9yaej8NYMNcVaSf XqtyvkLXt5YqLdlzpphq47OGHH+OeZy7r2B2N18RynHztByVbGHtS5TFif1hJbJHU07u tJDoy0uXDWO2GGEQaiHUmsab9HexIZ1KDJMT7u2B2P8h63wkqooHqVh55Iy0x7C5o64v zp/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vUfawtU5Ys8cWncdGkHdhCkjx5+l5nzjqt7tZ+fvOL0=; b=oJunzBorQ1ULC84/t01mAzqu9HCkyPbBldCHFqa1y+oheFLtuJ6WRtPUSG9cRc1Wlt 55VvbwgXf8MZ9DW/WSqYus1d2uzwKCtdTTHOx7DryThr7ZwwdkHOKTjm+Q4Th2zYydV0 yf945kCR81qyWFq9juat5CnHUSdnHuOr8nb/9sTyUaZVS8g0DpK+9SfNOfOet9WBqV9L vcdbRtXkr3bg3tn0XpTBJELcq4yklkDnXhu29ljMjFqC+3UaWJP8hFubi6cjcHMx5Kgr 2SYe/zk3SLxisDgjgKJrOnaLZ9nX8Zkh78vTR15aIw4zHJCkhIXL31pQNvEThTkKJcPH gsAg== X-Gm-Message-State: AOAM533qsv4iNbMo5P3Ugrk4A5rB10qRYnNABvGfOUJd8CUVtfWOW5Zo w9lVlLv7RY/gBb6C/lqEICcF/MuTVwfvonYi X-Google-Smtp-Source: ABdhPJz6EcZ/hy9d3BiqrUwSwejJSBfPRSyQaZdDzPv4IxypbKv3elLmKbQ+RmrUw8KAumo0wBRLDw== X-Received: by 2002:aa7:c4c2:: with SMTP id p2mr26281712edr.63.1607460288659; Tue, 08 Dec 2020 12:44:48 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id be6sm18441471edb.29.2020.12.08.12.44.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Dec 2020 12:44:48 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Tue, 8 Dec 2020 20:44:40 +0000 Message-Id: <20201208204441.9356-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201208204441.9356-1-david.plowman@raspberrypi.com> References: <20201208204441.9356-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 5/6] src: ipa: raspberrypi: Estimate the colour temerature if starting with fixed colour gains X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the AWB is started from "cold" with fixed colour gains, we try to estimate the colour temperature this corresponds to (if a calibrated CT curve was supplied). When fixed colour gains are set after the AWB has been running, we leave the CT estimate alone, as the one we have is probably sensible. This estimated colour is passed out in the metadata for other algorithms - notably ALSC - to use. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 6 +++++- src/ipa/raspberrypi/controller/rpi/awb.cpp | 14 ++++++++++++++ src/ipa/raspberrypi/controller/rpi/awb.hpp | 1 + 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index 183a0c95..c354c985 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -146,6 +146,7 @@ void Alsc::Read(boost::property_tree::ptree const ¶ms) config_.threshold = params.get("threshold", 1e-3); } +static double get_ct(Metadata *metadata, double default_ct); static void get_cal_table(double ct, std::vector const &calibrations, double cal_table[XY]); @@ -210,6 +211,9 @@ void Alsc::SwitchMode(CameraMode const &camera_mode, // change. bool reset_tables = first_time_ || compare_modes(camera_mode_, camera_mode); + // Believe the colour temperature from the AWB, if there is one. + ct_ = get_ct(metadata, ct_); + // Ensure the other thread isn't running while we do this. waitForAysncThread(); @@ -254,7 +258,7 @@ void Alsc::fetchAsyncResults() memcpy(sync_results_, async_results_, sizeof(sync_results_)); } -static double get_ct(Metadata *metadata, double default_ct) +double get_ct(Metadata *metadata, double default_ct) { AwbStatus awb_status; awb_status.temperature_K = default_ct; // in case nothing found diff --git a/src/ipa/raspberrypi/controller/rpi/awb.cpp b/src/ipa/raspberrypi/controller/rpi/awb.cpp index 10600f75..f66c2b29 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.cpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.cpp @@ -19,6 +19,9 @@ using namespace RPiController; const double Awb::RGB::INVALID = -1.0; +// todo - the locking in this algorithm needs some tidying up as has been done +// elsewhere (ALSC and AGC). + void AwbMode::Read(boost::property_tree::ptree const ¶ms) { ct_lo = params.get("lo"); @@ -121,6 +124,7 @@ Awb::Awb(Controller *controller) async_abort_ = async_start_ = async_started_ = async_finished_ = false; mode_ = nullptr; manual_r_ = manual_b_ = 0.0; + first_switch_mode_ = true; async_thread_ = std::thread(std::bind(&Awb::asyncFunc, this)); } @@ -199,9 +203,19 @@ void Awb::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, prev_sync_results_.gain_r = manual_r_; prev_sync_results_.gain_g = 1.0; prev_sync_results_.gain_b = manual_b_; + // If we're starting up for the first time, try and + // "dead reckon" the corresponding colour temperature. + if (first_switch_mode_ && config_.bayes) { + Pwl ct_r_inverse = config_.ct_r.Inverse(); + Pwl ct_b_inverse = config_.ct_b.Inverse(); + double ct_r = ct_r_inverse.Eval(ct_r_inverse.Domain().Clip(1 / manual_r_)); + double ct_b = ct_b_inverse.Eval(ct_b_inverse.Domain().Clip(1 / manual_b_)); + prev_sync_results_.temperature_K = (ct_r + ct_b) / 2; + } sync_results_ = prev_sync_results_; } metadata->Set("awb.status", prev_sync_results_); + first_switch_mode_ = false; } void Awb::fetchAsyncResults() diff --git a/src/ipa/raspberrypi/controller/rpi/awb.hpp b/src/ipa/raspberrypi/controller/rpi/awb.hpp index 6fc045ce..83b81498 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.hpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.hpp @@ -159,6 +159,7 @@ private: double manual_r_; // manual b setting double manual_b_; + bool first_switch_mode_; // is this the first call to SwitchMode? }; static inline Awb::RGB operator+(Awb::RGB const &a, Awb::RGB const &b) From patchwork Tue Dec 8 20:44:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10618 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BA8E6BDB21 for ; Tue, 8 Dec 2020 20:44:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 84EE267E6F; Tue, 8 Dec 2020 21:44:52 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Z3cgVBkH"; dkim-atps=neutral Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A918367F0B for ; Tue, 8 Dec 2020 21:44:50 +0100 (CET) Received: by mail-ed1-x52f.google.com with SMTP id q16so18975754edv.10 for ; Tue, 08 Dec 2020 12:44:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XdEmnit+rjT7DnH9yrl+C0JlXKdX3fb/QjsepWpG/tE=; b=Z3cgVBkHSUy9aY0kQDP+aifbzn7/IY8H8WpGujdVK69QZb7GaOZPqkN2lBwzinTGep MTc5VOsBpWLfutTqdwGlzGgqpOcbx99AB78ftQSHY8h2vAq65AZLBnuueXCD5IdJYppT PGNfM+F6qryNsW9d9TXkXQ5HDk2Nf/r6lsgFE/r5YMLJf4/AePxN84U3b9GXV8zpzUaY DsNZWJQfN+wzNTXOyrKplLX0hyH4bdlsIGoo8QJw/M6YDw0W2b/WyqzVz52kqQ9NpSmE rAHRyHWqdSUwUrUtCJuxLQj/JzXq6sxHUSsVvNRVriOOzRcN/N2OPIyyKVwJMHmJElD7 TuuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XdEmnit+rjT7DnH9yrl+C0JlXKdX3fb/QjsepWpG/tE=; b=mpAJT1bdT/3pElL27N60PE2YZmnWcjA8JSa4P+V6+AoUMGT3CXSjh8YsbXqCz1taoq B6bDQiAd5DvTCpchYwwC+ftX5OiO8qfTkJe5mqJamvWCCoAyqCliqxuBtaX+gwRJSDPq e1xRnNeW5aTU4UclPTb+Q2xFhXjt1IRSqJYRChDFr+79l8nq7nZIKLOyZyslgsqmx+Gl cNIbvu0lk2GZw26CWS4B7H7Hutd8hVbKACrkTfXhbhIH+pxCjUpUcHRl/CRNHBNzmr+a eppPd9wRfirLL4oNPoh/IeAMGN2voLMd10IhWS49ZUESEuP70xp1e3/iMiekJJx49Mvn gKeA== X-Gm-Message-State: AOAM5307Q4R6ESZkdtKkbAv0HE67G/+2iEqPQeRngUfRCjA9XEVW1swl kDRLodeIbWeW+u849Q8mDyXu+qw9cyV+tvXK X-Google-Smtp-Source: ABdhPJxJz/mT7ygHLT715uCKSlIAFp1G3ZmFKxLqUujpXs/hObqexjFLAGo/wpWwwmEYec8I/UQfNQ== X-Received: by 2002:a50:875b:: with SMTP id 27mr26588440edv.24.1607460290050; Tue, 08 Dec 2020 12:44:50 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id be6sm18441471edb.29.2020.12.08.12.44.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Dec 2020 12:44:49 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Tue, 8 Dec 2020 20:44:41 +0000 Message-Id: <20201208204441.9356-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201208204441.9356-1-david.plowman@raspberrypi.com> References: <20201208204441.9356-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 6/6] src: ipa: raspberrypi: Move initial frame drop decision to AGC/AWB X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Previously the CamHelper was returning the number of frames to drop (on account of AGC/AWB converging). This wasn't really appropriate, it's better for the algorithms to do it as they know how many frames they might need. The CamHelper::HideFramesStartup method should now just be returning the number of frames to hide because they're bad/invalid in some way, not worrying about the AGC/AWB. For many sensors, the correct value for this is zero. But the ov5647 needs updating as it must return 2. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/cam_helper.cpp | 6 ++--- src/ipa/raspberrypi/cam_helper_ov5647.cpp | 10 ++++++++ src/ipa/raspberrypi/raspberrypi.cpp | 28 +++++++++++++++++++++++ 3 files changed, 41 insertions(+), 3 deletions(-) diff --git a/src/ipa/raspberrypi/cam_helper.cpp b/src/ipa/raspberrypi/cam_helper.cpp index c8ac3232..6efa0d7f 100644 --- a/src/ipa/raspberrypi/cam_helper.cpp +++ b/src/ipa/raspberrypi/cam_helper.cpp @@ -82,10 +82,10 @@ bool CamHelper::SensorEmbeddedDataPresent() const unsigned int CamHelper::HideFramesStartup() const { /* - * By default, hide 6 frames completely at start-up while AGC etc. sort - * themselves out (converge). + * The number of frames when a camera first starts that shouldn't be + * displayed as they are invalid in some way. */ - return 6; + return 0; } unsigned int CamHelper::HideFramesModeSwitch() const diff --git a/src/ipa/raspberrypi/cam_helper_ov5647.cpp b/src/ipa/raspberrypi/cam_helper_ov5647.cpp index dc5d8275..0b841cd1 100644 --- a/src/ipa/raspberrypi/cam_helper_ov5647.cpp +++ b/src/ipa/raspberrypi/cam_helper_ov5647.cpp @@ -19,6 +19,7 @@ public: uint32_t GainCode(double gain) const override; double Gain(uint32_t gain_code) const override; void GetDelays(int &exposure_delay, int &gain_delay) const override; + unsigned int HideFramesStartup() const override; unsigned int HideFramesModeSwitch() const override; unsigned int MistrustFramesStartup() const override; unsigned int MistrustFramesModeSwitch() const override; @@ -54,6 +55,15 @@ void CamHelperOv5647::GetDelays(int &exposure_delay, int &gain_delay) const gain_delay = 2; } +unsigned int CamHelperOv5647::HideFramesStartup() const +{ + /* + * On startup, we get a couple of under-exposed frames which + * we don't want shown. + */ + return 2; +} + unsigned int CamHelperOv5647::HideFramesModeSwitch() const { /* diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index ba63480e..8dd796ed 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -194,6 +194,34 @@ int IPARPi::start(const IPAOperationData &data, IPAOperationData *result) if (firstStart_) { dropFrame = helper_->HideFramesStartup(); mistrustCount_ = helper_->MistrustFramesStartup(); + + /* + * Query the AGC/AWB for how many frames they may take to + * converge sufficiently. Where these numbers are non-zero + * we must allow for the frames with bad statistics + * (mistrustCount_) that they won't see. But if zero (i.e. + * no convergence necessary), no frames need to be dropped. + */ + unsigned int agcConvergenceFrames = 0; + RPiController::AgcAlgorithm *agc = dynamic_cast( + controller_.GetAlgorithm("agc")); + if (agc) { + agcConvergenceFrames = agc->GetConvergenceFrames(); + if (agcConvergenceFrames) + agcConvergenceFrames += mistrustCount_; + } + + unsigned int awbConvergenceFrames = 0; + RPiController::AwbAlgorithm *awb = dynamic_cast( + controller_.GetAlgorithm("awb")); + if (awb) { + awbConvergenceFrames = awb->GetConvergenceFrames(); + if (awbConvergenceFrames) + awbConvergenceFrames += mistrustCount_; + } + + dropFrame = std::max({ dropFrame, agcConvergenceFrames, awbConvergenceFrames }); + LOG(IPARPI, Debug) << "Drop " << dropFrame << " frames on startup"; } else { dropFrame = helper_->HideFramesModeSwitch(); mistrustCount_ = helper_->MistrustFramesModeSwitch();