From patchwork Mon Nov 23 07:37:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10460 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DF8C3BE08A for ; Mon, 23 Nov 2020 07:38:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 956C86332E; Mon, 23 Nov 2020 08:38:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="nmqyWuDu"; dkim-atps=neutral Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5E78163320 for ; Mon, 23 Nov 2020 08:38:31 +0100 (CET) Received: by mail-wr1-x432.google.com with SMTP id p8so17578082wrx.5 for ; Sun, 22 Nov 2020 23:38:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=u46CcECDOCNqmkIJKCIsZpcth4nWLgktLU0Jsd2RAiw=; b=nmqyWuDuca7SE4PrYHEMKsdmHIcof61/h9ocmJtQ18mDEESqNDGut6KNfp6ut90Gkz I7IoZ8ZSPO/XlDNjSE5+CvW1DbJYH8tp4dXuN0VZJghpA0I4Y6aHpfc/tBpEwxIUvD/2 M9SwUtTfFrv65ge4PQA8kGx8o84NgDl9105X9UIxpV1AWlgsPinedELvnbM/Znt2MYua vE/sTMu7G+9XL6EnyxC8yJF7BkOCcr78IjKvGPmRj3Sy8gSE19GgDHkHq7V/QlSJhVPk d7+2YZvA6KugLE8Rbqkhh7tW6wnrh0Gx+bM/tywmnMCfFRZ8+w3pkUC7/0AlIpwNfmoe 7q5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=u46CcECDOCNqmkIJKCIsZpcth4nWLgktLU0Jsd2RAiw=; b=iQiVa2nqGt7MuiOBywBnyu/7kcIa/XY1nCjtTe7u4rvW04mi8ZK1jcHLRgyJIFI3/P tOoegL8beE+w0/0TUTUMEzJBE5InJD8BgxtIljAQs9H7+acbl6XFs5gmmOZwx9CDNCth xYQA3Li4eFsLTtxryyDl5nw+1GE6Dw8I3zlnvkE7G0wOJTXyVXcYie8vHFiRdyngHyRS VG6/qX200J4S1HvBDriLbPgosO+vQUrAQ5Td2JCOYFZyqB339Ol7Uq20suhU3kyqZXlw YHYwSqyPs+LpRDgXK3WznTxFJCr8SIJ5D6ox/UjJ1DU3f4x2zZ3EM+AlQAdxaKz5u6TO WW3A== X-Gm-Message-State: AOAM532ZC7UuClSVSFPWlr+sSfzFjbHgb1PaSx1+pW6S01nqFy3G3q1r JPrytgxarJoFuBnjCSOEFuGisu1Ibo/9bKcF X-Google-Smtp-Source: ABdhPJw+18J+5cMXhCSFI+G9rob45GmtnNR/So1sGe1sIcEO/s5voLE6HR/d3bcOQlZHngQ7shZ56g== X-Received: by 2002:adf:814f:: with SMTP id 73mr29115330wrm.174.1606117110646; Sun, 22 Nov 2020 23:38:30 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id h15sm17841822wrw.15.2020.11.22.23.38.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Nov 2020 23:38:30 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 23 Nov 2020 07:37:55 +0000 Message-Id: <20201123073804.3125-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201123073804.3125-1-david.plowman@raspberrypi.com> References: <20201123073804.3125-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 01/10] libcamera: ipa: raspberrypi: agc: Use libcamera debug X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Replace Raspberry Pi debug with libcamera debug. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 90 +++++++++++----------- 1 file changed, 47 insertions(+), 43 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index df4d3647..8079345b 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -9,16 +9,20 @@ #include "linux/bcm2835-isp.h" +#include "libcamera/internal/log.h" + #include "../awb_status.h" #include "../device_status.h" #include "../histogram.hpp" -#include "../logging.hpp" #include "../lux_status.h" #include "../metadata.hpp" #include "agc.hpp" using namespace RPiController; +using namespace libcamera; + +LOG_DEFINE_CATEGORY(RPiAgc) #define NAME "rpi.agc" @@ -128,7 +132,7 @@ static std::string read_constraint_modes( void AgcConfig::Read(boost::property_tree::ptree const ¶ms) { - RPI_LOG("AgcConfig"); + LOG(RPiAgc, Debug) << "AgcConfig"; default_metering_mode = read_metering_modes( metering_modes, params.get_child("metering_modes")); default_exposure_mode = read_exposure_modes( @@ -166,7 +170,7 @@ char const *Agc::Name() const void Agc::Read(boost::property_tree::ptree const ¶ms) { - RPI_LOG("Agc"); + LOG(RPiAgc, Debug) << "Agc"; config_.Read(params); // Set the config's defaults (which are the first ones it read) as our // current modes, until someone changes them. (they're all known to @@ -254,15 +258,15 @@ void Agc::Prepare(Metadata *image_metadata) status.digital_gain = status_.total_exposure_value / actual_exposure; - RPI_LOG("Want total exposure " << status_.total_exposure_value); + LOG(RPiAgc, Debug) << "Want total exposure " << status_.total_exposure_value; // Never ask for a gain < 1.0, and also impose // some upper limit. Make it customisable? status.digital_gain = std::max( 1.0, std::min(status.digital_gain, 4.0)); - RPI_LOG("Actual exposure " << actual_exposure); - RPI_LOG("Use digital_gain " << status.digital_gain); - RPI_LOG("Effective exposure " << actual_exposure * status.digital_gain); + LOG(RPiAgc, Debug) << "Actual exposure " << actual_exposure; + LOG(RPiAgc, Debug) << "Use digital_gain " << status.digital_gain; + LOG(RPiAgc, Debug) << "Effective exposure " << actual_exposure * status.digital_gain; // Decide whether AEC/AGC has converged. // Insist AGC is steady for MAX_LOCK_COUNT // frames before we say we are "locked". @@ -285,11 +289,11 @@ void Agc::Prepare(Metadata *image_metadata) status.target_exposure_value - 1.5 * err) lock_count_ = lock_count; - RPI_LOG("Lock count: " << lock_count_); + LOG(RPiAgc, Debug) << "Lock count: " << lock_count_; } } } else - RPI_LOG(Name() << ": no device metadata"); + LOG(RPiAgc, Debug) << Name() << ": no device metadata"; status.locked = lock_count_ >= MAX_LOCK_COUNT; //printf("%s\n", status.locked ? "+++++++++" : "-"); image_metadata->Set("agc.status", status); @@ -343,9 +347,9 @@ void Agc::housekeepConfig() status_.fixed_analogue_gain = fixed_analogue_gain_; status_.flicker_period = flicker_period_; } - RPI_LOG("ev " << status_.ev << " fixed_shutter " - << status_.fixed_shutter << " fixed_analogue_gain " - << status_.fixed_analogue_gain); + LOG(RPiAgc, Debug) << "ev " << status_.ev << " fixed_shutter " + << status_.fixed_shutter << " fixed_analogue_gain " + << status_.fixed_analogue_gain; // Make sure the "mode" pointers point to the up-to-date things, if // they've changed. if (strcmp(new_metering_mode_name.c_str(), status_.metering_mode)) { @@ -376,10 +380,10 @@ void Agc::housekeepConfig() copy_string(new_constraint_mode_name, status_.constraint_mode, sizeof(status_.constraint_mode)); } - RPI_LOG("exposure_mode " - << new_exposure_mode_name << " constraint_mode " - << new_constraint_mode_name << " metering_mode " - << new_metering_mode_name); + LOG(RPiAgc, Debug) << "exposure_mode " + << new_exposure_mode_name << " constraint_mode " + << new_constraint_mode_name << " metering_mode " + << new_metering_mode_name; } void Agc::fetchCurrentExposure(Metadata *image_metadata) @@ -404,7 +408,7 @@ static double compute_initial_Y(bcm2835_isp_stats *stats, Metadata *image_metada struct AwbStatus awb; awb.gain_r = awb.gain_g = awb.gain_b = 1.0; // in case no metadata if (image_metadata->Get("awb.status", awb) != 0) - RPI_WARN("Agc: no AWB status found"); + LOG(RPiAgc, Warning) << "Agc: no AWB status found"; double Y_sum = 0, weight_sum = 0; for (int i = 0; i < AGC_STATS_SIZE; i++) { if (regions[i].counted == 0) @@ -443,7 +447,7 @@ void Agc::computeGain(bcm2835_isp_stats *statistics, Metadata *image_metadata, struct LuxStatus lux = {}; lux.lux = 400; // default lux level to 400 in case no metadata found if (image_metadata->Get("lux.status", lux) != 0) - RPI_WARN("Agc: no lux level found"); + LOG(RPiAgc, Warning) << "Agc: no lux level found"; Histogram h(statistics->hist[0].g_hist, NUM_HISTOGRAM_BINS); double ev_gain = status_.ev * config_.base_ev; // The initial gain and target_Y come from some of the regions. After @@ -454,28 +458,28 @@ void Agc::computeGain(bcm2835_isp_stats *statistics, Metadata *image_metadata, double initial_Y = compute_initial_Y(statistics, image_metadata, metering_mode_->weights); gain = std::min(10.0, target_Y / (initial_Y + .001)); - RPI_LOG("Initially Y " << initial_Y << " target " << target_Y - << " gives gain " << gain); + LOG(RPiAgc, Debug) << "Initially Y " << initial_Y << " target " << target_Y + << " gives gain " << gain; for (auto &c : *constraint_mode_) { double new_target_Y; double new_gain = constraint_compute_gain(c, h, lux.lux, ev_gain, new_target_Y); - RPI_LOG("Constraint has target_Y " - << new_target_Y << " giving gain " << new_gain); + LOG(RPiAgc, Debug) << "Constraint has target_Y " + << new_target_Y << " giving gain " << new_gain; if (c.bound == AgcConstraint::Bound::LOWER && new_gain > gain) { - RPI_LOG("Lower bound constraint adopted"); + LOG(RPiAgc, Debug) << "Lower bound constraint adopted"; gain = new_gain, target_Y = new_target_Y; } else if (c.bound == AgcConstraint::Bound::UPPER && new_gain < gain) { - RPI_LOG("Upper bound constraint adopted"); + LOG(RPiAgc, Debug) << "Upper bound constraint adopted"; gain = new_gain, target_Y = new_target_Y; } } - RPI_LOG("Final gain " << gain << " (target_Y " << target_Y << " ev " - << status_.ev << " base_ev " << config_.base_ev - << ")"); + LOG(RPiAgc, Debug) << "Final gain " << gain << " (target_Y " << target_Y << " ev " + << status_.ev << " base_ev " << config_.base_ev + << ")"; } void Agc::computeTargetExposure(double gain) @@ -494,7 +498,7 @@ void Agc::computeTargetExposure(double gain) : exposure_mode_->gain.back()); target_.total_exposure = std::min(target_.total_exposure, max_total_exposure); - RPI_LOG("Target total_exposure " << target_.total_exposure); + LOG(RPiAgc, Debug) << "Target total_exposure " << target_.total_exposure; } bool Agc::applyDigitalGain(Metadata *image_metadata, double gain, @@ -509,9 +513,9 @@ bool Agc::applyDigitalGain(Metadata *image_metadata, double gain, std::min(awb.gain_g, awb.gain_b)); dg *= std::max(1.0, 1.0 / min_gain); } else - RPI_WARN("Agc: no AWB status found"); - RPI_LOG("after AWB, target dg " << dg << " gain " << gain - << " target_Y " << target_Y); + LOG(RPiAgc, Warning) << "Agc: no AWB status found"; + LOG(RPiAgc, Debug) << "after AWB, target dg " << dg << " gain " << gain + << " target_Y " << target_Y; // Finally, if we're trying to reduce exposure but the target_Y is // "close" to 1.0, then the gain computed for that constraint will be // only slightly less than one, because the measured Y can never be @@ -523,9 +527,9 @@ bool Agc::applyDigitalGain(Metadata *image_metadata, double gain, gain < sqrt(target_Y); if (desaturate) dg /= config_.fast_reduce_threshold; - RPI_LOG("Digital gain " << dg << " desaturate? " << desaturate); + LOG(RPiAgc, Debug) << "Digital gain " << dg << " desaturate? " << desaturate; target_.total_exposure_no_dg = target_.total_exposure / dg; - RPI_LOG("Target total_exposure_no_dg " << target_.total_exposure_no_dg); + LOG(RPiAgc, Debug) << "Target total_exposure_no_dg " << target_.total_exposure_no_dg; return desaturate; } @@ -560,8 +564,8 @@ void Agc::filterExposure(bool desaturate) filtered_.total_exposure * config_.fast_reduce_threshold) filtered_.total_exposure_no_dg = filtered_.total_exposure * config_.fast_reduce_threshold; - RPI_LOG("After filtering, total_exposure " << filtered_.total_exposure << - " no dg " << filtered_.total_exposure_no_dg); + LOG(RPiAgc, Debug) << "After filtering, total_exposure " << filtered_.total_exposure + << " no dg " << filtered_.total_exposure_no_dg; } void Agc::divvyupExposure() @@ -602,8 +606,8 @@ void Agc::divvyupExposure() } } } - RPI_LOG("Divided up shutter and gain are " << shutter_time << " and " - << analogue_gain); + LOG(RPiAgc, Debug) << "Divided up shutter and gain are " << shutter_time << " and " + << analogue_gain; // Finally adjust shutter time for flicker avoidance (require both // shutter and gain not to be fixed). if (status_.fixed_shutter == 0.0 && @@ -621,8 +625,8 @@ void Agc::divvyupExposure() exposure_mode_->gain.back()); shutter_time = new_shutter_time; } - RPI_LOG("After flicker avoidance, shutter " - << shutter_time << " gain " << analogue_gain); + LOG(RPiAgc, Debug) << "After flicker avoidance, shutter " + << shutter_time << " gain " << analogue_gain; } filtered_.shutter = shutter_time; filtered_.analogue_gain = analogue_gain; @@ -641,10 +645,10 @@ void Agc::writeAndFinish(Metadata *image_metadata, bool desaturate) // Write to metadata as well, in case anyone wants to update the camera // immediately. image_metadata->Set("agc.status", status_); - RPI_LOG("Output written, total exposure requested is " - << filtered_.total_exposure); - RPI_LOG("Camera exposure update: shutter time " << filtered_.shutter << - " analogue gain " << filtered_.analogue_gain); + LOG(RPiAgc, Debug) << "Output written, total exposure requested is " + << filtered_.total_exposure; + LOG(RPiAgc, Debug) << "Camera exposure update: shutter time " << filtered_.shutter + << " analogue gain " << filtered_.analogue_gain; } // Register algorithm with the system. From patchwork Mon Nov 23 07:37:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10461 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CB33ABE08A for ; Mon, 23 Nov 2020 07:38:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3D26F63344; Mon, 23 Nov 2020 08:38:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="iXiHAFTC"; dkim-atps=neutral Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 55B5760332 for ; Mon, 23 Nov 2020 08:38:32 +0100 (CET) Received: by mail-wr1-x436.google.com with SMTP id b6so17594950wrt.4 for ; Sun, 22 Nov 2020 23:38:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZHnJxWPMbxEB7oCKged3AtYHg1uDAslixQCdBCXVkrU=; b=iXiHAFTCy8uEyPgsrLqsVMEcoB4tF5aNhQREWnXrJFdxV+LbkyT8kmJGUBJJaAQhwy VuiMaetnmTlHfPfBx0kRg/CYGA7A4W8XIn+S/7RkXgmigQH1AAUMjWuWyZ1xRigS47q5 hbqLH3hSYbDcdM9cEHChv2o3l1LyMhpBhlYMjYjhXui4ScFF8czqPIKErqMaMcVoB5ZP H2K3SVGvZh/pBtP0L8FJyyt2hoMQI/JuafoA1xjA0i/v8q9Y5p6bjlXVHk2bpzq0jMZH Pru7ySUeT9ykHAGJHRqfKcgCEH5kGbldA/DUa7aydGgl8rNB5eYt/sSgXmM/7NuskHsZ iiEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZHnJxWPMbxEB7oCKged3AtYHg1uDAslixQCdBCXVkrU=; b=Ry98rNv1wpxYXHiZXuQf2k9Ha7KD1WjCVAxfYcXs41IsIpXDOr8qSHYnAKJh+DY/gG jFKP4g5KHBgm5JGxxK44mGWa9L3DXp+zwX4TD2mdk/MeDiObD7G5UnKEMCUSIZEi0GXv BF2KJqqPDappuRLOjHbAvuazB8P3nLffnsjUDv3fmZtqgpWZ3Uwc4GeZ9SRh1TIbsxvP +Rrlq1RU+JYc4NLUZ3jdunR34RkiRIC/m8SCyKVKL+/4JnpXrYzYf/XOu2I+sx7GSkgD CIvlzKeCJiYDoY4wJZj9WzTY4lL2Y5JJiTfDCKkgUkKmPsN8uIlCBEV9e7KfdEzu5t0i 25/A== X-Gm-Message-State: AOAM533N/TULYAzoADg3SyQHR7ovxL5CkndUmF9GMHUJn/EJ+TG2AXPh 1ntGD4NEy3cOc1z9tHZv9UcMS2474WHo8dmf X-Google-Smtp-Source: ABdhPJzAUFAkk4ZqE+eza0Iz6gx3shiBrAzh/r/GR7FLS2HvvF7uf+U4FPUFSydfJzeSJlFt2lObXw== X-Received: by 2002:a05:6000:108:: with SMTP id o8mr28194536wrx.337.1606117111649; Sun, 22 Nov 2020 23:38:31 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id h15sm17841822wrw.15.2020.11.22.23.38.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Nov 2020 23:38:31 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 23 Nov 2020 07:37:56 +0000 Message-Id: <20201123073804.3125-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201123073804.3125-1-david.plowman@raspberrypi.com> References: <20201123073804.3125-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 02/10] libcamera: ipa: raspberrypi: agc: Remove unnecessary locking X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" On the libcamera/VC4 platform the AGC Prepare/Process methods, and any changes to the AGC settings, run synchronously - so a number of mutexes and copies are unnecessary and can be removed. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 98 ++++++++-------------- src/ipa/raspberrypi/controller/rpi/agc.hpp | 5 +- 2 files changed, 36 insertions(+), 67 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 8079345b..0a67f456 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -160,7 +160,6 @@ Agc::Agc(Controller *controller) status_.total_exposure_value = 0.0; status_.target_exposure_value = 0.0; status_.locked = false; - output_status_ = status_; } char const *Agc::Name() const @@ -185,43 +184,36 @@ void Agc::Read(boost::property_tree::ptree const ¶ms) void Agc::SetEv(double ev) { - std::unique_lock lock(settings_mutex_); ev_ = ev; } void Agc::SetFlickerPeriod(double flicker_period) { - std::unique_lock lock(settings_mutex_); flicker_period_ = flicker_period; } void Agc::SetFixedShutter(double fixed_shutter) { - std::unique_lock lock(settings_mutex_); fixed_shutter_ = fixed_shutter; } void Agc::SetFixedAnalogueGain(double fixed_analogue_gain) { - std::unique_lock lock(settings_mutex_); fixed_analogue_gain_ = fixed_analogue_gain; } void Agc::SetMeteringMode(std::string const &metering_mode_name) { - std::unique_lock lock(settings_mutex_); metering_mode_name_ = metering_mode_name; } void Agc::SetExposureMode(std::string const &exposure_mode_name) { - std::unique_lock lock(settings_mutex_); exposure_mode_name_ = exposure_mode_name; } void Agc::SetConstraintMode(std::string const &constraint_mode_name) { - std::unique_lock lock(settings_mutex_); constraint_mode_name_ = constraint_mode_name; } @@ -240,14 +232,9 @@ void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, void Agc::Prepare(Metadata *image_metadata) { - AgcStatus status; - { - std::unique_lock lock(output_mutex_); - status = output_status_; - } int lock_count = lock_count_; lock_count_ = 0; - status.digital_gain = 1.0; + status_.digital_gain = 1.0; if (status_.total_exposure_value) { // Process has run, so we have meaningful values. DeviceStatus device_status; @@ -255,48 +242,45 @@ void Agc::Prepare(Metadata *image_metadata) double actual_exposure = device_status.shutter_speed * device_status.analogue_gain; if (actual_exposure) { - status.digital_gain = + status_.digital_gain = status_.total_exposure_value / actual_exposure; LOG(RPiAgc, Debug) << "Want total exposure " << status_.total_exposure_value; // Never ask for a gain < 1.0, and also impose // some upper limit. Make it customisable? - status.digital_gain = std::max( + status_.digital_gain = std::max( 1.0, - std::min(status.digital_gain, 4.0)); + std::min(status_.digital_gain, 4.0)); LOG(RPiAgc, Debug) << "Actual exposure " << actual_exposure; - LOG(RPiAgc, Debug) << "Use digital_gain " << status.digital_gain; - LOG(RPiAgc, Debug) << "Effective exposure " << actual_exposure * status.digital_gain; + LOG(RPiAgc, Debug) << "Use digital_gain " << status_.digital_gain; + LOG(RPiAgc, Debug) << "Effective exposure " << actual_exposure * status_.digital_gain; // Decide whether AEC/AGC has converged. // Insist AGC is steady for MAX_LOCK_COUNT // frames before we say we are "locked". // (The hard-coded constants may need to // become customisable.) - if (status.target_exposure_value) { + if (status_.target_exposure_value) { #define MAX_LOCK_COUNT 3 - double err = 0.10 * status.target_exposure_value + 200; + double err = 0.10 * status_.target_exposure_value + 200; if (actual_exposure < - status.target_exposure_value + err - && actual_exposure > - status.target_exposure_value - err) + status_.target_exposure_value + err && + actual_exposure > + status_.target_exposure_value - err) lock_count_ = std::min(lock_count + 1, - MAX_LOCK_COUNT); + MAX_LOCK_COUNT); else if (actual_exposure < - status.target_exposure_value - + 1.5 * err && + status_.target_exposure_value + 1.5 * err && actual_exposure > - status.target_exposure_value - - 1.5 * err) + status_.target_exposure_value - 1.5 * err) lock_count_ = lock_count; LOG(RPiAgc, Debug) << "Lock count: " << lock_count_; } } } else LOG(RPiAgc, Debug) << Name() << ": no device metadata"; - status.locked = lock_count_ >= MAX_LOCK_COUNT; - //printf("%s\n", status.locked ? "+++++++++" : "-"); - image_metadata->Set("agc.status", status); + status_.locked = lock_count_ >= MAX_LOCK_COUNT; + image_metadata->Set("agc.status", status_); } } @@ -335,55 +319,47 @@ static void copy_string(std::string const &s, char *d, size_t size) void Agc::housekeepConfig() { // First fetch all the up-to-date settings, so no one else has to do it. - std::string new_exposure_mode_name, new_constraint_mode_name, - new_metering_mode_name; - { - std::unique_lock lock(settings_mutex_); - new_metering_mode_name = metering_mode_name_; - new_exposure_mode_name = exposure_mode_name_; - new_constraint_mode_name = constraint_mode_name_; - status_.ev = ev_; - status_.fixed_shutter = fixed_shutter_; - status_.fixed_analogue_gain = fixed_analogue_gain_; - status_.flicker_period = flicker_period_; - } + status_.ev = ev_; + status_.fixed_shutter = fixed_shutter_; + status_.fixed_analogue_gain = fixed_analogue_gain_; + status_.flicker_period = flicker_period_; LOG(RPiAgc, Debug) << "ev " << status_.ev << " fixed_shutter " << status_.fixed_shutter << " fixed_analogue_gain " << status_.fixed_analogue_gain; // Make sure the "mode" pointers point to the up-to-date things, if // they've changed. - if (strcmp(new_metering_mode_name.c_str(), status_.metering_mode)) { - auto it = config_.metering_modes.find(new_metering_mode_name); + if (strcmp(metering_mode_name_.c_str(), status_.metering_mode)) { + auto it = config_.metering_modes.find(metering_mode_name_); if (it == config_.metering_modes.end()) throw std::runtime_error("Agc: no metering mode " + - new_metering_mode_name); + metering_mode_name_); metering_mode_ = &it->second; - copy_string(new_metering_mode_name, status_.metering_mode, + copy_string(metering_mode_name_, status_.metering_mode, sizeof(status_.metering_mode)); } - if (strcmp(new_exposure_mode_name.c_str(), status_.exposure_mode)) { - auto it = config_.exposure_modes.find(new_exposure_mode_name); + if (strcmp(exposure_mode_name_.c_str(), status_.exposure_mode)) { + auto it = config_.exposure_modes.find(exposure_mode_name_); if (it == config_.exposure_modes.end()) throw std::runtime_error("Agc: no exposure profile " + - new_exposure_mode_name); + exposure_mode_name_); exposure_mode_ = &it->second; - copy_string(new_exposure_mode_name, status_.exposure_mode, + copy_string(exposure_mode_name_, status_.exposure_mode, sizeof(status_.exposure_mode)); } - if (strcmp(new_constraint_mode_name.c_str(), status_.constraint_mode)) { + if (strcmp(constraint_mode_name_.c_str(), status_.constraint_mode)) { auto it = - config_.constraint_modes.find(new_constraint_mode_name); + config_.constraint_modes.find(constraint_mode_name_); if (it == config_.constraint_modes.end()) throw std::runtime_error("Agc: no constraint list " + - new_constraint_mode_name); + constraint_mode_name_); constraint_mode_ = &it->second; - copy_string(new_constraint_mode_name, status_.constraint_mode, + copy_string(constraint_mode_name_, status_.constraint_mode, sizeof(status_.constraint_mode)); } LOG(RPiAgc, Debug) << "exposure_mode " - << new_exposure_mode_name << " constraint_mode " - << new_constraint_mode_name << " metering_mode " - << new_metering_mode_name; + << exposure_mode_name_ << " constraint_mode " + << constraint_mode_name_ << " metering_mode " + << metering_mode_name_; } void Agc::fetchCurrentExposure(Metadata *image_metadata) @@ -638,10 +614,6 @@ void Agc::writeAndFinish(Metadata *image_metadata, bool desaturate) status_.target_exposure_value = desaturate ? 0 : target_.total_exposure_no_dg; status_.shutter_time = filtered_.shutter; status_.analogue_gain = filtered_.analogue_gain; - { - std::unique_lock lock(output_mutex_); - output_status_ = status_; - } // Write to metadata as well, in case anyone wants to update the camera // immediately. image_metadata->Set("agc.status", status_); diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index ba7ae092..5a02df4e 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -106,12 +106,9 @@ private: ExposureValues current_; // values for the current frame ExposureValues target_; // calculate the values we want here ExposureValues filtered_; // these values are filtered towards target - AgcStatus status_; // to "latch" settings so they can't change - AgcStatus output_status_; // the status we will write out - std::mutex output_mutex_; + AgcStatus status_; int lock_count_; // Below here the "settings" that applications can change. - std::mutex settings_mutex_; std::string metering_mode_name_; std::string exposure_mode_name_; std::string constraint_mode_name_; From patchwork Mon Nov 23 07:37:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10462 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8998BBE08A for ; Mon, 23 Nov 2020 07:38:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BD3B56332A; Mon, 23 Nov 2020 08:38:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="eqFBNgqC"; dkim-atps=neutral Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3DFB1632B4 for ; Mon, 23 Nov 2020 08:38:33 +0100 (CET) Received: by mail-wr1-x42a.google.com with SMTP id e7so41598wrv.6 for ; Sun, 22 Nov 2020 23:38:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=67Ttd9klxjHbW25ZV2IOb9yHFsznpt0Ki1kGRGQT2Ok=; b=eqFBNgqCCnfZtlu5pMn1pe/NhVfpnJ+4ZtCcpg8ThtHabHKfVnSi1OeDz/JUl6k5Cu Gp7g03sA6Ao0LObj68kbdIGgHazYa2VbsCWC2u+srCxq71Kfx9Ff38O2UlSQlOyY0NKX xJoIHYokfg26/hIEdfeHVsxLuYu9jbnroheeJPFuWe0WcbjvMXI07vulxID2bq2qgz91 lSKY2/mVotXFHgGoFJL0xFEtO6mUjqjVaH9He72SDwMm3E8vWIVKRfbxONmNICWYHA1N IU1Ulued8viG6POPDa3FVlagxAG4siUp2ukSQAONRHrFKv6+1zvDdyaXPJMO7lAMBVN0 pTug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=67Ttd9klxjHbW25ZV2IOb9yHFsznpt0Ki1kGRGQT2Ok=; b=q7oj08AcKr0CxwzWWEaRz3+xOdlDMsplU5VILJS8audOTCLOOH9RZA/wrLWLI7JUhC IcT/5xcKCGdWh5tjjJ8O/0k35UZR1XSnisPLBPyQvZze4aNcI3FG1JACHbSGDE6LI5iv QV3lMQGOP7Ce9d1bQ3ezWgyu1xJV6EquwxDvqpgU7bK/Y1rylho54137X8ZbUv4p4QjM xrZFIwFJloBxJHj5YK7d0hePf2qfRQnLAhI9hMId+43emXIGM+vslbANA5FZ+5Nkws36 e6qoAdmvQuNfGaVW+NtGs0oFqUlZF5CvRVMeWf0nDFd2rvVQjY0pcm/5rUAMrtNoKKEr Vysw== X-Gm-Message-State: AOAM5316I+1h3X0yQyA2bEKhOCS/WjYdwJbvBNqwZ2SvhKyKBBJAb26s 68L/GdGcgFfIV1q91pCG71AYcn6XGSISHElI X-Google-Smtp-Source: ABdhPJxEKPgiIy/VRNyFt9MippLBFDqf/TqZfav85MmXdrTy/Jfg8sMx/p646QM32y9J/+cmABmJbQ== X-Received: by 2002:a5d:4acf:: with SMTP id y15mr21377119wrs.333.1606117112726; Sun, 22 Nov 2020 23:38:32 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id h15sm17841822wrw.15.2020.11.22.23.38.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Nov 2020 23:38:32 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 23 Nov 2020 07:37:57 +0000 Message-Id: <20201123073804.3125-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201123073804.3125-1-david.plowman@raspberrypi.com> References: <20201123073804.3125-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 03/10] libcamera: ipa: raspberrypi: agc: Rename method to divideUpExposure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The method formerly known as divvyupExposure is given a more understandable name. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 8 ++++---- src/ipa/raspberrypi/controller/rpi/agc.hpp | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 0a67f456..9a5d84f7 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -225,7 +225,7 @@ void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, // write the results into the metadata we've been given. if (status_.total_exposure_value) { housekeepConfig(); - divvyupExposure(); + divideUpExposure(); writeAndFinish(metadata, false); } } @@ -303,9 +303,9 @@ void Agc::Process(StatisticsPtr &stats, Metadata *image_metadata) bool desaturate = applyDigitalGain(image_metadata, gain, target_Y); // The results have to be filtered so as not to change too rapidly. filterExposure(desaturate); - // The last thing is to divvy up the exposure value into a shutter time + // The last thing is to divide up the exposure value into a shutter time // and analogue_gain, according to the current exposure mode. - divvyupExposure(); + divideUpExposure(); // Finally advertise what we've done. writeAndFinish(image_metadata, desaturate); } @@ -544,7 +544,7 @@ void Agc::filterExposure(bool desaturate) << " no dg " << filtered_.total_exposure_no_dg; } -void Agc::divvyupExposure() +void Agc::divideUpExposure() { // Sending the fixed shutter/gain cases through the same code may seem // unnecessary, but it will make more sense when extend this to cover diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index 5a02df4e..2442fc03 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -89,7 +89,7 @@ private: bool applyDigitalGain(Metadata *image_metadata, double gain, double target_Y); void filterExposure(bool desaturate); - void divvyupExposure(); + void divideUpExposure(); void writeAndFinish(Metadata *image_metadata, bool desaturate); AgcMeteringMode *metering_mode_; AgcExposureMode *exposure_mode_; From patchwork Mon Nov 23 07:37:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10463 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 48142BE08A for ; Mon, 23 Nov 2020 07:38:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1110263320; Mon, 23 Nov 2020 08:38:37 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="rxN2a/78"; dkim-atps=neutral Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3A4A963327 for ; Mon, 23 Nov 2020 08:38:34 +0100 (CET) Received: by mail-wr1-x431.google.com with SMTP id r17so17621935wrw.1 for ; Sun, 22 Nov 2020 23:38:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QATcowDQnYv31Wl12U+LjUBs2LFP1uTTOOAJsc0CaZk=; b=rxN2a/7885WjDU4E+/cuKRoaa3F0Hz4WJOZQ4mgH4y/cMTOrZys8po5EY5+XEANrPU mqmqZU+6m9xYan4rTRZZNsqduuQ47gzs3GFYH//s7kyBzczeZUedx5LTDWhhHCZvacDg papNWPF6xJwKLcVwn85l4ixeFJQtmA5Dul1FCl0seRmEVIwcQ6Lq5Sz0wUrLqRlPeQ3M 9Q1T30GvI7l2EiM1uV0lXdILa3gyNBkyfDg/xRVDVgsghmqWR+MYbXwK72wmHzwrHAW9 Kbt1zuvdRVir0UOeuECd/IZ/ykZssmJiRL1IGwox1YEIptJDwWFA7jYPub/q0CxTwcvm HHDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QATcowDQnYv31Wl12U+LjUBs2LFP1uTTOOAJsc0CaZk=; b=SYhIYnBQ2ZWYoOkn8zvo1AEJtuPOP+eeOEB+0r8/Ok2soRBHIj1Bg7+hVopwj6AkKJ P6szz/vuHiyFenw2ez+ELw+xqtloVqf8YV3s7iuHQsiB9j5611oJiztYcclI0pQ5ShE3 77YbJ+HaJnT3KfZEFqyU0yaMuYXa5H6H+6R1ykXP0E8HMhqU7Fe6I1pGZpkSEdVlv2Ci mEgh1rq+2uX+xzBbYqZeYkCpaX82iIDIq1P6Ut5OboeiOSLZmmpDcOksclNa8qteOCfC fHq/RyGuoXN+hsEdR6eFk6rVokkUK9yWxHUUKBFLmuhB7whermXPeAczoDau7Q8Oe4U/ 4Fjg== X-Gm-Message-State: AOAM530RLHstQq+O0F5tcIUCqcgGvB4WtCLijqyFriAqPagdFadKuc/m yByrJiVMfiNy0QXwvkj4Rz7Te6WAEh0SK2bA X-Google-Smtp-Source: ABdhPJxtTz78BBXcL1pf6MAVI2HT0pWVTG8gxgVd3lQu/A9QMWdae96iurU91PdWmruCs7fyMDECEw== X-Received: by 2002:adf:c452:: with SMTP id a18mr30006592wrg.189.1606117113639; Sun, 22 Nov 2020 23:38:33 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id h15sm17841822wrw.15.2020.11.22.23.38.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Nov 2020 23:38:33 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 23 Nov 2020 07:37:58 +0000 Message-Id: <20201123073804.3125-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201123073804.3125-1-david.plowman@raspberrypi.com> References: <20201123073804.3125-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 04/10] libcamera: ipa: raspberrypi: agc: Improve centre-weighted luminance calucation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Previously the calculation computed Y for each region before returning the weighted average, which "baked in" the over-importance of small statistics regions. The revised calculation will treat all pixels equally when the region weights are the same, making it easier to use. With the previous scheme, proper "average" metering was difficult to implement. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 25 +++++++++++++--------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 9a5d84f7..f0c70a0a 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -385,18 +385,23 @@ static double compute_initial_Y(bcm2835_isp_stats *stats, Metadata *image_metada awb.gain_r = awb.gain_g = awb.gain_b = 1.0; // in case no metadata if (image_metadata->Get("awb.status", awb) != 0) LOG(RPiAgc, Warning) << "Agc: no AWB status found"; - double Y_sum = 0, weight_sum = 0; + // Note how the calculation below means that equal weights give you + // "average" metering (i.e. all pixels equally important). + double R_sum = 0, G_sum = 0, B_sum = 0, pixel_sum = 0; for (int i = 0; i < AGC_STATS_SIZE; i++) { - if (regions[i].counted == 0) - continue; - weight_sum += weights[i]; - double Y = regions[i].r_sum * awb.gain_r * .299 + - regions[i].g_sum * awb.gain_g * .587 + - regions[i].b_sum * awb.gain_b * .114; - Y /= regions[i].counted; - Y_sum += Y * weights[i]; + R_sum += regions[i].r_sum * weights[i]; + G_sum += regions[i].g_sum * weights[i]; + B_sum += regions[i].b_sum * weights[i]; + pixel_sum += regions[i].counted * weights[i]; } - return Y_sum / weight_sum / (1 << PIPELINE_BITS); + if (pixel_sum == 0.0) { + LOG(RPiAgc, Warning) << "compute_initial_Y: pixel_sum is zero"; + return 0; + } + double Y_sum = R_sum * awb.gain_r * .299 + + G_sum * awb.gain_g * .587 + + B_sum * awb.gain_b * .114; + return Y_sum / pixel_sum / (1 << PIPELINE_BITS); } // We handle extra gain through EV by adjusting our Y targets. However, you From patchwork Mon Nov 23 07:37:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10464 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E0683BE08A for ; Mon, 23 Nov 2020 07:38:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 91BFC632EE; Mon, 23 Nov 2020 08:38:37 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="GSQ6mQRn"; dkim-atps=neutral Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 230C760332 for ; Mon, 23 Nov 2020 08:38:35 +0100 (CET) Received: by mail-wr1-x434.google.com with SMTP id l1so17534141wrb.9 for ; Sun, 22 Nov 2020 23:38:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=V8Nsaokvnt+3p3VR7LiNfq4cMjAbzSycTM1hXK3GrGw=; b=GSQ6mQRncupg35/N1SMZdUxl0Dtl1n5vsMLy2Kw8sPspXOc/MK16OlUWrkzcwN4hBw xeGNntViTck3j+ENpyuEe12cEwVTJTZpJCYKw7dwLYnJsNW/aRboi2WcBAKc8eVDTRPQ 1HJOkv9qEyPoqh9KEotPwH45PYo3U2q9d/o1lkrHYo56BrrjWRC743ihWssLWKJJZmZf SMvoCpXY6EBu3WOZtXmAfyoH+cB35111uUxQOMVZ771x7ZD9v0YGIl1t9AokO3aV2OfH Y3yvsOjgLELqGNb3p8V9qAYIqiIO8RE1hC6tmxgB98P4nTMThztrd3HQFidKa5ODj+Q+ O/YQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=V8Nsaokvnt+3p3VR7LiNfq4cMjAbzSycTM1hXK3GrGw=; b=W2S6rDJHwLwn49hQC0ccg/h87HwLMGNnIfa79psyPajvXCBfnCe023MbCvZ/6Nk9Mb dkAOQUo+jbJP4OpbfKuh8pHot89n/D3/brC/pA4oAqr+2dYGUCn1wPIDPQ5p7Ra75PMv YlfJo79o3iLarLgM7viYKf89k0jSFUXsjNOAPZTQe8045PFD4KnOUKHPhZWrJS1YSyLq oaoZQjefUSpG00+eZvISvZTYo1JPfnk73ajiwkjALYJ81EBROtxRZgjfef3PWMBIMChL Atnz7r6zWx0FLd6cxkhVrF/YmFjFsGLVzwoOw9wDtuvV1mUK4e5PDApCYjq8CXKpuOR1 UsoA== X-Gm-Message-State: AOAM533fTB3B1+Pkt5kq7R4dH6kgJI1bQ8Hg+Tn6UWFgLCMG3udkGsIY NoKtiR5SeyCfJU12yJczYpejh4CIrGjIog26 X-Google-Smtp-Source: ABdhPJwjXCMsbm0hPZwqQLY5NgPXcOY3OFyNdvm7Vhw0hCmN3CLVTkF6EpwbTSLSw+T0Lg0B0rIhXA== X-Received: by 2002:a5d:69d1:: with SMTP id s17mr30048175wrw.104.1606117114547; Sun, 22 Nov 2020 23:38:34 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id h15sm17841822wrw.15.2020.11.22.23.38.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Nov 2020 23:38:34 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 23 Nov 2020 07:37:59 +0000 Message-Id: <20201123073804.3125-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201123073804.3125-1-david.plowman@raspberrypi.com> References: <20201123073804.3125-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 05/10] libcamera: ipa: raspberrypi: agc: Fetch AWB status only once X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Introduce a function to fetch the AwbStatus (fetchAwbStatus), and call it unconditionally at the top of Prepare so that both Prepare and Process know thereafter that it's been done. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 35 +++++++++++----------- src/ipa/raspberrypi/controller/rpi/agc.hpp | 5 ++-- 2 files changed, 21 insertions(+), 19 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index f0c70a0a..4cf98e66 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -235,6 +235,8 @@ void Agc::Prepare(Metadata *image_metadata) int lock_count = lock_count_; lock_count_ = 0; status_.digital_gain = 1.0; + fetchAwbStatus(image_metadata); // always fetch it so that Process knows it's been done + if (status_.total_exposure_value) { // Process has run, so we have meaningful values. DeviceStatus device_status; @@ -300,7 +302,7 @@ void Agc::Process(StatisticsPtr &stats, Metadata *image_metadata) // Some of the exposure has to be applied as digital gain, so work out // what that is. This function also tells us whether it's decided to // "desaturate" the image more quickly. - bool desaturate = applyDigitalGain(image_metadata, gain, target_Y); + bool desaturate = applyDigitalGain(gain, target_Y); // The results have to be filtered so as not to change too rapidly. filterExposure(desaturate); // The last thing is to divide up the exposure value into a shutter time @@ -377,14 +379,19 @@ void Agc::fetchCurrentExposure(Metadata *image_metadata) current_.total_exposure_no_dg = current_.shutter * current_.analogue_gain; } -static double compute_initial_Y(bcm2835_isp_stats *stats, Metadata *image_metadata, +void Agc::fetchAwbStatus(Metadata *image_metadata) +{ + awb_.gain_r = 1.0; // in case not found in metadata + awb_.gain_g = 1.0; + awb_.gain_b = 1.0; + if (image_metadata->Get("awb.status", awb_) != 0) + LOG(RPiAgc, Warning) << "Agc: no AWB status found"; +} + +static double compute_initial_Y(bcm2835_isp_stats *stats, AwbStatus const &awb, double weights[]) { bcm2835_isp_stats_region *regions = stats->agc_stats; - struct AwbStatus awb; - awb.gain_r = awb.gain_g = awb.gain_b = 1.0; // in case no metadata - if (image_metadata->Get("awb.status", awb) != 0) - LOG(RPiAgc, Warning) << "Agc: no AWB status found"; // Note how the calculation below means that equal weights give you // "average" metering (i.e. all pixels equally important). double R_sum = 0, G_sum = 0, B_sum = 0, pixel_sum = 0; @@ -436,7 +443,7 @@ void Agc::computeGain(bcm2835_isp_stats *statistics, Metadata *image_metadata, target_Y = config_.Y_target.Eval(config_.Y_target.Domain().Clip(lux.lux)); target_Y = std::min(EV_GAIN_Y_TARGET_LIMIT, target_Y * ev_gain); - double initial_Y = compute_initial_Y(statistics, image_metadata, + double initial_Y = compute_initial_Y(statistics, awb_, metering_mode_->weights); gain = std::min(10.0, target_Y / (initial_Y + .001)); LOG(RPiAgc, Debug) << "Initially Y " << initial_Y << " target " << target_Y @@ -482,19 +489,13 @@ void Agc::computeTargetExposure(double gain) LOG(RPiAgc, Debug) << "Target total_exposure " << target_.total_exposure; } -bool Agc::applyDigitalGain(Metadata *image_metadata, double gain, - double target_Y) +bool Agc::applyDigitalGain(double gain, double target_Y) { - double dg = 1.0; + double min_colour_gain = std::min({ awb_.gain_r, awb_.gain_g, awb_.gain_b, 1.0 }); + ASSERT(min_colour_gain != 0.0); + double dg = 1.0 / min_colour_gain; // I think this pipeline subtracts black level and rescales before we // get the stats, so no need to worry about it. - struct AwbStatus awb; - if (image_metadata->Get("awb.status", awb) == 0) { - double min_gain = std::min(awb.gain_r, - std::min(awb.gain_g, awb.gain_b)); - dg *= std::max(1.0, 1.0 / min_gain); - } else - LOG(RPiAgc, Warning) << "Agc: no AWB status found"; LOG(RPiAgc, Debug) << "after AWB, target dg " << dg << " gain " << gain << " target_Y " << target_Y; // Finally, if we're trying to reduce exposure but the target_Y is diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index 2442fc03..e7ac480f 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -83,11 +83,11 @@ private: AgcConfig config_; void housekeepConfig(); void fetchCurrentExposure(Metadata *image_metadata); + void fetchAwbStatus(Metadata *image_metadata); void computeGain(bcm2835_isp_stats *statistics, Metadata *image_metadata, double &gain, double &target_Y); void computeTargetExposure(double gain); - bool applyDigitalGain(Metadata *image_metadata, double gain, - double target_Y); + bool applyDigitalGain(double gain, double target_Y); void filterExposure(bool desaturate); void divideUpExposure(); void writeAndFinish(Metadata *image_metadata, bool desaturate); @@ -95,6 +95,7 @@ private: AgcExposureMode *exposure_mode_; AgcConstraintMode *constraint_mode_; uint64_t frame_count_; + AwbStatus awb_; struct ExposureValues { ExposureValues() : shutter(0), analogue_gain(0), total_exposure(0), total_exposure_no_dg(0) {} From patchwork Mon Nov 23 07:38:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10465 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 810FDBE08A for ; Mon, 23 Nov 2020 07:38:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4FAD76332E; Mon, 23 Nov 2020 08:38:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="jGML+XRY"; dkim-atps=neutral Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 280F960332 for ; Mon, 23 Nov 2020 08:38:36 +0100 (CET) Received: by mail-wm1-x32d.google.com with SMTP id s13so16276193wmh.4 for ; Sun, 22 Nov 2020 23:38:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=43bWmHG17Nxr2zP+ZJpDTRe06JALjHw0tSt03Ca7jmI=; b=jGML+XRYoGfb3GJwgBAY+P3km6lJGuAVLQ8UTtxpLXbnjIMlD+/eptmL4/oZSoUJVc NkHmVTGiCm/QjOUd5pFNbb6dFFHZ8H9BIt2x51Pw/btNcxfoz8FrnF/ZdKDAT1iz9Wrb 52uUPICYEhvoxlAviTrqfoN11pZm2675o4YB8PBe4jpKrC8fq99unlsdwKqcB+Ob9oQF hePcZRbroOaNtAGd+iVex/VNkS+6IBz0gSiI8tCPvMloVlU7TP80PwUFfKwjRdbngqqb e56JnOKXd2gbc+IL2LG+x+uYn1ArldOnSWi9NoQSIWrkADh9DLuybc8jcjw7BK4uSj47 67KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=43bWmHG17Nxr2zP+ZJpDTRe06JALjHw0tSt03Ca7jmI=; b=GRy6x7OgpdwE13j5r0B0zsRAhQch1+mgzyEKh9HndtNe3pKG9K44/83aPqIHXVgLCM JKpT580QUVAGsMg/wIezefkusmDlXmuWcZel8eNCCZqYLWKgyBb3qFrPsHrpKG3bkEI+ IJTIKl/1m+ejdFJYVAB1uN4gbo9jFb26tsiS3vUaWZrre8DYOJ4e5WMSLANanRRAXFZ0 UvJA62sTCi1C8R2luiKk34zRgUtKCnv5/MuF+tNHoaOT2rHObh1EOsOxTStGNmc+Krf9 E6KATZaXpXvfxv95hvY5lUHqEWsjMjt/CMs0v4Hmwb7ocknhZpCrCHWYv9unLnsyQhsQ vboA== X-Gm-Message-State: AOAM530yW0EBlDVvxY0bqzhlPatOz5+yCOy9Fv7x8jRICtPBTzFSBEpS oyRY8tE5an/geW676GUJIzU+vAH2+XKIXdOJ X-Google-Smtp-Source: ABdhPJzde+fxoVyqCC/BPSc4bbK7xe2NVtmz9X9EKSdalQvkZ6YqB7J+hgbxGQ+w5NAWs96IDSVg1w== X-Received: by 2002:a7b:c5cf:: with SMTP id n15mr23347401wmk.9.1606117115574; Sun, 22 Nov 2020 23:38:35 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id h15sm17841822wrw.15.2020.11.22.23.38.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Nov 2020 23:38:34 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 23 Nov 2020 07:38:00 +0000 Message-Id: <20201123073804.3125-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201123073804.3125-1-david.plowman@raspberrypi.com> References: <20201123073804.3125-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 06/10] libcamera: ipa: raspberrypi: awb: Add SwitchMode method to output AWB status X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Awb class now implements a SwitchMode method which outputs its AwbStatus for other algorithms to read, should they be interested. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/awb.cpp | 14 ++++++++++++++ src/ipa/raspberrypi/controller/rpi/awb.hpp | 1 + 2 files changed, 15 insertions(+) diff --git a/src/ipa/raspberrypi/controller/rpi/awb.cpp b/src/ipa/raspberrypi/controller/rpi/awb.cpp index 469d0e53..020825e3 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.cpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.cpp @@ -179,6 +179,20 @@ void Awb::SetManualGains(double manual_r, double manual_b) manual_b_ = manual_b; } +void Awb::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, + Metadata *metadata) +{ + // If fixed colour gains have been set, we should let other algorithms + // know by writing it into the image metadata. + if (manual_r_ != 0.0 && manual_b_ != 0.0) { + prev_sync_results_.gain_r = manual_r_; + prev_sync_results_.gain_g = 1.0; + prev_sync_results_.gain_b = manual_b_; + sync_results_ = prev_sync_results_; + } + metadata->Set("awb.status", prev_sync_results_); +} + void Awb::fetchAsyncResults() { RPI_LOG("Fetch AWB results"); diff --git a/src/ipa/raspberrypi/controller/rpi/awb.hpp b/src/ipa/raspberrypi/controller/rpi/awb.hpp index 9124d042..545d85a8 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.hpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.hpp @@ -84,6 +84,7 @@ public: void Read(boost::property_tree::ptree const ¶ms) override; void SetMode(std::string const &name) override; void SetManualGains(double manual_r, double manual_b) override; + void SwitchMode(CameraMode const &camera_mode, Metadata *metadata) override; void Prepare(Metadata *image_metadata) override; void Process(StatisticsPtr &stats, Metadata *image_metadata) override; struct RGB { From patchwork Mon Nov 23 07:38:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10466 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E2799BE177 for ; Mon, 23 Nov 2020 07:38:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9B47663327; Mon, 23 Nov 2020 08:38:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="WajKYdJ7"; dkim-atps=neutral Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3E9EC6334E for ; Mon, 23 Nov 2020 08:38:37 +0100 (CET) Received: by mail-wr1-x436.google.com with SMTP id r17so17622056wrw.1 for ; Sun, 22 Nov 2020 23:38:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Xj/mIFBwOUaGUcNu8yackrW21gg3ew3bmgpzWscYFA8=; b=WajKYdJ7RUt1gRAoKxfphhvsFFBPZBjP8K8S0awtYcKOpqWVAOrjGCOiQNwEE7Lei1 yS/lNVJME67gscnn2GNLO/3r6//+d4/wHSpoaW1G87ziDZZcokVB32kgaGhsYAFhrpDn yerdT4tgrK1iCfW7mPUbvEs0A2Xn9/QzT4ytYvZI5uW7M7zE/S6HrdM+Lxc6M9zSAgvg HrHk2iCgfLOz1og9x2idT7pLt0fbsFkmUyOcVA/+1hKM9v5bmbcPUJmhhzWXfoLT4/Vl TK4GXGbCHcX3AEc9NmmZpHQ6/3HujNQwOjynNzsnJF/aepI9B7dl1Zp+ALOEYSE1J2tF uJcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Xj/mIFBwOUaGUcNu8yackrW21gg3ew3bmgpzWscYFA8=; b=XLgRIxWdooiYWlK+x8pyAzpsPUH0N6fL2tZMReOseWpKjFFt/B5c5nOwZOG9kSQCko LgxpUP4IQKBKCW0FaHoY+uF7JOwlYRmi/rGm81WnBdRSxqbyO0Fe+kUsFqG4EMi6z68R PXwgLaZEAY/8BbN+34nI2ePhKbgubqgxSxMlh4+viT/UhrnfP9Otl0WGmC+faJL9l4BX El2Z5Bx2gzvmJv19Nv6BxDlzU9pEHLzY4DqTy6bmT8kSXDyE0Fl/H4wd4OW2owQX6QiD K7WC90IcEfRO2AITe9cAwFcJDvq2hRW7YnqGiDikI5ag4pYT6iIsJnSVNTDl4LCPx6Qy c6MA== X-Gm-Message-State: AOAM532RgjK1hR35JYvlIgtYV7Yi1CJVoaH/PethlXhiBZEzXKdR4BZc 9zFssw+TKCGgh/FOw4zUSsd0tcvqAv+oCkbD X-Google-Smtp-Source: ABdhPJyR5mSrU7FJgc22jTzRcwApcN1aA4qND0+vGJd3QRsMFnDVNC91Acl0KH7uez8R4c/D8NDhpw== X-Received: by 2002:a5d:6452:: with SMTP id d18mr29174412wrw.200.1606117116505; Sun, 22 Nov 2020 23:38:36 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id h15sm17841822wrw.15.2020.11.22.23.38.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Nov 2020 23:38:35 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 23 Nov 2020 07:38:01 +0000 Message-Id: <20201123073804.3125-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201123073804.3125-1-david.plowman@raspberrypi.com> References: <20201123073804.3125-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 07/10] libcamera: ipa: raspberrypi: agc: Report fixed exposure/gain values during SwitchMode X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When an application has specified fixed exposure time and/or gain they must be programmed into the sensor immediately, even before the sensor has been started. For this to happen they must be written into the image metadata when the SwitchMode method is invoked. We also make the default exposure/gain, when nothing has been set, customisable in the tuning file. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 74 +++++++++++++++++----- src/ipa/raspberrypi/controller/rpi/agc.hpp | 2 + 2 files changed, 60 insertions(+), 16 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 4cf98e66..2533727e 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -145,6 +145,9 @@ void AgcConfig::Read(boost::property_tree::ptree const ¶ms) fast_reduce_threshold = params.get("fast_reduce_threshold", 0.4); base_ev = params.get("base_ev", 1.0); + // Start with quite a low value as ramping up is easier than ramping down. + default_exposure_time = params.get("default_exposure_time", 1000); + default_analogue_gain = params.get("default_analogue_gain", 1.0); } Agc::Agc(Controller *controller) @@ -220,14 +223,42 @@ void Agc::SetConstraintMode(std::string const &constraint_mode_name) void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, Metadata *metadata) { - // On a mode switch, it's possible the exposure profile could change, - // so we run through the dividing up of exposure/gain again and - // write the results into the metadata we've been given. - if (status_.total_exposure_value) { - housekeepConfig(); + housekeepConfig(); + + if (fixed_shutter_ != 0.0 && fixed_analogue_gain_ != 0.0) { + // We're going to reset the algorithm here with these fixed values. + + fetchAwbStatus(metadata); + double min_colour_gain = std::min({ awb_.gain_r, awb_.gain_g, awb_.gain_b, 1.0 }); + ASSERT(min_colour_gain != 0.0); + + // This is the equivalent of computeTargetExposure and applyDigitalGain. + target_.total_exposure_no_dg = fixed_shutter_ * fixed_analogue_gain_; + target_.total_exposure = target_.total_exposure_no_dg / min_colour_gain; + + // Equivalent of filterExposure. This resets any "history". + filtered_ = target_; + + // Equivalent of divideUpExposure. + filtered_.shutter = fixed_shutter_; + filtered_.analogue_gain = fixed_analogue_gain_; + } else if (status_.total_exposure_value) { + // On a mode switch, it's possible the exposure profile could change, + // or a fixed exposure/gain might be set so we divide up the exposure/ + // gain again, but we don't change any target values. divideUpExposure(); - writeAndFinish(metadata, false); + } else { + // We come through here on startup, when at least one of the shutter + // or gain has not been fixed. We must still write those values out so + // that they will be applied immediately. We supply some arbitrary defaults + // for any that weren't set. + + // Equivalent of divideUpExposure. + filtered_.shutter = fixed_shutter_ ? fixed_shutter_ : config_.default_exposure_time; + filtered_.analogue_gain = fixed_analogue_gain_ ? fixed_analogue_gain_ : config_.default_analogue_gain; } + + writeAndFinish(metadata, false); } void Agc::Prepare(Metadata *image_metadata) @@ -472,20 +503,31 @@ void Agc::computeGain(bcm2835_isp_stats *statistics, Metadata *image_metadata, void Agc::computeTargetExposure(double gain) { - // The statistics reflect the image without digital gain, so the final - // total exposure we're aiming for is: - target_.total_exposure = current_.total_exposure_no_dg * gain; - // The final target exposure is also limited to what the exposure - // mode allows. - double max_total_exposure = - (status_.fixed_shutter != 0.0 + if (status_.fixed_shutter != 0.0 && status_.fixed_analogue_gain != 0.0) { + // When ag and shutter are both fixed, we need to drive the + // total exposure so that we end up with a digital gain of at least + // 1/min_colour_gain. Otherwise we'd desaturate channels causing + // white to go cyan or magenta. + double min_colour_gain = std::min({ awb_.gain_r, awb_.gain_g, awb_.gain_b, 1.0 }); + ASSERT(min_colour_gain != 0.0); + target_.total_exposure = + status_.fixed_shutter * status_.fixed_analogue_gain / min_colour_gain; + } else { + // The statistics reflect the image without digital gain, so the final + // total exposure we're aiming for is: + target_.total_exposure = current_.total_exposure_no_dg * gain; + // The final target exposure is also limited to what the exposure + // mode allows. + double max_total_exposure = + (status_.fixed_shutter != 0.0 ? status_.fixed_shutter : exposure_mode_->shutter.back()) * - (status_.fixed_analogue_gain != 0.0 + (status_.fixed_analogue_gain != 0.0 ? status_.fixed_analogue_gain : exposure_mode_->gain.back()); - target_.total_exposure = std::min(target_.total_exposure, - max_total_exposure); + target_.total_exposure = std::min(target_.total_exposure, + max_total_exposure); + } LOG(RPiAgc, Debug) << "Target total_exposure " << target_.total_exposure; } diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index e7ac480f..859a9650 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -60,6 +60,8 @@ struct AgcConfig { std::string default_exposure_mode; std::string default_constraint_mode; double base_ev; + double default_exposure_time; + double default_analogue_gain; }; class Agc : public AgcAlgorithm From patchwork Mon Nov 23 07:38:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10467 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 52139BE08A for ; Mon, 23 Nov 2020 07:38:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1F9086334C; Mon, 23 Nov 2020 08:38:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="pRipBaTR"; dkim-atps=neutral Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ED7AE632B2 for ; Mon, 23 Nov 2020 08:38:37 +0100 (CET) Received: by mail-wr1-x42f.google.com with SMTP id b6so17595145wrt.4 for ; Sun, 22 Nov 2020 23:38:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ulb2sJuTBbnF03P2gKhOiyCPKEujg0gr7XFejXSRJME=; b=pRipBaTRGaF/ZphEHbxznaOL0l4n+DQTniJLBSi97anfoK9d1laRjcc1pETTldpwD4 NarCgqLDQ+FgJ8EyN5ta+l5qkh6NaCahqdhfFCfzuG967mjLxPoDSvspwBLmIe3zA+c1 p5wEHfVs4mJvEuVH35DTXUarPRxYabAIqbiE3YHsRBAN4g3HSHoIGKdYoixj6ub31eGB KDqfvG21DAfDUsxcjmq1EU6WhP0KSCzBOAvofYO05XqylR3xKgOzee55yYFO14KeYTEC VlPgCk/Qarg2mcPsYZ8DGHx6yLWD4SeZEs6O2+omRtHYFY2qxfVihvNHaQ2PJipu7Ig0 4LJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ulb2sJuTBbnF03P2gKhOiyCPKEujg0gr7XFejXSRJME=; b=i4BOMshtftbk87UXJysBWMUvBpHs2YlMqS2u8pPrlnqb6N6H3qfKVbc2n4bbzPORGb CFBB+hz7jS1lLePEewoNGTYloppJg3x+QDDL/FH0ejVeEal+FyBk3kQqKXN/ntjbXvcC fiRPHaQjal1vDadQY9rWjNafy1bxmk+gk9ayLWnoWbYtLE8NBcjM5Zrpyhx8tMnxBlwf AnAc0g5CA6I+nNUi2OmBs1r6yfxBvh5zw2FmHiyYf+Y1u9WrXnH3uDE91gPtTZfWRgSf TtuqaTeDWOcj6/pyxmglhLY23VZRTSU6/ENEgwCkhcAykPcHN4Am1XpAsimB9EUCVQdb H5sQ== X-Gm-Message-State: AOAM531vj0VfGLl2p+IgVDN2vDCiZNGuszCt5rI7ZyZI4yo7ZdLPKfYZ /YjB+2OvP64QL7INZE8xTwjZ07lP0CZ1meoO X-Google-Smtp-Source: ABdhPJyj2orV20O2uNaILg11mHzDKcBY43ZBSC6UWAncQV+ZeCiD6A5XLbdSj3M7ioyXwLFCyEgOgQ== X-Received: by 2002:adf:a343:: with SMTP id d3mr29500501wrb.91.1606117117490; Sun, 22 Nov 2020 23:38:37 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id h15sm17841822wrw.15.2020.11.22.23.38.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Nov 2020 23:38:36 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 23 Nov 2020 07:38:02 +0000 Message-Id: <20201123073804.3125-9-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201123073804.3125-1-david.plowman@raspberrypi.com> References: <20201123073804.3125-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 08/10] libcamera: src: ipa: raspberrypi: agc: Fix uninitialised members in Agc constructor X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use memset in the constructor for embedded structures, it is tidier and initialises everything. We use the initialiser list for other members. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 2533727e..d7d37055 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -153,16 +153,15 @@ void AgcConfig::Read(boost::property_tree::ptree const ¶ms) Agc::Agc(Controller *controller) : AgcAlgorithm(controller), metering_mode_(nullptr), exposure_mode_(nullptr), constraint_mode_(nullptr), - frame_count_(0), lock_count_(0) -{ - ev_ = status_.ev = 1.0; - flicker_period_ = status_.flicker_period = 0.0; - fixed_shutter_ = status_.fixed_shutter = 0; - fixed_analogue_gain_ = status_.fixed_analogue_gain = 0.0; - // set to zero initially, so we can tell it's not been calculated - status_.total_exposure_value = 0.0; - status_.target_exposure_value = 0.0; - status_.locked = false; + frame_count_(0), lock_count_(0), + ev_(1.0), flicker_period_(0.0), + fixed_shutter_(0), fixed_analogue_gain_(0.0) +{ + memset(&awb_, 0, sizeof(awb_)); + // Setting status_.total_exposure_value_ to zero initially tells us + // it's not been calculated yet (i.e. Process hasn't yet run). + memset(&status_, 0, sizeof(status_)); + status_.ev = ev_; } char const *Agc::Name() const From patchwork Mon Nov 23 07:38:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10468 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B3A31BE08A for ; Mon, 23 Nov 2020 07:38:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7F84463350; Mon, 23 Nov 2020 08:38:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="An+uLqiS"; dkim-atps=neutral Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E49FA63325 for ; Mon, 23 Nov 2020 08:38:38 +0100 (CET) Received: by mail-wm1-x330.google.com with SMTP id 1so16283273wme.3 for ; Sun, 22 Nov 2020 23:38:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=l1N2pr2q7Z9XN1viacAoQlqJnMssJB536U10h0wxdDc=; b=An+uLqiSKQdfdI4+8GOqCNCmhOyLbgyEca5cI2bYXtL81O2+t9BROT/sjehgRc/Xs3 dyLwwjq3sMLb2ygxbnZLByIrf1j5DVkcVw1Jcu+4MoF5GYfa1zRtA75EaD59SHz8kioc cgoyDIrx5WXr8ybaeo+9KGegL3a0tmf0/twiMLvB8bed41ok6319qiR+tvGlgLQiCVbe A6ZlgRo4S3PucC96Tnoi0pqnxrmktNVpdUwEN8+C9g5xtUgASpbj4MNjQ0aY/7PPBnS2 ifuy7h0QPlyr8DkZ6QSz/qiGXekaJvG3wd973+kQdIqZvQXPXf1EMPeFutdam4LHCU0q PH+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l1N2pr2q7Z9XN1viacAoQlqJnMssJB536U10h0wxdDc=; b=YbcrQt1VBvxd18rqFe2BqS8UJyulh4ocpHhiLM1vpniJl9XHyKDoVRSxpse/Ymh18j Gqu4k3yIseL8Sp+pYiFQkBEU1BMNlxgPUKg+VsSedFSK/5tjqgjuyWnUKwAdA6RCSXQ5 fTLzH/d8jVzT1IHWgzJmtK40DePyogGUM8CFkUDmUAxDdKMvWOZ8RoUMoy+QA61HQt/F GVFESYpcEgCYxD1Mw2BzBeOixYxusbrk3VSIPPOuK3xrbWXJklEt8QvgLFxKPTNIFB6N cUJBiKcRyDb7sk/ZuRvhPusFgHTf3cQPXWJoj9ViLe/H+qmbKaMViq/etg+3G4IiV8OG Y3vQ== X-Gm-Message-State: AOAM531qKTqRjD8z6n66OYFg4canIhJKCxqd4qz92ms/1jH5rcL7uzaN AsOCirR+6m4F93C6jswTYYqSffsCEPgN97ZC X-Google-Smtp-Source: ABdhPJx33csanBlg9ffLf0RgrNGI103mdr3LYwjBrjAhocfMD5DMPeiRUz71xxE80lDl57306OiYtQ== X-Received: by 2002:a1c:3c4:: with SMTP id 187mr8148864wmd.143.1606117118355; Sun, 22 Nov 2020 23:38:38 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id h15sm17841822wrw.15.2020.11.22.23.38.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Nov 2020 23:38:37 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 23 Nov 2020 07:38:03 +0000 Message-Id: <20201123073804.3125-10-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201123073804.3125-1-david.plowman@raspberrypi.com> References: <20201123073804.3125-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 09/10] libcamera: src: ipa: raspberrypi: agc: Improve gain update calculation for partly saturated images X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When parts of an image saturate then the image brightness no longer increases linearly with increased exposure/gain. Having calculated a linear gain value it's better then to try it, allowing for saturating regions, and if necessary increase the gain some more. We repeat this several times. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 34 +++++++++++++++------- 1 file changed, 24 insertions(+), 10 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index d7d37055..37806055 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -419,17 +419,21 @@ void Agc::fetchAwbStatus(Metadata *image_metadata) } static double compute_initial_Y(bcm2835_isp_stats *stats, AwbStatus const &awb, - double weights[]) + double weights[], double gain) { bcm2835_isp_stats_region *regions = stats->agc_stats; // Note how the calculation below means that equal weights give you // "average" metering (i.e. all pixels equally important). double R_sum = 0, G_sum = 0, B_sum = 0, pixel_sum = 0; for (int i = 0; i < AGC_STATS_SIZE; i++) { - R_sum += regions[i].r_sum * weights[i]; - G_sum += regions[i].g_sum * weights[i]; - B_sum += regions[i].b_sum * weights[i]; - pixel_sum += regions[i].counted * weights[i]; + double counted = regions[i].counted; + double r_sum = std::min(regions[i].r_sum * gain, ((1 << PIPELINE_BITS) - 1) * counted); + double g_sum = std::min(regions[i].g_sum * gain, ((1 << PIPELINE_BITS) - 1) * counted); + double b_sum = std::min(regions[i].b_sum * gain, ((1 << PIPELINE_BITS) - 1) * counted); + R_sum += r_sum * weights[i]; + G_sum += g_sum * weights[i]; + B_sum += b_sum * weights[i]; + pixel_sum += counted * weights[i]; } if (pixel_sum == 0.0) { LOG(RPiAgc, Warning) << "compute_initial_Y: pixel_sum is zero"; @@ -473,11 +477,21 @@ void Agc::computeGain(bcm2835_isp_stats *statistics, Metadata *image_metadata, target_Y = config_.Y_target.Eval(config_.Y_target.Domain().Clip(lux.lux)); target_Y = std::min(EV_GAIN_Y_TARGET_LIMIT, target_Y * ev_gain); - double initial_Y = compute_initial_Y(statistics, awb_, - metering_mode_->weights); - gain = std::min(10.0, target_Y / (initial_Y + .001)); - LOG(RPiAgc, Debug) << "Initially Y " << initial_Y << " target " << target_Y - << " gives gain " << gain; + + // Do this calculation a few times as brightness increase can be + // non-linear when there are saturated regions. + gain = 1.0; + for (int i = 0; i < 8; i++) { + double initial_Y = compute_initial_Y(statistics, awb_, + metering_mode_->weights, gain); + double extra_gain = std::min(10.0, target_Y / (initial_Y + .001)); + gain *= extra_gain; + LOG(RPiAgc, Debug) << "Initial Y " << initial_Y << " target " << target_Y + << " gives gain " << gain; + if (extra_gain < 1.01) // close enough + break; + } + for (auto &c : *constraint_mode_) { double new_target_Y; double new_gain = From patchwork Mon Nov 23 07:38:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10469 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1D4A3BE177 for ; Mon, 23 Nov 2020 07:38:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D9F1D63359; Mon, 23 Nov 2020 08:38:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="DuVqlZUW"; dkim-atps=neutral Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BE69263325 for ; Mon, 23 Nov 2020 08:38:39 +0100 (CET) Received: by mail-wr1-x42f.google.com with SMTP id 23so17535485wrc.8 for ; Sun, 22 Nov 2020 23:38:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xxsqE/MmwUS9M8cOxXNB6MzXfTJijIjyM9MywW0cEkY=; b=DuVqlZUWSTLxFVLr53wowlMaiE2XB3oeYiEOVEpqbxwPBJL+Cgq6UMtWenyT+U0myg 7L9J0ARHnIqA9KVhyIh3COaeYkdxopU2lLSVzk7chrtZKcvzooAKqjR06KBxCczN4e1u Wg6uzuEL2fBLKmi+8ARIqgNzYGi8/SvcPphWXOagh9B+yWP6rU61gMfnEHeBReqb5PjL CUHLhHX5iLcva+4zYrobbHRPELI5qr2qqy6npmrztz7SbC+Go1QcUBbaYU8ekeZkvkjy 2wBsMp4NROaqXAE/txbvNe/t8biXkq5lSklaG/EoRSnt5yKFC6ekaR2WVcRw5IjbxNnA VayQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xxsqE/MmwUS9M8cOxXNB6MzXfTJijIjyM9MywW0cEkY=; b=CUZv7jyvWLA1liXo2wy1slzPUIBblpEXY6Os7O7Jr/C39lKPRGNILJDL91Fvn0cVvC +WsBJACLMbY2q07jH3KCHbMj68QKK2NPjUa631YNMuPMfv9oxm1O/z9BVqcovKoLq22G ksqzSgS0y1YSZRWupj5wSerdTwAlVJO6+xm2gmHJK4TuXeFofVGW2HYRghFMF+WUj9Gz t2G/p46Y9Zs61ciVemqhSqTyomeHP3sD5jyVhH68IczSV1xR8sstgVPyKSktxPGlfdYg JlRHv7DE1EuqQLPeS7LTp7nR9uoeROIVI/fZfH9BumiUllk6Q9Iat4IXQiWbihyd4/V8 +NLw== X-Gm-Message-State: AOAM533Zd8cpNkFB/RHm662D0Q5H1IWUr6BY6o9bPG/JmqH4Vxnbdaen CWkMkUm4IO4gRL2Rme79TsaMkL4GYZtV1cy9 X-Google-Smtp-Source: ABdhPJzTdJCSbsrmKDH2+W+UIkpVEiEdAIwknfXK1QFoGBRWnJvb9CM+i0QmeTbE6bYrYhTvmQhJHA== X-Received: by 2002:a05:6000:1292:: with SMTP id f18mr29029212wrx.196.1606117119205; Sun, 22 Nov 2020 23:38:39 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id h15sm17841822wrw.15.2020.11.22.23.38.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Nov 2020 23:38:38 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 23 Nov 2020 07:38:04 +0000 Message-Id: <20201123073804.3125-11-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201123073804.3125-1-david.plowman@raspberrypi.com> References: <20201123073804.3125-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 10/10] libcamera: src: ipa: raspberrypi: agc: Improve AE locked logic X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Previously we required that the sensor absolutely reaches the target exposure, but this can fail if frame rates or analogue gains are limited. Instead insist only that we get several frames with the same exposure time, analogue gain and that the algorithm's target exposure hasn't changed either. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 66 ++++++++++++++-------- src/ipa/raspberrypi/controller/rpi/agc.hpp | 3 + 2 files changed, 44 insertions(+), 25 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 37806055..4c56bdc9 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -154,6 +154,7 @@ Agc::Agc(Controller *controller) : AgcAlgorithm(controller), metering_mode_(nullptr), exposure_mode_(nullptr), constraint_mode_(nullptr), frame_count_(0), lock_count_(0), + last_target_exposure_(0.0), ev_(1.0), flicker_period_(0.0), fixed_shutter_(0), fixed_analogue_gain_(0.0) { @@ -162,6 +163,7 @@ Agc::Agc(Controller *controller) // it's not been calculated yet (i.e. Process hasn't yet run). memset(&status_, 0, sizeof(status_)); status_.ev = ev_; + memset(&last_device_status_, 0, sizeof(last_device_status_)); } char const *Agc::Name() const @@ -262,8 +264,6 @@ void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, void Agc::Prepare(Metadata *image_metadata) { - int lock_count = lock_count_; - lock_count_ = 0; status_.digital_gain = 1.0; fetchAwbStatus(image_metadata); // always fetch it so that Process knows it's been done @@ -287,31 +287,10 @@ void Agc::Prepare(Metadata *image_metadata) LOG(RPiAgc, Debug) << "Use digital_gain " << status_.digital_gain; LOG(RPiAgc, Debug) << "Effective exposure " << actual_exposure * status_.digital_gain; // Decide whether AEC/AGC has converged. - // Insist AGC is steady for MAX_LOCK_COUNT - // frames before we say we are "locked". - // (The hard-coded constants may need to - // become customisable.) - if (status_.target_exposure_value) { -#define MAX_LOCK_COUNT 3 - double err = 0.10 * status_.target_exposure_value + 200; - if (actual_exposure < - status_.target_exposure_value + err && - actual_exposure > - status_.target_exposure_value - err) - lock_count_ = - std::min(lock_count + 1, - MAX_LOCK_COUNT); - else if (actual_exposure < - status_.target_exposure_value + 1.5 * err && - actual_exposure > - status_.target_exposure_value - 1.5 * err) - lock_count_ = lock_count; - LOG(RPiAgc, Debug) << "Lock count: " << lock_count_; - } + updateLockStatus(device_status); } } else - LOG(RPiAgc, Debug) << Name() << ": no device metadata"; - status_.locked = lock_count_ >= MAX_LOCK_COUNT; + LOG(RPiAgc, Warning) << Name() << ": no device metadata"; image_metadata->Set("agc.status", status_); } } @@ -342,6 +321,43 @@ void Agc::Process(StatisticsPtr &stats, Metadata *image_metadata) writeAndFinish(image_metadata, desaturate); } +void Agc::updateLockStatus(DeviceStatus const &device_status) +{ + const double ERROR_FACTOR = 0.10; // make these customisable? + const int MAX_LOCK_COUNT = 5; + // Reset "lock count" when we exceed this multiple of ERROR_FACTOR + const double RESET_MARGIN = 1.5; + + // Add 200us to the exposure time error to allow for line quantisation. + double exposure_error = last_device_status_.shutter_speed * ERROR_FACTOR + 200; + double gain_error = last_device_status_.analogue_gain * ERROR_FACTOR; + double target_error = last_target_exposure_ * ERROR_FACTOR; + + // Note that we don't know the exposure/gain limits of the sensor, so + // the values we keep requesting may be unachievable. For this reason + // we only insist that we're close to values in the past few frames. + if (device_status.shutter_speed > last_device_status_.shutter_speed - exposure_error && + device_status.shutter_speed < last_device_status_.shutter_speed + exposure_error && + device_status.analogue_gain > last_device_status_.analogue_gain - gain_error && + device_status.analogue_gain < last_device_status_.analogue_gain + gain_error && + status_.target_exposure_value > last_target_exposure_ - target_error && + status_.target_exposure_value < last_target_exposure_ + target_error) + lock_count_ = std::min(lock_count_ + 1, MAX_LOCK_COUNT); + else if (device_status.shutter_speed < last_device_status_.shutter_speed - RESET_MARGIN * exposure_error || + device_status.shutter_speed > last_device_status_.shutter_speed + RESET_MARGIN * exposure_error || + device_status.analogue_gain < last_device_status_.analogue_gain - RESET_MARGIN * gain_error || + device_status.analogue_gain > last_device_status_.analogue_gain + RESET_MARGIN * gain_error || + status_.target_exposure_value < last_target_exposure_ - RESET_MARGIN * target_error || + status_.target_exposure_value > last_target_exposure_ + RESET_MARGIN * target_error) + lock_count_ = 0; + + last_device_status_ = device_status; + last_target_exposure_ = status_.target_exposure_value; + + LOG(RPiAgc, Debug) << "Lock count updated to " << lock_count_; + status_.locked = lock_count_ == MAX_LOCK_COUNT; +} + static void copy_string(std::string const &s, char *d, size_t size) { size_t length = s.copy(d, size - 1); diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index 859a9650..47ebb324 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -82,6 +82,7 @@ public: void Process(StatisticsPtr &stats, Metadata *image_metadata) override; private: + void updateLockStatus(DeviceStatus const &device_status); AgcConfig config_; void housekeepConfig(); void fetchCurrentExposure(Metadata *image_metadata); @@ -111,6 +112,8 @@ private: ExposureValues filtered_; // these values are filtered towards target AgcStatus status_; int lock_count_; + DeviceStatus last_device_status_; + double last_target_exposure_; // Below here the "settings" that applications can change. std::string metering_mode_name_; std::string exposure_mode_name_;