From patchwork Mon Nov 16 16:49:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10430 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AA792BE082 for ; Mon, 16 Nov 2020 16:49:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9C5B1632CD; Mon, 16 Nov 2020 17:49:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="ZdDWY5Ua"; dkim-atps=neutral Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8C498632C6 for ; Mon, 16 Nov 2020 17:49:25 +0100 (CET) Received: by mail-wr1-x436.google.com with SMTP id u12so12196247wrt.0 for ; Mon, 16 Nov 2020 08:49:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L4kwk7FyYtVgnUS2q+EE1+JNZpjTiSe/toZBQoAzaEI=; b=ZdDWY5Ua7m4m+jPpdKHTmRmmcP91nJTKq2DEA9ivvXhiY9ObA5GcBBwTmf1mIFXkto juYHUiglFF1aq0Eyo2rz+UzBpes7n1gEziPbhkJqGmhhVWNqbJSsdbTX/PZ5SXPsEQay v8KLFirxw0VbOmkkA4VEVtoSZQxmjuttMwmGlZZMlu8rbcUljlTAcx5ivixXcoMJ7LK1 aygYA/YvILc75pKBokTvsvW4dPJO3xWCLHorEnckiEjuBcNg06BiFj+7oVA2G/4aw2s1 Fp87vRWTebVgGFjEKbkD9YE3I3gRzwIMbybdVqOV6D4KPvVS05bsUm4jcuNAmkojYo6R N4gA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L4kwk7FyYtVgnUS2q+EE1+JNZpjTiSe/toZBQoAzaEI=; b=NR4T3ecUfIGoAjZ/+puYnfq4lwxlZmnjuwAFgtEbA7zEKxYndyzjHy2NX8lMrTZi31 REOPkpsnhTqzJ2W9WiRmWkqrt9shWtwqxr1phE85EiLvrC5qwzZ8q8MDlhdzGuL3z9nO iw0CWVoLUL1ZPZMOUWAQkUAJIkydenmShb5G40YNT5fyO5Kf12sS25BBfSdI06XaKZSs 5l6bO30shO+AQTrlmkAaKMuxqBz2Lp/HcPBruqPZYWq9KrX+nq4TOoLnDiSYLddOOIkF hEC6lN/j0i9IRd7jx57oJvZTVZnWEmtyiZmej9aEbCbqSrq8pmuRgiQAqGj7V6Qx+3ke OYUA== X-Gm-Message-State: AOAM530cuV3IAM+5m2icXTXAeyVOrCgNKi246UVSYukcKdi76KLuJxow S1jfOWyijX97shcHTLM6k6XWcWD2hIjtCA== X-Google-Smtp-Source: ABdhPJwiJnzlfLpzOX3Y/zOm+dLZt6xrNE8c0OWMA/qdxwLRfB6ryPASmRRj5kUsa7zmGRPkpX943g== X-Received: by 2002:a5d:54c5:: with SMTP id x5mr20499466wrv.297.1605545364384; Mon, 16 Nov 2020 08:49:24 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id q16sm23716973wrn.13.2020.11.16.08.49.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 08:49:23 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 16 Nov 2020 16:49:09 +0000 Message-Id: <20201116164918.2055-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201116164918.2055-1-david.plowman@raspberrypi.com> References: <20201116164918.2055-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 01/10] libcamera: ipa: raspberrypi: agc: Use libcamera debug X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Replace Raspberry Pi debug with libcamera debug. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 90 +++++++++++----------- 1 file changed, 47 insertions(+), 43 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index df4d3647..8079345b 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -9,16 +9,20 @@ #include "linux/bcm2835-isp.h" +#include "libcamera/internal/log.h" + #include "../awb_status.h" #include "../device_status.h" #include "../histogram.hpp" -#include "../logging.hpp" #include "../lux_status.h" #include "../metadata.hpp" #include "agc.hpp" using namespace RPiController; +using namespace libcamera; + +LOG_DEFINE_CATEGORY(RPiAgc) #define NAME "rpi.agc" @@ -128,7 +132,7 @@ static std::string read_constraint_modes( void AgcConfig::Read(boost::property_tree::ptree const ¶ms) { - RPI_LOG("AgcConfig"); + LOG(RPiAgc, Debug) << "AgcConfig"; default_metering_mode = read_metering_modes( metering_modes, params.get_child("metering_modes")); default_exposure_mode = read_exposure_modes( @@ -166,7 +170,7 @@ char const *Agc::Name() const void Agc::Read(boost::property_tree::ptree const ¶ms) { - RPI_LOG("Agc"); + LOG(RPiAgc, Debug) << "Agc"; config_.Read(params); // Set the config's defaults (which are the first ones it read) as our // current modes, until someone changes them. (they're all known to @@ -254,15 +258,15 @@ void Agc::Prepare(Metadata *image_metadata) status.digital_gain = status_.total_exposure_value / actual_exposure; - RPI_LOG("Want total exposure " << status_.total_exposure_value); + LOG(RPiAgc, Debug) << "Want total exposure " << status_.total_exposure_value; // Never ask for a gain < 1.0, and also impose // some upper limit. Make it customisable? status.digital_gain = std::max( 1.0, std::min(status.digital_gain, 4.0)); - RPI_LOG("Actual exposure " << actual_exposure); - RPI_LOG("Use digital_gain " << status.digital_gain); - RPI_LOG("Effective exposure " << actual_exposure * status.digital_gain); + LOG(RPiAgc, Debug) << "Actual exposure " << actual_exposure; + LOG(RPiAgc, Debug) << "Use digital_gain " << status.digital_gain; + LOG(RPiAgc, Debug) << "Effective exposure " << actual_exposure * status.digital_gain; // Decide whether AEC/AGC has converged. // Insist AGC is steady for MAX_LOCK_COUNT // frames before we say we are "locked". @@ -285,11 +289,11 @@ void Agc::Prepare(Metadata *image_metadata) status.target_exposure_value - 1.5 * err) lock_count_ = lock_count; - RPI_LOG("Lock count: " << lock_count_); + LOG(RPiAgc, Debug) << "Lock count: " << lock_count_; } } } else - RPI_LOG(Name() << ": no device metadata"); + LOG(RPiAgc, Debug) << Name() << ": no device metadata"; status.locked = lock_count_ >= MAX_LOCK_COUNT; //printf("%s\n", status.locked ? "+++++++++" : "-"); image_metadata->Set("agc.status", status); @@ -343,9 +347,9 @@ void Agc::housekeepConfig() status_.fixed_analogue_gain = fixed_analogue_gain_; status_.flicker_period = flicker_period_; } - RPI_LOG("ev " << status_.ev << " fixed_shutter " - << status_.fixed_shutter << " fixed_analogue_gain " - << status_.fixed_analogue_gain); + LOG(RPiAgc, Debug) << "ev " << status_.ev << " fixed_shutter " + << status_.fixed_shutter << " fixed_analogue_gain " + << status_.fixed_analogue_gain; // Make sure the "mode" pointers point to the up-to-date things, if // they've changed. if (strcmp(new_metering_mode_name.c_str(), status_.metering_mode)) { @@ -376,10 +380,10 @@ void Agc::housekeepConfig() copy_string(new_constraint_mode_name, status_.constraint_mode, sizeof(status_.constraint_mode)); } - RPI_LOG("exposure_mode " - << new_exposure_mode_name << " constraint_mode " - << new_constraint_mode_name << " metering_mode " - << new_metering_mode_name); + LOG(RPiAgc, Debug) << "exposure_mode " + << new_exposure_mode_name << " constraint_mode " + << new_constraint_mode_name << " metering_mode " + << new_metering_mode_name; } void Agc::fetchCurrentExposure(Metadata *image_metadata) @@ -404,7 +408,7 @@ static double compute_initial_Y(bcm2835_isp_stats *stats, Metadata *image_metada struct AwbStatus awb; awb.gain_r = awb.gain_g = awb.gain_b = 1.0; // in case no metadata if (image_metadata->Get("awb.status", awb) != 0) - RPI_WARN("Agc: no AWB status found"); + LOG(RPiAgc, Warning) << "Agc: no AWB status found"; double Y_sum = 0, weight_sum = 0; for (int i = 0; i < AGC_STATS_SIZE; i++) { if (regions[i].counted == 0) @@ -443,7 +447,7 @@ void Agc::computeGain(bcm2835_isp_stats *statistics, Metadata *image_metadata, struct LuxStatus lux = {}; lux.lux = 400; // default lux level to 400 in case no metadata found if (image_metadata->Get("lux.status", lux) != 0) - RPI_WARN("Agc: no lux level found"); + LOG(RPiAgc, Warning) << "Agc: no lux level found"; Histogram h(statistics->hist[0].g_hist, NUM_HISTOGRAM_BINS); double ev_gain = status_.ev * config_.base_ev; // The initial gain and target_Y come from some of the regions. After @@ -454,28 +458,28 @@ void Agc::computeGain(bcm2835_isp_stats *statistics, Metadata *image_metadata, double initial_Y = compute_initial_Y(statistics, image_metadata, metering_mode_->weights); gain = std::min(10.0, target_Y / (initial_Y + .001)); - RPI_LOG("Initially Y " << initial_Y << " target " << target_Y - << " gives gain " << gain); + LOG(RPiAgc, Debug) << "Initially Y " << initial_Y << " target " << target_Y + << " gives gain " << gain; for (auto &c : *constraint_mode_) { double new_target_Y; double new_gain = constraint_compute_gain(c, h, lux.lux, ev_gain, new_target_Y); - RPI_LOG("Constraint has target_Y " - << new_target_Y << " giving gain " << new_gain); + LOG(RPiAgc, Debug) << "Constraint has target_Y " + << new_target_Y << " giving gain " << new_gain; if (c.bound == AgcConstraint::Bound::LOWER && new_gain > gain) { - RPI_LOG("Lower bound constraint adopted"); + LOG(RPiAgc, Debug) << "Lower bound constraint adopted"; gain = new_gain, target_Y = new_target_Y; } else if (c.bound == AgcConstraint::Bound::UPPER && new_gain < gain) { - RPI_LOG("Upper bound constraint adopted"); + LOG(RPiAgc, Debug) << "Upper bound constraint adopted"; gain = new_gain, target_Y = new_target_Y; } } - RPI_LOG("Final gain " << gain << " (target_Y " << target_Y << " ev " - << status_.ev << " base_ev " << config_.base_ev - << ")"); + LOG(RPiAgc, Debug) << "Final gain " << gain << " (target_Y " << target_Y << " ev " + << status_.ev << " base_ev " << config_.base_ev + << ")"; } void Agc::computeTargetExposure(double gain) @@ -494,7 +498,7 @@ void Agc::computeTargetExposure(double gain) : exposure_mode_->gain.back()); target_.total_exposure = std::min(target_.total_exposure, max_total_exposure); - RPI_LOG("Target total_exposure " << target_.total_exposure); + LOG(RPiAgc, Debug) << "Target total_exposure " << target_.total_exposure; } bool Agc::applyDigitalGain(Metadata *image_metadata, double gain, @@ -509,9 +513,9 @@ bool Agc::applyDigitalGain(Metadata *image_metadata, double gain, std::min(awb.gain_g, awb.gain_b)); dg *= std::max(1.0, 1.0 / min_gain); } else - RPI_WARN("Agc: no AWB status found"); - RPI_LOG("after AWB, target dg " << dg << " gain " << gain - << " target_Y " << target_Y); + LOG(RPiAgc, Warning) << "Agc: no AWB status found"; + LOG(RPiAgc, Debug) << "after AWB, target dg " << dg << " gain " << gain + << " target_Y " << target_Y; // Finally, if we're trying to reduce exposure but the target_Y is // "close" to 1.0, then the gain computed for that constraint will be // only slightly less than one, because the measured Y can never be @@ -523,9 +527,9 @@ bool Agc::applyDigitalGain(Metadata *image_metadata, double gain, gain < sqrt(target_Y); if (desaturate) dg /= config_.fast_reduce_threshold; - RPI_LOG("Digital gain " << dg << " desaturate? " << desaturate); + LOG(RPiAgc, Debug) << "Digital gain " << dg << " desaturate? " << desaturate; target_.total_exposure_no_dg = target_.total_exposure / dg; - RPI_LOG("Target total_exposure_no_dg " << target_.total_exposure_no_dg); + LOG(RPiAgc, Debug) << "Target total_exposure_no_dg " << target_.total_exposure_no_dg; return desaturate; } @@ -560,8 +564,8 @@ void Agc::filterExposure(bool desaturate) filtered_.total_exposure * config_.fast_reduce_threshold) filtered_.total_exposure_no_dg = filtered_.total_exposure * config_.fast_reduce_threshold; - RPI_LOG("After filtering, total_exposure " << filtered_.total_exposure << - " no dg " << filtered_.total_exposure_no_dg); + LOG(RPiAgc, Debug) << "After filtering, total_exposure " << filtered_.total_exposure + << " no dg " << filtered_.total_exposure_no_dg; } void Agc::divvyupExposure() @@ -602,8 +606,8 @@ void Agc::divvyupExposure() } } } - RPI_LOG("Divided up shutter and gain are " << shutter_time << " and " - << analogue_gain); + LOG(RPiAgc, Debug) << "Divided up shutter and gain are " << shutter_time << " and " + << analogue_gain; // Finally adjust shutter time for flicker avoidance (require both // shutter and gain not to be fixed). if (status_.fixed_shutter == 0.0 && @@ -621,8 +625,8 @@ void Agc::divvyupExposure() exposure_mode_->gain.back()); shutter_time = new_shutter_time; } - RPI_LOG("After flicker avoidance, shutter " - << shutter_time << " gain " << analogue_gain); + LOG(RPiAgc, Debug) << "After flicker avoidance, shutter " + << shutter_time << " gain " << analogue_gain; } filtered_.shutter = shutter_time; filtered_.analogue_gain = analogue_gain; @@ -641,10 +645,10 @@ void Agc::writeAndFinish(Metadata *image_metadata, bool desaturate) // Write to metadata as well, in case anyone wants to update the camera // immediately. image_metadata->Set("agc.status", status_); - RPI_LOG("Output written, total exposure requested is " - << filtered_.total_exposure); - RPI_LOG("Camera exposure update: shutter time " << filtered_.shutter << - " analogue gain " << filtered_.analogue_gain); + LOG(RPiAgc, Debug) << "Output written, total exposure requested is " + << filtered_.total_exposure; + LOG(RPiAgc, Debug) << "Camera exposure update: shutter time " << filtered_.shutter + << " analogue gain " << filtered_.analogue_gain; } // Register algorithm with the system. From patchwork Mon Nov 16 16:49:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10431 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4F079BE082 for ; Mon, 16 Nov 2020 16:49:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 10D80632D4; Mon, 16 Nov 2020 17:49:29 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="is8Te22X"; dkim-atps=neutral Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DF2C3632C9 for ; Mon, 16 Nov 2020 17:49:26 +0100 (CET) Received: by mail-wr1-x433.google.com with SMTP id p8so19427387wrx.5 for ; Mon, 16 Nov 2020 08:49:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fnx644/4e1aclgbmGPuB8wpXNNlNgk4rgNHPuNiIVwo=; b=is8Te22X41nTPWaQpy0bAQKCkBbatmToD+4Z+ORZ6yLkpcJ0sAOnL+idNU7cuSjIWE my1vYGYdBZ8helWF088826vEZXtFuUieTlpowcPnXEd3ZcyK6MeuciO5LidIch93JE1y 6khLEnsSQHDS1liB58YTD4O4PY/f/HbW5qR+/FmCzZ73AYkTLOu4CW7SVzqRaCJXIjNm MR8sgclWe3bvkwoK1d83KLHMBqo3NlEvjBmiZBW7Rk2G0PMSQwBCUtmJJBYUyUn8z11o DY1Y9JN8f1HDF4WJIqLorSKC5y3tZ3WRJDYotxogQtCpAZt0tX1Jx8vX1WynRba3WRQi 2jjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fnx644/4e1aclgbmGPuB8wpXNNlNgk4rgNHPuNiIVwo=; b=niecWTKt2izCRtNtxvemMxIbHvrei0gzkFWSyVERK8j5fkSjzUS1jHN3/132cNNJkg lhn4nEI/9sBUPxRUPKx8D01PHuyESEChbBkntrFOZnuO1L1j8n2hFcsXc4NxsUCxfSs+ RVg/+2cz92B7cP9HvzXMUqYt3F0X04FIComKMaQP4yA98BOILPFthAop4Cmt02uuF0nZ gJn2/l8PFeucAJXeHpR1bOJ6eSR6u7/2n2XOBmj/CJm8KZRUuON+jhOUJPnreyr68vah qnb1JsFpZLIp4Jfj5/fAQO9dIwfXxM91U+/VPA3vtTQdzbjU2nfsMCpgoiZ5BhG1zMXz ZRGQ== X-Gm-Message-State: AOAM533iWy5dij6CbqOmpC/Ve5f1OYjrqt5mwGbQUotaGCOL/7QrViBk zmX0zgjiFvlF/+vVMEWM/NTXue03liCZNA== X-Google-Smtp-Source: ABdhPJwAGHDfxAlPViJZus04GoHVbCy2opSke3vwWf4/PUWVOeJb2pSeSLwBXpRcoludENWDxd9EsQ== X-Received: by 2002:adf:f906:: with SMTP id b6mr20211716wrr.244.1605545366052; Mon, 16 Nov 2020 08:49:26 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id q16sm23716973wrn.13.2020.11.16.08.49.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 08:49:24 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 16 Nov 2020 16:49:10 +0000 Message-Id: <20201116164918.2055-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201116164918.2055-1-david.plowman@raspberrypi.com> References: <20201116164918.2055-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 02/10] libcamera: ipa: raspberrypi: agc: Remove unnecessary locking X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" On the libcamera/VC4 platform the AGC Prepare/Process methods, and any changes to the AGC settings, run synchronously - so a number of mutexes and copies are unnecessary and can be removed. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 99 ++++++++-------------- src/ipa/raspberrypi/controller/rpi/agc.hpp | 5 +- 2 files changed, 37 insertions(+), 67 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 8079345b..c4379b99 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -160,7 +160,6 @@ Agc::Agc(Controller *controller) status_.total_exposure_value = 0.0; status_.target_exposure_value = 0.0; status_.locked = false; - output_status_ = status_; } char const *Agc::Name() const @@ -185,43 +184,36 @@ void Agc::Read(boost::property_tree::ptree const ¶ms) void Agc::SetEv(double ev) { - std::unique_lock lock(settings_mutex_); ev_ = ev; } void Agc::SetFlickerPeriod(double flicker_period) { - std::unique_lock lock(settings_mutex_); flicker_period_ = flicker_period; } void Agc::SetFixedShutter(double fixed_shutter) { - std::unique_lock lock(settings_mutex_); fixed_shutter_ = fixed_shutter; } void Agc::SetFixedAnalogueGain(double fixed_analogue_gain) { - std::unique_lock lock(settings_mutex_); fixed_analogue_gain_ = fixed_analogue_gain; } void Agc::SetMeteringMode(std::string const &metering_mode_name) { - std::unique_lock lock(settings_mutex_); metering_mode_name_ = metering_mode_name; } void Agc::SetExposureMode(std::string const &exposure_mode_name) { - std::unique_lock lock(settings_mutex_); exposure_mode_name_ = exposure_mode_name; } void Agc::SetConstraintMode(std::string const &constraint_mode_name) { - std::unique_lock lock(settings_mutex_); constraint_mode_name_ = constraint_mode_name; } @@ -240,14 +232,9 @@ void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, void Agc::Prepare(Metadata *image_metadata) { - AgcStatus status; - { - std::unique_lock lock(output_mutex_); - status = output_status_; - } int lock_count = lock_count_; lock_count_ = 0; - status.digital_gain = 1.0; + status_.digital_gain = 1.0; if (status_.total_exposure_value) { // Process has run, so we have meaningful values. DeviceStatus device_status; @@ -255,48 +242,46 @@ void Agc::Prepare(Metadata *image_metadata) double actual_exposure = device_status.shutter_speed * device_status.analogue_gain; if (actual_exposure) { - status.digital_gain = + status_.digital_gain = status_.total_exposure_value / actual_exposure; LOG(RPiAgc, Debug) << "Want total exposure " << status_.total_exposure_value; // Never ask for a gain < 1.0, and also impose // some upper limit. Make it customisable? - status.digital_gain = std::max( + status_.digital_gain = std::max( 1.0, - std::min(status.digital_gain, 4.0)); + std::min(status_.digital_gain, 4.0)); LOG(RPiAgc, Debug) << "Actual exposure " << actual_exposure; - LOG(RPiAgc, Debug) << "Use digital_gain " << status.digital_gain; - LOG(RPiAgc, Debug) << "Effective exposure " << actual_exposure * status.digital_gain; + LOG(RPiAgc, Debug) << "Use digital_gain " << status_.digital_gain; + LOG(RPiAgc, Debug) << "Effective exposure " << actual_exposure * status_.digital_gain; // Decide whether AEC/AGC has converged. // Insist AGC is steady for MAX_LOCK_COUNT // frames before we say we are "locked". // (The hard-coded constants may need to // become customisable.) - if (status.target_exposure_value) { + if (status_.target_exposure_value) { #define MAX_LOCK_COUNT 3 - double err = 0.10 * status.target_exposure_value + 200; + double err = 0.10 * status_.target_exposure_value + 200; if (actual_exposure < - status.target_exposure_value + err - && actual_exposure > - status.target_exposure_value - err) + status_.target_exposure_value + err && + actual_exposure > + status_.target_exposure_value - err) lock_count_ = std::min(lock_count + 1, - MAX_LOCK_COUNT); + MAX_LOCK_COUNT); else if (actual_exposure < - status.target_exposure_value - + 1.5 * err && + status_.target_exposure_value + 1.5 * err && actual_exposure > - status.target_exposure_value - - 1.5 * err) + status_.target_exposure_value - 1.5 * err) lock_count_ = lock_count; LOG(RPiAgc, Debug) << "Lock count: " << lock_count_; } } } else LOG(RPiAgc, Debug) << Name() << ": no device metadata"; - status.locked = lock_count_ >= MAX_LOCK_COUNT; - //printf("%s\n", status.locked ? "+++++++++" : "-"); - image_metadata->Set("agc.status", status); + status_.locked = lock_count_ >= MAX_LOCK_COUNT; + //printf("%s\n", status_.locked ? "+++++++++" : "-"); + image_metadata->Set("agc.status", status_); } } @@ -335,55 +320,47 @@ static void copy_string(std::string const &s, char *d, size_t size) void Agc::housekeepConfig() { // First fetch all the up-to-date settings, so no one else has to do it. - std::string new_exposure_mode_name, new_constraint_mode_name, - new_metering_mode_name; - { - std::unique_lock lock(settings_mutex_); - new_metering_mode_name = metering_mode_name_; - new_exposure_mode_name = exposure_mode_name_; - new_constraint_mode_name = constraint_mode_name_; - status_.ev = ev_; - status_.fixed_shutter = fixed_shutter_; - status_.fixed_analogue_gain = fixed_analogue_gain_; - status_.flicker_period = flicker_period_; - } + status_.ev = ev_; + status_.fixed_shutter = fixed_shutter_; + status_.fixed_analogue_gain = fixed_analogue_gain_; + status_.flicker_period = flicker_period_; LOG(RPiAgc, Debug) << "ev " << status_.ev << " fixed_shutter " << status_.fixed_shutter << " fixed_analogue_gain " << status_.fixed_analogue_gain; // Make sure the "mode" pointers point to the up-to-date things, if // they've changed. - if (strcmp(new_metering_mode_name.c_str(), status_.metering_mode)) { - auto it = config_.metering_modes.find(new_metering_mode_name); + if (strcmp(metering_mode_name_.c_str(), status_.metering_mode)) { + auto it = config_.metering_modes.find(metering_mode_name_); if (it == config_.metering_modes.end()) throw std::runtime_error("Agc: no metering mode " + - new_metering_mode_name); + metering_mode_name_); metering_mode_ = &it->second; - copy_string(new_metering_mode_name, status_.metering_mode, + copy_string(metering_mode_name_, status_.metering_mode, sizeof(status_.metering_mode)); } - if (strcmp(new_exposure_mode_name.c_str(), status_.exposure_mode)) { - auto it = config_.exposure_modes.find(new_exposure_mode_name); + if (strcmp(exposure_mode_name_.c_str(), status_.exposure_mode)) { + auto it = config_.exposure_modes.find(exposure_mode_name_); if (it == config_.exposure_modes.end()) throw std::runtime_error("Agc: no exposure profile " + - new_exposure_mode_name); + exposure_mode_name_); exposure_mode_ = &it->second; - copy_string(new_exposure_mode_name, status_.exposure_mode, + copy_string(exposure_mode_name_, status_.exposure_mode, sizeof(status_.exposure_mode)); } - if (strcmp(new_constraint_mode_name.c_str(), status_.constraint_mode)) { + if (strcmp(constraint_mode_name_.c_str(), status_.constraint_mode)) { auto it = - config_.constraint_modes.find(new_constraint_mode_name); + config_.constraint_modes.find(constraint_mode_name_); if (it == config_.constraint_modes.end()) throw std::runtime_error("Agc: no constraint list " + - new_constraint_mode_name); + constraint_mode_name_); constraint_mode_ = &it->second; - copy_string(new_constraint_mode_name, status_.constraint_mode, + copy_string(constraint_mode_name_, status_.constraint_mode, sizeof(status_.constraint_mode)); } LOG(RPiAgc, Debug) << "exposure_mode " - << new_exposure_mode_name << " constraint_mode " - << new_constraint_mode_name << " metering_mode " - << new_metering_mode_name; + << exposure_mode_name_ << " constraint_mode " + << constraint_mode_name_ << " metering_mode " + << metering_mode_name_; } void Agc::fetchCurrentExposure(Metadata *image_metadata) @@ -638,10 +615,6 @@ void Agc::writeAndFinish(Metadata *image_metadata, bool desaturate) status_.target_exposure_value = desaturate ? 0 : target_.total_exposure_no_dg; status_.shutter_time = filtered_.shutter; status_.analogue_gain = filtered_.analogue_gain; - { - std::unique_lock lock(output_mutex_); - output_status_ = status_; - } // Write to metadata as well, in case anyone wants to update the camera // immediately. image_metadata->Set("agc.status", status_); diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index ba7ae092..5a02df4e 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -106,12 +106,9 @@ private: ExposureValues current_; // values for the current frame ExposureValues target_; // calculate the values we want here ExposureValues filtered_; // these values are filtered towards target - AgcStatus status_; // to "latch" settings so they can't change - AgcStatus output_status_; // the status we will write out - std::mutex output_mutex_; + AgcStatus status_; int lock_count_; // Below here the "settings" that applications can change. - std::mutex settings_mutex_; std::string metering_mode_name_; std::string exposure_mode_name_; std::string constraint_mode_name_; From patchwork Mon Nov 16 16:49:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10432 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CEE34BE082 for ; Mon, 16 Nov 2020 16:49:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8B5C9632D5; Mon, 16 Nov 2020 17:49:29 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="GNN1Gkod"; dkim-atps=neutral Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A72B9632C8 for ; Mon, 16 Nov 2020 17:49:27 +0100 (CET) Received: by mail-wm1-x32b.google.com with SMTP id c9so24343733wml.5 for ; Mon, 16 Nov 2020 08:49:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xqNctcBDmlL2x/krLkjaVmnpGJIy8M2P+aQ68PdFEbc=; b=GNN1GkodywIzjUO31ylKH+L9oKeuTQ3B2Nuk4PQLjNbYegE93B+GQuGoWhEd9w4+1L lvsI+S0g+3WJUoyXgpoLTfnwTZZTbnUeSgvFklk96IlXd5JmSkuGNP4I96anbwPE4y+e GcFoE5ftb9yv5tKTeHp693PtMogAM1GQZ8aA0NlVygu0/NWgBKB7zfaHXGDj4TAbCEiY jw2vvbGfO8STfL2LYheAMqpZcJTG3YkIvvXW29DcvZFnlAqUSNw2UEr6svuMrCcBG9D8 pXUQje/92FurOe5Z8PlOcz14U4NUS1qV8XI7mhW3WZVpD6FN2f3oEBNikwsKwXMESEdg EK/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xqNctcBDmlL2x/krLkjaVmnpGJIy8M2P+aQ68PdFEbc=; b=iBQwOAoqTgeI4BkJfMRIQG/lo16ryy3ok/BOi2ZLQv50VVArYEbMwSQnnQX9uhC/3R QR+/CAoqfjlEjTXqi6M5R0H0xknk4vxbAsDt4mInhUygNZuyuFdW4NGk5rnkT5tDdtdQ 1wTThtTrbVREcqrLJeUsZ5+/B03U2NDlO1/0hUWZw+5jfsp4UTRnzeqQk2haLx7Rddy0 AXm9IOnXThV4oDA2NbuAasoUcydvt673f6hgaoRSX0F0Det0Jb6c59Vi8CJhaxE2+UBF dFZeoEFa/r2WQHK6Gz+h4ugEsWs8l4i1mI4FB51IrCcwyXV3Wledwf+LNtaQPY74q9U0 uxdQ== X-Gm-Message-State: AOAM53382qP+SG2bEwQrLKi2a2Xi8GlRTZy8kq9zoUdEmLA0JQ5sgXvq N9o3cF8OFUdAJLWQTQCqzVZSAMGHq30/MA== X-Google-Smtp-Source: ABdhPJyGBt+eIZh1+23sLrpAA+yi2KhKL1E0+q1wAjVpLx9OsMtjjqcqPYgHOofu2DEi33BZe/db8g== X-Received: by 2002:a1c:99d3:: with SMTP id b202mr16116727wme.0.1605545367120; Mon, 16 Nov 2020 08:49:27 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id q16sm23716973wrn.13.2020.11.16.08.49.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 08:49:26 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 16 Nov 2020 16:49:11 +0000 Message-Id: <20201116164918.2055-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201116164918.2055-1-david.plowman@raspberrypi.com> References: <20201116164918.2055-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 03/10] libcamera: ipa: raspberrypi: agc: Rename method to divideUpExposure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The method formerly known as divvyupExposure is given a more understandable name. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 8 ++++---- src/ipa/raspberrypi/controller/rpi/agc.hpp | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index c4379b99..d29b1156 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -225,7 +225,7 @@ void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, // write the results into the metadata we've been given. if (status_.total_exposure_value) { housekeepConfig(); - divvyupExposure(); + divideUpExposure(); writeAndFinish(metadata, false); } } @@ -304,9 +304,9 @@ void Agc::Process(StatisticsPtr &stats, Metadata *image_metadata) bool desaturate = applyDigitalGain(image_metadata, gain, target_Y); // The results have to be filtered so as not to change too rapidly. filterExposure(desaturate); - // The last thing is to divvy up the exposure value into a shutter time + // The last thing is to divide up the exposure value into a shutter time // and analogue_gain, according to the current exposure mode. - divvyupExposure(); + divideUpExposure(); // Finally advertise what we've done. writeAndFinish(image_metadata, desaturate); } @@ -545,7 +545,7 @@ void Agc::filterExposure(bool desaturate) << " no dg " << filtered_.total_exposure_no_dg; } -void Agc::divvyupExposure() +void Agc::divideUpExposure() { // Sending the fixed shutter/gain cases through the same code may seem // unnecessary, but it will make more sense when extend this to cover diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index 5a02df4e..2442fc03 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -89,7 +89,7 @@ private: bool applyDigitalGain(Metadata *image_metadata, double gain, double target_Y); void filterExposure(bool desaturate); - void divvyupExposure(); + void divideUpExposure(); void writeAndFinish(Metadata *image_metadata, bool desaturate); AgcMeteringMode *metering_mode_; AgcExposureMode *exposure_mode_; From patchwork Mon Nov 16 16:49:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10433 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 73F11BE082 for ; Mon, 16 Nov 2020 16:49:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 347DB632DB; Mon, 16 Nov 2020 17:49:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="XsgOcAoZ"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 99E0D632CA for ; Mon, 16 Nov 2020 17:49:28 +0100 (CET) Received: by mail-wm1-x329.google.com with SMTP id d142so24363792wmd.4 for ; Mon, 16 Nov 2020 08:49:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FX6HZ/72E1uEy6eG+0sP6T7FgtQ8d+tG9LreXqCwQJA=; b=XsgOcAoZ9Zhw30bzQUzHkrWo4WxJWpZh0YHGZbxZDlDfe7WuC3wYRZtFumySiOkz1z 1l6yHgzLiTTKvgmxOX1fBa633xlNYdLrHi3FcfHPFAZiitvNr8/0HW1OU/CqwIr/qE/g pTV2s1mF3jJ7+15NEbWEi6E9BHvg/bbRj6wqrFKiqolEufz7Jn6Qqr1NlADj1FX8dB67 a6aJTUUFQHuZKo3epS5676sXzKJVRQAvuRQlsRTwK560ERCsRjUOgI8MhOwVp1q/bCHx aywGZN4XKdjChuFy4iUw67MDqzCROCT7m9hf4+nZv8nleNS/QuQIEcrsHARZwDdCptf4 Oj0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FX6HZ/72E1uEy6eG+0sP6T7FgtQ8d+tG9LreXqCwQJA=; b=c34JGJAQdHdiTXo4ou869UMlew+y2HDhfRRKJWOyehWeHiBSe9ks9rcrTxOZR6cmjG iCRVzO/cqO+Hs1gasfKH3VVejxWot1oc+o5N0oefSMk4R/Kd7ITEwCfv3071JiEfRScW 4VkngxEHfSMQJo/cv7UEpjdHjsIuvT306CIvjZzDB9YjNBdiLeNafmR+HSFjRfahpHPF axJ3GqK9zVm5fiEPRcxFuILHQ6IXmOlXGWqCIH12ol+DAnmcZ1kJhZWwhKE9hZQsNn9k JI24HW3iiktbxRoxHl3KjWruK3B/IYK4NYCSglU4AXnTFTnclOe/UBBctobKZsNlSpJQ TiWQ== X-Gm-Message-State: AOAM533zokIk32jOqNIaT417sMNoC0lluhrL12RmppIfvhnjwOqEEVIj XsaLMHNZlNCCiXszDXnCL5/SP89IgF80QQ== X-Google-Smtp-Source: ABdhPJzYCQu+19OTMC4l0xpWn9ykV9KHJJ9Vq1j/BP2TxpVcayNJjJS//mQ2L0/QhaSuWYRz/MnuVw== X-Received: by 2002:a1c:7c02:: with SMTP id x2mr12803544wmc.64.1605545368018; Mon, 16 Nov 2020 08:49:28 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id q16sm23716973wrn.13.2020.11.16.08.49.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 08:49:27 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 16 Nov 2020 16:49:12 +0000 Message-Id: <20201116164918.2055-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201116164918.2055-1-david.plowman@raspberrypi.com> References: <20201116164918.2055-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 04/10] libcamera: ipa: raspberrypi: agc: Improve centre-weighted luminance calucation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Previously the calculation computed Y for each region before returning the weighted average, which "baked in" the over-importance of small statistics regions. The revised calculation will treat all pixels equally when the region weights are the same, making it easier to use. With the previous scheme, proper "average" metering was difficult to implement. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 25 +++++++++++++--------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index d29b1156..ead28398 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -386,18 +386,23 @@ static double compute_initial_Y(bcm2835_isp_stats *stats, Metadata *image_metada awb.gain_r = awb.gain_g = awb.gain_b = 1.0; // in case no metadata if (image_metadata->Get("awb.status", awb) != 0) LOG(RPiAgc, Warning) << "Agc: no AWB status found"; - double Y_sum = 0, weight_sum = 0; + // Note how the calculation below means that equal weights give you + // "average" metering (i.e. all pixels equally important). + double R_sum = 0, G_sum = 0, B_sum = 0, pixel_sum = 0; for (int i = 0; i < AGC_STATS_SIZE; i++) { - if (regions[i].counted == 0) - continue; - weight_sum += weights[i]; - double Y = regions[i].r_sum * awb.gain_r * .299 + - regions[i].g_sum * awb.gain_g * .587 + - regions[i].b_sum * awb.gain_b * .114; - Y /= regions[i].counted; - Y_sum += Y * weights[i]; + R_sum += regions[i].r_sum * weights[i]; + G_sum += regions[i].g_sum * weights[i]; + B_sum += regions[i].b_sum * weights[i]; + pixel_sum += regions[i].counted * weights[i]; } - return Y_sum / weight_sum / (1 << PIPELINE_BITS); + if (pixel_sum == 0.0) { + LOG(RPiAgc, Warning) << "compute_initial_Y: pixel_sum is zero"; + return 0; + } + double Y_sum = R_sum * awb.gain_r * .299 + + G_sum * awb.gain_g * .587 + + B_sum * awb.gain_b * .114; + return Y_sum / pixel_sum / (1 << PIPELINE_BITS); } // We handle extra gain through EV by adjusting our Y targets. However, you From patchwork Mon Nov 16 16:49:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10434 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D0303BE082 for ; Mon, 16 Nov 2020 16:49:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9EF39632DE; Mon, 16 Nov 2020 17:49:32 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="NidqvYhV"; dkim-atps=neutral Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A6668632D8 for ; Mon, 16 Nov 2020 17:49:29 +0100 (CET) Received: by mail-wr1-x436.google.com with SMTP id j7so19410086wrp.3 for ; Mon, 16 Nov 2020 08:49:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3jOTqsdM+5vSReHOduxhhHeWX+jcNQzjs9vGsaKAjj8=; b=NidqvYhVeTQ16nbibmCfTfc8gH3CR+macAriZ0dsfCosr1dbDfzlnANSHKhtRb9x5q QqrSpX6WtBeVHbujcaasWrU9EF9a7v2CZM1HzTdAixVGHo9gRdyIUvAaW1bcjxqf2+Mw 8sJfec3CWnfcABuD3SAWA/NLbBmzLPjB4NevLeD5DUyD89olhpmdRLkYmtfaDtVdwqpu w1mx6LeLpzsgXP3mScAnLLH6kb5KAjAArTUA4axdXehE5EEg7g3kD6FCbJxI6PGsWF5P Fb344FX/E9sKwe9LFkeod99pnB94y12q/RsB26O1QrOb3gCKpHAWRZWVg4/KP8/byIM2 KIcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3jOTqsdM+5vSReHOduxhhHeWX+jcNQzjs9vGsaKAjj8=; b=PRIOU44nijD2JKVoG/3WvKwDc2k1FLXbpFEufoSArf6LnsQU0xfZPibdfFz1XYUnF0 UUa92j6N/dVI/Js7NancrqfjMhNlqb5UXixu+XPDg5Rzk2vjBCndAyCGBonSH+GRo0QF hDqewKXf33gL9Atodft3mTdHhDpGkKEuPNQjE+caacSBaZcKg/56afbcquRqyeESIute gwU53xOHbTNvOJz0GFN6U052WiyVgfuvvJ28l45bDzIEnJQTRf8g1Tw0xqMibBYxM1qE hqWdrFHXKs956hoA0od9pAZc1ynwUwGo/+LNQn0YOt7FQFCKqz4ME5wr2cLcxc/ujkAq GVMw== X-Gm-Message-State: AOAM533ybtmjC6XXZRn3/xtgdjPt0OTNiWQ1Opk1mBoUw7dUrGYtbSDE vGFAJAWDnTxS+OpFCVPT+FVvTP4rpeYD3w== X-Google-Smtp-Source: ABdhPJwA/o677mH523dHvj34Pah/tT3vRGIqXOvqVLXQj6UMJghFfsr7uWFGGCC+dYOBaD2uZkIpCQ== X-Received: by 2002:adf:e88b:: with SMTP id d11mr19378694wrm.4.1605545369068; Mon, 16 Nov 2020 08:49:29 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id q16sm23716973wrn.13.2020.11.16.08.49.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 08:49:28 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 16 Nov 2020 16:49:13 +0000 Message-Id: <20201116164918.2055-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201116164918.2055-1-david.plowman@raspberrypi.com> References: <20201116164918.2055-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 05/10] libcamera: ipa: raspberrypi: agc: Fetch AWB status only once X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Introduce a function to fetch the AwbStatus (fetchAwbStatus), and call it unconditionally at the top of Prepare so that both Prepare and Process know thereafter that it's been done. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 37 ++++++++++++---------- src/ipa/raspberrypi/controller/rpi/agc.hpp | 5 +-- 2 files changed, 23 insertions(+), 19 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index ead28398..6de56def 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -5,6 +5,8 @@ * agc.cpp - AGC/AEC control algorithm */ +#include + #include #include "linux/bcm2835-isp.h" @@ -235,6 +237,8 @@ void Agc::Prepare(Metadata *image_metadata) int lock_count = lock_count_; lock_count_ = 0; status_.digital_gain = 1.0; + fetchAwbStatus(image_metadata); // always fetch it so that Process knows it's been done + if (status_.total_exposure_value) { // Process has run, so we have meaningful values. DeviceStatus device_status; @@ -301,7 +305,7 @@ void Agc::Process(StatisticsPtr &stats, Metadata *image_metadata) // Some of the exposure has to be applied as digital gain, so work out // what that is. This function also tells us whether it's decided to // "desaturate" the image more quickly. - bool desaturate = applyDigitalGain(image_metadata, gain, target_Y); + bool desaturate = applyDigitalGain(gain, target_Y); // The results have to be filtered so as not to change too rapidly. filterExposure(desaturate); // The last thing is to divide up the exposure value into a shutter time @@ -378,14 +382,19 @@ void Agc::fetchCurrentExposure(Metadata *image_metadata) current_.total_exposure_no_dg = current_.shutter * current_.analogue_gain; } -static double compute_initial_Y(bcm2835_isp_stats *stats, Metadata *image_metadata, +void Agc::fetchAwbStatus(Metadata *image_metadata) +{ + awb_.gain_r = 1.0; // in case not found in metadata + awb_.gain_g = 1.0; + awb_.gain_b = 1.0; + if (image_metadata->Get("awb.status", awb_) != 0) + LOG(RPiAgc, Warning) << "Agc: no AWB status found"; +} + +static double compute_initial_Y(bcm2835_isp_stats *stats, AwbStatus const &awb, double weights[]) { bcm2835_isp_stats_region *regions = stats->agc_stats; - struct AwbStatus awb; - awb.gain_r = awb.gain_g = awb.gain_b = 1.0; // in case no metadata - if (image_metadata->Get("awb.status", awb) != 0) - LOG(RPiAgc, Warning) << "Agc: no AWB status found"; // Note how the calculation below means that equal weights give you // "average" metering (i.e. all pixels equally important). double R_sum = 0, G_sum = 0, B_sum = 0, pixel_sum = 0; @@ -437,7 +446,7 @@ void Agc::computeGain(bcm2835_isp_stats *statistics, Metadata *image_metadata, target_Y = config_.Y_target.Eval(config_.Y_target.Domain().Clip(lux.lux)); target_Y = std::min(EV_GAIN_Y_TARGET_LIMIT, target_Y * ev_gain); - double initial_Y = compute_initial_Y(statistics, image_metadata, + double initial_Y = compute_initial_Y(statistics, awb_, metering_mode_->weights); gain = std::min(10.0, target_Y / (initial_Y + .001)); LOG(RPiAgc, Debug) << "Initially Y " << initial_Y << " target " << target_Y @@ -483,19 +492,13 @@ void Agc::computeTargetExposure(double gain) LOG(RPiAgc, Debug) << "Target total_exposure " << target_.total_exposure; } -bool Agc::applyDigitalGain(Metadata *image_metadata, double gain, - double target_Y) +bool Agc::applyDigitalGain(double gain, double target_Y) { - double dg = 1.0; + double min_colour_gain = std::min({ awb_.gain_r, awb_.gain_g, awb_.gain_b, 1.0 }); + assert(min_colour_gain != 0.0); + double dg = 1.0 / min_colour_gain; // I think this pipeline subtracts black level and rescales before we // get the stats, so no need to worry about it. - struct AwbStatus awb; - if (image_metadata->Get("awb.status", awb) == 0) { - double min_gain = std::min(awb.gain_r, - std::min(awb.gain_g, awb.gain_b)); - dg *= std::max(1.0, 1.0 / min_gain); - } else - LOG(RPiAgc, Warning) << "Agc: no AWB status found"; LOG(RPiAgc, Debug) << "after AWB, target dg " << dg << " gain " << gain << " target_Y " << target_Y; // Finally, if we're trying to reduce exposure but the target_Y is diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index 2442fc03..e7ac480f 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -83,11 +83,11 @@ private: AgcConfig config_; void housekeepConfig(); void fetchCurrentExposure(Metadata *image_metadata); + void fetchAwbStatus(Metadata *image_metadata); void computeGain(bcm2835_isp_stats *statistics, Metadata *image_metadata, double &gain, double &target_Y); void computeTargetExposure(double gain); - bool applyDigitalGain(Metadata *image_metadata, double gain, - double target_Y); + bool applyDigitalGain(double gain, double target_Y); void filterExposure(bool desaturate); void divideUpExposure(); void writeAndFinish(Metadata *image_metadata, bool desaturate); @@ -95,6 +95,7 @@ private: AgcExposureMode *exposure_mode_; AgcConstraintMode *constraint_mode_; uint64_t frame_count_; + AwbStatus awb_; struct ExposureValues { ExposureValues() : shutter(0), analogue_gain(0), total_exposure(0), total_exposure_no_dg(0) {} From patchwork Mon Nov 16 16:49:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10435 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 47383BE082 for ; Mon, 16 Nov 2020 16:49:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1769B632C8; Mon, 16 Nov 2020 17:49:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="dnHuGQ6+"; dkim-atps=neutral Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 99F78632D7 for ; Mon, 16 Nov 2020 17:49:30 +0100 (CET) Received: by mail-wm1-x32b.google.com with SMTP id w24so24214662wmi.0 for ; Mon, 16 Nov 2020 08:49:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aisa3uaRSaovPCtBa0hhRtIHRsyDdDRuZE0+SvpTmd4=; b=dnHuGQ6+EqVuy7vXu6PNA2i9Xmrufe7M7u4fnihV7ay6g2q+hmohWnUgZtxMn7mPhB zDro9fDSCUiOdc6cazpWSgY7GtHFypx13lEVsNNozoLXR2tc8Rmq82hsVCcXgFyACLvl x+1tbrmlgSq8BbvRnF/T9s/1sq5sRK2Xa5zM66qvTiYYJH2EYA7rX4x8NMyVMA9tm+C3 j8tLuGzvGHTKmEDzuH+HpEcs1ed1BWasSeheUEjGEm36LGrIkqS5PkVn8bNPsbjaYpp0 u4JKSygVOXu6KuHRNJ49dasPF2zO3vEffSJ1YU1J+GV8avMcuvMs4YHi9sxft+f2JAPf ZJRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aisa3uaRSaovPCtBa0hhRtIHRsyDdDRuZE0+SvpTmd4=; b=CoYk5PBGwJ+htFqSe7OGAZTBY5GF2KBThoZDOMyhlOEv1X/34/URcfHzbdWPqOgLSH bwxH/t8MyQV32Y2Etq4x51UVHMMKufLlTa7JMh2rQUhVlfmsNUCBvU1jdUNGnQJIBE5l ZH6mDI2QvxjQdmg1Eun5iiGXE9+HJtqnQrRqK7WS73u4XUpVkEUksMRbE+EE2s4dgEEK xOO7zuShplbczKMKkis3Yf8tIPn3rzZ34XbbC0cHWs2k1il6x9nG9FmK2b/Q11RRKCnj kv3jJPAr6KVlbbr8hAHBZB9guA/AnlJ/YPbKj66ZbNVryD+TbVrXKlTdWFr7raehbbaE uKnQ== X-Gm-Message-State: AOAM530rUGKBD5WWmHbLwyRiCuj+OopsiIiyPIRpugDu7tjGFMx/DPKF zAXMvHCm93yn79vzWcrsmzlkUqDCQJxkYA== X-Google-Smtp-Source: ABdhPJz0lTJz43rMZ2AKhCd1mGPILHMOoCgD9F4UIOAMhrTQwogF0CblH2YVmDV19dnwtEOxV4E23w== X-Received: by 2002:a1c:103:: with SMTP id 3mr16101819wmb.81.1605545370018; Mon, 16 Nov 2020 08:49:30 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id q16sm23716973wrn.13.2020.11.16.08.49.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 08:49:29 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 16 Nov 2020 16:49:14 +0000 Message-Id: <20201116164918.2055-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201116164918.2055-1-david.plowman@raspberrypi.com> References: <20201116164918.2055-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 06/10] libcamera: ipa: raspberrypi: awb: Add SwitchMode method to output AWB status X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Awb class now implements a SwitchMode method which outputs its AwbStatus for other algorithms to read, should they be interested. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/awb.cpp | 14 ++++++++++++++ src/ipa/raspberrypi/controller/rpi/awb.hpp | 1 + 2 files changed, 15 insertions(+) diff --git a/src/ipa/raspberrypi/controller/rpi/awb.cpp b/src/ipa/raspberrypi/controller/rpi/awb.cpp index 469d0e53..020825e3 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.cpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.cpp @@ -179,6 +179,20 @@ void Awb::SetManualGains(double manual_r, double manual_b) manual_b_ = manual_b; } +void Awb::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, + Metadata *metadata) +{ + // If fixed colour gains have been set, we should let other algorithms + // know by writing it into the image metadata. + if (manual_r_ != 0.0 && manual_b_ != 0.0) { + prev_sync_results_.gain_r = manual_r_; + prev_sync_results_.gain_g = 1.0; + prev_sync_results_.gain_b = manual_b_; + sync_results_ = prev_sync_results_; + } + metadata->Set("awb.status", prev_sync_results_); +} + void Awb::fetchAsyncResults() { RPI_LOG("Fetch AWB results"); diff --git a/src/ipa/raspberrypi/controller/rpi/awb.hpp b/src/ipa/raspberrypi/controller/rpi/awb.hpp index 9124d042..545d85a8 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.hpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.hpp @@ -84,6 +84,7 @@ public: void Read(boost::property_tree::ptree const ¶ms) override; void SetMode(std::string const &name) override; void SetManualGains(double manual_r, double manual_b) override; + void SwitchMode(CameraMode const &camera_mode, Metadata *metadata) override; void Prepare(Metadata *image_metadata) override; void Process(StatisticsPtr &stats, Metadata *image_metadata) override; struct RGB { From patchwork Mon Nov 16 16:49:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10436 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9A5A1BE088 for ; Mon, 16 Nov 2020 16:49:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 645FF632DC; Mon, 16 Nov 2020 17:49:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="kCOeB65J"; dkim-atps=neutral Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9A2A8632C8 for ; Mon, 16 Nov 2020 17:49:31 +0100 (CET) Received: by mail-wr1-x431.google.com with SMTP id p1so19376533wrf.12 for ; Mon, 16 Nov 2020 08:49:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pV5Y4BC+e5WO5rCf4cwaFUFTslXt0uT26nNKHvbbNQI=; b=kCOeB65Jt5AOO+LZ9MqXg2LBwzbt9vz4MJPnrD38qKvuYUsXoOM0reyPFeegtXuFcV 53KYf0eC4fguVS9+B/+/VJSFtRxp24xDso2Pj3BWRjDnbsHJ3fLCP6Ej+L8tXIbEUFGX rzZSdhLKAVdKLymt4tAx3sQbixv5sbhZlzO61JksF/hevYfosLDD25tMaIlAdiGX6TLj h9HIBYD39vPoN0irBvU1OCZDzVYl+9Q9OLim8AI5gy6ao+2zsyMaGNK5JRK0sRy1/cYD d73U7JQX3Dcp7sA6UBsIyuuMqgPaeD9vfg5Q8TK2fJBy7Fzp/Y7b4HkYjNpmLZlunvsO tWHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pV5Y4BC+e5WO5rCf4cwaFUFTslXt0uT26nNKHvbbNQI=; b=M0azA8AOhtRzIjLjjv02lBisJBzcb5J93FiU7I+YkXt9nnWC0woxd6jpqarHVsi9V2 KzPUIGIupyCGKLe0fc6pblYggYmBTDnZDeErE/IuaLcH7g7fAZq6Ej5YjU3k0PpymiHg GQQNup593r/XdQGaciKiF7eSW+RdjYYuQx8QxM/7RuiU6HrLqXAmoetIhBZ/3thq9rL5 PsqHzQUkYBtZ5jyXI7Od9Ca31XBZ2IL/uhb+6/8bCCoO5ayAa1hIZkjaR3OG0M4En7FO BXoPKWpRRmsnWc7du2KTnFTus7UCU8lBNCKIn8k1B79wwF3c7TeVtHLUizSf5qbysksx 2l9Q== X-Gm-Message-State: AOAM530nxP8Gxx18ChC6yIDIee9i3GQrf3B/1Wsy4Qhpdvx5+gmYzbFG yRW6KIt4cXTKmcov7BnNnkl3Ny9eFG9VmQ== X-Google-Smtp-Source: ABdhPJwlewiG/ptox0APnvmUg8Uu+nc+/1b6K1zXwO8TwStwe5Q9b0y2GKxzThLURE6gjCq7TVrFXQ== X-Received: by 2002:adf:db85:: with SMTP id u5mr20856672wri.155.1605545370919; Mon, 16 Nov 2020 08:49:30 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id q16sm23716973wrn.13.2020.11.16.08.49.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 08:49:30 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 16 Nov 2020 16:49:15 +0000 Message-Id: <20201116164918.2055-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201116164918.2055-1-david.plowman@raspberrypi.com> References: <20201116164918.2055-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 07/10] libcamera: ipa: raspberrypi: agc: Report fixed exposure/gain values during SwitchMode X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When an application has specified fixed exposure time and/or gain they must be programmed into the sensor immediately, even before the sensor has been started. For this to happen they must be written into the image metadata when the SwitchMode method is invoked. We also make the default exposure/gain, when nothing has been set, customisable in the tuning file. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 74 +++++++++++++++++----- src/ipa/raspberrypi/controller/rpi/agc.hpp | 2 + 2 files changed, 60 insertions(+), 16 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 6de56def..7c7944e8 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -147,6 +147,9 @@ void AgcConfig::Read(boost::property_tree::ptree const ¶ms) fast_reduce_threshold = params.get("fast_reduce_threshold", 0.4); base_ev = params.get("base_ev", 1.0); + // Start with quite a low value as ramping up is easier than ramping down. + default_exposure_time = params.get("default_exposure_time", 1000); + default_analogue_gain = params.get("default_analogue_gain", 1.0); } Agc::Agc(Controller *controller) @@ -222,14 +225,42 @@ void Agc::SetConstraintMode(std::string const &constraint_mode_name) void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, Metadata *metadata) { - // On a mode switch, it's possible the exposure profile could change, - // so we run through the dividing up of exposure/gain again and - // write the results into the metadata we've been given. - if (status_.total_exposure_value) { - housekeepConfig(); + housekeepConfig(); + + if (fixed_shutter_ != 0.0 && fixed_analogue_gain_ != 0.0) { + // We're going to reset the algorithm here with these fixed values. + + fetchAwbStatus(metadata); + double min_colour_gain = std::min({ awb_.gain_r, awb_.gain_g, awb_.gain_b, 1.0 }); + assert(min_colour_gain != 0.0); + + // This is the equivalent of computeTargetExposure and applyDigitalGain. + target_.total_exposure_no_dg = fixed_shutter_ * fixed_analogue_gain_; + target_.total_exposure = target_.total_exposure_no_dg / min_colour_gain; + + // Equivalent of filterExposure. This resets any "history". + filtered_ = target_; + + // Equivalent of divideUpExposure. + filtered_.shutter = fixed_shutter_; + filtered_.analogue_gain = fixed_analogue_gain_; + } else if (status_.total_exposure_value) { + // On a mode switch, it's possible the exposure profile could change, + // or a fixed exposure/gain might be set so we divide up the exposure/ + // gain again, but we don't change any target values. divideUpExposure(); - writeAndFinish(metadata, false); + } else { + // We come through here on startup, when at least one of the shutter + // or gain has not been fixed. We must still write those values out so + // that they will be applied immediately. We supply some arbitrary defaults + // for any that weren't set. + + // Equivalent of divideUpExposure. + filtered_.shutter = fixed_shutter_ ? fixed_shutter_ : config_.default_exposure_time; + filtered_.analogue_gain = fixed_analogue_gain_ ? fixed_analogue_gain_ : config_.default_analogue_gain; } + + writeAndFinish(metadata, false); } void Agc::Prepare(Metadata *image_metadata) @@ -475,20 +506,31 @@ void Agc::computeGain(bcm2835_isp_stats *statistics, Metadata *image_metadata, void Agc::computeTargetExposure(double gain) { - // The statistics reflect the image without digital gain, so the final - // total exposure we're aiming for is: - target_.total_exposure = current_.total_exposure_no_dg * gain; - // The final target exposure is also limited to what the exposure - // mode allows. - double max_total_exposure = - (status_.fixed_shutter != 0.0 + if (status_.fixed_shutter != 0.0 && status_.fixed_analogue_gain != 0.0) { + // When ag and shutter are both fixed, we need to drive the + // total exposure so that we end up with a digital gain of at least + // 1/min_colour_gain. Otherwise we'd desaturate channels causing + // white to go cyan or magenta. + double min_colour_gain = std::min({ awb_.gain_r, awb_.gain_g, awb_.gain_b, 1.0 }); + assert(min_colour_gain != 0.0); + target_.total_exposure = + status_.fixed_shutter * status_.fixed_analogue_gain / min_colour_gain; + } else { + // The statistics reflect the image without digital gain, so the final + // total exposure we're aiming for is: + target_.total_exposure = current_.total_exposure_no_dg * gain; + // The final target exposure is also limited to what the exposure + // mode allows. + double max_total_exposure = + (status_.fixed_shutter != 0.0 ? status_.fixed_shutter : exposure_mode_->shutter.back()) * - (status_.fixed_analogue_gain != 0.0 + (status_.fixed_analogue_gain != 0.0 ? status_.fixed_analogue_gain : exposure_mode_->gain.back()); - target_.total_exposure = std::min(target_.total_exposure, - max_total_exposure); + target_.total_exposure = std::min(target_.total_exposure, + max_total_exposure); + } LOG(RPiAgc, Debug) << "Target total_exposure " << target_.total_exposure; } diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index e7ac480f..859a9650 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -60,6 +60,8 @@ struct AgcConfig { std::string default_exposure_mode; std::string default_constraint_mode; double base_ev; + double default_exposure_time; + double default_analogue_gain; }; class Agc : public AgcAlgorithm From patchwork Mon Nov 16 16:49:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10437 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3810DBE082 for ; Mon, 16 Nov 2020 16:49:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0080E632DB; Mon, 16 Nov 2020 17:49:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="eBYKjrbk"; dkim-atps=neutral Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8025C632D4 for ; Mon, 16 Nov 2020 17:49:32 +0100 (CET) Received: by mail-wm1-x331.google.com with SMTP id a3so24196581wmb.5 for ; Mon, 16 Nov 2020 08:49:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zzxO+7DUtU4wEkTcoTYpn2fnio2NuM0p+0OqFUa+9G0=; b=eBYKjrbkEFAWUpGYGtQ0Y92mVUtanvq4Hbog9icNuX1RK5S6LzphO12ThC8xqADi3n t3fZfque1QX4aXor6cc8n3pnUT0V51MUywiN7x9d0+iystD2jrHyj1uf0DEv/YFSATiR caypXhgvBUMM1Rpb8/LfSMDdUV91uEM/MSdpmsbnJeREk5OgEvlgFDc3X4lsxinoab1O K84A50AO3DBDUCnsnjJILLD/XeSIzbGHu9vDiQnufhXoPP+FQK9zOQMw6v5RLlvbhZ8R KJGTOFkim9XPTo5aFn3ppnlF6Wugsp6eMEorloFHzf2XN3GASNxrI5n+QY8e2eiEAZfO VoDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zzxO+7DUtU4wEkTcoTYpn2fnio2NuM0p+0OqFUa+9G0=; b=HlbjWK0EKe5GVMrqo9XgB/0r337L830Q/BLADtEBYJBMIJESmXytML12J721XLDYYf Wkysgc26Wg4ABzOX4dIAKxeWTH1fs7xkqtNKVtpGI5DWZgsy0Q+mDJrMKZSEi/FLWDze wtNUg8ZD9X+FJOGtgRZrn11qiYfxXd8LUDE8TR2L/mPcS3EuzxlT5/2N9f7RGcJIioka TPflYNRvr5DwAywgf6jXMiuKps0Foi1ecH486ffHtf5qa8xX2Xy9+dOcOXqyTPj3+7hv 1xUSnPw2Btq7asv4eZoh6b7zR2ewFkh4GIC6g5weIEBlHKWMIJ+WJ9vZFK5uVI1MQq9z VLSg== X-Gm-Message-State: AOAM5315oKNQlsUeSGsOMgdwCgq3aBpEn83Ary9kqZov8uh45HGVXs5e N/F0Fi2AAbT8lherTYbdskOOVPUqNznxgQ== X-Google-Smtp-Source: ABdhPJz/aeIHZrcy4YCzqoZQMkX+gKRmpc77fkV4geVK0kHPYer4uqy0pG9pRZFSk/pMPrQVdz4A/A== X-Received: by 2002:a1c:df89:: with SMTP id w131mr15777113wmg.164.1605545371965; Mon, 16 Nov 2020 08:49:31 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id q16sm23716973wrn.13.2020.11.16.08.49.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 08:49:31 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 16 Nov 2020 16:49:16 +0000 Message-Id: <20201116164918.2055-9-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201116164918.2055-1-david.plowman@raspberrypi.com> References: <20201116164918.2055-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 08/10] libcamera: src: ipa: raspberrypi: agc: Fix uninitialised members in status_ X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use memset in the constructor, it is tidier and initialises everything. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 7c7944e8..1037f968 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -157,14 +157,13 @@ Agc::Agc(Controller *controller) exposure_mode_(nullptr), constraint_mode_(nullptr), frame_count_(0), lock_count_(0) { + // Setting status_.total_exposure_value_ to zero initially tells us + // it's not been calculated yet (i.e. Process hasn't yet run). + memset(&status_, 0, sizeof(status_)); ev_ = status_.ev = 1.0; - flicker_period_ = status_.flicker_period = 0.0; - fixed_shutter_ = status_.fixed_shutter = 0; - fixed_analogue_gain_ = status_.fixed_analogue_gain = 0.0; - // set to zero initially, so we can tell it's not been calculated - status_.total_exposure_value = 0.0; - status_.target_exposure_value = 0.0; - status_.locked = false; + flicker_period_ = 0.0; + fixed_shutter_ = 0; + fixed_analogue_gain_ = 0.0; } char const *Agc::Name() const From patchwork Mon Nov 16 16:49:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10438 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 915CDBE088 for ; Mon, 16 Nov 2020 16:49:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5C9C0632DC; Mon, 16 Nov 2020 17:49:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Rgg51cvT"; dkim-atps=neutral Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 706F1632E2 for ; Mon, 16 Nov 2020 17:49:33 +0100 (CET) Received: by mail-wr1-x42b.google.com with SMTP id k2so19457248wrx.2 for ; Mon, 16 Nov 2020 08:49:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Jr6QAPf1iUIZbFIQfkDfohO+vSzKEvj6JIaoOlTvUoc=; b=Rgg51cvTaovB8Dp6I780n9mCSQwyLhqIqRHL4gBJdQXWdPe/rXRiceR4U95RYXiz1R CrCicfSbPevVWq23m6uEVZHaUuZrBhOnJb/3Fx2K2rfn+rv1hbXakrwlImwOVj10W6BK SKLrOyj3X32Q8JTWtSqNZwqkYgoMnY40XnfgcopI3kap+/n+CNmdA8WPtSo8d/5ZxkI5 k10SuBJGbRvAyZu2AgOPyO8ArJT8pV6/kNE1HDP5obMHoDsINu4h5ookO+JU+l2nUwUK t2T4i6hSzT17jbMBz9MIsOiqqOyIcQEzPo/LV3vA5frsACTDvVmIQwtp9yOaVg4P3bm9 goLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Jr6QAPf1iUIZbFIQfkDfohO+vSzKEvj6JIaoOlTvUoc=; b=JxPL6RGrxjxyjDso9uvaOaRqw5anIpwMvVk19uBgK2xPbECY2Qd5LS0EPlo1SnBw/n GV1bNzO3ewNgjrS8Ym0KWbBMlI+dC6jNTf6n0b8SmpInOwNWu8gHMmQ1CKRUZ/s9a3Dv 4zXm5GlBnq0IdIfqekDLB3prVQSEz5tJ+P+nVK4k42bhLeePu42QhbcF3RMgShv/URIO Knz6o1MDA9R1eHgBHRH4swAJacEJY1Gj7fxpsRMb+7wGubsWcvgo8V/IsBPFxhwTHJQk skJJPJ2TDoVdczlVn1d+Dnw2ThBVZ0HNveddQ7xJa0n9bylkhb2hbl3GqIRFbbHepOXx jCCg== X-Gm-Message-State: AOAM531g33PAiljWdQJeksCuHQ44DeDvy+hhiUeCbb+d7LeDEtUMA25M oqtyF2Zo1qF1eiQBSchnuxkv0Fsq/qNiow== X-Google-Smtp-Source: ABdhPJwgfcxfcDUiDhnMMUwPAGuu/kiRsxNvwNoYcjiE7axU39hApTG1MhkH3TwID8F7cCHu2NTxfw== X-Received: by 2002:a5d:698c:: with SMTP id g12mr20066181wru.36.1605545372908; Mon, 16 Nov 2020 08:49:32 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id q16sm23716973wrn.13.2020.11.16.08.49.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 08:49:32 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 16 Nov 2020 16:49:17 +0000 Message-Id: <20201116164918.2055-10-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201116164918.2055-1-david.plowman@raspberrypi.com> References: <20201116164918.2055-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 09/10] libcamera: src: ipa: raspberrypi: agc: Improve gain update calculation for partly saturated images X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When parts of an image saturate then the image brightness no longer increases linearly with increased exposure/gain. Having calculated a linear gain value it's better then to try it, allowing for saturating regions, and if necessary increase the gain some more. We repeat this several times. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 34 +++++++++++++++------- 1 file changed, 24 insertions(+), 10 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 1037f968..93b46a28 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -422,17 +422,21 @@ void Agc::fetchAwbStatus(Metadata *image_metadata) } static double compute_initial_Y(bcm2835_isp_stats *stats, AwbStatus const &awb, - double weights[]) + double weights[], double gain) { bcm2835_isp_stats_region *regions = stats->agc_stats; // Note how the calculation below means that equal weights give you // "average" metering (i.e. all pixels equally important). double R_sum = 0, G_sum = 0, B_sum = 0, pixel_sum = 0; for (int i = 0; i < AGC_STATS_SIZE; i++) { - R_sum += regions[i].r_sum * weights[i]; - G_sum += regions[i].g_sum * weights[i]; - B_sum += regions[i].b_sum * weights[i]; - pixel_sum += regions[i].counted * weights[i]; + double counted = regions[i].counted; + double r_sum = std::min(regions[i].r_sum * gain, ((1 << PIPELINE_BITS) - 1) * counted); + double g_sum = std::min(regions[i].g_sum * gain, ((1 << PIPELINE_BITS) - 1) * counted); + double b_sum = std::min(regions[i].b_sum * gain, ((1 << PIPELINE_BITS) - 1) * counted); + R_sum += r_sum * weights[i]; + G_sum += g_sum * weights[i]; + B_sum += b_sum * weights[i]; + pixel_sum += counted * weights[i]; } if (pixel_sum == 0.0) { LOG(RPiAgc, Warning) << "compute_initial_Y: pixel_sum is zero"; @@ -476,11 +480,21 @@ void Agc::computeGain(bcm2835_isp_stats *statistics, Metadata *image_metadata, target_Y = config_.Y_target.Eval(config_.Y_target.Domain().Clip(lux.lux)); target_Y = std::min(EV_GAIN_Y_TARGET_LIMIT, target_Y * ev_gain); - double initial_Y = compute_initial_Y(statistics, awb_, - metering_mode_->weights); - gain = std::min(10.0, target_Y / (initial_Y + .001)); - LOG(RPiAgc, Debug) << "Initially Y " << initial_Y << " target " << target_Y - << " gives gain " << gain; + + // Do this calculation a few times as brightness increase can be + // non-linear when there are saturated regions. + gain = 1.0; + for (int i = 0; i < 8; i++) { + double initial_Y = compute_initial_Y(statistics, awb_, + metering_mode_->weights, gain); + double extra_gain = std::min(10.0, target_Y / (initial_Y + .001)); + gain *= extra_gain; + LOG(RPiAgc, Debug) << "Initial Y " << initial_Y << " target " << target_Y + << " gives gain " << gain; + if (extra_gain < 1.01) // close enough + break; + } + for (auto &c : *constraint_mode_) { double new_target_Y; double new_gain = From patchwork Mon Nov 16 16:49:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 10439 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0D9E4BE082 for ; Mon, 16 Nov 2020 16:49:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CF907632D6; Mon, 16 Nov 2020 17:49:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="U74fOlTh"; dkim-atps=neutral Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 85835632D1 for ; Mon, 16 Nov 2020 17:49:34 +0100 (CET) Received: by mail-wm1-x332.google.com with SMTP id s13so24199948wmh.4 for ; Mon, 16 Nov 2020 08:49:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bc8pIXR1oLN0cPR7o76WwaFmRCJS7IeSD16dgScTV5Y=; b=U74fOlTh9+gRoGsA7I1ew8UZ8Gp0YnQ33ebmfHCu1139xp01lcQZxb9OJpt/4nYyoA t9WcREbC7Cy2IztwsFjvMWwaRhwTsoZPWPBIzf/uvksPkZm+oZHk3b7KUJX5bHmb4TZx jtw7kQyAuZFpH37rVTHRlL8Ytq4lRQZsW2jnz5avObmw0SfxURyHb8FBRunFXzH9j6ij Sjk/HmXgFpD4cd5NNdrJlx/IF40Ld+VSUxeflPJ9ytjrLWukU0NRcOWQH+fSHtLz8uRJ FXpUSvOkkUCZn3jL90uLLEjaPqx+t+GQFHhsx+33xKTEXcqWm5houfMi3sG3iKfz3DI1 p0XA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bc8pIXR1oLN0cPR7o76WwaFmRCJS7IeSD16dgScTV5Y=; b=HJjxV42W7uDyxOAKK2j6N1o3nhASgwXCNdclXGwC2X6vAWOYwgr3O0sQlraYmnUwiH HUNzYp8pdV2xlD5UWj/2H+j/cst1TA8E3w2LBQh5X9xpdrJnv0fSbFDbYeixElRaAU/r ryNMvBnCIWXdmRRNnjtX6LkwmPAZ+8qFxpmfV0zK+T9whWkMqn2mULTEH48u7gT2WbHN EUQQpJVXZwjlty3iq9Zqj/4nwE9aEheq4f89sZALpuaIb5RTYf/489ZmLzcnQV78oK7/ BRxWEPEk7sOHMN4oOFdh9f2Fut0fslRVvDhBBxJIatSXCNnaKhCdXcSUs4uuz6mgVWon vljA== X-Gm-Message-State: AOAM5332MSFGntw0FN5df0HfjR2cayhqByWbzHJOMUoffT0Rrdzs8FSE dezBq9/graIb+598MKmEkjtIc8xoUnaGzg== X-Google-Smtp-Source: ABdhPJzQczI9hMe766aBmspH5aNNtuon3uCub+duQ7RrywWrnOo6iU15nndMmoSjCjWCYnoSNyd4Lw== X-Received: by 2002:a1c:3d05:: with SMTP id k5mr17131635wma.151.1605545373897; Mon, 16 Nov 2020 08:49:33 -0800 (PST) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id q16sm23716973wrn.13.2020.11.16.08.49.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 08:49:33 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 16 Nov 2020 16:49:18 +0000 Message-Id: <20201116164918.2055-11-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201116164918.2055-1-david.plowman@raspberrypi.com> References: <20201116164918.2055-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 10/10] libcamera: src: ipa: raspberrypi: agc: Improve AE locked logic X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Previously we required that the sensor absolutely reaches the target exposure, but this can fail if frame rates or analogue gains are limited. Instead insist only that we get several frames with the same exposure time, analogue gain and that the algorithm's target exposure hasn't changed either. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 65 +++++++++++++--------- src/ipa/raspberrypi/controller/rpi/agc.hpp | 3 + 2 files changed, 42 insertions(+), 26 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 93b46a28..7be74763 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -164,6 +164,8 @@ Agc::Agc(Controller *controller) flicker_period_ = 0.0; fixed_shutter_ = 0; fixed_analogue_gain_ = 0.0; + memset(&last_device_status_, 0, sizeof(last_device_status_)); + last_target_exposure_ = 0.0; } char const *Agc::Name() const @@ -264,8 +266,6 @@ void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, void Agc::Prepare(Metadata *image_metadata) { - int lock_count = lock_count_; - lock_count_ = 0; status_.digital_gain = 1.0; fetchAwbStatus(image_metadata); // always fetch it so that Process knows it's been done @@ -289,32 +289,10 @@ void Agc::Prepare(Metadata *image_metadata) LOG(RPiAgc, Debug) << "Use digital_gain " << status_.digital_gain; LOG(RPiAgc, Debug) << "Effective exposure " << actual_exposure * status_.digital_gain; // Decide whether AEC/AGC has converged. - // Insist AGC is steady for MAX_LOCK_COUNT - // frames before we say we are "locked". - // (The hard-coded constants may need to - // become customisable.) - if (status_.target_exposure_value) { -#define MAX_LOCK_COUNT 3 - double err = 0.10 * status_.target_exposure_value + 200; - if (actual_exposure < - status_.target_exposure_value + err && - actual_exposure > - status_.target_exposure_value - err) - lock_count_ = - std::min(lock_count + 1, - MAX_LOCK_COUNT); - else if (actual_exposure < - status_.target_exposure_value + 1.5 * err && - actual_exposure > - status_.target_exposure_value - 1.5 * err) - lock_count_ = lock_count; - LOG(RPiAgc, Debug) << "Lock count: " << lock_count_; - } + updateLockStatus(device_status); } } else - LOG(RPiAgc, Debug) << Name() << ": no device metadata"; - status_.locked = lock_count_ >= MAX_LOCK_COUNT; - //printf("%s\n", status_.locked ? "+++++++++" : "-"); + LOG(RPiAgc, Warning) << Name() << ": no device metadata"; image_metadata->Set("agc.status", status_); } } @@ -345,6 +323,41 @@ void Agc::Process(StatisticsPtr &stats, Metadata *image_metadata) writeAndFinish(image_metadata, desaturate); } +void Agc::updateLockStatus(DeviceStatus const &device_status) +{ + const double ERROR_FACTOR = 0.10; // make these customisable? + const int MAX_LOCK_COUNT = 5; + + // Add 200us to the exposure time error to allow for line quantisation. + double exposure_error = last_device_status_.shutter_speed * ERROR_FACTOR + 200; + double gain_error = last_device_status_.analogue_gain * ERROR_FACTOR; + double target_error = last_target_exposure_ * ERROR_FACTOR; + + // Note that we don't know the exposure/gain limits of the sensor, so + // the values we keep requesting may be unachievable. For this reason + // we only insist that we're close to values in the past few frames. + if (device_status.shutter_speed > last_device_status_.shutter_speed - exposure_error && + device_status.shutter_speed < last_device_status_.shutter_speed + exposure_error && + device_status.analogue_gain > last_device_status_.analogue_gain - gain_error && + device_status.analogue_gain < last_device_status_.analogue_gain + gain_error && + status_.target_exposure_value > last_target_exposure_ - target_error && + status_.target_exposure_value < last_target_exposure_ + target_error) + lock_count_ = std::min(lock_count_ + 1, MAX_LOCK_COUNT); + else if (device_status.shutter_speed < last_device_status_.shutter_speed - 1.5 * exposure_error || + device_status.shutter_speed > last_device_status_.shutter_speed + 1.5 * exposure_error || + device_status.analogue_gain < last_device_status_.analogue_gain - 1.5 * gain_error || + device_status.analogue_gain > last_device_status_.analogue_gain + 1.5 * gain_error || + status_.target_exposure_value < last_target_exposure_ - 1.5 * target_error || + status_.target_exposure_value > last_target_exposure_ + 1.5 * target_error) + lock_count_ = 0; + + last_device_status_ = device_status; + last_target_exposure_ = status_.target_exposure_value; + + LOG(RPiAgc, Debug) << "Lock count updated to " << lock_count_; + status_.locked = lock_count_ >= MAX_LOCK_COUNT; +} + static void copy_string(std::string const &s, char *d, size_t size) { size_t length = s.copy(d, size - 1); diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index 859a9650..47ebb324 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -82,6 +82,7 @@ public: void Process(StatisticsPtr &stats, Metadata *image_metadata) override; private: + void updateLockStatus(DeviceStatus const &device_status); AgcConfig config_; void housekeepConfig(); void fetchCurrentExposure(Metadata *image_metadata); @@ -111,6 +112,8 @@ private: ExposureValues filtered_; // these values are filtered towards target AgcStatus status_; int lock_count_; + DeviceStatus last_device_status_; + double last_target_exposure_; // Below here the "settings" that applications can change. std::string metering_mode_name_; std::string exposure_mode_name_;