From patchwork Fri Sep 15 15:58:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 19038 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0A949C32B0 for ; Fri, 15 Sep 2023 15:59:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A196C6291C; Fri, 15 Sep 2023 17:58:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694793539; bh=Eo/H8q5i3+FT5oPUdT+DXN8pB6mxy+TQNhCh9E9Fw0k=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=L0xe4GKgpiCl4KHX/1ozarwgQtsm+z4a/rEtv6mCOq1N3b74HT0cyAHHBDNxBeBJ/ MzR1IJWZa4+mFMzdU0/8gtn3El/HrrELtTSx3qe0ppdSNt+gz65/IiW38IPpi7uWCe 3HO8f0YuYbclsS4k87YneIK7o7IzkL3YzFbC3D29anc3zfK6Nm0vvkcSEVl5VH8OLm uoA8EnM6BZ1kDqlk9pwW2bXSRCv2ohQmCgIhRcVLo/zphW/bFLqaiyYiixPmuyt/Mu x58tzuqwe6hdhgylKdUtp8nzPIOuR8Ia3wPqCmuTaIqm3v42rKcNFWfWAWBKlTMHYf 0Cdry3XaomF+g== Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 74A7762916 for ; Fri, 15 Sep 2023 17:58:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="necXUJas"; dkim-atps=neutral Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-403012f276dso25196005e9.0 for ; Fri, 15 Sep 2023 08:58:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1694793536; x=1695398336; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CWSOLadTJtKvYCIdSC1vIHqtxyxalOIgM1mgM1CtEyU=; b=necXUJas2zjLc9ffC2RyvFtrPr7wzxrG+zssWAlo8v7q/hp6Il4LVsFPlvlzEE9fGy ojxpw3fjK480ki/w/+EOdBZ2dOmSWsEPgp6kIk3bqCDYxq0O52H4Dm9DJDB9aIO/IZA+ E0Cy1H6E2If3zYsmGzkg9VpjGnbVMeqcVlfOTKyEEzKXpxin8ozj7uHKaTtXQsI2hPI6 P8wluqVjG5YI3cK1oADtpnWO7XSkcIiTtILO5EeS3nbUMSvXVCPQiDjlvwcRhp5G4gdS 20oLZh8eFe6fEaPukZHaMrmu1ZtCPU0xBDdg7mKI5cxmkSEPIUetiW22ddLsyfFT9Hp2 O8UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694793536; x=1695398336; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CWSOLadTJtKvYCIdSC1vIHqtxyxalOIgM1mgM1CtEyU=; b=Y3v3DaUwvkCqp04RcVBqz/rF9uO50eWPpOKLXsVXtUX4O5FvYOFRt2cEffB+fO7wZh +Bg+8gE2LPqspAK4JHdwQGWTktYXMbAt6jUiiTvDTMZofUIYjho+6TZ0Sz2RVpTrG/6c 5+pdMIXYza4WnuEUr+6LKl+jOp7RbdKNZaNpaBYwlJjG2retfoTttcesWJMQn15420fF 3h7yxXGHwqq1uKlAC886XOr7+nPJOVluucGWqhoLB7C9imQF+ondkb/BZsXfytu0mKzL RPdWzy47I16mwnK/l5FRTbuzx3BCewjZKIO5dl3ZD3RmdDvG75hazb5Jp58qBxKPANoM 2j8A== X-Gm-Message-State: AOJu0YyeMShsxp2XMhHzDZMBPPtnXkI+QZRwweXxdJnrrOyfra4lEEx/ hE6ZtA1AMeLx4xlhfp1/vYTZLas3bwYKeSSVHhI= X-Google-Smtp-Source: AGHT+IGKSnYOg6OYaFozE1b/GNHwX42q7UjAGTjEM0ZApSFAWbe0g6XmZA4pD7nSYfYn77KVX3R7QQ== X-Received: by 2002:a05:600c:2209:b0:404:746e:c601 with SMTP id z9-20020a05600c220900b00404746ec601mr2104418wml.9.1694793535672; Fri, 15 Sep 2023 08:58:55 -0700 (PDT) Received: from localhost.localdomain ([195.180.61.40]) by smtp.gmail.com with ESMTPSA id l13-20020a7bc44d000000b00401c595fcc7sm7810618wmi.11.2023.09.15.08.58.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 08:58:55 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 15 Sep 2023 16:58:44 +0100 Message-Id: <20230915155844.2388-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230915155844.2388-1-david.plowman@raspberrypi.com> References: <20230915155844.2388-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 5/5] ipa: rpi: agc: Use channel constraints in the AGC algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: David Plowman via libcamera-devel From: David Plowman Reply-To: David Plowman Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Whenever we run Agc::process(), we store the most recent total exposure requested for each channel. With these values we can apply the channel constraints after time-filtering the requested total exposure, but before working out how much digital gain is needed. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Jacopo Mondi --- src/ipa/rpi/controller/rpi/agc.cpp | 22 ++++++-- src/ipa/rpi/controller/rpi/agc.h | 1 + src/ipa/rpi/controller/rpi/agc_channel.cpp | 58 +++++++++++++++++++--- src/ipa/rpi/controller/rpi/agc_channel.h | 8 ++- 4 files changed, 75 insertions(+), 14 deletions(-) diff --git a/src/ipa/rpi/controller/rpi/agc.cpp b/src/ipa/rpi/controller/rpi/agc.cpp index 8b9d2026..870cb431 100644 --- a/src/ipa/rpi/controller/rpi/agc.cpp +++ b/src/ipa/rpi/controller/rpi/agc.cpp @@ -39,6 +39,7 @@ int Agc::read(const libcamera::YamlObject ¶ms) */ if (!params.contains("channels")) { LOG(RPiAgc, Debug) << "Single channel only"; + channelTotalExposures_.resize(1, 0s); channelData_.emplace_back(); return channelData_.back().channel.read(params, getHardwareConfig()); } @@ -58,6 +59,8 @@ int Agc::read(const libcamera::YamlObject ¶ms) return -1; } + channelTotalExposures_.resize(channelData_.size(), 0s); + return 0; } @@ -236,16 +239,22 @@ static void getDelayedChannelIndex(Metadata *metadata, const char *message, unsi } } -static void setCurrentChannelIndex(Metadata *metadata, const char *message, unsigned int channelIndex) +static libcamera::utils::Duration +setCurrentChannelIndexGetExposure(Metadata *metadata, const char *message, unsigned int channelIndex) { std::unique_lock lock(*metadata); AgcStatus *status = metadata->getLocked("agc.status"); - if (status) + libcamera::utils::Duration dur = 0s; + + if (status) { status->channel = channelIndex; - else { + dur = status->totalExposureValue; + } else { /* This does happen at startup, otherwise it would be a Warning or Error. */ LOG(RPiAgc, Debug) << message; } + + return dur; } void Agc::prepare(Metadata *imageMetadata) @@ -310,8 +319,11 @@ void Agc::process(StatisticsPtr &stats, Metadata *imageMetadata) LOG(RPiAgc, Debug) << "process: channel " << channelIndex << " not seen yet"; } - channelData.channel.process(stats, deviceStatus, imageMetadata); - setCurrentChannelIndex(imageMetadata, "process: no AGC status found", channelIndex); + channelData.channel.process(stats, deviceStatus, imageMetadata, channelTotalExposures_); + auto dur = setCurrentChannelIndexGetExposure(imageMetadata, "process: no AGC status found", + channelIndex); + if (dur) + channelTotalExposures_[channelIndex] = dur; /* And onto the next channel for the next call. */ index_ = (index_ + 1) % activeChannels_.size(); diff --git a/src/ipa/rpi/controller/rpi/agc.h b/src/ipa/rpi/controller/rpi/agc.h index ee85c693..90890439 100644 --- a/src/ipa/rpi/controller/rpi/agc.h +++ b/src/ipa/rpi/controller/rpi/agc.h @@ -55,6 +55,7 @@ private: std::vector channelData_; std::vector activeChannels_; unsigned int index_; /* index into the activeChannels_ */ + AgcChannelTotalExposures channelTotalExposures_; }; } /* namespace RPiController */ diff --git a/src/ipa/rpi/controller/rpi/agc_channel.cpp b/src/ipa/rpi/controller/rpi/agc_channel.cpp index dfe363ff..3957dbc3 100644 --- a/src/ipa/rpi/controller/rpi/agc_channel.cpp +++ b/src/ipa/rpi/controller/rpi/agc_channel.cpp @@ -493,7 +493,9 @@ void AgcChannel::prepare(Metadata *imageMetadata) } } -void AgcChannel::process(StatisticsPtr &stats, DeviceStatus const &deviceStatus, Metadata *imageMetadata) +void AgcChannel::process(StatisticsPtr &stats, DeviceStatus const &deviceStatus, + Metadata *imageMetadata, + const AgcChannelTotalExposures &channelTotalExposures) { frameCount_++; /* @@ -512,12 +514,17 @@ void AgcChannel::process(StatisticsPtr &stats, DeviceStatus const &deviceStatus, computeTargetExposure(gain); /* The results have to be filtered so as not to change too rapidly. */ filterExposure(); + /* + * We may be asked to limit the exposure using other channels. If another channel + * determines our upper bound we may want to know this later. + */ + bool channelBound = applyChannelConstraints(channelTotalExposures); /* * Some of the exposure has to be applied as digital gain, so work out - * what that is. This function also tells us whether it's decided to - * "desaturate" the image more quickly. + * what that is. It also tells us whether it's trying to desaturate the image + * more quickly, which can only happen when another channel is not limiting us. */ - bool desaturate = applyDigitalGain(gain, targetY); + bool desaturate = applyDigitalGain(gain, targetY, channelBound); /* * The last thing is to divide up the exposure value into a shutter time * and analogue gain, according to the current exposure mode. @@ -794,7 +801,44 @@ void AgcChannel::computeTargetExposure(double gain) LOG(RPiAgc, Debug) << "Target totalExposure " << target_.totalExposure; } -bool AgcChannel::applyDigitalGain(double gain, double targetY) +bool AgcChannel::applyChannelConstraints(const AgcChannelTotalExposures &channelTotalExposures) +{ + bool channelBound = false; + LOG(RPiAgc, Debug) + << "Total exposure before channel constraints " << filtered_.totalExposure; + + for (const auto &constraint : config_.channelConstraints) { + LOG(RPiAgc, Debug) + << "Check constraint: channel " << constraint.channel << " bound " + << (constraint.bound == AgcChannelConstraint::Bound::UPPER ? "UPPER" : "LOWER") + << " factor " << constraint.factor; + if (constraint.channel >= channelTotalExposures.size() || + !channelTotalExposures[constraint.channel]) { + LOG(RPiAgc, Debug) << "no such channel or no exposure available- skipped"; + continue; + } + + libcamera::utils::Duration limitExposure = + channelTotalExposures[constraint.channel] * constraint.factor; + LOG(RPiAgc, Debug) << "Limit exposure " << limitExposure; + if ((constraint.bound == AgcChannelConstraint::Bound::UPPER && + filtered_.totalExposure > limitExposure) || + (constraint.bound == AgcChannelConstraint::Bound::LOWER && + filtered_.totalExposure < limitExposure)) { + filtered_.totalExposure = limitExposure; + LOG(RPiAgc, Debug) << "Constraint applies"; + channelBound = true; + } else + LOG(RPiAgc, Debug) << "Constraint does not apply"; + } + + LOG(RPiAgc, Debug) + << "Total exposure after channel constraints " << filtered_.totalExposure; + + return channelBound; +} + +bool AgcChannel::applyDigitalGain(double gain, double targetY, bool channelBound) { double minColourGain = std::min({ awb_.gainR, awb_.gainG, awb_.gainB, 1.0 }); ASSERT(minColourGain != 0.0); @@ -814,8 +858,8 @@ bool AgcChannel::applyDigitalGain(double gain, double targetY) * quickly (and we then approach the correct value more quickly from * below). */ - bool desaturate = targetY > config_.fastReduceThreshold && - gain < sqrt(targetY); + bool desaturate = !channelBound && + targetY > config_.fastReduceThreshold && gain < sqrt(targetY); if (desaturate) dg /= config_.fastReduceThreshold; LOG(RPiAgc, Debug) << "Digital gain " << dg << " desaturate? " << desaturate; diff --git a/src/ipa/rpi/controller/rpi/agc_channel.h b/src/ipa/rpi/controller/rpi/agc_channel.h index ceaab12f..ae826fa8 100644 --- a/src/ipa/rpi/controller/rpi/agc_channel.h +++ b/src/ipa/rpi/controller/rpi/agc_channel.h @@ -21,6 +21,8 @@ namespace RPiController { +using AgcChannelTotalExposures = std::vector; + struct AgcMeteringMode { std::vector weights; int read(const libcamera::YamlObject ¶ms); @@ -95,7 +97,8 @@ public: void disableAuto(); void switchMode(CameraMode const &cameraMode, Metadata *metadata); void prepare(Metadata *imageMetadata); - void process(StatisticsPtr &stats, DeviceStatus const &deviceStatus, Metadata *imageMetadata); + void process(StatisticsPtr &stats, DeviceStatus const &deviceStatus, Metadata *imageMetadata, + const AgcChannelTotalExposures &channelTotalExposures); private: bool updateLockStatus(DeviceStatus const &deviceStatus); @@ -107,7 +110,8 @@ private: double &gain, double &targetY); void computeTargetExposure(double gain); void filterExposure(); - bool applyDigitalGain(double gain, double targetY); + bool applyChannelConstraints(const AgcChannelTotalExposures &channelTotalExposures); + bool applyDigitalGain(double gain, double targetY, bool channelBound); void divideUpExposure(); void writeAndFinish(Metadata *imageMetadata, bool desaturate); libcamera::utils::Duration limitShutter(libcamera::utils::Duration shutter);