From patchwork Wed Mar 22 16:13:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18439 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A1D66C3260 for ; Wed, 22 Mar 2023 16:13:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 403C262713; Wed, 22 Mar 2023 17:13:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679501596; bh=Q9nYWYgXbYyKB9xNapmUD/AEc/dU8XmqqZPszshPCeg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=XVhubd1js9UZY4zwsrLIOpq9r7XP1Fu3VSYSQiSgn4qRVwMDGJnxEu8n5pHm4jC7t 6KNy1xchazo6FBtL6HoPH6/sgPea8XY1pqHqd/DdNeJN9HKs1+IGOBwCCxdTBtb57e kpZUXzkf17E/O8Idqh9GijqgcdRXxSD9G3secN5BrykfM2+lN4CurhmxKq9BtqkvfX /Cd7e002KHRtoZYl1zJxaZH6HwMhSIq5m0sQez0xCI9vw/kcERilJK+IKOKyO3RZiz fmL9ZqMdbDdMIyNiOyvuAFLB9X8L54NL/Xt4b79iyFRT/Wb74FQ5pD+UOP7YKToOyF mxGqZ/7Sj2Hug== Received: from mail-io1-xd2a.google.com (mail-io1-xd2a.google.com [IPv6:2607:f8b0:4864:20::d2a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 97CE261ECE for ; Wed, 22 Mar 2023 17:13:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Pl+TGrxd"; dkim-atps=neutral Received: by mail-io1-xd2a.google.com with SMTP id s4so8682061ioj.11 for ; Wed, 22 Mar 2023 09:13:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679501592; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ldq/Rlt6NYA7xymrGzIpw8UgcgCXe3O7zbO3IBqcui4=; b=Pl+TGrxdt1F/3BwDdyMkcz6/yZ4I9vRfb+Fgvac1sho1BaoXA+ZLl3+khEzqTJ2s5l G8s4XYM0jwa2dCn3/dL/I7OQlgPpUMGJuvoLS5ExrcLXVS4Vj8PrQVpupVhuWanczX4V k7cJkuUY4DNrSyERteLxrCE4b/jisWhSrhy82JUxEqRES8ZanJZntSZiuFL6B4hcGB4C wA9AorsURSmLy1gBc+xJCmoBgPLEfZDQhkNCN9KieikAJ3//bfUbRtn/qO6EvY6Da2fy Mw1EmKIo3d8Acr1TP143xU/3MZe0lmmIpzt77O5nKzSOd7X4rFsaKjuYjlTzX/tn/7NF +evg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679501592; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ldq/Rlt6NYA7xymrGzIpw8UgcgCXe3O7zbO3IBqcui4=; b=StloSyRVZ2MIvXpsp1TQslocDG0r8/rAaExGYskvZoz3czFz63nEkRYq6N8K5Xfjjc dGKHP8zE6swsSbZ6N9P+PCd/+1uAOWznsWe78QfzAOD72EpfAWmT7AqfxhhHSmNHKVxL KgD/UMSklBrd+QE8LIz2lwcvvgfPHr/nvUnC3Et3J3Kh7Vf0GECYPMCE8o8cWVaU661t orRWD8UnCXduJFHWNAA56//iGLPnyzi6odD+0uXfllgCIuX6O8HXT54SDc196qRSvQ7n rjj8d56e3pA2p2QO2YHbvJTbJ/it2eQwZAX8Rm5i+Ue1w5iABgv9guuERelu4VNSPpyc zm+w== X-Gm-Message-State: AO0yUKXVU2iXUOc9HKbkO9EdeOBTlAfJlIgcRt7+ux5adwGLDl6nIUmj btVVCXYRfm7zAO8EMcsxXlvVaD19nleXNw+AmKR+Rg== X-Google-Smtp-Source: AK7set8B97V0PepPp/7ZdBpSQEwoBJZ7f6aDuc+dsXZoMwleLerxgtJBereC9qOwQXbPPpW57XFlPA== X-Received: by 2002:a6b:4f0b:0:b0:752:e9e5:49af with SMTP id d11-20020a6b4f0b000000b00752e9e549afmr5058931iob.2.1679501591975; Wed, 22 Mar 2023 09:13:11 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id n24-20020a02a918000000b004050d92f6d4sm5100119jam.50.2023.03.22.09.13.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 09:13:11 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Mar 2023 16:13:15 +0000 Message-Id: <20230322161317.18055-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322161317.18055-1-naush@raspberrypi.com> References: <20230322161317.18055-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 1/3] ipa: raspberrypi: Replace setMaxShutter with setSensorLimits in AGC X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Create a SensorLimits structure in the AgcAlgorithm class. The maximum shutter time is stored in this structure from the IPA. In a subsequent commit, all gain and shutter limits will additionally be stored. Pass the SensorLimits structure with the maximum shutter time into the AGC algorithm through the new setSensorLimits() member function, replacing the call to setMaxShutter(). Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/agc_algorithm.h | 11 ++++++++++- src/ipa/raspberrypi/controller/rpi/agc.cpp | 10 +++++----- src/ipa/raspberrypi/controller/rpi/agc.h | 4 ++-- src/ipa/raspberrypi/raspberrypi.cpp | 7 ++++--- 4 files changed, 21 insertions(+), 11 deletions(-) diff --git a/src/ipa/raspberrypi/controller/agc_algorithm.h b/src/ipa/raspberrypi/controller/agc_algorithm.h index 36e6c11058ee..c2759689c7f9 100644 --- a/src/ipa/raspberrypi/controller/agc_algorithm.h +++ b/src/ipa/raspberrypi/controller/agc_algorithm.h @@ -15,13 +15,22 @@ namespace RPiController { class AgcAlgorithm : public Algorithm { public: + struct SensorLimits { + libcamera::utils::Duration minShutter; + libcamera::utils::Duration maxShutter; + libcamera::utils::Duration minFrameDuration; + libcamera::utils::Duration maxFrameDuration; + double minAnalogueGain; + double maxAnalogueGain; + }; + AgcAlgorithm(Controller *controller) : Algorithm(controller) {} /* An AGC algorithm must provide the following: */ virtual unsigned int getConvergenceFrames() const = 0; virtual void setEv(double ev) = 0; virtual void setFlickerPeriod(libcamera::utils::Duration flickerPeriod) = 0; virtual void setFixedShutter(libcamera::utils::Duration fixedShutter) = 0; - virtual void setMaxShutter(libcamera::utils::Duration maxShutter) = 0; + virtual void setSensorLimits(const SensorLimits &limits) = 0; virtual void setFixedAnalogueGain(double fixedAnalogueGain) = 0; virtual void setMeteringMode(std::string const &meteringModeName) = 0; virtual void setExposureMode(std::string const &exposureModeName) = 0; diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 4ea0dd41e66c..eaabc5fa9cd0 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -225,7 +225,7 @@ Agc::Agc(Controller *controller) frameCount_(0), lockCount_(0), lastTargetExposure_(0s), lastSensitivity_(0.0), ev_(1.0), flickerPeriod_(0s), - maxShutter_(0s), fixedShutter_(0s), fixedAnalogueGain_(0.0) + fixedShutter_(0s), fixedAnalogueGain_(0.0) { memset(&awb_, 0, sizeof(awb_)); /* @@ -300,9 +300,9 @@ void Agc::setFlickerPeriod(Duration flickerPeriod) flickerPeriod_ = flickerPeriod; } -void Agc::setMaxShutter(Duration maxShutter) +void Agc::setSensorLimits(const SensorLimits &limits) { - maxShutter_ = maxShutter; + sensorLimits_ = limits; } void Agc::setFixedShutter(Duration fixedShutter) @@ -874,8 +874,8 @@ void Agc::writeAndFinish(Metadata *imageMetadata, bool desaturate) Duration Agc::clipShutter(Duration shutter) { - if (maxShutter_) - shutter = std::min(shutter, maxShutter_); + if (sensorLimits_.maxShutter) + shutter = std::min(shutter, sensorLimits_.maxShutter); return shutter; } diff --git a/src/ipa/raspberrypi/controller/rpi/agc.h b/src/ipa/raspberrypi/controller/rpi/agc.h index f04896ca25ad..150a5f6e4428 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.h +++ b/src/ipa/raspberrypi/controller/rpi/agc.h @@ -78,7 +78,7 @@ public: unsigned int getConvergenceFrames() const override; void setEv(double ev) override; void setFlickerPeriod(libcamera::utils::Duration flickerPeriod) override; - void setMaxShutter(libcamera::utils::Duration maxShutter) override; + void setSensorLimits(const SensorLimits &limits) override; void setFixedShutter(libcamera::utils::Duration fixedShutter) override; void setFixedAnalogueGain(double fixedAnalogueGain) override; void setMeteringMode(std::string const &meteringModeName) override; @@ -131,7 +131,7 @@ private: std::string constraintModeName_; double ev_; libcamera::utils::Duration flickerPeriod_; - libcamera::utils::Duration maxShutter_; + SensorLimits sensorLimits_; libcamera::utils::Duration fixedShutter_; double fixedAnalogueGain_; }; diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 1375795568e2..c3b2c375036f 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -1505,12 +1505,13 @@ void IPARPi::applyFrameDurations(Duration minFrameDuration, Duration maxFrameDur * getBlanking() will update maxShutter with the largest exposure * value possible. */ - Duration maxShutter = Duration::max(); - helper_->getBlanking(maxShutter, minFrameDuration_, maxFrameDuration_); + RPiController::AgcAlgorithm::SensorLimits limits; + limits.maxShutter = Duration::max(); + helper_->getBlanking(limits.maxShutter, minFrameDuration_, maxFrameDuration_); RPiController::AgcAlgorithm *agc = dynamic_cast( controller_.getAlgorithm("agc")); - agc->setMaxShutter(maxShutter); + agc->setSensorLimits(limits); } void IPARPi::applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls) From patchwork Wed Mar 22 16:13:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18440 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 587D7C329C for ; Wed, 22 Mar 2023 16:13:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3CD69626E6; Wed, 22 Mar 2023 17:13:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679501597; bh=VYKperBBYbHQZ4V5a/30P5cKliRg9gd4k2sKwfLuepQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=X61uoHXXAU0xbR4m4GVI3xdYUWNCHI/n5eWbEOAZU4Ttq3iRRbnVrw5iyv5wLMQya Ox/WFnpLsHfzMhNoP7zFJJwvjE9bGeJXmJTia2mSfGbWhxdbtX5PrKNcZH8MrxFLYw /HUo8/Bf/lfJUCpyhKznnbadVk2d7VQUYxaPuYMbt8Qen5SydzTWxQwhmDD/s9LuWh mbYleYZ1Bap05V87tU024zsGooMAczZMB+t0f+bcy0nrPdWMqwHWO3LDfhwmqDf6yL CIi/xMqkfpB1AAn1+8Lt+c8yRuSqJEQggjLA2Uvxmf+nvTEDJsVymW12fY+6bjUN4Y jeb78LjUP9RCw== Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 778DD61ECE for ; Wed, 22 Mar 2023 17:13:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="F4egNLot"; dkim-atps=neutral Received: by mail-io1-xd29.google.com with SMTP id e13so648119ioc.0 for ; Wed, 22 Mar 2023 09:13:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679501593; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AI7QePFulGJyiLBn7dSeGdrq5BdJQids08mpyLUydP4=; b=F4egNLotmWHGTzacRI9UkfyzGWsPJkwL7IZD97xYQSzFoLObee6uYFZWYaGZNVIo/p mLE+qt9J/LgcA2e/IhLW8jYwwszqcK9EPaE/lzV2ov3rMowWnpnXAJrTJ68V1riLHudy 1JglvtC8apYMk7Sh+3w+WkIrPnfQKT1IdRO90edQf0OMWJcsxbcdklrI5mUMl9ksr30Q 9vFjm6OxJtxBX5YZgIvsmQuPWTbe3fMAXBOxXzZtWEoRxuk47BQ4Q73r3vsZSG6ps6V3 lBQ0kiowF/HDx8YF4h/Pc/mEEZtT16Jz9mZ3CWD4JXd97eFghhhh0RQCqEuVNp/2BFU3 4ruw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679501593; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AI7QePFulGJyiLBn7dSeGdrq5BdJQids08mpyLUydP4=; b=1RAgXEc7nszvna3OEPxftsmeuH36d11lboqwXP1Aqtd9aXeVUm4M73UcDs5i1dCFst 4E7sOkAhpGN2wxRyr5l+KMI9bglrQvBJgg4VrP0DWxaOhmTyo2plEWJQ1OpoPCIKG/2I 53CCYC9Zpsy9ZlDvREBxPqVwwNnBtjeUvTyl43x+SP/SFp2DWm93mxBKH/tp7Ru0eaqS 3PXmcHlBZy9vwm7YuZXrLDBbbygJrGI6BE+9Hew0tliMrjfTBCepkyb6+PmmYJaol96a 1ccU7+Y+yN5S2+6lfwA/2kO/C4XG3AnaFe3Nddyn6BEYBJ6v+g2igpjdt6uhUbdgUD5m sDxw== X-Gm-Message-State: AO0yUKXenTCoaDJZ86spIO48gU3Nt+A60Cpjn775le2sMhc9R+30vjAf 08KSXI53IZJhZXDIdRuLAEJi9DZDrl4dywMyMjQ5Hw== X-Google-Smtp-Source: AK7set93R8PCjeemzJiLvNnDyuxgxfwLDI8Bht2ua74uYZhrMM/jTp5jP+W9QosvUAdU4t0y9WjwbQ== X-Received: by 2002:a6b:7613:0:b0:751:df6e:79ba with SMTP id g19-20020a6b7613000000b00751df6e79bamr1927007iom.5.1679501593055; Wed, 22 Mar 2023 09:13:13 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id n24-20020a02a918000000b004050d92f6d4sm5100119jam.50.2023.03.22.09.13.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 09:13:12 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Mar 2023 16:13:16 +0000 Message-Id: <20230322161317.18055-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322161317.18055-1-naush@raspberrypi.com> References: <20230322161317.18055-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 2/3] ipa: raspberrypi: Populate SensorLimits X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Populate all the fields of the SensorLimits structure in configure(). This allows us to use the cached values instead of re-computing them on every frame. For the gain -> code convertion, ensure we clamp to the analogue gain limits set in the SensorLimits structure. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/raspberrypi.cpp | 67 ++++++++++++++++------------- 1 file changed, 38 insertions(+), 29 deletions(-) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index c3b2c375036f..e8d8023bcfe7 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -224,8 +224,8 @@ private: Duration minFrameDuration_; Duration maxFrameDuration_; - /* Maximum gain code for the sensor. */ - uint32_t maxSensorGainCode_; + /* Mode specific sensor gain/exposure limits. */ + RPiController::AgcAlgorithm::SensorLimits sensorLimits_; /* Track the frame length times over FrameLengthsQueueSize frames. */ std::deque frameLengths_; @@ -439,8 +439,6 @@ int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, const IPAConfig &ip } } - maxSensorGainCode_ = sensorCtrls_.at(V4L2_CID_ANALOGUE_GAIN).max().get(); - /* Setup a metadata ControlList to output metadata. */ libcameraMetadata_ = ControlList(controls::controls); @@ -473,6 +471,28 @@ int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, const IPAConfig &ip /* Pass the camera mode to the CamHelper to setup algorithms. */ helper_->setCameraMode(mode_); + /* + * Store the sensor gain and shutter limits for the mode. + * + * The maximum shutter value are calculated from the frame duration limit + * as V4L2 will restrict the maximum control value based on the current + * VBLANK value. + * + * These limits get set in the AGC algorithm through applyFrameDurations() + * below. + */ + const ControlInfo &gainCtrl = sensorCtrls_.at(V4L2_CID_ANALOGUE_GAIN); + const ControlInfo &shutterCtrl = sensorCtrls_.at(V4L2_CID_EXPOSURE); + + sensorLimits_.minAnalogueGain = helper_->gain(gainCtrl.min().get()); + sensorLimits_.maxAnalogueGain = helper_->gain(gainCtrl.max().get()); + sensorLimits_.minFrameDuration = mode_.minFrameLength * mode_.minLineLength; + sensorLimits_.maxFrameDuration = mode_.maxFrameLength * mode_.maxLineLength; + sensorLimits_.minShutter = helper_->exposure(shutterCtrl.min().get(), mode_.minLineLength); + sensorLimits_.maxShutter = Duration::max(); + helper_->getBlanking(sensorLimits_.maxShutter, + sensorLimits_.minFrameDuration, sensorLimits_.maxFrameDuration); + /* * Initialise this ControlList correctly, even if empty, in case the IPA is * running is isolation mode (passing the ControlList through the IPC layer). @@ -501,26 +521,17 @@ int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, const IPAConfig &ip * based on the current sensor mode. */ ControlInfoMap::Map ctrlMap = ipaControls; - const Duration minSensorFrameDuration = mode_.minFrameLength * mode_.minLineLength; - const Duration maxSensorFrameDuration = mode_.maxFrameLength * mode_.maxLineLength; ctrlMap[&controls::FrameDurationLimits] = - ControlInfo(static_cast(minSensorFrameDuration.get()), - static_cast(maxSensorFrameDuration.get())); + ControlInfo(static_cast(sensorLimits_.minFrameDuration.get()), + static_cast(sensorLimits_.maxFrameDuration.get())); ctrlMap[&controls::AnalogueGain] = - ControlInfo(1.0f, static_cast(helper_->gain(maxSensorGainCode_))); - - /* - * Calculate the max exposure limit from the frame duration limit as V4L2 - * will limit the maximum control value based on the current VBLANK value. - */ - Duration maxShutter = Duration::max(); - helper_->getBlanking(maxShutter, minSensorFrameDuration, maxSensorFrameDuration); - const uint32_t exposureMin = sensorCtrls_.at(V4L2_CID_EXPOSURE).min().get(); + ControlInfo(static_cast(sensorLimits_.minAnalogueGain), + static_cast(sensorLimits_.maxAnalogueGain)); ctrlMap[&controls::ExposureTime] = - ControlInfo(static_cast(helper_->exposure(exposureMin, mode_.minLineLength).get()), - static_cast(maxShutter.get())); + ControlInfo(static_cast(sensorLimits_.minShutter.get()), + static_cast(sensorLimits_.maxShutter.get())); /* Declare Autofocus controls, only if we have a controllable lens */ if (lensPresent_) @@ -1480,9 +1491,6 @@ void IPARPi::applyAWB(const struct AwbStatus *awbStatus, ControlList &ctrls) void IPARPi::applyFrameDurations(Duration minFrameDuration, Duration maxFrameDuration) { - const Duration minSensorFrameDuration = mode_.minFrameLength * mode_.minLineLength; - const Duration maxSensorFrameDuration = mode_.maxFrameLength * mode_.maxLineLength; - /* * This will only be applied once AGC recalculations occur. * The values may be clamped based on the sensor mode capabilities as well. @@ -1490,9 +1498,9 @@ void IPARPi::applyFrameDurations(Duration minFrameDuration, Duration maxFrameDur minFrameDuration_ = minFrameDuration ? minFrameDuration : defaultMaxFrameDuration; maxFrameDuration_ = maxFrameDuration ? maxFrameDuration : defaultMinFrameDuration; minFrameDuration_ = std::clamp(minFrameDuration_, - minSensorFrameDuration, maxSensorFrameDuration); + sensorLimits_.minFrameDuration, sensorLimits_.maxFrameDuration); maxFrameDuration_ = std::clamp(maxFrameDuration_, - minSensorFrameDuration, maxSensorFrameDuration); + sensorLimits_.minFrameDuration, sensorLimits_.maxFrameDuration); maxFrameDuration_ = std::max(maxFrameDuration_, minFrameDuration_); /* Return the validated limits via metadata. */ @@ -1505,17 +1513,18 @@ void IPARPi::applyFrameDurations(Duration minFrameDuration, Duration maxFrameDur * getBlanking() will update maxShutter with the largest exposure * value possible. */ - RPiController::AgcAlgorithm::SensorLimits limits; - limits.maxShutter = Duration::max(); - helper_->getBlanking(limits.maxShutter, minFrameDuration_, maxFrameDuration_); + sensorLimits_.maxShutter = Duration::max(); + helper_->getBlanking(sensorLimits_.maxShutter, minFrameDuration_, maxFrameDuration_); RPiController::AgcAlgorithm *agc = dynamic_cast( controller_.getAlgorithm("agc")); - agc->setSensorLimits(limits); + agc->setSensorLimits(sensorLimits_); } void IPARPi::applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls) { + const int32_t minGainCode = helper_->gainCode(sensorLimits_.minAnalogueGain); + const int32_t maxGainCode = helper_->gainCode(sensorLimits_.maxAnalogueGain); int32_t gainCode = helper_->gainCode(agcStatus->analogueGain); /* @@ -1523,7 +1532,7 @@ void IPARPi::applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls) * DelayedControls. The AGC will correctly handle a lower gain returned * by the sensor, provided it knows the actual gain used. */ - gainCode = std::min(gainCode, maxSensorGainCode_); + gainCode = std::clamp(gainCode, minGainCode, maxGainCode); /* getBlanking might clip exposure time to the fps limits. */ Duration exposure = agcStatus->shutterTime; From patchwork Wed Mar 22 16:13:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18441 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1D290C329D for ; Wed, 22 Mar 2023 16:13:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 43A1A62715; Wed, 22 Mar 2023 17:13:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679501598; bh=T5UmdV72nfTRwNUVi7eQo2sbBcMiLO+uLhjRMQVEiy4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=oZqjzVbthTSITcs7FW4NkMCfpieY5oc/XOqLue0ZSRi6SEI+YF5ydIKlY+/nNPlLB RFSMdn5RTHP+QfLyWafwR+6rYvceOSBPZ5tTfUY4AIvzq83kSf+ouWsI34/nZ+a7Yk ov0w9vCZC0dLuNRtlEfBbVrwQHbcO8rLQc5pGXUKCpMe6WS0fAN2CQhYJLmfnqwreV FpI3od6RktHRITiIjB6vsdZWIrAf26XErK3HN0YKaYyWkbIjf2UJWvwWMJDK1ezyY+ UTa1YjxPEEijoIPOsMHcxMbZNMow6vGDpfhF6zy4MZFPG7mvyWfHx/Q9SDROH+qPNt muxSJtGv0XXMg== Received: from mail-io1-xd2a.google.com (mail-io1-xd2a.google.com [IPv6:2607:f8b0:4864:20::d2a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B3C45626DA for ; Wed, 22 Mar 2023 17:13:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="NIAoES7W"; dkim-atps=neutral Received: by mail-io1-xd2a.google.com with SMTP id s4so8682117ioj.11 for ; Wed, 22 Mar 2023 09:13:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679501594; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0clupoib3wTLjICcDff6DmuNnVx8wOrUSKdGYrk+q6I=; b=NIAoES7W8XFY2Ivk7m/YXYLKGKBKtwcjg1Dz/HSMkZY6Zz0GJLNF+qq06IcEVqn3B3 RmqwkjZlZepvRGqQDRdOysoS29+d9U0cYkhOtpaAlC8zFWGpezVtGkMwqRmPf0U3encd KNhDhYIS7KB6o10y99X2WmdTHEl26haKNsvFTpABFGYZWzgOHwEZKVz3vY++s+seBM8L 3aS+78nDTvUHRMjZCH8llp6dz5XVqoMPdLrn0SxThzw7oVElP47QIqo8twpkbDUol5Lj R+MA0CQsMedtP+aH8xdqkdAGxLgk9klA+1rSrxhaA1z28P+jBCJNw2vczd8e4zPwUci7 yE8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679501594; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0clupoib3wTLjICcDff6DmuNnVx8wOrUSKdGYrk+q6I=; b=eA8RjeIAdBsLBiLbZZbDZb3DFujgJXL9uDa2fKxSxmWykqUwtpIlyS3A9pP6vx7gTH aPFrYDMk42fDjfByp5Bxz8Sp5+IZkwxjm4m7e7GtDj/NqkD7yZevDyWvXeb5nAfmdGQm 1lJijGhhtcowNeJ6WR7r/D8/nWDyJ7fkmUm25xz61ySMXFHu9r12To8iSPNTTUUuDLLu ficpq+uzEslZdz65e4KCjg3opbzQP3FfLeKsBOnSiIhbLH18JMA++5mxkg1UHrs7niTr LHVybCruBHMKjJyfk6BaDhhAXr8Ub+wU0lHMNJn/a3kBrltJoGA9VkIDAz/B6bRpLOVJ t21Q== X-Gm-Message-State: AO0yUKXruoSV2VMVSRhpbCIMGjcGqxH4LjoJZjtWEZ+dw5i/nXkjM28G /dU6j/Wx8JWalqTOFLzWMpE8RFHcKh2sbwE0rVkUQQ== X-Google-Smtp-Source: AK7set8pXynw/dxNeSXaupCxLJrXzURHhpjr35ABPZOgD2lDJZbjMmimzNgknwlCYBJ6bUJ3tv6cmw== X-Received: by 2002:a5e:c605:0:b0:74c:8cec:548e with SMTP id f5-20020a5ec605000000b0074c8cec548emr4997612iok.4.1679501594087; Wed, 22 Mar 2023 09:13:14 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id n24-20020a02a918000000b004050d92f6d4sm5100119jam.50.2023.03.22.09.13.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 09:13:13 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Mar 2023 16:13:17 +0000 Message-Id: <20230322161317.18055-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322161317.18055-1-naush@raspberrypi.com> References: <20230322161317.18055-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 3/3] ipa: raspberrypi: Ensure shutter speed and gain are clipped in the AGC X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Limit the AGC gain calculations to the minimum value given by SensorLimits. The maximum value remains unclipped as any gain over the SensorLimits maximum will be made up by digital gain. Rename clipShutter to limitShutter for consistency, and have the latter limit the shutter speed to both upper and lower bounds. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 47 +++++++++++++++++----- src/ipa/raspberrypi/controller/rpi/agc.h | 3 +- 2 files changed, 39 insertions(+), 11 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index eaabc5fa9cd0..ee81537c653c 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -309,14 +309,14 @@ void Agc::setFixedShutter(Duration fixedShutter) { fixedShutter_ = fixedShutter; /* Set this in case someone calls disableAuto() straight after. */ - status_.shutterTime = clipShutter(fixedShutter_); + status_.shutterTime = limitShutter(fixedShutter_); } void Agc::setFixedAnalogueGain(double fixedAnalogueGain) { fixedAnalogueGain_ = fixedAnalogueGain; /* Set this in case someone calls disableAuto() straight after. */ - status_.analogueGain = fixedAnalogueGain; + status_.analogueGain = limitGain(fixedAnalogueGain); } void Agc::setMeteringMode(std::string const &meteringModeName) @@ -342,7 +342,7 @@ void Agc::switchMode(CameraMode const &cameraMode, housekeepConfig(); - Duration fixedShutter = clipShutter(fixedShutter_); + Duration fixedShutter = limitShutter(fixedShutter_); if (fixedShutter && fixedAnalogueGain_) { /* We're going to reset the algorithm here with these fixed values. */ @@ -516,7 +516,7 @@ void Agc::housekeepConfig() { /* First fetch all the up-to-date settings, so no one else has to do it. */ status_.ev = ev_; - status_.fixedShutter = clipShutter(fixedShutter_); + status_.fixedShutter = limitShutter(fixedShutter_); status_.fixedAnalogueGain = fixedAnalogueGain_; status_.flickerPeriod = flickerPeriod_; LOG(RPiAgc, Debug) << "ev " << status_.ev << " fixedShutter " @@ -703,7 +703,7 @@ void Agc::computeTargetExposure(double gain) Duration maxShutter = status_.fixedShutter ? status_.fixedShutter : exposureMode_->shutter.back(); - maxShutter = clipShutter(maxShutter); + maxShutter = limitShutter(maxShutter); Duration maxTotalExposure = maxShutter * (status_.fixedAnalogueGain != 0.0 @@ -803,15 +803,16 @@ void Agc::divideUpExposure() double analogueGain; shutterTime = status_.fixedShutter ? status_.fixedShutter : exposureMode_->shutter[0]; - shutterTime = clipShutter(shutterTime); + shutterTime = limitShutter(shutterTime); analogueGain = status_.fixedAnalogueGain != 0.0 ? status_.fixedAnalogueGain : exposureMode_->gain[0]; + analogueGain = limitGain(analogueGain); if (shutterTime * analogueGain < exposureValue) { for (unsigned int stage = 1; stage < exposureMode_->gain.size(); stage++) { if (!status_.fixedShutter) { Duration stageShutter = - clipShutter(exposureMode_->shutter[stage]); + limitShutter(exposureMode_->shutter[stage]); if (stageShutter * analogueGain >= exposureValue) { shutterTime = exposureValue / analogueGain; break; @@ -824,6 +825,7 @@ void Agc::divideUpExposure() break; } analogueGain = exposureMode_->gain[stage]; + analogueGain = limitGain(analogueGain); } } } @@ -846,6 +848,7 @@ void Agc::divideUpExposure() * gain as a side-effect. */ analogueGain = std::min(analogueGain, exposureMode_->gain.back()); + analogueGain = limitGain(analogueGain); shutterTime = newShutterTime; } LOG(RPiAgc, Debug) << "After flicker avoidance, shutter " @@ -872,13 +875,37 @@ void Agc::writeAndFinish(Metadata *imageMetadata, bool desaturate) << " analogue gain " << filtered_.analogueGain; } -Duration Agc::clipShutter(Duration shutter) +Duration Agc::limitShutter(Duration shutter) { - if (sensorLimits_.maxShutter) - shutter = std::min(shutter, sensorLimits_.maxShutter); + /* + * shutter == 0 is a special case for fixed shutter values, and must pass + * through unchanged + */ + if (!shutter) + return shutter; + + shutter = std::clamp(shutter, sensorLimits_.minShutter, + sensorLimits_.maxShutter); return shutter; } +double Agc::limitGain(double gain) const +{ + /* + * Only limit the lower bounds of the gain value to what the sensor limits. + * The upper bound on analogue gain will be made up with additional digital + * gain applied by the ISP. + * + * gain == 0.0 is a special case for fixed shutter values, and must pass + * through unchanged + */ + if (!gain) + return gain; + + gain = std::max(gain, sensorLimits_.minAnalogueGain); + return gain; +} + /* Register algorithm with the system. */ static Algorithm *create(Controller *controller) { diff --git a/src/ipa/raspberrypi/controller/rpi/agc.h b/src/ipa/raspberrypi/controller/rpi/agc.h index 150a5f6e4428..c48acbba61f0 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.h +++ b/src/ipa/raspberrypi/controller/rpi/agc.h @@ -103,7 +103,8 @@ private: void filterExposure(bool desaturate); void divideUpExposure(); void writeAndFinish(Metadata *imageMetadata, bool desaturate); - libcamera::utils::Duration clipShutter(libcamera::utils::Duration shutter); + libcamera::utils::Duration limitShutter(libcamera::utils::Duration shutter); + double limitGain(double gain) const; AgcMeteringMode *meteringMode_; AgcExposureMode *exposureMode_; AgcConstraintMode *constraintMode_;