From patchwork Wed Mar 22 16:13:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18440 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 587D7C329C for ; Wed, 22 Mar 2023 16:13:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3CD69626E6; Wed, 22 Mar 2023 17:13:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679501597; bh=VYKperBBYbHQZ4V5a/30P5cKliRg9gd4k2sKwfLuepQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=X61uoHXXAU0xbR4m4GVI3xdYUWNCHI/n5eWbEOAZU4Ttq3iRRbnVrw5iyv5wLMQya Ox/WFnpLsHfzMhNoP7zFJJwvjE9bGeJXmJTia2mSfGbWhxdbtX5PrKNcZH8MrxFLYw /HUo8/Bf/lfJUCpyhKznnbadVk2d7VQUYxaPuYMbt8Qen5SydzTWxQwhmDD/s9LuWh mbYleYZ1Bap05V87tU024zsGooMAczZMB+t0f+bcy0nrPdWMqwHWO3LDfhwmqDf6yL CIi/xMqkfpB1AAn1+8Lt+c8yRuSqJEQggjLA2Uvxmf+nvTEDJsVymW12fY+6bjUN4Y jeb78LjUP9RCw== Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 778DD61ECE for ; Wed, 22 Mar 2023 17:13:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="F4egNLot"; dkim-atps=neutral Received: by mail-io1-xd29.google.com with SMTP id e13so648119ioc.0 for ; Wed, 22 Mar 2023 09:13:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679501593; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AI7QePFulGJyiLBn7dSeGdrq5BdJQids08mpyLUydP4=; b=F4egNLotmWHGTzacRI9UkfyzGWsPJkwL7IZD97xYQSzFoLObee6uYFZWYaGZNVIo/p mLE+qt9J/LgcA2e/IhLW8jYwwszqcK9EPaE/lzV2ov3rMowWnpnXAJrTJ68V1riLHudy 1JglvtC8apYMk7Sh+3w+WkIrPnfQKT1IdRO90edQf0OMWJcsxbcdklrI5mUMl9ksr30Q 9vFjm6OxJtxBX5YZgIvsmQuPWTbe3fMAXBOxXzZtWEoRxuk47BQ4Q73r3vsZSG6ps6V3 lBQ0kiowF/HDx8YF4h/Pc/mEEZtT16Jz9mZ3CWD4JXd97eFghhhh0RQCqEuVNp/2BFU3 4ruw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679501593; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AI7QePFulGJyiLBn7dSeGdrq5BdJQids08mpyLUydP4=; b=1RAgXEc7nszvna3OEPxftsmeuH36d11lboqwXP1Aqtd9aXeVUm4M73UcDs5i1dCFst 4E7sOkAhpGN2wxRyr5l+KMI9bglrQvBJgg4VrP0DWxaOhmTyo2plEWJQ1OpoPCIKG/2I 53CCYC9Zpsy9ZlDvREBxPqVwwNnBtjeUvTyl43x+SP/SFp2DWm93mxBKH/tp7Ru0eaqS 3PXmcHlBZy9vwm7YuZXrLDBbbygJrGI6BE+9Hew0tliMrjfTBCepkyb6+PmmYJaol96a 1ccU7+Y+yN5S2+6lfwA/2kO/C4XG3AnaFe3Nddyn6BEYBJ6v+g2igpjdt6uhUbdgUD5m sDxw== X-Gm-Message-State: AO0yUKXenTCoaDJZ86spIO48gU3Nt+A60Cpjn775le2sMhc9R+30vjAf 08KSXI53IZJhZXDIdRuLAEJi9DZDrl4dywMyMjQ5Hw== X-Google-Smtp-Source: AK7set93R8PCjeemzJiLvNnDyuxgxfwLDI8Bht2ua74uYZhrMM/jTp5jP+W9QosvUAdU4t0y9WjwbQ== X-Received: by 2002:a6b:7613:0:b0:751:df6e:79ba with SMTP id g19-20020a6b7613000000b00751df6e79bamr1927007iom.5.1679501593055; Wed, 22 Mar 2023 09:13:13 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id n24-20020a02a918000000b004050d92f6d4sm5100119jam.50.2023.03.22.09.13.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 09:13:12 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Mar 2023 16:13:16 +0000 Message-Id: <20230322161317.18055-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322161317.18055-1-naush@raspberrypi.com> References: <20230322161317.18055-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 2/3] ipa: raspberrypi: Populate SensorLimits X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Populate all the fields of the SensorLimits structure in configure(). This allows us to use the cached values instead of re-computing them on every frame. For the gain -> code convertion, ensure we clamp to the analogue gain limits set in the SensorLimits structure. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/raspberrypi.cpp | 67 ++++++++++++++++------------- 1 file changed, 38 insertions(+), 29 deletions(-) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index c3b2c375036f..e8d8023bcfe7 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -224,8 +224,8 @@ private: Duration minFrameDuration_; Duration maxFrameDuration_; - /* Maximum gain code for the sensor. */ - uint32_t maxSensorGainCode_; + /* Mode specific sensor gain/exposure limits. */ + RPiController::AgcAlgorithm::SensorLimits sensorLimits_; /* Track the frame length times over FrameLengthsQueueSize frames. */ std::deque frameLengths_; @@ -439,8 +439,6 @@ int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, const IPAConfig &ip } } - maxSensorGainCode_ = sensorCtrls_.at(V4L2_CID_ANALOGUE_GAIN).max().get(); - /* Setup a metadata ControlList to output metadata. */ libcameraMetadata_ = ControlList(controls::controls); @@ -473,6 +471,28 @@ int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, const IPAConfig &ip /* Pass the camera mode to the CamHelper to setup algorithms. */ helper_->setCameraMode(mode_); + /* + * Store the sensor gain and shutter limits for the mode. + * + * The maximum shutter value are calculated from the frame duration limit + * as V4L2 will restrict the maximum control value based on the current + * VBLANK value. + * + * These limits get set in the AGC algorithm through applyFrameDurations() + * below. + */ + const ControlInfo &gainCtrl = sensorCtrls_.at(V4L2_CID_ANALOGUE_GAIN); + const ControlInfo &shutterCtrl = sensorCtrls_.at(V4L2_CID_EXPOSURE); + + sensorLimits_.minAnalogueGain = helper_->gain(gainCtrl.min().get()); + sensorLimits_.maxAnalogueGain = helper_->gain(gainCtrl.max().get()); + sensorLimits_.minFrameDuration = mode_.minFrameLength * mode_.minLineLength; + sensorLimits_.maxFrameDuration = mode_.maxFrameLength * mode_.maxLineLength; + sensorLimits_.minShutter = helper_->exposure(shutterCtrl.min().get(), mode_.minLineLength); + sensorLimits_.maxShutter = Duration::max(); + helper_->getBlanking(sensorLimits_.maxShutter, + sensorLimits_.minFrameDuration, sensorLimits_.maxFrameDuration); + /* * Initialise this ControlList correctly, even if empty, in case the IPA is * running is isolation mode (passing the ControlList through the IPC layer). @@ -501,26 +521,17 @@ int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, const IPAConfig &ip * based on the current sensor mode. */ ControlInfoMap::Map ctrlMap = ipaControls; - const Duration minSensorFrameDuration = mode_.minFrameLength * mode_.minLineLength; - const Duration maxSensorFrameDuration = mode_.maxFrameLength * mode_.maxLineLength; ctrlMap[&controls::FrameDurationLimits] = - ControlInfo(static_cast(minSensorFrameDuration.get()), - static_cast(maxSensorFrameDuration.get())); + ControlInfo(static_cast(sensorLimits_.minFrameDuration.get()), + static_cast(sensorLimits_.maxFrameDuration.get())); ctrlMap[&controls::AnalogueGain] = - ControlInfo(1.0f, static_cast(helper_->gain(maxSensorGainCode_))); - - /* - * Calculate the max exposure limit from the frame duration limit as V4L2 - * will limit the maximum control value based on the current VBLANK value. - */ - Duration maxShutter = Duration::max(); - helper_->getBlanking(maxShutter, minSensorFrameDuration, maxSensorFrameDuration); - const uint32_t exposureMin = sensorCtrls_.at(V4L2_CID_EXPOSURE).min().get(); + ControlInfo(static_cast(sensorLimits_.minAnalogueGain), + static_cast(sensorLimits_.maxAnalogueGain)); ctrlMap[&controls::ExposureTime] = - ControlInfo(static_cast(helper_->exposure(exposureMin, mode_.minLineLength).get()), - static_cast(maxShutter.get())); + ControlInfo(static_cast(sensorLimits_.minShutter.get()), + static_cast(sensorLimits_.maxShutter.get())); /* Declare Autofocus controls, only if we have a controllable lens */ if (lensPresent_) @@ -1480,9 +1491,6 @@ void IPARPi::applyAWB(const struct AwbStatus *awbStatus, ControlList &ctrls) void IPARPi::applyFrameDurations(Duration minFrameDuration, Duration maxFrameDuration) { - const Duration minSensorFrameDuration = mode_.minFrameLength * mode_.minLineLength; - const Duration maxSensorFrameDuration = mode_.maxFrameLength * mode_.maxLineLength; - /* * This will only be applied once AGC recalculations occur. * The values may be clamped based on the sensor mode capabilities as well. @@ -1490,9 +1498,9 @@ void IPARPi::applyFrameDurations(Duration minFrameDuration, Duration maxFrameDur minFrameDuration_ = minFrameDuration ? minFrameDuration : defaultMaxFrameDuration; maxFrameDuration_ = maxFrameDuration ? maxFrameDuration : defaultMinFrameDuration; minFrameDuration_ = std::clamp(minFrameDuration_, - minSensorFrameDuration, maxSensorFrameDuration); + sensorLimits_.minFrameDuration, sensorLimits_.maxFrameDuration); maxFrameDuration_ = std::clamp(maxFrameDuration_, - minSensorFrameDuration, maxSensorFrameDuration); + sensorLimits_.minFrameDuration, sensorLimits_.maxFrameDuration); maxFrameDuration_ = std::max(maxFrameDuration_, minFrameDuration_); /* Return the validated limits via metadata. */ @@ -1505,17 +1513,18 @@ void IPARPi::applyFrameDurations(Duration minFrameDuration, Duration maxFrameDur * getBlanking() will update maxShutter with the largest exposure * value possible. */ - RPiController::AgcAlgorithm::SensorLimits limits; - limits.maxShutter = Duration::max(); - helper_->getBlanking(limits.maxShutter, minFrameDuration_, maxFrameDuration_); + sensorLimits_.maxShutter = Duration::max(); + helper_->getBlanking(sensorLimits_.maxShutter, minFrameDuration_, maxFrameDuration_); RPiController::AgcAlgorithm *agc = dynamic_cast( controller_.getAlgorithm("agc")); - agc->setSensorLimits(limits); + agc->setSensorLimits(sensorLimits_); } void IPARPi::applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls) { + const int32_t minGainCode = helper_->gainCode(sensorLimits_.minAnalogueGain); + const int32_t maxGainCode = helper_->gainCode(sensorLimits_.maxAnalogueGain); int32_t gainCode = helper_->gainCode(agcStatus->analogueGain); /* @@ -1523,7 +1532,7 @@ void IPARPi::applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls) * DelayedControls. The AGC will correctly handle a lower gain returned * by the sensor, provided it knows the actual gain used. */ - gainCode = std::min(gainCode, maxSensorGainCode_); + gainCode = std::clamp(gainCode, minGainCode, maxGainCode); /* getBlanking might clip exposure time to the fps limits. */ Duration exposure = agcStatus->shutterTime;