From patchwork Mon Mar 27 14:06:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18480 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F3217C0F2A for ; Mon, 27 Mar 2023 14:06:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1EAF66271C; Mon, 27 Mar 2023 16:06:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679926000; bh=GKuSCN9Yy/eVZDMQ1xwKyTmP6GB4FWUbJg8lH/iBHGI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=dFmeVfL/g9KtbkPuaWcvIUZ0puoKDgG3jDDIg/n2uQ/X5zDflZ31pb5C/kfybhzMy lscl5GUpoXqMQibb9VS38jHgFiIvQfqVJLifgXkCcF5xXWVIWFUixDPaQrjs2AGNom irRnG/+R3U9FiRMKPvbN/pklwaeUqd9E2Pk8M2kKVxmRWoPBSHZN/pDM6WAa1TUtDv T8deDmAS8HAFjiuxtXGQ/vWrWnUfWslXZfHB8OME+x+Amew6ElP/yBowe3gUmI2esP C4X2mcvprya1igjW6mw1RXOimusdgLi9RcFIlBfvv7bmc1Cxxjr8x4MMCHauaHJWU4 4/mX38vJ1vMVg== Received: from mail-il1-x12f.google.com (mail-il1-x12f.google.com [IPv6:2607:f8b0:4864:20::12f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 539F4626D7 for ; Mon, 27 Mar 2023 16:06:38 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="L+wNnZwg"; dkim-atps=neutral Received: by mail-il1-x12f.google.com with SMTP id h14so3313852ilj.0 for ; Mon, 27 Mar 2023 07:06:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679925997; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/4bT4+XhpmutDlxLJohIF3zTi9O9kJXYZaoNU/eOy8Q=; b=L+wNnZwg5LfMsby8+7PtEuc1wu8SQSGeLl9thf6L+L3BjWBKK1wBBGatbJ/gxPkyoY 4LZq9if4H6O8WAUspmuEctSo4wQ9yiOmdV7qt2v/nBGp5ymikjjgBxoIb7bUKuYQ4fjt J83KD3liTRGq4ihJrQA1kQAH92z4fxUpro7mUCGZicJuU7PMizUOsgKPAHRXGPPEa3de IDToN3PkKt8zPd7s9zVkJ1jJMLaOTIJ573PTFgNqpShCZvXidacn/Q3n3cCwg2roQr6A SRHZyKhwtNwwonDA+XepPLIGcNtok5G/8QU6MhDamlvJrnz+Esk/oEOpMunOe7GgSLCE M8rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679925997; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/4bT4+XhpmutDlxLJohIF3zTi9O9kJXYZaoNU/eOy8Q=; b=RUvz2aVk1gHVPjC5DN5lej6J7Mc9MOOsfYupmvDq+S+HzaqI6F7Xo8JkHlJcNI1RJF xOS++mjcoigdj5bodQAuVIVvm9NeKrpExPdcH7/3Ag4jIFmkH7WDQKBXY5EsbZXC5dR7 7bDXgI9VNnKZh1UBmUvaNYVm7qw0dMNJ3lasDgClfTIb6Tja0RNab0rhyFsWukpK4no4 l76tFZlyRxf3bElsnUNOXd7BZEqNsgUgFl2FWaOlv3V7KbGPHxCZPxN/qz2xBFx9q7vX aH0BTAtGmBlC7db37VPxfVBIY2kv9c6AWlCpyqXnVMYeT8aGAqt/obpQ3xayGEx7dMDl X4Jw== X-Gm-Message-State: AAQBX9e0w0GYS9Jw3jLhtr1th6HCLPXeGT1dAXT2h6itLuus5VPia+7m wz5lCaz4FI3NXdVquPzxPDkdlLTL315zARgMxR85MA== X-Google-Smtp-Source: AKy350a8n+f46HQQTEoUsQmvd8qxsswwxuqNNk/qjBK/Cagb/GLiqyxZhgJchEJCPKFjqfuoxQSOBQ== X-Received: by 2002:a92:d812:0:b0:325:cfe4:8c3 with SMTP id y18-20020a92d812000000b00325cfe408c3mr9117397ilm.30.1679925996748; Mon, 27 Mar 2023 07:06:36 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id a12-20020a056e020e0c00b00325f7bfc651sm1483297ilk.59.2023.03.27.07.06.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Mar 2023 07:06:36 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Mar 2023 15:06:25 +0100 Message-Id: <20230327140625.6624-1-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230327093439.24670-4-naush@raspberrypi.com> References: <20230327093439.24670-4-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH] ipa: raspberrypi: Ensure shutter speed and gain are clipped in the AGC X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Make a copy of the CameraMode structure on a switch mode call. This replaces the existing lastSensitivity_ field. Limit the AGC gain calculations to the minimum value given by the CameraMode structure. The maximum value remains unclipped as any gain over the sensor maximum will be made up by digital gain. Rename clipShutter to limitShutter for consistency, and have the latter limit the shutter speed to both upper and lower bounds. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 58 ++++++++++++++++------ src/ipa/raspberrypi/controller/rpi/agc.h | 5 +- 2 files changed, 47 insertions(+), 16 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 4ea0dd41e66c..1776262edeb5 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -309,14 +309,14 @@ void Agc::setFixedShutter(Duration fixedShutter) { fixedShutter_ = fixedShutter; /* Set this in case someone calls disableAuto() straight after. */ - status_.shutterTime = clipShutter(fixedShutter_); + status_.shutterTime = limitShutter(fixedShutter_); } void Agc::setFixedAnalogueGain(double fixedAnalogueGain) { fixedAnalogueGain_ = fixedAnalogueGain; /* Set this in case someone calls disableAuto() straight after. */ - status_.analogueGain = fixedAnalogueGain; + status_.analogueGain = limitGain(fixedAnalogueGain); } void Agc::setMeteringMode(std::string const &meteringModeName) @@ -342,7 +342,14 @@ void Agc::switchMode(CameraMode const &cameraMode, housekeepConfig(); - Duration fixedShutter = clipShutter(fixedShutter_); + /* + * Store the mode in the local state. We must cache the sensitivity of + * of the previous mode for the calculations below. + */ + double lastSensitivity = mode_.sensitivity; + mode_ = cameraMode; + + Duration fixedShutter = limitShutter(fixedShutter_); if (fixedShutter && fixedAnalogueGain_) { /* We're going to reset the algorithm here with these fixed values. */ @@ -371,7 +378,7 @@ void Agc::switchMode(CameraMode const &cameraMode, * current exposure profile, which takes care of everything else. */ - double ratio = lastSensitivity_ / cameraMode.sensitivity; + double ratio = lastSensitivity / cameraMode.sensitivity; target_.totalExposureNoDG *= ratio; target_.totalExposure *= ratio; filtered_.totalExposureNoDG *= ratio; @@ -392,9 +399,6 @@ void Agc::switchMode(CameraMode const &cameraMode, } writeAndFinish(metadata, false); - - /* We must remember the sensitivity of this mode for the next SwitchMode. */ - lastSensitivity_ = cameraMode.sensitivity; } void Agc::prepare(Metadata *imageMetadata) @@ -516,7 +520,7 @@ void Agc::housekeepConfig() { /* First fetch all the up-to-date settings, so no one else has to do it. */ status_.ev = ev_; - status_.fixedShutter = clipShutter(fixedShutter_); + status_.fixedShutter = limitShutter(fixedShutter_); status_.fixedAnalogueGain = fixedAnalogueGain_; status_.flickerPeriod = flickerPeriod_; LOG(RPiAgc, Debug) << "ev " << status_.ev << " fixedShutter " @@ -703,7 +707,7 @@ void Agc::computeTargetExposure(double gain) Duration maxShutter = status_.fixedShutter ? status_.fixedShutter : exposureMode_->shutter.back(); - maxShutter = clipShutter(maxShutter); + maxShutter = limitShutter(maxShutter); Duration maxTotalExposure = maxShutter * (status_.fixedAnalogueGain != 0.0 @@ -803,15 +807,16 @@ void Agc::divideUpExposure() double analogueGain; shutterTime = status_.fixedShutter ? status_.fixedShutter : exposureMode_->shutter[0]; - shutterTime = clipShutter(shutterTime); + shutterTime = limitShutter(shutterTime); analogueGain = status_.fixedAnalogueGain != 0.0 ? status_.fixedAnalogueGain : exposureMode_->gain[0]; + analogueGain = limitGain(analogueGain); if (shutterTime * analogueGain < exposureValue) { for (unsigned int stage = 1; stage < exposureMode_->gain.size(); stage++) { if (!status_.fixedShutter) { Duration stageShutter = - clipShutter(exposureMode_->shutter[stage]); + limitShutter(exposureMode_->shutter[stage]); if (stageShutter * analogueGain >= exposureValue) { shutterTime = exposureValue / analogueGain; break; @@ -824,6 +829,7 @@ void Agc::divideUpExposure() break; } analogueGain = exposureMode_->gain[stage]; + analogueGain = limitGain(analogueGain); } } } @@ -846,6 +852,7 @@ void Agc::divideUpExposure() * gain as a side-effect. */ analogueGain = std::min(analogueGain, exposureMode_->gain.back()); + analogueGain = limitGain(analogueGain); shutterTime = newShutterTime; } LOG(RPiAgc, Debug) << "After flicker avoidance, shutter " @@ -872,13 +879,36 @@ void Agc::writeAndFinish(Metadata *imageMetadata, bool desaturate) << " analogue gain " << filtered_.analogueGain; } -Duration Agc::clipShutter(Duration shutter) +Duration Agc::limitShutter(Duration shutter) { - if (maxShutter_) - shutter = std::min(shutter, maxShutter_); + /* + * shutter == 0 is a special case for fixed shutter values, and must pass + * through unchanged + */ + if (!shutter) + return shutter; + + shutter = std::clamp(shutter, mode_.minShutter, maxShutter_); return shutter; } +double Agc::limitGain(double gain) const +{ + /* + * Only limit the lower bounds of the gain value to what the sensor limits. + * The upper bound on analogue gain will be made up with additional digital + * gain applied by the ISP. + * + * gain == 0.0 is a special case for fixed shutter values, and must pass + * through unchanged + */ + if (!gain) + return gain; + + gain = std::max(gain, mode_.minAnalogueGain); + return gain; +} + /* Register algorithm with the system. */ static Algorithm *create(Controller *controller) { diff --git a/src/ipa/raspberrypi/controller/rpi/agc.h b/src/ipa/raspberrypi/controller/rpi/agc.h index f04896ca25ad..4fa9b8de5fb2 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.h +++ b/src/ipa/raspberrypi/controller/rpi/agc.h @@ -103,10 +103,12 @@ private: void filterExposure(bool desaturate); void divideUpExposure(); void writeAndFinish(Metadata *imageMetadata, bool desaturate); - libcamera::utils::Duration clipShutter(libcamera::utils::Duration shutter); + libcamera::utils::Duration limitShutter(libcamera::utils::Duration shutter); + double limitGain(double gain) const; AgcMeteringMode *meteringMode_; AgcExposureMode *exposureMode_; AgcConstraintMode *constraintMode_; + CameraMode mode_; uint64_t frameCount_; AwbStatus awb_; struct ExposureValues { @@ -124,7 +126,6 @@ private: int lockCount_; DeviceStatus lastDeviceStatus_; libcamera::utils::Duration lastTargetExposure_; - double lastSensitivity_; /* sensitivity of the previous camera mode */ /* Below here the "settings" that applications can change. */ std::string meteringModeName_; std::string exposureModeName_;