From patchwork Sun Jul 18 18:51:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 13037 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6CDB4C0109 for ; Sun, 18 Jul 2021 18:51:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2373368540; Sun, 18 Jul 2021 20:51:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Mmq7XsnF"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E916568521 for ; Sun, 18 Jul 2021 20:51:30 +0200 (CEST) Received: by mail-wm1-x336.google.com with SMTP id a23-20020a05600c2257b0290236ec98bebaso6638548wmm.1 for ; Sun, 18 Jul 2021 11:51:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=okO5up/zx0X+4DJQ5oxsTFvRqTM5be38sIC2FgZlx80=; b=Mmq7XsnFeLVYvTM/clIBTIFzjrR8eWA+zFsEEz0OySXATNU0iGplO+Rp9ocN7lM/vn 2HgsBC2HMRJn/+pvxmj5KJY2Xi6TydoJ43AzynM/Y81DVU0Ent0jD0lNBHqcAZUOv/81 Kykt25CalZoyfwd7zQXMI7dTQvVrzY8GOrV6LMGGGg7FinU+mpqDhU99mgkka17xJjsg DWTEBHcdcGal9BJdR75uWTacjypC1URBG/SOFHvwjxL6nYyrWq1vgSJtYzTE7GQwPCcc JTQIYigVHWEaKFy919kUsdjvKE3NOJGD/+mu/8xA5VvcfG/DPyD3b5FdF83eOfd7jsE6 9knw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=okO5up/zx0X+4DJQ5oxsTFvRqTM5be38sIC2FgZlx80=; b=hGDdZftGv/hyNU23Z9jB63ho7p/BwEG1C6Z9wkTBlqICByJKo5YXQIeHHneXZPb4oX pR8OtGfmy7DvwHKsGNpffoBYI/NEdNwq+vdbO4kJgPC8sLyZ/VZFyFyMMO8CHs1kDT3G K2fZUfefFG8RestnrELQptppdDwP+/Z0oRYJCz6yftn9M5aWIgPKGnhmVfNZrmeIC8LC fDDwaXzrNX940rgElrye/g/k2/8HXMRCYO+M+OOzsiUM1n9vOWgzlbx+YY2ui5QOE0BD l9zFOjT2LGRSkTrF+eAi3QibXO1OdVZMaSQQS1TBzqXaCHJPXzd0t9sgUUem/yxtejCp 00Gw== X-Gm-Message-State: AOAM530molNNK1NZ9o0ziz7wRZjnST0jXzckl7PDDfd8xilKApnbSJqi KwB+05Gq3+mP/X2n6CGiiuxX/dzYU7vMhBMr X-Google-Smtp-Source: ABdhPJzb+PCijbghfN/J6NdWsoJo514Y7sgcYewcww1/IGNGLVGr9TSiA8kUG/oJe+89webKCKDyMA== X-Received: by 2002:a7b:c013:: with SMTP id c19mr28053559wmb.158.1626634290425; Sun, 18 Jul 2021 11:51:30 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id o19sm17459634wmc.12.2021.07.18.11.51.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 18 Jul 2021 11:51:29 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Sun, 18 Jul 2021 19:51:24 +0100 Message-Id: <20210718185125.17472-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210718185125.17472-1-david.plowman@raspberrypi.com> References: <20210718185125.17472-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 1/2] ipa: raspberrypi: Add sensitivity field to camera mode X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We use the CamHelper class to initialise it to the usual value of 1. The CamHelper's GetModeSensitivity method can be redefined to implement a different behaviour for sensors that require it. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Acked-by: Kieran Bingham --- src/ipa/raspberrypi/cam_helper.cpp | 11 +++++++++++ src/ipa/raspberrypi/cam_helper.hpp | 3 +++ src/ipa/raspberrypi/controller/camera_mode.h | 2 ++ src/ipa/raspberrypi/raspberrypi.cpp | 6 ++++++ 4 files changed, 22 insertions(+) diff --git a/src/ipa/raspberrypi/cam_helper.cpp b/src/ipa/raspberrypi/cam_helper.cpp index 3c6afce7..8f64e014 100644 --- a/src/ipa/raspberrypi/cam_helper.cpp +++ b/src/ipa/raspberrypi/cam_helper.cpp @@ -129,6 +129,17 @@ bool CamHelper::SensorEmbeddedDataPresent() const return false; } +double CamHelper::GetModeSensitivity([[maybe_unused]] const CameraMode &mode) const +{ + /* + * Most sensors have the same sensitivity in every mode, but this + * method can be overridden for those that do not. Note that it is + * called before mode_ is set, so it must return the sensitivity + * of the mode that is passed in. + */ + return 1.0; +} + unsigned int CamHelper::HideFramesStartup() const { /* diff --git a/src/ipa/raspberrypi/cam_helper.hpp b/src/ipa/raspberrypi/cam_helper.hpp index 200cc83f..0643b77a 100644 --- a/src/ipa/raspberrypi/cam_helper.hpp +++ b/src/ipa/raspberrypi/cam_helper.hpp @@ -41,6 +41,8 @@ namespace RPiController { // // A method to query if the sensor outputs embedded data that can be parsed. // +// A method to return the sensitivity of a given camera mode. +// // A parser to parse the embedded data buffers provided by some sensors (for // example, the imx219 does; the ov5647 doesn't). This allows us to know for // sure the exposure and gain of the frame we're looking at. CamHelper @@ -84,6 +86,7 @@ public: virtual void GetDelays(int &exposure_delay, int &gain_delay, int &vblank_delay) const; virtual bool SensorEmbeddedDataPresent() const; + virtual double GetModeSensitivity(const CameraMode &mode) const; virtual unsigned int HideFramesStartup() const; virtual unsigned int HideFramesModeSwitch() const; virtual unsigned int MistrustFramesStartup() const; diff --git a/src/ipa/raspberrypi/controller/camera_mode.h b/src/ipa/raspberrypi/controller/camera_mode.h index 65888230..e2b82828 100644 --- a/src/ipa/raspberrypi/controller/camera_mode.h +++ b/src/ipa/raspberrypi/controller/camera_mode.h @@ -41,6 +41,8 @@ struct CameraMode { libcamera::Transform transform; // minimum and maximum fame lengths in units of lines uint32_t min_frame_length, max_frame_length; + // sensitivity of this mode + double sensitivity; }; #ifdef __cplusplus diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 5cd33304..2fdb60a9 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -325,6 +325,12 @@ void IPARPi::setMode(const IPACameraSensorInfo &sensorInfo) */ mode_.min_frame_length = sensorInfo.minFrameLength; mode_.max_frame_length = sensorInfo.maxFrameLength; + + /* + * Some sensors may have different sensitivities in different modes; + * the CamHelper will know the correct value. + */ + mode_.sensitivity = helper_->GetModeSensitivity(mode_); } int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, From patchwork Sun Jul 18 18:51:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 13038 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 36FE8C3229 for ; Sun, 18 Jul 2021 18:51:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6F0FB68543; Sun, 18 Jul 2021 20:51:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="VGLrKUJK"; dkim-atps=neutral Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 79CC368521 for ; Sun, 18 Jul 2021 20:51:31 +0200 (CEST) Received: by mail-wm1-x331.google.com with SMTP id f190so7389877wmf.4 for ; Sun, 18 Jul 2021 11:51:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0bO8wlqoaF3DO4FeBDhDlKGfNfhap7UA/slY9vWYmm8=; b=VGLrKUJKkmAKk3BUsRM7pl7Wk2+MJklb2s/GEr4ZQK9h2KF3Ti6OyMr/NdvoCsrdrp hCCiVN9fjJCG/YDg6Jhc/17eJ3Tw4beRsxOU3IAD3Cw35OaKSAGtGpMZ4nRgtd9SWiUf xrkdFS8GDhdUw5VuZA3z3CcznHGgzqr2Nbi+nZNVUFCLfzai56tqE490cTbN6dYtl+3F S1I4zLqqUFNnEp7LpNLyolmxjMguJd4Sw6Nm2BJ/yX3f32r+1vVukkZ1bEu8aMbaHyih XrW2A4iY5d2d1ddQgyDxA/H95JqmwqQFSJqS4YEZERau01DjBsd9BWnJ0KZSIbW+sBKT Oe2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0bO8wlqoaF3DO4FeBDhDlKGfNfhap7UA/slY9vWYmm8=; b=mSCDIyZDmgEGk5otDwVU8GW0vhNUSh82qzq3l+KYn9vJNKICl2Ufb0vk9Orioi9HW+ 9X8YZ0SF82PSo7K0RsXduN/MfnGJ/3mhutBNMlPWWQtOstdE/lMXf2D/7/3t9GYkVAsW q6ocHH54af71BZJAihUizgtPc9P0R5BD5nTyVFmpqQnW41xsC+dc7YLQKIX4EZX2uCLF I5rxzQnL2hpnBJhUyuprRgxhae6XnWva8W5swmehn2S4ipaluy85CxqUcjYBJDZ8kSiF gT872UF67jeK5DR1VYYsA302a6pfrlk+OktYO9A8C60M9vI5ujO9fF6JATJ3C+Yo32qS C90A== X-Gm-Message-State: AOAM5332l+Ous1N39enX9NIfnaCJb7EPTBh4RMHlGrmjFGNb4fbceP4E hwB3F9EkxG8C7+3OME2rpAiimA2vTgkoFYsi X-Google-Smtp-Source: ABdhPJzSXUc6dx85PM8NQBqkn1EGSmONDO8lTtDEcxxNTnaoodLfdFhXec+bTDR0lps1FjI3lVkR1Q== X-Received: by 2002:a7b:c850:: with SMTP id c16mr12592442wml.177.1626634291079; Sun, 18 Jul 2021 11:51:31 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id o19sm17459634wmc.12.2021.07.18.11.51.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 18 Jul 2021 11:51:30 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Sun, 18 Jul 2021 19:51:25 +0100 Message-Id: <20210718185125.17472-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210718185125.17472-1-david.plowman@raspberrypi.com> References: <20210718185125.17472-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 2/2] ipa: raspberrypi: AGC: handle modes with different sensitivities X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the sensor is switched to a mode with a different sensitivity, the target exposure values need to be adjusted proportionately to maintain the same image brightness. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 27 ++++++++++++++++++---- src/ipa/raspberrypi/controller/rpi/agc.hpp | 1 + 2 files changed, 23 insertions(+), 5 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index f57783f8..1cfdd3bb 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -163,7 +163,7 @@ Agc::Agc(Controller *controller) : AgcAlgorithm(controller), metering_mode_(nullptr), exposure_mode_(nullptr), constraint_mode_(nullptr), frame_count_(0), lock_count_(0), - last_target_exposure_(0s), + last_target_exposure_(0s), last_sensitivity_(0.0), ev_(1.0), flicker_period_(0s), max_shutter_(0s), fixed_shutter_(0s), fixed_analogue_gain_(0.0) { @@ -268,9 +268,12 @@ void Agc::SetConstraintMode(std::string const &constraint_mode_name) constraint_mode_name_ = constraint_mode_name; } -void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, +void Agc::SwitchMode(CameraMode const &camera_mode, Metadata *metadata) { + /* AGC expects the mode sensitivity always to be non-zero. */ + ASSERT(camera_mode.sensitivity); + housekeepConfig(); Duration fixed_shutter = clipShutter(fixed_shutter_); @@ -292,9 +295,20 @@ void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, filtered_.shutter = fixed_shutter; filtered_.analogue_gain = fixed_analogue_gain_; } else if (status_.total_exposure_value) { - // On a mode switch, it's possible the exposure profile could change, - // or a fixed exposure/gain might be set so we divide up the exposure/ - // gain again, but we don't change any target values. + // On a mode switch, various things could happen: + // - the exposure profile might change + // - a fixed exposure or gain might be set + // - the new mode's sensitivity might be different + // We cope with the last of these by scaling the target values. After + // that we just need to re-divide the exposure/gain according to the + // current exposure profile, which takes care of everything else. + + double ratio = last_sensitivity_ / camera_mode.sensitivity; + target_.total_exposure_no_dg *= ratio; + target_.total_exposure *= ratio; + filtered_.total_exposure_no_dg *= ratio; + filtered_.total_exposure *= ratio; + divideUpExposure(); } else { // We come through here on startup, when at least one of the shutter @@ -308,6 +322,9 @@ void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, } writeAndFinish(metadata, false); + + // We must remember the sensitivity of this mode for the next SwitchMode. + last_sensitivity_ = camera_mode.sensitivity; } void Agc::Prepare(Metadata *image_metadata) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index 85067dc6..82063636 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -126,6 +126,7 @@ private: int lock_count_; DeviceStatus last_device_status_; libcamera::utils::Duration last_target_exposure_; + double last_sensitivity_; // sensitivity of the previous camera mode // Below here the "settings" that applications can change. std::string metering_mode_name_; std::string exposure_mode_name_;