From patchwork Wed Jun 2 15:21:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 12490 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E6C09C3208 for ; Wed, 2 Jun 2021 15:21:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7812B68923; Wed, 2 Jun 2021 17:21:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="aJHTdCGn"; dkim-atps=neutral Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2C256602A4 for ; Wed, 2 Jun 2021 17:21:27 +0200 (CEST) Received: by mail-wm1-x330.google.com with SMTP id b145-20020a1c80970000b029019c8c824054so4006766wmd.5 for ; Wed, 02 Jun 2021 08:21:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VcVXDLtZMS/ZdvZzR6ROcLDdyirQXTZQvOsQmAI0Pzk=; b=aJHTdCGnQB9kpd2TAJvuC8Iq2SoQCs9dOHR+4t6IQA9PpRLl6hnTfLkINvfkSLNB3j AB6JPVjBMhIjAeUCijGLnDVcuB3A57yZ6ZonBlGkL4a4WW8jmZ98/SoalR9yOUBKNi2o BhBj3nTRW7zKMxdlDYdErr0YfJU7IG6Y+Bsj3XOW2ikKKXkBUiLEpRBMuOloIDmNQNrH xflWx9sjGMKNU8bVLC6GNFwc9s0BmT1GhE4Hgcf4K0HnStBetxLd7vWpF9gzLU6/XneT WnvJ0IKQGI28IEg+pAlJ4VGl19zvQfCxwktYTuZkgmXTtqn2yXVV8QtfNCSTrZ4ofeaO 6UTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VcVXDLtZMS/ZdvZzR6ROcLDdyirQXTZQvOsQmAI0Pzk=; b=DNAoYSzesWzHrtQVoxYHcc7bFIWCrJ/JRUhdSRAnm1VFbkY74/H7zi7UJQ+v/hJBhO GsH/eKxBJzVQN+rBSDQLMYjEnk2CFAxFq0OG/m4dvxV6bvyTef5b3ASSNLB/BhgG7B9s WcUQdMx7y87aBjK6hBMUwg5CHXo52lVVXDqSsMra74Tqlfdb/ZHTMnY1pLDW8rHssCeq +6IOd6ZhUCorBv3EM1JFo13XsrTZ/eMMN+JP6YEgAVobhI6AAbNmdoSQH/ar1Anh8VG6 DU9ik3YE7JTPCdsQbk1slXOIOB05X7x9Fade4bnq7NC0eHyToKRtDzX+vh01rhZJ9bsf fxfQ== X-Gm-Message-State: AOAM532o/jSoZwDkeaRseFR3aj0pUqBXAtC7SIRea1rqUsMw62zCbLI/ L/ApbWScsrsly5G+0uz99dQQn/vWgfjA+UWJ X-Google-Smtp-Source: ABdhPJyZlFUuijQtWPWWxs6sxbxBi3IfaVwAJjXlAtJB7YUwSJc1bRzlrTZY/zF94HCO7KYnyK0GZA== X-Received: by 2002:a1c:9804:: with SMTP id a4mr32486779wme.34.1622647286440; Wed, 02 Jun 2021 08:21:26 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id p1sm57044wmc.11.2021.06.02.08.21.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Jun 2021 08:21:26 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Wed, 2 Jun 2021 16:21:21 +0100 Message-Id: <20210602152122.14836-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210602152122.14836-1-david.plowman@raspberrypi.com> References: <20210602152122.14836-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/2] ipa: raspberrypi: Add sensitivity field to camera mode X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We use the CamHelper class to initialise it to the usual value of 1. The CamHelper's GetModeSensitivity method can be redefined to implement a different behaviour for sensors that require it. Signed-off-by: David Plowman --- src/ipa/raspberrypi/cam_helper.cpp | 11 +++++++++++ src/ipa/raspberrypi/cam_helper.hpp | 3 +++ src/ipa/raspberrypi/controller/camera_mode.h | 2 ++ src/ipa/raspberrypi/raspberrypi.cpp | 6 ++++++ 4 files changed, 22 insertions(+) diff --git a/src/ipa/raspberrypi/cam_helper.cpp b/src/ipa/raspberrypi/cam_helper.cpp index 09917f3c..c1dd0daa 100644 --- a/src/ipa/raspberrypi/cam_helper.cpp +++ b/src/ipa/raspberrypi/cam_helper.cpp @@ -128,6 +128,17 @@ bool CamHelper::SensorEmbeddedDataPresent() const return false; } +double CamHelper::GetModeSensitivity([[maybe_unused]] const CameraMode &mode) const +{ + /* + * Most sensors have the same sensitivity in every mode, but this + * method can be overridden for those that do not. Note that it is + * called before mode_ is set, so it must return the sensitivity + * of the mode that is passed in. + */ + return 1.0; +} + unsigned int CamHelper::HideFramesStartup() const { /* diff --git a/src/ipa/raspberrypi/cam_helper.hpp b/src/ipa/raspberrypi/cam_helper.hpp index a52f3f0b..bc034877 100644 --- a/src/ipa/raspberrypi/cam_helper.hpp +++ b/src/ipa/raspberrypi/cam_helper.hpp @@ -39,6 +39,8 @@ namespace RPiController { // // A method to query if the sensor outputs embedded data that can be parsed. // +// A method to return the sensitivity of a given camera mode. +// // A parser to parse the embedded data buffers provided by some sensors (for // example, the imx219 does; the ov5647 doesn't). This allows us to know for // sure the exposure and gain of the frame we're looking at. CamHelper @@ -81,6 +83,7 @@ public: virtual void GetDelays(int &exposure_delay, int &gain_delay, int &vblank_delay) const; virtual bool SensorEmbeddedDataPresent() const; + virtual double GetModeSensitivity(const CameraMode &mode) const; virtual unsigned int HideFramesStartup() const; virtual unsigned int HideFramesModeSwitch() const; virtual unsigned int MistrustFramesStartup() const; diff --git a/src/ipa/raspberrypi/controller/camera_mode.h b/src/ipa/raspberrypi/controller/camera_mode.h index 256438c9..326f4f20 100644 --- a/src/ipa/raspberrypi/controller/camera_mode.h +++ b/src/ipa/raspberrypi/controller/camera_mode.h @@ -39,6 +39,8 @@ struct CameraMode { libcamera::Transform transform; // minimum and maximum fame lengths in units of lines uint32_t min_frame_length, max_frame_length; + // sensitivity of this mode + double sensitivity; }; #ifdef __cplusplus diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 0c4752ec..3da7e232 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -321,6 +321,12 @@ void IPARPi::setMode(const IPACameraSensorInfo &sensorInfo) */ mode_.min_frame_length = sensorInfo.minFrameLength; mode_.max_frame_length = sensorInfo.maxFrameLength; + + /* + * Some sensors may have different sensitivities in different modes; + * the CamHelper will know the correct value. + */ + mode_.sensitivity = helper_->GetModeSensitivity(mode_); } int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, From patchwork Wed Jun 2 15:21:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 12491 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7FCD7C3209 for ; Wed, 2 Jun 2021 15:21:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CB3326892D; Wed, 2 Jun 2021 17:21:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="naJpCNqn"; dkim-atps=neutral Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 50362602AF for ; Wed, 2 Jun 2021 17:21:27 +0200 (CEST) Received: by mail-wm1-x330.google.com with SMTP id p13-20020a05600c358db029019f44afc845so1900113wmq.3 for ; Wed, 02 Jun 2021 08:21:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/68s40iRMrqonldzS1duTI3qAQU2QsASwzXb6nw6uwQ=; b=naJpCNqnwP1Rufj3fDk1QFbA/094uAWw49JiCGxGT6vH3+79/VaK5ZRR3cbfLZkIXM 9JVTMrSLfJsOvuDh8wb5tw0rpzHyGFvZmsyAWkdq8K7AjwGXI/M8oeQqF4yQLCwWO++p YMkNIoOBZDbZaXy4BKJ5sPZhQbIas11k4yaA8C5uUdPN9N6T3hkqWNgYaW0VPe5qtdKr kOFyKvrfB0cC4KSeorr0vRg1lhzCUWCZRVi5A3/wlv1I8gbkPWDVy+IYtbvwuRGtS6iv 3HT9m8XUARLZTSmw29fJm+w1AAuf/d5ApJREyebdtr3pFJDTvS54I+Y2aPjmeYHNJdIj I1PA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/68s40iRMrqonldzS1duTI3qAQU2QsASwzXb6nw6uwQ=; b=iv8fRdCIJCsMTHqHXzUseZ9g0ZkkSJyZGFqlZMoBMMB4kitWuXDnl6BAre6MJ0PelQ R3NxohIS1IsJSTQ766+mvCCr0Z5s86OV62NBbgcW3WZXFoVdQOt1T93VHXjAvNmjjiQo S089T758b+avySDLMurKSvla7PRXbWO3aV5BAkriyvlzh6Wy0gAnodIUIxM42l45FrWo 3mvpeMv6PYzMFxuYSQ5Clo/8HxgcmfIrHAS4ZsSp5Cdy/OlV4o1/0iScjiPkiBhJzsl+ KDUDYgr+KU+6BfstRXwehjParyWACoVXVCOwjdekEu8u2ifGFm7DhfGkmfgnccJaMRu4 CuhQ== X-Gm-Message-State: AOAM530Bw/aqL7335SRvEeKvTj7FKQLNdn8G7GqhmtMjpSTXftNK8aRj bh0V3Ou0KwUmQ9CILBS6yI2fVPaWhfM7yNL0 X-Google-Smtp-Source: ABdhPJywsj/9e4IcE6FaabRvjdlo0huX/p2yDyDxSB9MKAC9uxWhdfxTPgPm6JhkAgoe9bI2I3B1sw== X-Received: by 2002:a7b:c44f:: with SMTP id l15mr5809899wmi.151.1622647286906; Wed, 02 Jun 2021 08:21:26 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id p1sm57044wmc.11.2021.06.02.08.21.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Jun 2021 08:21:26 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Wed, 2 Jun 2021 16:21:22 +0100 Message-Id: <20210602152122.14836-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210602152122.14836-1-david.plowman@raspberrypi.com> References: <20210602152122.14836-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/2] ipa: raspberrypi: AGC: handle modes with different sensitivities X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the sensor is switched to a mode with a different sensitivity, the target exposure values need to be adjusted proportionately to maintain the same image brightness. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 25 +++++++++++++++++----- src/ipa/raspberrypi/controller/rpi/agc.hpp | 1 + 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index f4cd5d26..6f0e9702 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -155,7 +155,7 @@ Agc::Agc(Controller *controller) : AgcAlgorithm(controller), metering_mode_(nullptr), exposure_mode_(nullptr), constraint_mode_(nullptr), frame_count_(0), lock_count_(0), - last_target_exposure_(0.0), + last_target_exposure_(0.0), last_sensitivity_(0.0), ev_(1.0), flicker_period_(0.0), max_shutter_(0), fixed_shutter_(0), fixed_analogue_gain_(0.0) { @@ -261,7 +261,7 @@ void Agc::SetConstraintMode(std::string const &constraint_mode_name) constraint_mode_name_ = constraint_mode_name; } -void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, +void Agc::SwitchMode(CameraMode const &camera_mode, Metadata *metadata) { housekeepConfig(); @@ -285,9 +285,20 @@ void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, filtered_.shutter = fixed_shutter; filtered_.analogue_gain = fixed_analogue_gain_; } else if (status_.total_exposure_value) { - // On a mode switch, it's possible the exposure profile could change, - // or a fixed exposure/gain might be set so we divide up the exposure/ - // gain again, but we don't change any target values. + // On a mode switch, various things could happen: + // - the exposure profile might change + // - a fixed exposure or gain might be set + // - the new mode's sensitivity might be different + // We cope with the last of these by scaling the target values. After + // that we just need to re-divide the exposure/gain according to the + // current exposure profile, which takes care of everything else. + + double ratio = last_sensitivity_ / camera_mode.sensitivity; + target_.total_exposure_no_dg *= ratio; + target_.total_exposure *= ratio; + filtered_.total_exposure_no_dg *= ratio; + filtered_.total_exposure *= ratio; + divideUpExposure(); } else { // We come through here on startup, when at least one of the shutter @@ -301,6 +312,10 @@ void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, } writeAndFinish(metadata, false); + + // We must remember the sensitivity of this mode for the next SwitchMode. + ASSERT(camera_mode.sensitivity != 0.0); + last_sensitivity_ = camera_mode.sensitivity; } void Agc::Prepare(Metadata *image_metadata) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index 0427fb59..025878a2 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -122,6 +122,7 @@ private: int lock_count_; DeviceStatus last_device_status_; double last_target_exposure_; + double last_sensitivity_; // sensitivity of the previous camera mode // Below here the "settings" that applications can change. std::string metering_mode_name_; std::string exposure_mode_name_;