From patchwork Fri Jun 11 10:07:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 12569 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C1820C3214 for ; Fri, 11 Jun 2021 10:07:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 52F4068934; Fri, 11 Jun 2021 12:07:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qz5QvdHO"; dkim-atps=neutral Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6AC8F6892C for ; Fri, 11 Jun 2021 12:07:08 +0200 (CEST) Received: by mail-wm1-x32a.google.com with SMTP id f16-20020a05600c1550b02901b00c1be4abso8321729wmg.2 for ; Fri, 11 Jun 2021 03:07:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XACfAo3TeGNF0rp/SYZW+awF5o1J3CDFB0MciljeeOQ=; b=qz5QvdHOUGe1IfPGt8bnaBRkzgD5JoJGSk//DmlSH1ufFG3pdMkbMXZO7hSGNFKIGe ess5jQITrVMiupk3fDL4409lo3L0KC7vwIT0T69woQBsPBvZDbUZ4iK3gOA9eAWMCo93 wfgRmfvf/Fcd8KjxZ5YvRYlSwa+Of2OY0NnUbzAHlcXAvp5LmZZTDdLX00TZ5a2758S0 DhVIbpg1H+oi1H1psLsk00arjRF1+SOMV/SiD6fAmtIMhUNUvMhUqutOU7Y8I63gZpwI LCifjxJL6F2Sm3Hon9GT+hNaLfx8fc3XnMMbASDvZ7HoCG9uU4K2UQaYr/HFRwjG4ddg LfhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XACfAo3TeGNF0rp/SYZW+awF5o1J3CDFB0MciljeeOQ=; b=ZaYKcS6ZGIxTOhRcNc8PbLRqbZO6NQ5ZfLwRjJlJ9/zN998Pp5bV+yFutJ3lfjasVv Ws2S7akucn6jfZJn+EYiX2wJLmIE7gYLrKL3/4yWQxK5Du2QI9e1GFaklLO6XqEAD0fI RmnzM8Jqi6UWuMX8h9Er3B3WACnc9z91/fGmSE3V468kC4mjUcfK3pV7/cTRzKMfuQvp t9x1K5V/aAg2v5t6T5siWMrYnhO2yZDT1hAdopYmtdAKBhqCxfznFcnOD5mjfk7JiFG7 ZE234/eFML7mjFtxi9Yl5s1sb/p9aNCTn5O7Z3cUWDdgIGvlYDbsikpyqxtNuX7J1lWN UgvQ== X-Gm-Message-State: AOAM532QlMpPxSwa9aLEk2MJrZYVGp7zB9NDnBx73FZVFBf+BwHlE/1d sReRG05dh9vX8Dt57y0L8f2U23KEcpgARbuO X-Google-Smtp-Source: ABdhPJwM9Ne/sexF84I3DfjmyrvDEfd8qIm/sObvjMw+VMtx5t/Om4ri/zLR6NjNy4HF4YUS1VcIrw== X-Received: by 2002:a7b:c10b:: with SMTP id w11mr19565668wmi.186.1623406027675; Fri, 11 Jun 2021 03:07:07 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id b135sm12568155wmb.5.2021.06.11.03.07.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Jun 2021 03:07:07 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Fri, 11 Jun 2021 11:07:02 +0100 Message-Id: <20210611100703.2285-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210611100703.2285-1-david.plowman@raspberrypi.com> References: <20210611100703.2285-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/2] ipa: raspberrypi: Add sensitivity field to camera mode X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We use the CamHelper class to initialise it to the usual value of 1. The CamHelper's GetModeSensitivity method can be redefined to implement a different behaviour for sensors that require it. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Acked-by: Kieran Bingham --- src/ipa/raspberrypi/cam_helper.cpp | 11 +++++++++++ src/ipa/raspberrypi/cam_helper.hpp | 3 +++ src/ipa/raspberrypi/controller/camera_mode.h | 2 ++ src/ipa/raspberrypi/raspberrypi.cpp | 6 ++++++ 4 files changed, 22 insertions(+) diff --git a/src/ipa/raspberrypi/cam_helper.cpp b/src/ipa/raspberrypi/cam_helper.cpp index 062e94c4..d0104da7 100644 --- a/src/ipa/raspberrypi/cam_helper.cpp +++ b/src/ipa/raspberrypi/cam_helper.cpp @@ -130,6 +130,17 @@ bool CamHelper::SensorEmbeddedDataPresent() const return false; } +double CamHelper::GetModeSensitivity([[maybe_unused]] const CameraMode &mode) const +{ + /* + * Most sensors have the same sensitivity in every mode, but this + * method can be overridden for those that do not. Note that it is + * called before mode_ is set, so it must return the sensitivity + * of the mode that is passed in. + */ + return 1.0; +} + unsigned int CamHelper::HideFramesStartup() const { /* diff --git a/src/ipa/raspberrypi/cam_helper.hpp b/src/ipa/raspberrypi/cam_helper.hpp index f53f5c39..f524fe84 100644 --- a/src/ipa/raspberrypi/cam_helper.hpp +++ b/src/ipa/raspberrypi/cam_helper.hpp @@ -40,6 +40,8 @@ namespace RPiController { // // A method to query if the sensor outputs embedded data that can be parsed. // +// A method to return the sensitivity of a given camera mode. +// // A parser to parse the embedded data buffers provided by some sensors (for // example, the imx219 does; the ov5647 doesn't). This allows us to know for // sure the exposure and gain of the frame we're looking at. CamHelper @@ -83,6 +85,7 @@ public: virtual void GetDelays(int &exposure_delay, int &gain_delay, int &vblank_delay) const; virtual bool SensorEmbeddedDataPresent() const; + virtual double GetModeSensitivity(const CameraMode &mode) const; virtual unsigned int HideFramesStartup() const; virtual unsigned int HideFramesModeSwitch() const; virtual unsigned int MistrustFramesStartup() const; diff --git a/src/ipa/raspberrypi/controller/camera_mode.h b/src/ipa/raspberrypi/controller/camera_mode.h index 2aa2335d..72c68dc6 100644 --- a/src/ipa/raspberrypi/controller/camera_mode.h +++ b/src/ipa/raspberrypi/controller/camera_mode.h @@ -41,6 +41,8 @@ struct CameraMode { libcamera::Transform transform; // minimum and maximum fame lengths in units of lines uint32_t min_frame_length, max_frame_length; + // sensitivity of this mode + double sensitivity; }; #ifdef __cplusplus diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 1c1e802a..9ce0db08 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -324,6 +324,12 @@ void IPARPi::setMode(const IPACameraSensorInfo &sensorInfo) */ mode_.min_frame_length = sensorInfo.minFrameLength; mode_.max_frame_length = sensorInfo.maxFrameLength; + + /* + * Some sensors may have different sensitivities in different modes; + * the CamHelper will know the correct value. + */ + mode_.sensitivity = helper_->GetModeSensitivity(mode_); } int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, From patchwork Fri Jun 11 10:07:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 12570 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 31D48C320B for ; Fri, 11 Jun 2021 10:07:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E0F0568932; Fri, 11 Jun 2021 12:07:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="gg5ToUIu"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 24C1368932 for ; Fri, 11 Jun 2021 12:07:09 +0200 (CEST) Received: by mail-wm1-x32c.google.com with SMTP id t4-20020a1c77040000b029019d22d84ebdso8303364wmi.3 for ; Fri, 11 Jun 2021 03:07:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4dd9tk74djwk5oBxLFFfp2oyTcPgl7Oc6QfpzeJ4gA8=; b=gg5ToUIum1IZsfDBfCW9IN+Dyft7TQf6PSR0KcGK66ItbjdqsUypGumQ45cCNfO/gt uQghA3ZktiTuwrb9T6BEsvJAyuWBRAUqWrojWMdHQNS2MVt30u/kgdSrk1Q2LAPqsRR4 HWKeRIA7LEcdMdTTrry2KBA9hrz9oa31POMbxVaj5pAoD644tkrJgWa3nXxKbsj4XbpH FoL1ocMZj7MofoEXntpCbvYka1+vlM9HMErHzlVnfr45AnqeU3RwF0PvGHnubDrPv62Z DY5hiRDs2um1XRNv9bKuv9LRv9+5sRQjOjz22x+gB6L/KPOmjqwLoco0EL+MSP8/eo5B 41mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4dd9tk74djwk5oBxLFFfp2oyTcPgl7Oc6QfpzeJ4gA8=; b=HFPj/WRVj2tJGnP+o9S1bRogmCEIXbWPkhQucFr18PFibYfS4R9GTt+0KU5WCVQPjp LWidaCeiwJtWT5Q8JX11b8VWah+lfldN0omooCGYLFVsHdPQEV9vyml2LFlP1EzvpbBE NniH+g2RGQqqw8yhNIfHx1bnPfyVzz4DjFxEZamjfJp+IpQj2jNlEXJCjGbFyJ8UVIou 5SV57jzBXZFotvyJvfnXdcDycr3YnAY9uPYATQl75MHxW/Bnqp57BLqrSeWJSCto3hUA x9Oy7f3twSouufhDjXWW2GLhRwoH8YKaOM2xNSbE8w3MNQZyiZUfM43ROBAbO5wQMeJD 847w== X-Gm-Message-State: AOAM533q6maJ6eXmnTYfCUBqqvYlBeSMMOSywsjLhJxwzuM6R0t77lcH KfTlrnTUgrkhL0e9wDyUF/Y8MTzO5UmtAUb/ X-Google-Smtp-Source: ABdhPJxGn2rkGO/hWrGI4mdd1dzzq2tY9pIE3Nq6CxRz2KW7frXHQlwB1IXtnGwJvaN+yvb3Ky/okw== X-Received: by 2002:a05:600c:410a:: with SMTP id j10mr18966453wmi.6.1623406028489; Fri, 11 Jun 2021 03:07:08 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id b135sm12568155wmb.5.2021.06.11.03.07.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Jun 2021 03:07:07 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Fri, 11 Jun 2021 11:07:03 +0100 Message-Id: <20210611100703.2285-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210611100703.2285-1-david.plowman@raspberrypi.com> References: <20210611100703.2285-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/2] ipa: raspberrypi: AGC: handle modes with different sensitivities X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the sensor is switched to a mode with a different sensitivity, the target exposure values need to be adjusted proportionately to maintain the same image brightness. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 25 +++++++++++++++++----- src/ipa/raspberrypi/controller/rpi/agc.hpp | 1 + 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 55e80ac7..431029d0 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -163,7 +163,7 @@ Agc::Agc(Controller *controller) : AgcAlgorithm(controller), metering_mode_(nullptr), exposure_mode_(nullptr), constraint_mode_(nullptr), frame_count_(0), lock_count_(0), - last_target_exposure_(0s), + last_target_exposure_(0s), last_sensitivity_(0.0), ev_(1.0), flicker_period_(0s), max_shutter_(0s), fixed_shutter_(0s), fixed_analogue_gain_(0.0) { @@ -269,7 +269,7 @@ void Agc::SetConstraintMode(std::string const &constraint_mode_name) constraint_mode_name_ = constraint_mode_name; } -void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, +void Agc::SwitchMode(CameraMode const &camera_mode, Metadata *metadata) { housekeepConfig(); @@ -293,9 +293,20 @@ void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, filtered_.shutter = fixed_shutter; filtered_.analogue_gain = fixed_analogue_gain_; } else if (status_.total_exposure_value) { - // On a mode switch, it's possible the exposure profile could change, - // or a fixed exposure/gain might be set so we divide up the exposure/ - // gain again, but we don't change any target values. + // On a mode switch, various things could happen: + // - the exposure profile might change + // - a fixed exposure or gain might be set + // - the new mode's sensitivity might be different + // We cope with the last of these by scaling the target values. After + // that we just need to re-divide the exposure/gain according to the + // current exposure profile, which takes care of everything else. + + double ratio = last_sensitivity_ / camera_mode.sensitivity; + target_.total_exposure_no_dg *= ratio; + target_.total_exposure *= ratio; + filtered_.total_exposure_no_dg *= ratio; + filtered_.total_exposure *= ratio; + divideUpExposure(); } else { // We come through here on startup, when at least one of the shutter @@ -309,6 +320,10 @@ void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, } writeAndFinish(metadata, false); + + // We must remember the sensitivity of this mode for the next SwitchMode. + ASSERT(camera_mode.sensitivity != 0.0); + last_sensitivity_ = camera_mode.sensitivity; } void Agc::Prepare(Metadata *image_metadata) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index 75078948..0d08dcd7 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -126,6 +126,7 @@ private: int lock_count_; DeviceStatus last_device_status_; libcamera::utils::Duration last_target_exposure_; + double last_sensitivity_; // sensitivity of the previous camera mode // Below here the "settings" that applications can change. std::string metering_mode_name_; std::string exposure_mode_name_;