From patchwork Thu May 27 08:45:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 12443 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7B660BDB80 for ; Thu, 27 May 2021 08:45:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1045168924; Thu, 27 May 2021 10:45:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="MKTnTcuE"; dkim-atps=neutral Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E34F068920 for ; Thu, 27 May 2021 10:45:46 +0200 (CEST) Received: by mail-wr1-x434.google.com with SMTP id x7so3825434wrt.12 for ; Thu, 27 May 2021 01:45:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=V5Re0dpffDCKWoUA3ShNyWGXAsBd9y/diewHASCjpvA=; b=MKTnTcuE4epic7dPtnFTMjIzR6K4hI74Ayvrl+8FrRW7Kgkm1xlh5vR3Qdi9k2dOTG rD9/DyhwKhGM6EYXxeBwcejLtT0jiiwXXiruHKsrkeJAzeTkcAdPqfKhE+FlMfd/IoPV bkHp15fgeqp0v7z9tnHgl6Cs6Rd73xY3YQpUYu+lRWhBNIGsXoftH4lvz1EUz7pBPcfI OBt32r+7iQC51dBFQzZah6+qVhYWW0nXMRyROCm6uGerWAIr2rkq76pyjoWL6odVk/zv iKHQWN9znW/hWjaKXo2wqsLQ4M4Qu6R8y5lELex2oXyv0DhxnHXk/TH3mmtCkHdC5NXM nAwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=V5Re0dpffDCKWoUA3ShNyWGXAsBd9y/diewHASCjpvA=; b=eZ86DgfguOu32RbXFqvVH9zID2HKBGigDmlx6TKMjhvtUQ+fnSyQZ7qRTeoNQSC0Vu SnXDQNfYQVAkhaJNE9B89DeNeZuP2j3N3r8FhnUD3z6EbtiS9p6+wzVGmlMGJsHSnMU2 MXDtDVegShDhiOblQShE+YP4zoi9M5juK8+/ZChYrQiuILxQkAjSLoGHsdsronaG/EGM fcD0CyKCWLbcSEyiyCmSoQ7BBPf/qgYJQo/1UM2vBbzS1w5c6hXnBLfqKp7qDn9l26r6 6RF6MgaZ6N8vzCBazUK7HXaxvtQI+ItBnmvgZJkXEez4oim0P0yOCZYDbk/aH18Rc/eP pWKg== X-Gm-Message-State: AOAM531q/0iXSpUVPhLoSg/Oj7pejbNy3tE5TQgdWyNhhKAxWwCsm8Ok X2QjspD9mvIq3h0Tzlc7/iTkzPO4/7A3Ug== X-Google-Smtp-Source: ABdhPJylCcXoiW50lB7a/jt5gDQg2RKUFzj5KisbT2eJeuE9ii6Qt5Oi5GseXozLGbeFuULtJHH9Vg== X-Received: by 2002:adf:e291:: with SMTP id v17mr2137648wri.149.1622105146393; Thu, 27 May 2021 01:45:46 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id q11sm2325553wrx.80.2021.05.27.01.45.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 01:45:46 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Thu, 27 May 2021 09:45:39 +0100 Message-Id: <20210527084540.9983-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210527084540.9983-1-david.plowman@raspberrypi.com> References: <20210527084540.9983-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/2] ipa: raspberrypi: Add sensitivity field to camera mode X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Initialise it to the usual value of 1, until such time as we have a mechanism that allows us to discover the correct value. The CamHelper class also gets a method to return this sensitivity value. This method is virtual so that it can be overridden for specific sensors. Once the correct value is obtainable elsewhere, this can be removed. Signed-off-by: David Plowman --- src/ipa/raspberrypi/cam_helper.cpp | 5 +++++ src/ipa/raspberrypi/cam_helper.hpp | 3 +++ src/ipa/raspberrypi/controller/camera_mode.h | 2 ++ src/ipa/raspberrypi/raspberrypi.cpp | 12 ++++++++++++ 4 files changed, 22 insertions(+) diff --git a/src/ipa/raspberrypi/cam_helper.cpp b/src/ipa/raspberrypi/cam_helper.cpp index 09917f3c..71ea21ff 100644 --- a/src/ipa/raspberrypi/cam_helper.cpp +++ b/src/ipa/raspberrypi/cam_helper.cpp @@ -128,6 +128,11 @@ bool CamHelper::SensorEmbeddedDataPresent() const return false; } +double CamHelper::Sensitivity() const +{ + return mode_.sensitivity; +} + unsigned int CamHelper::HideFramesStartup() const { /* diff --git a/src/ipa/raspberrypi/cam_helper.hpp b/src/ipa/raspberrypi/cam_helper.hpp index a52f3f0b..14c4714c 100644 --- a/src/ipa/raspberrypi/cam_helper.hpp +++ b/src/ipa/raspberrypi/cam_helper.hpp @@ -39,6 +39,8 @@ namespace RPiController { // // A method to query if the sensor outputs embedded data that can be parsed. // +// A method to return the sensitivity of the current camera mode. +// // A parser to parse the embedded data buffers provided by some sensors (for // example, the imx219 does; the ov5647 doesn't). This allows us to know for // sure the exposure and gain of the frame we're looking at. CamHelper @@ -81,6 +83,7 @@ public: virtual void GetDelays(int &exposure_delay, int &gain_delay, int &vblank_delay) const; virtual bool SensorEmbeddedDataPresent() const; + virtual double Sensitivity() const; virtual unsigned int HideFramesStartup() const; virtual unsigned int HideFramesModeSwitch() const; virtual unsigned int MistrustFramesStartup() const; diff --git a/src/ipa/raspberrypi/controller/camera_mode.h b/src/ipa/raspberrypi/controller/camera_mode.h index 256438c9..326f4f20 100644 --- a/src/ipa/raspberrypi/controller/camera_mode.h +++ b/src/ipa/raspberrypi/controller/camera_mode.h @@ -39,6 +39,8 @@ struct CameraMode { libcamera::Transform transform; // minimum and maximum fame lengths in units of lines uint32_t min_frame_length, max_frame_length; + // sensitivity of this mode + double sensitivity; }; #ifdef __cplusplus diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index e5bb8159..da42d279 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -321,6 +321,12 @@ void IPARPi::setMode(const IPACameraSensorInfo &sensorInfo) */ mode_.min_frame_length = sensorInfo.minFrameLength; mode_.max_frame_length = sensorInfo.maxFrameLength; + + /* + * For now, initialise the mode sensitivity to the usual value of 1. + * \todo Obtain the correct sensitivity number automatically. + */ + mode_.sensitivity = 1.0; } int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, @@ -379,6 +385,12 @@ int IPARPi::configure(const IPACameraSensorInfo &sensorInfo, /* Pass the camera mode to the CamHelper to setup algorithms. */ helper_->SetCameraMode(mode_); + /* + * For now, the CamHelper knows the sensitivity correctly. + * \todo This can be removed once the sensitivity is initialised properly. + */ + mode_.sensitivity = helper_->Sensitivity(); + if (firstStart_) { /* Supply initial values for frame durations. */ applyFrameDurations(defaultMinFrameDuration, defaultMaxFrameDuration); From patchwork Thu May 27 08:45:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 12444 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7ED26BDB80 for ; Thu, 27 May 2021 08:45:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4C9A168925; Thu, 27 May 2021 10:45:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qapg+gY6"; dkim-atps=neutral Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9C403602AA for ; Thu, 27 May 2021 10:45:47 +0200 (CEST) Received: by mail-wm1-x334.google.com with SMTP id f6-20020a1c1f060000b0290175ca89f698so2072969wmf.5 for ; Thu, 27 May 2021 01:45:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NzHu98sOk4xja6FwzR2w3UuPTGhHwGZQofZOG9a8B6c=; b=qapg+gY6snTHYEDOSQWKHNWFlssL/SqMcNimPEcaAJaJl7Fq1ONEhh2jmvIye3GYo2 f0BoEwEpxiXRj06Fx8splWXSUgl6Rd30YmpyAylNKUgKtgeWXFrBPElbwgSj53R6sqcC 5L7ij7IG7MPUSav45ySpy3wnPtZ0GQY2MbmsC6EYF8qNz+CLxXxBr+UWTl7yGSa4Knin ZM6pclqeLAAoR/yG6r/HoaKWdAqE4IYxTkUMyGiI/swBATZVrndzz7LIEcZBHjd+GLoi tziqLC35lG7UUAeveA8NOx7i05LslzvanqjV5+tcGnSSyVI0tGJvvjc0t4v/a04vmbD4 GgUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NzHu98sOk4xja6FwzR2w3UuPTGhHwGZQofZOG9a8B6c=; b=hLncUranLXXUjM4VCDFRrfas2m2SHTYvI7hdExTjkG96Maq3iW9mOuIgpPBfgsJwr+ 3U4PWiBvUa1Dtesu2hvVKKTO5ZMzyuzZSJLmBdn9XsNgvo+XIYY6j+h5oY9Az78ONvS2 7Qnkv6TiQHfWG6MKv3yzVlv+/sPwFARhZ3qcNdvqec4vT6SebsmeYOFWaCnkFBvvdMVI p28heyQW270FfrTSwo1IClAClqbcM7VhGs+RL9j5uUJAB5H+qEb5919d3Fwyb9OKOaV+ QvrDBMTbGZMSmKhFNUjxNrMagyXuK0F4goBuN5ochYQW62rKq18mMkhGIeLu+ywUfMvD z3fw== X-Gm-Message-State: AOAM5318utH3bNgGIfPFqobZA7w+/elzt9AX4UpOBt0O3w1KYbBKUVNS tHfoTgEEwCUwTTZQg9pdI84BqMR7Izgq4A== X-Google-Smtp-Source: ABdhPJxkmpHiZTin2VUZ0Ld+cf1FB735XmmyD+O4aLBtlPuQE5rL2iw0/WRLlUa56Xwd+O7QtbenmA== X-Received: by 2002:a7b:cd98:: with SMTP id y24mr7326698wmj.4.1622105147141; Thu, 27 May 2021 01:45:47 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id q11sm2325553wrx.80.2021.05.27.01.45.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 01:45:46 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Thu, 27 May 2021 09:45:40 +0100 Message-Id: <20210527084540.9983-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210527084540.9983-1-david.plowman@raspberrypi.com> References: <20210527084540.9983-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/2] ipa: raspberrypi: AGC: handle modes with different sensitivities X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the sensor is switched to a mode with a different sensitivity, the target exposure values need to be adjusted proportionately to maintain the same image brightness. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 25 +++++++++++++++++----- src/ipa/raspberrypi/controller/rpi/agc.hpp | 1 + 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index f4cd5d26..6f0e9702 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -155,7 +155,7 @@ Agc::Agc(Controller *controller) : AgcAlgorithm(controller), metering_mode_(nullptr), exposure_mode_(nullptr), constraint_mode_(nullptr), frame_count_(0), lock_count_(0), - last_target_exposure_(0.0), + last_target_exposure_(0.0), last_sensitivity_(0.0), ev_(1.0), flicker_period_(0.0), max_shutter_(0), fixed_shutter_(0), fixed_analogue_gain_(0.0) { @@ -261,7 +261,7 @@ void Agc::SetConstraintMode(std::string const &constraint_mode_name) constraint_mode_name_ = constraint_mode_name; } -void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, +void Agc::SwitchMode(CameraMode const &camera_mode, Metadata *metadata) { housekeepConfig(); @@ -285,9 +285,20 @@ void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, filtered_.shutter = fixed_shutter; filtered_.analogue_gain = fixed_analogue_gain_; } else if (status_.total_exposure_value) { - // On a mode switch, it's possible the exposure profile could change, - // or a fixed exposure/gain might be set so we divide up the exposure/ - // gain again, but we don't change any target values. + // On a mode switch, various things could happen: + // - the exposure profile might change + // - a fixed exposure or gain might be set + // - the new mode's sensitivity might be different + // We cope with the last of these by scaling the target values. After + // that we just need to re-divide the exposure/gain according to the + // current exposure profile, which takes care of everything else. + + double ratio = last_sensitivity_ / camera_mode.sensitivity; + target_.total_exposure_no_dg *= ratio; + target_.total_exposure *= ratio; + filtered_.total_exposure_no_dg *= ratio; + filtered_.total_exposure *= ratio; + divideUpExposure(); } else { // We come through here on startup, when at least one of the shutter @@ -301,6 +312,10 @@ void Agc::SwitchMode([[maybe_unused]] CameraMode const &camera_mode, } writeAndFinish(metadata, false); + + // We must remember the sensitivity of this mode for the next SwitchMode. + ASSERT(camera_mode.sensitivity != 0.0); + last_sensitivity_ = camera_mode.sensitivity; } void Agc::Prepare(Metadata *image_metadata) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index 0427fb59..025878a2 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -122,6 +122,7 @@ private: int lock_count_; DeviceStatus last_device_status_; double last_target_exposure_; + double last_sensitivity_; // sensitivity of the previous camera mode // Below here the "settings" that applications can change. std::string metering_mode_name_; std::string exposure_mode_name_;