From patchwork Fri Dec 18 10:06:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 10678 X-Patchwork-Delegate: laurent.pinchart@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BD971C0F1A for ; Fri, 18 Dec 2020 10:07:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 766AE61593; Fri, 18 Dec 2020 11:07:02 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="T851rUO2"; dkim-atps=neutral Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 014C361593 for ; Fri, 18 Dec 2020 11:07:00 +0100 (CET) Received: by mail-wm1-x334.google.com with SMTP id k10so1762962wmi.3 for ; Fri, 18 Dec 2020 02:07:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=toNVQHPqikeKHtw7h2O9IrIFWTVBv1gYRnO0mkIhmkI=; b=T851rUO2KEyKQjqzHFtOjycWN2hK6AsAwcIx6JRUomxEDneQ0j/9Rvvj/s+xyHRHBt AAJiWfwJ0Yw1RxC51dtvGY+tXz96DMlz5FuChrwW5WStswiu/MTL4GUlu0fX0FBWBECu e4rGgbWMuXlERP5Ty2CmOQqzcM1hnGgMNZG6GAGHnTkRIugs6rPo3LMr0aik4VpN+zhQ tdkkFKA9qjeZVj6HDwMfSxa5My8D0ad4AfjTDkk1gE8jCp+/6JjmpnBLhjgSCXO8p9s+ 8Oi770goABjH3+CQXHuUBX2lKqhEPYr9KDx5EKFIPDPk4h3+B55QlNj1nIum9sR9i8yC hfIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=toNVQHPqikeKHtw7h2O9IrIFWTVBv1gYRnO0mkIhmkI=; b=suAH5i0YEp6tapWA68vgMqFdUyd3aGE0hUwwSwlTRs6S/OUU11d1iqW+ZwA6lIL0u1 i4hMFlybADuOflI4eg1y4o4N5omUUW79fPcitTgrrGJYK8/9q1Meet/XFriGmAPUDeLm QuIcqmNFi2GKnXt05uaWo+0O2FdOWRlm3m9K3dVdwNlSVsUB9jrKmmnf6T/KrEga+TUq dtKMSXaE6GkGFZnTUOfoNQGDvrFVCg6Ws75UCJcaeOhsos+V8uPRuDj10SkzKwEip9Gn seMvqEv18aQ8tscFUztOhawV/o4m+gkZ5/mOIdJKwne3JPZHGnQ3vQqNZHgsWzeNzWSO 3L0g== X-Gm-Message-State: AOAM533Jjf7hkY9IKpAhE8RAvbnkY7SJmmkLHoO7+QETXYlSeQSoG1EF PdkVmaQQKJKmP3AXfB6BZjZtqbVCa3SoPg== X-Google-Smtp-Source: ABdhPJxXt10EUc//r3RT/SsVg4FT5ogpacwT/BlK79WJDifkZ+xalms015oviK6fcqZPdWR0dI/QCQ== X-Received: by 2002:a05:600c:208:: with SMTP id 8mr3523582wmi.146.1608286019428; Fri, 18 Dec 2020 02:06:59 -0800 (PST) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id c4sm11836424wmf.19.2020.12.18.02.06.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Dec 2020 02:06:58 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Dec 2020 10:06:25 +0000 Message-Id: <20201218100626.205134-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201218100626.205134-1-naush@raspberrypi.com> References: <20201218100626.205134-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v10 1/3] libcamera: controls: Add frame duration control X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add an int64_t array control (controls::FrameDurations) to specify the minimum and maximum (in that order) frame duration to be used by the camera sensor. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- src/libcamera/control_ids.yaml | 41 ++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml index 6d6f0fee..a58bff18 100644 --- a/src/libcamera/control_ids.yaml +++ b/src/libcamera/control_ids.yaml @@ -306,6 +306,47 @@ controls: maximum valid value is given by the properties::ScalerCropMaximum property, and the two can be used to implement digital zoom. + - FrameDurations: + type: int64_t + description: | + The minimum and maximum (in that order) frame duration, + expressed in micro-seconds. + + When provided by applications, the control specifies the sensor frame + duration interval the pipeline has to use. This could also limit the + largest exposure time the sensor can use. For example, if a maximum + frame duration of 33ms is requested (corresponding to 30 frames per + second), the sensor will not be able to raise the exposure time above + 33ms. A fixed frame duration is achieved by setting the minimum and + maximum values to be the same. + + The maximum frame duration provides the absolute limit to the shutter + speed computed by the AE algorithm and it overrides any exposure mode + setting specified with controls::AeExposureMode. Similarly, when a + manual exposure time is set through controls::ExposureTime, it also + gets clipped to the limits set by this control. + + \sa AeExposureMode + \sa ExposureTime + + \todo Refer to the frame duration limits property to describe how + application-provided values gets clipped and reset. + + When reported by pipelines, the control expresses the minimum and + maximum frame durations used after being clipped to what the current + sensor mode supports, and what is achievable based on the exposure + mode setting specified with controls::AeExposureMode or manual + exposure time set through controls::ExposureTime. The sensor frame + duration is one of the parameter that defines the capture frame rate + but it does not alone provide enough information to fully calculate it + as it does not account for pipeline processing delays. + + \todo Define how to calculate the capture frame rate by + defining controls to report additional delays introduced by + the capture pipeline or post-processing stages (ie JPEG + conversion, frame scaling). + size: [2] + # ---------------------------------------------------------------------------- # Draft controls section From patchwork Fri Dec 18 10:06:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 10679 X-Patchwork-Delegate: laurent.pinchart@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F382CC0F1A for ; Fri, 18 Dec 2020 10:07:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D6A1F61595; Fri, 18 Dec 2020 11:07:02 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="pek73Po2"; dkim-atps=neutral Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 70A1061593 for ; Fri, 18 Dec 2020 11:07:01 +0100 (CET) Received: by mail-wm1-x335.google.com with SMTP id q75so1955639wme.2 for ; Fri, 18 Dec 2020 02:07:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XjMr54AfPGB1mbaXknZT2iwhN4fPzpsjc89BlML7OHo=; b=pek73Po2IP7KOr1hhO+o04WxGL9yy7ZcFkFkAOFpW5sDaf3B3KSIOh7+q4+bUazeFX Xqtd6Tyg0r0QNa7VhVzYjjqbepB3cHz95GnZY9+CcXYZtaGW9a5S+ZSM+nEB2QBNIn9w y4VUmc2dMKXl5PTySbfk3bC6fIq7FhxXeRifp/Ehsi7HdYbqkJTUlBxEtNwjRDnxDDT2 323/NNJ/gxRZCs6mt7iyZCjw+7IT/Ny6yRifNzdNPeu5JCeTuk86PE7pgLCqgjdVyi5B BNkKfOnEKWFyLCGHyWWtt8K809e6wdP23Xh3EH8moU75RwKWDQKqjWA/7boyzA35AaOm npbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XjMr54AfPGB1mbaXknZT2iwhN4fPzpsjc89BlML7OHo=; b=Jlzv4mErc/kuk5W5wVdBrAqV1NRpDFQjJ7xy9E9sxvURSuzIkQeo2ckHk978c4FgaK wNrkBhL8Wn4DnIa/HageC6qWWQLU/FAvrVmA9aAEVCeZelYWJrngsaEAJ4QVhCyAXjRM s5hx60L9stcqbVsCcKmdmdrpKMlNtCVrTjli3uUMuZvkC0XSstTXJSfIZYyevZlQQFlK K6WYrnhI72Jgf0yfntgEOwy1T+GEpJWEnnDsPMHdGYNd0jJ51FrfpztxDsoInSADyIJm HcGDR5v19dLj2kYdVy7f5O+NUUeweLOzZ/B/7uUW/dLxFGIDwxTpOSshqOKel+YD5ocb bggg== X-Gm-Message-State: AOAM5332KkyGxKoSv7qnYRGraOLE3t48BufbFvP2twseV/FqsyQhM2+E CCu0lFdJOn8YmDLI8RiVeqY2Cq7EykDTmA== X-Google-Smtp-Source: ABdhPJwUpMh+Laz28d9JiZS/lIyO8FlJGgDCiypAb/6u9gw5R9PvS/HRtqaHL2m3ofhDhzBSMuc3cA== X-Received: by 2002:a1c:6a0e:: with SMTP id f14mr3391949wmc.102.1608286020618; Fri, 18 Dec 2020 02:07:00 -0800 (PST) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id c4sm11836424wmf.19.2020.12.18.02.06.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Dec 2020 02:06:59 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Dec 2020 10:06:26 +0000 Message-Id: <20201218100626.205134-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201218100626.205134-1-naush@raspberrypi.com> References: <20201218100626.205134-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v10 2/3] libcamera: raspberrypi: Add control of sensor vblanking X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for setting V4L2_CID_VBLANK appropriately when setting V4L2_CID_EXPOSURE. This will allow adaptive framerates during viewfinder use cases (e.g. when the exposure time goes above 33ms, we can reduce the framerate to lower than 30fps). The minimum and maximum frame durations are provided via libcamera controls, and will prioritise exposure time limits over any AGC request. V4L2_CID_VBLANK is controlled through the staggered writer, just like the exposure and gain controls. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- include/libcamera/ipa/raspberrypi.h | 1 + src/ipa/raspberrypi/cam_helper.cpp | 35 ++++++++++++- src/ipa/raspberrypi/cam_helper.hpp | 15 +++++- src/ipa/raspberrypi/cam_helper_imx219.cpp | 13 ++++- src/ipa/raspberrypi/cam_helper_imx477.cpp | 11 +++- src/ipa/raspberrypi/cam_helper_ov5647.cpp | 11 +++- src/ipa/raspberrypi/raspberrypi.cpp | 52 ++++++++++++++++--- .../pipeline/raspberrypi/raspberrypi.cpp | 3 +- 8 files changed, 127 insertions(+), 14 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index 01fe5abc..1de36039 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -65,6 +65,7 @@ static const ControlInfoMap Controls = { { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, + { &controls::FrameDurations, ControlInfo(1000, 1000000000) }, }; } /* namespace RPi */ diff --git a/src/ipa/raspberrypi/cam_helper.cpp b/src/ipa/raspberrypi/cam_helper.cpp index 6efa0d7f..018c17b9 100644 --- a/src/ipa/raspberrypi/cam_helper.cpp +++ b/src/ipa/raspberrypi/cam_helper.cpp @@ -34,8 +34,10 @@ CamHelper *CamHelper::Create(std::string const &cam_name) return nullptr; } -CamHelper::CamHelper(MdParser *parser) - : parser_(parser), initialized_(false) +CamHelper::CamHelper(MdParser *parser, unsigned int maxFrameLength, + unsigned int frameIntegrationDiff) + : parser_(parser), initialized_(false), maxFrameLength_(maxFrameLength), + frameIntegrationDiff_(frameIntegrationDiff) { } @@ -56,6 +58,35 @@ double CamHelper::Exposure(uint32_t exposure_lines) const return exposure_lines * mode_.line_length / 1000.0; } +uint32_t CamHelper::GetVBlanking(double &exposure, double minFrameDuration, + double maxFrameDuration) const +{ + uint32_t frameLengthMin, frameLengthMax, vblank; + uint32_t exposureLines = ExposureLines(exposure); + + assert(initialized_); + + /* + * Clip frame length by the frame duration range and the maximum allowable + * value in the sensor, given by maxFrameLength_. + */ + frameLengthMin = std::clamp(1e3 * minFrameDuration / mode_.line_length, + mode_.height, maxFrameLength_); + frameLengthMax = std::clamp(1e3 * maxFrameDuration / mode_.line_length, + mode_.height, maxFrameLength_); + /* + * Limit the exposure to the maximum frame duration requested, and + * re-calculate if it has been clipped. + */ + exposureLines = std::min(frameLengthMax - frameIntegrationDiff_, exposureLines); + exposure = Exposure(exposureLines); + + /* Limit the vblank to the range allowed by the frame length limits. */ + vblank = std::clamp(exposureLines + frameIntegrationDiff_, + frameLengthMin, frameLengthMax) - mode_.height; + return vblank; +} + void CamHelper::SetCameraMode(const CameraMode &mode) { mode_ = mode; diff --git a/src/ipa/raspberrypi/cam_helper.hpp b/src/ipa/raspberrypi/cam_helper.hpp index 044c2866..b1739a57 100644 --- a/src/ipa/raspberrypi/cam_helper.hpp +++ b/src/ipa/raspberrypi/cam_helper.hpp @@ -62,12 +62,15 @@ class CamHelper { public: static CamHelper *Create(std::string const &cam_name); - CamHelper(MdParser *parser); + CamHelper(MdParser *parser, unsigned int maxFrameLength, + unsigned int frameIntegrationDiff); virtual ~CamHelper(); void SetCameraMode(const CameraMode &mode); MdParser &Parser() const { return *parser_; } uint32_t ExposureLines(double exposure_us) const; double Exposure(uint32_t exposure_lines) const; // in us + virtual uint32_t GetVBlanking(double &exposure_us, double minFrameDuration, + double maxFrameDuration) const; virtual uint32_t GainCode(double gain) const = 0; virtual double Gain(uint32_t gain_code) const = 0; virtual void GetDelays(int &exposure_delay, int &gain_delay) const; @@ -76,10 +79,20 @@ public: virtual unsigned int HideFramesModeSwitch() const; virtual unsigned int MistrustFramesStartup() const; virtual unsigned int MistrustFramesModeSwitch() const; + protected: MdParser *parser_; CameraMode mode_; + +private: bool initialized_; + /* Largest possible frame length, in units of lines. */ + unsigned int maxFrameLength_; + /* + * Smallest difference between the frame length and integration time, + * in units of lines. + */ + unsigned int frameIntegrationDiff_; }; // This is for registering camera helpers with the system, so that the diff --git a/src/ipa/raspberrypi/cam_helper_imx219.cpp b/src/ipa/raspberrypi/cam_helper_imx219.cpp index db8ab879..8688ec09 100644 --- a/src/ipa/raspberrypi/cam_helper_imx219.cpp +++ b/src/ipa/raspberrypi/cam_helper_imx219.cpp @@ -49,13 +49,22 @@ public: double Gain(uint32_t gain_code) const override; unsigned int MistrustFramesModeSwitch() const override; bool SensorEmbeddedDataPresent() const override; + +private: + /* + * Smallest difference between the frame length and integration time, + * in units of lines. + */ + static constexpr int frameIntegrationDiff = 4; + /* Largest possible frame length, in units of lines. */ + static constexpr int maxFrameLength = 0xffff; }; CamHelperImx219::CamHelperImx219() #if ENABLE_EMBEDDED_DATA - : CamHelper(new MdParserImx219()) + : CamHelper(new MdParserImx219(), maxFrameLength, frameIntegrationDiff) #else - : CamHelper(new MdParserRPi()) + : CamHelper(new MdParserRPi(), maxFrameLength, frameIntegrationDiff) #endif { } diff --git a/src/ipa/raspberrypi/cam_helper_imx477.cpp b/src/ipa/raspberrypi/cam_helper_imx477.cpp index 0e896ac7..53961310 100644 --- a/src/ipa/raspberrypi/cam_helper_imx477.cpp +++ b/src/ipa/raspberrypi/cam_helper_imx477.cpp @@ -38,10 +38,19 @@ public: uint32_t GainCode(double gain) const override; double Gain(uint32_t gain_code) const override; bool SensorEmbeddedDataPresent() const override; + +private: + /* + * Smallest difference between the frame length and integration time, + * in units of lines. + */ + static constexpr int frameIntegrationDiff = 22; + /* Largest possible frame length, in units of lines. */ + static constexpr int maxFrameLength = 0xffdc; }; CamHelperImx477::CamHelperImx477() - : CamHelper(new MdParserImx477()) + : CamHelper(new MdParserImx477(), maxFrameLength, frameIntegrationDiff) { } diff --git a/src/ipa/raspberrypi/cam_helper_ov5647.cpp b/src/ipa/raspberrypi/cam_helper_ov5647.cpp index 0b841cd1..2bd8a754 100644 --- a/src/ipa/raspberrypi/cam_helper_ov5647.cpp +++ b/src/ipa/raspberrypi/cam_helper_ov5647.cpp @@ -23,6 +23,15 @@ public: unsigned int HideFramesModeSwitch() const override; unsigned int MistrustFramesStartup() const override; unsigned int MistrustFramesModeSwitch() const override; + +private: + /* + * Smallest difference between the frame length and integration time, + * in units of lines. + */ + static constexpr int frameIntegrationDiff = 4; + /* Largest possible frame length, in units of lines. */ + static constexpr int maxFrameLength = 0xffff; }; /* @@ -31,7 +40,7 @@ public: */ CamHelperOv5647::CamHelperOv5647() - : CamHelper(new MdParserRPi()) + : CamHelper(new MdParserRPi(), maxFrameLength, frameIntegrationDiff) { } diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index d087b07e..d309536b 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -58,6 +58,8 @@ namespace libcamera { /* Configure the sensor with these values initially. */ constexpr double DefaultAnalogueGain = 1.0; constexpr unsigned int DefaultExposureTime = 20000; +constexpr double defaultMinFrameDuration = 1e6 / 30.0; +constexpr double defaultMaxFrameDuration = 1e6 / 0.01; LOG_DEFINE_CATEGORY(IPARPI) @@ -150,6 +152,9 @@ private: /* Distinguish the first camera start from others. */ bool firstStart_; + + /* Frame duration (1/fps) given in microseconds. */ + double minFrameDuration_, maxFrameDuration_; }; int IPARPi::init(const IPASettings &settings) @@ -332,7 +337,8 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, sensorMetadata = helper_->SensorEmbeddedDataPresent(); result->data.push_back(gainDelay); - result->data.push_back(exposureDelay); + result->data.push_back(exposureDelay); /* For EXPOSURE ctrl */ + result->data.push_back(exposureDelay); /* For VBLANK ctrl */ result->data.push_back(sensorMetadata); result->operation |= RPi::IPA_CONFIG_STAGGERED_WRITE; @@ -377,6 +383,9 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, controller_.Initialise(); controllerInit_ = true; + minFrameDuration_ = defaultMinFrameDuration; + maxFrameDuration_ = defaultMaxFrameDuration; + /* Supply initial values for gain and exposure. */ ControlList ctrls(sensorCtrls_); AgcStatus agcStatus; @@ -524,7 +533,8 @@ bool IPARPi::validateSensorControls() { static const uint32_t ctrls[] = { V4L2_CID_ANALOGUE_GAIN, - V4L2_CID_EXPOSURE + V4L2_CID_EXPOSURE, + V4L2_CID_VBLANK }; for (auto c : ctrls) { @@ -804,6 +814,24 @@ void IPARPi::queueRequest(const ControlList &controls) break; } + case controls::FRAME_DURATIONS: { + auto frameDurations = ctrl.second.get>(); + + /* This will be applied once AGC recalculations occur. */ + minFrameDuration_ = frameDurations[0] ? frameDurations[0] : defaultMinFrameDuration; + maxFrameDuration_ = frameDurations[1] ? frameDurations[1] : defaultMaxFrameDuration; + + /* + * \todo The values returned in the metadata below must be + * correctly clipped by what the sensor mode supports and + * what the AGC exposure mode or manual shutter speed limits + */ + libcameraMetadata_.set(controls::FrameDurations, + { static_cast(minFrameDuration_), + static_cast(maxFrameDuration_) }); + break; + } + default: LOG(IPARPI, Warning) << "Ctrl " << controls::controls.at(ctrl.first)->name() @@ -962,15 +990,27 @@ void IPARPi::applyAWB(const struct AwbStatus *awbStatus, ControlList &ctrls) void IPARPi::applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls) { int32_t gainCode = helper_->GainCode(agcStatus->analogue_gain); - int32_t exposureLines = helper_->ExposureLines(agcStatus->shutter_time); - LOG(IPARPI, Debug) << "Applying AGC Exposure: " << agcStatus->shutter_time - << " (Shutter lines: " << exposureLines << ") Gain: " + /* GetVBlanking might clip exposure time to the fps limits. */ + double exposure = agcStatus->shutter_time; + int32_t vblanking = helper_->GetVBlanking(exposure, minFrameDuration_, + maxFrameDuration_); + int32_t exposureLines = helper_->ExposureLines(exposure); + + LOG(IPARPI, Debug) << "Applying AGC Exposure: " << exposure + << " (Shutter lines: " << exposureLines << ", AGC requested " + << agcStatus->shutter_time << ") Gain: " << agcStatus->analogue_gain << " (Gain Code: " << gainCode << ")"; - ctrls.set(V4L2_CID_ANALOGUE_GAIN, gainCode); + /* + * Due to the behavior of V4L2, the current value of VBLANK could clip the + * exposure time without us knowing. The next time though this function should + * clip exposure correctly. + */ + ctrls.set(V4L2_CID_VBLANK, vblanking); ctrls.set(V4L2_CID_EXPOSURE, exposureLines); + ctrls.set(V4L2_CID_ANALOGUE_GAIN, gainCode); } void IPARPi::applyDG(const struct AgcStatus *dgStatus, ControlList &ctrls) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 7a5f5881..252cab64 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1233,7 +1233,8 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) if (!staggeredCtrl_) { staggeredCtrl_.init(unicam_[Unicam::Image].dev(), { { V4L2_CID_ANALOGUE_GAIN, result.data[resultIdx++] }, - { V4L2_CID_EXPOSURE, result.data[resultIdx++] } }); + { V4L2_CID_EXPOSURE, result.data[resultIdx++] }, + { V4L2_CID_VBLANK, result.data[resultIdx++] } }); sensorMetadata_ = result.data[resultIdx++]; } } From patchwork Fri Dec 18 10:06:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 10680 X-Patchwork-Delegate: laurent.pinchart@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 82B4AC0F1A for ; Fri, 18 Dec 2020 10:07:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 510FD615D5; Fri, 18 Dec 2020 11:07:03 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="eY2Z1NAo"; dkim-atps=neutral Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D8F4161593 for ; Fri, 18 Dec 2020 11:07:01 +0100 (CET) Received: by mail-wm1-x330.google.com with SMTP id y23so1949783wmi.1 for ; Fri, 18 Dec 2020 02:07:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UivwIO3ktu5j2EoxINYFa7ri9ABorkJNi+C3XacBQ4A=; b=eY2Z1NAocx9yFOiyR1wjbkFL9isX6hx1qnPwWgVBtU9thpiSYpN+xl49ptESMF/k9z TpUhVXnx9Uc9KbACmswnNrtLFcVacmwikzojclNtajGO/sG6tgTFafqV36NIzp16XO/u SfGDiYbB0MFJEP827e5MXB8SWoOvqUwWIo/Wla/kPPO3r0gl5qnvK26nV3KhupMAPPJR E2g5G4lpEFnbtlz2JhPbRa+9Pz1qV5f9/TAdUwIv1GqJW/HcLg9/GBWwfzACh68CGcua 0fJjrvn9jlqzLAzAzaafhuqSDBM6Lb/Ampdx3Cq7osAXMJfjOFHfLXvFS9dw+1fWEY0+ P0nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UivwIO3ktu5j2EoxINYFa7ri9ABorkJNi+C3XacBQ4A=; b=RRjzpbdI82GODUrIkBsL8SI7ZrtARhOXbpDjGmHLMOv+6U0Q8+wOOcxd1uA7cagJlp fT9V46Px5iMYTzAjM5AgrO81/nY+JJLwETHLGcPsxr+Zb8hPNXh4uVD2YmDDhlNEk5M9 ySxIjGJ5LqawyNSZgX9sUvrXyiOiqAFbtwe94J2QIs4VcsjzZEekSQ7wEFb+iDP+RvA3 ZU6q4NLX4ez7FCed+6WolCvDf1TiKVZmypouH1+PWI3OVrh7xgyaCBbv1VgU33nCtszT uoTvbUMyloz+PZZhSdryCbbOClFKVh4gipLNWpwL2R1FJ7lJZcCehcfUB1VcrDdjLkM9 iF4g== X-Gm-Message-State: AOAM530ajPq65ynFhX2MxyD/uP+b6kKHgdv6TNrj/4MJ8JaLU37pBwFo bi7kj3aHsrtjAzKae8p6O8EPAWtiCLUTWw== X-Google-Smtp-Source: ABdhPJzzBHjAnXlx9ly6AZFAaV1R39O8gz15byyaEmaLP0b0LPfzzs9d+boaaiQJhx/66+9yUVkHvg== X-Received: by 2002:a7b:c10e:: with SMTP id w14mr3419405wmi.161.1608286021402; Fri, 18 Dec 2020 02:07:01 -0800 (PST) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id c4sm11836424wmf.19.2020.12.18.02.07.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Dec 2020 02:07:00 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Dec 2020 10:06:27 +0000 Message-Id: <20201218100626.205134-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201218100626.205134-1-naush@raspberrypi.com> References: <20201218100626.205134-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v10 3/3] ipa: raspberrypi: config: Update shutter speeds for imx219/477 and ov5647 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Set the maximum shutter speed for the normal exposure profile to 66ms. Set the maximum shutter speed for the sport exposure profile to 33ms. Add a long exposure profile to allow shutter speeds of up to 120ms. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/data/imx219.json | 15 +++++++++++++-- src/ipa/raspberrypi/data/imx477.json | 15 +++++++++++++-- src/ipa/raspberrypi/data/ov5647.json | 15 +++++++++++++-- 3 files changed, 39 insertions(+), 6 deletions(-) diff --git a/src/ipa/raspberrypi/data/imx219.json b/src/ipa/raspberrypi/data/imx219.json index 212f8b9a..1ec338be 100644 --- a/src/ipa/raspberrypi/data/imx219.json +++ b/src/ipa/raspberrypi/data/imx219.json @@ -133,7 +133,7 @@ { "shutter": [ - 100, 10000, 30000, 30000, 30000 + 100, 10000, 30000, 60000, 66666 ], "gain": [ @@ -144,7 +144,18 @@ { "shutter": [ - 100, 5000, 10000, 20000, 30000 + 100, 5000, 10000, 20000, 33333 + ], + "gain": + [ + 1.0, 2.0, 4.0, 6.0, 6.0 + ] + }, + "long": + { + "shutter": + [ + 100, 10000, 30000, 60000, 120000 ], "gain": [ diff --git a/src/ipa/raspberrypi/data/imx477.json b/src/ipa/raspberrypi/data/imx477.json index 12fe2f41..9bee3f16 100644 --- a/src/ipa/raspberrypi/data/imx477.json +++ b/src/ipa/raspberrypi/data/imx477.json @@ -133,7 +133,7 @@ { "shutter": [ - 100, 10000, 30000, 60000, 120000 + 100, 10000, 30000, 60000, 66666 ], "gain": [ @@ -144,7 +144,18 @@ { "shutter": [ - 100, 5000, 10000, 20000, 120000 + 100, 5000, 10000, 20000, 33333 + ], + "gain": + [ + 1.0, 2.0, 4.0, 6.0, 6.0 + ] + }, + "long": + { + "shutter": + [ + 100, 10000, 30000, 60000, 120000 ], "gain": [ diff --git a/src/ipa/raspberrypi/data/ov5647.json b/src/ipa/raspberrypi/data/ov5647.json index 3d8a7c8f..1a354f7c 100644 --- a/src/ipa/raspberrypi/data/ov5647.json +++ b/src/ipa/raspberrypi/data/ov5647.json @@ -133,7 +133,7 @@ { "shutter": [ - 100, 10000, 30000, 30000, 30000 + 100, 10000, 30000, 60000, 66666 ], "gain": [ @@ -144,7 +144,18 @@ { "shutter": [ - 100, 5000, 10000, 20000, 30000 + 100, 5000, 10000, 20000, 33333 + ], + "gain": + [ + 1.0, 2.0, 4.0, 6.0, 6.0 + ] + }, + "long": + { + "shutter": + [ + 100, 10000, 30000, 60000, 120000 ], "gain": [