From patchwork Tue Jan 19 15:30:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 10900 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 31759C0F1C for ; Tue, 19 Jan 2021 15:30:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A99456815B; Tue, 19 Jan 2021 16:30:53 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="VVpGZT6M"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 818DD6814E for ; Tue, 19 Jan 2021 16:30:52 +0100 (CET) Received: by mail-wm1-x336.google.com with SMTP id c127so116322wmf.5 for ; Tue, 19 Jan 2021 07:30:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=HDZCTbJRCBRdw1De4jc2Td6h4U0eYFlyY/7WXINUGog=; b=VVpGZT6M6iTk4R1HyCkHUKz2Tvnhcbzn/PpzGQLqOjnOl1PsRNNaDHmJK7JpBKqvy6 gHerrl14jT+nX2r3qxTN4XqrMNFocVADbtVy+oq73+7bOlE8/R0VwYUnDhD5AQuyNK3M yC1lyg0zK9HRmtxnPLHhJdSxcINs1tg0EpDhvtUYI8haUlXIz7+1JAsOE2lRkGZ0yJoK ZnMfJFeuId4+L9kM9KTb9SjYmCf8Lr4RMIi1nms6TM8A1gyNOOoUDRBPlxlKozSHW1VZ kkEOrBTKkUf6uU7osk0ad7VYxFtGOQAMbP9LFVyqASj6UtxXSqEIRawV4IhNUM95KV5G 9utg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=HDZCTbJRCBRdw1De4jc2Td6h4U0eYFlyY/7WXINUGog=; b=l/i354k8jDnrprY8Pfx77XMQRFK/oC6V5pYiguqLw78Mqs142J6a8852ouuEGQe4jN HO9JMDzQlNN7m0qGOxvo5vabIU2fVyOGNy/Qf3sj3QO41XmkGUbaHUPBqv/t+RKqLI6d rNwFx+e4vfsCLDo0FGqvU9hVOd0ZgJPT6eu4ifRdlWR0feau71uXKE+IhWcpIbLTT2BJ 0pJHXyN/Vr+MQLaC8xjDlZKg8DYhXFDF2fAH/3wfrgn1BLbGYWYkn3etFBKk8CpaiKq2 kU4PBvxSSic5gH3LJbnYW3tmgKN+Cz39f9tWIklDdKw8XxyTjMwhdhA/+NUan4+wIvk2 88hQ== X-Gm-Message-State: AOAM5329KmepCyNIyyvEX1kLLQKiJBB0hnfLbQDQPIgBEPtE/yZjBtX0 SfMWltBo0NCjBrOAB4GN2v16BGEv7nQ0Fg== X-Google-Smtp-Source: ABdhPJzMfW4qX3qnbcVKZnncFaiK9dUmvbxDB0y46fE//+X0znkzkv1fTxMQfiXlf0jaHX4fxBd5mA== X-Received: by 2002:a1c:a784:: with SMTP id q126mr197637wme.52.1611070251950; Tue, 19 Jan 2021 07:30:51 -0800 (PST) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id k9sm5373970wma.17.2021.01.19.07.30.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jan 2021 07:30:51 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 19 Jan 2021 15:30:45 +0000 Message-Id: <20210119153047.468190-1-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v12 1/3] libcamera: controls: Add frame duration control X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add an int64_t array control (controls::FrameDurations) to specify the minimum and maximum (in that order) frame duration to be used by the camera sensor. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- src/libcamera/control_ids.yaml | 36 ++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml index 6d6f0fee508b..36c37161a3d5 100644 --- a/src/libcamera/control_ids.yaml +++ b/src/libcamera/control_ids.yaml @@ -306,6 +306,42 @@ controls: maximum valid value is given by the properties::ScalerCropMaximum property, and the two can be used to implement digital zoom. + - FrameDurations: + type: int64_t + description: | + The minimum and maximum (in that order) frame duration, + expressed in micro-seconds. + + When provided by applications, the control specifies the sensor frame + duration interval the pipeline has to use. This limits the largest + exposure time the sensor can use. For example, if a maximum frame + duration of 33ms is requested (corresponding to 30 frames per second), + the sensor will not be able to raise the exposure time above 33ms. + A fixed frame duration is achieved by setting the minimum and maximum + values to be the same. Setting both values to 0 reverts to using the + IPA provided defaults. + + The maximum frame duration provides the absolute limit to the shutter + speed computed by the AE algorithm and it overrides any exposure mode + setting specified with controls::AeExposureMode. Similarly, when a + manual exposure time is set through controls::ExposureTime, it also + gets clipped to the limits set by this control. When reported in + metadata, the control expresses the minimum and maximum frame + durations used after being clipped to the sensor provided frame + duration limits. + + \sa AeExposureMode + \sa ExposureTime + + \todo Define how to calculate the capture frame rate by + defining controls to report additional delays introduced by + the capture pipeline or post-processing stages (ie JPEG + conversion, frame scaling). + + \todo Provide an explicit definition of default control values, for + this and all other controls. + size: [2] + # ---------------------------------------------------------------------------- # Draft controls section From patchwork Tue Jan 19 15:30:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 10901 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5ADAEC0F1C for ; Tue, 19 Jan 2021 15:30:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0EDC768162; Tue, 19 Jan 2021 16:30:56 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="So1tkQrY"; dkim-atps=neutral Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CFFBC6815E for ; Tue, 19 Jan 2021 16:30:53 +0100 (CET) Received: by mail-wm1-x331.google.com with SMTP id j18so130944wmi.3 for ; Tue, 19 Jan 2021 07:30:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kaqoy9cyAz0Aq56p9abJNexhBLCETzE4+zcaazjRgJg=; b=So1tkQrYZghKif6d/iO+EEC54oa4+qodrmyXMjd2k7i64Pkeg17jHC/hGdgxZh43mb OrqrCtKxPmQce4m/1wGbNO1AiUgad0lEZVypGmcXab5lpXtiP+3wg1qY1E6IdOzWwTs1 4CuOh4OqHU7rrbKQKOTmD95K/f1WDKfRAQszqkXoHUVADlXXVeKTiB5psml12LPS0gl7 FQOIxPTIbtQvrAUlcWn7GC+8mbqg5TWJrES2iSATANkpedLNjfXagDpF772rXsU2SfsL S8y1sssMG3BHKDRlX8nCZ9nSmypWJsOvMphKq4pJr0Fgi47ZYjnzP+ohpArOcUzKlzYu KfeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kaqoy9cyAz0Aq56p9abJNexhBLCETzE4+zcaazjRgJg=; b=BxdRd0MdivTqUf0MTpebAXG+9c46oiTrHA/8Lc1+JBpvkEsaW7b9YZTPNHN8sxBwKR oFxsJ333K+OY9Irp0m8xV+Zh6FC9Zuqh29o9+lbPkGOX1eWbcg+Mc5URpx85DsVJMY6F 7eGPMdZON6c/G/ybvUS+9v9hJeVgHJS7bOvs3eihn8CCWBwvkBvItaY+/ajhV1Cn196k vG4YuK4jRB73KduRQKWsVLrXPbb+V+hCDFlIr+SrNK/PAlDlbF9AN9F1A0avltZBsEzB ANEzfvdicYQmcDHdZjKPRcQiiLjSc1+FRdUKFHJshf34Q14Cgj4xBqeK8l70BQP2ckKj pmKw== X-Gm-Message-State: AOAM531H5OaPhaBos5OG8/aiQzVubNbPeg+W7seF2l/gqBnkkB/6pSqC zupydZpq3fUcmoGnRbzVEWoqnRw17PJINA== X-Google-Smtp-Source: ABdhPJxesZjo0+7+LzxgaXvr38m49rFINhCr5Xf0H1/3NskuVZjxIOnEepusN6SZczerS9kWc7UtUw== X-Received: by 2002:a1c:dd83:: with SMTP id u125mr154572wmg.93.1611070253039; Tue, 19 Jan 2021 07:30:53 -0800 (PST) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id k9sm5373970wma.17.2021.01.19.07.30.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jan 2021 07:30:52 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 19 Jan 2021 15:30:46 +0000 Message-Id: <20210119153047.468190-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210119153047.468190-1-naush@raspberrypi.com> References: <20210119153047.468190-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v12 2/3] libcamera: raspberrypi: Add control of sensor vblanking X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for setting V4L2_CID_VBLANK appropriately when setting V4L2_CID_EXPOSURE. This will allow adaptive framerates during viewfinder use cases (e.g. when the exposure time goes above 33ms, we can reduce the framerate to lower than 30fps). The minimum and maximum frame durations are provided via libcamera controls, and will prioritise exposure time limits over any AGC request. V4L2_CID_VBLANK is controlled through the staggered writer, just like the exposure and gain controls. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/raspberrypi.h | 1 + src/ipa/raspberrypi/cam_helper.cpp | 35 +++++++++++- src/ipa/raspberrypi/cam_helper.hpp | 15 ++++- src/ipa/raspberrypi/cam_helper_imx219.cpp | 13 ++++- src/ipa/raspberrypi/cam_helper_imx477.cpp | 11 +++- src/ipa/raspberrypi/cam_helper_ov5647.cpp | 11 +++- src/ipa/raspberrypi/raspberrypi.cpp | 56 ++++++++++++++++--- .../pipeline/raspberrypi/raspberrypi.cpp | 3 +- 8 files changed, 130 insertions(+), 15 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index 01fe5abce9a6..1de36039cee0 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -65,6 +65,7 @@ static const ControlInfoMap Controls = { { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, + { &controls::FrameDurations, ControlInfo(1000, 1000000000) }, }; } /* namespace RPi */ diff --git a/src/ipa/raspberrypi/cam_helper.cpp b/src/ipa/raspberrypi/cam_helper.cpp index 6efa0d7fd030..b7b8faf09c69 100644 --- a/src/ipa/raspberrypi/cam_helper.cpp +++ b/src/ipa/raspberrypi/cam_helper.cpp @@ -34,8 +34,10 @@ CamHelper *CamHelper::Create(std::string const &cam_name) return nullptr; } -CamHelper::CamHelper(MdParser *parser) - : parser_(parser), initialized_(false) +CamHelper::CamHelper(MdParser *parser, unsigned int maxFrameLength, + unsigned int frameIntegrationDiff) + : parser_(parser), initialized_(false), maxFrameLength_(maxFrameLength), + frameIntegrationDiff_(frameIntegrationDiff) { } @@ -56,6 +58,35 @@ double CamHelper::Exposure(uint32_t exposure_lines) const return exposure_lines * mode_.line_length / 1000.0; } +uint32_t CamHelper::GetVBlanking(double &exposure, double minFrameDuration, + double maxFrameDuration) const +{ + uint32_t frameLengthMin, frameLengthMax, vblank; + uint32_t exposureLines = ExposureLines(exposure); + + assert(initialized_); + + /* + * Clamp frame length by the frame duration range and the maximum allowable + * value in the sensor, given by maxFrameLength_. + */ + frameLengthMin = std::clamp(1e3 * minFrameDuration / mode_.line_length, + mode_.height, maxFrameLength_); + frameLengthMax = std::clamp(1e3 * maxFrameDuration / mode_.line_length, + mode_.height, maxFrameLength_); + /* + * Limit the exposure to the maximum frame duration requested, and + * re-calculate if it has been clipped. + */ + exposureLines = std::min(frameLengthMax - frameIntegrationDiff_, exposureLines); + exposure = Exposure(exposureLines); + + /* Limit the vblank to the range allowed by the frame length limits. */ + vblank = std::clamp(exposureLines + frameIntegrationDiff_, + frameLengthMin, frameLengthMax) - mode_.height; + return vblank; +} + void CamHelper::SetCameraMode(const CameraMode &mode) { mode_ = mode; diff --git a/src/ipa/raspberrypi/cam_helper.hpp b/src/ipa/raspberrypi/cam_helper.hpp index 044c28667f75..b1739a57e342 100644 --- a/src/ipa/raspberrypi/cam_helper.hpp +++ b/src/ipa/raspberrypi/cam_helper.hpp @@ -62,12 +62,15 @@ class CamHelper { public: static CamHelper *Create(std::string const &cam_name); - CamHelper(MdParser *parser); + CamHelper(MdParser *parser, unsigned int maxFrameLength, + unsigned int frameIntegrationDiff); virtual ~CamHelper(); void SetCameraMode(const CameraMode &mode); MdParser &Parser() const { return *parser_; } uint32_t ExposureLines(double exposure_us) const; double Exposure(uint32_t exposure_lines) const; // in us + virtual uint32_t GetVBlanking(double &exposure_us, double minFrameDuration, + double maxFrameDuration) const; virtual uint32_t GainCode(double gain) const = 0; virtual double Gain(uint32_t gain_code) const = 0; virtual void GetDelays(int &exposure_delay, int &gain_delay) const; @@ -76,10 +79,20 @@ public: virtual unsigned int HideFramesModeSwitch() const; virtual unsigned int MistrustFramesStartup() const; virtual unsigned int MistrustFramesModeSwitch() const; + protected: MdParser *parser_; CameraMode mode_; + +private: bool initialized_; + /* Largest possible frame length, in units of lines. */ + unsigned int maxFrameLength_; + /* + * Smallest difference between the frame length and integration time, + * in units of lines. + */ + unsigned int frameIntegrationDiff_; }; // This is for registering camera helpers with the system, so that the diff --git a/src/ipa/raspberrypi/cam_helper_imx219.cpp b/src/ipa/raspberrypi/cam_helper_imx219.cpp index db8ab879deb7..8688ec091c44 100644 --- a/src/ipa/raspberrypi/cam_helper_imx219.cpp +++ b/src/ipa/raspberrypi/cam_helper_imx219.cpp @@ -49,13 +49,22 @@ public: double Gain(uint32_t gain_code) const override; unsigned int MistrustFramesModeSwitch() const override; bool SensorEmbeddedDataPresent() const override; + +private: + /* + * Smallest difference between the frame length and integration time, + * in units of lines. + */ + static constexpr int frameIntegrationDiff = 4; + /* Largest possible frame length, in units of lines. */ + static constexpr int maxFrameLength = 0xffff; }; CamHelperImx219::CamHelperImx219() #if ENABLE_EMBEDDED_DATA - : CamHelper(new MdParserImx219()) + : CamHelper(new MdParserImx219(), maxFrameLength, frameIntegrationDiff) #else - : CamHelper(new MdParserRPi()) + : CamHelper(new MdParserRPi(), maxFrameLength, frameIntegrationDiff) #endif { } diff --git a/src/ipa/raspberrypi/cam_helper_imx477.cpp b/src/ipa/raspberrypi/cam_helper_imx477.cpp index 0e896ac74f88..5396131003f1 100644 --- a/src/ipa/raspberrypi/cam_helper_imx477.cpp +++ b/src/ipa/raspberrypi/cam_helper_imx477.cpp @@ -38,10 +38,19 @@ public: uint32_t GainCode(double gain) const override; double Gain(uint32_t gain_code) const override; bool SensorEmbeddedDataPresent() const override; + +private: + /* + * Smallest difference between the frame length and integration time, + * in units of lines. + */ + static constexpr int frameIntegrationDiff = 22; + /* Largest possible frame length, in units of lines. */ + static constexpr int maxFrameLength = 0xffdc; }; CamHelperImx477::CamHelperImx477() - : CamHelper(new MdParserImx477()) + : CamHelper(new MdParserImx477(), maxFrameLength, frameIntegrationDiff) { } diff --git a/src/ipa/raspberrypi/cam_helper_ov5647.cpp b/src/ipa/raspberrypi/cam_helper_ov5647.cpp index 0b841cd175fa..2bd8a754f133 100644 --- a/src/ipa/raspberrypi/cam_helper_ov5647.cpp +++ b/src/ipa/raspberrypi/cam_helper_ov5647.cpp @@ -23,6 +23,15 @@ public: unsigned int HideFramesModeSwitch() const override; unsigned int MistrustFramesStartup() const override; unsigned int MistrustFramesModeSwitch() const override; + +private: + /* + * Smallest difference between the frame length and integration time, + * in units of lines. + */ + static constexpr int frameIntegrationDiff = 4; + /* Largest possible frame length, in units of lines. */ + static constexpr int maxFrameLength = 0xffff; }; /* @@ -31,7 +40,7 @@ public: */ CamHelperOv5647::CamHelperOv5647() - : CamHelper(new MdParserRPi()) + : CamHelper(new MdParserRPi(), maxFrameLength, frameIntegrationDiff) { } diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index d087b07e186b..ba9bc398ef87 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -58,6 +58,8 @@ namespace libcamera { /* Configure the sensor with these values initially. */ constexpr double DefaultAnalogueGain = 1.0; constexpr unsigned int DefaultExposureTime = 20000; +constexpr double defaultMinFrameDuration = 1e6 / 30.0; +constexpr double defaultMaxFrameDuration = 1e6 / 0.01; LOG_DEFINE_CATEGORY(IPARPI) @@ -150,6 +152,10 @@ private: /* Distinguish the first camera start from others. */ bool firstStart_; + + /* Frame duration (1/fps) limits, given in microseconds. */ + double minFrameDuration_; + double maxFrameDuration_; }; int IPARPi::init(const IPASettings &settings) @@ -332,7 +338,8 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, sensorMetadata = helper_->SensorEmbeddedDataPresent(); result->data.push_back(gainDelay); - result->data.push_back(exposureDelay); + result->data.push_back(exposureDelay); /* For EXPOSURE ctrl */ + result->data.push_back(exposureDelay); /* For VBLANK ctrl */ result->data.push_back(sensorMetadata); result->operation |= RPi::IPA_CONFIG_STAGGERED_WRITE; @@ -377,6 +384,9 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, controller_.Initialise(); controllerInit_ = true; + minFrameDuration_ = defaultMinFrameDuration; + maxFrameDuration_ = defaultMaxFrameDuration; + /* Supply initial values for gain and exposure. */ ControlList ctrls(sensorCtrls_); AgcStatus agcStatus; @@ -524,7 +534,8 @@ bool IPARPi::validateSensorControls() { static const uint32_t ctrls[] = { V4L2_CID_ANALOGUE_GAIN, - V4L2_CID_EXPOSURE + V4L2_CID_EXPOSURE, + V4L2_CID_VBLANK, }; for (auto c : ctrls) { @@ -551,7 +562,7 @@ bool IPARPi::validateIspControls() V4L2_CID_USER_BCM2835_ISP_DENOISE, V4L2_CID_USER_BCM2835_ISP_SHARPEN, V4L2_CID_USER_BCM2835_ISP_DPC, - V4L2_CID_USER_BCM2835_ISP_LENS_SHADING + V4L2_CID_USER_BCM2835_ISP_LENS_SHADING, }; for (auto c : ctrls) { @@ -804,6 +815,25 @@ void IPARPi::queueRequest(const ControlList &controls) break; } + case controls::FRAME_DURATIONS: { + auto frameDurations = ctrl.second.get>(); + + /* This will be applied once AGC recalculations occur. */ + minFrameDuration_ = frameDurations[0] ? frameDurations[0] : defaultMinFrameDuration; + maxFrameDuration_ = frameDurations[1] ? frameDurations[1] : defaultMaxFrameDuration; + maxFrameDuration_ = std::max(maxFrameDuration_, minFrameDuration_); + + /* + * \todo The values returned in the metadata below must be + * correctly clipped by what the sensor mode suppots and + * what the AGC exposure mode or manual shutter speed limits + */ + libcameraMetadata_.set(controls::FrameDurations, + { static_cast(minFrameDuration_), + static_cast(maxFrameDuration_) }); + break; + } + default: LOG(IPARPI, Warning) << "Ctrl " << controls::controls.at(ctrl.first)->name() @@ -962,15 +992,27 @@ void IPARPi::applyAWB(const struct AwbStatus *awbStatus, ControlList &ctrls) void IPARPi::applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls) { int32_t gainCode = helper_->GainCode(agcStatus->analogue_gain); - int32_t exposureLines = helper_->ExposureLines(agcStatus->shutter_time); - LOG(IPARPI, Debug) << "Applying AGC Exposure: " << agcStatus->shutter_time - << " (Shutter lines: " << exposureLines << ") Gain: " + /* GetVBlanking might clip exposure time to the fps limits. */ + double exposure = agcStatus->shutter_time; + int32_t vblanking = helper_->GetVBlanking(exposure, minFrameDuration_, + maxFrameDuration_); + int32_t exposureLines = helper_->ExposureLines(exposure); + + LOG(IPARPI, Debug) << "Applying AGC Exposure: " << exposure + << " (Shutter lines: " << exposureLines << ", AGC requested " + << agcStatus->shutter_time << ") Gain: " << agcStatus->analogue_gain << " (Gain Code: " << gainCode << ")"; - ctrls.set(V4L2_CID_ANALOGUE_GAIN, gainCode); + /* + * Due to the behavior of V4L2, the current value of VBLANK could clip the + * exposure time without us knowing. The next time though this function should + * clip exposure correctly. + */ + ctrls.set(V4L2_CID_VBLANK, vblanking); ctrls.set(V4L2_CID_EXPOSURE, exposureLines); + ctrls.set(V4L2_CID_ANALOGUE_GAIN, gainCode); } void IPARPi::applyDG(const struct AgcStatus *dgStatus, ControlList &ctrls) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 7a5f5881b9b3..252cab64023e 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1233,7 +1233,8 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) if (!staggeredCtrl_) { staggeredCtrl_.init(unicam_[Unicam::Image].dev(), { { V4L2_CID_ANALOGUE_GAIN, result.data[resultIdx++] }, - { V4L2_CID_EXPOSURE, result.data[resultIdx++] } }); + { V4L2_CID_EXPOSURE, result.data[resultIdx++] }, + { V4L2_CID_VBLANK, result.data[resultIdx++] } }); sensorMetadata_ = result.data[resultIdx++]; } } From patchwork Tue Jan 19 15:30:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 10902 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0110FC0F1C for ; Tue, 19 Jan 2021 15:30:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 799AD68165; Tue, 19 Jan 2021 16:30:56 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="TcsBsWQx"; dkim-atps=neutral Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E8E366815A for ; Tue, 19 Jan 2021 16:30:54 +0100 (CET) Received: by mail-wr1-x42a.google.com with SMTP id w5so20125505wrm.11 for ; Tue, 19 Jan 2021 07:30:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=R6nrMbyhXO9BMSJttWjg2nXYMZuCVQpLluw5KYD+yPA=; b=TcsBsWQxlWc7OiAqmsIUT5XwFIVSB40uizr04AiUacakl65DxW6IdmmQ1os0XKuQtV bWKP82gdq2Ko2vAPJNhkELWk7kHeS4ywFxWesp+hvzFIedDC5tCUQ9g/MqfES1KB82oS 87pmOKVRKluFo+EKPnxqKUFL7LOCFnrUFM7jkuoySPgyLvSxr2vmgw8GB24w4gzwRkAy CvKiTwWEDgAGFFjGPLa863oedUjx6U2+ZCv6GFMb/0SqbOpmbVbFuGVjqTmch7maQ3Z3 RmrgQmYQnADsRtWol6aECQo9k+X1DwOwZiTErumDZNtIS0tQKGrE8w2z45GvuVUCWI6Q 4dBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R6nrMbyhXO9BMSJttWjg2nXYMZuCVQpLluw5KYD+yPA=; b=rAoG525d4kni1AYAL5dsYCdJZX5iGI+eja2r80jah7l4gDfqshMRdLlq/f4Px1WWnB zSsBpJ0EmCKT8b3CJooifMB/mhwCZJIozu5L51uk57aaW7hyH8SkTS8FwzRs18Xt6n69 8NYOKUoYMwqpF8mSEF4eY1QAXwiGQqjNs0zNozPA1j9xP3cidWhyZ0xNS3Om7ecRWcrk PsdqVT3KJ+/SK1Dbnt4Kqe5bvEjn39wdWZHxPvnXSZ1X3jRLwhU7NdnMQCcMXl+MCLb+ 9SV0/i/jMKZOdB9joxet5q5mOavBMpKzP9srAP6H1EcB+nJYV51PWZNTpoo5cwWv64vj f9Dg== X-Gm-Message-State: AOAM532vpNovgeXmaTqqyUAmpzE+a4FEIr7B7RL1/N67BKB4aFe76Dc/ w2Cfwtc3nZ3W3aCK9TAsag1S4zRNe/nU9A== X-Google-Smtp-Source: ABdhPJyLeWdQRMCRZjOmlPX8jEdFwanHx3xE1sH4RVttfMNNDwNdEGY7pth4j/BGtQBglOJ7j7OdAQ== X-Received: by 2002:a5d:62c7:: with SMTP id o7mr4865287wrv.257.1611070254441; Tue, 19 Jan 2021 07:30:54 -0800 (PST) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id k9sm5373970wma.17.2021.01.19.07.30.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jan 2021 07:30:53 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 19 Jan 2021 15:30:47 +0000 Message-Id: <20210119153047.468190-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210119153047.468190-1-naush@raspberrypi.com> References: <20210119153047.468190-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v12 3/3] ipa: raspberrypi: config: Update shutter speeds for imx219/477 and ov5647 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Set the maximum shutter speed for the normal exposure profile to 66ms, allowing viewfinder framerates to go down to approx. 15fps. Set the maximum shutter speed for the sport exposure profile to 33ms, limiting the minimum framerate to approx. 30fps. Add a long exposure profile to allow shutter speeds of up to 120ms. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/raspberrypi/data/imx219.json | 15 +++++++++++++-- src/ipa/raspberrypi/data/imx477.json | 15 +++++++++++++-- src/ipa/raspberrypi/data/ov5647.json | 15 +++++++++++++-- 3 files changed, 39 insertions(+), 6 deletions(-) diff --git a/src/ipa/raspberrypi/data/imx219.json b/src/ipa/raspberrypi/data/imx219.json index 212f8b9af460..1ec338be2ccb 100644 --- a/src/ipa/raspberrypi/data/imx219.json +++ b/src/ipa/raspberrypi/data/imx219.json @@ -133,7 +133,7 @@ { "shutter": [ - 100, 10000, 30000, 30000, 30000 + 100, 10000, 30000, 60000, 66666 ], "gain": [ @@ -144,7 +144,18 @@ { "shutter": [ - 100, 5000, 10000, 20000, 30000 + 100, 5000, 10000, 20000, 33333 + ], + "gain": + [ + 1.0, 2.0, 4.0, 6.0, 6.0 + ] + }, + "long": + { + "shutter": + [ + 100, 10000, 30000, 60000, 120000 ], "gain": [ diff --git a/src/ipa/raspberrypi/data/imx477.json b/src/ipa/raspberrypi/data/imx477.json index 12fe2f418560..9bee3f16e108 100644 --- a/src/ipa/raspberrypi/data/imx477.json +++ b/src/ipa/raspberrypi/data/imx477.json @@ -133,7 +133,7 @@ { "shutter": [ - 100, 10000, 30000, 60000, 120000 + 100, 10000, 30000, 60000, 66666 ], "gain": [ @@ -144,7 +144,18 @@ { "shutter": [ - 100, 5000, 10000, 20000, 120000 + 100, 5000, 10000, 20000, 33333 + ], + "gain": + [ + 1.0, 2.0, 4.0, 6.0, 6.0 + ] + }, + "long": + { + "shutter": + [ + 100, 10000, 30000, 60000, 120000 ], "gain": [ diff --git a/src/ipa/raspberrypi/data/ov5647.json b/src/ipa/raspberrypi/data/ov5647.json index 3d8a7c8f15c7..1a354f7cb036 100644 --- a/src/ipa/raspberrypi/data/ov5647.json +++ b/src/ipa/raspberrypi/data/ov5647.json @@ -133,7 +133,7 @@ { "shutter": [ - 100, 10000, 30000, 30000, 30000 + 100, 10000, 30000, 60000, 66666 ], "gain": [ @@ -144,7 +144,18 @@ { "shutter": [ - 100, 5000, 10000, 20000, 30000 + 100, 5000, 10000, 20000, 33333 + ], + "gain": + [ + 1.0, 2.0, 4.0, 6.0, 6.0 + ] + }, + "long": + { + "shutter": + [ + 100, 10000, 30000, 60000, 120000 ], "gain": [