{"id":3748,"url":"https://patchwork.libcamera.org/api/1.1/patches/3748/?format=json","web_url":"https://patchwork.libcamera.org/patch/3748/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20200511100150.5205-2-naush@raspberrypi.com>","date":"2020-05-11T10:01:49","name":"[libcamera-devel,1/2] libcamera: raspberrypi: Add control of sensor vblanking","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"8e308b4a96f6afd16e97d7bb50b7df9762eed71e","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/1.1/people/34/?format=json","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/3748/mbox/","series":[{"id":892,"url":"https://patchwork.libcamera.org/api/1.1/series/892/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=892","date":"2020-05-11T10:01:48","name":"raspberrypi: FPS control","version":1,"mbox":"https://patchwork.libcamera.org/series/892/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/3748/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/3748/checks/","tags":{},"headers":{"Return-Path":"","Received":["from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com\n\t[IPv6:2a00:1450:4864:20::42a])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4DF81603E2\n\tfor ;\n\tMon, 11 May 2020 12:01:59 +0200 (CEST)","by mail-wr1-x42a.google.com with SMTP id 50so9603147wrc.11\n\tfor ;\n\tMon, 11 May 2020 03:01:59 -0700 (PDT)","from naushir-VirtualBox.patuck.local ([88.97.76.4])\n\tby smtp.gmail.com with ESMTPSA id\n\tu74sm26391908wmu.13.2020.05.11.03.01.57\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tMon, 11 May 2020 03:01:58 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=raspberrypi.com\n\theader.i=@raspberrypi.com\n\theader.b=\"lFmEm4rN\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=from:to:cc:subject:date:message-id:in-reply-to:references;\n\tbh=TrA51dLWrdSSmq/21+LQQUS9w/a+8P4nyXisMlvm0n0=;\n\tb=lFmEm4rN0MC4FCCIB0pR8kbk3N7fXxgc4WAM9h+ba0kgBlZb6QS4Cw+86LFxZ3eEnY\n\tE2eKKYmYELRe2/0obvh1CTCesXShTEQtitoa8UGPaVOrqTK1LQeKh53Ct9IEhKMjmmoa\n\tJYu+9mMdtKPleXR+jg7HoNlzuszpk6ydhX+W8x/JC4DVWP3OLBsLf9x+l4p3Pymq3VtE\n\tcZJrXPAwQPmmaMA76Uj0i2+/sGTyhxybKWe0Xya1tkQ0g7r7wgsxMfUj2ei/+BVpzYRL\n\tfIbDwqNGaUIpc3lnkTAuwkhzIfaf1zi6VPqFHlFfCgf72wyniukRJ0UAx0dQBMRDbk83\n\tUDeg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references;\n\tbh=TrA51dLWrdSSmq/21+LQQUS9w/a+8P4nyXisMlvm0n0=;\n\tb=BNvxc1qzVrM1em642DMd4bNbyjHiX3XMhirwFNH4L6AsXxkusa5bBmpTy4iAd0HYXj\n\tCKUIv+BfCqJDdJb0L9A6HEXDXqPa+H42URCKBvdUestUQVDwBSo8W9CKCRaIrV1klXo3\n\tOkTdKwkgN+tenSoijDj1M0YYUAcAzerhJ52lapgK0ULsVshoL6SEtnNLsH4nrAoEpaJg\n\tB0WAzDL6dfd1RFgbtlHQMSw/91dMC2AuUa2ocuLNMctiZNFRIpaCsDr1QioDMhUyN5cb\n\tc/ky3yNIoDbDVyYYi3xJRBO3WHKgdy79EMxj3YEWL3zzejGkEbYwKibK3w8PLaMmGt4+\n\t57YA==","X-Gm-Message-State":"AGi0PuZMI2Rs/U0fAwwEgvV7sO1bvfoQQGw8gnJi7JBySU//gq1uhusb\n\tq3TlMUT5/fOchcq8/uh7UPF/XySwKkytyQ==","X-Google-Smtp-Source":"APiQypKu3V8sxz8TQDPdZdOMToB69EL3wFJHiiwTO02by9SHuKjjGqrUAdGN9NRmONazsYUHGJI2LQ==","X-Received":"by 2002:adf:ffc2:: with SMTP id\n\tx2mr17644148wrs.273.1589191318686; \n\tMon, 11 May 2020 03:01:58 -0700 (PDT)","From":"Naushir Patuck ","To":"libcamera-devel@lists.libcamera.org","Date":"Mon, 11 May 2020 11:01:49 +0100","Message-Id":"<20200511100150.5205-2-naush@raspberrypi.com>","X-Mailer":"git-send-email 2.17.1","In-Reply-To":"<20200511100150.5205-1-naush@raspberrypi.com>","References":"<20200511100150.5205-1-naush@raspberrypi.com>","Subject":"[libcamera-devel] [PATCH 1/2] libcamera: raspberrypi: Add control\n\tof sensor vblanking","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Mon, 11 May 2020 10:01:59 -0000"},"content":"Add support for setting V4L2_CID_VBLANK appropriately when setting\nV4L2_CID_EXPOSURE. This will allow adaptive framerates during\nviewfinder use cases (e.g. when the exposure time goes above 33ms, we\ncan reduce the framerate to lower than 30fps).\n\nCurrently, the minimum framerate is dictated by the lowest available\nexposure time in the tuning, and the maximum framerate is limited by\nthe sensor mode.\n\nV4L2_CID_VBLANK is controlled through the staggered writer, just like\nthe exposure and gain controls.\n\nSigned-off-by: Naushir Patuck \n---\n src/ipa/raspberrypi/cam_helper.hpp | 1 +\n src/ipa/raspberrypi/cam_helper_imx219.cpp | 13 +++++++++++++\n src/ipa/raspberrypi/cam_helper_imx477.cpp | 13 +++++++++++++\n src/ipa/raspberrypi/cam_helper_ov5647.cpp | 13 +++++++++++++\n src/ipa/raspberrypi/raspberrypi.cpp | 8 +++++++-\n src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 2 ++\n 6 files changed, 49 insertions(+), 1 deletion(-)","diff":"diff --git a/src/ipa/raspberrypi/cam_helper.hpp b/src/ipa/raspberrypi/cam_helper.hpp\nindex 0c8aa29a..37281c78 100644\n--- a/src/ipa/raspberrypi/cam_helper.hpp\n+++ b/src/ipa/raspberrypi/cam_helper.hpp\n@@ -74,6 +74,7 @@ public:\n \tMdParser &Parser() const { return *parser_; }\n \tuint32_t ExposureLines(double exposure_us) const;\n \tdouble Exposure(uint32_t exposure_lines) const; // in us\n+\tvirtual uint32_t GetVBlanking(double exposure_us) const = 0;\n \tvirtual uint32_t GainCode(double gain) const = 0;\n \tvirtual double Gain(uint32_t gain_code) const = 0;\n \tvirtual void GetDelays(int &exposure_delay, int &gain_delay) const;\ndiff --git a/src/ipa/raspberrypi/cam_helper_imx219.cpp b/src/ipa/raspberrypi/cam_helper_imx219.cpp\nindex 35c6597c..d34a1f1f 100644\n--- a/src/ipa/raspberrypi/cam_helper_imx219.cpp\n+++ b/src/ipa/raspberrypi/cam_helper_imx219.cpp\n@@ -45,11 +45,16 @@ class CamHelperImx219 : public CamHelper\n {\n public:\n \tCamHelperImx219();\n+\tuint32_t GetVBlanking(double exposure) const override;\n \tuint32_t GainCode(double gain) const override;\n \tdouble Gain(uint32_t gain_code) const override;\n \tunsigned int MistrustFramesModeSwitch() const override;\n \tbool SensorEmbeddedDataPresent() const override;\n \tCamTransform GetOrientation() const override;\n+\n+private:\n+\t/* Smallest difference between the frame length and integration time. */\n+\tstatic constexpr int frameIntegrationDiff = 4;\n };\n \n CamHelperImx219::CamHelperImx219()\n@@ -61,6 +66,14 @@ CamHelperImx219::CamHelperImx219()\n {\n }\n \n+uint32_t CamHelperImx219::GetVBlanking(double exposure) const\n+{\n+\tuint32_t exposureLines = ExposureLines(exposure);\n+\n+\treturn std::max(mode_.height, exposureLines) -\n+\t mode_.height + frameIntegrationDiff;\n+}\n+\n uint32_t CamHelperImx219::GainCode(double gain) const\n {\n \treturn (uint32_t)(256 - 256 / gain);\ndiff --git a/src/ipa/raspberrypi/cam_helper_imx477.cpp b/src/ipa/raspberrypi/cam_helper_imx477.cpp\nindex 69544456..242d9689 100644\n--- a/src/ipa/raspberrypi/cam_helper_imx477.cpp\n+++ b/src/ipa/raspberrypi/cam_helper_imx477.cpp\n@@ -35,10 +35,15 @@ class CamHelperImx477 : public CamHelper\n {\n public:\n \tCamHelperImx477();\n+\tuint32_t GetVBlanking(double exposure) const override;\n \tuint32_t GainCode(double gain) const override;\n \tdouble Gain(uint32_t gain_code) const override;\n \tbool SensorEmbeddedDataPresent() const override;\n \tCamTransform GetOrientation() const override;\n+\n+private:\n+\t/* Smallest difference between the frame length and integration time. */\n+\tstatic constexpr int frameIntegrationDiff = 22;\n };\n \n CamHelperImx477::CamHelperImx477()\n@@ -46,6 +51,14 @@ CamHelperImx477::CamHelperImx477()\n {\n }\n \n+uint32_t CamHelperImx477::GetVBlanking(double exposure) const\n+{\n+\tuint32_t exposureLines = ExposureLines(exposure);\n+\n+\treturn std::max(mode_.height, exposureLines) -\n+\t mode_.height + frameIntegrationDiff;\n+}\n+\n uint32_t CamHelperImx477::GainCode(double gain) const\n {\n \treturn static_cast(1024 - 1024 / gain);\ndiff --git a/src/ipa/raspberrypi/cam_helper_ov5647.cpp b/src/ipa/raspberrypi/cam_helper_ov5647.cpp\nindex 3dbcb164..ff37779c 100644\n--- a/src/ipa/raspberrypi/cam_helper_ov5647.cpp\n+++ b/src/ipa/raspberrypi/cam_helper_ov5647.cpp\n@@ -16,12 +16,17 @@ class CamHelperOv5647 : public CamHelper\n {\n public:\n \tCamHelperOv5647();\n+\tuint32_t GetVBlanking(double exposure) const override;\n \tuint32_t GainCode(double gain) const override;\n \tdouble Gain(uint32_t gain_code) const override;\n \tvoid GetDelays(int &exposure_delay, int &gain_delay) const override;\n \tunsigned int HideFramesModeSwitch() const override;\n \tunsigned int MistrustFramesStartup() const override;\n \tunsigned int MistrustFramesModeSwitch() const override;\n+\n+private:\n+\t/* Smallest difference between the frame length and integration time. */\n+\tstatic constexpr int frameIntegrationDiff = 4;\n };\n \n /*\n@@ -34,6 +39,14 @@ CamHelperOv5647::CamHelperOv5647()\n {\n }\n \n+uint32_t CamHelperOv5647::GetVBlanking(double exposure) const\n+{\n+\tuint32_t exposureLines = ExposureLines(exposure);\n+\n+\treturn std::max(mode_.height, exposureLines) -\n+\t mode_.height + frameIntegrationDiff;\n+}\n+\n uint32_t CamHelperOv5647::GainCode(double gain) const\n {\n \treturn static_cast(gain * 16.0);\ndiff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp\nindex 80dc77a3..104727ea 100644\n--- a/src/ipa/raspberrypi/raspberrypi.cpp\n+++ b/src/ipa/raspberrypi/raspberrypi.cpp\n@@ -795,6 +795,7 @@ void IPARPi::applyAGC(const struct AgcStatus *agcStatus)\n \n \tint32_t gain_code = helper_->GainCode(agcStatus->analogue_gain);\n \tint32_t exposure_lines = helper_->ExposureLines(agcStatus->shutter_time);\n+\tint32_t vblanking = helper_->GetVBlanking(agcStatus->shutter_time);\n \n \tif (unicam_ctrls_.find(V4L2_CID_ANALOGUE_GAIN) == unicam_ctrls_.end()) {\n \t\tLOG(IPARPI, Error) << \"Can't find analogue gain control\";\n@@ -812,8 +813,13 @@ void IPARPi::applyAGC(const struct AgcStatus *agcStatus)\n \t\t\t << gain_code << \")\";\n \n \tControlList ctrls(unicam_ctrls_);\n-\tctrls.set(V4L2_CID_ANALOGUE_GAIN, gain_code);\n+\t/*\n+\t * VBLANK must be set before EXPOSURE as the former will adjust the\n+\t * limits of the latter control.\n+\t */\n+\tctrls.set(V4L2_CID_VBLANK, vblanking);\n \tctrls.set(V4L2_CID_EXPOSURE, exposure_lines);\n+\tctrls.set(V4L2_CID_ANALOGUE_GAIN, gain_code);\n \top.controls.push_back(ctrls);\n \tqueueFrameAction.emit(0, op);\n }\ndiff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\nindex 16850819..e4c5d8f0 100644\n--- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n+++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n@@ -1159,7 +1159,9 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData\n \t\tif (!staggeredCtrl_) {\n \t\t\tstaggeredCtrl_.init(unicam_[Unicam::Image].dev(),\n \t\t\t\t\t { { V4L2_CID_ANALOGUE_GAIN, action.data[0] },\n+\t\t\t\t\t { V4L2_CID_VBLANK, action.data[1] },\n \t\t\t\t\t { V4L2_CID_EXPOSURE, action.data[1] } });\n+\n \t\t\tsensorMetadata_ = action.data[2];\n \t\t}\n \n","prefixes":["libcamera-devel","1/2"]}