{"id":4015,"url":"https://patchwork.libcamera.org/api/patches/4015/?format=json","web_url":"https://patchwork.libcamera.org/patch/4015/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20200610142455.4641-1-david.plowman@raspberrypi.com>","date":"2020-06-10T14:24:55","name":"[libcamera-devel] libcamera: raspberrypi: correct exposure values when camera mode changes","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"7e4890a237459e3e511eaab2f407f4fcb39fca7d","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/people/42/?format=json","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/4015/mbox/","series":[{"id":984,"url":"https://patchwork.libcamera.org/api/series/984/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=984","date":"2020-06-10T14:24:55","name":"[libcamera-devel] libcamera: raspberrypi: correct exposure values when camera mode changes","version":1,"mbox":"https://patchwork.libcamera.org/series/984/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/4015/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/4015/checks/","tags":{},"headers":{"Return-Path":"<david.plowman@raspberrypi.com>","Received":["from mail-wr1-x431.google.com (mail-wr1-x431.google.com\n\t[IPv6:2a00:1450:4864:20::431])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id ECA5B603C4\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 10 Jun 2020 16:24:59 +0200 (CEST)","by mail-wr1-x431.google.com with SMTP id r7so2527952wro.1\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 10 Jun 2020 07:24:59 -0700 (PDT)","from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72])\n\tby smtp.gmail.com with ESMTPSA id\n\to18sm7535615wme.19.2020.06.10.07.24.58\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tWed, 10 Jun 2020 07:24:58 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=raspberrypi.com\n\theader.i=@raspberrypi.com\n\theader.b=\"cpb5vVD0\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=from:to:cc:subject:date:message-id:mime-version\n\t:content-transfer-encoding;\n\tbh=cGYOngz5wHFZ0iZ1KsBkdaPYjzQo9fShw7QLyNjhwDc=;\n\tb=cpb5vVD0xCVoWEk87OJ/4SvgI8KsqAHGF41kXd2VBgrIs8Mo7BcxQri512a6zEkhL7\n\tPXNCdZPtyHZMNqY5XOZlD6+q+OKInJgfPCkEKCgVl11OZXTy4bBE2y4c8vWFALxAkLFK\n\tCexT7LmdLKbXoQF4H41Qlr1abVFLIdrzhJ3CHM8ZlDFXsb8r9gwD3AtdopYyDe1UdhGc\n\th3ctks8KtqBC8FgYV1FoAfEFwg3kCEFpKjKcRFgDNOLqVBxK80LQV1W+q6D6A1hv8cSz\n\t4CJS0YW8NbYEzAnOoc5aCC3ao05Gq5Aj3dUdhao/YvfVOXkOgAItmZ8PVv1J5/1dem8u\n\tJVog==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version\n\t:content-transfer-encoding;\n\tbh=cGYOngz5wHFZ0iZ1KsBkdaPYjzQo9fShw7QLyNjhwDc=;\n\tb=DqikPUqrfhFrfdst3DrvzjWy6TC6kkGeEtgoY6nZwAGYWaLTqXP86cdJlEbLIB/Ll4\n\t2MXaKO6TOvH61G2klLCRgpoBwt4vw8AVzgQlIrdoDS5nAE9+REbf1MJkgL2Jo6dExUoT\n\t1+1pxZR3kjG5gvjmI3u+dsAxmA32AVTFvhmzAQXA46xNgXIfWyqr5aQWqzOG6IVPfj3B\n\t8l6I9lSNM2FJSOLTjdcOR/VmQFygmSIwHWYU3rcUxA4H2qi/AdNiqXKsYBWKn3qn1r7H\n\t0KZi0SstX5i7NwfrVsT2chFnYh2FLgAHv5K36WW2aECrr7dmCDjxHuMaO3+PkgS6xsvk\n\t2vpQ==","X-Gm-Message-State":"AOAM532/0NapvYhB5f+9nUH3LAHN3OvRo2XlWOpI3vSX+gEtbxYSLxXV\n\tLXi9/wkXHu1U2j8F2hNd3Lq4jf3LWBA=","X-Google-Smtp-Source":"ABdhPJz/qtq3mOPFm0Qp7NvpN0F5/EIjsUCCgA7sMJYC0UzZNHtYduycCm9mxQeyHedblBsk94GMZg==","X-Received":"by 2002:a5d:4ec3:: with SMTP id s3mr4312855wrv.103.1591799098963;\n\tWed, 10 Jun 2020 07:24:58 -0700 (PDT)","From":"David Plowman <david.plowman@raspberrypi.com>","To":"libcamera-devel@lists.libcamera.org","Date":"Wed, 10 Jun 2020 15:24:55 +0100","Message-Id":"<20200610142455.4641-1-david.plowman@raspberrypi.com>","X-Mailer":"git-send-email 2.20.1","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH] libcamera: raspberrypi: correct exposure\n\tvalues when camera mode changes","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","X-List-Received-Date":"Wed, 10 Jun 2020 14:25:00 -0000"},"content":"When the sensor is switched into a different mode the line\ntimings may change, yet the V4L2 driver remembers the number of\nexposure lines, not the total time. So this will change \"under\nour feet\".\n\nThese patches extend the IPA handling of the camera mode switch\nso that correct exposure values are recalculated for the new mode\nand written to the sensor before it starts streaming again. They\nalso allow the IPA to alter how the total exposure is divided\nbetween actual exposure and analogue gain, according to the\nselected exposure profile in the camera tuning.\n\nSigned-off-by: David Plowman <david.plowman@raspberrypi.com>\n---\n src/ipa/raspberrypi/controller/algorithm.cpp  |  3 +-\n src/ipa/raspberrypi/controller/algorithm.hpp  |  2 +-\n src/ipa/raspberrypi/controller/controller.cpp |  4 +--\n src/ipa/raspberrypi/controller/controller.hpp |  2 +-\n src/ipa/raspberrypi/controller/rpi/agc.cpp    | 12 ++++++++\n src/ipa/raspberrypi/controller/rpi/agc.hpp    |  1 +\n src/ipa/raspberrypi/controller/rpi/alsc.cpp   |  4 ++-\n src/ipa/raspberrypi/controller/rpi/alsc.hpp   |  2 +-\n src/ipa/raspberrypi/controller/rpi/noise.cpp  |  4 ++-\n src/ipa/raspberrypi/controller/rpi/noise.hpp  |  2 +-\n .../raspberrypi/controller/rpi/sharpen.cpp    |  4 ++-\n .../raspberrypi/controller/rpi/sharpen.hpp    |  2 +-\n src/ipa/raspberrypi/raspberrypi.cpp           | 28 ++++++++++---------\n 13 files changed, 46 insertions(+), 24 deletions(-)","diff":"diff --git a/src/ipa/raspberrypi/controller/algorithm.cpp b/src/ipa/raspberrypi/controller/algorithm.cpp\nindex 9bd3df8..55cb201 100644\n--- a/src/ipa/raspberrypi/controller/algorithm.cpp\n+++ b/src/ipa/raspberrypi/controller/algorithm.cpp\n@@ -16,9 +16,10 @@ void Algorithm::Read(boost::property_tree::ptree const &params)\n \n void Algorithm::Initialise() {}\n \n-void Algorithm::SwitchMode(CameraMode const &camera_mode)\n+void Algorithm::SwitchMode(CameraMode const &camera_mode, Metadata *metadata)\n {\n \t(void)camera_mode;\n+\t(void)metadata;\n }\n \n void Algorithm::Prepare(Metadata *image_metadata)\ndiff --git a/src/ipa/raspberrypi/controller/algorithm.hpp b/src/ipa/raspberrypi/controller/algorithm.hpp\nindex b82c184..187c50c 100644\n--- a/src/ipa/raspberrypi/controller/algorithm.hpp\n+++ b/src/ipa/raspberrypi/controller/algorithm.hpp\n@@ -37,7 +37,7 @@ public:\n \tvirtual void Resume() { paused_ = false; }\n \tvirtual void Read(boost::property_tree::ptree const &params);\n \tvirtual void Initialise();\n-\tvirtual void SwitchMode(CameraMode const &camera_mode);\n+\tvirtual void SwitchMode(CameraMode const &camera_mode, Metadata *metadata);\n \tvirtual void Prepare(Metadata *image_metadata);\n \tvirtual void Process(StatisticsPtr &stats, Metadata *image_metadata);\n \tMetadata &GetGlobalMetadata() const\ndiff --git a/src/ipa/raspberrypi/controller/controller.cpp b/src/ipa/raspberrypi/controller/controller.cpp\nindex 20dd4c7..7c4b04f 100644\n--- a/src/ipa/raspberrypi/controller/controller.cpp\n+++ b/src/ipa/raspberrypi/controller/controller.cpp\n@@ -56,11 +56,11 @@ void Controller::Initialise()\n \tRPI_LOG(\"Controller finished\");\n }\n \n-void Controller::SwitchMode(CameraMode const &camera_mode)\n+void Controller::SwitchMode(CameraMode const &camera_mode, Metadata *metadata)\n {\n \tRPI_LOG(\"Controller starting\");\n \tfor (auto &algo : algorithms_)\n-\t\talgo->SwitchMode(camera_mode);\n+\t\talgo->SwitchMode(camera_mode, metadata);\n \tswitch_mode_called_ = true;\n \tRPI_LOG(\"Controller finished\");\n }\ndiff --git a/src/ipa/raspberrypi/controller/controller.hpp b/src/ipa/raspberrypi/controller/controller.hpp\nindex d685386..6ba9412 100644\n--- a/src/ipa/raspberrypi/controller/controller.hpp\n+++ b/src/ipa/raspberrypi/controller/controller.hpp\n@@ -39,7 +39,7 @@ public:\n \tAlgorithm *CreateAlgorithm(char const *name);\n \tvoid Read(char const *filename);\n \tvoid Initialise();\n-\tvoid SwitchMode(CameraMode const &camera_mode);\n+\tvoid SwitchMode(CameraMode const &camera_mode, Metadata *metadata);\n \tvoid Prepare(Metadata *image_metadata);\n \tvoid Process(StatisticsPtr stats, Metadata *image_metadata);\n \tMetadata &GetGlobalMetadata();\ndiff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp\nindex a474287..c02b5ec 100644\n--- a/src/ipa/raspberrypi/controller/rpi/agc.cpp\n+++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp\n@@ -221,6 +221,18 @@ void Agc::SetConstraintMode(std::string const &constraint_mode_name)\n \tconstraint_mode_name_ = constraint_mode_name;\n }\n \n+void Agc::SwitchMode(CameraMode const &camera_mode, Metadata *metadata)\n+{\n+\t// On a mode switch, it's possible the exposure profile could change,\n+\t// so we run through the dividing up of exposure/gain again and\n+\t// write the results into the metadata we've been given.\n+\tif (status_.total_exposure_value) {\n+\t\thousekeepConfig();\n+\t\tdivvyupExposure();\n+\t\twriteAndFinish(metadata, false);\n+\t}\n+}\n+\n void Agc::Prepare(Metadata *image_metadata)\n {\n \tAgcStatus status;\ndiff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp\nindex dbcefba..9a7e89c 100644\n--- a/src/ipa/raspberrypi/controller/rpi/agc.hpp\n+++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp\n@@ -75,6 +75,7 @@ public:\n \tvoid SetMeteringMode(std::string const &metering_mode_name) override;\n \tvoid SetExposureMode(std::string const &exposure_mode_name) override;\n \tvoid SetConstraintMode(std::string const &contraint_mode_name) override;\n+\tvoid SwitchMode(CameraMode const &camera_mode, Metadata *metadata) override;\n \tvoid Prepare(Metadata *image_metadata) override;\n \tvoid Process(StatisticsPtr &stats, Metadata *image_metadata) override;\n \ndiff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp\nindex 821a0ca..76e2f04 100644\n--- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp\n+++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp\n@@ -173,8 +173,10 @@ void Alsc::Initialise()\n \t\tlambda_r_[i] = lambda_b_[i] = 1.0;\n }\n \n-void Alsc::SwitchMode(CameraMode const &camera_mode)\n+void Alsc::SwitchMode(CameraMode const &camera_mode, Metadata *metadata)\n {\n+\t(void)metadata;\n+\n \t// There's a bit of a question what we should do if the \"crop\" of the\n \t// camera mode has changed.  Any calculation currently in flight would\n \t// not be useful to the new mode, so arguably we should abort it, and\ndiff --git a/src/ipa/raspberrypi/controller/rpi/alsc.hpp b/src/ipa/raspberrypi/controller/rpi/alsc.hpp\nindex c8ed3d2..3806257 100644\n--- a/src/ipa/raspberrypi/controller/rpi/alsc.hpp\n+++ b/src/ipa/raspberrypi/controller/rpi/alsc.hpp\n@@ -50,7 +50,7 @@ public:\n \t~Alsc();\n \tchar const *Name() const override;\n \tvoid Initialise() override;\n-\tvoid SwitchMode(CameraMode const &camera_mode) override;\n+\tvoid SwitchMode(CameraMode const &camera_mode, Metadata *metadata) override;\n \tvoid Read(boost::property_tree::ptree const &params) override;\n \tvoid Prepare(Metadata *image_metadata) override;\n \tvoid Process(StatisticsPtr &stats, Metadata *image_metadata) override;\ndiff --git a/src/ipa/raspberrypi/controller/rpi/noise.cpp b/src/ipa/raspberrypi/controller/rpi/noise.cpp\nindex 2209d79..2cafde3 100644\n--- a/src/ipa/raspberrypi/controller/rpi/noise.cpp\n+++ b/src/ipa/raspberrypi/controller/rpi/noise.cpp\n@@ -27,8 +27,10 @@ char const *Noise::Name() const\n \treturn NAME;\n }\n \n-void Noise::SwitchMode(CameraMode const &camera_mode)\n+void Noise::SwitchMode(CameraMode const &camera_mode, Metadata *metadata)\n {\n+\t(void)metadata;\n+\n \t// For example, we would expect a 2x2 binned mode to have a \"noise\n \t// factor\" of sqrt(2x2) = 2. (can't be less than one, right?)\n \tmode_factor_ = std::max(1.0, camera_mode.noise_factor);\ndiff --git a/src/ipa/raspberrypi/controller/rpi/noise.hpp b/src/ipa/raspberrypi/controller/rpi/noise.hpp\nindex 51d46a3..25bf188 100644\n--- a/src/ipa/raspberrypi/controller/rpi/noise.hpp\n+++ b/src/ipa/raspberrypi/controller/rpi/noise.hpp\n@@ -18,7 +18,7 @@ class Noise : public Algorithm\n public:\n \tNoise(Controller *controller);\n \tchar const *Name() const override;\n-\tvoid SwitchMode(CameraMode const &camera_mode) override;\n+\tvoid SwitchMode(CameraMode const &camera_mode, Metadata *metadata) override;\n \tvoid Read(boost::property_tree::ptree const &params) override;\n \tvoid Prepare(Metadata *image_metadata) override;\n \ndiff --git a/src/ipa/raspberrypi/controller/rpi/sharpen.cpp b/src/ipa/raspberrypi/controller/rpi/sharpen.cpp\nindex 1f07bb6..086952f 100644\n--- a/src/ipa/raspberrypi/controller/rpi/sharpen.cpp\n+++ b/src/ipa/raspberrypi/controller/rpi/sharpen.cpp\n@@ -26,8 +26,10 @@ char const *Sharpen::Name() const\n \treturn NAME;\n }\n \n-void Sharpen::SwitchMode(CameraMode const &camera_mode)\n+void Sharpen::SwitchMode(CameraMode const &camera_mode, Metadata *metadata)\n {\n+\t(void)metadata;\n+\n \t// can't be less than one, right?\n \tmode_factor_ = std::max(1.0, camera_mode.noise_factor);\n }\ndiff --git a/src/ipa/raspberrypi/controller/rpi/sharpen.hpp b/src/ipa/raspberrypi/controller/rpi/sharpen.hpp\nindex 3b0d680..f871aa6 100644\n--- a/src/ipa/raspberrypi/controller/rpi/sharpen.hpp\n+++ b/src/ipa/raspberrypi/controller/rpi/sharpen.hpp\n@@ -18,7 +18,7 @@ class Sharpen : public Algorithm\n public:\n \tSharpen(Controller *controller);\n \tchar const *Name() const override;\n-\tvoid SwitchMode(CameraMode const &camera_mode) override;\n+\tvoid SwitchMode(CameraMode const &camera_mode, Metadata *metadata) override;\n \tvoid Read(boost::property_tree::ptree const &params) override;\n \tvoid Prepare(Metadata *image_metadata) override;\n \ndiff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp\nindex 9669f21..42c84b1 100644\n--- a/src/ipa/raspberrypi/raspberrypi.cpp\n+++ b/src/ipa/raspberrypi/raspberrypi.cpp\n@@ -247,27 +247,29 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo,\n \t\tmistrust_count_ = helper_->MistrustFramesStartup();\n \t}\n \n+\tstruct AgcStatus agcStatus;\n+\t/* These zero values mean not program anything (unless overwritten). */\n+\tagcStatus.shutter_time = 0.0;\n+\tagcStatus.analogue_gain = 0.0;\n+\n \tif (!controllerInit_) {\n \t\t/* Load the tuning file for this sensor. */\n \t\tcontroller_.Read(tuningFile_.c_str());\n \t\tcontroller_.Initialise();\n \t\tcontrollerInit_ = true;\n \n-\t\t/* Calculate initial values for gain and exposure. */\n-\t\tint32_t gain_code = helper_->GainCode(DEFAULT_ANALOGUE_GAIN);\n-\t\tint32_t exposure_lines = helper_->ExposureLines(DEFAULT_EXPOSURE_TIME);\n-\n-\t\tControlList ctrls(unicam_ctrls_);\n-\t\tctrls.set(V4L2_CID_ANALOGUE_GAIN, gain_code);\n-\t\tctrls.set(V4L2_CID_EXPOSURE, exposure_lines);\n-\n-\t\tIPAOperationData op;\n-\t\top.operation = RPI_IPA_ACTION_V4L2_SET_STAGGERED;\n-\t\top.controls.push_back(ctrls);\n-\t\tqueueFrameAction.emit(0, op);\n+\t\t/* Supply initial values for gain and exposure. */\n+\t\tagcStatus.shutter_time = DEFAULT_EXPOSURE_TIME;\n+\t\tagcStatus.analogue_gain = DEFAULT_ANALOGUE_GAIN;\n \t}\n \n-\tcontroller_.SwitchMode(mode_);\n+\tRPi::Metadata metadata;\n+\tcontroller_.SwitchMode(mode_, &metadata);\n+\n+\t/* SwitchMode may supply updated exposure/gain values to use. */\n+\tmetadata.Get(\"agc.status\", agcStatus);\n+\tif (agcStatus.shutter_time != 0.0 && agcStatus.analogue_gain != 0.0)\n+\t\tapplyAGC(&agcStatus);\n \n \tlastMode_ = mode_;\n }\n","prefixes":["libcamera-devel"]}