[{"id":14986,"web_url":"https://patchwork.libcamera.org/comment/14986/","msgid":"","date":"2021-02-04T20:22:01","subject":"Re: [libcamera-devel] [PATCH v4 5/5] pipeline: raspberrypi: Add\n\tnotion of priority write to StaggeredCtrl","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Naush,\n\nThank you for the patch.\n\nOn Fri, Jan 29, 2021 at 11:16:16AM +0000, Naushir Patuck wrote:\n> If an exposure time change adjusts the vblanking limits, and we write\n> both VBLANK and EXPOSURE controls as a set, the latter may fail if the\n> value is outside of the limits calculated by the old VBLANK value. This\n> is a limitation in V4L2 and cannot be fixed by writing VBLANK before\n> EXPOSURE.\n> \n> The workaround here is to have the StaggeredCtrl mark the\n> VBLANK control as \"priority write\", which then write VBLANK separately\n> from (and ahead of) any other controls. This way, the sensor driver will\n> update the EXPOSURE control with new limits before the new values is\n> presented, and will thus be seen as valid.\n> \n> In addition to this, the following changes have also been made to\n> the module:\n> \n> - The initializer list passed into init() now uses a structure type\n> instead of a std::pair.\n> - Use unsigned int to store control delays to avoid unnecessary casts.\n> \n> StaggeredCtrl is due to be deprecated and replaced by DelayedCtrl, so\n> this change serves more a working proof-of-concept on the workaround,\n> and not much care has been taken to provide a nice new API for applying\n> this \"priority write\" flag to the control. A similar workaround must be\n> available to DelayedCtrl eventually.\n> \n> Signed-off-by: Naushir Patuck \n> Reviewed-by: Jacopo Mondi \n\nI'll drop this patch for now, as the StaggeredCtrl implementation has\nbeen replaced.\n\n> ---\n> src/ipa/raspberrypi/raspberrypi.cpp | 5 ++-\n> .../pipeline/raspberrypi/raspberrypi.cpp | 12 ++++--\n> .../pipeline/raspberrypi/staggered_ctrl.cpp | 41 +++++++++++++------\n> .../pipeline/raspberrypi/staggered_ctrl.h | 17 ++++++--\n> 4 files changed, 54 insertions(+), 21 deletions(-)\n> \n> diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp\n> index 8c0e699184f6..2ad7b7dabb3e 100644\n> --- a/src/ipa/raspberrypi/raspberrypi.cpp\n> +++ b/src/ipa/raspberrypi/raspberrypi.cpp\n> @@ -1038,8 +1038,9 @@ void IPARPi::applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls)\n> \n> \t/*\n> \t * Due to the behavior of V4L2, the current value of VBLANK could clip the\n> -\t * exposure time without us knowing. The next time though this function should\n> -\t * clip exposure correctly.\n> +\t * exposure time without us knowing. We get around this by ensuring the\n> +\t * staggered write component submits VBLANK separately from, and before the\n> +\t * EXPOSURE control.\n> \t */\n> \tctrls.set(V4L2_CID_VBLANK, vblanking);\n> \tctrls.set(V4L2_CID_EXPOSURE, exposureLines);\n> diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n> index 524cc960dd37..2118f2e72486 100644\n> --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n> +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n> @@ -1229,12 +1229,18 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config)\n> \t\t/*\n> \t\t * Setup our staggered control writer with the sensor default\n> \t\t * gain and exposure delays.\n> +\t\t *\n> +\t\t * VBLANK must be flagged as \"priority write\" to allow it to\n> +\t\t * be set ahead of (and separate from) all other controls that\n> +\t\t * are batched together. This is needed so that any update to the\n> +\t\t * EXPOSURE control will be validated based on the new VBLANK\n> +\t\t * control value.\n> \t\t */\n> \t\tif (!staggeredCtrl_) {\n> \t\t\tstaggeredCtrl_.init(unicam_[Unicam::Image].dev(),\n> -\t\t\t\t\t { { V4L2_CID_ANALOGUE_GAIN, result.data[resultIdx++] },\n> -\t\t\t\t\t { V4L2_CID_EXPOSURE, result.data[resultIdx++] },\n> -\t\t\t\t\t { V4L2_CID_VBLANK, result.data[resultIdx++] } });\n> +\t\t\t\t\t { { V4L2_CID_ANALOGUE_GAIN, result.data[resultIdx++], false },\n> +\t\t\t\t\t { V4L2_CID_EXPOSURE, result.data[resultIdx++], false },\n> +\t\t\t\t\t { V4L2_CID_VBLANK, result.data[resultIdx++], true } });\n> \t\t\tsensorMetadata_ = result.data[resultIdx++];\n> \t\t}\n> \t}\n> diff --git a/src/libcamera/pipeline/raspberrypi/staggered_ctrl.cpp b/src/libcamera/pipeline/raspberrypi/staggered_ctrl.cpp\n> index 62605c0fceee..498cd65b4cb6 100644\n> --- a/src/libcamera/pipeline/raspberrypi/staggered_ctrl.cpp\n> +++ b/src/libcamera/pipeline/raspberrypi/staggered_ctrl.cpp\n> @@ -22,21 +22,23 @@ LOG_DEFINE_CATEGORY(RPI_S_W)\n> namespace RPi {\n> \n> void StaggeredCtrl::init(V4L2VideoDevice *dev,\n> -\t std::initializer_list> delayList)\n> +\t\t\t std::initializer_list ctrlList)\n> {\n> \tstd::lock_guard lock(lock_);\n> \n> \tdev_ = dev;\n> -\tdelay_ = delayList;\n> +\tctrlParams_.clear();\n> \tctrl_.clear();\n> \n> -\t/* Find the largest delay across all controls. */\n> \tmaxDelay_ = 0;\n> -\tfor (auto const &p : delay_) {\n> +\tfor (auto const &c : ctrlList) {\n> \t\tLOG(RPI_S_W, Info) << \"Init ctrl \"\n> -\t\t\t\t << utils::hex(p.first) << \" with delay \"\n> -\t\t\t\t << static_cast(p.second);\n> -\t\tmaxDelay_ = std::max(maxDelay_, p.second);\n> +\t\t\t\t << utils::hex(c.id) << \" with delay \"\n> +\t\t\t\t << static_cast(c.delay);\n> +\n> +\t\tctrlParams_[c.id] = { c.delay, c.priorityWrite };\n> +\t\t/* Find the largest delay across all controls. */\n> +\t\tmaxDelay_ = std::max(maxDelay_, c.delay);\n> \t}\n> \n> \tinit_ = true;\n> @@ -67,7 +69,7 @@ bool StaggeredCtrl::set(uint32_t ctrl, int32_t value)\n> \tstd::lock_guard lock(lock_);\n> \n> \t/* Can we find this ctrl as one that is registered? */\n> -\tif (delay_.find(ctrl) == delay_.end())\n> +\tif (ctrlParams_.find(ctrl) == ctrlParams_.end())\n> \t\treturn false;\n> \n> \tctrl_[ctrl][setCount_].value = value;\n> @@ -82,7 +84,7 @@ bool StaggeredCtrl::set(std::initializer_list\n> \n> \tfor (auto const &p : ctrlList) {\n> \t\t/* Can we find this ctrl? */\n> -\t\tif (delay_.find(p.first) == delay_.end())\n> +\t\tif (ctrlParams_.find(p.first) == ctrlParams_.end())\n> \t\t\treturn false;\n> \n> \t\tctrl_[p.first][setCount_] = CtrlInfo(p.second);\n> @@ -97,7 +99,7 @@ bool StaggeredCtrl::set(const ControlList &controls)\n> \n> \tfor (auto const &p : controls) {\n> \t\t/* Can we find this ctrl? */\n> -\t\tif (delay_.find(p.first) == delay_.end())\n> +\t\tif (ctrlParams_.find(p.first) == ctrlParams_.end())\n> \t\t\treturn false;\n> \n> \t\tctrl_[p.first][setCount_] = CtrlInfo(p.second.get());\n> @@ -117,12 +119,25 @@ int StaggeredCtrl::write()\n> \tControlList controls(dev_->controls());\n> \n> \tfor (auto &p : ctrl_) {\n> -\t\tint delayDiff = maxDelay_ - delay_[p.first];\n> +\t\tint delayDiff = maxDelay_ - ctrlParams_[p.first].delay;\n> \t\tint index = std::max(0, setCount_ - delayDiff);\n> \n> \t\tif (p.second[index].updated) {\n> -\t\t\t/* We need to write this value out. */\n> -\t\t\tcontrols.set(p.first, p.second[index].value);\n> +\t\t\tif (ctrlParams_[p.first].priorityWrite) {\n> +\t\t\t\t/*\n> +\t\t\t\t * This control must be written now, it could\n> +\t\t\t\t * affect validity of the other controls.\n> +\t\t\t\t */\n> +\t\t\t\tControlList immediate(dev_->controls());\n> +\t\t\t\timmediate.set(p.first, p.second[index].value);\n> +\t\t\t\tdev_->setControls(&immediate);\n> +\t\t\t} else {\n> +\t\t\t\t/*\n> +\t\t\t\t * Batch up the list of controls and write them\n> +\t\t\t\t * at the end of the function.\n> +\t\t\t\t */\n> +\t\t\t\tcontrols.set(p.first, p.second[index].value);\n> +\t\t\t}\n> \t\t\tp.second[index].updated = false;\n> \t\t\tLOG(RPI_S_W, Debug) << \"Writing ctrl \"\n> \t\t\t\t\t << utils::hex(p.first) << \" to \"\n> diff --git a/src/libcamera/pipeline/raspberrypi/staggered_ctrl.h b/src/libcamera/pipeline/raspberrypi/staggered_ctrl.h\n> index 382fa31a6853..637629c0d9a8 100644\n> --- a/src/libcamera/pipeline/raspberrypi/staggered_ctrl.h\n> +++ b/src/libcamera/pipeline/raspberrypi/staggered_ctrl.h\n> @@ -23,6 +23,12 @@ namespace RPi {\n> class StaggeredCtrl\n> {\n> public:\n> +\tstruct CtrlInitParams {\n> +\t\tunsigned int id;\n> +\t\tunsigned int delay;\n> +\t\tbool priorityWrite;\n> +\t};\n> +\n> \tStaggeredCtrl()\n> \t\t: init_(false), setCount_(0), getCount_(0), maxDelay_(0)\n> \t{\n> @@ -34,7 +40,7 @@ public:\n> \t}\n> \n> \tvoid init(V4L2VideoDevice *dev,\n> -\t\t std::initializer_list> delayList);\n> +\t\t std::initializer_list ctrlList);\n> \tvoid reset();\n> \n> \tvoid get(std::unordered_map &ctrl, uint8_t offset = 0);\n> @@ -79,12 +85,17 @@ private:\n> \t\t}\n> \t};\n> \n> +\tstruct CtrlParams {\n> +\t\tunsigned int delay;\n> +\t\tbool priorityWrite;\n> +\t};\n> +\n> \tbool init_;\n> \tuint32_t setCount_;\n> \tuint32_t getCount_;\n> -\tuint8_t maxDelay_;\n> +\tunsigned int maxDelay_;\n> \tV4L2VideoDevice *dev_;\n> -\tstd::unordered_map delay_;\n> +\tstd::unordered_map ctrlParams_;\n> \tstd::unordered_map ctrl_;\n> \tstd::mutex lock_;\n> };","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 190C7BD160\n\tfor ;\n\tThu, 4 Feb 2021 20:22:26 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9FADA61430;\n\tThu, 4 Feb 2021 21:22:25 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B512161430\n\tfor ;\n\tThu, 4 Feb 2021 21:22:24 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 3429C45D;\n\tThu, 4 Feb 2021 21:22:24 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"d+ioN8bX\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1612470144;\n\tbh=KM6xJVfCysNi5dmutrlV16HdHc1D2UbyqaXBADeJqbQ=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=d+ioN8bXCMeW01uQZiYMkm9u/4G1LYqz+lsW1O2xOABdryoq8HFhH6X/8CcludD7j\n\t3x58EwlB67aavglwwgB/KQny1jG9VZpAkEGhbDSRQPKyhBlrCposuQWvAkMe9LyTOP\n\tUsdrjpQqW5NyzBQUzaXOuep9m+/HtfWsKl19w0zM=","Date":"Thu, 4 Feb 2021 22:22:01 +0200","From":"Laurent Pinchart ","To":"Naushir Patuck ","Message-ID":"","References":"<20210129111616.1047483-1-naush@raspberrypi.com>\n\t<20210129111616.1047483-6-naush@raspberrypi.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20210129111616.1047483-6-naush@raspberrypi.com>","Subject":"Re: [libcamera-devel] [PATCH v4 5/5] pipeline: raspberrypi: Add\n\tnotion of priority write to StaggeredCtrl","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]