Patch Detail
Show a patch.
GET /api/1.1/patches/10662/?format=api
{ "id": 10662, "url": "https://patchwork.libcamera.org/api/1.1/patches/10662/?format=api", "web_url": "https://patchwork.libcamera.org/patch/10662/", "project": { "id": 1, "url": "https://patchwork.libcamera.org/api/1.1/projects/1/?format=api", "name": "libcamera", "link_name": "libcamera", "list_id": "libcamera_core", "list_email": "libcamera-devel@lists.libcamera.org", "web_url": "", "scm_url": "", "webscm_url": "" }, "msgid": "<20201215004811.602429-2-niklas.soderlund@ragnatech.se>", "date": "2020-12-15T00:48:04", "name": "[libcamera-devel,v4,1/8] libcamera: delayed_controls: Add helper for controls that applies with a delay", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": false, "hash": "6f310997fdc9b433b98da2bae1cdd52a82b6e375", "submitter": { "id": 5, "url": "https://patchwork.libcamera.org/api/1.1/people/5/?format=api", "name": "Niklas Söderlund", "email": "niklas.soderlund@ragnatech.se" }, "delegate": { "id": 16, "url": "https://patchwork.libcamera.org/api/1.1/users/16/?format=api", "username": "neg", "first_name": "Niklas", "last_name": "Söderlund", "email": "niklas.soderlund@ragnatech.se" }, "mbox": "https://patchwork.libcamera.org/patch/10662/mbox/", "series": [ { "id": 1533, "url": "https://patchwork.libcamera.org/api/1.1/series/1533/?format=api", "web_url": "https://patchwork.libcamera.org/project/libcamera/list/?series=1533", "date": "2020-12-15T00:48:03", "name": "libcamera: Add helper for controls that take effect with a delay", "version": 4, "mbox": "https://patchwork.libcamera.org/series/1533/mbox/" } ], "comments": "https://patchwork.libcamera.org/api/patches/10662/comments/", "check": "pending", "checks": "https://patchwork.libcamera.org/api/patches/10662/checks/", "tags": {}, "headers": { "Return-Path": "<libcamera-devel-bounces@lists.libcamera.org>", "X-Original-To": "parsemail@patchwork.libcamera.org", "Delivered-To": "parsemail@patchwork.libcamera.org", "Received": [ "from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 15C74BD80A\n\tfor <parsemail@patchwork.libcamera.org>;\n\tTue, 15 Dec 2020 00:48:23 +0000 (UTC)", "from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id E298361594;\n\tTue, 15 Dec 2020 01:48:22 +0100 (CET)", "from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net\n\t[195.74.38.228])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id D9D8161590\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tTue, 15 Dec 2020 01:48:20 +0100 (CET)", "from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de\n\t[79.202.36.88])\n\tby bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA\n\tid 397d6a07-3e6f-11eb-a076-005056917f90;\n\tTue, 15 Dec 2020 01:48:19 +0100 (CET)" ], "X-Halon-ID": "397d6a07-3e6f-11eb-a076-005056917f90", "Authorized-sender": "niklas.soderlund@fsdn.se", "From": "=?utf-8?q?Niklas_S=C3=B6derlund?= <niklas.soderlund@ragnatech.se>", "To": "libcamera-devel@lists.libcamera.org,\n\tnaush@raspberrypi.com", "Date": "Tue, 15 Dec 2020 01:48:04 +0100", "Message-Id": "<20201215004811.602429-2-niklas.soderlund@ragnatech.se>", "X-Mailer": "git-send-email 2.29.2", "In-Reply-To": "<20201215004811.602429-1-niklas.soderlund@ragnatech.se>", "References": "<20201215004811.602429-1-niklas.soderlund@ragnatech.se>", "MIME-Version": "1.0", "Subject": "[libcamera-devel] [PATCH v4 1/8] libcamera: delayed_controls: Add\n\thelper for controls that applies with a delay", "X-BeenThere": "libcamera-devel@lists.libcamera.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "<libcamera-devel.lists.libcamera.org>", "List-Unsubscribe": "<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>", "List-Archive": "<https://lists.libcamera.org/pipermail/libcamera-devel/>", "List-Post": "<mailto:libcamera-devel@lists.libcamera.org>", "List-Help": "<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>", "List-Subscribe": "<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>", "Content-Type": "text/plain; charset=\"utf-8\"", "Content-Transfer-Encoding": "base64", "Errors-To": "libcamera-devel-bounces@lists.libcamera.org", "Sender": "\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>" }, "content": "Some sensor controls take effect with a delay as the sensor needs time\nto adjust, for example exposure. Add an optional helper DelayedControls\nto help pipelines deal with such controls.\n\nThe idea is to provide a queue of controls towards the V4L2 device and\napply individual controls with the specified delay with the aim to get\npredictable and retrievable control values for any given frame. To do\nthis the queue of controls needs to be at least as deep as the control\nwith the largest delay.\n\nThe DelayedControls needs to be informed of every start of exposure.\nThis can be emulated but the helper is designed to be used with this\nevent being provide by the kernel thru V4L2 events.\n\nThis helper is based on StaggeredCtrl from the Raspberry Pi pipeline\nhandler but expands on its API. This helpers aims to replace the\nRaspberry Pi implementations and mimics it behavior perfectly.\n\nSigned-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>\n---\n* Changes since v2\n- Drop optional argument to reset().\n- Update commit message.\n- Remove usage of Mutex.\n- Rename frameStart() to applyControls)_.\n- Rename ControlInfo to Into.\n- Rename ControlArray to ControlRingBuffer.\n- Drop ControlsDelays and ControlsValues.\n- Sort headers.\n- Rename iterators.\n- Simplify queueCount_ handeling in reset().\n- Add more warnings.\n- Update documentation.\n\n* Changes since v2\n- Improve error logic in queue() as suggested by Jean-Michel Hautbois.\n- s/fistSequence_/firstSequence_/\n\n* Changes since v1\n- Correct copyright to reflect work is derived from Raspberry Pi\n pipeline handler. This was always the intention and was wrong in v1.\n- Rewrite large parts of the documentation.\n- Join two loops to one in DelayedControls::DelayedControls()\n---\n include/libcamera/internal/delayed_controls.h | 82 ++++++\n src/libcamera/delayed_controls.cpp | 252 ++++++++++++++++++\n src/libcamera/meson.build | 1 +\n 3 files changed, 335 insertions(+)\n create mode 100644 include/libcamera/internal/delayed_controls.h\n create mode 100644 src/libcamera/delayed_controls.cpp", "diff": "diff --git a/include/libcamera/internal/delayed_controls.h b/include/libcamera/internal/delayed_controls.h\nnew file mode 100644\nindex 0000000000000000..1292b484ec9f53e9\n--- /dev/null\n+++ b/include/libcamera/internal/delayed_controls.h\n@@ -0,0 +1,82 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2020, Raspberry Pi (Trading) Ltd.\n+ *\n+ * delayed_controls.h - Helper to deal with controls that are applied with a delay\n+ */\n+#ifndef __LIBCAMERA_INTERNAL_DELAYED_CONTROLS_H__\n+#define __LIBCAMERA_INTERNAL_DELAYED_CONTROLS_H__\n+\n+#include <stdint.h>\n+#include <unordered_map>\n+\n+#include <libcamera/controls.h>\n+\n+namespace libcamera {\n+\n+class V4L2Device;\n+\n+class DelayedControls\n+{\n+public:\n+\tDelayedControls(V4L2Device *device,\n+\t\t\tconst std::unordered_map<uint32_t, unsigned int> &delays);\n+\n+\tvoid reset();\n+\n+\tbool push(const ControlList &controls);\n+\tControlList get(uint32_t sequence);\n+\n+\tvoid applyControls(uint32_t sequence);\n+\n+private:\n+\tclass Info\n+\t{\n+\tpublic:\n+\t\tInfo()\n+\t\t\t: updated(false)\n+\t\t{\n+\t\t}\n+\n+\t\tInfo(const ControlValue &v)\n+\t\t\t: value(v), updated(true)\n+\t\t{\n+\t\t}\n+\n+\t\tControlValue value;\n+\t\tbool updated;\n+\t};\n+\n+\t/* \\todo: Make the listSize configurable at instance creation time. */\n+\tstatic constexpr int listSize = 16;\n+\tclass ControlRingBuffer : public std::array<Info, listSize>\n+\t{\n+\tpublic:\n+\t\tInfo &operator[](unsigned int index)\n+\t\t{\n+\t\t\treturn std::array<Info, listSize>::operator[](index % listSize);\n+\t\t}\n+\n+\t\tconst Info &operator[](unsigned int index) const\n+\t\t{\n+\t\t\treturn std::array<Info, listSize>::operator[](index % listSize);\n+\t\t}\n+\t};\n+\n+\tbool queue(const ControlList &controls);\n+\n+\tV4L2Device *device_;\n+\tstd::unordered_map<const ControlId *, unsigned int> delays_;\n+\tunsigned int maxDelay_;\n+\n+\tbool running_;\n+\tuint32_t firstSequence_;\n+\n+\tuint32_t queueCount_;\n+\tuint32_t writeCount_;\n+\tstd::unordered_map<const ControlId *, ControlRingBuffer> values_;\n+};\n+\n+} /* namespace libcamera */\n+\n+#endif /* __LIBCAMERA_INTERNAL_DELAYED_CONTROLS_H__ */\ndiff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp\nnew file mode 100644\nindex 0000000000000000..db2e51f8c93c4755\n--- /dev/null\n+++ b/src/libcamera/delayed_controls.cpp\n@@ -0,0 +1,252 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2020, Raspberry Pi (Trading) Ltd.\n+ *\n+ * delayed_controls.h - Helper to deal with controls that are applied with a delay\n+ */\n+\n+#include \"libcamera/internal/delayed_controls.h\"\n+\n+#include <libcamera/controls.h>\n+\n+#include \"libcamera/internal/log.h\"\n+#include \"libcamera/internal/v4l2_device.h\"\n+\n+/**\n+ * \\file delayed_controls.h\n+ * \\brief Helper to deal with controls that are applied with a delay\n+ */\n+\n+namespace libcamera {\n+\n+LOG_DEFINE_CATEGORY(DelayedControls)\n+\n+/**\n+ * \\class DelayedControls\n+ * \\brief Helper to deal with controls that take effect with a delay\n+ *\n+ * Some sensor controls take effect with a delay as the sensor needs time to\n+ * adjust, for example exposure and focus. This is a helper class to deal with\n+ * such controls and the intended users are pipeline handlers.\n+ *\n+ * The idea is to extend the concept of the buffer depth of a pipeline the\n+ * application needs to maintain to also cover controls. Just as with buffer\n+ * depth if the application keeps the number of requests queued above the\n+ * control depth the controls are guaranteed to take effect for the correct\n+ * request. The control depth is determined by the control with the greatest\n+ * delay.\n+ */\n+\n+/**\n+ * \\brief Construct a DelayedControls instance\n+ * \\param[in] device The V4L2 device the controls have to be applied to\n+ * \\param[in] delays Map of the numerical V4L2 control ids to their associated\n+ * delays (in frames)\n+ *\n+ * Only controls specified in \\a delays are handled. If it's desired to mix\n+ * delayed controls and controls that take effect immediately the immediate\n+ * controls must be listed in the \\a delays map with a delay value of 0.\n+ */\n+DelayedControls::DelayedControls(V4L2Device *device,\n+\t\t\t\t const std::unordered_map<uint32_t, unsigned int> &delays)\n+\t: device_(device), maxDelay_(0)\n+{\n+\tconst ControlInfoMap &controls = device_->controls();\n+\n+\t/*\n+\t * Create a map of control ids to delays for controls exposed by the\n+\t * device.\n+\t */\n+\tfor (auto const &delay : delays) {\n+\t\tauto it = controls.find(delay.first);\n+\t\tif (it == controls.end()) {\n+\t\t\tLOG(DelayedControls, Error)\n+\t\t\t\t<< \"Delay request for control id \"\n+\t\t\t\t<< utils::hex(delay.first)\n+\t\t\t\t<< \" but control is not exposed by device \"\n+\t\t\t\t<< device_->deviceNode();\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tconst ControlId *id = it->first;\n+\n+\t\tdelays_[id] = delay.second;\n+\n+\t\tLOG(DelayedControls, Debug)\n+\t\t\t<< \"Set a delay of \" << delays_[id]\n+\t\t\t<< \" for \" << id->name();\n+\n+\t\tmaxDelay_ = std::max(maxDelay_, delays_[id]);\n+\t}\n+\n+\treset();\n+}\n+\n+/**\n+ * \\brief Reset state machine\n+ *\n+ * Resets the state machine to a starting position based on control values\n+ * retrieved from the device.\n+ */\n+void DelayedControls::reset()\n+{\n+\trunning_ = false;\n+\tfirstSequence_ = 0;\n+\tqueueCount_ = 1;\n+\twriteCount_ = 0;\n+\n+\t/* Retrieve control as reported by the device. */\n+\tstd::vector<uint32_t> ids;\n+\tfor (auto const &delay : delays_)\n+\t\tids.push_back(delay.first->id());\n+\n+\tControlList controls = device_->getControls(ids);\n+\n+\t/* Seed the control queue with the controls reported by the device. */\n+\tvalues_.clear();\n+\tfor (const auto &ctrl : controls) {\n+\t\tconst ControlId *id = device_->controls().idmap().at(ctrl.first);\n+\t\tvalues_[id][0] = Info(ctrl.second);\n+\t}\n+}\n+\n+/**\n+ * \\brief Push a set of controls on the queue\n+ * \\param[in] controls List of controls to add to the device queue\n+ *\n+ * Push a set of controls to the control queue. This increases the control queue\n+ * depth by one.\n+ *\n+ * \\returns true if \\a controls are accepted, or false otherwise\n+ */\n+bool DelayedControls::push(const ControlList &controls)\n+{\n+\treturn queue(controls);\n+}\n+\n+bool DelayedControls::queue(const ControlList &controls)\n+{\n+\t/* Copy state from previous frame. */\n+\tfor (auto &ctrl : values_) {\n+\t\tInfo &info = ctrl.second[queueCount_];\n+\t\tinfo.value = values_[ctrl.first][queueCount_ - 1].value;\n+\t\tinfo.updated = false;\n+\t}\n+\n+\t/* Update with new controls. */\n+\tconst ControlIdMap &idmap = device_->controls().idmap();\n+\tfor (const auto &control : controls) {\n+\t\tconst auto &it = idmap.find(control.first);\n+\t\tif (it == idmap.end()) {\n+\t\t\tLOG(DelayedControls, Warning)\n+\t\t\t\t<< \"Unknown control \" << control.first;\n+\t\t\treturn false;\n+\t\t}\n+\n+\t\tconst ControlId *id = it->second;\n+\n+\t\tif (delays_.find(id) == delays_.end())\n+\t\t\treturn false;\n+\n+\t\tInfo &info = values_[id][queueCount_];\n+\n+\t\tinfo.value = control.second;\n+\t\tinfo.updated = true;\n+\n+\t\tLOG(DelayedControls, Debug)\n+\t\t\t<< \"Queuing \" << id->name()\n+\t\t\t<< \" to \" << info.value.toString()\n+\t\t\t<< \" at index \" << queueCount_;\n+\t}\n+\n+\tqueueCount_++;\n+\n+\treturn true;\n+}\n+\n+/**\n+ * \\brief Read back controls in effect at a sequence number\n+ * \\param[in] sequence The sequence number to get controls for\n+ *\n+ * Read back what controls where in effect at a specific sequence number. The\n+ * history is a ring buffer of 16 entries where new and old values coexist. It's\n+ * the callers responsibility to not read too old sequence numbers that have been\n+ * pushed out of the history.\n+ *\n+ * Historic values are evicted by pushing new values onto the queue using\n+ * push(). The max history from the current sequence number that yields valid\n+ * values are thus 16 minus number of controls pushed.\n+ *\n+ * \\return The controls at \\a sequence number\n+ */\n+ControlList DelayedControls::get(uint32_t sequence)\n+{\n+\tuint32_t adjustedSeq = sequence - firstSequence_ + 1;\n+\tunsigned int index = std::max<int>(0, adjustedSeq - maxDelay_);\n+\n+\tControlList out(device_->controls());\n+\tfor (const auto &ctrl : values_) {\n+\t\tconst ControlId *id = ctrl.first;\n+\t\tconst Info &info = ctrl.second[index];\n+\n+\t\tout.set(id->id(), info.value);\n+\n+\t\tLOG(DelayedControls, Debug)\n+\t\t\t<< \"Reading \" << id->name()\n+\t\t\t<< \" to \" << info.value.toString()\n+\t\t\t<< \" at index \" << index;\n+\t}\n+\n+\treturn out;\n+}\n+\n+/**\n+ * \\brief Inform DelayedControls of the start of a new frame\n+ * \\param[in] sequence Sequence number of the frame that started\n+ *\n+ * Inform the state machine that a new frame has started and of its sequence\n+ * number. Any user of these helpers is responsible to inform the helper about\n+ * the start of any frame.This can be connected with ease to the start of a\n+ * exposure (SOE) V4L2 event.\n+ */\n+void DelayedControls::applyControls(uint32_t sequence)\n+{\n+\tLOG(DelayedControls, Debug) << \"frame \" << sequence << \" started\";\n+\n+\tif (!running_) {\n+\t\tfirstSequence_ = sequence;\n+\t\trunning_ = true;\n+\t}\n+\n+\t/*\n+\t * Create control list peaking ahead in the value queue to ensure\n+\t * values are set in time to satisfy the sensor delay.\n+\t */\n+\tControlList out(device_->controls());\n+\tfor (const auto &ctrl : values_) {\n+\t\tconst ControlId *id = ctrl.first;\n+\t\tunsigned int delayDiff = maxDelay_ - delays_[id];\n+\t\tunsigned int index = std::max<int>(0, writeCount_ - delayDiff);\n+\t\tconst Info &info = ctrl.second[index];\n+\n+\t\tif (info.updated) {\n+\t\t\tout.set(id->id(), info.value);\n+\t\t\tLOG(DelayedControls, Debug)\n+\t\t\t\t<< \"Setting \" << id->name()\n+\t\t\t\t<< \" to \" << info.value.toString()\n+\t\t\t\t<< \" at index \" << index;\n+\t\t}\n+\t}\n+\n+\twriteCount_++;\n+\n+\twhile (writeCount_ >= queueCount_) {\n+\t\tLOG(DelayedControls, Debug)\n+\t\t\t<< \"Queue is empty, auto queue no-op.\";\n+\t\tqueue({});\n+\t}\n+\n+\tdevice_->setControls(&out);\n+}\n+\n+} /* namespace libcamera */\ndiff --git a/src/libcamera/meson.build b/src/libcamera/meson.build\nindex 387d5d88ecae11ad..5a4bf0d7ba4fd231 100644\n--- a/src/libcamera/meson.build\n+++ b/src/libcamera/meson.build\n@@ -12,6 +12,7 @@ libcamera_sources = files([\n 'controls.cpp',\n 'control_serializer.cpp',\n 'control_validator.cpp',\n+ 'delayed_controls.cpp',\n 'device_enumerator.cpp',\n 'device_enumerator_sysfs.cpp',\n 'event_dispatcher.cpp',\n", "prefixes": [ "libcamera-devel", "v4", "1/8" ] }