From patchwork Mon Nov 23 22:12:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10481 X-Patchwork-Delegate: niklas.soderlund@ragnatech.se Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6F570BE176 for ; Mon, 23 Nov 2020 22:12:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3BB4B633FA; Mon, 23 Nov 2020 23:12:49 +0100 (CET) Received: from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net [195.74.38.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6C467631D0 for ; Mon, 23 Nov 2020 23:12:47 +0100 (CET) X-Halon-ID: 259d007e-2dd8-11eb-a076-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de [79.202.36.88]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 259d007e-2dd8-11eb-a076-005056917f90; Mon, 23 Nov 2020 23:06:33 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com Date: Mon, 23 Nov 2020 23:12:27 +0100 Message-Id: <20201123221234.485933-2-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> References: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/8] libcamera: delayed_controls: Add helper for controls that applies with a delay X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Some sensor controls take effect with a delay as the sensor needs time to adjust, for example exposure. Add a optional helper DelayedControls to help pipelines deal with such controls. The idea is to provide a queue of controls towards the V4L2 device and apply individual controls with the specified delay with the aim to get predictable and retrievable control values for any given frame. To do this the queue of controls needs to be at least as deep as the control with the largest delay. The DelayedControls needs to be informed of every start of exposure. This can be emulated but the helper is designed to be used with this event being provide by the kernel thru V4L2 events. This helper is based on StaggeredCtrl from the Raspberry Pi pipeline handler but expands on its API. This helpers aims to replace the Raspberry Pi implementations and mimics it behavior perfectly. Signed-off-by: Niklas Söderlund --- * Changes since v2 - Improve error logic in queue() as suggested by Jean-Michel Hautbois. - s/fistSequence_/firstSequence_/ * Changes since v1 - Correct copyright to reflect work is derived from Raspberry Pi pipeline handler. This was always the intention and was wrong in v1. - Rewrite large parts of the documentation. - Join two loops to one in DelayedControls::DelayedControls() --- include/libcamera/internal/delayed_controls.h | 87 ++++++ src/libcamera/delayed_controls.cpp | 265 ++++++++++++++++++ src/libcamera/meson.build | 1 + 3 files changed, 353 insertions(+) create mode 100644 include/libcamera/internal/delayed_controls.h create mode 100644 src/libcamera/delayed_controls.cpp diff --git a/include/libcamera/internal/delayed_controls.h b/include/libcamera/internal/delayed_controls.h new file mode 100644 index 0000000000000000..e9cb0b90aac99847 --- /dev/null +++ b/include/libcamera/internal/delayed_controls.h @@ -0,0 +1,87 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi (Trading) Ltd. + * + * delayed_controls.h - Helper to deal with controls that are applied with a delay + */ +#ifndef __LIBCAMERA_INTERNAL_DELAYED_CONTROLS_H__ +#define __LIBCAMERA_INTERNAL_DELAYED_CONTROLS_H__ + +#include +#include +#include + +#include + +namespace libcamera { + +class V4L2Device; + +class DelayedControls +{ +public: + DelayedControls(V4L2Device *device, + const std::unordered_map &delays); + + void reset(ControlList *controls = nullptr); + + bool push(const ControlList &controls); + ControlList get(uint32_t sequence); + + void frameStart(uint32_t sequence); + +private: + class ControlInfo + { + public: + ControlInfo() + : updated(false) + { + } + + ControlInfo(const ControlValue &v) + : value(v), updated(true) + { + } + + ControlValue value; + bool updated; + }; + + static constexpr int listSize = 16; + class ControlArray : public std::array + { + public: + ControlInfo &operator[](unsigned int index) + { + return std::array::operator[](index % listSize); + } + + const ControlInfo &operator[](unsigned int index) const + { + return std::array::operator[](index % listSize); + } + }; + + using ControlsDelays = std::unordered_map; + using ControlsValues = std::unordered_map; + + bool queue(const ControlList &controls); + + std::mutex lock_; + + V4L2Device *device_; + ControlsDelays delays_; + unsigned int maxDelay_; + + bool running_; + uint32_t firstSequence_; + + uint32_t queueCount_; + uint32_t writeCount_; + ControlsValues ctrls_; +}; + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_INTERNAL_DELAYED_CONTROLS_H__ */ diff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp new file mode 100644 index 0000000000000000..c810be48a36be515 --- /dev/null +++ b/src/libcamera/delayed_controls.cpp @@ -0,0 +1,265 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi (Trading) Ltd. + * + * delayed_controls.h - Helper to deal with controls that are applied with a delay + */ + +#include "libcamera/internal/delayed_controls.h" +#include "libcamera/internal/v4l2_device.h" + +#include + +#include "libcamera/internal/log.h" + +/** + * \file delayed_controls.h + * \brief Helper to deal with controls that are applied with a delay + */ + +namespace libcamera { + +LOG_DEFINE_CATEGORY(DelayedControls) + +/** + * \class DelayedControls + * \brief Helper to deal with controls that take effect with a delay + * + * Some sensor controls take effect with a delay as the sensor needs time to + * adjust, for example exposure and focus. This is an optional helper class to + * deal with such controls and the intended users are pipeline handlers. + * + * The idea is to extend the concept of the buffer depth of a pipeline the + * application need to maintain to also cover controls. Just as with buffer + * depth if the application keeps the number of requests queued above the + * control depth the controls are guaranteed to take effect for the correct + * request. The control depth is determined by the control with the grates + * delay. + */ + +/** + * \brief Construct a DelayedControls + * \param[in] device The V4L2 device the controls have to be applied to + * \param[in] delays Map of the numerical V4L2 control ids to their associated + * delays (in frames) + * + * Only controls specified in \a delays are handled. If it's desired to mix + * delayed controls and controls that take effect immediately the immediate + * controls must be listed in the \a delays map with a delay value of 0. + */ +DelayedControls::DelayedControls(V4L2Device *device, + const std::unordered_map &delays) + : device_(device), maxDelay_(0) +{ + const ControlInfoMap &controls = device_->controls(); + + /* + * Create a map of control ids to delays for controls exposed by the + * device. + */ + for (auto const &delay : delays) { + auto itControl = controls.find(delay.first); + if (itControl == controls.end()) { + LOG(DelayedControls, Error) + << "Delay request for control id " + << utils::hex(delay.first) + << " but control is not exposed by device " + << device_->deviceNode(); + continue; + } + + const ControlId *id = itControl->first; + + delays_[id] = delay.second; + + LOG(DelayedControls, Debug) + << "Set a delay of " << delays_[id] + << " for " << id->name(); + + maxDelay_ = std::max(maxDelay_, delays_[id]); + } + + reset(); +} + +/** + * \brief Reset state machine and controls + * \param[in] controls Optional list of controls to apply to the device + * + * Resets the state machine to a starting position based on control values + * retrieved from the device. Controls may optionally be set before they are + * read back using \a controls. + */ +void DelayedControls::reset(ControlList *controls) +{ + std::lock_guard lock(lock_); + + running_ = false; + firstSequence_ = 0; + queueCount_ = 0; + writeCount_ = 0; + + /* Set the controls on the device if requested. */ + if (controls) + device_->setControls(controls); + + /* Retrieve current control values reported by the device. */ + std::vector ids; + for (auto const &delay : delays_) + ids.push_back(delay.first->id()); + + ControlList devCtrls = device_->getControls(ids); + + /* Seed the control queue with the controls reported by the device. */ + ctrls_.clear(); + for (const auto &ctrl : devCtrls) { + const ControlId *id = devCtrls.infoMap()->idmap().at(ctrl.first); + ctrls_[id][queueCount_] = ControlInfo(ctrl.second); + } + + queueCount_++; +} + +/** + * \brief Push a set of controls on the queue + * \param[in] controls List of controls to add to the device queue + * + * Push a set of controls to the control queue. This increases the control queue + * depth by one. + * + * \returns true if \a controls are accepted, or false otherwise + */ +bool DelayedControls::push(const ControlList &controls) +{ + std::lock_guard lock(lock_); + + return queue(controls); +} + +bool DelayedControls::queue(const ControlList &controls) +{ + /* Copy state from previous frame. */ + for (auto &ctrl : ctrls_) { + ControlInfo &info = ctrl.second[queueCount_]; + info.value = ctrls_[ctrl.first][queueCount_ - 1].value; + info.updated = false; + } + + /* Update with new controls. */ + const ControlIdMap &idmap = device_->controls().idmap(); + for (const auto &control : controls) { + const auto &itCtrl = idmap.find(control.first); + if (itCtrl == idmap.end()) + return false; + + const ControlId *id = itCtrl->second; + + if (delays_.find(id) == delays_.end()) + return false; + + ControlInfo &info = ctrls_[id][queueCount_]; + + info.value = control.second; + info.updated = true; + + LOG(DelayedControls, Debug) + << "Queuing " << id->name() + << " to " << info.value.toString() + << " at index " << queueCount_; + } + + queueCount_++; + + return true; +} + +/** + * \brief Read back controls in effect at a sequence number + * \param[in] sequence The sequence number to get controls for + * + * Read back what controls where in effect at a specific sequence number. The + * history is a ring buffer of 16 entries where new and old values coexist. It's + * the callers responsibility to not read to old sequence numbers that have been + * pushed out of the history. + * + * Historic values are evicted by pushing new values onto the queue using + * push(). The max history from the current sequence number that yields valid + * values are thus 16 minus number of controls pushed. + * + * \return The controls at \a sequence number + */ +ControlList DelayedControls::get(uint32_t sequence) +{ + std::lock_guard lock(lock_); + + uint32_t adjustedSeq = sequence - firstSequence_ + 1; + unsigned int index = std::max(0, adjustedSeq - maxDelay_); + + ControlList out(device_->controls()); + for (const auto &ctrl : ctrls_) { + const ControlId *id = ctrl.first; + const ControlInfo &info = ctrl.second[index]; + + out.set(id->id(), info.value); + + LOG(DelayedControls, Debug) + << "Reading " << id->name() + << " to " << info.value.toString() + << " at index " << index; + } + + return out; +} + +/** + * \brief Inform DelayedControls of the start of a new frame + * \param[in] sequence Sequence number of the frame that started + * + * Inform the state machine that a new frame has started and of its sequence + * number. Any user of these helpers is responsible to inform the helper about + * the start of any frame.This can be connected with ease to the start of a + * exposure (SOE) V4L2 event. + */ +void DelayedControls::frameStart(uint32_t sequence) +{ + LOG(DelayedControls, Debug) << "frame " << sequence << " started"; + + std::lock_guard lock(lock_); + + if (!running_) { + firstSequence_ = sequence; + running_ = true; + } + + /* + * Create control list peaking ahead in the value queue to ensure + * values are set in time to satisfy the sensor delay. + */ + ControlList out(device_->controls()); + for (const auto &ctrl : ctrls_) { + const ControlId *id = ctrl.first; + unsigned int delayDiff = maxDelay_ - delays_[id]; + unsigned int index = std::max(0, writeCount_ - delayDiff); + const ControlInfo &info = ctrl.second[index]; + + if (info.updated) { + out.set(id->id(), info.value); + LOG(DelayedControls, Debug) + << "Setting " << id->name() + << " to " << info.value.toString() + << " at index " << index; + } + } + + writeCount_++; + + while (writeCount_ >= queueCount_) { + LOG(DelayedControls, Debug) + << "Queue is empty, auto queue no-op."; + queue({}); + } + + device_->setControls(&out); +} + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 387d5d88ecae11ad..5a4bf0d7ba4fd231 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -12,6 +12,7 @@ libcamera_sources = files([ 'controls.cpp', 'control_serializer.cpp', 'control_validator.cpp', + 'delayed_controls.cpp', 'device_enumerator.cpp', 'device_enumerator_sysfs.cpp', 'event_dispatcher.cpp', From patchwork Mon Nov 23 22:12:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10482 X-Patchwork-Delegate: niklas.soderlund@ragnatech.se Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ED67CBE176 for ; Mon, 23 Nov 2020 22:12:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AB2B7633FF; Mon, 23 Nov 2020 23:12:51 +0100 (CET) Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 61C76633F3 for ; Mon, 23 Nov 2020 23:12:48 +0100 (CET) X-Halon-ID: 265e7c3a-2dd8-11eb-a076-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de [79.202.36.88]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 265e7c3a-2dd8-11eb-a076-005056917f90; Mon, 23 Nov 2020 23:06:34 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com Date: Mon, 23 Nov 2020 23:12:28 +0100 Message-Id: <20201123221234.485933-3-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> References: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/8] test: delayed_controls: Add test case for DelayedControls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a test-case for DelayedControls that exercise the setting of controls and reading back what controls where used for a particular frame. Also exercise corner case such as a V4L2 devices that do not reset its sequence number to 0 at stream on and sequence number wrapping around the uint32_t value space. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi --- * Changes since v2 - Remove a unused LoC. - Add and remove blank lines. - Align number of loop iterations. --- test/delayed_contols.cpp | 300 +++++++++++++++++++++++++++++++++++++++ test/meson.build | 1 + 2 files changed, 301 insertions(+) create mode 100644 test/delayed_contols.cpp diff --git a/test/delayed_contols.cpp b/test/delayed_contols.cpp new file mode 100644 index 0000000000000000..aaf321bf4f1b4cd8 --- /dev/null +++ b/test/delayed_contols.cpp @@ -0,0 +1,300 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * libcamera delayed controls tests + */ + +#include + +#include "libcamera/internal/delayed_controls.h" +#include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/v4l2_videodevice.h" + +#include "test.h" + +using namespace std; +using namespace libcamera; + +class DelayedControlsTest : public Test +{ +public: + DelayedControlsTest() + : dev_(nullptr) + { + } + + int init() + { + enumerator_ = DeviceEnumerator::create(); + if (!enumerator_) { + cerr << "Failed to create device enumerator" << endl; + return TestFail; + } + + if (enumerator_->enumerate()) { + cerr << "Failed to enumerate media devices" << endl; + return TestFail; + } + + DeviceMatch dm("vivid"); + dm.add("vivid-000-vid-cap"); + + media_ = enumerator_->search(dm); + if (!media_) { + cerr << "vivid video device found" << endl; + return TestSkip; + } + + MediaEntity *entity = media_->getEntityByName("vivid-000-vid-cap"); + dev_ = new V4L2VideoDevice(entity->deviceNode()); + if (dev_->open()) { + cerr << "Failed to open video device" << endl; + return TestFail; + } + + const ControlInfoMap &infoMap = dev_->controls(); + + /* Test control enumeration. */ + if (infoMap.empty()) { + cerr << "Failed to enumerate controls" << endl; + return TestFail; + } + + if (infoMap.find(V4L2_CID_BRIGHTNESS) == infoMap.end() || + infoMap.find(V4L2_CID_CONTRAST) == infoMap.end()) { + cerr << "Missing controls" << endl; + return TestFail; + } + + return TestPass; + } + + int singleControlNoDelay() + { + std::unordered_map delays = { + { V4L2_CID_BRIGHTNESS, 0 }, + }; + std::unique_ptr delayed = + std::make_unique(dev_, delays); + ControlList ctrls; + + /* Reset control to value not used in test. */ + ctrls.set(V4L2_CID_BRIGHTNESS, 1); + delayed->reset(&ctrls); + + /* Test control without delay are set at once. */ + for (int32_t i = 0; i < 100; i++) { + int32_t value = 100 + i; + + ctrls.set(V4L2_CID_BRIGHTNESS, value); + delayed->push(ctrls); + + delayed->frameStart(i); + + ControlList result = delayed->get(i); + int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); + if (brightness != value) { + cerr << "Failed single control without delay" + << " frame " << i + << " expected " << value + << " got " << brightness + << endl; + return TestFail; + } + } + + return TestPass; + } + + int singleControlWithDelay() + { + std::unordered_map delays = { + { V4L2_CID_BRIGHTNESS, 1 }, + }; + std::unique_ptr delayed = + std::make_unique(dev_, delays); + ControlList ctrls; + + /* Reset control to value that will be first in test. */ + int32_t expected = 4; + ctrls.set(V4L2_CID_BRIGHTNESS, expected); + delayed->reset(&ctrls); + + /* Test single control with delay. */ + for (int32_t i = 0; i < 100; i++) { + int32_t value = 10 + i; + + ctrls.set(V4L2_CID_BRIGHTNESS, value); + delayed->push(ctrls); + + delayed->frameStart(i); + + ControlList result = delayed->get(i); + int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); + if (brightness != expected) { + cerr << "Failed single control with delay" + << " frame " << i + << " expected " << expected + << " got " << brightness + << endl; + return TestFail; + } + + expected = value; + } + + return TestPass; + } + + int dualControlsWithDelay(uint32_t startOffset) + { + std::unordered_map delays = { + { V4L2_CID_BRIGHTNESS, 1 }, + { V4L2_CID_CONTRAST, 2 }, + }; + std::unique_ptr delayed = + std::make_unique(dev_, delays); + ControlList ctrls; + + /* Reset control to value that will be first two frames in test. */ + int32_t expected = 200; + ctrls.set(V4L2_CID_BRIGHTNESS, expected); + ctrls.set(V4L2_CID_CONTRAST, expected); + delayed->reset(&ctrls); + + /* Test dual control with delay. */ + for (int32_t i = 0; i < 100; i++) { + uint32_t frame = startOffset + i; + int32_t value = 10 + i; + + ctrls.set(V4L2_CID_BRIGHTNESS, value); + ctrls.set(V4L2_CID_CONTRAST, value); + delayed->push(ctrls); + + delayed->frameStart(frame); + + ControlList result = delayed->get(frame); + int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); + int32_t contrast = result.get(V4L2_CID_CONTRAST).get(); + if (brightness != expected || contrast != expected) { + cerr << "Failed dual controls" + << " frame " << frame + << " brightness " << brightness + << " contrast " << contrast + << " expected " << expected + << endl; + return TestFail; + } + + expected = i < 1 ? expected : value - 1; + } + + return TestPass; + } + + int dualControlsMultiQueue() + { + std::unordered_map delays = { + { V4L2_CID_BRIGHTNESS, 1 }, + { V4L2_CID_CONTRAST, 2 }, + }; + std::unique_ptr delayed = + std::make_unique(dev_, delays); + ControlList ctrls; + + /* Reset control to value that will be first two frames in test. */ + int32_t expected = 100; + ctrls.set(V4L2_CID_BRIGHTNESS, expected); + ctrls.set(V4L2_CID_CONTRAST, expected); + delayed->reset(&ctrls); + + /* + * Queue all controls before any fake frame start. Note we + * can't queue up more then the delayed controls history size + * which is 16. Where one spot is used by the reset control. + */ + for (int32_t i = 0; i < 15; i++) { + int32_t value = 10 + i; + + ctrls.set(V4L2_CID_BRIGHTNESS, value); + ctrls.set(V4L2_CID_CONTRAST, value); + delayed->push(ctrls); + } + + /* Process all queued controls. */ + for (int32_t i = 0; i < 16; i++) { + int32_t value = 10 + i; + + delayed->frameStart(i); + + ControlList result = delayed->get(i); + + int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); + int32_t contrast = result.get(V4L2_CID_CONTRAST).get(); + if (brightness != expected || contrast != expected) { + cerr << "Failed multi queue" + << " frame " << i + << " brightness " << brightness + << " contrast " << contrast + << " expected " << expected + << endl; + return TestFail; + } + + expected = i < 1 ? expected : value - 1; + } + + return TestPass; + } + + int run() + { + int ret; + + /* Test single control without delay. */ + ret = singleControlNoDelay(); + if (ret) + return ret; + + /* Test single control with delay. */ + ret = singleControlWithDelay(); + if (ret) + return ret; + + /* Test dual controls with different delays. */ + ret = dualControlsWithDelay(0); + if (ret) + return ret; + + /* Test dual controls with non-zero sequence start. */ + ret = dualControlsWithDelay(10000); + if (ret) + return ret; + + /* Test dual controls with sequence number wraparound. */ + ret = dualControlsWithDelay(UINT32_MAX - 50); + if (ret) + return ret; + + /* Test control values produced faster then consumed. */ + ret = dualControlsMultiQueue(); + if (ret) + return ret; + + return TestPass; + } + + void cleanup() + { + delete dev_; + } + +private: + std::unique_ptr enumerator_; + std::shared_ptr media_; + V4L2VideoDevice *dev_; +}; + +TEST_REGISTER(DelayedControlsTest) diff --git a/test/meson.build b/test/meson.build index 0a1d434e399641bb..a683a657a439b4ff 100644 --- a/test/meson.build +++ b/test/meson.build @@ -25,6 +25,7 @@ public_tests = [ internal_tests = [ ['byte-stream-buffer', 'byte-stream-buffer.cpp'], ['camera-sensor', 'camera-sensor.cpp'], + ['delayed_contols', 'delayed_contols.cpp'], ['event', 'event.cpp'], ['event-dispatcher', 'event-dispatcher.cpp'], ['event-thread', 'event-thread.cpp'], From patchwork Mon Nov 23 22:12:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10483 X-Patchwork-Delegate: niklas.soderlund@ragnatech.se Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5BDD1BE177 for ; Mon, 23 Nov 2020 22:12:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1DBA8633FC; Mon, 23 Nov 2020 23:12:52 +0100 (CET) Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 24021633F9 for ; Mon, 23 Nov 2020 23:12:49 +0100 (CET) X-Halon-ID: 26edcb38-2dd8-11eb-a076-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de [79.202.36.88]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 26edcb38-2dd8-11eb-a076-005056917f90; Mon, 23 Nov 2020 23:06:35 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com Date: Mon, 23 Nov 2020 23:12:29 +0100 Message-Id: <20201123221234.485933-4-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> References: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 3/8] libcamera: raspberrypi: Switch to DelayedControls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the libcamera core helper DelayedControls instead of the local StaggeredCtrl. The new helper is modeled after the StaggeredCtrl implementation and behaves the same. Signed-off-by: Niklas Söderlund --- .../pipeline/raspberrypi/raspberrypi.cpp | 44 +++++++++---------- 1 file changed, 20 insertions(+), 24 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 66b483f73e2d6625..43bb4c2ecabd1f15 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -26,6 +26,7 @@ #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/buffer.h" #include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/media_device.h" @@ -36,7 +37,6 @@ #include "dma_heaps.h" #include "rpi_stream.h" -#include "staggered_ctrl.h" namespace libcamera { @@ -176,8 +176,7 @@ public: RPi::DmaHeap dmaHeap_; FileDescriptor lsTable_; - RPi::StaggeredCtrl staggeredCtrl_; - uint32_t expectedSequence_; + std::unique_ptr delayedCtrls_; bool sensorMetadata_; /* @@ -780,10 +779,7 @@ int PipelineHandlerRPi::start(Camera *camera) * starting. First check that the staggered ctrl has been initialised * by configure(). */ - ASSERT(data->staggeredCtrl_); - data->staggeredCtrl_.reset(); - data->staggeredCtrl_.write(); - data->expectedSequence_ = 0; + data->delayedCtrls_->reset(); data->state_ = RPiCameraData::State::Idle; @@ -1122,7 +1118,7 @@ void RPiCameraData::frameStarted(uint32_t sequence) LOG(RPI, Debug) << "frame start " << sequence; /* Write any controls for the next frame as soon as we can. */ - staggeredCtrl_.write(); + delayedCtrls_->frameStart(sequence); } int RPiCameraData::loadIPA() @@ -1200,18 +1196,22 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) * Setup our staggered control writer with the sensor default * gain and exposure delays. */ - if (!staggeredCtrl_) { - staggeredCtrl_.init(unicam_[Unicam::Image].dev(), - { { V4L2_CID_ANALOGUE_GAIN, result.data[resultIdx++] }, - { V4L2_CID_EXPOSURE, result.data[resultIdx++] } }); + + if (!delayedCtrls_) { + std::unordered_map delays = { + { V4L2_CID_ANALOGUE_GAIN, result.data[resultIdx++] }, + { V4L2_CID_EXPOSURE, result.data[resultIdx++] } + }; + + delayedCtrls_ = std::make_unique(unicam_[Unicam::Image].dev(), delays); + sensorMetadata_ = result.data[resultIdx++]; } } if (result.operation & RPi::IPA_CONFIG_SENSOR) { - const ControlList &ctrls = result.controls[0]; - if (!staggeredCtrl_.set(ctrls)) - LOG(RPI, Error) << "V4L2 staggered set failed"; + ControlList ctrls = result.controls[0]; + delayedCtrls_->reset(&ctrls); } if (result.operation & RPi::IPA_CONFIG_DROP_FRAMES) { @@ -1245,8 +1245,8 @@ void RPiCameraData::queueFrameAction([[maybe_unused]] unsigned int frame, switch (action.operation) { case RPi::IPA_ACTION_V4L2_SET_STAGGERED: { const ControlList &controls = action.controls[0]; - if (!staggeredCtrl_.set(controls)) - LOG(RPI, Error) << "V4L2 staggered set failed"; + if (!delayedCtrls_->push(controls)) + LOG(RPI, Error) << "V4L2 delay set failed"; goto done; } @@ -1350,11 +1350,7 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) } else { embeddedQueue_.push(buffer); - std::unordered_map ctrl; - int offset = buffer->metadata().sequence - expectedSequence_; - staggeredCtrl_.get(ctrl, offset); - - expectedSequence_ = buffer->metadata().sequence + 1; + ControlList ctrl = delayedCtrls_->get(buffer->metadata().sequence); /* * Sensor metadata is unavailable, so put the expected ctrl @@ -1366,8 +1362,8 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) auto it = mappedEmbeddedBuffers_.find(bufferId); if (it != mappedEmbeddedBuffers_.end()) { uint32_t *mem = reinterpret_cast(it->second.maps()[0].data()); - mem[0] = ctrl[V4L2_CID_EXPOSURE]; - mem[1] = ctrl[V4L2_CID_ANALOGUE_GAIN]; + mem[0] = ctrl.get(V4L2_CID_EXPOSURE).get(); + mem[1] = ctrl.get(V4L2_CID_ANALOGUE_GAIN).get(); } else { LOG(RPI, Warning) << "Failed to find embedded buffer " << bufferId; From patchwork Mon Nov 23 22:12:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10484 X-Patchwork-Delegate: niklas.soderlund@ragnatech.se Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B1CDFBE176 for ; Mon, 23 Nov 2020 22:12:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 86F29633FE; Mon, 23 Nov 2020 23:12:54 +0100 (CET) Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F22F9633F3 for ; Mon, 23 Nov 2020 23:12:49 +0100 (CET) X-Halon-ID: 275b14ee-2dd8-11eb-a076-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de [79.202.36.88]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 275b14ee-2dd8-11eb-a076-005056917f90; Mon, 23 Nov 2020 23:06:36 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com Date: Mon, 23 Nov 2020 23:12:30 +0100 Message-Id: <20201123221234.485933-5-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> References: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 4/8] libcamera: raspberrypi: Remove StaggeredCtrl X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" There are no users left, remove it. Signed-off-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- .../pipeline/raspberrypi/meson.build | 1 - .../pipeline/raspberrypi/staggered_ctrl.cpp | 174 ------------------ .../pipeline/raspberrypi/staggered_ctrl.h | 96 ---------- 3 files changed, 271 deletions(-) delete mode 100644 src/libcamera/pipeline/raspberrypi/staggered_ctrl.cpp delete mode 100644 src/libcamera/pipeline/raspberrypi/staggered_ctrl.h diff --git a/src/libcamera/pipeline/raspberrypi/meson.build b/src/libcamera/pipeline/raspberrypi/meson.build index 7c5b6ff73741c968..f1a2f5ee72c2a0dd 100644 --- a/src/libcamera/pipeline/raspberrypi/meson.build +++ b/src/libcamera/pipeline/raspberrypi/meson.build @@ -4,5 +4,4 @@ libcamera_sources += files([ 'dma_heaps.cpp', 'raspberrypi.cpp', 'rpi_stream.cpp', - 'staggered_ctrl.cpp', ]) diff --git a/src/libcamera/pipeline/raspberrypi/staggered_ctrl.cpp b/src/libcamera/pipeline/raspberrypi/staggered_ctrl.cpp deleted file mode 100644 index 62605c0fceee7d11..0000000000000000 --- a/src/libcamera/pipeline/raspberrypi/staggered_ctrl.cpp +++ /dev/null @@ -1,174 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2020, Raspberry Pi (Trading) Ltd. - * - * staggered_ctrl.cpp - Helper for writing staggered ctrls to a V4L2 device. - */ - -#include "staggered_ctrl.h" - -#include - -#include - -#include "libcamera/internal/log.h" -#include "libcamera/internal/utils.h" -#include "libcamera/internal/v4l2_videodevice.h" - -namespace libcamera { - -LOG_DEFINE_CATEGORY(RPI_S_W) - -namespace RPi { - -void StaggeredCtrl::init(V4L2VideoDevice *dev, - std::initializer_list> delayList) -{ - std::lock_guard lock(lock_); - - dev_ = dev; - delay_ = delayList; - ctrl_.clear(); - - /* Find the largest delay across all controls. */ - maxDelay_ = 0; - for (auto const &p : delay_) { - LOG(RPI_S_W, Info) << "Init ctrl " - << utils::hex(p.first) << " with delay " - << static_cast(p.second); - maxDelay_ = std::max(maxDelay_, p.second); - } - - init_ = true; -} - -void StaggeredCtrl::reset() -{ - std::lock_guard lock(lock_); - - int lastSetCount = setCount_; - std::unordered_map lastVal; - - /* Reset the counters. */ - setCount_ = getCount_ = 0; - - /* Look for the last set values. */ - for (auto const &c : ctrl_) - lastVal[c.first] = c.second[lastSetCount].value; - - /* Apply the last set values as the next to be applied. */ - ctrl_.clear(); - for (auto &c : lastVal) - ctrl_[c.first][setCount_] = CtrlInfo(c.second); -} - -bool StaggeredCtrl::set(uint32_t ctrl, int32_t value) -{ - std::lock_guard lock(lock_); - - /* Can we find this ctrl as one that is registered? */ - if (delay_.find(ctrl) == delay_.end()) - return false; - - ctrl_[ctrl][setCount_].value = value; - ctrl_[ctrl][setCount_].updated = true; - - return true; -} - -bool StaggeredCtrl::set(std::initializer_list> ctrlList) -{ - std::lock_guard lock(lock_); - - for (auto const &p : ctrlList) { - /* Can we find this ctrl? */ - if (delay_.find(p.first) == delay_.end()) - return false; - - ctrl_[p.first][setCount_] = CtrlInfo(p.second); - } - - return true; -} - -bool StaggeredCtrl::set(const ControlList &controls) -{ - std::lock_guard lock(lock_); - - for (auto const &p : controls) { - /* Can we find this ctrl? */ - if (delay_.find(p.first) == delay_.end()) - return false; - - ctrl_[p.first][setCount_] = CtrlInfo(p.second.get()); - LOG(RPI_S_W, Debug) << "Setting ctrl " - << utils::hex(p.first) << " to " - << ctrl_[p.first][setCount_].value - << " at index " - << setCount_; - } - - return true; -} - -int StaggeredCtrl::write() -{ - std::lock_guard lock(lock_); - ControlList controls(dev_->controls()); - - for (auto &p : ctrl_) { - int delayDiff = maxDelay_ - delay_[p.first]; - int index = std::max(0, setCount_ - delayDiff); - - if (p.second[index].updated) { - /* We need to write this value out. */ - controls.set(p.first, p.second[index].value); - p.second[index].updated = false; - LOG(RPI_S_W, Debug) << "Writing ctrl " - << utils::hex(p.first) << " to " - << p.second[index].value - << " at index " - << index; - } - } - - nextFrame(); - return dev_->setControls(&controls); -} - -void StaggeredCtrl::get(std::unordered_map &ctrl, uint8_t offset) -{ - std::lock_guard lock(lock_); - - /* Account for the offset to reset the getCounter. */ - getCount_ += offset + 1; - - ctrl.clear(); - for (auto &p : ctrl_) { - int index = std::max(0, getCount_ - maxDelay_); - ctrl[p.first] = p.second[index].value; - LOG(RPI_S_W, Debug) << "Getting ctrl " - << utils::hex(p.first) << " to " - << p.second[index].value - << " at index " - << index; - } -} - -void StaggeredCtrl::nextFrame() -{ - /* Advance the control history to the next frame */ - int prevCount = setCount_; - setCount_++; - - LOG(RPI_S_W, Debug) << "Next frame, set index is " << setCount_; - - for (auto &p : ctrl_) { - p.second[setCount_].value = p.second[prevCount].value; - p.second[setCount_].updated = false; - } -} - -} /* namespace RPi */ - -} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/raspberrypi/staggered_ctrl.h b/src/libcamera/pipeline/raspberrypi/staggered_ctrl.h deleted file mode 100644 index 382fa31a685357f2..0000000000000000 --- a/src/libcamera/pipeline/raspberrypi/staggered_ctrl.h +++ /dev/null @@ -1,96 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2020, Raspberry Pi (Trading) Ltd. - * - * staggered_ctrl.h - Helper for writing staggered ctrls to a V4L2 device. - */ -#ifndef __LIBCAMERA_PIPELINE_RASPBERRYPI_STAGGERED_CTRL_H__ -#define __LIBCAMERA_PIPELINE_RASPBERRYPI_STAGGERED_CTRL_H__ - -#include -#include -#include -#include -#include - -namespace libcamera { - -class ControlList; -class V4L2VideoDevice; - -namespace RPi { - -class StaggeredCtrl -{ -public: - StaggeredCtrl() - : init_(false), setCount_(0), getCount_(0), maxDelay_(0) - { - } - - operator bool() const - { - return init_; - } - - void init(V4L2VideoDevice *dev, - std::initializer_list> delayList); - void reset(); - - void get(std::unordered_map &ctrl, uint8_t offset = 0); - - bool set(uint32_t ctrl, int32_t value); - bool set(std::initializer_list> ctrlList); - bool set(const ControlList &controls); - - int write(); - -private: - void nextFrame(); - - /* listSize must be a power of 2. */ - static constexpr int listSize = (1 << 4); - struct CtrlInfo { - CtrlInfo() - : value(0), updated(false) - { - } - - CtrlInfo(int32_t value_) - : value(value_), updated(true) - { - } - - int32_t value; - bool updated; - }; - - class CircularArray : public std::array - { - public: - CtrlInfo &operator[](int index) - { - return std::array::operator[](index & (listSize - 1)); - } - - const CtrlInfo &operator[](int index) const - { - return std::array::operator[](index & (listSize - 1)); - } - }; - - bool init_; - uint32_t setCount_; - uint32_t getCount_; - uint8_t maxDelay_; - V4L2VideoDevice *dev_; - std::unordered_map delay_; - std::unordered_map ctrl_; - std::mutex lock_; -}; - -} /* namespace RPi */ - -} /* namespace libcamera */ - -#endif /* __LIBCAMERA_PIPELINE_RASPBERRYPI_STAGGERED_CTRL_H__ */ From patchwork Mon Nov 23 22:12:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10485 X-Patchwork-Delegate: niklas.soderlund@ragnatech.se Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 077EEBE177 for ; Mon, 23 Nov 2020 22:12:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CC04063409; Mon, 23 Nov 2020 23:12:54 +0100 (CET) Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9D624633F2 for ; Mon, 23 Nov 2020 23:12:50 +0100 (CET) X-Halon-ID: 27ec7f73-2dd8-11eb-a076-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de [79.202.36.88]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 27ec7f73-2dd8-11eb-a076-005056917f90; Mon, 23 Nov 2020 23:06:37 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com Date: Mon, 23 Nov 2020 23:12:31 +0100 Message-Id: <20201123221234.485933-6-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> References: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 5/8] libcamera: camera_sensor: Expose a DelayedControls interface X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Expose the optional DelayedControls interface to pipeline handlers. If used by a pipeline generic delays are used. In the future the delay values should be fetched to match the specific sensor module, either from a new kernel interface or worst case a sensors database. Signed-off-by: Niklas Söderlund --- include/libcamera/internal/camera_sensor.h | 5 ++++ src/libcamera/camera_sensor.cpp | 31 ++++++++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index b9eba89f00fba882..6be1cfd49134c534 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -14,6 +14,7 @@ #include #include +#include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/formats.h" #include "libcamera/internal/log.h" #include "libcamera/internal/v4l2_subdevice.h" @@ -61,6 +62,8 @@ public: ControlList getControls(const std::vector &ids); int setControls(ControlList *ctrls); + DelayedControls *delayedContols(); + const ControlList &properties() const { return properties_; } int sensorInfo(CameraSensorInfo *info) const; @@ -83,6 +86,8 @@ private: std::vector sizes_; ControlList properties_; + + std::unique_ptr delayedControls_; }; } /* namespace libcamera */ diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 935de528c4963453..6c92c97f4cc2547a 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -493,6 +493,37 @@ int CameraSensor::setControls(ControlList *ctrls) return subdev_->setControls(ctrls); } +/** + * \brief Get the sensors delayed controls interface + * + * Access the sensors delayed controls interface. This interface aids pipelines + * keeping tack of controls that needs time to take effect. The interface is + * optional to use and does not interact with setControls() and getControls() + * that operates directly on the sensor device. + * + * \sa DelayedControls + * \return The delayed controls interface + */ +DelayedControls *CameraSensor::delayedContols() +{ + if (!delayedControls_) { + /* + * \todo Read dealy values from the sensor itself or from a + * a sensor database. For now use generic values taken from + * the Raspberry Pi and listed as generic values. + */ + std::unordered_map delays = { + { V4L2_CID_ANALOGUE_GAIN, 1 }, + { V4L2_CID_EXPOSURE, 2 }, + }; + + delayedControls_ = + std::make_unique(subdev_.get(), delays); + } + + return delayedControls_.get(); +} + /** * \brief Assemble and return the camera sensor info * \param[out] info The camera sensor info From patchwork Mon Nov 23 22:12:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10486 X-Patchwork-Delegate: niklas.soderlund@ragnatech.se Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4D3A4BE176 for ; Mon, 23 Nov 2020 22:12:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1F6FD6340B; Mon, 23 Nov 2020 23:12:55 +0100 (CET) Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4D98D633F2 for ; Mon, 23 Nov 2020 23:12:51 +0100 (CET) X-Halon-ID: 284e5541-2dd8-11eb-a076-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de [79.202.36.88]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 284e5541-2dd8-11eb-a076-005056917f90; Mon, 23 Nov 2020 23:06:37 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com Date: Mon, 23 Nov 2020 23:12:32 +0100 Message-Id: <20201123221234.485933-7-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> References: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 6/8] libcamera: pipeline: rkisp1: Use CameraSensor and delayed controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Instead of setting controls using the RkISP1 local Timeline helper use the DelayedControls interface provided by the CameraSensor. The result are the same, the controls are applied with a delay. The values of the delays are however different between the two methods. The values used in the Timeline solution was chosen after some experimentation and the values used in DelayedControls are taken from a generic sensor. None of the two are a perfect match as the delays can be different for different sensors used with the pipeline. However using the interface provided by CameraSensor we prepare for the future where sensor specific delays will provided by the CameraSensor and used without any change in the pipeline. Signed-off-by: Niklas Söderlund --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 35 +++++++++--------------- 1 file changed, 13 insertions(+), 22 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 6e74a49abfda1b55..c3c4b5a65e3d9afe 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -121,8 +121,9 @@ class RkISP1CameraData : public CameraData public: RkISP1CameraData(PipelineHandler *pipe, RkISP1MainPath *mainPath, RkISP1SelfPath *selfPath) - : CameraData(pipe), sensor_(nullptr), frame_(0), - frameInfo_(pipe), mainPath_(mainPath), selfPath_(selfPath) + : CameraData(pipe), sensor_(nullptr), delayedCtrls_(nullptr), + frame_(0), frameInfo_(pipe), mainPath_(mainPath), + selfPath_(selfPath) { } @@ -136,6 +137,7 @@ public: Stream mainPathStream_; Stream selfPathStream_; CameraSensor *sensor_; + DelayedControls *delayedCtrls_; unsigned int frame_; std::vector ipaBuffers_; RkISP1Frames frameInfo_; @@ -346,23 +348,6 @@ RkISP1FrameInfo *RkISP1Frames::find(Request *request) return nullptr; } -class RkISP1ActionSetSensor : public FrameAction -{ -public: - RkISP1ActionSetSensor(unsigned int frame, CameraSensor *sensor, const ControlList &controls) - : FrameAction(frame, SetSensor), sensor_(sensor), controls_(controls) {} - -protected: - void run() override - { - sensor_->setControls(&controls_); - } - -private: - CameraSensor *sensor_; - ControlList controls_; -}; - class RkISP1ActionQueueBuffers : public FrameAction { public: @@ -430,9 +415,7 @@ void RkISP1CameraData::queueFrameAction(unsigned int frame, switch (action.operation) { case RKISP1_IPA_ACTION_V4L2_SET: { const ControlList &controls = action.controls[0]; - timeline_.scheduleAction(std::make_unique(frame, - sensor_, - controls)); + delayedCtrls_->push(controls); break; } case RKISP1_IPA_ACTION_PARAM_FILLED: { @@ -906,6 +889,8 @@ int PipelineHandlerRkISP1::start(Camera *camera) }; } + isp_->setFrameStartEnabled(true); + activeCamera_ = camera; /* Inform IPA of stream configuration and sensor controls. */ @@ -933,6 +918,8 @@ void PipelineHandlerRkISP1::stop(Camera *camera) RkISP1CameraData *data = cameraData(camera); int ret; + isp_->setFrameStartEnabled(false); + selfPath_.stop(); mainPath_.stop(); @@ -1051,6 +1038,10 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); + data->delayedCtrls_ = data->sensor_->delayedContols(); + isp_->frameStart.connect(data->delayedCtrls_, + &DelayedControls::frameStart); + ret = data->loadIPA(); if (ret) return ret; From patchwork Mon Nov 23 22:12:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10487 X-Patchwork-Delegate: niklas.soderlund@ragnatech.se Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8F67DBE178 for ; Mon, 23 Nov 2020 22:12:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 696AB633FB; Mon, 23 Nov 2020 23:12:55 +0100 (CET) Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1DD8363401 for ; Mon, 23 Nov 2020 23:12:52 +0100 (CET) X-Halon-ID: 28b5035f-2dd8-11eb-a076-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de [79.202.36.88]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 28b5035f-2dd8-11eb-a076-005056917f90; Mon, 23 Nov 2020 23:06:38 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com Date: Mon, 23 Nov 2020 23:12:33 +0100 Message-Id: <20201123221234.485933-8-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> References: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 7/8] libcamera: pipeline: rkisp1: Use SOF event to warn about late parameters X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In the Timeline approach the idea was to delay queuing buffers to the device until the IPA had a chance to prepare the parameters buffer. A check was still added to warn if the IPA queued buffers before the parameters buffer was filled in. This check happened much sooner then needed as the parameter buffers does not have to be ready when the buffer is queued but just before its consumed. As the pipeline now has true knowledge of each SOF we can move the check there and remove the delaying of queuing of buffers. This change really speeds up the IPA reactions as the delays used in the Timeline where approximated while with this change they are driven by events reported by the device. Signed-off-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 78 ++++++++---------------- 1 file changed, 24 insertions(+), 54 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index c3c4b5a65e3d9afe..3662a53ac4a43fcd 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -40,7 +40,6 @@ namespace libcamera { LOG_DEFINE_CATEGORY(RkISP1) class PipelineHandlerRkISP1; -class RkISP1ActionQueueBuffers; class RkISP1CameraData; enum RkISP1ActionType { @@ -203,7 +202,6 @@ private: PipelineHandler::cameraData(camera)); } - friend RkISP1ActionQueueBuffers; friend RkISP1CameraData; friend RkISP1Frames; @@ -214,6 +212,7 @@ private: void bufferReady(FrameBuffer *buffer); void paramReady(FrameBuffer *buffer); void statReady(FrameBuffer *buffer); + void frameStart(uint32_t sequence); int allocateBuffers(Camera *camera); int freeBuffers(Camera *camera); @@ -348,53 +347,6 @@ RkISP1FrameInfo *RkISP1Frames::find(Request *request) return nullptr; } -class RkISP1ActionQueueBuffers : public FrameAction -{ -public: - RkISP1ActionQueueBuffers(unsigned int frame, RkISP1CameraData *data, - PipelineHandlerRkISP1 *pipe) - : FrameAction(frame, QueueBuffers), data_(data), pipe_(pipe) - { - } - -protected: - void run() override - { - RkISP1FrameInfo *info = data_->frameInfo_.find(frame()); - if (!info) - LOG(RkISP1, Fatal) << "Frame not known"; - - /* - * \todo: If parameters are not filled a better method to handle - * the situation than queuing a buffer with unknown content - * should be used. - * - * It seems excessive to keep an internal zeroed scratch - * parameters buffer around as this should not happen unless the - * devices is under too much load. Perhaps failing the request - * and returning it to the application with an error code is - * better than queue it to hardware? - */ - if (!info->paramFilled) - LOG(RkISP1, Error) - << "Parameters not ready on time for frame " - << frame(); - - pipe_->param_->queueBuffer(info->paramBuffer); - pipe_->stat_->queueBuffer(info->statBuffer); - - if (info->mainPathBuffer) - pipe_->mainPath_.queueBuffer(info->mainPathBuffer); - - if (info->selfPathBuffer) - pipe_->selfPath_.queueBuffer(info->selfPathBuffer); - } - -private: - RkISP1CameraData *data_; - PipelineHandlerRkISP1 *pipe_; -}; - int RkISP1CameraData::loadIPA() { ipa_ = IPAManager::createIPA(pipe_, 1, 1); @@ -958,9 +910,14 @@ int PipelineHandlerRkISP1::queueRequestDevice(Camera *camera, Request *request) op.controls = { request->controls() }; data->ipa_->processEvent(op); - data->timeline_.scheduleAction(std::make_unique(data->frame_, - data, - this)); + param_->queueBuffer(info->paramBuffer); + stat_->queueBuffer(info->statBuffer); + + if (info->mainPathBuffer) + mainPath_.queueBuffer(info->mainPathBuffer); + + if (info->selfPathBuffer) + selfPath_.queueBuffer(info->selfPathBuffer); data->frame_++; @@ -1098,6 +1055,7 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) mainPath_.bufferReady().connect(this, &PipelineHandlerRkISP1::bufferReady); selfPath_.bufferReady().connect(this, &PipelineHandlerRkISP1::bufferReady); stat_->bufferReady.connect(this, &PipelineHandlerRkISP1::statReady); + isp_->frameStart.connect(this, &PipelineHandlerRkISP1::frameStart); param_->bufferReady.connect(this, &PipelineHandlerRkISP1::paramReady); /* @@ -1177,8 +1135,6 @@ void PipelineHandlerRkISP1::statReady(FrameBuffer *buffer) if (!info) return; - data->timeline_.bufferReady(buffer); - if (data->frame_ <= buffer->metadata().sequence) data->frame_ = buffer->metadata().sequence + 1; @@ -1188,6 +1144,20 @@ void PipelineHandlerRkISP1::statReady(FrameBuffer *buffer) data->ipa_->processEvent(op); } +void PipelineHandlerRkISP1::frameStart(uint32_t sequence) +{ + if (!activeCamera_) + return; + + RkISP1CameraData *data = cameraData(activeCamera_); + + RkISP1FrameInfo *info = data->frameInfo_.find(sequence); + if (!info || !info->paramFilled) + LOG(RkISP1, Info) + << "Parameters not ready on time for frame " + << sequence; +} + REGISTER_PIPELINE_HANDLER(PipelineHandlerRkISP1) } /* namespace libcamera */ From patchwork Mon Nov 23 22:12:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10488 X-Patchwork-Delegate: niklas.soderlund@ragnatech.se Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DBC84BE177 for ; Mon, 23 Nov 2020 22:12:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AFC626340D; Mon, 23 Nov 2020 23:12:55 +0100 (CET) Received: from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net [195.74.38.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 44D0D633F9 for ; Mon, 23 Nov 2020 23:12:53 +0100 (CET) X-Halon-ID: 292ddfb2-2dd8-11eb-a076-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de [79.202.36.88]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 292ddfb2-2dd8-11eb-a076-005056917f90; Mon, 23 Nov 2020 23:06:39 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com Date: Mon, 23 Nov 2020 23:12:34 +0100 Message-Id: <20201123221234.485933-9-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> References: <20201123221234.485933-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 8/8] libcamera: pipeline: rkisp1: Remove Timeline X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" There are no users left of the Timeline and there is no longer a need to keep emulating a start of exposure event as the CSI-2 resciver reports it. Remove the Timeline helper and what's left of it's integration in the pipeline. There is no functional change as nothing i the pipeline uses the Timeline. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/rkisp1/meson.build | 1 - src/libcamera/pipeline/rkisp1/rkisp1.cpp | 45 ---- src/libcamera/pipeline/rkisp1/timeline.cpp | 227 --------------------- src/libcamera/pipeline/rkisp1/timeline.h | 71 ------- 4 files changed, 344 deletions(-) delete mode 100644 src/libcamera/pipeline/rkisp1/timeline.cpp delete mode 100644 src/libcamera/pipeline/rkisp1/timeline.h diff --git a/src/libcamera/pipeline/rkisp1/meson.build b/src/libcamera/pipeline/rkisp1/meson.build index 5cd40d949d543fa9..cad66535c25f8cc0 100644 --- a/src/libcamera/pipeline/rkisp1/meson.build +++ b/src/libcamera/pipeline/rkisp1/meson.build @@ -3,5 +3,4 @@ libcamera_sources += files([ 'rkisp1.cpp', 'rkisp1_path.cpp', - 'timeline.cpp', ]) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 3662a53ac4a43fcd..c2b61e88ec34aa60 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -33,7 +33,6 @@ #include "libcamera/internal/v4l2_videodevice.h" #include "rkisp1_path.h" -#include "timeline.h" namespace libcamera { @@ -42,12 +41,6 @@ LOG_DEFINE_CATEGORY(RkISP1) class PipelineHandlerRkISP1; class RkISP1CameraData; -enum RkISP1ActionType { - SetSensor, - SOE, - QueueBuffers, -}; - struct RkISP1FrameInfo { unsigned int frame; Request *request; @@ -80,41 +73,6 @@ private: std::map frameInfo_; }; -class RkISP1Timeline : public Timeline -{ -public: - RkISP1Timeline() - : Timeline() - { - setDelay(SetSensor, -1, 5); - setDelay(SOE, 0, -1); - setDelay(QueueBuffers, -1, 10); - } - - void bufferReady(FrameBuffer *buffer) - { - /* - * Calculate SOE by taking the end of DMA set by the kernel and applying - * the time offsets provideprovided by the IPA to find the best estimate - * of SOE. - */ - - ASSERT(frameOffset(SOE) == 0); - - utils::time_point soe = std::chrono::time_point() - + std::chrono::nanoseconds(buffer->metadata().timestamp) - + timeOffset(SOE); - - notifyStartOfExposure(buffer->metadata().sequence, soe); - } - - void setDelay(unsigned int type, int frame, int msdelay) - { - utils::duration delay = std::chrono::milliseconds(msdelay); - setRawDelay(type, frame, delay); - } -}; - class RkISP1CameraData : public CameraData { public: @@ -140,7 +98,6 @@ public: unsigned int frame_; std::vector ipaBuffers_; RkISP1Frames frameInfo_; - RkISP1Timeline timeline_; RkISP1MainPath *mainPath_; RkISP1SelfPath *selfPath_; @@ -887,8 +844,6 @@ void PipelineHandlerRkISP1::stop(Camera *camera) data->ipa_->stop(); - data->timeline_.reset(); - data->frameInfo_.clear(); freeBuffers(camera); diff --git a/src/libcamera/pipeline/rkisp1/timeline.cpp b/src/libcamera/pipeline/rkisp1/timeline.cpp deleted file mode 100644 index 6b83bbe5b5892531..0000000000000000 --- a/src/libcamera/pipeline/rkisp1/timeline.cpp +++ /dev/null @@ -1,227 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2019, Google Inc. - * - * timeline.cpp - Timeline for per-frame control - */ - -#include "timeline.h" - -#include "libcamera/internal/log.h" - -/** - * \file timeline.h - * \brief Timeline for per-frame control - */ - -namespace libcamera { - -LOG_DEFINE_CATEGORY(Timeline) - -/** - * \class FrameAction - * \brief Action that can be schedule on a Timeline - * - * A frame action is an event schedule to be executed on a Timeline. A frame - * action has two primal attributes a frame number and a type. - * - * The frame number describes the frame to which the action is associated. The - * type is a numerical ID which identifies the action within the pipeline and - * IPA protocol. - */ - -/** - * \class Timeline - * \brief Executor of FrameAction - * - * The timeline has three primary functions: - * - * 1. Keep track of the Start of Exposure (SOE) for every frame processed by - * the hardware. Using this information it shall keep an up-to-date estimate - * of the frame interval (time between two consecutive SOE events). - * - * The estimated frame interval together with recorded SOE events are the - * foundation for how the timeline schedule FrameAction at specific points - * in time. - * \todo Improve the frame interval estimation algorithm. - * - * 2. Keep track of current delays for different types of actions. The delays - * for different actions might differ during a capture session. Exposure time - * effects the over all FPS and different ISP parameters might impacts its - * processing time. - * - * The action type delays shall be updated by the IPA in conjunction with - * how it changes the capture parameters. - * - * 3. Schedule actions on the timeline. This is the process of taking a - * FrameAction which contains an abstract description of what frame and - * what type of action it contains and turning that into an time point - * and make sure the action is executed at that time. - */ - -Timeline::Timeline() - : frameInterval_(0) -{ - timer_.timeout.connect(this, &Timeline::timeout); -} - -/** - * \brief Reset and stop the timeline - * - * The timeline needs to be reset when the timeline should no longer execute - * actions. A timeline should be reset between two capture sessions to prevent - * the old capture session to effect the second one. - */ -void Timeline::reset() -{ - timer_.stop(); - - actions_.clear(); - history_.clear(); -} - -/** - * \brief Schedule an action on the timeline - * \param[in] action FrameAction to schedule - * - * The act of scheduling an action to the timeline is the process of taking - * the properties of the action (type, frame and time offsets) and translating - * that to a time point using the current values for the action type timings - * value recorded in the timeline. If an action is scheduled too late, execute - * it immediately. - */ -void Timeline::scheduleAction(std::unique_ptr action) -{ - unsigned int lastFrame; - utils::time_point lastTime; - - if (history_.empty()) { - lastFrame = 0; - lastTime = std::chrono::steady_clock::now(); - } else { - lastFrame = history_.back().first; - lastTime = history_.back().second; - } - - /* - * Calculate when the action shall be schedule by first finding out how - * many frames in the future the action acts on and then add the actions - * frame offset. After the spatial frame offset is found out translate - * that to a time point by using the last estimated start of exposure - * (SOE) as the fixed offset. Lastly add the action time offset to the - * time point. - */ - int frame = action->frame() - lastFrame + frameOffset(action->type()); - utils::time_point deadline = lastTime + frame * frameInterval_ - + timeOffset(action->type()); - - utils::time_point now = std::chrono::steady_clock::now(); - if (deadline < now) { - LOG(Timeline, Warning) - << "Action scheduled too late " - << utils::time_point_to_string(deadline) - << ", run now " << utils::time_point_to_string(now); - action->run(); - } else { - actions_.emplace(deadline, std::move(action)); - updateDeadline(); - } -} - -void Timeline::notifyStartOfExposure(unsigned int frame, utils::time_point time) -{ - history_.push_back(std::make_pair(frame, time)); - - if (history_.size() <= HISTORY_DEPTH / 2) - return; - - while (history_.size() > HISTORY_DEPTH) - history_.pop_front(); - - /* Update esitmated time between two start of exposures. */ - utils::duration sumExposures(0); - unsigned int numExposures = 0; - - utils::time_point lastTime; - for (auto it = history_.begin(); it != history_.end(); it++) { - if (it != history_.begin()) { - sumExposures += it->second - lastTime; - numExposures++; - } - - lastTime = it->second; - } - - frameInterval_ = sumExposures; - if (numExposures) - frameInterval_ /= numExposures; -} - -int Timeline::frameOffset(unsigned int type) const -{ - const auto it = delays_.find(type); - if (it == delays_.end()) { - LOG(Timeline, Error) - << "No frame offset set for action type " << type; - return 0; - } - - return it->second.first; -} - -utils::duration Timeline::timeOffset(unsigned int type) const -{ - const auto it = delays_.find(type); - if (it == delays_.end()) { - LOG(Timeline, Error) - << "No time offset set for action type " << type; - return utils::duration::zero(); - } - - return it->second.second; -} - -void Timeline::setRawDelay(unsigned int type, int frame, utils::duration time) -{ - delays_[type] = std::make_pair(frame, time); -} - -void Timeline::updateDeadline() -{ - if (actions_.empty()) - return; - - const utils::time_point &deadline = actions_.begin()->first; - - if (timer_.isRunning() && deadline >= timer_.deadline()) - return; - - if (deadline <= std::chrono::steady_clock::now()) { - timeout(&timer_); - return; - } - - timer_.start(deadline); -} - -void Timeline::timeout([[maybe_unused]] Timer *timer) -{ - utils::time_point now = std::chrono::steady_clock::now(); - - for (auto it = actions_.begin(); it != actions_.end();) { - const utils::time_point &sched = it->first; - - if (sched > now) - break; - - FrameAction *action = it->second.get(); - - action->run(); - - it = actions_.erase(it); - } - - updateDeadline(); -} - -} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rkisp1/timeline.h b/src/libcamera/pipeline/rkisp1/timeline.h deleted file mode 100644 index 35a085159b8f545b..0000000000000000 --- a/src/libcamera/pipeline/rkisp1/timeline.h +++ /dev/null @@ -1,71 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2019, Google Inc. - * - * timeline.h - Timeline for per-frame controls - */ -#ifndef __LIBCAMERA_TIMELINE_H__ -#define __LIBCAMERA_TIMELINE_H__ - -#include -#include - -#include "libcamera/internal/timer.h" -#include "libcamera/internal/utils.h" - -namespace libcamera { - -class FrameAction -{ -public: - FrameAction(unsigned int frame, unsigned int type) - : frame_(frame), type_(type) {} - - virtual ~FrameAction() = default; - - unsigned int frame() const { return frame_; } - unsigned int type() const { return type_; } - - virtual void run() = 0; - -private: - unsigned int frame_; - unsigned int type_; -}; - -class Timeline -{ -public: - Timeline(); - virtual ~Timeline() = default; - - virtual void reset(); - virtual void scheduleAction(std::unique_ptr action); - virtual void notifyStartOfExposure(unsigned int frame, utils::time_point time); - - utils::duration frameInterval() const { return frameInterval_; } - -protected: - int frameOffset(unsigned int type) const; - utils::duration timeOffset(unsigned int type) const; - - void setRawDelay(unsigned int type, int frame, utils::duration time); - - std::map> delays_; - -private: - static constexpr unsigned int HISTORY_DEPTH = 10; - - void timeout(Timer *timer); - void updateDeadline(); - - std::list> history_; - std::multimap> actions_; - utils::duration frameInterval_; - - Timer timer_; -}; - -} /* namespace libcamera */ - -#endif /* __LIBCAMERA_TIMELINE_H__ */