From patchwork Tue Dec 15 00:48:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10662 X-Patchwork-Delegate: niklas.soderlund@ragnatech.se Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 15C74BD80A for ; Tue, 15 Dec 2020 00:48:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E298361594; Tue, 15 Dec 2020 01:48:22 +0100 (CET) Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D9D8161590 for ; Tue, 15 Dec 2020 01:48:20 +0100 (CET) X-Halon-ID: 397d6a07-3e6f-11eb-a076-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de [79.202.36.88]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 397d6a07-3e6f-11eb-a076-005056917f90; Tue, 15 Dec 2020 01:48:19 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com Date: Tue, 15 Dec 2020 01:48:04 +0100 Message-Id: <20201215004811.602429-2-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201215004811.602429-1-niklas.soderlund@ragnatech.se> References: <20201215004811.602429-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 1/8] libcamera: delayed_controls: Add helper for controls that applies with a delay X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Some sensor controls take effect with a delay as the sensor needs time to adjust, for example exposure. Add an optional helper DelayedControls to help pipelines deal with such controls. The idea is to provide a queue of controls towards the V4L2 device and apply individual controls with the specified delay with the aim to get predictable and retrievable control values for any given frame. To do this the queue of controls needs to be at least as deep as the control with the largest delay. The DelayedControls needs to be informed of every start of exposure. This can be emulated but the helper is designed to be used with this event being provide by the kernel thru V4L2 events. This helper is based on StaggeredCtrl from the Raspberry Pi pipeline handler but expands on its API. This helpers aims to replace the Raspberry Pi implementations and mimics it behavior perfectly. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Naushir Patuck --- * Changes since v2 - Drop optional argument to reset(). - Update commit message. - Remove usage of Mutex. - Rename frameStart() to applyControls)_. - Rename ControlInfo to Into. - Rename ControlArray to ControlRingBuffer. - Drop ControlsDelays and ControlsValues. - Sort headers. - Rename iterators. - Simplify queueCount_ handeling in reset(). - Add more warnings. - Update documentation. * Changes since v2 - Improve error logic in queue() as suggested by Jean-Michel Hautbois. - s/fistSequence_/firstSequence_/ * Changes since v1 - Correct copyright to reflect work is derived from Raspberry Pi pipeline handler. This was always the intention and was wrong in v1. - Rewrite large parts of the documentation. - Join two loops to one in DelayedControls::DelayedControls() --- include/libcamera/internal/delayed_controls.h | 82 ++++++ src/libcamera/delayed_controls.cpp | 252 ++++++++++++++++++ src/libcamera/meson.build | 1 + 3 files changed, 335 insertions(+) create mode 100644 include/libcamera/internal/delayed_controls.h create mode 100644 src/libcamera/delayed_controls.cpp diff --git a/include/libcamera/internal/delayed_controls.h b/include/libcamera/internal/delayed_controls.h new file mode 100644 index 0000000000000000..1292b484ec9f53e9 --- /dev/null +++ b/include/libcamera/internal/delayed_controls.h @@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi (Trading) Ltd. + * + * delayed_controls.h - Helper to deal with controls that are applied with a delay + */ +#ifndef __LIBCAMERA_INTERNAL_DELAYED_CONTROLS_H__ +#define __LIBCAMERA_INTERNAL_DELAYED_CONTROLS_H__ + +#include +#include + +#include + +namespace libcamera { + +class V4L2Device; + +class DelayedControls +{ +public: + DelayedControls(V4L2Device *device, + const std::unordered_map &delays); + + void reset(); + + bool push(const ControlList &controls); + ControlList get(uint32_t sequence); + + void applyControls(uint32_t sequence); + +private: + class Info + { + public: + Info() + : updated(false) + { + } + + Info(const ControlValue &v) + : value(v), updated(true) + { + } + + ControlValue value; + bool updated; + }; + + /* \todo: Make the listSize configurable at instance creation time. */ + static constexpr int listSize = 16; + class ControlRingBuffer : public std::array + { + public: + Info &operator[](unsigned int index) + { + return std::array::operator[](index % listSize); + } + + const Info &operator[](unsigned int index) const + { + return std::array::operator[](index % listSize); + } + }; + + bool queue(const ControlList &controls); + + V4L2Device *device_; + std::unordered_map delays_; + unsigned int maxDelay_; + + bool running_; + uint32_t firstSequence_; + + uint32_t queueCount_; + uint32_t writeCount_; + std::unordered_map values_; +}; + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_INTERNAL_DELAYED_CONTROLS_H__ */ diff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp new file mode 100644 index 0000000000000000..db2e51f8c93c4755 --- /dev/null +++ b/src/libcamera/delayed_controls.cpp @@ -0,0 +1,252 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi (Trading) Ltd. + * + * delayed_controls.h - Helper to deal with controls that are applied with a delay + */ + +#include "libcamera/internal/delayed_controls.h" + +#include + +#include "libcamera/internal/log.h" +#include "libcamera/internal/v4l2_device.h" + +/** + * \file delayed_controls.h + * \brief Helper to deal with controls that are applied with a delay + */ + +namespace libcamera { + +LOG_DEFINE_CATEGORY(DelayedControls) + +/** + * \class DelayedControls + * \brief Helper to deal with controls that take effect with a delay + * + * Some sensor controls take effect with a delay as the sensor needs time to + * adjust, for example exposure and focus. This is a helper class to deal with + * such controls and the intended users are pipeline handlers. + * + * The idea is to extend the concept of the buffer depth of a pipeline the + * application needs to maintain to also cover controls. Just as with buffer + * depth if the application keeps the number of requests queued above the + * control depth the controls are guaranteed to take effect for the correct + * request. The control depth is determined by the control with the greatest + * delay. + */ + +/** + * \brief Construct a DelayedControls instance + * \param[in] device The V4L2 device the controls have to be applied to + * \param[in] delays Map of the numerical V4L2 control ids to their associated + * delays (in frames) + * + * Only controls specified in \a delays are handled. If it's desired to mix + * delayed controls and controls that take effect immediately the immediate + * controls must be listed in the \a delays map with a delay value of 0. + */ +DelayedControls::DelayedControls(V4L2Device *device, + const std::unordered_map &delays) + : device_(device), maxDelay_(0) +{ + const ControlInfoMap &controls = device_->controls(); + + /* + * Create a map of control ids to delays for controls exposed by the + * device. + */ + for (auto const &delay : delays) { + auto it = controls.find(delay.first); + if (it == controls.end()) { + LOG(DelayedControls, Error) + << "Delay request for control id " + << utils::hex(delay.first) + << " but control is not exposed by device " + << device_->deviceNode(); + continue; + } + + const ControlId *id = it->first; + + delays_[id] = delay.second; + + LOG(DelayedControls, Debug) + << "Set a delay of " << delays_[id] + << " for " << id->name(); + + maxDelay_ = std::max(maxDelay_, delays_[id]); + } + + reset(); +} + +/** + * \brief Reset state machine + * + * Resets the state machine to a starting position based on control values + * retrieved from the device. + */ +void DelayedControls::reset() +{ + running_ = false; + firstSequence_ = 0; + queueCount_ = 1; + writeCount_ = 0; + + /* Retrieve control as reported by the device. */ + std::vector ids; + for (auto const &delay : delays_) + ids.push_back(delay.first->id()); + + ControlList controls = device_->getControls(ids); + + /* Seed the control queue with the controls reported by the device. */ + values_.clear(); + for (const auto &ctrl : controls) { + const ControlId *id = device_->controls().idmap().at(ctrl.first); + values_[id][0] = Info(ctrl.second); + } +} + +/** + * \brief Push a set of controls on the queue + * \param[in] controls List of controls to add to the device queue + * + * Push a set of controls to the control queue. This increases the control queue + * depth by one. + * + * \returns true if \a controls are accepted, or false otherwise + */ +bool DelayedControls::push(const ControlList &controls) +{ + return queue(controls); +} + +bool DelayedControls::queue(const ControlList &controls) +{ + /* Copy state from previous frame. */ + for (auto &ctrl : values_) { + Info &info = ctrl.second[queueCount_]; + info.value = values_[ctrl.first][queueCount_ - 1].value; + info.updated = false; + } + + /* Update with new controls. */ + const ControlIdMap &idmap = device_->controls().idmap(); + for (const auto &control : controls) { + const auto &it = idmap.find(control.first); + if (it == idmap.end()) { + LOG(DelayedControls, Warning) + << "Unknown control " << control.first; + return false; + } + + const ControlId *id = it->second; + + if (delays_.find(id) == delays_.end()) + return false; + + Info &info = values_[id][queueCount_]; + + info.value = control.second; + info.updated = true; + + LOG(DelayedControls, Debug) + << "Queuing " << id->name() + << " to " << info.value.toString() + << " at index " << queueCount_; + } + + queueCount_++; + + return true; +} + +/** + * \brief Read back controls in effect at a sequence number + * \param[in] sequence The sequence number to get controls for + * + * Read back what controls where in effect at a specific sequence number. The + * history is a ring buffer of 16 entries where new and old values coexist. It's + * the callers responsibility to not read too old sequence numbers that have been + * pushed out of the history. + * + * Historic values are evicted by pushing new values onto the queue using + * push(). The max history from the current sequence number that yields valid + * values are thus 16 minus number of controls pushed. + * + * \return The controls at \a sequence number + */ +ControlList DelayedControls::get(uint32_t sequence) +{ + uint32_t adjustedSeq = sequence - firstSequence_ + 1; + unsigned int index = std::max(0, adjustedSeq - maxDelay_); + + ControlList out(device_->controls()); + for (const auto &ctrl : values_) { + const ControlId *id = ctrl.first; + const Info &info = ctrl.second[index]; + + out.set(id->id(), info.value); + + LOG(DelayedControls, Debug) + << "Reading " << id->name() + << " to " << info.value.toString() + << " at index " << index; + } + + return out; +} + +/** + * \brief Inform DelayedControls of the start of a new frame + * \param[in] sequence Sequence number of the frame that started + * + * Inform the state machine that a new frame has started and of its sequence + * number. Any user of these helpers is responsible to inform the helper about + * the start of any frame.This can be connected with ease to the start of a + * exposure (SOE) V4L2 event. + */ +void DelayedControls::applyControls(uint32_t sequence) +{ + LOG(DelayedControls, Debug) << "frame " << sequence << " started"; + + if (!running_) { + firstSequence_ = sequence; + running_ = true; + } + + /* + * Create control list peaking ahead in the value queue to ensure + * values are set in time to satisfy the sensor delay. + */ + ControlList out(device_->controls()); + for (const auto &ctrl : values_) { + const ControlId *id = ctrl.first; + unsigned int delayDiff = maxDelay_ - delays_[id]; + unsigned int index = std::max(0, writeCount_ - delayDiff); + const Info &info = ctrl.second[index]; + + if (info.updated) { + out.set(id->id(), info.value); + LOG(DelayedControls, Debug) + << "Setting " << id->name() + << " to " << info.value.toString() + << " at index " << index; + } + } + + writeCount_++; + + while (writeCount_ >= queueCount_) { + LOG(DelayedControls, Debug) + << "Queue is empty, auto queue no-op."; + queue({}); + } + + device_->setControls(&out); +} + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 387d5d88ecae11ad..5a4bf0d7ba4fd231 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -12,6 +12,7 @@ libcamera_sources = files([ 'controls.cpp', 'control_serializer.cpp', 'control_validator.cpp', + 'delayed_controls.cpp', 'device_enumerator.cpp', 'device_enumerator_sysfs.cpp', 'event_dispatcher.cpp', From patchwork Tue Dec 15 00:48:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10663 X-Patchwork-Delegate: niklas.soderlund@ragnatech.se Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6F14ABD80A for ; Tue, 15 Dec 2020 00:48:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4B80461598; Tue, 15 Dec 2020 01:48:25 +0100 (CET) Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C33EA61590 for ; Tue, 15 Dec 2020 01:48:22 +0100 (CET) X-Halon-ID: 3a4aa245-3e6f-11eb-a076-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de [79.202.36.88]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 3a4aa245-3e6f-11eb-a076-005056917f90; Tue, 15 Dec 2020 01:48:21 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com Date: Tue, 15 Dec 2020 01:48:05 +0100 Message-Id: <20201215004811.602429-3-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201215004811.602429-1-niklas.soderlund@ragnatech.se> References: <20201215004811.602429-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 2/8] test: delayed_controls: Add test case for DelayedControls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a test-case for DelayedControls that exercise the setting of controls and reading back what controls were used for a particular frame. Also exercise corner case such as a V4L2 devices that do not reset its sequence number to 0 at stream on and sequence number wrapping around the uint32_t value space. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi --- * Changes since v3 - Update commit message. - Update header documentation. - Add protected and override. - Use unsigned int instead of int32_t in loops. * Changes since v2 - Remove a unused LoC. - Add and remove blank lines. - Align number of loop iterations. --- test/delayed_contols.cpp | 304 +++++++++++++++++++++++++++++++++++++++ test/meson.build | 1 + 2 files changed, 305 insertions(+) create mode 100644 test/delayed_contols.cpp diff --git a/test/delayed_contols.cpp b/test/delayed_contols.cpp new file mode 100644 index 0000000000000000..0a2b47aebf06b7fb --- /dev/null +++ b/test/delayed_contols.cpp @@ -0,0 +1,304 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * delayed_controls.cpp - libcamera delayed controls test + */ + +#include + +#include "libcamera/internal/delayed_controls.h" +#include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/v4l2_videodevice.h" + +#include "test.h" + +using namespace std; +using namespace libcamera; + +class DelayedControlsTest : public Test +{ +public: + DelayedControlsTest() + : dev_(nullptr) + { + } + +protected: + int init() override + { + enumerator_ = DeviceEnumerator::create(); + if (!enumerator_) { + cerr << "Failed to create device enumerator" << endl; + return TestFail; + } + + if (enumerator_->enumerate()) { + cerr << "Failed to enumerate media devices" << endl; + return TestFail; + } + + DeviceMatch dm("vivid"); + dm.add("vivid-000-vid-cap"); + + media_ = enumerator_->search(dm); + if (!media_) { + cerr << "vivid video device found" << endl; + return TestSkip; + } + + MediaEntity *entity = media_->getEntityByName("vivid-000-vid-cap"); + dev_ = new V4L2VideoDevice(entity->deviceNode()); + if (dev_->open()) { + cerr << "Failed to open video device" << endl; + return TestFail; + } + + const ControlInfoMap &infoMap = dev_->controls(); + + /* Test control enumeration. */ + if (infoMap.empty()) { + cerr << "Failed to enumerate controls" << endl; + return TestFail; + } + + if (infoMap.find(V4L2_CID_BRIGHTNESS) == infoMap.end() || + infoMap.find(V4L2_CID_CONTRAST) == infoMap.end()) { + cerr << "Missing controls" << endl; + return TestFail; + } + + return TestPass; + } + + int singleControlNoDelay() + { + std::unordered_map delays = { + { V4L2_CID_BRIGHTNESS, 0 }, + }; + std::unique_ptr delayed = + std::make_unique(dev_, delays); + ControlList ctrls; + + /* Reset control to value not used in test. */ + ctrls.set(V4L2_CID_BRIGHTNESS, 1); + dev_->setControls(&ctrls); + + /* Test control without delay are set at once. */ + for (int32_t i = 0; i < 100; i++) { + int32_t value = 100 + i; + + ctrls.set(V4L2_CID_BRIGHTNESS, value); + delayed->push(ctrls); + + delayed->applyControls(i); + + ControlList result = delayed->get(i); + int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); + if (brightness != value) { + cerr << "Failed single control without delay" + << " frame " << i + << " expected " << value + << " got " << brightness + << endl; + return TestFail; + } + } + + return TestPass; + } + + int singleControlWithDelay() + { + std::unordered_map delays = { + { V4L2_CID_BRIGHTNESS, 1 }, + }; + std::unique_ptr delayed = + std::make_unique(dev_, delays); + ControlList ctrls; + + /* Reset control to value that will be first in test. */ + int32_t expected = 4; + ctrls.set(V4L2_CID_BRIGHTNESS, expected); + dev_->setControls(&ctrls); + delayed->reset(); + + /* Test single control with delay. */ + for (int32_t i = 0; i < 100; i++) { + int32_t value = 10 + i; + + ctrls.set(V4L2_CID_BRIGHTNESS, value); + delayed->push(ctrls); + + delayed->applyControls(i); + + ControlList result = delayed->get(i); + int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); + if (brightness != expected) { + cerr << "Failed single control with delay" + << " frame " << i + << " expected " << expected + << " got " << brightness + << endl; + return TestFail; + } + + expected = value; + } + + return TestPass; + } + + int dualControlsWithDelay(uint32_t startOffset) + { + std::unordered_map delays = { + { V4L2_CID_BRIGHTNESS, 1 }, + { V4L2_CID_CONTRAST, 2 }, + }; + std::unique_ptr delayed = + std::make_unique(dev_, delays); + ControlList ctrls; + + /* Reset control to value that will be first two frames in test. */ + int32_t expected = 200; + ctrls.set(V4L2_CID_BRIGHTNESS, expected); + ctrls.set(V4L2_CID_CONTRAST, expected); + dev_->setControls(&ctrls); + delayed->reset(); + + /* Test dual control with delay. */ + for (int32_t i = 0; i < 100; i++) { + uint32_t frame = startOffset + i; + int32_t value = 10 + i; + + ctrls.set(V4L2_CID_BRIGHTNESS, value); + ctrls.set(V4L2_CID_CONTRAST, value); + delayed->push(ctrls); + + delayed->applyControls(frame); + + ControlList result = delayed->get(frame); + int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); + int32_t contrast = result.get(V4L2_CID_CONTRAST).get(); + if (brightness != expected || contrast != expected) { + cerr << "Failed dual controls" + << " frame " << frame + << " brightness " << brightness + << " contrast " << contrast + << " expected " << expected + << endl; + return TestFail; + } + + expected = i < 1 ? expected : value - 1; + } + + return TestPass; + } + + int dualControlsMultiQueue() + { + std::unordered_map delays = { + { V4L2_CID_BRIGHTNESS, 1 }, + { V4L2_CID_CONTRAST, 2 }, + }; + std::unique_ptr delayed = + std::make_unique(dev_, delays); + ControlList ctrls; + + /* Reset control to value that will be first two frames in test. */ + int32_t expected = 100; + ctrls.set(V4L2_CID_BRIGHTNESS, expected); + ctrls.set(V4L2_CID_CONTRAST, expected); + dev_->setControls(&ctrls); + delayed->reset(); + + /* + * Queue all controls before any fake frame start. Note we + * can't queue up more then the delayed controls history size + * which is 16. Where one spot is used by the reset control. + */ + for (unsigned int i = 0; i < 15; i++) { + int32_t value = 10 + i; + + ctrls.set(V4L2_CID_BRIGHTNESS, value); + ctrls.set(V4L2_CID_CONTRAST, value); + delayed->push(ctrls); + } + + /* Process all queued controls. */ + for (unsigned int i = 0; i < 16; i++) { + int32_t value = 10 + i; + + delayed->applyControls(i); + + ControlList result = delayed->get(i); + + int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); + int32_t contrast = result.get(V4L2_CID_CONTRAST).get(); + if (brightness != expected || contrast != expected) { + cerr << "Failed multi queue" + << " frame " << i + << " brightness " << brightness + << " contrast " << contrast + << " expected " << expected + << endl; + return TestFail; + } + + expected = i < 1 ? expected : value - 1; + } + + return TestPass; + } + + int run() override + { + int ret; + + /* Test single control without delay. */ + ret = singleControlNoDelay(); + if (ret) + return ret; + + /* Test single control with delay. */ + ret = singleControlWithDelay(); + if (ret) + return ret; + + /* Test dual controls with different delays. */ + ret = dualControlsWithDelay(0); + if (ret) + return ret; + + /* Test dual controls with non-zero sequence start. */ + ret = dualControlsWithDelay(10000); + if (ret) + return ret; + + /* Test dual controls with sequence number wraparound. */ + ret = dualControlsWithDelay(UINT32_MAX - 50); + if (ret) + return ret; + + /* Test control values produced faster then consumed. */ + ret = dualControlsMultiQueue(); + if (ret) + return ret; + + return TestPass; + } + + void cleanup() override + { + delete dev_; + } + +private: + std::unique_ptr enumerator_; + std::shared_ptr media_; + V4L2VideoDevice *dev_; +}; + +TEST_REGISTER(DelayedControlsTest) diff --git a/test/meson.build b/test/meson.build index 0a1d434e399641bb..a683a657a439b4ff 100644 --- a/test/meson.build +++ b/test/meson.build @@ -25,6 +25,7 @@ public_tests = [ internal_tests = [ ['byte-stream-buffer', 'byte-stream-buffer.cpp'], ['camera-sensor', 'camera-sensor.cpp'], + ['delayed_contols', 'delayed_contols.cpp'], ['event', 'event.cpp'], ['event-dispatcher', 'event-dispatcher.cpp'], ['event-thread', 'event-thread.cpp'], From patchwork Tue Dec 15 00:48:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10664 X-Patchwork-Delegate: niklas.soderlund@ragnatech.se Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 05EADBD80A for ; Tue, 15 Dec 2020 00:48:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 90736615AC; Tue, 15 Dec 2020 01:48:25 +0100 (CET) Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C618B61590 for ; Tue, 15 Dec 2020 01:48:23 +0100 (CET) X-Halon-ID: 3b713277-3e6f-11eb-a076-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de [79.202.36.88]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 3b713277-3e6f-11eb-a076-005056917f90; Tue, 15 Dec 2020 01:48:22 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com Date: Tue, 15 Dec 2020 01:48:06 +0100 Message-Id: <20201215004811.602429-4-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201215004811.602429-1-niklas.soderlund@ragnatech.se> References: <20201215004811.602429-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 3/8] libcamera: raspberrypi: Switch to DelayedControls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the libcamera core helper DelayedControls instead of the local StaggeredCtrl. The new helper is modeled after the StaggeredCtrl implementation and behaves the same. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- .../pipeline/raspberrypi/raspberrypi.cpp | 54 +++++++++---------- 1 file changed, 24 insertions(+), 30 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 7a5f5881b9b30129..a0186bee9926f945 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -27,6 +27,7 @@ #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/buffer.h" #include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/media_device.h" @@ -37,7 +38,6 @@ #include "dma_heaps.h" #include "rpi_stream.h" -#include "staggered_ctrl.h" namespace libcamera { @@ -178,8 +178,7 @@ public: RPi::DmaHeap dmaHeap_; FileDescriptor lsTable_; - RPi::StaggeredCtrl staggeredCtrl_; - uint32_t expectedSequence_; + std::unique_ptr delayedCtrls_; bool sensorMetadata_; /* @@ -766,11 +765,9 @@ int PipelineHandlerRPi::start(Camera *camera, [[maybe_unused]] ControlList *cont } /* Apply any gain/exposure settings that the IPA may have passed back. */ - ASSERT(data->staggeredCtrl_); if (result.operation & RPi::IPA_CONFIG_SENSOR) { - const ControlList &ctrls = result.controls[0]; - if (!data->staggeredCtrl_.set(ctrls)) - LOG(RPI, Error) << "V4L2 staggered set failed"; + ControlList ctrls = result.controls[0]; + data->unicam_[Unicam::Image].dev()->setControls(&ctrls); } if (result.operation & RPi::IPA_CONFIG_DROP_FRAMES) { @@ -802,13 +799,10 @@ int PipelineHandlerRPi::start(Camera *camera, [[maybe_unused]] ControlList *cont data->unicam_[Unicam::Image].dev()->setFrameStartEnabled(true); /* - * Write the last set of gain and exposure values to the camera before - * starting. First check that the staggered ctrl has been initialised - * by configure(). + * Reset the delayed controls with the gain and exposure values set by + * the IPA. */ - data->staggeredCtrl_.reset(); - data->staggeredCtrl_.write(); - data->expectedSequence_ = 0; + data->delayedCtrls_->reset(); data->state_ = RPiCameraData::State::Idle; @@ -1147,7 +1141,7 @@ void RPiCameraData::frameStarted(uint32_t sequence) LOG(RPI, Debug) << "frame start " << sequence; /* Write any controls for the next frame as soon as we can. */ - staggeredCtrl_.write(); + delayedCtrls_->applyControls(sequence); } int RPiCameraData::loadIPA() @@ -1230,18 +1224,22 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) * Setup our staggered control writer with the sensor default * gain and exposure delays. */ - if (!staggeredCtrl_) { - staggeredCtrl_.init(unicam_[Unicam::Image].dev(), - { { V4L2_CID_ANALOGUE_GAIN, result.data[resultIdx++] }, - { V4L2_CID_EXPOSURE, result.data[resultIdx++] } }); + + if (!delayedCtrls_) { + std::unordered_map delays = { + { V4L2_CID_ANALOGUE_GAIN, result.data[resultIdx++] }, + { V4L2_CID_EXPOSURE, result.data[resultIdx++] } + }; + + delayedCtrls_ = std::make_unique(unicam_[Unicam::Image].dev(), delays); + sensorMetadata_ = result.data[resultIdx++]; } } if (result.operation & RPi::IPA_CONFIG_SENSOR) { - const ControlList &ctrls = result.controls[0]; - if (!staggeredCtrl_.set(ctrls)) - LOG(RPI, Error) << "V4L2 staggered set failed"; + ControlList ctrls = result.controls[0]; + unicam_[Unicam::Image].dev()->setControls(&ctrls); } /* @@ -1270,8 +1268,8 @@ void RPiCameraData::queueFrameAction([[maybe_unused]] unsigned int frame, switch (action.operation) { case RPi::IPA_ACTION_V4L2_SET_STAGGERED: { const ControlList &controls = action.controls[0]; - if (!staggeredCtrl_.set(controls)) - LOG(RPI, Error) << "V4L2 staggered set failed"; + if (!delayedCtrls_->push(controls)) + LOG(RPI, Error) << "V4L2 delay set failed"; goto done; } @@ -1375,11 +1373,7 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) } else { embeddedQueue_.push(buffer); - std::unordered_map ctrl; - int offset = buffer->metadata().sequence - expectedSequence_; - staggeredCtrl_.get(ctrl, offset); - - expectedSequence_ = buffer->metadata().sequence + 1; + ControlList ctrl = delayedCtrls_->get(buffer->metadata().sequence); /* * Sensor metadata is unavailable, so put the expected ctrl @@ -1391,8 +1385,8 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) auto it = mappedEmbeddedBuffers_.find(bufferId); if (it != mappedEmbeddedBuffers_.end()) { uint32_t *mem = reinterpret_cast(it->second.maps()[0].data()); - mem[0] = ctrl[V4L2_CID_EXPOSURE]; - mem[1] = ctrl[V4L2_CID_ANALOGUE_GAIN]; + mem[0] = ctrl.get(V4L2_CID_EXPOSURE).get(); + mem[1] = ctrl.get(V4L2_CID_ANALOGUE_GAIN).get(); } else { LOG(RPI, Warning) << "Failed to find embedded buffer " << bufferId; From patchwork Tue Dec 15 00:48:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10665 X-Patchwork-Delegate: niklas.soderlund@ragnatech.se Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6F0AFBD80A for ; Tue, 15 Dec 2020 00:48:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4C375615D5; Tue, 15 Dec 2020 01:48:29 +0100 (CET) Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5FA2F61590 for ; Tue, 15 Dec 2020 01:48:27 +0100 (CET) X-Halon-ID: 3d4daf92-3e6f-11eb-a076-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de [79.202.36.88]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 3d4daf92-3e6f-11eb-a076-005056917f90; Tue, 15 Dec 2020 01:48:26 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com Date: Tue, 15 Dec 2020 01:48:08 +0100 Message-Id: <20201215004811.602429-6-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201215004811.602429-1-niklas.soderlund@ragnatech.se> References: <20201215004811.602429-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 5/8] libcamera: camera_sensor: Expose the camera device X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Expose the device backing the CameraSensor instance. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- * Changes since v3 - New, needed to be able to move the DelayedControls handling out of CameraSensor. --- include/libcamera/internal/camera_sensor.h | 2 ++ src/libcamera/camera_sensor.cpp | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index f80d836161a564e7..a70e024aa327f69b 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -61,6 +61,8 @@ public: ControlList getControls(const std::vector &ids); int setControls(ControlList *ctrls); + V4L2Subdevice *device() { return subdev_.get(); } + const ControlList &properties() const { return properties_; } int sensorInfo(CameraSensorInfo *info) const; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 1628ba9c892b0eaf..df45b2b803617bfd 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -486,6 +486,12 @@ ControlList CameraSensor::getControls(const std::vector &ids) return subdev_->getControls(ids); } +/** + * \fn CameraSensor::device() + * \brief Retrieve the camera sensor device + * \return The camera sensor device + */ + /** * \fn CameraSensor::properties() * \brief Retrieve the camera sensor properties From patchwork Tue Dec 15 00:48:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10666 X-Patchwork-Delegate: niklas.soderlund@ragnatech.se Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C565FC0F1B for ; Tue, 15 Dec 2020 00:48:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6C09161D25; Tue, 15 Dec 2020 01:48:29 +0100 (CET) Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 25F2E61592 for ; Tue, 15 Dec 2020 01:48:28 +0100 (CET) X-Halon-ID: 3e2af113-3e6f-11eb-a076-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de [79.202.36.88]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 3e2af113-3e6f-11eb-a076-005056917f90; Tue, 15 Dec 2020 01:48:27 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com Date: Tue, 15 Dec 2020 01:48:09 +0100 Message-Id: <20201215004811.602429-7-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201215004811.602429-1-niklas.soderlund@ragnatech.se> References: <20201215004811.602429-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 6/8] libcamera: pipeline: rkisp1: Use delayed controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Instead of setting controls using the RkISP1 local Timeline helper use the DelayedControls. The result is the same, the controls are applied with a delay. The values of the delays are however different between the two methods. The values used in the Timeline solution were chosen after some experimentation and the values used in DelayedControls are taken from a generic sensor. None of the two are a perfect match as the delays can be different for different sensors used with the pipeline. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- * Changes since v3 - Rewrite to not use CameraSensor. --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 43 +++++++++++++----------- 1 file changed, 23 insertions(+), 20 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 4d98c9027f42c759..513a60b04e5f2e21 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -23,6 +23,7 @@ #include #include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/log.h" @@ -136,6 +137,7 @@ public: Stream mainPathStream_; Stream selfPathStream_; CameraSensor *sensor_; + std::unique_ptr delayedCtrls_; unsigned int frame_; std::vector ipaBuffers_; RkISP1Frames frameInfo_; @@ -345,23 +347,6 @@ RkISP1FrameInfo *RkISP1Frames::find(Request *request) return nullptr; } -class RkISP1ActionSetSensor : public FrameAction -{ -public: - RkISP1ActionSetSensor(unsigned int frame, CameraSensor *sensor, const ControlList &controls) - : FrameAction(frame, SetSensor), sensor_(sensor), controls_(controls) {} - -protected: - void run() override - { - sensor_->setControls(&controls_); - } - -private: - CameraSensor *sensor_; - ControlList controls_; -}; - class RkISP1ActionQueueBuffers : public FrameAction { public: @@ -429,9 +414,7 @@ void RkISP1CameraData::queueFrameAction(unsigned int frame, switch (action.operation) { case RKISP1_IPA_ACTION_V4L2_SET: { const ControlList &controls = action.controls[0]; - timeline_.scheduleAction(std::make_unique(frame, - sensor_, - controls)); + delayedCtrls_->push(controls); break; } case RKISP1_IPA_ACTION_PARAM_FILLED: { @@ -898,6 +881,8 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] ControlList *c }; } + isp_->setFrameStartEnabled(true); + activeCamera_ = camera; /* Inform IPA of stream configuration and sensor controls. */ @@ -925,6 +910,8 @@ void PipelineHandlerRkISP1::stop(Camera *camera) RkISP1CameraData *data = cameraData(camera); int ret; + isp_->setFrameStartEnabled(false); + selfPath_.stop(); mainPath_.stop(); @@ -1043,6 +1030,22 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); + /* + * \todo Read dealy values from the sensor itself or from a + * a sensor database. For now use generic values taken from + * the Raspberry Pi and listed as generic values. + */ + std::unordered_map delays = { + { V4L2_CID_ANALOGUE_GAIN, 1 }, + { V4L2_CID_EXPOSURE, 2 }, + }; + + data->delayedCtrls_ = + std::make_unique(data->sensor_->device(), + delays); + isp_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + ret = data->loadIPA(); if (ret) return ret; From patchwork Tue Dec 15 00:48:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10667 X-Patchwork-Delegate: niklas.soderlund@ragnatech.se Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0F8E4BD80A for ; Tue, 15 Dec 2020 00:48:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DB6E361592; Tue, 15 Dec 2020 01:48:31 +0100 (CET) Received: from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net [195.74.38.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1271961591 for ; Tue, 15 Dec 2020 01:48:29 +0100 (CET) X-Halon-ID: 3eab7125-3e6f-11eb-a076-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de [79.202.36.88]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 3eab7125-3e6f-11eb-a076-005056917f90; Tue, 15 Dec 2020 01:48:28 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org, naush@raspberrypi.com Date: Tue, 15 Dec 2020 01:48:10 +0100 Message-Id: <20201215004811.602429-8-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201215004811.602429-1-niklas.soderlund@ragnatech.se> References: <20201215004811.602429-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 7/8] libcamera: pipeline: rkisp1: Use SOF event to warn about late parameters X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In the Timeline approach the idea was to delay queuing buffers to the device until the IPA had a chance to prepare the parameters buffer. A check was still added to warn if the IPA queued buffers before the parameters buffer was filled in. This check happened much sooner then needed as the parameter buffers does not have to be ready when the buffer is queued but just before it's consumed. As the pipeline now has true knowledge of each SOF we can move the check there and remove the delaying of queuing of buffers. This change really speeds up the IPA reactions as the delays used in the Timeline where approximated while with this change they are driven by events reported by the device. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 78 ++++++++---------------- 1 file changed, 24 insertions(+), 54 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 513a60b04e5f2e21..3851d94e7b133356 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -41,7 +41,6 @@ namespace libcamera { LOG_DEFINE_CATEGORY(RkISP1) class PipelineHandlerRkISP1; -class RkISP1ActionQueueBuffers; class RkISP1CameraData; enum RkISP1ActionType { @@ -202,7 +201,6 @@ private: PipelineHandler::cameraData(camera)); } - friend RkISP1ActionQueueBuffers; friend RkISP1CameraData; friend RkISP1Frames; @@ -213,6 +211,7 @@ private: void bufferReady(FrameBuffer *buffer); void paramReady(FrameBuffer *buffer); void statReady(FrameBuffer *buffer); + void frameStart(uint32_t sequence); int allocateBuffers(Camera *camera); int freeBuffers(Camera *camera); @@ -347,53 +346,6 @@ RkISP1FrameInfo *RkISP1Frames::find(Request *request) return nullptr; } -class RkISP1ActionQueueBuffers : public FrameAction -{ -public: - RkISP1ActionQueueBuffers(unsigned int frame, RkISP1CameraData *data, - PipelineHandlerRkISP1 *pipe) - : FrameAction(frame, QueueBuffers), data_(data), pipe_(pipe) - { - } - -protected: - void run() override - { - RkISP1FrameInfo *info = data_->frameInfo_.find(frame()); - if (!info) - LOG(RkISP1, Fatal) << "Frame not known"; - - /* - * \todo: If parameters are not filled a better method to handle - * the situation than queuing a buffer with unknown content - * should be used. - * - * It seems excessive to keep an internal zeroed scratch - * parameters buffer around as this should not happen unless the - * devices is under too much load. Perhaps failing the request - * and returning it to the application with an error code is - * better than queue it to hardware? - */ - if (!info->paramFilled) - LOG(RkISP1, Error) - << "Parameters not ready on time for frame " - << frame(); - - pipe_->param_->queueBuffer(info->paramBuffer); - pipe_->stat_->queueBuffer(info->statBuffer); - - if (info->mainPathBuffer) - pipe_->mainPath_.queueBuffer(info->mainPathBuffer); - - if (info->selfPathBuffer) - pipe_->selfPath_.queueBuffer(info->selfPathBuffer); - } - -private: - RkISP1CameraData *data_; - PipelineHandlerRkISP1 *pipe_; -}; - int RkISP1CameraData::loadIPA() { ipa_ = IPAManager::createIPA(pipe_, 1, 1); @@ -950,9 +902,14 @@ int PipelineHandlerRkISP1::queueRequestDevice(Camera *camera, Request *request) op.controls = { request->controls() }; data->ipa_->processEvent(op); - data->timeline_.scheduleAction(std::make_unique(data->frame_, - data, - this)); + param_->queueBuffer(info->paramBuffer); + stat_->queueBuffer(info->statBuffer); + + if (info->mainPathBuffer) + mainPath_.queueBuffer(info->mainPathBuffer); + + if (info->selfPathBuffer) + selfPath_.queueBuffer(info->selfPathBuffer); data->frame_++; @@ -1102,6 +1059,7 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) mainPath_.bufferReady().connect(this, &PipelineHandlerRkISP1::bufferReady); selfPath_.bufferReady().connect(this, &PipelineHandlerRkISP1::bufferReady); stat_->bufferReady.connect(this, &PipelineHandlerRkISP1::statReady); + isp_->frameStart.connect(this, &PipelineHandlerRkISP1::frameStart); param_->bufferReady.connect(this, &PipelineHandlerRkISP1::paramReady); /* @@ -1181,8 +1139,6 @@ void PipelineHandlerRkISP1::statReady(FrameBuffer *buffer) if (!info) return; - data->timeline_.bufferReady(buffer); - if (data->frame_ <= buffer->metadata().sequence) data->frame_ = buffer->metadata().sequence + 1; @@ -1192,6 +1148,20 @@ void PipelineHandlerRkISP1::statReady(FrameBuffer *buffer) data->ipa_->processEvent(op); } +void PipelineHandlerRkISP1::frameStart(uint32_t sequence) +{ + if (!activeCamera_) + return; + + RkISP1CameraData *data = cameraData(activeCamera_); + + RkISP1FrameInfo *info = data->frameInfo_.find(sequence); + if (!info || !info->paramFilled) + LOG(RkISP1, Info) + << "Parameters not ready on time for frame " + << sequence; +} + REGISTER_PIPELINE_HANDLER(PipelineHandlerRkISP1) } /* namespace libcamera */