From patchwork Sun Feb 16 16:16:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 2836 Return-Path: Received: from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net [195.74.38.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 632DE60438 for ; Sun, 16 Feb 2020 17:17:34 +0100 (CET) X-Halon-ID: d64c5259-50d7-11ea-9f85-005056917a89 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (p4fca2392.dip0.t-ipconnect.de [79.202.35.146]) by bin-vsp-out-01.atm.binero.net (Halon) with ESMTPA id d64c5259-50d7-11ea-9f85-005056917a89; Sun, 16 Feb 2020 17:17:33 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Sun, 16 Feb 2020 17:16:41 +0100 Message-Id: <20200216161642.152263-2-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200216161642.152263-1-niklas.soderlund@ragnatech.se> References: <20200216161642.152263-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/2] libcamera: V4L2BufferCache: Improve cache eviction strategy X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Feb 2020 16:17:34 -0000 The strategy used to find a free cache entry in the first implementation was not the smartest, it picked the first free entry. This lead to unwanted performance issues as they cache was not used as good as it could for imported buffers. Improve this by adding a last usage timestamp to the cache entries and change the eviction strategy to use the oldest free entry instead of the first one it finds. Signed-off-by: Niklas Söderlund Reviewed-by: Naushir Patuck --- src/libcamera/include/v4l2_videodevice.h | 1 + src/libcamera/v4l2_videodevice.cpp | 9 ++++++--- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/src/libcamera/include/v4l2_videodevice.h b/src/libcamera/include/v4l2_videodevice.h index fcf072641420dacf..f04feed87b24f28f 100644 --- a/src/libcamera/include/v4l2_videodevice.h +++ b/src/libcamera/include/v4l2_videodevice.h @@ -125,6 +125,7 @@ private: bool operator==(const FrameBuffer &buffer); bool free; + utils::time_point lastUsed; private: struct Plane { diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 99470ce11421c77c..9a4d2479b20f5e27 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -205,6 +205,7 @@ int V4L2BufferCache::get(const FrameBuffer &buffer) { bool hit = false; int use = -1; + utils::time_point oldest = utils::clock::now(); for (unsigned int index = 0; index < cache_.size(); index++) { const Entry &entry = cache_[index]; @@ -212,8 +213,10 @@ int V4L2BufferCache::get(const FrameBuffer &buffer) if (!entry.free) continue; - if (use < 0) + if (cache_[index].lastUsed < oldest) { use = index; + oldest = cache_[index].lastUsed; + } /* Try to find a cache hit by comparing the planes. */ if (cache_[index] == buffer) { @@ -245,12 +248,12 @@ void V4L2BufferCache::put(unsigned int index) } V4L2BufferCache::Entry::Entry() - : free(true) + : free(true), lastUsed(utils::clock::now()) { } V4L2BufferCache::Entry::Entry(bool free, const FrameBuffer &buffer) - : free(free) + : free(free), lastUsed(utils::clock::now()) { for (const FrameBuffer::Plane &plane : buffer.planes()) planes_.emplace_back(plane); From patchwork Sun Feb 16 16:16:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 2837 Return-Path: Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C9ACF61A27 for ; Sun, 16 Feb 2020 17:17:35 +0100 (CET) X-Halon-ID: d6b90541-50d7-11ea-9f85-005056917a89 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (p4fca2392.dip0.t-ipconnect.de [79.202.35.146]) by bin-vsp-out-01.atm.binero.net (Halon) with ESMTPA id d6b90541-50d7-11ea-9f85-005056917a89; Sun, 16 Feb 2020 17:17:34 +0100 (CET) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Sun, 16 Feb 2020 17:16:42 +0100 Message-Id: <20200216161642.152263-3-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200216161642.152263-1-niklas.soderlund@ragnatech.se> References: <20200216161642.152263-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/2] test: v4l2_videodevice: Add test for V4L2BufferCache X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Feb 2020 16:17:36 -0000 Add test to test the different modes and situations the V4L2BufferCache can be put to. The tests verify that a FrameBuffer used with the cache results in a V4L2 video device index can be resolved, and that the cache implementation is capable of keeping buffers in a hot state. Signed-off-by: Niklas Söderlund --- test/v4l2_videodevice/buffer_cache.cpp | 286 +++++++++++++++++++++++++ test/v4l2_videodevice/meson.build | 1 + 2 files changed, 287 insertions(+) create mode 100644 test/v4l2_videodevice/buffer_cache.cpp diff --git a/test/v4l2_videodevice/buffer_cache.cpp b/test/v4l2_videodevice/buffer_cache.cpp new file mode 100644 index 0000000000000000..6244361130525d29 --- /dev/null +++ b/test/v4l2_videodevice/buffer_cache.cpp @@ -0,0 +1,286 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * Test the buffer cache different operation modes + */ + +#include +#include +#include + +#include + +#include "device_enumerator.h" +#include "media_device.h" +#include "v4l2_videodevice.h" + +#include "test.h" + +using namespace libcamera; + +namespace { + +/* A provider of source of external buffers, suitable for to use by V4L2BufferCache. */ +class BufferSource +{ +public: + BufferSource() + : video_(nullptr) + { + } + + ~BufferSource() + { + if (video_) { + video_->releaseBuffers(); + video_->close(); + } + + delete video_; + video_ = nullptr; + + if (media_) + media_->release(); + } + + int allocate(const StreamConfiguration &config) + { + /* Locate and open the video device. */ + std::string videoDeviceName = "vivid-000-vid-out"; + + std::unique_ptr enumerator = + DeviceEnumerator::create(); + if (!enumerator) { + std::cout << "Failed to create device enumerator" << std::endl; + return TestFail; + } + + if (enumerator->enumerate()) { + std::cout << "Failed to enumerate media devices" << std::endl; + return TestFail; + } + + DeviceMatch dm("vivid"); + dm.add(videoDeviceName); + + media_ = enumerator->search(dm); + if (!media_) { + std::cout << "No vivid output device available" << std::endl; + return TestSkip; + } + + video_ = V4L2VideoDevice::fromEntityName(media_.get(), videoDeviceName); + if (!video_) { + std::cout << "Unable to open " << videoDeviceName << std::endl; + return TestFail; + } + + if (video_->open()) + return TestFail; + + /* Configure the format. */ + V4L2DeviceFormat format; + if (video_->getFormat(&format)) { + std::cout << "Failed to get format on output device" << std::endl; + return TestFail; + } + + format.size = config.size; + format.fourcc = V4L2VideoDevice::toV4L2Fourcc(config.pixelFormat, false); + if (video_->setFormat(&format)) { + std::cout << "Failed to set format on output device" << std::endl; + return TestFail; + } + + if (video_->exportBuffers(config.bufferCount, &buffers_) < 0) { + std::cout << "Failed to export buffers" << std::endl; + return TestFail; + } + + return TestPass; + } + + const std::vector> &buffers() + { + return buffers_; + } + +private: + std::shared_ptr media_; + V4L2VideoDevice *video_; + std::vector> buffers_; +}; + +class BufferCacheTest : public Test +{ +public: + /* + * Test that a cache with the same size as there are buffers results in + * a sequential run over; 0, 1, 2, 3, 4, 0, 1, 2, 3, 4, ... + * + * Test only valid when the cache size is as least as big as there are + * number of buffers. + */ + int testSequential(V4L2BufferCache *cache, + const std::vector> &buffers) + { + for (unsigned int i = 0; i < buffers.size() * 100; i++) { + int nBuffer = i % buffers.size(); + int index = cache->get(*buffers[nBuffer].get()); + + if (index != nBuffer) { + std::cout << "Expected index " << nBuffer + << " got " << index << std::endl; + return TestFail; + } + + cache->put(index); + } + + return TestPass; + } + + /* + * Test that randomly putting buffers to the cache always results in a + * valid index. + */ + int testRandom(V4L2BufferCache *cache, + const std::vector> &buffers) + { + for (unsigned int i = 0; i < buffers.size() * 100; i++) { + int nBuffer = rand() % buffers.size(); + int index = cache->get(*buffers[nBuffer].get()); + + if (index < 0) { + std::cout << "Failed lookup from cache" + << std::endl; + return TestFail; + } + + cache->put(index); + } + + return TestPass; + } + + /* + * Test that using a buffer more frequently keeps it hot in the cache at + * all times. + */ + int testHot(V4L2BufferCache *cache, + const std::vector> &buffers, + unsigned int hotFrequency) + { + /* Run the random test on the cache to make it messy. */ + if (testRandom(cache, buffers) != TestPass) + return TestFail; + + /* Pick a hot buffer at random and store its index */ + int hotBuffer = rand() % buffers.size(); + int hotIndex = cache->get(*buffers[hotBuffer].get()); + cache->put(hotIndex); + + /* + * Queue hot buffer at the requested frequency and make sure + * it stays hot. + */ + for (unsigned int i = 0; i < buffers.size() * 100; i++) { + int nBuffer, index; + bool hotQueue = i % hotFrequency == 0; + + if (hotQueue) + nBuffer = hotBuffer; + else + nBuffer = rand() % buffers.size(); + + index = cache->get(*buffers[nBuffer].get()); + + if (index < 0) { + std::cout << "Failed lookup from cache" + << std::endl; + return TestFail; + } + + if (hotQueue && index != hotIndex) { + std::cout << "Hot buffer got cold" + << std::endl; + return TestFail; + } + + cache->put(index); + } + + return TestPass; + } + + int run() override + { + const unsigned int numBuffers = 8; + + StreamConfiguration cfg; + cfg.pixelFormat = V4L2_PIX_FMT_YUYV; + cfg.size = Size(600, 800); + cfg.bufferCount = numBuffers; + + BufferSource source; + int ret = source.allocate(cfg); + if (ret != TestPass) + return ret; + + if (source.buffers().size() != numBuffers) { + std::cout << "Got " << source.buffers().size() + << " buffers, expected " << numBuffers + << std::endl; + return TestFail; + } + + /* + * Test cache of same size as there are buffers, cache is + * created from a list of buffers and will be pre-populated. + */ + V4L2BufferCache cacheFromBuffers(source.buffers()); + + if (testSequential(&cacheFromBuffers, source.buffers()) != TestPass) + return TestFail; + + if (testRandom(&cacheFromBuffers, source.buffers()) != TestPass) + return TestFail; + + if (testHot(&cacheFromBuffers, source.buffers(), numBuffers - 1) != TestPass) + return TestFail; + + /* + * Test cache of same size as there are buffers, cache is not + * pre-populated. + */ + V4L2BufferCache cacheFromNumbers(numBuffers); + + if (testSequential(&cacheFromNumbers, source.buffers()) != TestPass) + return TestFail; + + if (testRandom(&cacheFromNumbers, source.buffers()) != TestPass) + return TestFail; + + if (testHot(&cacheFromNumbers, source.buffers(), numBuffers - 1) != TestPass) + return TestFail; + + /* + * Test cache half the size of number of buffers used, cache is + * not pre-populated. + */ + V4L2BufferCache cacheHalf(numBuffers / 2); + + if (testRandom(&cacheHalf, source.buffers()) != TestPass) + return TestFail; + + if (testHot(&cacheHalf, source.buffers(), numBuffers / 2 - 1) != TestPass) + return TestFail; + + return TestPass; + } +}; + +} /* namespace */ + +TEST_REGISTER(BufferCacheTest); diff --git a/test/v4l2_videodevice/meson.build b/test/v4l2_videodevice/meson.build index 5c52da7219c21cc3..685fcf6d16d72182 100644 --- a/test/v4l2_videodevice/meson.build +++ b/test/v4l2_videodevice/meson.build @@ -5,6 +5,7 @@ v4l2_videodevice_tests = [ [ 'controls', 'controls.cpp' ], [ 'formats', 'formats.cpp' ], [ 'request_buffers', 'request_buffers.cpp' ], + [ 'buffer_cache', 'buffer_cache.cpp' ], [ 'stream_on_off', 'stream_on_off.cpp' ], [ 'capture_async', 'capture_async.cpp' ], [ 'buffer_sharing', 'buffer_sharing.cpp' ],