From patchwork Fri Sep 18 09:42:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9646 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 09AFBC3B5B for ; Fri, 18 Sep 2020 09:44:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C370862FBC; Fri, 18 Sep 2020 11:43:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="k61d1c1N"; dkim-atps=neutral Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4C53360367 for ; Fri, 18 Sep 2020 11:43:56 +0200 (CEST) Received: by mail-wr1-x42e.google.com with SMTP id z1so4954908wrt.3 for ; Fri, 18 Sep 2020 02:43:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RwctxmJSFTe6+XjCaT5M2+31UMCk6sU6+3dJg6rPVGI=; b=k61d1c1NSw1X4LmkbnEBhr7Bx4KRtQU4EQSKMyk/Eh5+n3+DslNCYU7bKxxq+vCrAx mjHAsWrhYmBZodJyWBHY+vuPD3gatlixWCMVW5/q57fVPRtNuJVX5+sxrI6vWrYCloCT esWq4pPXsNGwtlJDMD2ibHMMvceUC4Gapbs6h/m2RDDWGuXNyKlFbqb33E9y3mmQuO3s 3kDI7HN8dv2JVl2WBCgJnst+jmnKVm7AHsJWYocNVxIX1YPACA7vSsFQEAQrDvFCWfhJ h69A1pyPTayp43XORpyPKQ51OCgiCGM/2eYuG90ajVMSOHkJ+xesGPo4F6leolfKIBU/ FXVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RwctxmJSFTe6+XjCaT5M2+31UMCk6sU6+3dJg6rPVGI=; b=M56GdSwid3ClofNeeD/a1Xxo+9PX3FVxKCsYns9ogxQIIZtfr+l3WuL+IwQyyRHhYh k+EojSYFDpYX8mdiiru3qvortwlTAqVsE0NRXmsxiIr6560n7vHzUn7q+oxAf3r17/vT w5UVKd1ZZhFnQyqYDvBor9fKm3SMzBMxXbIiteR/Bc82UzK4aZ3L+BgzabOC9keH8wLJ GBhnOHR5S0ln6rQ+jZOQV3eOOi/UMMZSNZM5cjGe8cmNLC3cbZAwkQG+Pg9r5LwU7R41 /vPRaDjpWv/C/WNsTHmvcuZvllL5ocXwHM9MGvpQ+AwE25ABsYpEdHEP0jPfo/ydDaKS ZfuA== X-Gm-Message-State: AOAM532+IN6zMgDkjLq3WQJKjOcFI5c75sNpKybGRcEiiAF3JFCw/E70 SxqjQdr43w618U6PydfADt0mqU0OudfjHA== X-Google-Smtp-Source: ABdhPJy4GWehmPpVPZCi9KdY7vcitG36y3mt+j9zZy1l3gLkNQfqbhaEkhJ77ugasYv7XuC1ES9gOw== X-Received: by 2002:a5d:67d2:: with SMTP id n18mr36603553wrw.223.1600422235360; Fri, 18 Sep 2020 02:43:55 -0700 (PDT) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id n2sm4792472wma.29.2020.09.18.02.43.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 02:43:54 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Sep 2020 10:42:22 +0100 Message-Id: <20200918094233.5273-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918094233.5273-1-naush@raspberrypi.com> References: <20200918094233.5273-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 01/12] libcamera: pipeline: raspberrypi: Move RPiStream into a separate file X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Put RPiStream into the RPi namespace and add a new log category (RPISTREAM). Reorder methods into logical groups for readability. There are no functional changes in this commit. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Tested-by: David Plowman --- .../pipeline/raspberrypi/meson.build | 1 + .../pipeline/raspberrypi/raspberrypi.cpp | 193 ++---------------- .../pipeline/raspberrypi/rpi_stream.cpp | 116 +++++++++++ .../pipeline/raspberrypi/rpi_stream.h | 107 ++++++++++ 4 files changed, 243 insertions(+), 174 deletions(-) create mode 100644 src/libcamera/pipeline/raspberrypi/rpi_stream.cpp create mode 100644 src/libcamera/pipeline/raspberrypi/rpi_stream.h diff --git a/src/libcamera/pipeline/raspberrypi/meson.build b/src/libcamera/pipeline/raspberrypi/meson.build index ae0aed3b..7c5b6ff7 100644 --- a/src/libcamera/pipeline/raspberrypi/meson.build +++ b/src/libcamera/pipeline/raspberrypi/meson.build @@ -3,5 +3,6 @@ libcamera_sources += files([ 'dma_heaps.cpp', 'raspberrypi.cpp', + 'rpi_stream.cpp', 'staggered_ctrl.cpp', ]) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index ce43af34..3d94b3ed 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -19,7 +19,6 @@ #include #include #include -#include #include @@ -33,6 +32,7 @@ #include "libcamera/internal/v4l2_videodevice.h" #include "dma_heaps.h" +#include "rpi_stream.h" #include "staggered_ctrl.h" namespace libcamera { @@ -122,165 +122,10 @@ V4L2DeviceFormat findBestMode(V4L2VideoDevice::Formats &formatsMap, return bestMode; } -} /* namespace */ - -/* - * Device stream abstraction for either an internal or external stream. - * Used for both Unicam and the ISP. - */ -class RPiStream : public Stream -{ -public: - RPiStream() - { - } - - RPiStream(const char *name, MediaEntity *dev, bool importOnly = false) - : external_(false), importOnly_(importOnly), name_(name), - dev_(std::make_unique(dev)) - { - } - - V4L2VideoDevice *dev() const - { - return dev_.get(); - } - - void setExternal(bool external) - { - external_ = external; - } - - bool isExternal() const - { - /* - * Import streams cannot be external. - * - * RAW capture is a special case where we simply copy the RAW - * buffer out of the request. All other buffer handling happens - * as if the stream is internal. - */ - return external_ && !importOnly_; - } - - bool isImporter() const - { - return importOnly_; - } - - void reset() - { - external_ = false; - internalBuffers_.clear(); - } - - std::string name() const - { - return name_; - } - - void setExternalBuffers(std::vector> *buffers) - { - externalBuffers_ = buffers; - } - - const std::vector> *getBuffers() const - { - return external_ ? externalBuffers_ : &internalBuffers_; - } - - void releaseBuffers() - { - dev_->releaseBuffers(); - if (!external_ && !importOnly_) - internalBuffers_.clear(); - } - - int importBuffers(unsigned int count) - { - return dev_->importBuffers(count); - } - - int allocateBuffers(unsigned int count) - { - return dev_->allocateBuffers(count, &internalBuffers_); - } - - int queueBuffers() - { - if (external_) - return 0; - - for (auto &b : internalBuffers_) { - int ret = dev_->queueBuffer(b.get()); - if (ret) { - LOG(RPI, Error) << "Failed to queue buffers for " - << name_; - return ret; - } - } - - return 0; - } - - bool findFrameBuffer(FrameBuffer *buffer) const - { - auto start = external_ ? externalBuffers_->begin() : internalBuffers_.begin(); - auto end = external_ ? externalBuffers_->end() : internalBuffers_.end(); - - if (importOnly_) - return false; - - if (std::find_if(start, end, - [buffer](std::unique_ptr const &ref) { return ref.get() == buffer; }) != end) - return true; - - return false; - } - -private: - /* - * Indicates that this stream is active externally, i.e. the buffers - * are provided by the application. - */ - bool external_; - /* Indicates that this stream only imports buffers, e.g. ISP input. */ - bool importOnly_; - /* Stream name identifier. */ - std::string name_; - /* The actual device stream. */ - std::unique_ptr dev_; - /* Internally allocated framebuffers associated with this device stream. */ - std::vector> internalBuffers_; - /* Externally allocated framebuffers associated with this device stream. */ - std::vector> *externalBuffers_; -}; - -/* - * The following class is just a convenient (and typesafe) array of device - * streams indexed with an enum class. - */ enum class Unicam : unsigned int { Image, Embedded }; enum class Isp : unsigned int { Input, Output0, Output1, Stats }; -template -class RPiDevice : public std::array -{ -private: - constexpr auto index(E e) const noexcept - { - return static_cast>(e); - } -public: - RPiStream &operator[](E e) - { - return std::array::operator[](index(e)); - } - const RPiStream &operator[](E e) const - { - return std::array::operator[](index(e)); - } -}; +} /* namespace */ class RPiCameraData : public CameraData { @@ -304,15 +149,15 @@ public: void ispOutputDequeue(FrameBuffer *buffer); void clearIncompleteRequests(); - void handleStreamBuffer(FrameBuffer *buffer, const RPiStream *stream); + void handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream); void handleState(); CameraSensor *sensor_; /* Array of Unicam and ISP device streams and associated buffers/streams. */ - RPiDevice unicam_; - RPiDevice isp_; + RPi::RPiDevice unicam_; + RPi::RPiDevice isp_; /* The vector below is just for convenience when iterating over all streams. */ - std::vector streams_; + std::vector streams_; /* Buffers passed to the IPA. */ std::vector ipaBuffers_; @@ -761,7 +606,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) int PipelineHandlerRPi::exportFrameBuffers([[maybe_unused]] Camera *camera, Stream *stream, std::vector> *buffers) { - RPiStream *s = static_cast(stream); + RPi::RPiStream *s = static_cast(stream); unsigned int count = stream->configuration().bufferCount; int ret = s->dev()->exportBuffers(count, buffers); @@ -908,14 +753,14 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) return false; /* Locate and open the unicam video streams. */ - data->unicam_[Unicam::Embedded] = RPiStream("Unicam Embedded", unicam_->getEntityByName("unicam-embedded")); - data->unicam_[Unicam::Image] = RPiStream("Unicam Image", unicam_->getEntityByName("unicam-image")); + data->unicam_[Unicam::Embedded] = RPi::RPiStream("Unicam Embedded", unicam_->getEntityByName("unicam-embedded")); + data->unicam_[Unicam::Image] = RPi::RPiStream("Unicam Image", unicam_->getEntityByName("unicam-image")); /* Tag the ISP input stream as an import stream. */ - data->isp_[Isp::Input] = RPiStream("ISP Input", isp_->getEntityByName("bcm2835-isp0-output0"), true); - data->isp_[Isp::Output0] = RPiStream("ISP Output0", isp_->getEntityByName("bcm2835-isp0-capture1")); - data->isp_[Isp::Output1] = RPiStream("ISP Output1", isp_->getEntityByName("bcm2835-isp0-capture2")); - data->isp_[Isp::Stats] = RPiStream("ISP Stats", isp_->getEntityByName("bcm2835-isp0-capture3")); + data->isp_[Isp::Input] = RPi::RPiStream("ISP Input", isp_->getEntityByName("bcm2835-isp0-output0"), true); + data->isp_[Isp::Output0] = RPi::RPiStream("ISP Output0", isp_->getEntityByName("bcm2835-isp0-capture1")); + data->isp_[Isp::Output1] = RPi::RPiStream("ISP Output1", isp_->getEntityByName("bcm2835-isp0-capture2")); + data->isp_[Isp::Stats] = RPi::RPiStream("ISP Stats", isp_->getEntityByName("bcm2835-isp0-capture3")); /* This is just for convenience so that we can easily iterate over all streams. */ for (auto &stream : data->unicam_) @@ -1013,7 +858,7 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) */ unsigned int maxBuffers = 0; for (const Stream *s : camera->streams()) - if (static_cast(s)->isExternal()) + if (static_cast(s)->isExternal()) maxBuffers = std::max(maxBuffers, s->configuration().bufferCount); for (auto const stream : data->streams_) { @@ -1257,12 +1102,12 @@ done: void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) { - const RPiStream *stream = nullptr; + const RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPiStream const &s : unicam_) { + for (RPi::RPiStream const &s : unicam_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1318,12 +1163,12 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) { - const RPiStream *stream = nullptr; + const RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPiStream const &s : isp_) { + for (RPi::RPiStream const &s : isp_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1404,7 +1249,7 @@ void RPiCameraData::clearIncompleteRequests() } } -void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPiStream *stream) +void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream) { if (stream->isExternal()) { if (!dropFrame_) { diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp new file mode 100644 index 00000000..2e52bd5c --- /dev/null +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -0,0 +1,116 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi (Trading) Ltd. + * + * rpi_stream.cpp - Raspberry Pi device stream abstraction class. + */ +#include "rpi_stream.h" + +#include "libcamera/internal/log.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(RPISTREAM) + +namespace RPi { + +V4L2VideoDevice *RPiStream::dev() const +{ + return dev_.get(); +} + +std::string RPiStream::name() const +{ + return name_; +} + +void RPiStream::reset() +{ + external_ = false; + internalBuffers_.clear(); +} + +bool RPiStream::isImporter() const +{ + return importOnly_; +} + +void RPiStream::setExternal(bool external) +{ + external_ = external; +} + +bool RPiStream::isExternal() const +{ + /* + * Import streams cannot be external. + * + * RAW capture is a special case where we simply copy the RAW + * buffer out of the request. All other buffer handling happens + * as if the stream is internal. + */ + return external_ && !importOnly_; +} + +void RPiStream::setExternalBuffers(std::vector> *buffers) +{ + externalBuffers_ = buffers; +} + +const std::vector> *RPiStream::getBuffers() const +{ + return external_ ? externalBuffers_ : &internalBuffers_; +} + +bool RPiStream::findFrameBuffer(FrameBuffer *buffer) const +{ + auto start = external_ ? externalBuffers_->begin() : internalBuffers_.begin(); + auto end = external_ ? externalBuffers_->end() : internalBuffers_.end(); + + if (importOnly_) + return false; + + if (std::find_if(start, end, + [buffer](std::unique_ptr const &ref) { return ref.get() == buffer; }) != end) + return true; + + return false; +} + +int RPiStream::importBuffers(unsigned int count) +{ + return dev_->importBuffers(count); +} + +int RPiStream::allocateBuffers(unsigned int count) +{ + return dev_->allocateBuffers(count, &internalBuffers_); +} + +int RPiStream::queueBuffers() +{ + if (external_) + return 0; + + for (auto &b : internalBuffers_) { + int ret = dev_->queueBuffer(b.get()); + if (ret) { + LOG(RPISTREAM, Error) << "Failed to queue buffers for " + << name_; + return ret; + } + } + + return 0; +} + +void RPiStream::releaseBuffers() +{ + dev_->releaseBuffers(); + if (!external_ && !importOnly_) + internalBuffers_.clear(); +} + +} /* namespace RPi */ + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h new file mode 100644 index 00000000..8fcf522b --- /dev/null +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -0,0 +1,107 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi (Trading) Ltd. + * + * rpi_stream.h - Raspberry Pi device stream abstraction class. + */ +#ifndef __LIBCAMERA_PIPELINE_RPI_STREAM_H__ +#define __LIBCAMERA_PIPELINE_RPI_STREAM_H__ + +#include +#include +#include + +#include + +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +namespace RPi { + +/* + * Device stream abstraction for either an internal or external stream. + * Used for both Unicam and the ISP. + */ +class RPiStream : public Stream +{ +public: + RPiStream() + { + } + + RPiStream(const char *name, MediaEntity *dev, bool importOnly = false) + : external_(false), importOnly_(importOnly), name_(name), + dev_(std::make_unique(dev)) + { + } + + V4L2VideoDevice *dev() const; + std::string name() const; + bool isImporter() const; + void reset(); + + void setExternal(bool external); + bool isExternal() const; + + void setExternalBuffers(std::vector> *buffers); + const std::vector> *getBuffers() const; + bool findFrameBuffer(FrameBuffer *buffer) const; + + int importBuffers(unsigned int count); + int allocateBuffers(unsigned int count); + + int queueBuffers(); + void releaseBuffers(); + +private: + /* + * Indicates that this stream is active externally, i.e. the buffers + * are provided by the application. + */ + bool external_; + + /* Indicates that this stream only imports buffers, e.g. ISP input. */ + bool importOnly_; + + /* Stream name identifier. */ + std::string name_; + + /* The actual device stream. */ + std::unique_ptr dev_; + + /* Internally allocated framebuffers associated with this device stream. */ + std::vector> internalBuffers_; + + /* Externally allocated framebuffers associated with this device stream. */ + std::vector> *externalBuffers_; +}; + +/* + * The following class is just a convenient (and typesafe) array of device + * streams indexed with an enum class. + */ +template +class RPiDevice : public std::array +{ +private: + constexpr auto index(E e) const noexcept + { + return static_cast>(e); + } +public: + RPiStream &operator[](E e) + { + return std::array::operator[](index(e)); + } + const RPiStream &operator[](E e) const + { + return std::array::operator[](index(e)); + } +}; + +} /* namespace RPi */ + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_PIPELINE_RPI_STREAM_H__ */ From patchwork Fri Sep 18 09:42:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9647 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F12BEC3B5B for ; Fri, 18 Sep 2020 09:44:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 87BEE62FB7; Fri, 18 Sep 2020 11:44:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="C+uphoZ1"; dkim-atps=neutral Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 182FD60367 for ; Fri, 18 Sep 2020 11:43:57 +0200 (CEST) Received: by mail-wr1-x435.google.com with SMTP id j2so4939527wrx.7 for ; Fri, 18 Sep 2020 02:43:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3WK0rRdnEn+I4wH+SGznB3/njHWyIksYVpF7BQwluB4=; b=C+uphoZ1SB08BkVGx5i2XaQZeO2TAcGrYFVIVdNUHEFqLm8bjrbzAfbwmIanfkF/5x KYTQkWKQE4+Hwo613Uk4GrrOZMw8SCrhh/bOGJaFH2BegFUsoppdZKRlQ+WRlXriu9Ho m87x6NmwfhDVoS8mIr0ugqqh5lmVLlqY0CAidza62t2l0u/YmBb+IYuYnW9qTacJo8bb K5ROLiwHi/Jqv5L/t/HuUsK1M4HkqOtRbsrWq1Gx23qWQgKt2H3uqBLmTRUbuTEDrFzS X+ugrKNjB12/nFv7Z5YvNJvDKXw0hObImtr3yfHC66/iZ8IGeckbgwSFnoSVXpvMdPrB dDcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3WK0rRdnEn+I4wH+SGznB3/njHWyIksYVpF7BQwluB4=; b=C/A+fg6ZpHCRezQArJjrq3nGPyFenzb5p60/rv+H8XyyrX4mKByQtO4rLlHNgYMulv ZgzW86AshKxneCYLp5C9QxyAVA3F0TG+1NT5VgCKyLhs5+dZiBCOAg/pjuv+xdlykQh0 AAt2zRxnGZiN1tSPMLVRg5LsSuoxvo08bRmlLz/BcSLIKu6dQER3F7weXPnq3/u9W56j jH6uyTyUdwqNq6t4Ts/qjlYp4rKEBX4XVg3ay3cF1Q8YAiSRzRBCVUwrsn7SfWjuYzta rhay5Ti1pSVSHxYwiO73oRGt0Hedrgd31KUGVpvHeeF1r/tw7hF3WQVdKH/mFiJRWuuH k7cg== X-Gm-Message-State: AOAM531LyCcP1E3o23323XpZmQzxkbh3QeXpWn7+9Hd+t3X4cV4Pg5aA nDKclL8QrhbG1x+mVVXbzF720a5G8fODCQ== X-Google-Smtp-Source: ABdhPJxotzF5ol9nJpcLEr2y/yaIGFUPDf8I7pJs5IMMTincKkF5ra8pAv8FkMJI5swqsNG32RzSfw== X-Received: by 2002:a5d:6547:: with SMTP id z7mr36535000wrv.322.1600422236310; Fri, 18 Sep 2020 02:43:56 -0700 (PDT) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id n2sm4792472wma.29.2020.09.18.02.43.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 02:43:55 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Sep 2020 10:42:23 +0100 Message-Id: <20200918094233.5273-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918094233.5273-1-naush@raspberrypi.com> References: <20200918094233.5273-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 02/12] libcamera: pipeline: ipa: raspberrypi: Rework drop frame signalling X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The IPA now signals up front how many frames it wants the pipeline handler to drop. This makes it easier to handle up-coming changes to the buffer handling for import/export buffers. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Tested-by: David Plowman --- include/libcamera/ipa/raspberrypi.h | 2 +- src/ipa/raspberrypi/raspberrypi.cpp | 20 +++++------ .../pipeline/raspberrypi/raspberrypi.cpp | 36 +++++++++++-------- 3 files changed, 33 insertions(+), 25 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index ca62990e..262fc6f3 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -14,6 +14,7 @@ enum RPiConfigParameters { RPI_IPA_CONFIG_LS_TABLE = (1 << 0), RPI_IPA_CONFIG_STAGGERED_WRITE = (1 << 1), RPI_IPA_CONFIG_SENSOR = (1 << 2), + RPI_IPA_CONFIG_DROP_FRAMES = (1 << 3), }; enum RPiOperations { @@ -21,7 +22,6 @@ enum RPiOperations { RPI_IPA_ACTION_V4L2_SET_ISP, RPI_IPA_ACTION_STATS_METADATA_COMPLETE, RPI_IPA_ACTION_RUN_ISP, - RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME, RPI_IPA_ACTION_EMBEDDED_COMPLETE, RPI_IPA_EVENT_SIGNAL_STAT_READY, RPI_IPA_EVENT_SIGNAL_ISP_PREPARE, diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 4557016c..688d2efc 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -65,8 +65,8 @@ class IPARPi : public IPAInterface public: IPARPi() : lastMode_({}), controller_(), controllerInit_(false), - frame_count_(0), check_count_(0), hide_count_(0), - mistrust_count_(0), lsTable_(nullptr) + frame_count_(0), check_count_(0), mistrust_count_(0), + lsTable_(nullptr) { } @@ -137,8 +137,6 @@ private: uint64_t frame_count_; /* For checking the sequencing of Prepare/Process calls. */ uint64_t check_count_; - /* How many frames the pipeline handler should hide, or "drop". */ - unsigned int hide_count_; /* How many frames we should avoid running control algos on. */ unsigned int mistrust_count_; /* LS table allocation passed in from the pipeline handler. */ @@ -242,14 +240,18 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, */ frame_count_ = 0; check_count_ = 0; + unsigned int drop_frame = 0; if (controllerInit_) { - hide_count_ = helper_->HideFramesModeSwitch(); + drop_frame = helper_->HideFramesModeSwitch(); mistrust_count_ = helper_->MistrustFramesModeSwitch(); } else { - hide_count_ = helper_->HideFramesStartup(); + drop_frame = helper_->HideFramesStartup(); mistrust_count_ = helper_->MistrustFramesStartup(); } + result->data.push_back(drop_frame); + result->operation |= RPI_IPA_CONFIG_DROP_FRAMES; + struct AgcStatus agcStatus; /* These zero values mean not program anything (unless overwritten). */ agcStatus.shutter_time = 0.0; @@ -366,13 +368,11 @@ void IPARPi::processEvent(const IPAOperationData &event) * they are "unreliable". */ prepareISP(embeddedbufferId); + frame_count_++; /* Ready to push the input buffer into the ISP. */ IPAOperationData op; - if (++frame_count_ > hide_count_) - op.operation = RPI_IPA_ACTION_RUN_ISP; - else - op.operation = RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME; + op.operation = RPI_IPA_ACTION_RUN_ISP; op.data = { bayerbufferId & RPiIpaMask::ID }; queueFrameAction.emit(0, op); break; diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 3d94b3ed..4c878003 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -132,7 +132,7 @@ class RPiCameraData : public CameraData public: RPiCameraData(PipelineHandler *pipe) : CameraData(pipe), sensor_(nullptr), state_(State::Stopped), - dropFrame_(false), ispOutputCount_(0) + dropFrameCount_(0), ispOutputCount_(0) { } @@ -180,14 +180,15 @@ public: std::queue embeddedQueue_; std::deque requestQueue_; + unsigned int dropFrameCount_; + private: void checkRequestCompleted(); void tryRunPipeline(); void tryFlushQueues(); FrameBuffer *updateQueue(std::queue &q, uint64_t timestamp, V4L2VideoDevice *dev); - bool dropFrame_; - int ispOutputCount_; + unsigned int ispOutputCount_; }; class RPiCameraConfiguration : public CameraConfiguration @@ -1007,6 +1008,7 @@ int RPiCameraData::configureIPA() ipa_->configure(sensorInfo, streamConfig, entityControls, ipaConfig, &result); + unsigned int resultIdx = 0; if (result.operation & RPI_IPA_CONFIG_STAGGERED_WRITE) { /* * Setup our staggered control writer with the sensor default @@ -1014,9 +1016,9 @@ int RPiCameraData::configureIPA() */ if (!staggeredCtrl_) { staggeredCtrl_.init(unicam_[Unicam::Image].dev(), - { { V4L2_CID_ANALOGUE_GAIN, result.data[0] }, - { V4L2_CID_EXPOSURE, result.data[1] } }); - sensorMetadata_ = result.data[2]; + { { V4L2_CID_ANALOGUE_GAIN, result.data[resultIdx++] }, + { V4L2_CID_EXPOSURE, result.data[resultIdx++] } }); + sensorMetadata_ = result.data[resultIdx++]; } } @@ -1026,6 +1028,11 @@ int RPiCameraData::configureIPA() LOG(RPI, Error) << "V4L2 staggered set failed"; } + if (result.operation & RPI_IPA_CONFIG_DROP_FRAMES) { + /* Configure the number of dropped frames required on startup. */ + dropFrameCount_ = result.data[resultIdx++]; + } + return 0; } @@ -1077,7 +1084,6 @@ void RPiCameraData::queueFrameAction([[maybe_unused]] unsigned int frame, break; } - case RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME: case RPI_IPA_ACTION_RUN_ISP: { unsigned int bufferId = action.data[0]; FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers()->at(bufferId).get(); @@ -1086,7 +1092,6 @@ void RPiCameraData::queueFrameAction([[maybe_unused]] unsigned int frame, << ", timestamp: " << buffer->metadata().timestamp; isp_[Isp::Input].dev()->queueBuffer(buffer); - dropFrame_ = (action.operation == RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME) ? true : false; ispOutputCount_ = 0; break; } @@ -1252,7 +1257,7 @@ void RPiCameraData::clearIncompleteRequests() void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream) { if (stream->isExternal()) { - if (!dropFrame_) { + if (!dropFrameCount_) { Request *request = buffer->request(); pipe_->completeBuffer(camera_, request, buffer); } @@ -1264,7 +1269,7 @@ void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream * simply memcpy to the Request buffer and requeue back to the * device. */ - if (stream == &unicam_[Unicam::Image] && !dropFrame_) { + if (stream == &unicam_[Unicam::Image] && !dropFrameCount_) { const Stream *rawStream = static_cast(&isp_[Isp::Input]); Request *request = requestQueue_.front(); FrameBuffer *raw = request->findBuffer(const_cast(rawStream)); @@ -1309,7 +1314,7 @@ void RPiCameraData::checkRequestCompleted() * If we are dropping this frame, do not touch the request, simply * change the state to IDLE when ready. */ - if (!dropFrame_) { + if (!dropFrameCount_) { Request *request = requestQueue_.front(); if (request->hasPendingBuffers()) return; @@ -1328,10 +1333,13 @@ void RPiCameraData::checkRequestCompleted() * frame. */ if (state_ == State::IpaComplete && - ((ispOutputCount_ == 3 && dropFrame_) || requestCompleted)) { + ((ispOutputCount_ == 3 && dropFrameCount_) || requestCompleted)) { state_ = State::Idle; - if (dropFrame_) - LOG(RPI, Info) << "Dropping frame at the request of the IPA"; + if (dropFrameCount_) { + dropFrameCount_--; + LOG(RPI, Info) << "Dropping frame at the request of the IPA (" + << dropFrameCount_ << " left)"; + } } } From patchwork Fri Sep 18 09:42:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9648 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9C8C6C3B5B for ; Fri, 18 Sep 2020 09:44:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 49B3B62FB6; Fri, 18 Sep 2020 11:44:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="jZXR4TXX"; dkim-atps=neutral Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B075162FAF for ; Fri, 18 Sep 2020 11:43:57 +0200 (CEST) Received: by mail-wm1-x343.google.com with SMTP id x23so4667622wmi.3 for ; Fri, 18 Sep 2020 02:43:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7z1YhK/QItKgF24LCdggoTXw9ex8XokQHsbkI0ox0fs=; b=jZXR4TXXdKVf2OrxYWPsAFx2q2uSThoujCNFvYjGMRTiAwmQhZ5npsvATrs0AlCtIg fmBRzdqPpdPuFjXdTtxsr6yidC2bqHW9lsfcWMQRd2dZexvIHVhYvO22Ud71Oh/iCSKb wrz1tIVcazUL7DoaV6Viz/ErO383ZqIQANnC1PtmVLwWl8e0sghVtWziDyNrX8jwRNzy HMDaAXrfF7cdTC7saf6sdV12VJG38PAcW24sMUi4eh0PbBh+EYlEtemZKzGufjuQUPU9 Jj8QhXRJnwW4F5jc6Vd1oHmlHXTsx8Gibt47ygquV3UX20bhPQPOK582Scuviuocfh4G W5ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7z1YhK/QItKgF24LCdggoTXw9ex8XokQHsbkI0ox0fs=; b=QwrETdBXdo1ESYgFLuxF9iru87roFXQXPr+65Cp4zKladYK27F+J6flnACQGsdO74w 9X7ULXzuYnNNBsKY57tXzQLjhtclxwJLUP7t9NCsiRkCvyERJkm1SxjfKac/rsMsWjbL N6gm+R7RpytV03DkXz1IDieXyF6LlV6eEzwFwIEdc1FtcIMo3foYU8kY8r45+gxGOZfl 38DQnySUn1D/LyxC0t4LCKUbmyG4HUrxgsDa0pDzxCY3Y5DIEkvmyrwcFCB5Z+psc909 yyFuibcDFoYSqpvuvS/IyGT7pxnTcWJ9fwwgAPXDAhqYFFDhL1hustmef3iTQ7oBr3F2 C8Tw== X-Gm-Message-State: AOAM532idK1+R/OaKtr/9LeBzFnrgbBQRuk0Eb1SCxta6LA8OuyT1GNE PP7BbMAvSSUvPGk09QMPCQo+8NWZbkyf6w== X-Google-Smtp-Source: ABdhPJzF66pZ9vlWw/d8U0e6CapAefqWpJLVoy7sTVvhrn3RV9EmSKCSOHlnrOynEB3fgkWVTP2mug== X-Received: by 2002:a05:600c:414e:: with SMTP id h14mr14296194wmm.2.1600422237108; Fri, 18 Sep 2020 02:43:57 -0700 (PDT) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id n2sm4792472wma.29.2020.09.18.02.43.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 02:43:56 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Sep 2020 10:42:24 +0100 Message-Id: <20200918094233.5273-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918094233.5273-1-naush@raspberrypi.com> References: <20200918094233.5273-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 03/12] libcamera: pipeline: raspberrypi: Add some debug logging X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" No functional changes, only added some more trace points. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Tested-by: David Plowman --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 4c878003..f15345ea 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1162,6 +1162,11 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) if (state_ == State::Stopped) return; + LOG(RPI, Debug) << "Stream ISP Input buffer complete" + << ", buffer id " << buffer->cookie() + << ", timestamp: " << buffer->metadata().timestamp; + + /* The ISP input buffer gets re-queued into Unicam. */ handleStreamBuffer(buffer, &unicam_[Unicam::Image]); handleState(); } @@ -1464,7 +1469,7 @@ FrameBuffer *RPiCameraData::updateQueue(std::queue &q, uint64_t t if (b->metadata().timestamp < timestamp) { q.pop(); dev->queueBuffer(b); - LOG(RPI, Error) << "Dropping input frame!"; + LOG(RPI, Warning) << "Dropping input frame!"; } else if (b->metadata().timestamp == timestamp) { /* The calling function will pop the item from the queue. */ return b; From patchwork Fri Sep 18 09:42:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9649 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 05ED1C3B5D for ; Fri, 18 Sep 2020 09:44:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BA4E562FC2; Fri, 18 Sep 2020 11:44:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="U0DSjait"; dkim-atps=neutral Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9F0DC62FAF for ; Fri, 18 Sep 2020 11:43:58 +0200 (CEST) Received: by mail-wm1-x335.google.com with SMTP id z9so4874666wmk.1 for ; Fri, 18 Sep 2020 02:43:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nE/emN/9YQ8F/OJCPwpwoa+soM6JSc3SLzfzjKB3NuU=; b=U0DSjaitaVqcUq8uu9G+GYfyAMP7BY3asPt5PGXll5LBR34UgA0MsS0zoFfchMhqIZ Routru0keEdD0g8239b8H34Ah17Rxopwg3xmG6uUgtwMihBYqaykaRvJHBBteTXk1dRN oaRoXyvXWj5jhXR1AsrBhjpuKe6duAii4JialE1Wrz5IBYGMS/eEJoi74EriBGZKgjro Yp82e5zT34QNFuTlZkfLAlXA4wAGKeGOWkYlT+Il5BPXaE+gpvz+UTuXgtHvyFdW8Exl dxXWN32eM0x/iQhZ4sf0hzlRYZd2k84aBAEouutRV9wXjbVp1hiGpBu3A6dpIFxNyIHv Q35A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nE/emN/9YQ8F/OJCPwpwoa+soM6JSc3SLzfzjKB3NuU=; b=UI66IZVDDERX6cihfXP0sqqXqNagZvlGtjMKvfIfmWsJ5LMNNUAJ96e2+SVp8eSv2L LJw5XuaAGFgmsL+rOlPA+NVj26EJLawLYVdFVfZUzn0Emb+9houlMCKqlFzFX1j3Vq10 z7pppOzIJkAKBtZVlWsnxntmbpXBXFNH9Tu68FzSj1hLqCmU2LrlVpTZdHsa4xsG5eS8 Z47CkgnocGCo/bqIj7wFmQqt94hHzNCVyMteki/kAdYxTYltlarwK4Vd1DwoOEKKYDzo H71oinfYCVXLQvtTNSMcy3podUuknyQBLrk8kQaqd0ZonlD1NrsmQg33w/acHnZcb3l3 JsfQ== X-Gm-Message-State: AOAM532sVc8/Jj9krCPzaPa/LfGqeU0TqKQTS5Bh2sxadsFW4lZIwylO XxPVKtplTfu0auiS49z+gL1KPRTfkfrIzQ== X-Google-Smtp-Source: ABdhPJzWLm9TsmDCMIMArNdCmNs3FVYZUhPKLn7fHRBdgIOA62fZ9O2uGOir19WQIy3zfwNXUpq8Zg== X-Received: by 2002:a1c:2ed0:: with SMTP id u199mr14222107wmu.125.1600422238133; Fri, 18 Sep 2020 02:43:58 -0700 (PDT) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id n2sm4792472wma.29.2020.09.18.02.43.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 02:43:57 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Sep 2020 10:42:25 +0100 Message-Id: <20200918094233.5273-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918094233.5273-1-naush@raspberrypi.com> References: <20200918094233.5273-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 04/12] libcamera: pipeline: raspberrypi: Increase the number of RAW buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Increase the number of expected RAW buffers in the stream configuration to 2. This will avoid dropping Unicam frames when exporting RAW streams. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Tested-by: David Plowman --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index f15345ea..ed83a2c6 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -378,7 +378,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, sensorFormat = findBestMode(fmts, size); pixelFormat = sensorFormat.fourcc.toPixelFormat(); ASSERT(pixelFormat.isValid()); - bufferCount = 1; + bufferCount = 2; rawCount++; break; From patchwork Fri Sep 18 09:42:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9650 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 66A02C3B5B for ; Fri, 18 Sep 2020 09:44:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3592562FC8; Fri, 18 Sep 2020 11:44:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="L8yE7iSU"; dkim-atps=neutral Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C13EF62FBB for ; Fri, 18 Sep 2020 11:43:59 +0200 (CEST) Received: by mail-wr1-x434.google.com with SMTP id z1so4955094wrt.3 for ; Fri, 18 Sep 2020 02:43:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2HZNXIdwEKmIQyD2h2XM7I6cmzMuhwlSR4pJrDGbzE4=; b=L8yE7iSUzPJQ5OtAfqY9b47MzNzzWAIZSYn4Jamm6Kof+lsKJeBdeegXM/f9McspB9 OL1o3PNo+NtcIy/9EdkzQHFbpFiWU/grn3ISVNu9FFKrp6z8U+lnsShT50K92YN28XSz gcGmLXRAfPnQom7UK2fCS7K3PHHv4HOOaWjXZKylXMByLfTulvPfhUQYOlzTPOd2NA/w lkFsWuS71C4nGoYGlYiimYiY/8RrivCAMux1/Y4I3nzuO9aoiQ0DA/isgeowE4z16NUI Zhy3o76AGZDSQAiXfcwzFhB8j0CphgrieoxhlvLrTHgpeqhXPQXIZssPfDZQr+sjvlSv ul3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2HZNXIdwEKmIQyD2h2XM7I6cmzMuhwlSR4pJrDGbzE4=; b=ML4sTWtU8u7UwWBjbJF1rj6jhNHIsAoOEkR4pXV327bngOeXMsrV2ir9Evi9rVxMCW r04kJrMsscOWv1bjuyMY7VpvSHKlP4ykw+PDZktXSId4zDH8w1ayJ55btgliyB0deBXk 1pzqxJJ3RkvWhD58PLyoPa8h4epAobmWQToXNCtmCwwZKbfU7W2U8NAjFmq4IC20x77h IUBwaVcWdlmCLMHB4JfHQTCSK5hv/6UiOA98vHVwaRnZpGxF6vM/vMugbtOBTc0Tntbx NMy4vpCNP/s4W/VTfOj6Zp+UPvgJpg4DasAH6rSh42msiiUGxTeHEeOvMl90yWc9hWei SOHQ== X-Gm-Message-State: AOAM530konwvCN8bzyoOvh9bVTU4hFdRPAZz8TE0CRUliyrulizBwrvX ZF/4M0i3clfFcJGwaGXYUXq5Brbp5mAvGQ== X-Google-Smtp-Source: ABdhPJw79wJ3cxnuUoLgVYcaH5GeKUI/waIZGAzLsc7atSg5qHqnnZPyDeHhH0IJEqgD6YFEz2jdOQ== X-Received: by 2002:a5d:660f:: with SMTP id n15mr39653272wru.103.1600422239137; Fri, 18 Sep 2020 02:43:59 -0700 (PDT) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id n2sm4792472wma.29.2020.09.18.02.43.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 02:43:58 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Sep 2020 10:42:26 +0100 Message-Id: <20200918094233.5273-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918094233.5273-1-naush@raspberrypi.com> References: <20200918094233.5273-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 05/12] libcamera: pipeline: raspberrypi: Remove const qualifier from RPiStream X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For the zero-copy RAW capture feature, the RPiStream will have to be modified. Remove the const qualifier in anticipation of the future commits for this feature. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Tested-by: David Plowman --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index ed83a2c6..6b9de029 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -149,7 +149,7 @@ public: void ispOutputDequeue(FrameBuffer *buffer); void clearIncompleteRequests(); - void handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream); + void handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream); void handleState(); CameraSensor *sensor_; @@ -1107,12 +1107,12 @@ done: void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) { - const RPi::RPiStream *stream = nullptr; + RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPi::RPiStream const &s : unicam_) { + for (RPi::RPiStream &s : unicam_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1173,12 +1173,12 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) { - const RPi::RPiStream *stream = nullptr; + RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPi::RPiStream const &s : isp_) { + for (RPi::RPiStream &s : isp_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1259,7 +1259,7 @@ void RPiCameraData::clearIncompleteRequests() } } -void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream) +void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream) { if (stream->isExternal()) { if (!dropFrameCount_) { From patchwork Fri Sep 18 09:42:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9651 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B7738C3B5D for ; Fri, 18 Sep 2020 09:44:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7FD0A62FCE; Fri, 18 Sep 2020 11:44:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="q4F9fWXe"; dkim-atps=neutral Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 771F062FBF for ; Fri, 18 Sep 2020 11:44:01 +0200 (CEST) Received: by mail-wr1-x42e.google.com with SMTP id z1so4955182wrt.3 for ; Fri, 18 Sep 2020 02:44:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=65JYsargR9kCb4D8AS4OavGt1hHHa9Vm6NURziBlw2M=; b=q4F9fWXe2PwJ/MRdN9N6vFLpCaayAxFAWJBU43KCEPwXVWFqJ0jNA+ZRFAgIUm+oRN zVms/8YbKam4QDx7No4pPT7HuvoFbLL4tJ2qbBfCY4feYN5swRCF+CM6C0epP3tIkIWR JHQDqy/XjBRrop4Z4ooyuq/2oEduwBo/ZJA2XTasDxg6qrBTLRgDvqMuiRB97MoflcrV zMYKWi06R5zH7IHKWkrh3vrC666uUMKSqOs0wfKT/UO+/j6dyAIrZ0FY7afhVWGGT0BI tdmhMidugaLJ4FRoA1L/UQUwdHPFXMKfoC37yi3exfY1HIKusggmpQ7Cdbz9uvcGmzZc jw4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=65JYsargR9kCb4D8AS4OavGt1hHHa9Vm6NURziBlw2M=; b=VAioAZ7s9NFSw20PbnO3fyIsFOvkGD+QIjRmS0EHsYrCjTTX6TpOF+uyLNOP0Ze9TS zu3UBTg7rLwEpN2d3xOXc9isnWn0JOxQFZ6bnXyRTJsrMhN5ydfWeupBkBYg8L0Fq3RW BPQUU++XcWYAa/T5/ked8p3SZtaTZtZnAOs7IM3nYUee7nHNpA9k4RHyXW1p9TDO/Heb CLbmqUQfQH460IZiMzia58hthuMnd5vQIJYzEtXgwTY7th7y5SGxR5ZLi2r6n87cr0Lh B5LLWdKq+QTW0DGka+G4xvy59RaEpppl+3v3TrxBuLiBAKVSLOHH8HSSWaPCJJJcqaVh znLQ== X-Gm-Message-State: AOAM530ZDZs8sN7aBl+nlvu3Fe5PR1k1rPP8mIxt7wVSKFMMJu3WEo0o Z4k9cWXD1Nae7xifzdWB/fANh+hPNbG/Pw== X-Google-Smtp-Source: ABdhPJwldit7Kh2k+RtCIf5MgrqygP6iEOlFdLdnH79anO8kNNHdR5nvXVvWuGpgvXRUeyDJgg6otg== X-Received: by 2002:adf:eecb:: with SMTP id a11mr37294942wrp.356.1600422240230; Fri, 18 Sep 2020 02:44:00 -0700 (PDT) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id n2sm4792472wma.29.2020.09.18.02.43.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 02:43:59 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Sep 2020 10:42:27 +0100 Message-Id: <20200918094233.5273-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918094233.5273-1-naush@raspberrypi.com> References: <20200918094233.5273-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 06/12] libcamera: pipeline: raspberrypi: Rework stream buffer logic for zero-copy X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Stop using v4l2_videodevice::allocateBuffer() for internal buffers and instead export/import all buffers. This allows the pipeline to return any stream buffer requested by the application as zero-copy. Advertise the Unicam Image stream as the RAW capture stream now. The RPiStream object now maintains a new list of buffers that are available to queue into a device. This is needed to distinguish between FrameBuffers allocated for internal use vs externally provided buffers. When a Request comes in, if a buffer is not provided for an exported stream, we re-use a buffer from this list. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Tested-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 235 ++++++++++-------- .../pipeline/raspberrypi/rpi_stream.cpp | 118 ++++++--- .../pipeline/raspberrypi/rpi_stream.h | 33 ++- 3 files changed, 235 insertions(+), 151 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 6b9de029..9fbe5089 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -186,7 +186,8 @@ private: void checkRequestCompleted(); void tryRunPipeline(); void tryFlushQueues(); - FrameBuffer *updateQueue(std::queue &q, uint64_t timestamp, V4L2VideoDevice *dev); + FrameBuffer *updateQueue(std::queue &q, uint64_t timestamp, + RPi::RPiStream *stream); unsigned int ispOutputCount_; }; @@ -507,8 +508,15 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) StreamConfiguration &cfg = config->at(i); if (isRaw(cfg.pixelFormat)) { - cfg.setStream(&data->isp_[Isp::Input]); - data->isp_[Isp::Input].setExternal(true); + cfg.setStream(&data->unicam_[Unicam::Image]); + /* + * We must set both Unicam streams as external, even + * though the application may only request RAW frames. + * This is because we match timestamps on both streams + * to synchronise buffers. + */ + data->unicam_[Unicam::Image].setExternal(true); + data->unicam_[Unicam::Embedded].setExternal(true); continue; } @@ -611,7 +619,7 @@ int PipelineHandlerRPi::exportFrameBuffers([[maybe_unused]] Camera *camera, Stre unsigned int count = stream->configuration().bufferCount; int ret = s->dev()->exportBuffers(count, buffers); - s->setExternalBuffers(buffers); + s->setExportedBuffers(buffers); return ret; } @@ -625,6 +633,7 @@ int PipelineHandlerRPi::start(Camera *camera) ret = prepareBuffers(camera); if (ret) { LOG(RPI, Error) << "Failed to allocate buffers"; + stop(camera); return ret; } @@ -710,14 +719,26 @@ int PipelineHandlerRPi::queueRequestDevice(Camera *camera, Request *request) if (data->state_ == RPiCameraData::State::Stopped) return -EINVAL; - /* Ensure all external streams have associated buffers! */ - for (auto &stream : data->isp_) { - if (!stream.isExternal()) - continue; + LOG(RPI, Debug) << "queueRequestDevice: New request."; - if (!request->findBuffer(&stream)) { - LOG(RPI, Error) << "Attempt to queue request with invalid stream."; - return -ENOENT; + /* Push all buffers supplied in the Request to the respective streams. */ + for (auto stream : data->streams_) { + if (stream->isExternal()) { + FrameBuffer *buffer = request->findBuffer(stream); + /* + * If no buffer is provided by the request for this stream, we + * queue a nullptr to the stream to signify that it must use an + * internally allocated buffer for this capture request. This + * buffer will not be given back to the application, but is used + * to support the internal pipeline flow. + * + * The below queueBuffer() call will do nothing if there are not + * enough internal buffers allocated, but this will be handled by + * queuing the request for buffers in the RPiStream object. + */ + int ret = stream->queueBuffer(buffer); + if (ret) + return ret; } } @@ -816,14 +837,14 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) data->unicam_[Unicam::Image].dev()->setControls(&ctrls); /* - * List the available output streams. - * Currently cannot do Unicam streams! + * List the available streams an application may request. At present, we + * do not advertise Unicam Embedded and ISP Statistics streams, as there + * is no mechanism for the application to request non-image buffer formats. */ std::set streams; - streams.insert(&data->isp_[Isp::Input]); + streams.insert(&data->unicam_[Unicam::Image]); streams.insert(&data->isp_[Isp::Output0]); streams.insert(&data->isp_[Isp::Output1]); - streams.insert(&data->isp_[Isp::Stats]); /* Create and register the camera. */ std::shared_ptr camera = @@ -839,9 +860,28 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera) int ret; for (auto const stream : data->streams_) { - ret = stream->queueBuffers(); - if (ret < 0) - return ret; + if (!stream->isExternal()) { + ret = stream->queueAllBuffers(); + if (ret < 0) + return ret; + } else { + /* + * For external streams, we must queue up a set of internal + * buffers to handle the number of drop frames requested by + * the IPA. This is done by passing nullptr in queueBuffer(). + * + * The below queueBuffer() call will do nothing if there + * are not enough internal buffers allocated, but this will + * be handled by queuing the request for buffers in the + * RPiStream object. + */ + unsigned int i; + for (i = 0; i < data->dropFrameCount_; i++) { + int ret = stream->queueBuffer(nullptr); + if (ret) + return ret; + } + } } return 0; @@ -855,7 +895,8 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) /* * Decide how many internal buffers to allocate. For now, simply look * at how many external buffers will be provided. Will need to improve - * this logic. + * this logic. However, we really must have all streams allocate the same + * number of buffers to simplify error handling in queueRequestDevice(). */ unsigned int maxBuffers = 0; for (const Stream *s : camera->streams()) @@ -863,33 +904,9 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) maxBuffers = std::max(maxBuffers, s->configuration().bufferCount); for (auto const stream : data->streams_) { - if (stream->isExternal() || stream->isImporter()) { - /* - * If a stream is marked as external reserve memory to - * prepare to import as many buffers are requested in - * the stream configuration. - * - * If a stream is an internal stream with importer - * role, reserve as many buffers as possible. - */ - unsigned int count = stream->isExternal() - ? stream->configuration().bufferCount - : maxBuffers; - ret = stream->importBuffers(count); - if (ret < 0) - return ret; - } else { - /* - * If the stream is an internal exporter allocate and - * export as many buffers as possible to its internal - * pool. - */ - ret = stream->allocateBuffers(maxBuffers); - if (ret < 0) { - freeBuffers(camera); - return ret; - } - } + ret = stream->prepareBuffers(maxBuffers); + if (ret < 0) + return ret; } /* @@ -897,7 +914,7 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * the IPA and RPI_IPA_EVENT_SIGNAL_ISP_PREPARE event. */ count = 0; - for (auto const &b : *data->unicam_[Unicam::Image].getBuffers()) { + for (auto const &b : data->unicam_[Unicam::Image].getBuffers()) { b->setCookie(count++); } @@ -906,14 +923,14 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * the IPA. */ count = 0; - for (auto const &b : *data->isp_[Isp::Stats].getBuffers()) { + for (auto const &b : data->isp_[Isp::Stats].getBuffers()) { b->setCookie(count++); data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | b->cookie(), .planes = b->planes() }); } count = 0; - for (auto const &b : *data->unicam_[Unicam::Embedded].getBuffers()) { + for (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) { b->setCookie(count++); data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | b->cookie(), .planes = b->planes() }); @@ -1068,7 +1085,7 @@ void RPiCameraData::queueFrameAction([[maybe_unused]] unsigned int frame, switch (action.operation) { case RPI_IPA_ACTION_STATS_METADATA_COMPLETE: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = isp_[Isp::Stats].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = isp_[Isp::Stats].getBuffers().at(bufferId); handleStreamBuffer(buffer, &isp_[Isp::Stats]); /* Fill the Request metadata buffer with what the IPA has provided */ @@ -1079,19 +1096,19 @@ void RPiCameraData::queueFrameAction([[maybe_unused]] unsigned int frame, case RPI_IPA_ACTION_EMBEDDED_COMPLETE: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = unicam_[Unicam::Embedded].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = unicam_[Unicam::Embedded].getBuffers().at(bufferId); handleStreamBuffer(buffer, &unicam_[Unicam::Embedded]); break; } case RPI_IPA_ACTION_RUN_ISP: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers().at(bufferId); LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << buffer->cookie() << ", timestamp: " << buffer->metadata().timestamp; - isp_[Isp::Input].dev()->queueBuffer(buffer); + isp_[Isp::Input].queueBuffer(buffer); ispOutputCount_ = 0; break; } @@ -1192,22 +1209,26 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) << ", buffer id " << buffer->cookie() << ", timestamp: " << buffer->metadata().timestamp; - handleStreamBuffer(buffer, stream); - /* - * Increment the number of ISP outputs generated. - * This is needed to track dropped frames. + * ISP statistics buffer must not be re-queued or sent back to the + * application until after the IPA signals so. */ - ispOutputCount_++; - - /* If this is a stats output, hand it to the IPA now. */ if (stream == &isp_[Isp::Stats]) { IPAOperationData op; op.operation = RPI_IPA_EVENT_SIGNAL_STAT_READY; op.data = { RPiIpaMask::STATS | buffer->cookie() }; ipa_->processEvent(op); + } else { + /* Any other ISP output can be handed back to the application now. */ + handleStreamBuffer(buffer, stream); } + /* + * Increment the number of ISP outputs generated. + * This is needed to track dropped frames. + */ + ispOutputCount_++; + handleState(); } @@ -1220,8 +1241,12 @@ void RPiCameraData::clearIncompleteRequests() */ for (auto const request : requestQueue_) { for (auto const stream : streams_) { - if (stream->isExternal()) - stream->dev()->queueBuffer(request->findBuffer(stream)); + if (!stream->isExternal()) + continue; + + FrameBuffer *buffer = request->findBuffer(stream); + if (buffer) + stream->queueBuffer(buffer); } } @@ -1250,7 +1275,7 @@ void RPiCameraData::clearIncompleteRequests() * Has the buffer already been handed back to the * request? If not, do so now. */ - if (buffer->request()) + if (buffer && buffer->request()) pipe_->completeBuffer(camera_, request, buffer); } @@ -1262,30 +1287,28 @@ void RPiCameraData::clearIncompleteRequests() void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream) { if (stream->isExternal()) { - if (!dropFrameCount_) { - Request *request = buffer->request(); + /* + * It is possible to be here without a pending request, so check + * that we actually have one to action, otherwise we just return + * buffer back to the stream. + */ + Request *request = requestQueue_.empty() ? nullptr : requestQueue_.front(); + if (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) { + /* + * Tag the buffer as completed, returning it to the + * application. + */ pipe_->completeBuffer(camera_, request, buffer); + } else { + /* + * This buffer was not part of the Request, or there is no + * pending request, so we can recycle it. + */ + stream->returnBuffer(buffer); } } else { - /* Special handling for RAW buffer Requests. - * - * The ISP input stream is alway an import stream, but if the - * current Request has been made for a buffer on the stream, - * simply memcpy to the Request buffer and requeue back to the - * device. - */ - if (stream == &unicam_[Unicam::Image] && !dropFrameCount_) { - const Stream *rawStream = static_cast(&isp_[Isp::Input]); - Request *request = requestQueue_.front(); - FrameBuffer *raw = request->findBuffer(const_cast(rawStream)); - if (raw) { - raw->copyFrom(buffer); - pipe_->completeBuffer(camera_, request, raw); - } - } - - /* Simply requeue the buffer. */ - stream->dev()->queueBuffer(buffer); + /* Simply re-queue the buffer to the requested stream. */ + stream->queueBuffer(buffer); } } @@ -1367,7 +1390,7 @@ void RPiCameraData::tryRunPipeline() * current bayer buffer will be removed and re-queued to the driver. */ embeddedBuffer = updateQueue(embeddedQueue_, bayerBuffer->metadata().timestamp, - unicam_[Unicam::Embedded].dev()); + &unicam_[Unicam::Embedded]); if (!embeddedBuffer) { LOG(RPI, Debug) << "Could not find matching embedded buffer"; @@ -1386,7 +1409,7 @@ void RPiCameraData::tryRunPipeline() embeddedBuffer = embeddedQueue_.front(); bayerBuffer = updateQueue(bayerQueue_, embeddedBuffer->metadata().timestamp, - unicam_[Unicam::Image].dev()); + &unicam_[Unicam::Image]); if (!bayerBuffer) { LOG(RPI, Debug) << "Could not find matching bayer buffer - ending."; @@ -1394,11 +1417,7 @@ void RPiCameraData::tryRunPipeline() } } - /* - * Take the first request from the queue and action the IPA. - * Unicam buffers for the request have already been queued as they come - * in. - */ + /* Take the first request from the queue and action the IPA. */ Request *request = requestQueue_.front(); /* @@ -1410,12 +1429,6 @@ void RPiCameraData::tryRunPipeline() op.controls = { request->controls() }; ipa_->processEvent(op); - /* Queue up any ISP buffers passed into the request. */ - for (auto &stream : isp_) { - if (stream.isExternal()) - stream.dev()->queueBuffer(request->findBuffer(&stream)); - } - /* Ready to use the buffers, pop them off the queue. */ bayerQueue_.pop(); embeddedQueue_.pop(); @@ -1445,32 +1458,42 @@ void RPiCameraData::tryFlushQueues() * and give a chance for the hardware to return to lock-step. We do have * to drop all interim frames. */ - if (unicam_[Unicam::Image].getBuffers()->size() == bayerQueue_.size() && - unicam_[Unicam::Embedded].getBuffers()->size() == embeddedQueue_.size()) { + if (unicam_[Unicam::Image].getBuffers().size() == bayerQueue_.size() && + unicam_[Unicam::Embedded].getBuffers().size() == embeddedQueue_.size()) { + /* This cannot happen when Unicam streams are external. */ + assert(!unicam_[Unicam::Image].isExternal()); + LOG(RPI, Warning) << "Flushing all buffer queues!"; while (!bayerQueue_.empty()) { - unicam_[Unicam::Image].dev()->queueBuffer(bayerQueue_.front()); + unicam_[Unicam::Image].queueBuffer(bayerQueue_.front()); bayerQueue_.pop(); } while (!embeddedQueue_.empty()) { - unicam_[Unicam::Embedded].dev()->queueBuffer(embeddedQueue_.front()); + unicam_[Unicam::Embedded].queueBuffer(embeddedQueue_.front()); embeddedQueue_.pop(); } } } FrameBuffer *RPiCameraData::updateQueue(std::queue &q, uint64_t timestamp, - V4L2VideoDevice *dev) + RPi::RPiStream *stream) { + /* + * If the unicam streams are external (both have be to the same), then we + * can only return out the top buffer in the queue, and assume they have + * been synced by queuing at the same time. We cannot drop these frames, + * as they may have been provided externally. + */ while (!q.empty()) { FrameBuffer *b = q.front(); - if (b->metadata().timestamp < timestamp) { + if (!stream->isExternal() && b->metadata().timestamp < timestamp) { q.pop(); - dev->queueBuffer(b); - LOG(RPI, Warning) << "Dropping input frame!"; - } else if (b->metadata().timestamp == timestamp) { + stream->queueBuffer(b); + LOG(RPI, Warning) << "Dropping unmatched input frame in stream " + << stream->name(); + } else if (stream->isExternal() || b->metadata().timestamp == timestamp) { /* The calling function will pop the item from the queue. */ return b; } else { diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index 2e52bd5c..b4d737db 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -27,78 +27,120 @@ std::string RPiStream::name() const void RPiStream::reset() { external_ = false; - internalBuffers_.clear(); -} - -bool RPiStream::isImporter() const -{ - return importOnly_; + clearBuffers(); } void RPiStream::setExternal(bool external) { + /* Import streams cannot be external. */ + ASSERT(!external || !importOnly_); external_ = external; } bool RPiStream::isExternal() const { - /* - * Import streams cannot be external. - * - * RAW capture is a special case where we simply copy the RAW - * buffer out of the request. All other buffer handling happens - * as if the stream is internal. - */ - return external_ && !importOnly_; + return external_; } -void RPiStream::setExternalBuffers(std::vector> *buffers) +void RPiStream::setExportedBuffers(std::vector> *buffers) { - externalBuffers_ = buffers; + std::transform(buffers->begin(), buffers->end(), std::back_inserter(bufferList_), + [](std::unique_ptr &b) { return b.get(); }); } -const std::vector> *RPiStream::getBuffers() const +const std::vector &RPiStream::getBuffers() const { - return external_ ? externalBuffers_ : &internalBuffers_; + return bufferList_; } bool RPiStream::findFrameBuffer(FrameBuffer *buffer) const { - auto start = external_ ? externalBuffers_->begin() : internalBuffers_.begin(); - auto end = external_ ? externalBuffers_->end() : internalBuffers_.end(); - if (importOnly_) return false; - if (std::find_if(start, end, - [buffer](std::unique_ptr const &ref) { return ref.get() == buffer; }) != end) + if (std::find(bufferList_.begin(), bufferList_.end(), buffer) != bufferList_.end()) return true; return false; } -int RPiStream::importBuffers(unsigned int count) +int RPiStream::prepareBuffers(unsigned int count) { + int ret; + + if (!importOnly_) { + if (count) { + /* Export some frame buffers for internal use. */ + ret = dev_->exportBuffers(count, &internalBuffers_); + if (ret < 0) + return ret; + + /* Add these exported buffers to the internal/external buffer list. */ + setExportedBuffers(&internalBuffers_); + + /* Add these buffers to the queue of internal usable buffers. */ + for (auto const &buffer : internalBuffers_) + availableBuffers_.push(buffer.get()); + } + + /* We must import all internal/external exported buffers. */ + count = bufferList_.size(); + } + return dev_->importBuffers(count); } -int RPiStream::allocateBuffers(unsigned int count) +int RPiStream::queueBuffer(FrameBuffer *buffer) +{ + /* + * A nullptr buffer implies an external stream, but no external + * buffer has been supplied. So, pick one from the availableBuffers_ + * queue. + */ + if (!buffer) { + if (availableBuffers_.empty()) { + LOG(RPISTREAM, Warning) << "No buffers available for " + << name_; + return -EINVAL; + } + + buffer = availableBuffers_.front(); + availableBuffers_.pop(); + } + + LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + << " for " << name_; + + int ret = dev_->queueBuffer(buffer); + if (ret) { + LOG(RPISTREAM, Error) << "Failed to queue buffer for " + << name_; + } + + return ret; +} + +void RPiStream::returnBuffer(FrameBuffer *buffer) { - return dev_->allocateBuffers(count, &internalBuffers_); + /* This can only be called for external streams. */ + assert(external_); + + availableBuffers_.push(buffer); } -int RPiStream::queueBuffers() +int RPiStream::queueAllBuffers() { + int ret; + if (external_) return 0; - for (auto &b : internalBuffers_) { - int ret = dev_->queueBuffer(b.get()); - if (ret) { - LOG(RPISTREAM, Error) << "Failed to queue buffers for " - << name_; + while (!availableBuffers_.empty()) { + ret = queueBuffer(availableBuffers_.front()); + if (ret < 0) return ret; - } + + availableBuffers_.pop(); } return 0; @@ -107,8 +149,14 @@ int RPiStream::queueBuffers() void RPiStream::releaseBuffers() { dev_->releaseBuffers(); - if (!external_ && !importOnly_) - internalBuffers_.clear(); + clearBuffers(); +} + +void RPiStream::clearBuffers() +{ + availableBuffers_ = std::queue{}; + internalBuffers_.clear(); + bufferList_.clear(); } } /* namespace RPi */ diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index 8fcf522b..73954ec7 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -44,20 +44,23 @@ public: void setExternal(bool external); bool isExternal() const; - void setExternalBuffers(std::vector> *buffers); - const std::vector> *getBuffers() const; + void setExportedBuffers(std::vector> *buffers); + const std::vector &getBuffers() const; bool findFrameBuffer(FrameBuffer *buffer) const; - int importBuffers(unsigned int count); - int allocateBuffers(unsigned int count); + int prepareBuffers(unsigned int count); + int queueBuffer(FrameBuffer *buffer); + void returnBuffer(FrameBuffer *buffer); - int queueBuffers(); + int queueAllBuffers(); void releaseBuffers(); private: + void clearBuffers(); + /* * Indicates that this stream is active externally, i.e. the buffers - * are provided by the application. + * might be provided by (and returned to) the application. */ bool external_; @@ -70,11 +73,21 @@ private: /* The actual device stream. */ std::unique_ptr dev_; - /* Internally allocated framebuffers associated with this device stream. */ - std::vector> internalBuffers_; + /* All framebuffers associated with this device stream. */ + std::vector bufferList_; + + /* + * List of frame buffers that we can use if none have been provided by + * the application for external streams. This is populated by the + * buffers exported internally. + */ + std::queue availableBuffers_; - /* Externally allocated framebuffers associated with this device stream. */ - std::vector> *externalBuffers_; + /* + * This is a list of buffers exported internally. Need to keep this around + * as the stream needs to maintain ownership of these buffers. + */ + std::vector> internalBuffers_; }; /* From patchwork Fri Sep 18 09:42:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9652 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3F655C3B5B for ; Fri, 18 Sep 2020 09:44:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 044CC62FC7; Fri, 18 Sep 2020 11:44:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Tv1R8ciq"; dkim-atps=neutral Received: from mail-wr1-x442.google.com (mail-wr1-x442.google.com [IPv6:2a00:1450:4864:20::442]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A955062FB9 for ; Fri, 18 Sep 2020 11:44:01 +0200 (CEST) Received: by mail-wr1-x442.google.com with SMTP id c18so4930986wrm.9 for ; Fri, 18 Sep 2020 02:44:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=N8jhUK8hBWFefqL29hZIlulkNawiu+/q5f/uuUe0Gpk=; b=Tv1R8ciq9xfZYfw4VPgmcfudWhOXAzF3SwJHeczooTJq/P5pYZbbAutpvV2IL+u6lX gTg91ihlliTpHB0qMybupRXvrOZg+OROBFr8A602vacK+9q/X2wvHQ3W3mwqgtzq8p7z mCc+vCCHMNgKyhi/44yfB2cYoHvjxnld5CJODLxJZJYdpJhcYkObGuB11TJlu94cxeIe yu+uvQVh84qoWQsEQxWxLPFQ6C6BmyUm/ADt2dwk/zNNu9+E/a2DpOocwTsLVeaIzb47 NGq0cv0TYrtk/w8JqAL93biL++p3UDUJIC5jWZFI1zu5mv/p30OKOY/9uehLeAZ5pMQy S7wQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=N8jhUK8hBWFefqL29hZIlulkNawiu+/q5f/uuUe0Gpk=; b=BubyjqmFURN6okik0LhxXiRZYlJLTzrzNct6ERfmcXgid8g7Vm1eG9sppCvKYcuFBK cp8XLhysEzAO1rqC7YkcJWgdxHAPAgHaLUU0u5BDKR9nL61DDs5f/vGs7GWcpVhkzLdr 5RdYfg06J02+DLadyEWPlHxDxRml+H0CkRn7b/ruuHPcCQUkV2SUOOZUtJHheNXv5siX yvvnnkCWjO3f47j6xXSFzBNioOWw6J6o3rq/LyyTHyvv8l27pK7SlBMqIfsI0IBfRRck +IU9Krd+rGfcjgwgJBnQ72EUBzRsqT9lYj+Av8124w0cnMUJlb8OR4Fqf+J84kFQv7Oz 9e+Q== X-Gm-Message-State: AOAM530F2LvJJwlM9uP7acaYE7DgefdLyfpJGy2Jp3I7/jk3qZq5sDcs ziQP5DdFbKst3fS5keFh4J8SDwnenV8DyA== X-Google-Smtp-Source: ABdhPJzMel5UH0bBaLkZgRYC1iBdn9vpcujhV6Wr/25ZclsVYSctca1WbG1i89nF/wmSfSul7CkbhA== X-Received: by 2002:adf:f843:: with SMTP id d3mr39601463wrq.226.1600422241064; Fri, 18 Sep 2020 02:44:01 -0700 (PDT) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id n2sm4792472wma.29.2020.09.18.02.44.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 02:44:00 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Sep 2020 10:42:28 +0100 Message-Id: <20200918094233.5273-8-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918094233.5273-1-naush@raspberrypi.com> References: <20200918094233.5273-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 07/12] libcamera: pipeline: raspberrypi: Fix bug in passing configuration to IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The counter was not incremented, so multiple streams would only pass the last stream config to the IPA. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Tested-by: David Plowman --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 9fbe5089..3caf3655 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -992,12 +992,13 @@ int RPiCameraData::configureIPA() unsigned int i = 0; for (auto const &stream : isp_) { if (stream.isExternal()) { - streamConfig[i] = { + streamConfig[i++] = { .pixelFormat = stream.configuration().pixelFormat, .size = stream.configuration().size }; } } + entityControls.emplace(0, unicam_[Unicam::Image].dev()->controls()); entityControls.emplace(1, isp_[Isp::Input].dev()->controls()); From patchwork Fri Sep 18 09:42:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9653 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E3C05C3B5B for ; Fri, 18 Sep 2020 09:44:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B2C7862FCF; Fri, 18 Sep 2020 11:44:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="hF2Ilwxb"; dkim-atps=neutral Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1196E62FBF for ; Fri, 18 Sep 2020 11:44:03 +0200 (CEST) Received: by mail-wr1-x433.google.com with SMTP id j2so4939849wrx.7 for ; Fri, 18 Sep 2020 02:44:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qXKgORLdFipnKwdsuEbwQtPSTduRWQCqLlS+WdmGLhg=; b=hF2IlwxbbpcgE73JQiAxCoCYppspsN5SkWBlgWvnxEqPoCa4IrtgFsvBIs7wOGimmw JtFIAUUPkd5mlucVO50IM2IkM0AF3wgliKqQ4TT7WTFoAPrmxVPnxH0llgQLZcvGJ9MI XixfypC9MQRScVgp2nJx2gNDxvOVXa3nvJM1cCuQxnpBFlmdKvOD/t+cKkY6GkB3IMXY T6AXsFGSbc9rNJpLmsums4JGd0YdSAGv39OrBFSVhynsi+X+5nQsgxPrfsrJbk2bFLXo xbPCQTfciFu/Z+T+8tvqFySxx+O2tExH/+DXALVa0pVnrSuv+amrtkXFuif82fM4O8zw K6pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qXKgORLdFipnKwdsuEbwQtPSTduRWQCqLlS+WdmGLhg=; b=N7hrQM8eQGeUf+UUdAh56v11trrPZE2Y4w6Ou4mdPoceUFPVzPz03NeYtMZ8MMqpeb kOdyz88sAeGYQljxXLSxXCNn+Nvtm4Bq2spjnLySEY1um4tkVeC8Ct6cXjSC73MXhP3l F9gKhif/LWW82wVt05jxRpFbg+swPz0x6j/giikH3ibZjUll1jPrETTdanms/VMuVhVv oOweFFKF0+U9OCnnE+dElMXRnxRypwsVkDmCJ1zV8q9adD17vpyyX2DOWEyvuq6GxOt4 37qbYuh84aXhjQsp9dIRfB1+ZgEKT+Ho7lhJH7Y6GvzNRKytqXhjy18wdbYxqPt8dfRE YMGQ== X-Gm-Message-State: AOAM5335zCCKdU0EYbq6ZGucaHtvunDa9baiqSRwkyy960NOnGDLCS3P gm40qWVZE9ZwguUmX5La8LVYI3ju1x6SUQ== X-Google-Smtp-Source: ABdhPJxG2pIlJmsaUGcrcdAgQvUfIiRylfmd5iWri9S745xqhl/LaxbuSL0Xnu3VbDHmM1GoJU5ImA== X-Received: by 2002:adf:de8b:: with SMTP id w11mr36595619wrl.401.1600422242406; Fri, 18 Sep 2020 02:44:02 -0700 (PDT) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id n2sm4792472wma.29.2020.09.18.02.44.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 02:44:01 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Sep 2020 10:42:29 +0100 Message-Id: <20200918094233.5273-9-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918094233.5273-1-naush@raspberrypi.com> References: <20200918094233.5273-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 08/12] libcamera: pipeline: raspberrypi: Add more robust stream buffer logic X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add further queueing into the RPiStream object to ensure that we always follow the buffer ordering (be it internal or external) given by incoming Requests. This is essential, otherwise we risk dropping frames that are meant to be part of a Request, and can cause the pipeline to stall indefinitely. This also prevents any possibility of mismatched frame buffers going through the pipeline and out to the application. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Tested-by: David Plowman --- .../pipeline/raspberrypi/rpi_stream.cpp | 76 +++++++++++++++---- .../pipeline/raspberrypi/rpi_stream.h | 13 +++- 2 files changed, 75 insertions(+), 14 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index b4d737db..b2a5dfa7 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -94,38 +94,75 @@ int RPiStream::queueBuffer(FrameBuffer *buffer) { /* * A nullptr buffer implies an external stream, but no external - * buffer has been supplied. So, pick one from the availableBuffers_ - * queue. + * buffer has been supplied in the Request. So, pick one from the + * availableBuffers_ queue. */ if (!buffer) { if (availableBuffers_.empty()) { - LOG(RPISTREAM, Warning) << "No buffers available for " + LOG(RPISTREAM, Info) << "No buffers available for " << name_; - return -EINVAL; + /* + * Note that we need to queue an internal buffer as soon + * as one becomes available. + */ + requestBuffers_.push(nullptr); + return 0; } buffer = availableBuffers_.front(); availableBuffers_.pop(); } - LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() - << " for " << name_; + /* + * If no earlier requests are pending to be queued we can go ahead and + * queue this buffer into the device. + */ + if (requestBuffers_.empty()) + return queueToDevice(buffer); - int ret = dev_->queueBuffer(buffer); - if (ret) { - LOG(RPISTREAM, Error) << "Failed to queue buffer for " - << name_; - } + /* + * There are earlier Request buffers to be queued, so this buffer must go + * on the waiting list. + */ + requestBuffers_.push(buffer); - return ret; + return 0; } void RPiStream::returnBuffer(FrameBuffer *buffer) { /* This can only be called for external streams. */ - assert(external_); + ASSERT(external_); + /* Push this buffer back into the queue to be used again. */ availableBuffers_.push(buffer); + + /* + * Do we have any Request buffers that are waiting to be queued? + * If so, do it now as availableBuffers_ will not be empty. + */ + while (!requestBuffers_.empty()) { + FrameBuffer *buffer = requestBuffers_.front(); + + if (!buffer) { + /* + * We want to queue an internal buffer, but none + * are available. Can't do anything, quit the loop. + */ + if (availableBuffers_.empty()) + break; + + /* + * We want to queue an internal buffer, and at least one + * is available. + */ + buffer = availableBuffers_.front(); + availableBuffers_.pop(); + } + + requestBuffers_.pop(); + queueToDevice(buffer); + } } int RPiStream::queueAllBuffers() @@ -155,10 +192,23 @@ void RPiStream::releaseBuffers() void RPiStream::clearBuffers() { availableBuffers_ = std::queue{}; + requestBuffers_ = std::queue{}; internalBuffers_.clear(); bufferList_.clear(); } +int RPiStream::queueToDevice(FrameBuffer *buffer) +{ + LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + << " for " << name_; + + int ret = dev_->queueBuffer(buffer); + if (ret) + LOG(RPISTREAM, Error) << "Failed to queue buffer for " + << name_; + return ret; +} + } /* namespace RPi */ } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index 73954ec7..f42e25b9 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -57,6 +57,7 @@ public: private: void clearBuffers(); + int queueToDevice(FrameBuffer *buffer); /* * Indicates that this stream is active externally, i.e. the buffers @@ -73,7 +74,7 @@ private: /* The actual device stream. */ std::unique_ptr dev_; - /* All framebuffers associated with this device stream. */ + /* All frame buffers associated with this device stream. */ std::vector bufferList_; /* @@ -83,6 +84,16 @@ private: */ std::queue availableBuffers_; + /* + * List of frame buffers that are to be queued into the device from a Request. + * A nullptr indicates any internal buffer can be used (from availableBuffers_), + * whereas a valid pointer indicates an external buffer to be queued. + * + * Ordering buffers to be queued is important here as it must match the + * requests coming from the application. + */ + std::queue requestBuffers_; + /* * This is a list of buffers exported internally. Need to keep this around * as the stream needs to maintain ownership of these buffers. From patchwork Fri Sep 18 09:42:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9654 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 589AAC3B5B for ; Fri, 18 Sep 2020 09:44:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 174F062FD1; Fri, 18 Sep 2020 11:44:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Mu9DPTGW"; dkim-atps=neutral Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6DEEC62FC1 for ; Fri, 18 Sep 2020 11:44:04 +0200 (CEST) Received: by mail-wr1-x435.google.com with SMTP id x14so4923085wrl.12 for ; Fri, 18 Sep 2020 02:44:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZgkjYrRlb6RnGOD3nqFpnULCo+StRs9OlxWaznIBygQ=; b=Mu9DPTGWEe2p/GVu9CyXXMqKaV1RrPUgdE1f2BEOM9gd3pMTyFZVZpL1Fnc0yDIcoE wC/PNkP0sqfLp5TA67ckZpu8SyIbt4AMt1BzdK8g5pDY3m5hjay3WMYtuRU6ad1k+a2V iXgL+72KZb7s6kDTi+9/SJH3H+tYtGw1PtI59DEkKFX26Jmll7viCDtTh7/oKnFfGX1r mzqIZun2MNKIZl9F23cNIYh5VoWwygGyuO7DiCGA5QcSHOgUZxa2jhovXyzHmtPIGXlq f3uHpFQHL2y/4euDLgHRlWLhEeDh9fqfGWPGsS99x9l7mflyc/3fqpbrz/wY/OxFQSUD gEHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZgkjYrRlb6RnGOD3nqFpnULCo+StRs9OlxWaznIBygQ=; b=TWgMFeblLOxmr5gerlEzaKZ3IBaA9vLuiEr4DnOzPjxWqVHpA0cS+ta1j26wvH5AKe XgKrfxLTk8hO1WbfIwXcLLVzf1vgSkJqPQcw+2Tq0atiTDKSElScbLx539oeCOvL+0YR k2mGEmtvFzRCh9sKrKntT8LEGx9l6ZyWOeJR0orFOGosQULXtWBW0EqkJ7mLQHLVVO0B cdjU1xK0DKgalgJ++kA6qD/EV9GMAC3uncz1ncnSdJt9IGnInc/mcf1o1JRzK+j64RQZ efnSxFh4FDdcHGPlX4SnYFNnF3sFEm9LT+WFfTt382SqAptyOauwbFSgoXPqYSsS6Cij clzQ== X-Gm-Message-State: AOAM530uRyNVSuKguAOWWL3E9s7PTYJB+gslD0hDgBEAg43n7Hn8luKC /faw36Ci13lPA8hP55jE59YYIZVGL1TtvQ== X-Google-Smtp-Source: ABdhPJwtsk9lj8I7nZ4Fc/qim+RHEoB1EVKG8BzDauyxZVvkFwq3cW+/8sjUkqjuCUQzUoY36fi7iQ== X-Received: by 2002:a5d:40cd:: with SMTP id b13mr36981648wrq.297.1600422243736; Fri, 18 Sep 2020 02:44:03 -0700 (PDT) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id n2sm4792472wma.29.2020.09.18.02.44.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 02:44:02 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Sep 2020 10:42:30 +0100 Message-Id: <20200918094233.5273-10-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918094233.5273-1-naush@raspberrypi.com> References: <20200918094233.5273-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 09/12] libcamera: pipeline: ipa: raspberrypi: Remove use of FrameBuffer cookie X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The FrameBuffer cookie may be set by the application, so this cannot be set by the pipeline handler as well. Revert to using a simple index into the buffer list to identify buffers passing to and from the IPA. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Tested-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 60 ++++++++++--------- .../pipeline/raspberrypi/rpi_stream.cpp | 14 +++-- .../pipeline/raspberrypi/rpi_stream.h | 2 +- 3 files changed, 40 insertions(+), 36 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 3caf3655..7d91188c 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -890,7 +890,8 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera) int PipelineHandlerRPi::prepareBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera); - int count, ret; + unsigned int index; + int ret; /* * Decide how many internal buffers to allocate. For now, simply look @@ -910,30 +911,24 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) } /* - * Add cookies to the ISP Input buffers so that we can link them with - * the IPA and RPI_IPA_EVENT_SIGNAL_ISP_PREPARE event. - */ - count = 0; - for (auto const &b : data->unicam_[Unicam::Image].getBuffers()) { - b->setCookie(count++); - } - - /* - * Add cookies to the stats and embedded data buffers and link them with - * the IPA. + * Link the FrameBuffers with the index of their position in the vector + * stored in the RPi stream object. + * + * This will allow us to identify buffers passed between the pipeline + * handler and the IPA. */ - count = 0; + index = 0; for (auto const &b : data->isp_[Isp::Stats].getBuffers()) { - b->setCookie(count++); - data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | b->cookie(), + data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | index, .planes = b->planes() }); + index++; } - count = 0; + index = 0; for (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) { - b->setCookie(count++); - data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | b->cookie(), + data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | index, .planes = b->planes() }); + index++; } data->ipa_->mapBuffers(data->ipaBuffers_); @@ -1106,7 +1101,7 @@ void RPiCameraData::queueFrameAction([[maybe_unused]] unsigned int frame, unsigned int bufferId = action.data[0]; FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers().at(bufferId); - LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << buffer->cookie() + LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << bufferId << ", timestamp: " << buffer->metadata().timestamp; isp_[Isp::Input].queueBuffer(buffer); @@ -1126,12 +1121,14 @@ done: void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) { RPi::RPiStream *stream = nullptr; + int index; if (state_ == State::Stopped) return; for (RPi::RPiStream &s : unicam_) { - if (s.findFrameBuffer(buffer)) { + index = s.getBufferIndex(buffer); + if (index != -1) { stream = &s; break; } @@ -1141,7 +1138,7 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) ASSERT(stream); LOG(RPI, Debug) << "Stream " << stream->name() << " buffer dequeue" - << ", buffer id " << buffer->cookie() + << ", buffer id " << index << ", timestamp: " << buffer->metadata().timestamp; if (stream == &unicam_[Unicam::Image]) { @@ -1181,7 +1178,7 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) return; LOG(RPI, Debug) << "Stream ISP Input buffer complete" - << ", buffer id " << buffer->cookie() + << ", buffer id " << unicam_[Unicam::Image].getBufferIndex(buffer) << ", timestamp: " << buffer->metadata().timestamp; /* The ISP input buffer gets re-queued into Unicam. */ @@ -1192,12 +1189,14 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) { RPi::RPiStream *stream = nullptr; + int index; if (state_ == State::Stopped) return; for (RPi::RPiStream &s : isp_) { - if (s.findFrameBuffer(buffer)) { + index = s.getBufferIndex(buffer); + if (index != -1) { stream = &s; break; } @@ -1207,7 +1206,7 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) ASSERT(stream); LOG(RPI, Debug) << "Stream " << stream->name() << " buffer complete" - << ", buffer id " << buffer->cookie() + << ", buffer id " << index << ", timestamp: " << buffer->metadata().timestamp; /* @@ -1217,7 +1216,7 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) if (stream == &isp_[Isp::Stats]) { IPAOperationData op; op.operation = RPI_IPA_EVENT_SIGNAL_STAT_READY; - op.data = { RPiIpaMask::STATS | buffer->cookie() }; + op.data = { RPiIpaMask::STATS | static_cast(index) }; ipa_->processEvent(op); } else { /* Any other ISP output can be handed back to the application now. */ @@ -1437,13 +1436,16 @@ void RPiCameraData::tryRunPipeline() /* Set our state to say the pipeline is active. */ state_ = State::Busy; + unsigned int bayerIndex = unicam_[Unicam::Image].getBufferIndex(bayerBuffer); + unsigned int embeddedIndex = unicam_[Unicam::Embedded].getBufferIndex(embeddedBuffer); + LOG(RPI, Debug) << "Signalling RPI_IPA_EVENT_SIGNAL_ISP_PREPARE:" - << " Bayer buffer id: " << bayerBuffer->cookie() - << " Embedded buffer id: " << embeddedBuffer->cookie(); + << " Bayer buffer id: " << bayerIndex + << " Embedded buffer id: " << embeddedIndex; op.operation = RPI_IPA_EVENT_SIGNAL_ISP_PREPARE; - op.data = { RPiIpaMask::EMBEDDED_DATA | embeddedBuffer->cookie(), - RPiIpaMask::BAYER_DATA | bayerBuffer->cookie() }; + op.data = { RPiIpaMask::EMBEDDED_DATA | embeddedIndex, + RPiIpaMask::BAYER_DATA | bayerIndex }; ipa_->processEvent(op); } diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index b2a5dfa7..80170d64 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -53,15 +53,17 @@ const std::vector &RPiStream::getBuffers() const return bufferList_; } -bool RPiStream::findFrameBuffer(FrameBuffer *buffer) const +int RPiStream::getBufferIndex(FrameBuffer *buffer) const { if (importOnly_) - return false; + return -1; - if (std::find(bufferList_.begin(), bufferList_.end(), buffer) != bufferList_.end()) - return true; + /* Find the buffer in the list, and return the index position. */ + auto it = std::find(bufferList_.begin(), bufferList_.end(), buffer); + if (it != bufferList_.end()) + return std::distance(bufferList_.begin(), it); - return false; + return -1; } int RPiStream::prepareBuffers(unsigned int count) @@ -199,7 +201,7 @@ void RPiStream::clearBuffers() int RPiStream::queueToDevice(FrameBuffer *buffer) { - LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + LOG(RPISTREAM, Debug) << "Queuing buffer " << getBufferIndex(buffer) << " for " << name_; int ret = dev_->queueBuffer(buffer); diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index f42e25b9..959e9147 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -46,7 +46,7 @@ public: void setExportedBuffers(std::vector> *buffers); const std::vector &getBuffers() const; - bool findFrameBuffer(FrameBuffer *buffer) const; + int getBufferIndex(FrameBuffer *buffer) const; int prepareBuffers(unsigned int count); int queueBuffer(FrameBuffer *buffer); From patchwork Fri Sep 18 09:42:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9655 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CD8FAC3B5B for ; Fri, 18 Sep 2020 09:44:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 99E7162FCD; Fri, 18 Sep 2020 11:44:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="dFh78i+M"; dkim-atps=neutral Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AC5C062FCD for ; Fri, 18 Sep 2020 11:44:05 +0200 (CEST) Received: by mail-wr1-x42c.google.com with SMTP id w5so4935075wrp.8 for ; Fri, 18 Sep 2020 02:44:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hxq5IjBzS+WFtXE8uRz4DB3Vof4h0VUl6+n57VaVinw=; b=dFh78i+MVzoYpWElAuZu6fWI6gsk9HLfyeXxpY2EJoqZLfayccypX2S19SgSQKcZc/ 465piy5df0rLZMFo01YPk1j+/KMiPCuQ4cRpZcOOExwp1gensUM0qCT3fNZNnt7ojz7k 5gPgVkiYTPj9N2XcfQGLhp3c7t8T5xAoi3L+5fM6LlvZgKE9R09jDEXvFU/JQhKl3Bwx r3KTKHqfwW0REIaFMdoMO2zAQNIfX0PFFe82enxYhTeWSSqP+SMBKJu8iWmcAbGxqEol a0Ro/xWaxhQfv6jEnoN6MIQJYaqkg7aLRw7A3j6+w/kVLsiXrViTtmQUJW15txIHbHZ6 vszw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hxq5IjBzS+WFtXE8uRz4DB3Vof4h0VUl6+n57VaVinw=; b=LgSLpp9zPYrBcmk83G5E0BwLRNQ8AcW/3VSOo2VdrfXGUYV53Pa51260VnEts8NB3Z A0Q1Xd15aAQ5G026EkjFzo90T9vz6zdPFY/R1NmwwHBzGNH6KPuprMaaQ1xXRfppRftV /3I4pRPiHHBOtcm0iSTwKsRsmBYaeQ89ZecsoHfLprRtHWYFgxMZ719aer+VMZ7YhMMU fexi+8w66UJou6VhfKH2BHJuK3R7kG4n/yw4jA936ClaC+60R38rMq05/iIvvGyJH4WA 3/3sM90uagyOk0FemMnvzH1bNweULz/2CEdVlHd5yIdm8H4D2VONFY063gFL51nRPy6Z Ushw== X-Gm-Message-State: AOAM532BOLqnGBhuV7BIbLu4ef1V3RGTkwoimsCjX0nmfHGs6YtUOBhT 3VYzn25Brcr3rdyaIPiNmFeIZbvNphu8ew== X-Google-Smtp-Source: ABdhPJzJMD4BF/TArLXjKhYW3WRr61qAfTMefLr+AVVcFe/AcIgoQvR8cjDIzQiIrGf47WRVlrvIeg== X-Received: by 2002:adf:d4c7:: with SMTP id w7mr37494836wrk.263.1600422244922; Fri, 18 Sep 2020 02:44:04 -0700 (PDT) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id n2sm4792472wma.29.2020.09.18.02.44.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 02:44:03 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Sep 2020 10:42:31 +0100 Message-Id: <20200918094233.5273-11-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918094233.5273-1-naush@raspberrypi.com> References: <20200918094233.5273-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 10/12] pipeline: raspberrypi: Use an unordered_map for the stream buffer list X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" By using a map container, we can easily insert/remove buffers from the buffer list. This will be required to track buffers allocated externally and passed to the pipeline handler through a Request. Replace the buffer index tracking with an id generated internally by the stream object. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: Kieran Bingham Reviewed-by: Niklas Söderlund --- .../pipeline/raspberrypi/raspberrypi.cpp | 31 +++++------ .../pipeline/raspberrypi/rpi_stream.cpp | 32 ++++++----- .../pipeline/raspberrypi/rpi_stream.h | 54 +++++++++++++++++-- 3 files changed, 82 insertions(+), 35 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 7d91188c..51544233 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -890,7 +890,6 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera) int PipelineHandlerRPi::prepareBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera); - unsigned int index; int ret; /* @@ -917,18 +916,14 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * This will allow us to identify buffers passed between the pipeline * handler and the IPA. */ - index = 0; for (auto const &b : data->isp_[Isp::Stats].getBuffers()) { - data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | index, - .planes = b->planes() }); - index++; + data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | b.first, + .planes = b.second->planes() }); } - index = 0; for (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) { - data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | index, - .planes = b->planes() }); - index++; + data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | b.first, + .planes = b.second->planes() }); } data->ipa_->mapBuffers(data->ipaBuffers_); @@ -1127,7 +1122,7 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) return; for (RPi::RPiStream &s : unicam_) { - index = s.getBufferIndex(buffer); + index = s.getBufferId(buffer); if (index != -1) { stream = &s; break; @@ -1178,7 +1173,7 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) return; LOG(RPI, Debug) << "Stream ISP Input buffer complete" - << ", buffer id " << unicam_[Unicam::Image].getBufferIndex(buffer) + << ", buffer id " << unicam_[Unicam::Image].getBufferId(buffer) << ", timestamp: " << buffer->metadata().timestamp; /* The ISP input buffer gets re-queued into Unicam. */ @@ -1195,7 +1190,7 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) return; for (RPi::RPiStream &s : isp_) { - index = s.getBufferIndex(buffer); + index = s.getBufferId(buffer); if (index != -1) { stream = &s; break; @@ -1436,16 +1431,16 @@ void RPiCameraData::tryRunPipeline() /* Set our state to say the pipeline is active. */ state_ = State::Busy; - unsigned int bayerIndex = unicam_[Unicam::Image].getBufferIndex(bayerBuffer); - unsigned int embeddedIndex = unicam_[Unicam::Embedded].getBufferIndex(embeddedBuffer); + unsigned int bayerId = unicam_[Unicam::Image].getBufferId(bayerBuffer); + unsigned int embeddedId = unicam_[Unicam::Embedded].getBufferId(embeddedBuffer); LOG(RPI, Debug) << "Signalling RPI_IPA_EVENT_SIGNAL_ISP_PREPARE:" - << " Bayer buffer id: " << bayerIndex - << " Embedded buffer id: " << embeddedIndex; + << " Bayer buffer id: " << bayerId + << " Embedded buffer id: " << embeddedId; op.operation = RPI_IPA_EVENT_SIGNAL_ISP_PREPARE; - op.data = { RPiIpaMask::EMBEDDED_DATA | embeddedIndex, - RPiIpaMask::BAYER_DATA | bayerIndex }; + op.data = { RPiIpaMask::EMBEDDED_DATA | embeddedId, + RPiIpaMask::BAYER_DATA | bayerId }; ipa_->processEvent(op); } diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index 80170d64..aee0aa2d 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -44,26 +44,28 @@ bool RPiStream::isExternal() const void RPiStream::setExportedBuffers(std::vector> *buffers) { - std::transform(buffers->begin(), buffers->end(), std::back_inserter(bufferList_), - [](std::unique_ptr &b) { return b.get(); }); + for (auto const &buffer : *buffers) + bufferMap_.emplace(id_.get(), buffer.get()); } -const std::vector &RPiStream::getBuffers() const +const BufferMap &RPiStream::getBuffers() const { - return bufferList_; + return bufferMap_; } -int RPiStream::getBufferIndex(FrameBuffer *buffer) const +int RPiStream::getBufferId(FrameBuffer *buffer) const { if (importOnly_) return -1; - /* Find the buffer in the list, and return the index position. */ - auto it = std::find(bufferList_.begin(), bufferList_.end(), buffer); - if (it != bufferList_.end()) - return std::distance(bufferList_.begin(), it); + /* Find the buffer in the map, and return the buffer id. */ + auto it = std::find_if(bufferMap_.begin(), bufferMap_.end(), + [&buffer](auto const &p) { return p.second == buffer; }); - return -1; + if (it == bufferMap_.end()) + return -1; + + return it->first; } int RPiStream::prepareBuffers(unsigned int count) @@ -86,7 +88,7 @@ int RPiStream::prepareBuffers(unsigned int count) } /* We must import all internal/external exported buffers. */ - count = bufferList_.size(); + count = bufferMap_.size(); } return dev_->importBuffers(count); @@ -139,6 +141,9 @@ void RPiStream::returnBuffer(FrameBuffer *buffer) /* Push this buffer back into the queue to be used again. */ availableBuffers_.push(buffer); + /* Allow the buffer id to be reused. */ + id_.release(getBufferId(buffer)); + /* * Do we have any Request buffers that are waiting to be queued? * If so, do it now as availableBuffers_ will not be empty. @@ -196,12 +201,13 @@ void RPiStream::clearBuffers() availableBuffers_ = std::queue{}; requestBuffers_ = std::queue{}; internalBuffers_.clear(); - bufferList_.clear(); + bufferMap_.clear(); + id_.reset(); } int RPiStream::queueToDevice(FrameBuffer *buffer) { - LOG(RPISTREAM, Debug) << "Queuing buffer " << getBufferIndex(buffer) + LOG(RPISTREAM, Debug) << "Queuing buffer " << getBufferId(buffer) << " for " << name_; int ret = dev_->queueBuffer(buffer); diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index 959e9147..df986367 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -9,8 +9,10 @@ #include #include +#include #include +#include #include #include "libcamera/internal/v4l2_videodevice.h" @@ -19,6 +21,8 @@ namespace libcamera { namespace RPi { +using BufferMap = std::unordered_map; + /* * Device stream abstraction for either an internal or external stream. * Used for both Unicam and the ISP. @@ -27,12 +31,13 @@ class RPiStream : public Stream { public: RPiStream() + : id_(RPiIpaMask::ID) { } RPiStream(const char *name, MediaEntity *dev, bool importOnly = false) : external_(false), importOnly_(importOnly), name_(name), - dev_(std::make_unique(dev)) + dev_(std::make_unique(dev)), id_(RPiIpaMask::ID) { } @@ -45,8 +50,8 @@ public: bool isExternal() const; void setExportedBuffers(std::vector> *buffers); - const std::vector &getBuffers() const; - int getBufferIndex(FrameBuffer *buffer) const; + const BufferMap &getBuffers() const; + int getBufferId(FrameBuffer *buffer) const; int prepareBuffers(unsigned int count); int queueBuffer(FrameBuffer *buffer); @@ -56,6 +61,44 @@ public: void releaseBuffers(); private: + class IdGenerator + { + public: + IdGenerator(int max) + : max_(max), id_(0) + { + } + + int get() + { + int id; + if (!recycle_.empty()) { + id = recycle_.front(); + recycle_.pop(); + } else { + id = id_++; + ASSERT(id_ <= max_); + } + return id; + } + + void release(int id) + { + recycle_.push(id); + } + + void reset() + { + id_ = 0; + recycle_ = {}; + } + + private: + int max_; + int id_; + std::queue recycle_; + }; + void clearBuffers(); int queueToDevice(FrameBuffer *buffer); @@ -74,8 +117,11 @@ private: /* The actual device stream. */ std::unique_ptr dev_; + /* Tracks a unique id key for the bufferMap_ */ + IdGenerator id_; + /* All frame buffers associated with this device stream. */ - std::vector bufferList_; + BufferMap bufferMap_; /* * List of frame buffers that we can use if none have been provided by From patchwork Fri Sep 18 09:42:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9656 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3AB88C3B5B for ; Fri, 18 Sep 2020 09:44:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 092E762FC4; Fri, 18 Sep 2020 11:44:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="MI43aiZL"; dkim-atps=neutral Received: from mail-wr1-x441.google.com (mail-wr1-x441.google.com [IPv6:2a00:1450:4864:20::441]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 10FBB62FAD for ; Fri, 18 Sep 2020 11:44:07 +0200 (CEST) Received: by mail-wr1-x441.google.com with SMTP id t10so4961873wrv.1 for ; Fri, 18 Sep 2020 02:44:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VeqvmYrPOcZHFidx2F0bdpH4CTa6e7IqqNLmBrr4td4=; b=MI43aiZL93zDzh+/ViAjdRerpI+1DL/L8hHpGbSNt7rrPccMv2A6f43JNbyIw9mBbz uKBTYNp7XT6iy64PGw7RBaU6O8PS1VsslzEo7tSD/9hHDuIi05FWbL/YeZeYfCyFXhLL 8xg25QLlY0TGRyGKbAMOnETFpa7kPQYu8XloOP9HevXzvnTqmLw7S9vtqYIP2LGNZqgo Acl7iHLVkvyYouoi/i5JUx0urWR9RrXa3zPJ80dVfW1OwJvnEqtxUJ5aiUykuz1owivo 5mjXxI8j5NRExU3Q2FTwiIG/+cHhEIjIpeGxkSnhbyMPTSyQPtNJHsbxli+VsdwOUjWY t00g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VeqvmYrPOcZHFidx2F0bdpH4CTa6e7IqqNLmBrr4td4=; b=Z4PrSTVcto3IL6S+Ol2AMQqJJhoUSwk+E7JCoYAIIFAKgPmGZDiBL55/sEJRnffTgL Ob2JfS2SEdHD4vbNSRIlntVWUK5kJw234K0SRHnZXgcDsKaHuGPnz3gTkmm30qqEsuO4 eBBfm6vGv3ZWaIJ26zuuAbUvgRtGcNmkt1K1aPH8xdhcYqD2dehYGqrx6Ji7pNnv7jNN GMt6ReWXuud3A8/lUWd+ZGJQWr9ujyhCkTDpVUqNA3r7XxK2TKGysNHkUN3RGJxLTMrM iIqV2p6G5wttnTuoewOln8TKp/YvzkhxTboTfSbOYw2psGdJB+PKFlaMuX9pZv1S8uE6 oGtQ== X-Gm-Message-State: AOAM532o2+LcfvcXbZ/XyMkCfgyslHIvPoYD2QlydkcF03oA174UkTMp TNNzUYscL7ckUJgfNrV0YCibWFwM/PSxBg== X-Google-Smtp-Source: ABdhPJwETLDlad2lov/ngClBjEcrSBvF5UMHYy51vR1Imp5SGah7ihGxn0RjaQ/yr/plzrAw3/+mEQ== X-Received: by 2002:a5d:50cd:: with SMTP id f13mr32695803wrt.211.1600422246215; Fri, 18 Sep 2020 02:44:06 -0700 (PDT) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id n2sm4792472wma.29.2020.09.18.02.44.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 02:44:05 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Sep 2020 10:42:32 +0100 Message-Id: <20200918094233.5273-12-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918094233.5273-1-naush@raspberrypi.com> References: <20200918094233.5273-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 11/12] pipeline: raspberrypi: Use an unordered_set to store IPA buffer ids X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" By using a set container, we can easily insert/remove buffer ids that have been mmaped by the IPA. This will be required to track buffers allocated externally and passed to the pipeline handler through a Request. Move the IPA buffer mapping code into a function to remove duplicated code. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham --- .../pipeline/raspberrypi/raspberrypi.cpp | 48 +++++++++++-------- 1 file changed, 28 insertions(+), 20 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 51544233..4e12c7d6 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -158,8 +159,8 @@ public: RPi::RPiDevice isp_; /* The vector below is just for convenience when iterating over all streams. */ std::vector streams_; - /* Buffers passed to the IPA. */ - std::vector ipaBuffers_; + /* Stores the ids of the buffers mapped in the IPA. */ + std::unordered_set ipaBuffers_; /* DMAHEAP allocation helper. */ RPi::DmaHeap dmaHeap_; @@ -230,6 +231,7 @@ private: int queueAllBuffers(Camera *camera); int prepareBuffers(Camera *camera); void freeBuffers(Camera *camera); + void mapBuffers(Camera *camera, const RPi::BufferMap &buffers, unsigned int mask); MediaDevice *unicam_; MediaDevice *isp_; @@ -910,36 +912,42 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) } /* - * Link the FrameBuffers with the index of their position in the vector - * stored in the RPi stream object. + * Pass the stats and embedded data buffers to the IPA. No other + * buffers need to be passed. + */ + mapBuffers(camera, data->isp_[Isp::Stats].getBuffers(), RPiIpaMask::STATS); + mapBuffers(camera, data->unicam_[Unicam::Embedded].getBuffers(), RPiIpaMask::EMBEDDED_DATA); + + return 0; +} + +void PipelineHandlerRPi::mapBuffers(Camera *camera, const RPi::BufferMap &buffers, unsigned int mask) +{ + RPiCameraData *data = cameraData(camera); + std::vector ipaBuffers; + /* + * Link the FrameBuffers with the id (key value) in the map stored in + * the RPi stream object - along with an identifier mask. * * This will allow us to identify buffers passed between the pipeline * handler and the IPA. */ - for (auto const &b : data->isp_[Isp::Stats].getBuffers()) { - data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | b.first, - .planes = b.second->planes() }); - } - - for (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) { - data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | b.first, - .planes = b.second->planes() }); + for (auto const &it : buffers) { + ipaBuffers.push_back({ .id = mask | it.first, + .planes = it.second->planes() }); + data->ipaBuffers_.insert(mask | it.first); } - data->ipa_->mapBuffers(data->ipaBuffers_); - - return 0; + data->ipa_->mapBuffers(ipaBuffers); } void PipelineHandlerRPi::freeBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera); - std::vector ids; - for (IPABuffer &ipabuf : data->ipaBuffers_) - ids.push_back(ipabuf.id); - - data->ipa_->unmapBuffers(ids); + /* Copy the buffer ids from the unordered_set to a vector to pass to the IPA. */ + std::vector ipaBuffers(data->ipaBuffers_.begin(), data->ipaBuffers_.end()); + data->ipa_->unmapBuffers(ipaBuffers); data->ipaBuffers_.clear(); for (auto const stream : data->streams_) From patchwork Fri Sep 18 09:42:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9657 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CC51DC3B5B for ; Fri, 18 Sep 2020 09:44:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9308362FBC; Fri, 18 Sep 2020 11:44:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qsW1Sjpt"; dkim-atps=neutral Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C4EF862FC2 for ; Fri, 18 Sep 2020 11:44:07 +0200 (CEST) Received: by mail-wm1-x334.google.com with SMTP id e17so4681851wme.0 for ; Fri, 18 Sep 2020 02:44:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aNQtb8h4yI4yUYZQspEf/26vhR7dF0lxXt+8riCrbNs=; b=qsW1SjptF4i+OrNNE3MBOFNV/Pm2CbvC4dTQH3dtFmytAY4QItan/Xp3ZMmoiQAWEX EGWVwh1GVWc/A3QlwE2aBuT6LW8EOZUTWf0PaESHgJNYmUXGfM4iDMNEX6KePB8K0whs JFm5dRTDFyN2iKTvho2Zk+uB+Sc0ea9Kk8PWVQdd+dt3QmV8vN47Wsoq/0LLkkkLTKtd t3QfWmibv+5chRKXciUkGHmrabifCJhmBQ5ZioWqUOrIp3KKk3A5d2P9WktivjtAT2bz LJk5LV2cef8aP5saJwkDwi7PCVOM5Kt4Tvf0rJQoyOeO05DM3KGRf3dwTRGEPOIzF9Xq kqKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aNQtb8h4yI4yUYZQspEf/26vhR7dF0lxXt+8riCrbNs=; b=I6YWMOB2/yp06bPQzr6esF2UlsYPiHMMpTeug1X+15y5Ak1wvgusqXNRSMGpmKXKnI nAtI0pZhhzhIyBvqb52RwDb8sFGE2YY4aPcJAapzZxT9wjai14hdbWT1rXglbe/7Sf/P s9LD1Zc7C1O9oxtmWsKZuhYNBNkw4iRnDrz50KaI1PniPXifR/k4u74osABWutLhpQqe 50eBVKri9p5nwm9xH50gGNs/EEwpPCuPX7tt4Z1C3IhrJRBmYh/Uf2zQkh3jpWidzRsI AOBlUBnjjWR0WHgy1OtKBqAmHBidQwXRCkvWzbRJm+5KdxgGDPXcM5i/s8RTVcdFMro1 DmGw== X-Gm-Message-State: AOAM531dN9BciHWgXe/DZRwX1D3L5I6soPlxUXUpgBYPBTut0i1aus09 ZUPrE1DD/mtN7fhqZjMG8z5Hg/s7TkPnHQ== X-Google-Smtp-Source: ABdhPJwf0zOUa5+a+eBkJIZrvY6iV+/1Tn8hUb+UliRLOYY7GAq002JsoOBYKOoEiT9OgJ/NRi1AKw== X-Received: by 2002:a1c:5988:: with SMTP id n130mr313854wmb.95.1600422247067; Fri, 18 Sep 2020 02:44:07 -0700 (PDT) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id n2sm4792472wma.29.2020.09.18.02.44.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 02:44:06 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 18 Sep 2020 10:42:33 +0100 Message-Id: <20200918094233.5273-13-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200918094233.5273-1-naush@raspberrypi.com> References: <20200918094233.5273-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 12/12] pipeline: ipa: raspberrypi: Handle any externally allocated FrameBuffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Handle the case where a FrameBuffer that has been externally allocated (i.e. not through the v4l2 video device) is passed into a Request. We must store the buffer pointer in the stream internal buffer list to identify when used. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham Reviewed-by: Niklas Söderlund --- include/libcamera/ipa/raspberrypi.h | 11 ++-- src/ipa/raspberrypi/raspberrypi.cpp | 6 +- .../pipeline/raspberrypi/raspberrypi.cpp | 64 +++++++++++++------ .../pipeline/raspberrypi/rpi_stream.cpp | 14 ++++ .../pipeline/raspberrypi/rpi_stream.h | 7 +- 5 files changed, 73 insertions(+), 29 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index 262fc6f3..dd6ebeac 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -28,11 +28,12 @@ enum RPiOperations { RPI_IPA_EVENT_QUEUE_REQUEST, }; -enum RPiIpaMask { - ID = 0x0ffff, - STATS = 0x10000, - EMBEDDED_DATA = 0x20000, - BAYER_DATA = 0x40000 +enum RPiBufferMask { + ID = 0x00ffff, + STATS = 0x010000, + EMBEDDED_DATA = 0x020000, + BAYER_DATA = 0x040000, + EXTERNAL_BUFFER = 0x100000, }; /* Size of the LS grid allocation. */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 688d2efc..0555cc4e 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -352,7 +352,7 @@ void IPARPi::processEvent(const IPAOperationData &event) IPAOperationData op; op.operation = RPI_IPA_ACTION_STATS_METADATA_COMPLETE; - op.data = { bufferId & RPiIpaMask::ID }; + op.data = { bufferId & RPiBufferMask::ID }; op.controls = { libcameraMetadata_ }; queueFrameAction.emit(0, op); break; @@ -373,7 +373,7 @@ void IPARPi::processEvent(const IPAOperationData &event) /* Ready to push the input buffer into the ISP. */ IPAOperationData op; op.operation = RPI_IPA_ACTION_RUN_ISP; - op.data = { bayerbufferId & RPiIpaMask::ID }; + op.data = { bayerbufferId & RPiBufferMask::ID }; queueFrameAction.emit(0, op); break; } @@ -700,7 +700,7 @@ void IPARPi::returnEmbeddedBuffer(unsigned int bufferId) { IPAOperationData op; op.operation = RPI_IPA_ACTION_EMBEDDED_COMPLETE; - op.data = { bufferId & RPiIpaMask::ID }; + op.data = { bufferId & RPiBufferMask::ID }; queueFrameAction.emit(0, op); } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 4e12c7d6..50f07182 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -151,6 +151,7 @@ public: void clearIncompleteRequests(); void handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream); + void handleExternalBuffer(FrameBuffer *buffer, RPi::RPiStream *stream); void handleState(); CameraSensor *sensor_; @@ -725,23 +726,32 @@ int PipelineHandlerRPi::queueRequestDevice(Camera *camera, Request *request) /* Push all buffers supplied in the Request to the respective streams. */ for (auto stream : data->streams_) { - if (stream->isExternal()) { - FrameBuffer *buffer = request->findBuffer(stream); + if (!stream->isExternal()) + continue; + + FrameBuffer *buffer = request->findBuffer(stream); + if (buffer && stream->getBufferId(buffer) == -1) { /* - * If no buffer is provided by the request for this stream, we - * queue a nullptr to the stream to signify that it must use an - * internally allocated buffer for this capture request. This - * buffer will not be given back to the application, but is used - * to support the internal pipeline flow. - * - * The below queueBuffer() call will do nothing if there are not - * enough internal buffers allocated, but this will be handled by - * queuing the request for buffers in the RPiStream object. + * This buffer is not recognised, so it must have been allocated + * outside the v4l2 device. Store it in the stream buffer list + * so we can track it. */ - int ret = stream->queueBuffer(buffer); - if (ret) - return ret; + stream->setExternalBuffer(buffer); } + /* + * If no buffer is provided by the request for this stream, we + * queue a nullptr to the stream to signify that it must use an + * internally allocated buffer for this capture request. This + * buffer will not be given back to the application, but is used + * to support the internal pipeline flow. + * + * The below queueBuffer() call will do nothing if there are not + * enough internal buffers allocated, but this will be handled by + * queuing the request for buffers in the RPiStream object. + */ + int ret = stream->queueBuffer(buffer); + if (ret) + return ret; } /* Push the request to the back of the queue. */ @@ -915,8 +925,8 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * Pass the stats and embedded data buffers to the IPA. No other * buffers need to be passed. */ - mapBuffers(camera, data->isp_[Isp::Stats].getBuffers(), RPiIpaMask::STATS); - mapBuffers(camera, data->unicam_[Unicam::Embedded].getBuffers(), RPiIpaMask::EMBEDDED_DATA); + mapBuffers(camera, data->isp_[Isp::Stats].getBuffers(), RPiBufferMask::STATS); + mapBuffers(camera, data->unicam_[Unicam::Embedded].getBuffers(), RPiBufferMask::EMBEDDED_DATA); return 0; } @@ -1219,7 +1229,7 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) if (stream == &isp_[Isp::Stats]) { IPAOperationData op; op.operation = RPI_IPA_EVENT_SIGNAL_STAT_READY; - op.data = { RPiIpaMask::STATS | static_cast(index) }; + op.data = { RPiBufferMask::STATS | static_cast(index) }; ipa_->processEvent(op); } else { /* Any other ISP output can be handed back to the application now. */ @@ -1297,6 +1307,11 @@ void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stre */ Request *request = requestQueue_.empty() ? nullptr : requestQueue_.front(); if (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) { + /* + * Check if this is an externally provided buffer, and if + * so, we must stop tracking it in the pipeline handler. + */ + handleExternalBuffer(buffer, stream); /* * Tag the buffer as completed, returning it to the * application. @@ -1315,6 +1330,17 @@ void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stre } } +void RPiCameraData::handleExternalBuffer(FrameBuffer *buffer, RPi::RPiStream *stream) +{ + unsigned int id = stream->getBufferId(buffer); + + if (!(id & RPiBufferMask::EXTERNAL_BUFFER)) + return; + + /* Stop the Stream object from tracking the buffer. */ + stream->removeExternalBuffer(buffer); +} + void RPiCameraData::handleState() { switch (state_) { @@ -1447,8 +1473,8 @@ void RPiCameraData::tryRunPipeline() << " Embedded buffer id: " << embeddedId; op.operation = RPI_IPA_EVENT_SIGNAL_ISP_PREPARE; - op.data = { RPiIpaMask::EMBEDDED_DATA | embeddedId, - RPiIpaMask::BAYER_DATA | bayerId }; + op.data = { RPiBufferMask::EMBEDDED_DATA | embeddedId, + RPiBufferMask::BAYER_DATA | bayerId }; ipa_->processEvent(op); } diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index aee0aa2d..2dcf96ca 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -68,6 +68,20 @@ int RPiStream::getBufferId(FrameBuffer *buffer) const return it->first; } +void RPiStream::setExternalBuffer(FrameBuffer *buffer) +{ + bufferMap_.emplace(RPiBufferMask::EXTERNAL_BUFFER | id_.get(), buffer); +} + +void RPiStream::removeExternalBuffer(FrameBuffer *buffer) +{ + int id = getBufferId(buffer); + + /* Ensure we have this buffer in the stream, and it is marked external. */ + ASSERT(id != -1 && (id & RPiBufferMask::EXTERNAL_BUFFER)); + bufferMap_.erase(id); +} + int RPiStream::prepareBuffers(unsigned int count) { int ret; diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index df986367..a2c21bcd 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -31,13 +31,13 @@ class RPiStream : public Stream { public: RPiStream() - : id_(RPiIpaMask::ID) + : id_(RPiBufferMask::ID) { } RPiStream(const char *name, MediaEntity *dev, bool importOnly = false) : external_(false), importOnly_(importOnly), name_(name), - dev_(std::make_unique(dev)), id_(RPiIpaMask::ID) + dev_(std::make_unique(dev)), id_(RPiBufferMask::ID) { } @@ -53,6 +53,9 @@ public: const BufferMap &getBuffers() const; int getBufferId(FrameBuffer *buffer) const; + void setExternalBuffer(FrameBuffer *buffer); + void removeExternalBuffer(FrameBuffer *buffer); + int prepareBuffers(unsigned int count); int queueBuffer(FrameBuffer *buffer); void returnBuffer(FrameBuffer *buffer);