From patchwork Tue Aug 4 09:58:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9166 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 755DFBD86F for ; Tue, 4 Aug 2020 09:58:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4334C60499; Tue, 4 Aug 2020 11:58:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="g0ajAWrB"; dkim-atps=neutral Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 942216042B for ; Tue, 4 Aug 2020 11:58:57 +0200 (CEST) Received: by mail-wr1-x42b.google.com with SMTP id z18so33237638wrm.12 for ; Tue, 04 Aug 2020 02:58:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BX33WxPjQxFM26LTJ/1Sd5KBaSlmZ/nG5OzAMvNlULg=; b=g0ajAWrBUI/oEvr+Gwe0fNn+GKOgFUrijL4JwKHOnhEDxSZZjm6lTD6g4H7h3o0Lup +IvHTWq69d9qhHvSkNrs9j7n6FOEq0rl0kqgVtJDKu+nNlbTufNur+ZsauD1qKstbeeZ Kzfjay1bWR+D3efRUjjb0inmEj/n4BZmE+GowpZg92gARnEp4FnA/YGIq4BwQqKX15pP DQNQiGe3WVtFbeDLAawwUwXdGZJKov2zwzcP4EGr26edCaCBsiahQI4QGStDq8auZ+DT H/7Ln0oTZN2IMftzluZI85VBf/rplcunHk9gL8BDN4G2eNrgUb/CenaTXGi+cF50k8A7 79FA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BX33WxPjQxFM26LTJ/1Sd5KBaSlmZ/nG5OzAMvNlULg=; b=chEh6PW6dgWBeMZFEaKcSNHEbYz4JhXw5oEYEaweCzjc8lo3TSOeVew4s9yCToi1j7 QsrokkSerqU792NkZ2OJOI11uVvXq68yCaE8GipzSpcP63sXB1hjawXBmAhJz2Fk09BJ e1F4y9i1LA/QgYTdPk8xhhcNQvhzPwgW4Mx5a12V+nwcXC8EE3dgKsXFRgTBw6pXOHnK BGnzZCF/I6uB9PotXijhIIYJ+s4p3FVKkiz+zxIb1X1yanjb2ThxFdtr2jO8nwpuHIqZ a6CsCZebMG4SWvPMEA5dtyk1xfIDB3lxEm7MCdM4/dhZYiQ2l/CLreAL+4b/BTbPciI3 qN0w== X-Gm-Message-State: AOAM532bLhPjIP04AyfPTBhK4NhBzpX6hyAj1sPgqmU3P+KgRTRIS+MP wsvqH5pidHuVOg71o/qv7ftplgNPUqNW2w== X-Google-Smtp-Source: ABdhPJy4Y2l6wkEd1V3fFQDQn4i46LDPlD2+FNpsHQGE0GwP54wCwqNpQO+ae4x/ReF7irjCTaXCxA== X-Received: by 2002:adf:82d5:: with SMTP id 79mr18178566wrc.282.1596535136648; Tue, 04 Aug 2020 02:58:56 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id 32sm34047234wrh.18.2020.08.04.02.58.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 02:58:55 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Aug 2020 10:58:39 +0100 Message-Id: <20200804095850.275499-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200804095850.275499-1-naush@raspberrypi.com> References: <20200804095850.275499-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 01/12] libcamera: pipeline: raspberrypi: Move RPiStream into a separate file X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Put RPiStream into the RPi namespace and add a new log category (RPISTREAM). Reorder methods into logical groups for readability. There are no functional changes in this commit. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Tested-by: David Plowman --- .../pipeline/raspberrypi/meson.build | 1 + .../pipeline/raspberrypi/raspberrypi.cpp | 193 ++---------------- .../pipeline/raspberrypi/rpi_stream.cpp | 116 +++++++++++ .../pipeline/raspberrypi/rpi_stream.h | 107 ++++++++++ 4 files changed, 243 insertions(+), 174 deletions(-) create mode 100644 src/libcamera/pipeline/raspberrypi/rpi_stream.cpp create mode 100644 src/libcamera/pipeline/raspberrypi/rpi_stream.h diff --git a/src/libcamera/pipeline/raspberrypi/meson.build b/src/libcamera/pipeline/raspberrypi/meson.build index ae0aed3b..7c5b6ff7 100644 --- a/src/libcamera/pipeline/raspberrypi/meson.build +++ b/src/libcamera/pipeline/raspberrypi/meson.build @@ -3,5 +3,6 @@ libcamera_sources += files([ 'dma_heaps.cpp', 'raspberrypi.cpp', + 'rpi_stream.cpp', 'staggered_ctrl.cpp', ]) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 82a0a4df..90331ede 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -19,7 +19,6 @@ #include #include #include -#include #include @@ -33,6 +32,7 @@ #include "libcamera/internal/v4l2_videodevice.h" #include "dma_heaps.h" +#include "rpi_stream.h" #include "staggered_ctrl.h" namespace libcamera { @@ -122,165 +122,10 @@ V4L2DeviceFormat findBestMode(V4L2VideoDevice::Formats &formatsMap, return bestMode; } -} /* namespace */ - -/* - * Device stream abstraction for either an internal or external stream. - * Used for both Unicam and the ISP. - */ -class RPiStream : public Stream -{ -public: - RPiStream() - { - } - - RPiStream(const char *name, MediaEntity *dev, bool importOnly = false) - : external_(false), importOnly_(importOnly), name_(name), - dev_(std::make_unique(dev)) - { - } - - V4L2VideoDevice *dev() const - { - return dev_.get(); - } - - void setExternal(bool external) - { - external_ = external; - } - - bool isExternal() const - { - /* - * Import streams cannot be external. - * - * RAW capture is a special case where we simply copy the RAW - * buffer out of the request. All other buffer handling happens - * as if the stream is internal. - */ - return external_ && !importOnly_; - } - - bool isImporter() const - { - return importOnly_; - } - - void reset() - { - external_ = false; - internalBuffers_.clear(); - } - - std::string name() const - { - return name_; - } - - void setExternalBuffers(std::vector> *buffers) - { - externalBuffers_ = buffers; - } - - const std::vector> *getBuffers() const - { - return external_ ? externalBuffers_ : &internalBuffers_; - } - - void releaseBuffers() - { - dev_->releaseBuffers(); - if (!external_ && !importOnly_) - internalBuffers_.clear(); - } - - int importBuffers(unsigned int count) - { - return dev_->importBuffers(count); - } - - int allocateBuffers(unsigned int count) - { - return dev_->allocateBuffers(count, &internalBuffers_); - } - - int queueBuffers() - { - if (external_) - return 0; - - for (auto &b : internalBuffers_) { - int ret = dev_->queueBuffer(b.get()); - if (ret) { - LOG(RPI, Error) << "Failed to queue buffers for " - << name_; - return ret; - } - } - - return 0; - } - - bool findFrameBuffer(FrameBuffer *buffer) const - { - auto start = external_ ? externalBuffers_->begin() : internalBuffers_.begin(); - auto end = external_ ? externalBuffers_->end() : internalBuffers_.end(); - - if (importOnly_) - return false; - - if (std::find_if(start, end, - [buffer](std::unique_ptr const &ref) { return ref.get() == buffer; }) != end) - return true; - - return false; - } - -private: - /* - * Indicates that this stream is active externally, i.e. the buffers - * are provided by the application. - */ - bool external_; - /* Indicates that this stream only imports buffers, e.g. ISP input. */ - bool importOnly_; - /* Stream name identifier. */ - std::string name_; - /* The actual device stream. */ - std::unique_ptr dev_; - /* Internally allocated framebuffers associated with this device stream. */ - std::vector> internalBuffers_; - /* Externally allocated framebuffers associated with this device stream. */ - std::vector> *externalBuffers_; -}; - -/* - * The following class is just a convenient (and typesafe) array of device - * streams indexed with an enum class. - */ enum class Unicam : unsigned int { Image, Embedded }; enum class Isp : unsigned int { Input, Output0, Output1, Stats }; -template -class RPiDevice : public std::array -{ -private: - constexpr auto index(E e) const noexcept - { - return static_cast>(e); - } -public: - RPiStream &operator[](E e) - { - return std::array::operator[](index(e)); - } - const RPiStream &operator[](E e) const - { - return std::array::operator[](index(e)); - } -}; +} /* namespace */ class RPiCameraData : public CameraData { @@ -304,15 +149,15 @@ public: void ispOutputDequeue(FrameBuffer *buffer); void clearIncompleteRequests(); - void handleStreamBuffer(FrameBuffer *buffer, const RPiStream *stream); + void handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream); void handleState(); CameraSensor *sensor_; /* Array of Unicam and ISP device streams and associated buffers/streams. */ - RPiDevice unicam_; - RPiDevice isp_; + RPi::RPiDevice unicam_; + RPi::RPiDevice isp_; /* The vector below is just for convenience when iterating over all streams. */ - std::vector streams_; + std::vector streams_; /* Buffers passed to the IPA. */ std::vector ipaBuffers_; @@ -761,7 +606,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) int PipelineHandlerRPi::exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) { - RPiStream *s = static_cast(stream); + RPi::RPiStream *s = static_cast(stream); unsigned int count = stream->configuration().bufferCount; int ret = s->dev()->exportBuffers(count, buffers); @@ -907,14 +752,14 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) std::unique_ptr data = std::make_unique(this); /* Locate and open the unicam video streams. */ - data->unicam_[Unicam::Embedded] = RPiStream("Unicam Embedded", unicam_->getEntityByName("unicam-embedded")); - data->unicam_[Unicam::Image] = RPiStream("Unicam Image", unicam_->getEntityByName("unicam-image")); + data->unicam_[Unicam::Embedded] = RPi::RPiStream("Unicam Embedded", unicam_->getEntityByName("unicam-embedded")); + data->unicam_[Unicam::Image] = RPi::RPiStream("Unicam Image", unicam_->getEntityByName("unicam-image")); /* Tag the ISP input stream as an import stream. */ - data->isp_[Isp::Input] = RPiStream("ISP Input", isp_->getEntityByName("bcm2835-isp0-output0"), true); - data->isp_[Isp::Output0] = RPiStream("ISP Output0", isp_->getEntityByName("bcm2835-isp0-capture1")); - data->isp_[Isp::Output1] = RPiStream("ISP Output1", isp_->getEntityByName("bcm2835-isp0-capture2")); - data->isp_[Isp::Stats] = RPiStream("ISP Stats", isp_->getEntityByName("bcm2835-isp0-capture3")); + data->isp_[Isp::Input] = RPi::RPiStream("ISP Input", isp_->getEntityByName("bcm2835-isp0-output0"), true); + data->isp_[Isp::Output0] = RPi::RPiStream("ISP Output0", isp_->getEntityByName("bcm2835-isp0-capture1")); + data->isp_[Isp::Output1] = RPi::RPiStream("ISP Output1", isp_->getEntityByName("bcm2835-isp0-capture2")); + data->isp_[Isp::Stats] = RPi::RPiStream("ISP Stats", isp_->getEntityByName("bcm2835-isp0-capture3")); /* This is just for convenience so that we can easily iterate over all streams. */ for (auto &stream : data->unicam_) @@ -1004,7 +849,7 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) */ unsigned int maxBuffers = 0; for (const Stream *s : camera->streams()) - if (static_cast(s)->isExternal()) + if (static_cast(s)->isExternal()) maxBuffers = std::max(maxBuffers, s->configuration().bufferCount); for (auto const stream : data->streams_) { @@ -1254,12 +1099,12 @@ done: void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) { - const RPiStream *stream = nullptr; + const RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPiStream const &s : unicam_) { + for (RPi::RPiStream const &s : unicam_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1315,12 +1160,12 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) { - const RPiStream *stream = nullptr; + const RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPiStream const &s : isp_) { + for (RPi::RPiStream const &s : isp_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1401,7 +1246,7 @@ void RPiCameraData::clearIncompleteRequests() } } -void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPiStream *stream) +void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream) { if (stream->isExternal()) { if (!dropFrame_) { diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp new file mode 100644 index 00000000..2e52bd5c --- /dev/null +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -0,0 +1,116 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi (Trading) Ltd. + * + * rpi_stream.cpp - Raspberry Pi device stream abstraction class. + */ +#include "rpi_stream.h" + +#include "libcamera/internal/log.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(RPISTREAM) + +namespace RPi { + +V4L2VideoDevice *RPiStream::dev() const +{ + return dev_.get(); +} + +std::string RPiStream::name() const +{ + return name_; +} + +void RPiStream::reset() +{ + external_ = false; + internalBuffers_.clear(); +} + +bool RPiStream::isImporter() const +{ + return importOnly_; +} + +void RPiStream::setExternal(bool external) +{ + external_ = external; +} + +bool RPiStream::isExternal() const +{ + /* + * Import streams cannot be external. + * + * RAW capture is a special case where we simply copy the RAW + * buffer out of the request. All other buffer handling happens + * as if the stream is internal. + */ + return external_ && !importOnly_; +} + +void RPiStream::setExternalBuffers(std::vector> *buffers) +{ + externalBuffers_ = buffers; +} + +const std::vector> *RPiStream::getBuffers() const +{ + return external_ ? externalBuffers_ : &internalBuffers_; +} + +bool RPiStream::findFrameBuffer(FrameBuffer *buffer) const +{ + auto start = external_ ? externalBuffers_->begin() : internalBuffers_.begin(); + auto end = external_ ? externalBuffers_->end() : internalBuffers_.end(); + + if (importOnly_) + return false; + + if (std::find_if(start, end, + [buffer](std::unique_ptr const &ref) { return ref.get() == buffer; }) != end) + return true; + + return false; +} + +int RPiStream::importBuffers(unsigned int count) +{ + return dev_->importBuffers(count); +} + +int RPiStream::allocateBuffers(unsigned int count) +{ + return dev_->allocateBuffers(count, &internalBuffers_); +} + +int RPiStream::queueBuffers() +{ + if (external_) + return 0; + + for (auto &b : internalBuffers_) { + int ret = dev_->queueBuffer(b.get()); + if (ret) { + LOG(RPISTREAM, Error) << "Failed to queue buffers for " + << name_; + return ret; + } + } + + return 0; +} + +void RPiStream::releaseBuffers() +{ + dev_->releaseBuffers(); + if (!external_ && !importOnly_) + internalBuffers_.clear(); +} + +} /* namespace RPi */ + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h new file mode 100644 index 00000000..8fcf522b --- /dev/null +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -0,0 +1,107 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi (Trading) Ltd. + * + * rpi_stream.h - Raspberry Pi device stream abstraction class. + */ +#ifndef __LIBCAMERA_PIPELINE_RPI_STREAM_H__ +#define __LIBCAMERA_PIPELINE_RPI_STREAM_H__ + +#include +#include +#include + +#include + +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +namespace RPi { + +/* + * Device stream abstraction for either an internal or external stream. + * Used for both Unicam and the ISP. + */ +class RPiStream : public Stream +{ +public: + RPiStream() + { + } + + RPiStream(const char *name, MediaEntity *dev, bool importOnly = false) + : external_(false), importOnly_(importOnly), name_(name), + dev_(std::make_unique(dev)) + { + } + + V4L2VideoDevice *dev() const; + std::string name() const; + bool isImporter() const; + void reset(); + + void setExternal(bool external); + bool isExternal() const; + + void setExternalBuffers(std::vector> *buffers); + const std::vector> *getBuffers() const; + bool findFrameBuffer(FrameBuffer *buffer) const; + + int importBuffers(unsigned int count); + int allocateBuffers(unsigned int count); + + int queueBuffers(); + void releaseBuffers(); + +private: + /* + * Indicates that this stream is active externally, i.e. the buffers + * are provided by the application. + */ + bool external_; + + /* Indicates that this stream only imports buffers, e.g. ISP input. */ + bool importOnly_; + + /* Stream name identifier. */ + std::string name_; + + /* The actual device stream. */ + std::unique_ptr dev_; + + /* Internally allocated framebuffers associated with this device stream. */ + std::vector> internalBuffers_; + + /* Externally allocated framebuffers associated with this device stream. */ + std::vector> *externalBuffers_; +}; + +/* + * The following class is just a convenient (and typesafe) array of device + * streams indexed with an enum class. + */ +template +class RPiDevice : public std::array +{ +private: + constexpr auto index(E e) const noexcept + { + return static_cast>(e); + } +public: + RPiStream &operator[](E e) + { + return std::array::operator[](index(e)); + } + const RPiStream &operator[](E e) const + { + return std::array::operator[](index(e)); + } +}; + +} /* namespace RPi */ + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_PIPELINE_RPI_STREAM_H__ */ From patchwork Tue Aug 4 09:58:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9167 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 64BA3BD86F for ; Tue, 4 Aug 2020 09:59:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AEC3260543; Tue, 4 Aug 2020 11:58:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="GGScWJet"; dkim-atps=neutral Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5A00B6042B for ; Tue, 4 Aug 2020 11:58:58 +0200 (CEST) Received: by mail-wm1-x32f.google.com with SMTP id 184so2300472wmb.0 for ; Tue, 04 Aug 2020 02:58:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=T6gOmPnyM8lbTRfP3cyq3DCmCB/koQWeu23FNz57wTQ=; b=GGScWJetbBN6UkxNm7X8tYm3UO5uDT8BcoZBi1nKHieP5/60tV2LGH7V+f/8fsn9Lc DC8uzumPRagfem1RpZscWqARLblRIeDULygJO6d8bv+ybGxP+Yu/FAKaz19riZV2LIb6 KrMEhFc6O4GjtzNHQgnvT6LwZCw/KLGwWUxocNcdxWjk47VMwuGhf6ympqU9GOSJN81m riiH9lFaPKqoni9WRbix1/2e+bdz29bKvgYrdflmtgk5GiNUTzoqyXEMXpjs3xLBWKeo wK+LJkhsnQ9UHgWs9zfVz8NskMTPM5w/VeZTBBZSHhYjdyfsb/8iK0qctv2D5RIi7PII rHsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=T6gOmPnyM8lbTRfP3cyq3DCmCB/koQWeu23FNz57wTQ=; b=Y4r+XzwZDjpd+VOmaE8rEhHCzDK2/7s5vmr9Jfw1w6DoF3UmEEbYb0tcLavzODEO0K xcRxGQMY0ILOZ1MeaxjulIjA9B0jaRvPLeXtuZCAxmLEQL+Y8Q4KCuLxCLPQWBcaZUVH 7+vuIDR9vdf1IgXS9a05MCrvZvj6EPqGB9AH1hbUy+3aXJL6k17jbwa5jxvkPySHf7lz zw2uQjSkT9VG7qrSXm43UejFeCUleLzs0C8sjf3BN2z/UWoy+D4UoHuvZDp5Aovra3bJ sBmjm7jdDN4fDrc74IR2WoFb8Go4YsOxBE3zHTuYry+PVSNRnGY8LCUUnSY582XYnMAr yY9A== X-Gm-Message-State: AOAM531Sap/obbNEU7t0Ci5+4t+aSfM+AjgfIYEy4hcFNlPWQ1xjWOKP tV8wbK4cPPHqdoOwhuQzXLviSNDCyI2Y7w== X-Google-Smtp-Source: ABdhPJxrvhLT+NdnQp8sJrhehysZXlX8poT18uS3lq1D2XLEK6PJZtPN53mwqkjpAn4b2hP0MTCFBA== X-Received: by 2002:a05:600c:2209:: with SMTP id z9mr3260562wml.70.1596535137630; Tue, 04 Aug 2020 02:58:57 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id 32sm34047234wrh.18.2020.08.04.02.58.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 02:58:56 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Aug 2020 10:58:40 +0100 Message-Id: <20200804095850.275499-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200804095850.275499-1-naush@raspberrypi.com> References: <20200804095850.275499-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 02/12] libcamera: pipeline: ipa: raspberrypi: Rework drop frame signalling X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The IPA now signals up front how many frames it wants the pipeline handler to drop. This makes it easier to handle up-coming changes to the buffer handling for import/export buffers. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Tested-by: David Plowman --- include/libcamera/ipa/raspberrypi.h | 2 +- src/ipa/raspberrypi/raspberrypi.cpp | 20 +++++------ .../pipeline/raspberrypi/raspberrypi.cpp | 36 +++++++++++-------- 3 files changed, 33 insertions(+), 25 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index ca62990e..262fc6f3 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -14,6 +14,7 @@ enum RPiConfigParameters { RPI_IPA_CONFIG_LS_TABLE = (1 << 0), RPI_IPA_CONFIG_STAGGERED_WRITE = (1 << 1), RPI_IPA_CONFIG_SENSOR = (1 << 2), + RPI_IPA_CONFIG_DROP_FRAMES = (1 << 3), }; enum RPiOperations { @@ -21,7 +22,6 @@ enum RPiOperations { RPI_IPA_ACTION_V4L2_SET_ISP, RPI_IPA_ACTION_STATS_METADATA_COMPLETE, RPI_IPA_ACTION_RUN_ISP, - RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME, RPI_IPA_ACTION_EMBEDDED_COMPLETE, RPI_IPA_EVENT_SIGNAL_STAT_READY, RPI_IPA_EVENT_SIGNAL_ISP_PREPARE, diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 37472087..cd49de80 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -65,8 +65,8 @@ class IPARPi : public IPAInterface public: IPARPi() : lastMode_({}), controller_(), controllerInit_(false), - frame_count_(0), check_count_(0), hide_count_(0), - mistrust_count_(0), lsTable_(nullptr) + frame_count_(0), check_count_(0), mistrust_count_(0), + lsTable_(nullptr) { } @@ -137,8 +137,6 @@ private: uint64_t frame_count_; /* For checking the sequencing of Prepare/Process calls. */ uint64_t check_count_; - /* How many frames the pipeline handler should hide, or "drop". */ - unsigned int hide_count_; /* How many frames we should avoid running control algos on. */ unsigned int mistrust_count_; /* LS table allocation passed in from the pipeline handler. */ @@ -242,14 +240,18 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, */ frame_count_ = 0; check_count_ = 0; + unsigned int drop_frame = 0; if (controllerInit_) { - hide_count_ = helper_->HideFramesModeSwitch(); + drop_frame = helper_->HideFramesModeSwitch(); mistrust_count_ = helper_->MistrustFramesModeSwitch(); } else { - hide_count_ = helper_->HideFramesStartup(); + drop_frame = helper_->HideFramesStartup(); mistrust_count_ = helper_->MistrustFramesStartup(); } + result->data.push_back(drop_frame); + result->operation |= RPI_IPA_CONFIG_DROP_FRAMES; + struct AgcStatus agcStatus; /* These zero values mean not program anything (unless overwritten). */ agcStatus.shutter_time = 0.0; @@ -366,13 +368,11 @@ void IPARPi::processEvent(const IPAOperationData &event) * they are "unreliable". */ prepareISP(embeddedbufferId); + frame_count_++; /* Ready to push the input buffer into the ISP. */ IPAOperationData op; - if (++frame_count_ > hide_count_) - op.operation = RPI_IPA_ACTION_RUN_ISP; - else - op.operation = RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME; + op.operation = RPI_IPA_ACTION_RUN_ISP; op.data = { bayerbufferId & RPiIpaMask::ID }; queueFrameAction.emit(0, op); break; diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 90331ede..8fb9df72 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -132,7 +132,7 @@ class RPiCameraData : public CameraData public: RPiCameraData(PipelineHandler *pipe) : CameraData(pipe), sensor_(nullptr), state_(State::Stopped), - dropFrame_(false), ispOutputCount_(0) + dropFrameCount_(0), ispOutputCount_(0) { } @@ -180,14 +180,15 @@ public: std::queue embeddedQueue_; std::deque requestQueue_; + unsigned int dropFrameCount_; + private: void checkRequestCompleted(); void tryRunPipeline(); void tryFlushQueues(); FrameBuffer *updateQueue(std::queue &q, uint64_t timestamp, V4L2VideoDevice *dev); - bool dropFrame_; - int ispOutputCount_; + unsigned int ispOutputCount_; }; class RPiCameraConfiguration : public CameraConfiguration @@ -998,6 +999,7 @@ int RPiCameraData::configureIPA() ipa_->configure(sensorInfo, streamConfig, entityControls, ipaConfig, &result); + unsigned int resultIdx = 0; if (result.operation & RPI_IPA_CONFIG_STAGGERED_WRITE) { /* * Setup our staggered control writer with the sensor default @@ -1005,9 +1007,9 @@ int RPiCameraData::configureIPA() */ if (!staggeredCtrl_) { staggeredCtrl_.init(unicam_[Unicam::Image].dev(), - { { V4L2_CID_ANALOGUE_GAIN, result.data[0] }, - { V4L2_CID_EXPOSURE, result.data[1] } }); - sensorMetadata_ = result.data[2]; + { { V4L2_CID_ANALOGUE_GAIN, result.data[resultIdx++] }, + { V4L2_CID_EXPOSURE, result.data[resultIdx++] } }); + sensorMetadata_ = result.data[resultIdx++]; } /* Configure the H/V flip controls based on the sensor rotation. */ @@ -1024,6 +1026,11 @@ int RPiCameraData::configureIPA() LOG(RPI, Error) << "V4L2 staggered set failed"; } + if (result.operation & RPI_IPA_CONFIG_DROP_FRAMES) { + /* Configure the number of dropped frames required on startup. */ + dropFrameCount_ = result.data[resultIdx++]; + } + return 0; } @@ -1074,7 +1081,6 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData break; } - case RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME: case RPI_IPA_ACTION_RUN_ISP: { unsigned int bufferId = action.data[0]; FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers()->at(bufferId).get(); @@ -1083,7 +1089,6 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData << ", timestamp: " << buffer->metadata().timestamp; isp_[Isp::Input].dev()->queueBuffer(buffer); - dropFrame_ = (action.operation == RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME) ? true : false; ispOutputCount_ = 0; break; } @@ -1249,7 +1254,7 @@ void RPiCameraData::clearIncompleteRequests() void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream) { if (stream->isExternal()) { - if (!dropFrame_) { + if (!dropFrameCount_) { Request *request = buffer->request(); pipe_->completeBuffer(camera_, request, buffer); } @@ -1261,7 +1266,7 @@ void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream * simply memcpy to the Request buffer and requeue back to the * device. */ - if (stream == &unicam_[Unicam::Image] && !dropFrame_) { + if (stream == &unicam_[Unicam::Image] && !dropFrameCount_) { const Stream *rawStream = static_cast(&isp_[Isp::Input]); Request *request = requestQueue_.front(); FrameBuffer *raw = request->findBuffer(const_cast(rawStream)); @@ -1306,7 +1311,7 @@ void RPiCameraData::checkRequestCompleted() * If we are dropping this frame, do not touch the request, simply * change the state to IDLE when ready. */ - if (!dropFrame_) { + if (!dropFrameCount_) { Request *request = requestQueue_.front(); if (request->hasPendingBuffers()) return; @@ -1325,10 +1330,13 @@ void RPiCameraData::checkRequestCompleted() * frame. */ if (state_ == State::IpaComplete && - ((ispOutputCount_ == 3 && dropFrame_) || requestCompleted)) { + ((ispOutputCount_ == 3 && dropFrameCount_) || requestCompleted)) { state_ = State::Idle; - if (dropFrame_) - LOG(RPI, Info) << "Dropping frame at the request of the IPA"; + if (dropFrameCount_) { + dropFrameCount_--; + LOG(RPI, Info) << "Dropping frame at the request of the IPA (" + << dropFrameCount_ << " left)"; + } } } From patchwork Tue Aug 4 09:58:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9168 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5ED7DBD86F for ; Tue, 4 Aug 2020 09:59:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2A6F16053B; Tue, 4 Aug 2020 11:59:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="KCNgohgf"; dkim-atps=neutral Received: from mail-wr1-x441.google.com (mail-wr1-x441.google.com [IPv6:2a00:1450:4864:20::441]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5BB286053B for ; Tue, 4 Aug 2020 11:58:59 +0200 (CEST) Received: by mail-wr1-x441.google.com with SMTP id f7so36827040wrw.1 for ; Tue, 04 Aug 2020 02:58:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RplsxPmRs6AHgDVous0OsVaIaGRrWTjjB/MXfVuAQ/A=; b=KCNgohgfiFXajlQatn2OnWepTG+5Ng7BRKa0fXsPi41FehzLFu7lL9La0lVYrFO8B4 PlDdaaoBfcvkpD9uvit4/EDdwCUkoq4Zo8QuDrV//pZectOus67afuZjqX+xXqWmoLaQ AvCEpFjPKu6mrgeUbAmUsQOs+/Lj2VmZC4M4QsLqgzHCZjYDr8VgvsWYM/J6cxzZSZOD hvaW260KtVgVkdQ9aEIyivajALJzwslNCkNob3euVIm5d/PEc3HfZc3biQz6drCVYwJ+ iS/bReHF6u+H0YdmKfdIXoBUP3C0n4c3Qcv1/mHRkh31WAQUu7ibMLGubosgfArUC8uc GLag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RplsxPmRs6AHgDVous0OsVaIaGRrWTjjB/MXfVuAQ/A=; b=ExOAFb0lpt9Mi6ZPEdxBoIqNsRZKIqkwcSADECm6f7eaox9VZzkjRh520+UNbXkgXU bO1d6KjN8/55OzzfxG9EOCmyokJj9uYKoNitMTwGsB2MElcRxu6oP6W09K5Qs1VdC8RX 0zVDOaW2SSYQLJEOwgia0tWp4zl7x6C3LtGcSDjrYFHnrr1iOOGiYGeHs+zT/Cag8gcY fE7vzOmdV3sksmnlGBEYTnjvKsSV2N0Css82IoOm+xSGo6aHf8KopsFwJPpDLrJBR+/G 0iFCJxKKsZUD/llS5n63XzTrOebejMfZ05xUu/bYgwiGYoGqWXm2Rvn0liVw0oRM4RsC tkUg== X-Gm-Message-State: AOAM5331ww+XLBQePa3cA+DVqZWlcHhX1wCu8wdLRhiHmMf4cLXnfDrL ALEFIL2FHnCBpIJv7FKyflpXD3mVMW4M2Q== X-Google-Smtp-Source: ABdhPJx6ezbjHnMhVOtHeGkJCws3VNp5BkrYDgabAvb1XURWc5ucXVgJxPix1tywMt5c9Bcs94O69Q== X-Received: by 2002:a5d:4ecf:: with SMTP id s15mr18883892wrv.202.1596535138773; Tue, 04 Aug 2020 02:58:58 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id 32sm34047234wrh.18.2020.08.04.02.58.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 02:58:58 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Aug 2020 10:58:41 +0100 Message-Id: <20200804095850.275499-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200804095850.275499-1-naush@raspberrypi.com> References: <20200804095850.275499-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 03/12] libcamera: pipeline: raspberrypi: Add some debug logging X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" No functional changes, only added some more trace points. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Tested-by: David Plowman --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 8fb9df72..61925033 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1159,6 +1159,11 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) if (state_ == State::Stopped) return; + LOG(RPI, Debug) << "Stream ISP Input buffer complete" + << ", buffer id " << buffer->cookie() + << ", timestamp: " << buffer->metadata().timestamp; + + /* The ISP input buffer gets re-queued into Unicam. */ handleStreamBuffer(buffer, &unicam_[Unicam::Image]); handleState(); } @@ -1461,7 +1466,7 @@ FrameBuffer *RPiCameraData::updateQueue(std::queue &q, uint64_t t if (b->metadata().timestamp < timestamp) { q.pop(); dev->queueBuffer(b); - LOG(RPI, Error) << "Dropping input frame!"; + LOG(RPI, Warning) << "Dropping input frame!"; } else if (b->metadata().timestamp == timestamp) { /* The calling function will pop the item from the queue. */ return b; From patchwork Tue Aug 4 09:58:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9169 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B6245BD86F for ; Tue, 4 Aug 2020 09:59:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7DE0360499; Tue, 4 Aug 2020 11:59:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="CfVIQt+A"; dkim-atps=neutral Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7D60760495 for ; Tue, 4 Aug 2020 11:59:00 +0200 (CEST) Received: by mail-wm1-x343.google.com with SMTP id g8so2103982wmk.3 for ; Tue, 04 Aug 2020 02:59:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3+ZXMgg8PooDj2zFEFpjzLpTNquT9wyZ/ayyxCK3lyQ=; b=CfVIQt+A3cQvlnJq1iUcCiNtLyZ8Z2jM6J7YeiV5D6j4QlQkiN6yIMi2iUcJcE/EwN 7v+cPL621l2SxJwjZuGKpUclrbPh17j+MiyZwyCR89Z1nd7tZW0yGLS8F0wj5I6Qkejr Rw5W7bWscD4BGKJy3+Dt2CDBRb7bhxkukKgJck4ddF3BDnCjaCWAG2mv1WByz3n616Yg EsFTAXLj0uMxZv33d+nh8PqhL8UqqZ6m9cIgQELQwQ4znElmgd9Dp8fuQdug5GD4BzjX 8SyiDO3PdaoJsgu26rMZ+I9UJ5T3KaQtLVLrm2F36sPkXEkScH4C6CWupkeQsRBcURBt 5Zog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3+ZXMgg8PooDj2zFEFpjzLpTNquT9wyZ/ayyxCK3lyQ=; b=R1263fSlxY6wnqpYMzJxrMsNI4+RU+jZlUx5W5zjwcy+ar/coBNtnlDsoOCVtL4jDy VD5PIfWv9mrOOQ1feyBEvztL8epTY08jv+GgAqgE+woMLT//cUJSanB+oa8IyF8Gcukm cPL6tZSuoIgkaE0TnYoNGq0Gm1x2qewPja0JmcGE+RiCw2pNhhQoG6DBYbbezE3vBqfH G8F22ka8PoqUbUKJ9nW4fSdKGrySMs9LJyhTOph/njZ/oUg0X8kMvaIGQsxYCKa9S1YE lXBgIidNjh/OuhIyLrlxDerasrDgnUtIgpVhXavJl2a2U6fDD2L/qVhPw2Bvu8dWRgdy N8qA== X-Gm-Message-State: AOAM5321693ej6MQNxpdQkUxZpGQgK7agwiWrSfBopvbvSr3dYP50pMz C5gen820oXh4KuBF/iJUmZ9bGRTiNzravg== X-Google-Smtp-Source: ABdhPJwZSX+zijS9rXo7dT6kQUguTM9cKeEGGubqQae9WFbew5EzDcFG+HFo66a0VdZ3SuQJI3/K0A== X-Received: by 2002:a05:600c:2302:: with SMTP id 2mr3431722wmo.151.1596535140040; Tue, 04 Aug 2020 02:59:00 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id 32sm34047234wrh.18.2020.08.04.02.58.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 02:58:59 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Aug 2020 10:58:42 +0100 Message-Id: <20200804095850.275499-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200804095850.275499-1-naush@raspberrypi.com> References: <20200804095850.275499-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 04/12] libcamera: pipeline: raspberrypi: Increase the number of RAW buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Increase the number of expected RAW buffers in the stream configuration to 2. This will avoid dropping Unicam frames when exporting RAW streams. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Tested-by: David Plowman --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 61925033..a6b87df1 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -378,7 +378,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, sensorFormat = findBestMode(fmts, size); pixelFormat = sensorFormat.fourcc.toPixelFormat(); ASSERT(pixelFormat.isValid()); - bufferCount = 1; + bufferCount = 2; rawCount++; break; From patchwork Tue Aug 4 09:58:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9170 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 23CC0BD87B for ; Tue, 4 Aug 2020 09:59:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DDEB86054A; Tue, 4 Aug 2020 11:59:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="l+ykXuL1"; dkim-atps=neutral Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8E3B260499 for ; Tue, 4 Aug 2020 11:59:01 +0200 (CEST) Received: by mail-wr1-x42a.google.com with SMTP id f1so36250240wro.2 for ; Tue, 04 Aug 2020 02:59:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=i6xPt6jLjmnkByxhoVw+shMv1UBgVkah1UXeWaWM0rU=; b=l+ykXuL1/iyJwk3vhd0SfXyTnWr+M9t6hgA4j+/e+XRIfhpErSJMkZ4MlhXuwqWwi6 KEtZiHUKblKa0KAtXU9lSpgCihDLVv7IJjD/locvVA9wq4iV6MZgq+dU5A3rnskXrbHG f0Q7gPLr2ENRA+IqMEj4m+yW4Zql/PWlop6Jw6XWUu5I0rbtOCneVWf5eSbzylrNyugF v5qlYYVW4PyfpQApk3zVXKkyyUOFJm6keEYVSHPRN65bnCbB1CZnLoTG61mXNL8iPiBE 3jAr0Xzsy6VEzQhadvkIpYDPL/uXzvND2yi0b7h5du7C0CmeLemU8p+PXjRz+ERhAMQE hU1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=i6xPt6jLjmnkByxhoVw+shMv1UBgVkah1UXeWaWM0rU=; b=lUWxkHklHJF3frjiRrWj3QP9ExxIMUd6BL5SmIHb0WOyJ0+qY25/jlJObRDGGf9Xww ZOu837zYMvoL0mkFaGGJxoOa03UhlVrXOIuJ24+INWCrv/8nOmkoUk6hem0ppIozJvIQ Dxw/2hIWLi4RtjOfXEOvgOjaPh3O15tZL9QFq8ifsOE8hqT2F2gQHhWPTRc8FqOEu5Lg OhiD34ZyHqINl3dZCrDakz7arlgGiDZbtmV2DkcHTuuTuyz3aGsMgz42zFv0o2apZZpg NKLvT5ZLViaaV4x6qmXtaWDBrHJggOhhDk06f7Nwc2ocGdNiWM1u3co5EuVs0FNznOrH +OXQ== X-Gm-Message-State: AOAM532MpuIdMFnn17YfAFAyg+L9TvgO7F6bWQWJZCC/otoJ6O5HtmGe QbCAV5sZPLK7KxFGjUxNfN6HB6SyCO0Lnw== X-Google-Smtp-Source: ABdhPJz5+92fD7Bs6o95H446wE0i2dvJdWBonNUcHMBe35yh69oFfK+kGktNX+oGGChAJnfUfZy/EQ== X-Received: by 2002:adf:ea0c:: with SMTP id q12mr18183383wrm.382.1596535141090; Tue, 04 Aug 2020 02:59:01 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id 32sm34047234wrh.18.2020.08.04.02.59.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 02:59:00 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Aug 2020 10:58:43 +0100 Message-Id: <20200804095850.275499-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200804095850.275499-1-naush@raspberrypi.com> References: <20200804095850.275499-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 05/12] libcamera: pipeline: raspberrypi: Remove const qualifier from RPiStream X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For the zero-copy RAW capture feature, the RPiStream will have to be modified. Remove the const qualifier in anticipation of the future commits for this feature. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Tested-by: David Plowman --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index a6b87df1..53a07361 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -149,7 +149,7 @@ public: void ispOutputDequeue(FrameBuffer *buffer); void clearIncompleteRequests(); - void handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream); + void handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream); void handleState(); CameraSensor *sensor_; @@ -1104,12 +1104,12 @@ done: void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) { - const RPi::RPiStream *stream = nullptr; + RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPi::RPiStream const &s : unicam_) { + for (RPi::RPiStream &s : unicam_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1170,12 +1170,12 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) { - const RPi::RPiStream *stream = nullptr; + RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPi::RPiStream const &s : isp_) { + for (RPi::RPiStream &s : isp_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1256,7 +1256,7 @@ void RPiCameraData::clearIncompleteRequests() } } -void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream) +void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream) { if (stream->isExternal()) { if (!dropFrameCount_) { From patchwork Tue Aug 4 09:58:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9171 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 82717BD86F for ; Tue, 4 Aug 2020 09:59:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4F97B6049B; Tue, 4 Aug 2020 11:59:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Tcmg2+tI"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 892DC60495 for ; Tue, 4 Aug 2020 11:59:03 +0200 (CEST) Received: by mail-wr1-x429.google.com with SMTP id a15so36787508wrh.10 for ; Tue, 04 Aug 2020 02:59:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WyUMgdVpAn4EKQU1q0Py72yq7qPLwGImGumfb+AouDg=; b=Tcmg2+tIW7FfFbCB0OyEGJLGNUIe8oLzYZpRuecDs988ggiBHcEaHDE7N9gbBoY8F1 XbjP6WzAAgw+0J8O6dQZjZRFhGyLOwhUzcXddk4k4GkuigJ6Z6FrLSDOkYkcW1Je52Ih KKZc+6GBf4BApc5CTjG/vSW4sodd7AX7Avt1ukCAXj3oIHn/anoBjG9cFU4vTIM2klVT YHbuZbSC7OJkC7tm3vqRc9rHyEaSu0NXx3ZbT1PtB9tQJjbbsLvAMN90yDsLjvFBJfVP 5jLrEfMzCfaTDyEpXhFUXItDNt+U6N5utO7hf0AVc+uYAyAwq2QbRERISZvy1fGFy6Ze 5N8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WyUMgdVpAn4EKQU1q0Py72yq7qPLwGImGumfb+AouDg=; b=mD1iQJavLZm9H27slZXl3jMNU5Y+UHVUfowk2YHpPya8L1gHdyfxg2FWoQBhY2x2bI kXIUIiKJngmCmBI2SaeDTwgc40frIVMA6Lr9a8et1ft1dqQNFAahiIobSwEobO+TY77B x0yIKybtNjG8O8io76s+BBjEpROltPaSbWkBGNgjCG/J2Q3Zt9b67gopS2fz5nc34U01 +4rcd+QHs4utD5VxNx/uHRxH8wb/90vkch8RDfY7sJyVP6UCPqk1MkD/99344RwtrWqA WFmIuecy6C07ib9EEDJUm68RqRjGcmPejfwT1y12d83C6S/oYbVq29FLZVpn151STsvK 8AvQ== X-Gm-Message-State: AOAM532+Kc4JTynfo5nKubQY8LmHiH7jNe4U+KK8Jpl68lhZx5rn00wS C5z+a5U/IpGQyd6Hr9DIz3dUDTanXmVmEQ== X-Google-Smtp-Source: ABdhPJze8duR/zfeUbKw1Fz3GL8A4BAR9wQyyLCvreMtfvIZA+ngSb5sjc/WnhxtvOV5jWQytvHFMw== X-Received: by 2002:adf:bc13:: with SMTP id s19mr1822764wrg.412.1596535142208; Tue, 04 Aug 2020 02:59:02 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id 32sm34047234wrh.18.2020.08.04.02.59.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 02:59:01 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Aug 2020 10:58:44 +0100 Message-Id: <20200804095850.275499-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200804095850.275499-1-naush@raspberrypi.com> References: <20200804095850.275499-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 06/12] libcamera: pipeline: raspberrypi: Rework stream buffer logic for zero-copy X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Stop using v4l2_videodevice::allocateBuffer() for internal buffers and instead export/import all buffers. This allows the pipeline to return any stream buffer requested by the application as zero-copy. Advertise the Unicam Image stream as the RAW capture stream now. The RPiStream object now maintains a new list of buffers that are available to queue into a device. This is needed to distinguish between FrameBuffers allocated for internal use vs externally provided buffers. When a Request comes in, if a buffer is not provided for an exported stream, we re-use a buffer from this list. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Tested-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 231 ++++++++++-------- .../pipeline/raspberrypi/rpi_stream.cpp | 118 ++++++--- .../pipeline/raspberrypi/rpi_stream.h | 33 ++- 3 files changed, 231 insertions(+), 151 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 53a07361..d4791a16 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -186,7 +186,8 @@ private: void checkRequestCompleted(); void tryRunPipeline(); void tryFlushQueues(); - FrameBuffer *updateQueue(std::queue &q, uint64_t timestamp, V4L2VideoDevice *dev); + FrameBuffer *updateQueue(std::queue &q, uint64_t timestamp, + RPi::RPiStream *stream); unsigned int ispOutputCount_; }; @@ -507,8 +508,15 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) StreamConfiguration &cfg = config->at(i); if (isRaw(cfg.pixelFormat)) { - cfg.setStream(&data->isp_[Isp::Input]); - data->isp_[Isp::Input].setExternal(true); + cfg.setStream(&data->unicam_[Unicam::Image]); + /* + * We must set both Unicam streams as external, even + * though the application may only request RAW frames. + * This is because we match timestamps on both streams + * to synchronise buffers. + */ + data->unicam_[Unicam::Image].setExternal(true); + data->unicam_[Unicam::Embedded].setExternal(true); continue; } @@ -611,7 +619,7 @@ int PipelineHandlerRPi::exportFrameBuffers(Camera *camera, Stream *stream, unsigned int count = stream->configuration().bufferCount; int ret = s->dev()->exportBuffers(count, buffers); - s->setExternalBuffers(buffers); + s->setExportedBuffers(buffers); return ret; } @@ -625,6 +633,7 @@ int PipelineHandlerRPi::start(Camera *camera) ret = prepareBuffers(camera); if (ret) { LOG(RPI, Error) << "Failed to allocate buffers"; + stop(camera); return ret; } @@ -711,14 +720,26 @@ int PipelineHandlerRPi::queueRequestDevice(Camera *camera, Request *request) if (data->state_ == RPiCameraData::State::Stopped) return -EINVAL; - /* Ensure all external streams have associated buffers! */ - for (auto &stream : data->isp_) { - if (!stream.isExternal()) - continue; + LOG(RPI, Debug) << "queueRequestDevice: New request."; - if (!request->findBuffer(&stream)) { - LOG(RPI, Error) << "Attempt to queue request with invalid stream."; - return -ENOENT; + /* Push all buffers supplied in the Request to the respective streams. */ + for (auto stream : data->streams_) { + if (stream->isExternal()) { + FrameBuffer *buffer = request->findBuffer(stream); + /* + * If no buffer is provided by the request for this stream, we + * queue a nullptr to the stream to signify that it must use an + * internally allocated buffer for this capture request. This + * buffer will not be given back to the application, but is used + * to support the internal pipeline flow. + * + * The below queueBuffer() call will do nothing if there are not + * enough internal buffers allocated, but this will be handled by + * queuing the request for buffers in the RPiStream object. + */ + int ret = stream->queueBuffer(buffer); + if (ret) + return ret; } } @@ -808,14 +829,14 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) data->properties_ = data->sensor_->properties(); /* - * List the available output streams. - * Currently cannot do Unicam streams! + * List the available streams an application may request. At present, we + * do not advertise Unicam Embedded and ISP Statistics streams, as there + * is no mechanism for the application to request non-image buffer formats. */ std::set streams; - streams.insert(&data->isp_[Isp::Input]); + streams.insert(&data->unicam_[Unicam::Image]); streams.insert(&data->isp_[Isp::Output0]); streams.insert(&data->isp_[Isp::Output1]); - streams.insert(&data->isp_[Isp::Stats]); /* Create and register the camera. */ std::shared_ptr camera = Camera::create(this, data->sensor_->model(), streams); @@ -830,9 +851,28 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera) int ret; for (auto const stream : data->streams_) { - ret = stream->queueBuffers(); - if (ret < 0) - return ret; + if (!stream->isExternal()) { + ret = stream->queueAllBuffers(); + if (ret < 0) + return ret; + } else { + /* + * For external streams, we must queue up a set of internal + * buffers to handle the number of drop frames requested by + * the IPA. This is done by passing nullptr in queueBuffer(). + * + * The below queueBuffer() call will do nothing if there + * are not enough internal buffers allocated, but this will + * be handled by queuing the request for buffers in the + * RPiStream object. + */ + unsigned int i; + for (i = 0; i < data->dropFrameCount_; i++) { + int ret = stream->queueBuffer(nullptr); + if (ret) + return ret; + } + } } return 0; @@ -846,7 +886,8 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) /* * Decide how many internal buffers to allocate. For now, simply look * at how many external buffers will be provided. Will need to improve - * this logic. + * this logic. However, we really must have all streams allocate the same + * number of buffers to simplify error handling in queueRequestDevice(). */ unsigned int maxBuffers = 0; for (const Stream *s : camera->streams()) @@ -854,33 +895,9 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) maxBuffers = std::max(maxBuffers, s->configuration().bufferCount); for (auto const stream : data->streams_) { - if (stream->isExternal() || stream->isImporter()) { - /* - * If a stream is marked as external reserve memory to - * prepare to import as many buffers are requested in - * the stream configuration. - * - * If a stream is an internal stream with importer - * role, reserve as many buffers as possible. - */ - unsigned int count = stream->isExternal() - ? stream->configuration().bufferCount - : maxBuffers; - ret = stream->importBuffers(count); - if (ret < 0) - return ret; - } else { - /* - * If the stream is an internal exporter allocate and - * export as many buffers as possible to its internal - * pool. - */ - ret = stream->allocateBuffers(maxBuffers); - if (ret < 0) { - freeBuffers(camera); - return ret; - } - } + ret = stream->prepareBuffers(maxBuffers); + if (ret < 0) + return ret; } /* @@ -888,7 +905,7 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * the IPA and RPI_IPA_EVENT_SIGNAL_ISP_PREPARE event. */ count = 0; - for (auto const &b : *data->unicam_[Unicam::Image].getBuffers()) { + for (auto const &b : data->unicam_[Unicam::Image].getBuffers()) { b->setCookie(count++); } @@ -897,14 +914,14 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * the IPA. */ count = 0; - for (auto const &b : *data->isp_[Isp::Stats].getBuffers()) { + for (auto const &b : data->isp_[Isp::Stats].getBuffers()) { b->setCookie(count++); data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | b->cookie(), .planes = b->planes() }); } count = 0; - for (auto const &b : *data->unicam_[Unicam::Embedded].getBuffers()) { + for (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) { b->setCookie(count++); data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | b->cookie(), .planes = b->planes() }); @@ -1065,7 +1082,7 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData switch (action.operation) { case RPI_IPA_ACTION_STATS_METADATA_COMPLETE: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = isp_[Isp::Stats].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = isp_[Isp::Stats].getBuffers().at(bufferId); handleStreamBuffer(buffer, &isp_[Isp::Stats]); /* Fill the Request metadata buffer with what the IPA has provided */ @@ -1076,19 +1093,19 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData case RPI_IPA_ACTION_EMBEDDED_COMPLETE: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = unicam_[Unicam::Embedded].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = unicam_[Unicam::Embedded].getBuffers().at(bufferId); handleStreamBuffer(buffer, &unicam_[Unicam::Embedded]); break; } case RPI_IPA_ACTION_RUN_ISP: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers().at(bufferId); LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << buffer->cookie() << ", timestamp: " << buffer->metadata().timestamp; - isp_[Isp::Input].dev()->queueBuffer(buffer); + isp_[Isp::Input].queueBuffer(buffer); ispOutputCount_ = 0; break; } @@ -1189,22 +1206,26 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) << ", buffer id " << buffer->cookie() << ", timestamp: " << buffer->metadata().timestamp; - handleStreamBuffer(buffer, stream); - /* - * Increment the number of ISP outputs generated. - * This is needed to track dropped frames. + * ISP statistics buffer must not be re-queued or sent back to the + * application until after the IPA signals so. */ - ispOutputCount_++; - - /* If this is a stats output, hand it to the IPA now. */ if (stream == &isp_[Isp::Stats]) { IPAOperationData op; op.operation = RPI_IPA_EVENT_SIGNAL_STAT_READY; op.data = { RPiIpaMask::STATS | buffer->cookie() }; ipa_->processEvent(op); + } else { + /* Any other ISP output can be handed back to the application now. */ + handleStreamBuffer(buffer, stream); } + /* + * Increment the number of ISP outputs generated. + * This is needed to track dropped frames. + */ + ispOutputCount_++; + handleState(); } @@ -1217,8 +1238,12 @@ void RPiCameraData::clearIncompleteRequests() */ for (auto const request : requestQueue_) { for (auto const stream : streams_) { - if (stream->isExternal()) - stream->dev()->queueBuffer(request->findBuffer(stream)); + if (!stream->isExternal()) + continue; + + FrameBuffer *buffer = request->findBuffer(stream); + if (buffer) + stream->queueBuffer(buffer); } } @@ -1247,7 +1272,7 @@ void RPiCameraData::clearIncompleteRequests() * Has the buffer already been handed back to the * request? If not, do so now. */ - if (buffer->request()) + if (buffer && buffer->request()) pipe_->completeBuffer(camera_, request, buffer); } @@ -1259,30 +1284,24 @@ void RPiCameraData::clearIncompleteRequests() void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream) { if (stream->isExternal()) { - if (!dropFrameCount_) { - Request *request = buffer->request(); + Request *request = requestQueue_.front(); + + if (!dropFrameCount_ && request->findBuffer(stream) == buffer) { + /* + * Tag the buffer as completed, returning it to the + * application. + */ pipe_->completeBuffer(camera_, request, buffer); + } else { + /* + * This buffer was not part of the Request, so we can + * recycle it. + */ + stream->returnBuffer(buffer); } } else { - /* Special handling for RAW buffer Requests. - * - * The ISP input stream is alway an import stream, but if the - * current Request has been made for a buffer on the stream, - * simply memcpy to the Request buffer and requeue back to the - * device. - */ - if (stream == &unicam_[Unicam::Image] && !dropFrameCount_) { - const Stream *rawStream = static_cast(&isp_[Isp::Input]); - Request *request = requestQueue_.front(); - FrameBuffer *raw = request->findBuffer(const_cast(rawStream)); - if (raw) { - raw->copyFrom(buffer); - pipe_->completeBuffer(camera_, request, raw); - } - } - - /* Simply requeue the buffer. */ - stream->dev()->queueBuffer(buffer); + /* Simply re-queue the buffer to the requested stream. */ + stream->queueBuffer(buffer); } } @@ -1364,7 +1383,7 @@ void RPiCameraData::tryRunPipeline() * current bayer buffer will be removed and re-queued to the driver. */ embeddedBuffer = updateQueue(embeddedQueue_, bayerBuffer->metadata().timestamp, - unicam_[Unicam::Embedded].dev()); + &unicam_[Unicam::Embedded]); if (!embeddedBuffer) { LOG(RPI, Debug) << "Could not find matching embedded buffer"; @@ -1383,7 +1402,7 @@ void RPiCameraData::tryRunPipeline() embeddedBuffer = embeddedQueue_.front(); bayerBuffer = updateQueue(bayerQueue_, embeddedBuffer->metadata().timestamp, - unicam_[Unicam::Image].dev()); + &unicam_[Unicam::Image]); if (!bayerBuffer) { LOG(RPI, Debug) << "Could not find matching bayer buffer - ending."; @@ -1391,11 +1410,7 @@ void RPiCameraData::tryRunPipeline() } } - /* - * Take the first request from the queue and action the IPA. - * Unicam buffers for the request have already been queued as they come - * in. - */ + /* Take the first request from the queue and action the IPA. */ Request *request = requestQueue_.front(); /* @@ -1407,12 +1422,6 @@ void RPiCameraData::tryRunPipeline() op.controls = { request->controls() }; ipa_->processEvent(op); - /* Queue up any ISP buffers passed into the request. */ - for (auto &stream : isp_) { - if (stream.isExternal()) - stream.dev()->queueBuffer(request->findBuffer(&stream)); - } - /* Ready to use the buffers, pop them off the queue. */ bayerQueue_.pop(); embeddedQueue_.pop(); @@ -1442,32 +1451,42 @@ void RPiCameraData::tryFlushQueues() * and give a chance for the hardware to return to lock-step. We do have * to drop all interim frames. */ - if (unicam_[Unicam::Image].getBuffers()->size() == bayerQueue_.size() && - unicam_[Unicam::Embedded].getBuffers()->size() == embeddedQueue_.size()) { + if (unicam_[Unicam::Image].getBuffers().size() == bayerQueue_.size() && + unicam_[Unicam::Embedded].getBuffers().size() == embeddedQueue_.size()) { + /* This cannot happen when Unicam streams are external. */ + assert(!unicam_[Unicam::Image].isExternal()); + LOG(RPI, Warning) << "Flushing all buffer queues!"; while (!bayerQueue_.empty()) { - unicam_[Unicam::Image].dev()->queueBuffer(bayerQueue_.front()); + unicam_[Unicam::Image].queueBuffer(bayerQueue_.front()); bayerQueue_.pop(); } while (!embeddedQueue_.empty()) { - unicam_[Unicam::Embedded].dev()->queueBuffer(embeddedQueue_.front()); + unicam_[Unicam::Embedded].queueBuffer(embeddedQueue_.front()); embeddedQueue_.pop(); } } } FrameBuffer *RPiCameraData::updateQueue(std::queue &q, uint64_t timestamp, - V4L2VideoDevice *dev) + RPi::RPiStream *stream) { + /* + * If the unicam streams are external (both have be to the same), then we + * can only return out the top buffer in the queue, and assume they have + * been synced by queuing at the same time. We cannot drop these frames, + * as they may have been provided externally. + */ while (!q.empty()) { FrameBuffer *b = q.front(); - if (b->metadata().timestamp < timestamp) { + if (!stream->isExternal() && b->metadata().timestamp < timestamp) { q.pop(); - dev->queueBuffer(b); - LOG(RPI, Warning) << "Dropping input frame!"; - } else if (b->metadata().timestamp == timestamp) { + stream->queueBuffer(b); + LOG(RPI, Warning) << "Dropping unmatched input frame in stream " + << stream->name(); + } else if (stream->isExternal() || b->metadata().timestamp == timestamp) { /* The calling function will pop the item from the queue. */ return b; } else { diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index 2e52bd5c..b4d737db 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -27,78 +27,120 @@ std::string RPiStream::name() const void RPiStream::reset() { external_ = false; - internalBuffers_.clear(); -} - -bool RPiStream::isImporter() const -{ - return importOnly_; + clearBuffers(); } void RPiStream::setExternal(bool external) { + /* Import streams cannot be external. */ + ASSERT(!external || !importOnly_); external_ = external; } bool RPiStream::isExternal() const { - /* - * Import streams cannot be external. - * - * RAW capture is a special case where we simply copy the RAW - * buffer out of the request. All other buffer handling happens - * as if the stream is internal. - */ - return external_ && !importOnly_; + return external_; } -void RPiStream::setExternalBuffers(std::vector> *buffers) +void RPiStream::setExportedBuffers(std::vector> *buffers) { - externalBuffers_ = buffers; + std::transform(buffers->begin(), buffers->end(), std::back_inserter(bufferList_), + [](std::unique_ptr &b) { return b.get(); }); } -const std::vector> *RPiStream::getBuffers() const +const std::vector &RPiStream::getBuffers() const { - return external_ ? externalBuffers_ : &internalBuffers_; + return bufferList_; } bool RPiStream::findFrameBuffer(FrameBuffer *buffer) const { - auto start = external_ ? externalBuffers_->begin() : internalBuffers_.begin(); - auto end = external_ ? externalBuffers_->end() : internalBuffers_.end(); - if (importOnly_) return false; - if (std::find_if(start, end, - [buffer](std::unique_ptr const &ref) { return ref.get() == buffer; }) != end) + if (std::find(bufferList_.begin(), bufferList_.end(), buffer) != bufferList_.end()) return true; return false; } -int RPiStream::importBuffers(unsigned int count) +int RPiStream::prepareBuffers(unsigned int count) { + int ret; + + if (!importOnly_) { + if (count) { + /* Export some frame buffers for internal use. */ + ret = dev_->exportBuffers(count, &internalBuffers_); + if (ret < 0) + return ret; + + /* Add these exported buffers to the internal/external buffer list. */ + setExportedBuffers(&internalBuffers_); + + /* Add these buffers to the queue of internal usable buffers. */ + for (auto const &buffer : internalBuffers_) + availableBuffers_.push(buffer.get()); + } + + /* We must import all internal/external exported buffers. */ + count = bufferList_.size(); + } + return dev_->importBuffers(count); } -int RPiStream::allocateBuffers(unsigned int count) +int RPiStream::queueBuffer(FrameBuffer *buffer) +{ + /* + * A nullptr buffer implies an external stream, but no external + * buffer has been supplied. So, pick one from the availableBuffers_ + * queue. + */ + if (!buffer) { + if (availableBuffers_.empty()) { + LOG(RPISTREAM, Warning) << "No buffers available for " + << name_; + return -EINVAL; + } + + buffer = availableBuffers_.front(); + availableBuffers_.pop(); + } + + LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + << " for " << name_; + + int ret = dev_->queueBuffer(buffer); + if (ret) { + LOG(RPISTREAM, Error) << "Failed to queue buffer for " + << name_; + } + + return ret; +} + +void RPiStream::returnBuffer(FrameBuffer *buffer) { - return dev_->allocateBuffers(count, &internalBuffers_); + /* This can only be called for external streams. */ + assert(external_); + + availableBuffers_.push(buffer); } -int RPiStream::queueBuffers() +int RPiStream::queueAllBuffers() { + int ret; + if (external_) return 0; - for (auto &b : internalBuffers_) { - int ret = dev_->queueBuffer(b.get()); - if (ret) { - LOG(RPISTREAM, Error) << "Failed to queue buffers for " - << name_; + while (!availableBuffers_.empty()) { + ret = queueBuffer(availableBuffers_.front()); + if (ret < 0) return ret; - } + + availableBuffers_.pop(); } return 0; @@ -107,8 +149,14 @@ int RPiStream::queueBuffers() void RPiStream::releaseBuffers() { dev_->releaseBuffers(); - if (!external_ && !importOnly_) - internalBuffers_.clear(); + clearBuffers(); +} + +void RPiStream::clearBuffers() +{ + availableBuffers_ = std::queue{}; + internalBuffers_.clear(); + bufferList_.clear(); } } /* namespace RPi */ diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index 8fcf522b..73954ec7 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -44,20 +44,23 @@ public: void setExternal(bool external); bool isExternal() const; - void setExternalBuffers(std::vector> *buffers); - const std::vector> *getBuffers() const; + void setExportedBuffers(std::vector> *buffers); + const std::vector &getBuffers() const; bool findFrameBuffer(FrameBuffer *buffer) const; - int importBuffers(unsigned int count); - int allocateBuffers(unsigned int count); + int prepareBuffers(unsigned int count); + int queueBuffer(FrameBuffer *buffer); + void returnBuffer(FrameBuffer *buffer); - int queueBuffers(); + int queueAllBuffers(); void releaseBuffers(); private: + void clearBuffers(); + /* * Indicates that this stream is active externally, i.e. the buffers - * are provided by the application. + * might be provided by (and returned to) the application. */ bool external_; @@ -70,11 +73,21 @@ private: /* The actual device stream. */ std::unique_ptr dev_; - /* Internally allocated framebuffers associated with this device stream. */ - std::vector> internalBuffers_; + /* All framebuffers associated with this device stream. */ + std::vector bufferList_; + + /* + * List of frame buffers that we can use if none have been provided by + * the application for external streams. This is populated by the + * buffers exported internally. + */ + std::queue availableBuffers_; - /* Externally allocated framebuffers associated with this device stream. */ - std::vector> *externalBuffers_; + /* + * This is a list of buffers exported internally. Need to keep this around + * as the stream needs to maintain ownership of these buffers. + */ + std::vector> internalBuffers_; }; /* From patchwork Tue Aug 4 09:58:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9172 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E5DB4BD87B for ; Tue, 4 Aug 2020 09:59:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B09A260547; Tue, 4 Aug 2020 11:59:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="TNAIreQY"; dkim-atps=neutral Received: from mail-wm1-x344.google.com (mail-wm1-x344.google.com [IPv6:2a00:1450:4864:20::344]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9D96C6053B for ; Tue, 4 Aug 2020 11:59:03 +0200 (CEST) Received: by mail-wm1-x344.google.com with SMTP id g8so2104110wmk.3 for ; Tue, 04 Aug 2020 02:59:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4Ux2KzIjE78zos/U1JdkYUuRbOMWY4gvRVijLHAsfZQ=; b=TNAIreQY5W+iz/dHNsdlvsKxaiHGEqQz7PdO4T8uy2QGKvYyhwT8hEvGRLdTY6K5AX XrJ8iO4E2p8q2WaRznt2+vdJFb2nI2X37jdUllSn7/qpZwkFu0r7DdBQjo2+ilmMfmpM 6JeaB2LU+hcWb5T8fb/hTP/6i8RagbDNoffa9MkJ3YC7jV2+DSt8FQ4+++h01p3pNqqp eRDBqNHa6FP/7VpFXQz/CXoH686DfzrB0OpYEoQqU2vs4gNHx7v+USrXKjD5TbtxQQXc KpMr5Mj7RmrwAhOQGMV8aF0XtoiADQwWGNvWzkA1lM9idYO3ekPteHUh5SwsTt2HTFEK ln0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4Ux2KzIjE78zos/U1JdkYUuRbOMWY4gvRVijLHAsfZQ=; b=SYnFAw85zuAUpGY5lAQF2WlPojKRfDYFssQS05P2CRs+iYTyOR4iXLp5HnwVAqk/Q+ qVgsSGKF5eh0wow3jNPY7lsNd5HmRC9Kp5qPnIP9+EYpZclucqmgdRLSHmByVruPl3ol HwRhAY2/maC6SIp+BGMu73fkxt3ypKB6lJ77QaQ2ouKgC1MSkEKtCfQF2gpG2YlwY+DZ RI2Bpu0rqdsWaMTmncpbD+uZKVrLMSl3Gy2aj13pLOXHeFibmWNCt/d6FTxpHyx800ZI Tx/hxy+CPoTpTpkGvHDOGDRxURMCTlLzkfJr21BEzjCV0zdR12amf16kY8mgkEU11WmA LLvg== X-Gm-Message-State: AOAM531VObKLBMk4PYreigz8gEepeZ2sF1mPWNG2n2EBP2CvwLpAwphU En0Xm8MG7j+Ga6sI7BWOKMmYaN0smvqUSQ== X-Google-Smtp-Source: ABdhPJzlXQTPU/GjtpJWK/HzRPt/zZNOAApAI9yFAbsYFssbF0nFC4Z6JFZxLTkcyTcmhYeIPkNR2Q== X-Received: by 2002:a1c:9952:: with SMTP id b79mr3360022wme.68.1596535143102; Tue, 04 Aug 2020 02:59:03 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id 32sm34047234wrh.18.2020.08.04.02.59.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 02:59:02 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Aug 2020 10:58:45 +0100 Message-Id: <20200804095850.275499-8-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200804095850.275499-1-naush@raspberrypi.com> References: <20200804095850.275499-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 07/12] libcamera: pipeline: raspberrypi: Fix bug in passing configuration to IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The counter was not incremented, so multiple streams would only pass the last stream config to the IPA. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Tested-by: David Plowman --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index d4791a16..0c8e352d 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -983,12 +983,13 @@ int RPiCameraData::configureIPA() unsigned int i = 0; for (auto const &stream : isp_) { if (stream.isExternal()) { - streamConfig[i] = { + streamConfig[i++] = { .pixelFormat = stream.configuration().pixelFormat, .size = stream.configuration().size }; } } + entityControls.emplace(0, unicam_[Unicam::Image].dev()->controls()); entityControls.emplace(1, isp_[Isp::Input].dev()->controls()); From patchwork Tue Aug 4 09:58:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9173 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 97931BD86F for ; Tue, 4 Aug 2020 09:59:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4EE2060550; Tue, 4 Aug 2020 11:59:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Wn43jHT2"; dkim-atps=neutral Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C869760541 for ; Tue, 4 Aug 2020 11:59:04 +0200 (CEST) Received: by mail-wm1-x32f.google.com with SMTP id c19so1828295wmd.1 for ; Tue, 04 Aug 2020 02:59:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pO9m/3WEaO+YsGGYT5406UzcJktNRpMQ9mP+ggh1N14=; b=Wn43jHT2lDmhm1ixtcsr8nut+yjAfRXFZTzflnCXYl0m1gwaaKhD08jnJUyYhYueoa 1/KLXGyye+1Byw+UcAfaF8kKb3biXPgGrH4zsIu/uS8+eYZbH2lkutr1DH5Muf7PrQbY zxSXUAdO7IUfwkpSgX+H3PmRBtn5JeCLF49zyqTfx92N/ES4OF3n6DgmddALkAKK+BXU EOoIlEn0Wg+AaTjxdXSEbmAuVuZwUmzhoPDgtEPVTTazBfF7zJvuKi52jS0WUOd4NPmE Uehk6mYqG8b+FjGCoeGXP7z8d9KRjzEQI1VTAfJ5QiiXzsbLan60Z/hpdNMreE1MIdql QymQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pO9m/3WEaO+YsGGYT5406UzcJktNRpMQ9mP+ggh1N14=; b=rL2yXy6QhNc5xl0t51Ymy/5bUWIeWf+cNJaMiTNfv1RCWoaBts8GKz53go17sV7e+P 3kbL3A11usOTFw5fSTE2kLFXxADhtIqX50Td6cWLhxKY05sK8emRZHI4MaPIEe70zegS OVvgdD8fJXqRvdpBgPIMtlLOOSS2cviwLXBN2twl/iGSdiYxVDWXtkf2FyIUsrcWYBEt dRgUxRLCs/U407553qwWQwVYvkuHGvX2Dpz9sNsr6vA28SDR061EqpeOwsBndHRvkXGj 7UPmtA/wQZhsyocbkrNPB75kNg2qQL/i2PMMII+C/4ie7MsZfrxMs4Dm3E0wwA37G0UA fIyA== X-Gm-Message-State: AOAM532RdbLzgFEDCJvR2DnRc3TeBszig4xRX/UCMCfSmuLZZKNGwtqa 3F59immNVM5Emk4haMEoYERSQt/59ebQog== X-Google-Smtp-Source: ABdhPJwia8NG68gl32xX8spUlUThPqC3xgv0TaZqQmpblEDULnD5asJ//s0wzrl4kdTNiteT3L2WFw== X-Received: by 2002:a1c:6a05:: with SMTP id f5mr3238232wmc.162.1596535144204; Tue, 04 Aug 2020 02:59:04 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id 32sm34047234wrh.18.2020.08.04.02.59.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 02:59:03 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Aug 2020 10:58:46 +0100 Message-Id: <20200804095850.275499-9-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200804095850.275499-1-naush@raspberrypi.com> References: <20200804095850.275499-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 08/12] libcamera: pipeline: raspberrypi: Add more robust stream buffer logic X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add further queueing into the RPiStream object to ensure that we always follow the buffer ordering (be it internal or external) given by incoming Requests. This is essential, otherwise we risk dropping frames that are meant to be part of a Request, and can cause the pipeline to stall indefinitely. This also prevents any possibility of mismatched frame buffers going through the pipeline and out to the application. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Tested-by: David Plowman --- .../pipeline/raspberrypi/rpi_stream.cpp | 76 +++++++++++++++---- .../pipeline/raspberrypi/rpi_stream.h | 13 +++- 2 files changed, 75 insertions(+), 14 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index b4d737db..b2a5dfa7 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -94,38 +94,75 @@ int RPiStream::queueBuffer(FrameBuffer *buffer) { /* * A nullptr buffer implies an external stream, but no external - * buffer has been supplied. So, pick one from the availableBuffers_ - * queue. + * buffer has been supplied in the Request. So, pick one from the + * availableBuffers_ queue. */ if (!buffer) { if (availableBuffers_.empty()) { - LOG(RPISTREAM, Warning) << "No buffers available for " + LOG(RPISTREAM, Info) << "No buffers available for " << name_; - return -EINVAL; + /* + * Note that we need to queue an internal buffer as soon + * as one becomes available. + */ + requestBuffers_.push(nullptr); + return 0; } buffer = availableBuffers_.front(); availableBuffers_.pop(); } - LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() - << " for " << name_; + /* + * If no earlier requests are pending to be queued we can go ahead and + * queue this buffer into the device. + */ + if (requestBuffers_.empty()) + return queueToDevice(buffer); - int ret = dev_->queueBuffer(buffer); - if (ret) { - LOG(RPISTREAM, Error) << "Failed to queue buffer for " - << name_; - } + /* + * There are earlier Request buffers to be queued, so this buffer must go + * on the waiting list. + */ + requestBuffers_.push(buffer); - return ret; + return 0; } void RPiStream::returnBuffer(FrameBuffer *buffer) { /* This can only be called for external streams. */ - assert(external_); + ASSERT(external_); + /* Push this buffer back into the queue to be used again. */ availableBuffers_.push(buffer); + + /* + * Do we have any Request buffers that are waiting to be queued? + * If so, do it now as availableBuffers_ will not be empty. + */ + while (!requestBuffers_.empty()) { + FrameBuffer *buffer = requestBuffers_.front(); + + if (!buffer) { + /* + * We want to queue an internal buffer, but none + * are available. Can't do anything, quit the loop. + */ + if (availableBuffers_.empty()) + break; + + /* + * We want to queue an internal buffer, and at least one + * is available. + */ + buffer = availableBuffers_.front(); + availableBuffers_.pop(); + } + + requestBuffers_.pop(); + queueToDevice(buffer); + } } int RPiStream::queueAllBuffers() @@ -155,10 +192,23 @@ void RPiStream::releaseBuffers() void RPiStream::clearBuffers() { availableBuffers_ = std::queue{}; + requestBuffers_ = std::queue{}; internalBuffers_.clear(); bufferList_.clear(); } +int RPiStream::queueToDevice(FrameBuffer *buffer) +{ + LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + << " for " << name_; + + int ret = dev_->queueBuffer(buffer); + if (ret) + LOG(RPISTREAM, Error) << "Failed to queue buffer for " + << name_; + return ret; +} + } /* namespace RPi */ } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index 73954ec7..f42e25b9 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -57,6 +57,7 @@ public: private: void clearBuffers(); + int queueToDevice(FrameBuffer *buffer); /* * Indicates that this stream is active externally, i.e. the buffers @@ -73,7 +74,7 @@ private: /* The actual device stream. */ std::unique_ptr dev_; - /* All framebuffers associated with this device stream. */ + /* All frame buffers associated with this device stream. */ std::vector bufferList_; /* @@ -83,6 +84,16 @@ private: */ std::queue availableBuffers_; + /* + * List of frame buffers that are to be queued into the device from a Request. + * A nullptr indicates any internal buffer can be used (from availableBuffers_), + * whereas a valid pointer indicates an external buffer to be queued. + * + * Ordering buffers to be queued is important here as it must match the + * requests coming from the application. + */ + std::queue requestBuffers_; + /* * This is a list of buffers exported internally. Need to keep this around * as the stream needs to maintain ownership of these buffers. From patchwork Tue Aug 4 09:58:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9174 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 10227BD86F for ; Tue, 4 Aug 2020 09:59:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A4E536053B; Tue, 4 Aug 2020 11:59:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="KJVI4bJ7"; dkim-atps=neutral Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DE9E060499 for ; Tue, 4 Aug 2020 11:59:05 +0200 (CEST) Received: by mail-wm1-x32a.google.com with SMTP id t14so2288733wmi.3 for ; Tue, 04 Aug 2020 02:59:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CHpP2lRZl26kBCIlTMJ5AwlzrNhgHly/NGIe9X8vj2Y=; b=KJVI4bJ7xXoSxV9iPtun83CO0iUlyB0Vvz029HHjHZIJ3nDDE8R0AnOwjoFR/ySHY7 8+S/tifkktjHIkF4iVLfOhOF+rhShd3vZC58k0+gzR959uUJD+Q46Sxkdcs8o0adANYT ir40mli2AqxK2P8si+I90M9OGUxvIcOyn8/9hfEe4Xn3+ZxJVacHJSIe/Wq7A9PrZEbl L/8TX9FyPF6AvPmPvOpPV0/L/fmGr8Rzrb81b8VBJtWjoSrodXLhaW+F3vnglNHaWw5F iGTboRmOPncebaI2McLtyQavXE3YVb3eBMup1YuPv+a5cPJQVyTQ1wgv2Ig2G+R97nzB 3bZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CHpP2lRZl26kBCIlTMJ5AwlzrNhgHly/NGIe9X8vj2Y=; b=Ry+ALNzMbzE6mBk71W1ekIIhif51M5SHhIq7FysY46hOcqrGFj2bZ9vkGSmGUFKHRO Oog3p6e0v51DLcTqSuRUcMDKI3xg/3y5/TtzCI/CQ3tPN5+fFAWlZ+zo3IHhTkTZWeli Z6GFumgxFpkMrX/H5U7Bb8an8677tNJEvgX4LSqFEJdfAtO0585lEQ44wTfqQTYxmyC4 NrvBJkeym9SMZ3EWAMWIRAcQaFz10TNw9hX+4DcR3jUjpZxahRQRWDJ3tomsFrJvZGpi Fwd3+m22GjdYsh4bxgrTEEU5OOQwzDU/SMvrG2YH34q97uXMj5Q+PvFeWe7IrXZXIfms Fpfw== X-Gm-Message-State: AOAM5330n9eGCLYfluagIsMoVpfudFMFesqYNERgG03x3yaRVIfTO3Pm 9df4A6vKB3wMiI0i2EtkG0J6XS4MZ3BRMw== X-Google-Smtp-Source: ABdhPJyI+1K32pk/1cOyPS8VWsyIYoc7JYDrpNDIbzemkBAn5ajIr+dFh5pA9WdxNNCTrY0Icg7YLQ== X-Received: by 2002:a7b:ce83:: with SMTP id q3mr3264603wmj.5.1596535145141; Tue, 04 Aug 2020 02:59:05 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id 32sm34047234wrh.18.2020.08.04.02.59.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 02:59:04 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Aug 2020 10:58:47 +0100 Message-Id: <20200804095850.275499-10-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200804095850.275499-1-naush@raspberrypi.com> References: <20200804095850.275499-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 09/12] libcamera: pipeline: ipa: raspberrypi: Remove use of FrameBuffer cookie X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The FrameBuffer cookie may be set by the application, so this cannot be set by the pipeline handler as well. Revert to using a simple index into the buffer list to identify buffers passing to and from the IPA. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Tested-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 60 ++++++++++--------- .../pipeline/raspberrypi/rpi_stream.cpp | 14 +++-- .../pipeline/raspberrypi/rpi_stream.h | 2 +- 3 files changed, 40 insertions(+), 36 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 0c8e352d..250bbf32 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -881,7 +881,8 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera) int PipelineHandlerRPi::prepareBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera); - int count, ret; + unsigned int index; + int ret; /* * Decide how many internal buffers to allocate. For now, simply look @@ -901,30 +902,24 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) } /* - * Add cookies to the ISP Input buffers so that we can link them with - * the IPA and RPI_IPA_EVENT_SIGNAL_ISP_PREPARE event. - */ - count = 0; - for (auto const &b : data->unicam_[Unicam::Image].getBuffers()) { - b->setCookie(count++); - } - - /* - * Add cookies to the stats and embedded data buffers and link them with - * the IPA. + * Link the FrameBuffers with the index of their position in the vector + * stored in the RPi stream object. + * + * This will allow us to identify buffers passed between the pipeline + * handler and the IPA. */ - count = 0; + index = 0; for (auto const &b : data->isp_[Isp::Stats].getBuffers()) { - b->setCookie(count++); - data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | b->cookie(), + data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | index, .planes = b->planes() }); + index++; } - count = 0; + index = 0; for (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) { - b->setCookie(count++); - data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | b->cookie(), + data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | index, .planes = b->planes() }); + index++; } data->ipa_->mapBuffers(data->ipaBuffers_); @@ -1103,7 +1098,7 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData unsigned int bufferId = action.data[0]; FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers().at(bufferId); - LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << buffer->cookie() + LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << bufferId << ", timestamp: " << buffer->metadata().timestamp; isp_[Isp::Input].queueBuffer(buffer); @@ -1123,12 +1118,14 @@ done: void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) { RPi::RPiStream *stream = nullptr; + int index; if (state_ == State::Stopped) return; for (RPi::RPiStream &s : unicam_) { - if (s.findFrameBuffer(buffer)) { + index = s.getBufferIndex(buffer); + if (index != -1) { stream = &s; break; } @@ -1138,7 +1135,7 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) ASSERT(stream); LOG(RPI, Debug) << "Stream " << stream->name() << " buffer dequeue" - << ", buffer id " << buffer->cookie() + << ", buffer id " << index << ", timestamp: " << buffer->metadata().timestamp; if (stream == &unicam_[Unicam::Image]) { @@ -1178,7 +1175,7 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) return; LOG(RPI, Debug) << "Stream ISP Input buffer complete" - << ", buffer id " << buffer->cookie() + << ", buffer id " << unicam_[Unicam::Image].getBufferIndex(buffer) << ", timestamp: " << buffer->metadata().timestamp; /* The ISP input buffer gets re-queued into Unicam. */ @@ -1189,12 +1186,14 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) { RPi::RPiStream *stream = nullptr; + int index; if (state_ == State::Stopped) return; for (RPi::RPiStream &s : isp_) { - if (s.findFrameBuffer(buffer)) { + index = s.getBufferIndex(buffer); + if (index != -1) { stream = &s; break; } @@ -1204,7 +1203,7 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) ASSERT(stream); LOG(RPI, Debug) << "Stream " << stream->name() << " buffer complete" - << ", buffer id " << buffer->cookie() + << ", buffer id " << index << ", timestamp: " << buffer->metadata().timestamp; /* @@ -1214,7 +1213,7 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) if (stream == &isp_[Isp::Stats]) { IPAOperationData op; op.operation = RPI_IPA_EVENT_SIGNAL_STAT_READY; - op.data = { RPiIpaMask::STATS | buffer->cookie() }; + op.data = { RPiIpaMask::STATS | static_cast(index) }; ipa_->processEvent(op); } else { /* Any other ISP output can be handed back to the application now. */ @@ -1430,13 +1429,16 @@ void RPiCameraData::tryRunPipeline() /* Set our state to say the pipeline is active. */ state_ = State::Busy; + unsigned int bayerIndex = unicam_[Unicam::Image].getBufferIndex(bayerBuffer); + unsigned int embeddedIndex = unicam_[Unicam::Embedded].getBufferIndex(embeddedBuffer); + LOG(RPI, Debug) << "Signalling RPI_IPA_EVENT_SIGNAL_ISP_PREPARE:" - << " Bayer buffer id: " << bayerBuffer->cookie() - << " Embedded buffer id: " << embeddedBuffer->cookie(); + << " Bayer buffer id: " << bayerIndex + << " Embedded buffer id: " << embeddedIndex; op.operation = RPI_IPA_EVENT_SIGNAL_ISP_PREPARE; - op.data = { RPiIpaMask::EMBEDDED_DATA | embeddedBuffer->cookie(), - RPiIpaMask::BAYER_DATA | bayerBuffer->cookie() }; + op.data = { RPiIpaMask::EMBEDDED_DATA | embeddedIndex, + RPiIpaMask::BAYER_DATA | bayerIndex }; ipa_->processEvent(op); } diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index b2a5dfa7..80170d64 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -53,15 +53,17 @@ const std::vector &RPiStream::getBuffers() const return bufferList_; } -bool RPiStream::findFrameBuffer(FrameBuffer *buffer) const +int RPiStream::getBufferIndex(FrameBuffer *buffer) const { if (importOnly_) - return false; + return -1; - if (std::find(bufferList_.begin(), bufferList_.end(), buffer) != bufferList_.end()) - return true; + /* Find the buffer in the list, and return the index position. */ + auto it = std::find(bufferList_.begin(), bufferList_.end(), buffer); + if (it != bufferList_.end()) + return std::distance(bufferList_.begin(), it); - return false; + return -1; } int RPiStream::prepareBuffers(unsigned int count) @@ -199,7 +201,7 @@ void RPiStream::clearBuffers() int RPiStream::queueToDevice(FrameBuffer *buffer) { - LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + LOG(RPISTREAM, Debug) << "Queuing buffer " << getBufferIndex(buffer) << " for " << name_; int ret = dev_->queueBuffer(buffer); diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index f42e25b9..959e9147 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -46,7 +46,7 @@ public: void setExportedBuffers(std::vector> *buffers); const std::vector &getBuffers() const; - bool findFrameBuffer(FrameBuffer *buffer) const; + int getBufferIndex(FrameBuffer *buffer) const; int prepareBuffers(unsigned int count); int queueBuffer(FrameBuffer *buffer); From patchwork Tue Aug 4 09:58:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9175 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 621E4BD87B for ; Tue, 4 Aug 2020 09:59:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3235E60555; Tue, 4 Aug 2020 11:59:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="GAvIDGRw"; dkim-atps=neutral Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2E9446049B for ; Tue, 4 Aug 2020 11:59:07 +0200 (CEST) Received: by mail-wr1-x431.google.com with SMTP id a5so26836760wrm.6 for ; Tue, 04 Aug 2020 02:59:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lt8mUVcYxjuDMyoiYlgrDc1BgLMPQm+IF3xYVvJO6nE=; b=GAvIDGRwH3HW+ReFTq6lpAp9r7mnOueCbMwygc+C+LAw0ojD7ss4Xjx23uOoy5vFJ9 2ryCQ/SVCFcMTwv4MN7Gs4eo0YJUXkrvyXOptXQqXYctjV3EXx6Ner96/VzQHk0ZQhqz g62l4zleBB5Ormy1NXr42sUpoQjzSFFmeYPHKDbcbpS1OQJO026GegjghOtpCtbqkcUD 8Z/ho0pFtVVudhzTES1wW7zjeJaDvjpY4RBxWeBTY9fEAHk87LcKhYfwdNhC4t0Zftwq qjyk1Yis2FnhVpJO2paxKT1PBYsHka8x25LwimoIoqXO/6R3g+x8VVTiLzkb9GMlGsNW luvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lt8mUVcYxjuDMyoiYlgrDc1BgLMPQm+IF3xYVvJO6nE=; b=SCJLM4ssQDPKwMXmpa9fNAN/3NJsHwHIqWdcjzr30EB3PIX2k/6ZD0IcYQcUhS+Ey4 fyAYI5qYuBCAzEpPQOz8bjc6f5acuI76NTO4GojzhVypwOOoZayj7ZyrVFm+FN4+cCdr mQukrmf+dquN/4O9t+gqvi4jNA3jY+rxvnhLOaR+5cbS3n+LRQmolRVBGtEuD7A2e/X5 OQvuZ00CxpRgQB87W+1pF+yrFbQ7V6B7/NbtYiQbxXqFxhpQXFXAP2+BnuaISFjfZ8LF XYin/RDtCs6kp7Q/aLt1bGpY7P6etDHUsj/dQro9dPkwWY5UdMZrp8RkHOfEpVoe6fMv E6Sg== X-Gm-Message-State: AOAM532hzkDe+7l7HCwcZ7O7yKRf49ilPwLUa8NXX91ae8wt9oH8Irdy nsiaHXGTm69rBe2Tg+ul2gMARPn2rLg0Hg== X-Google-Smtp-Source: ABdhPJx71tB9tTTJPiLXhEahmipwzYZ31Rkx8knKqf/dbHwMDmMcBkZAZY3tzHFMquhho5fRBeMZDQ== X-Received: by 2002:adf:e992:: with SMTP id h18mr17933568wrm.383.1596535146432; Tue, 04 Aug 2020 02:59:06 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id 32sm34047234wrh.18.2020.08.04.02.59.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 02:59:05 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Aug 2020 10:58:48 +0100 Message-Id: <20200804095850.275499-11-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200804095850.275499-1-naush@raspberrypi.com> References: <20200804095850.275499-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 10/12] pipeline: raspberrypi: Use an unordered_map for the stream buffer list X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" By using a map container, we can easily insert/remove buffers from the buffer list. This will be required to track buffers allocated externally and passed to the pipeline handler through a Request. Replace the buffer index tracking with an id generated internally by the stream object. Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 23 ++++++--------- .../pipeline/raspberrypi/rpi_stream.cpp | 28 ++++++++++--------- .../pipeline/raspberrypi/rpi_stream.h | 14 +++++++--- 3 files changed, 34 insertions(+), 31 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 250bbf32..8cfa87be 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -881,7 +881,6 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera) int PipelineHandlerRPi::prepareBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera); - unsigned int index; int ret; /* @@ -908,18 +907,14 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * This will allow us to identify buffers passed between the pipeline * handler and the IPA. */ - index = 0; for (auto const &b : data->isp_[Isp::Stats].getBuffers()) { - data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | index, - .planes = b->planes() }); - index++; + data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | b.first, + .planes = b.second->planes() }); } - index = 0; for (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) { - data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | index, - .planes = b->planes() }); - index++; + data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | b.first, + .planes = b.second->planes() }); } data->ipa_->mapBuffers(data->ipaBuffers_); @@ -1124,7 +1119,7 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) return; for (RPi::RPiStream &s : unicam_) { - index = s.getBufferIndex(buffer); + index = s.getBufferId(buffer); if (index != -1) { stream = &s; break; @@ -1175,7 +1170,7 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) return; LOG(RPI, Debug) << "Stream ISP Input buffer complete" - << ", buffer id " << unicam_[Unicam::Image].getBufferIndex(buffer) + << ", buffer id " << unicam_[Unicam::Image].getBufferId(buffer) << ", timestamp: " << buffer->metadata().timestamp; /* The ISP input buffer gets re-queued into Unicam. */ @@ -1192,7 +1187,7 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) return; for (RPi::RPiStream &s : isp_) { - index = s.getBufferIndex(buffer); + index = s.getBufferId(buffer); if (index != -1) { stream = &s; break; @@ -1429,8 +1424,8 @@ void RPiCameraData::tryRunPipeline() /* Set our state to say the pipeline is active. */ state_ = State::Busy; - unsigned int bayerIndex = unicam_[Unicam::Image].getBufferIndex(bayerBuffer); - unsigned int embeddedIndex = unicam_[Unicam::Embedded].getBufferIndex(embeddedBuffer); + unsigned int bayerIndex = unicam_[Unicam::Image].getBufferId(bayerBuffer); + unsigned int embeddedIndex = unicam_[Unicam::Embedded].getBufferId(embeddedBuffer); LOG(RPI, Debug) << "Signalling RPI_IPA_EVENT_SIGNAL_ISP_PREPARE:" << " Bayer buffer id: " << bayerIndex diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index 80170d64..879e25ba 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -44,26 +44,28 @@ bool RPiStream::isExternal() const void RPiStream::setExportedBuffers(std::vector> *buffers) { - std::transform(buffers->begin(), buffers->end(), std::back_inserter(bufferList_), - [](std::unique_ptr &b) { return b.get(); }); + for (auto const &buffer : *buffers) + bufferMap_.emplace(id_++, buffer.get()); } -const std::vector &RPiStream::getBuffers() const +const BufferMap &RPiStream::getBuffers() const { - return bufferList_; + return bufferMap_; } -int RPiStream::getBufferIndex(FrameBuffer *buffer) const +int RPiStream::getBufferId(FrameBuffer *buffer) const { if (importOnly_) return -1; - /* Find the buffer in the list, and return the index position. */ - auto it = std::find(bufferList_.begin(), bufferList_.end(), buffer); - if (it != bufferList_.end()) - return std::distance(bufferList_.begin(), it); + /* Find the buffer in the map, and return the buffer id. */ + auto it = std::find_if(bufferMap_.begin(), bufferMap_.end(), + [&buffer](auto const &p) { return p.second == buffer; }); - return -1; + if (it == bufferMap_.end()) + return -1; + + return it->first; } int RPiStream::prepareBuffers(unsigned int count) @@ -86,7 +88,7 @@ int RPiStream::prepareBuffers(unsigned int count) } /* We must import all internal/external exported buffers. */ - count = bufferList_.size(); + count = bufferMap_.size(); } return dev_->importBuffers(count); @@ -196,12 +198,12 @@ void RPiStream::clearBuffers() availableBuffers_ = std::queue{}; requestBuffers_ = std::queue{}; internalBuffers_.clear(); - bufferList_.clear(); + bufferMap_.clear(); } int RPiStream::queueToDevice(FrameBuffer *buffer) { - LOG(RPISTREAM, Debug) << "Queuing buffer " << getBufferIndex(buffer) + LOG(RPISTREAM, Debug) << "Queuing buffer " << getBufferId(buffer) << " for " << name_; int ret = dev_->queueBuffer(buffer); diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index 959e9147..ed517c22 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -9,6 +9,7 @@ #include #include +#include #include #include @@ -19,6 +20,8 @@ namespace libcamera { namespace RPi { +using BufferMap = std::unordered_map; + /* * Device stream abstraction for either an internal or external stream. * Used for both Unicam and the ISP. @@ -32,7 +35,7 @@ public: RPiStream(const char *name, MediaEntity *dev, bool importOnly = false) : external_(false), importOnly_(importOnly), name_(name), - dev_(std::make_unique(dev)) + dev_(std::make_unique(dev)), id_(0) { } @@ -45,8 +48,8 @@ public: bool isExternal() const; void setExportedBuffers(std::vector> *buffers); - const std::vector &getBuffers() const; - int getBufferIndex(FrameBuffer *buffer) const; + const BufferMap &getBuffers() const; + int getBufferId(FrameBuffer *buffer) const; int prepareBuffers(unsigned int count); int queueBuffer(FrameBuffer *buffer); @@ -74,8 +77,11 @@ private: /* The actual device stream. */ std::unique_ptr dev_; + /* Tracks a unique id key for the bufferMap_ */ + unsigned int id_; + /* All frame buffers associated with this device stream. */ - std::vector bufferList_; + BufferMap bufferMap_; /* * List of frame buffers that we can use if none have been provided by From patchwork Tue Aug 4 09:58:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9176 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CF995BD86F for ; Tue, 4 Aug 2020 09:59:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7E5FB60550; Tue, 4 Aug 2020 11:59:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="hpy6X1SM"; dkim-atps=neutral Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E9D2A60553 for ; Tue, 4 Aug 2020 11:59:07 +0200 (CEST) Received: by mail-wm1-x343.google.com with SMTP id k8so2295194wma.2 for ; Tue, 04 Aug 2020 02:59:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ryz+jGxeBqsKM7XD/dNKRoalEXGdhWc/8wLotbzodnc=; b=hpy6X1SMG640fhafI/7ZOndJb7T698yNo1CAqXPeBy/V5YYS63yjFVKqgVszx334Bi bEK14MRKXjmQ83YKG4TxWyxG81aBds9jUk/F0LF8yH2w/29vwOlLK72JsvadKZbQWs9U uZk4WYMKrrupnOaZYGheoWFQb8IXf3TmNeQfSwQFd3AgN0nWbPp4SHsrZwI1VvwpyqJj fWC4guxQZF0f/yCoSUQnbDKF1izUS0+3gr7RQzKq7QiBK/7lPK6/mrVUwSqK1uud5AyX Sk7XxTTEwZ/EFnw83vhr+FgQ4WVjTep1kgIXeMz6zACzm315UEC06Fa1T7Pu31778tI3 k8bA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ryz+jGxeBqsKM7XD/dNKRoalEXGdhWc/8wLotbzodnc=; b=pePGEjCvou32IacNKjibglElOWR9sb2Gb+iNkCf/9V4O48lz1R60dKTHB7XWS2ZA2+ rA0K5mnFJhCC0aFpBqWCHoYKe49XcxqTUTT8vCh7uYRqXj1876y3pDlzcGZyGFPpbTgQ /TMm+jOPHuzGW9ydilIEVtHqQFueZaA9aSmoTK/xWjjCGcRWeFHRRVYmXPoCryO5ix3i w7p9ztxnrgx1rYLXkVe08XNf5BwQEfthKd1XXpi6cIHtxrYezXBhxD3ueA3LYPTkt1D+ yL+w7vOnrKnMwKnnjf20F37fslN+bRaov+U8q/XL//pqJDrXSGbFTcLhbCffdFwCsOrp O6xg== X-Gm-Message-State: AOAM533kyY7St53JZprdbbISg9EiW/AlVMKZLucbA2PpG0x9HQe/42Kx jbl4XboefZl93W36s31Yzz40MnO7vv0wWQ== X-Google-Smtp-Source: ABdhPJx2rLcGRa2zpPy50K6RliXtgW69q7a2kbinFvuEoWDWQ7NzNYUb31E8YJoGZl4JzAiyzDAxBQ== X-Received: by 2002:a1c:1f12:: with SMTP id f18mr3513251wmf.66.1596535147261; Tue, 04 Aug 2020 02:59:07 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id 32sm34047234wrh.18.2020.08.04.02.59.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 02:59:06 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Aug 2020 10:58:49 +0100 Message-Id: <20200804095850.275499-12-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200804095850.275499-1-naush@raspberrypi.com> References: <20200804095850.275499-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 11/12] pipeline: raspberrypi: Use an unordered_set to store IPA buffer ids X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" By using a set container, we can easily insert/remove buffer ids that have been mmaped by the IPA. This will be required to track buffers allocated externally and passed to the pipeline handler through a Request. Move the IPA buffer mapping code into a function to remove duplicated code. Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 48 +++++++++++-------- 1 file changed, 28 insertions(+), 20 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 8cfa87be..b059c582 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -158,8 +159,8 @@ public: RPi::RPiDevice isp_; /* The vector below is just for convenience when iterating over all streams. */ std::vector streams_; - /* Buffers passed to the IPA. */ - std::vector ipaBuffers_; + /* Stores the ids of the buffers mapped in the IPA. */ + std::unordered_set ipaBuffers_; /* DMAHEAP allocation helper. */ RPi::DmaHeap dmaHeap_; @@ -230,6 +231,7 @@ private: int queueAllBuffers(Camera *camera); int prepareBuffers(Camera *camera); void freeBuffers(Camera *camera); + void mapBuffers(Camera *camera, const RPi::BufferMap &buffers, unsigned int mask); MediaDevice *unicam_; MediaDevice *isp_; @@ -901,36 +903,42 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) } /* - * Link the FrameBuffers with the index of their position in the vector - * stored in the RPi stream object. + * Pass the stats and embedded data buffers to the IPA. No other + * buffers need to be passed. + */ + mapBuffers(camera, data->isp_[Isp::Stats].getBuffers(), RPiIpaMask::STATS); + mapBuffers(camera, data->unicam_[Unicam::Embedded].getBuffers(), RPiIpaMask::EMBEDDED_DATA); + + return 0; +} + +void PipelineHandlerRPi::mapBuffers(Camera *camera, const RPi::BufferMap &buffers, unsigned int mask) +{ + RPiCameraData *data = cameraData(camera); + std::vector ipaBuffers; + /* + * Link the FrameBuffers with the id (key value) in the map stored in + * the RPi stream object - along with an identifer mask. * * This will allow us to identify buffers passed between the pipeline * handler and the IPA. */ - for (auto const &b : data->isp_[Isp::Stats].getBuffers()) { - data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | b.first, - .planes = b.second->planes() }); - } - - for (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) { - data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | b.first, - .planes = b.second->planes() }); + for (auto const &it : buffers) { + ipaBuffers.push_back({ .id = mask | it.first, + .planes = it.second->planes() }); + data->ipaBuffers_.insert(mask | it.first); } - data->ipa_->mapBuffers(data->ipaBuffers_); - - return 0; + data->ipa_->mapBuffers(ipaBuffers); } void PipelineHandlerRPi::freeBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera); - std::vector ids; - for (IPABuffer &ipabuf : data->ipaBuffers_) - ids.push_back(ipabuf.id); - - data->ipa_->unmapBuffers(ids); + /* Copy the buffer ids from the unordered_set to a vector to pass to the IPA. */ + std::vector ipaBuffers(data->ipaBuffers_.begin(), data->ipaBuffers_.end()); + data->ipa_->unmapBuffers(ipaBuffers); data->ipaBuffers_.clear(); for (auto const stream : data->streams_) From patchwork Tue Aug 4 09:58:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9177 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7956DBD86F for ; Tue, 4 Aug 2020 09:59:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 430EB6049B; Tue, 4 Aug 2020 11:59:12 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="aS/ATfxi"; dkim-atps=neutral Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1005B60550 for ; Tue, 4 Aug 2020 11:59:09 +0200 (CEST) Received: by mail-wr1-x42c.google.com with SMTP id f7so36827481wrw.1 for ; Tue, 04 Aug 2020 02:59:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VrnZVRzAFcTlfpKwg6N0sIpXHNob2AVqtYJvsi6G/f4=; b=aS/ATfxiQDBlhTPdF9BqRoymrGLfHs3LwduRyuAy2pzxZQ29PPd7q9ReZwdMwncv1e Yr+5tH/B+dH3Uxau/+jvQTTtmW2vdk1oa7sdTG17eChzlHzVLItp0h48bV/0zNyS2Ae6 mWRyNZ77TWZlgAOSOTJeedSZD/EedBL2cgRt0WU+/vYpn7VRt37sVJSJfZm4F246zg2S wfuNs0xUVCKAUsGr4IQENJ3i7LjU93xlTsJiDljGOB2EC8z93g3D1nQMxlP0YVeN4/4j nrGTLDXzDvJGl8P+L2gaCYw8CIwplenJLLWLzRMyOUqHGVd4batplq4NafEsxrPl9qfZ KA9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VrnZVRzAFcTlfpKwg6N0sIpXHNob2AVqtYJvsi6G/f4=; b=TL1nqVZKiR80uAGzWq/EEH+OAw6Pl2rb8GZZltM1Y5s5a76jeDa0SGiMbOU31j3o7y GvBnQXB7L5T65CP4cPxQaR+6rWwWHILECtqPTmeLupPRIzrIJ1V82zyEHECT5jBWnrIW naaNC4AjHFahylFg5nllFUEFilOVV52ouzf+N4qIzNbW0cjxC86A9J7DVmpILxTMp7H9 fby9VDeCbxKxmGx5RUfF+h6eXH5b8uGnB7/7OohJ9kXvv4P8DT1VA8hKmgG/EJy+MXvt 9hzx9v19AltkF39DC2oPjQO9j5sFQxaLAaNIHwie9Osi9AoBqwN3hN74ekxOe+qHcjQ1 wUnA== X-Gm-Message-State: AOAM533S9YO49Hz+TuBQQtTsoQAwIuBSmovCXrOOIJKBRARUUp4Jiv1B okkCokFSK2lgcdf94g4urAxIwpT3QMBGBA== X-Google-Smtp-Source: ABdhPJzt9EiSGL8oe6ZjQxDaB5G4zJ7FMr4/lK6xqmEA2+Let7409S1qLH/AqHlCu2S6Ax+ew4TRAw== X-Received: by 2002:a5d:4802:: with SMTP id l2mr17877581wrq.183.1596535148230; Tue, 04 Aug 2020 02:59:08 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id 32sm34047234wrh.18.2020.08.04.02.59.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 02:59:07 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Aug 2020 10:58:50 +0100 Message-Id: <20200804095850.275499-13-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200804095850.275499-1-naush@raspberrypi.com> References: <20200804095850.275499-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 12/12] pipeline: ipa: raspberrypi: Handle any externally allocated FrameBuffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Handle the case where a FrameBuffer that has been externally allocated (i.e. not through the v4l2 video device) is passed into a Request. We must store the buffer pointer in the stream internal buffer list to identify when used, as well as mmap the buffer in the IPA if needed. Signed-off-by: Naushir Patuck --- include/libcamera/ipa/raspberrypi.h | 11 +-- src/ipa/raspberrypi/raspberrypi.cpp | 6 +- .../pipeline/raspberrypi/raspberrypi.cpp | 82 ++++++++++++++----- .../pipeline/raspberrypi/rpi_stream.cpp | 30 ++++++- .../pipeline/raspberrypi/rpi_stream.h | 3 + 5 files changed, 103 insertions(+), 29 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index 262fc6f3..dd6ebeac 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -28,11 +28,12 @@ enum RPiOperations { RPI_IPA_EVENT_QUEUE_REQUEST, }; -enum RPiIpaMask { - ID = 0x0ffff, - STATS = 0x10000, - EMBEDDED_DATA = 0x20000, - BAYER_DATA = 0x40000 +enum RPiBufferMask { + ID = 0x00ffff, + STATS = 0x010000, + EMBEDDED_DATA = 0x020000, + BAYER_DATA = 0x040000, + EXTERNAL_BUFFER = 0x100000, }; /* Size of the LS grid allocation. */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index cd49de80..93dfef27 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -352,7 +352,7 @@ void IPARPi::processEvent(const IPAOperationData &event) IPAOperationData op; op.operation = RPI_IPA_ACTION_STATS_METADATA_COMPLETE; - op.data = { bufferId & RPiIpaMask::ID }; + op.data = { bufferId & RPiBufferMask::ID }; op.controls = { libcameraMetadata_ }; queueFrameAction.emit(0, op); break; @@ -373,7 +373,7 @@ void IPARPi::processEvent(const IPAOperationData &event) /* Ready to push the input buffer into the ISP. */ IPAOperationData op; op.operation = RPI_IPA_ACTION_RUN_ISP; - op.data = { bayerbufferId & RPiIpaMask::ID }; + op.data = { bayerbufferId & RPiBufferMask::ID }; queueFrameAction.emit(0, op); break; } @@ -700,7 +700,7 @@ void IPARPi::returnEmbeddedBuffer(unsigned int bufferId) { IPAOperationData op; op.operation = RPI_IPA_ACTION_EMBEDDED_COMPLETE; - op.data = { bufferId & RPiIpaMask::ID }; + op.data = { bufferId & RPiBufferMask::ID }; queueFrameAction.emit(0, op); } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index b059c582..e3240914 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -151,6 +151,7 @@ public: void clearIncompleteRequests(); void handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream); + void handleExternalBuffer(FrameBuffer *buffer, RPi::RPiStream *stream); void handleState(); CameraSensor *sensor_; @@ -726,23 +727,41 @@ int PipelineHandlerRPi::queueRequestDevice(Camera *camera, Request *request) /* Push all buffers supplied in the Request to the respective streams. */ for (auto stream : data->streams_) { - if (stream->isExternal()) { - FrameBuffer *buffer = request->findBuffer(stream); + if (!stream->isExternal()) + continue; + + FrameBuffer *buffer = request->findBuffer(stream); + if (buffer && stream->getBufferId(buffer) == -1) { /* - * If no buffer is provided by the request for this stream, we - * queue a nullptr to the stream to signify that it must use an - * internally allocated buffer for this capture request. This - * buffer will not be given back to the application, but is used - * to support the internal pipeline flow. - * - * The below queueBuffer() call will do nothing if there are not - * enough internal buffers allocated, but this will be handled by - * queuing the request for buffers in the RPiStream object. + * This buffer is not recognised, so it must have been allocated + * outside the v4l2 device. Store it in the stream buffer list + * so we can track it. */ - int ret = stream->queueBuffer(buffer); - if (ret) - return ret; + stream->setExternalBuffer(buffer); + + /* Also get the IPA to mmap it if needed. */ + if (stream == &data->unicam_[Unicam::Embedded] || + stream == &data->isp_[Isp::Stats]) { + unsigned int id = stream->getBufferId(buffer); + unsigned int mask = stream == &data->unicam_[Unicam::Embedded] ? + RPiBufferMask::EMBEDDED_DATA : RPiBufferMask::STATS; + mapBuffers(camera, {{ id, buffer }}, mask); + } } + /* + * If no buffer is provided by the request for this stream, we + * queue a nullptr to the stream to signify that it must use an + * internally allocated buffer for this capture request. This + * buffer will not be given back to the application, but is used + * to support the internal pipeline flow. + * + * The below queueBuffer() call will do nothing if there are not + * enough internal buffers allocated, but this will be handled by + * queuing the request for buffers in the RPiStream object. + */ + int ret = stream->queueBuffer(buffer); + if (ret) + return ret; } /* Push the request to the back of the queue. */ @@ -906,8 +925,8 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * Pass the stats and embedded data buffers to the IPA. No other * buffers need to be passed. */ - mapBuffers(camera, data->isp_[Isp::Stats].getBuffers(), RPiIpaMask::STATS); - mapBuffers(camera, data->unicam_[Unicam::Embedded].getBuffers(), RPiIpaMask::EMBEDDED_DATA); + mapBuffers(camera, data->isp_[Isp::Stats].getBuffers(), RPiBufferMask::STATS); + mapBuffers(camera, data->unicam_[Unicam::Embedded].getBuffers(), RPiBufferMask::EMBEDDED_DATA); return 0; } @@ -1216,7 +1235,7 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) if (stream == &isp_[Isp::Stats]) { IPAOperationData op; op.operation = RPI_IPA_EVENT_SIGNAL_STAT_READY; - op.data = { RPiIpaMask::STATS | static_cast(index) }; + op.data = { RPiBufferMask::STATS | static_cast(index) }; ipa_->processEvent(op); } else { /* Any other ISP output can be handed back to the application now. */ @@ -1290,6 +1309,12 @@ void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stre Request *request = requestQueue_.front(); if (!dropFrameCount_ && request->findBuffer(stream) == buffer) { + /* + * Check if this is an externally provided buffer, and if + * so, we must get the IPA to unmap it and stop tracking + * it in the pipeline handler. + */ + handleExternalBuffer(buffer, stream); /* * Tag the buffer as completed, returning it to the * application. @@ -1308,6 +1333,25 @@ void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stre } } +void RPiCameraData::handleExternalBuffer(FrameBuffer *buffer, RPi::RPiStream *stream) +{ + unsigned int id = stream->getBufferId(buffer); + + if (!(id & RPiBufferMask::EXTERNAL_BUFFER)) + return; + + if (stream == &unicam_[Unicam::Embedded] || stream == &isp_[Isp::Stats]) { + unsigned int mask = stream == &unicam_[Unicam::Embedded] ? + RPiBufferMask::EMBEDDED_DATA : RPiBufferMask::STATS; + /* Ask the IPA to unmap the buffer. */ + ipa_->unmapBuffers({ mask | id }); + ipaBuffers_.erase(mask | id); + } + + /* Stop the Stream object from tracking the buffer. */ + stream->removeExternalBuffer(buffer); +} + void RPiCameraData::handleState() { switch (state_) { @@ -1440,8 +1484,8 @@ void RPiCameraData::tryRunPipeline() << " Embedded buffer id: " << embeddedIndex; op.operation = RPI_IPA_EVENT_SIGNAL_ISP_PREPARE; - op.data = { RPiIpaMask::EMBEDDED_DATA | embeddedIndex, - RPiIpaMask::BAYER_DATA | bayerIndex }; + op.data = { RPiBufferMask::EMBEDDED_DATA | embeddedIndex, + RPiBufferMask::BAYER_DATA | bayerIndex }; ipa_->processEvent(op); } diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index 879e25ba..c09f14c9 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -6,6 +6,8 @@ */ #include "rpi_stream.h" +#include + #include "libcamera/internal/log.h" namespace libcamera { @@ -44,8 +46,13 @@ bool RPiStream::isExternal() const void RPiStream::setExportedBuffers(std::vector> *buffers) { - for (auto const &buffer : *buffers) - bufferMap_.emplace(id_++, buffer.get()); + /* Ensure we are using a sensible number of buffers. */ + ASSERT(id_ < RPiBufferMask::ID); + + for (auto const &buffer : *buffers) { + bufferMap_.emplace(id_ & RPiBufferMask::ID, buffer.get()); + id_++; + } } const BufferMap &RPiStream::getBuffers() const @@ -68,6 +75,25 @@ int RPiStream::getBufferId(FrameBuffer *buffer) const return it->first; } +void RPiStream::setExternalBuffer(FrameBuffer *buffer) +{ + /* Ensure we are using a sensible number of buffers. */ + ASSERT(id_ < RPiBufferMask::ID); + + bufferMap_.emplace(RPiBufferMask::EXTERNAL_BUFFER | (id_ & RPiBufferMask::ID), + buffer); + id_++; +} + +void RPiStream::removeExternalBuffer(FrameBuffer *buffer) +{ + int id = getBufferId(buffer); + + /* Ensure we have this buffer in the stream, and it is marked external. */ + ASSERT(id != -1 && (id & RPiBufferMask::EXTERNAL_BUFFER)); + bufferMap_.erase(id); +} + int RPiStream::prepareBuffers(unsigned int count) { int ret; diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index ed517c22..8b23c4b2 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -51,6 +51,9 @@ public: const BufferMap &getBuffers() const; int getBufferId(FrameBuffer *buffer) const; + void setExternalBuffer(FrameBuffer *buffer); + void removeExternalBuffer(FrameBuffer *buffer); + int prepareBuffers(unsigned int count); int queueBuffer(FrameBuffer *buffer); void returnBuffer(FrameBuffer *buffer);