From patchwork Wed Jul 15 14:06:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8824 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 42126BD790 for ; Wed, 15 Jul 2020 14:07:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ECB2A609A9; Wed, 15 Jul 2020 16:07:12 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="I2L5i+Mt"; dkim-atps=neutral Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DADF760487 for ; Wed, 15 Jul 2020 16:07:11 +0200 (CEST) Received: by mail-wr1-x42c.google.com with SMTP id f18so2911806wrs.0 for ; Wed, 15 Jul 2020 07:07:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ruut7Hd3XX0EZMUdhakXLnqZr9tDy0Fu04H3y4CmE5g=; b=I2L5i+MtmPjY8KYhfA+LvMuGtK019VtahwAFbEJ7AITIlObohvr8KnLcXsipgsBREP TQQdATPY+QfaZ3CmFR5fks2D/rfC2uLnPH668jDsfRhwXCpP3feRb/g9YoqLcuWgqcs+ MhyEltcmXUDClrlKASSIDr4J8OTcVPFM5k19LaVAtO91lxeppZTuFkBXOQjitnVDJrmx tVU4GsdYhi63tp92Mwcyhwb4VK3IVAMLz+GuqKRP+erwZLNZKfF77gVtKDB9HuQJtO3Q ntq2BlMJhyUzpiT4Pc9E+BBEzehyThjhUmqzeodnESY2jbiup9kBFxQhMxKMWpzpywsO Eczg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ruut7Hd3XX0EZMUdhakXLnqZr9tDy0Fu04H3y4CmE5g=; b=RuDZ++EKhdLBnQdQwa9Mq8g6XXKau8ugpZDKeolYZLvQJ2Ij7wjjIjLFNtwkjumJQS Rk2s6kCYA397izWFb3MMFktE5gsVgl62kg4IrTeJ6P5hBq9TWF7MDgwYhp6xzEGps8Rc Dr5YL4rs93ggAXOrHRRPM22ExCYirBiS0jtN31CFg51IDxGLHpMD3xJ1bTvDM8zd7QqY FUj1CWfntWJ9aJC/7mP9Rf1x6RznJAAz/Q9eAA4YNUKBtXuRmF/V+zq/NweAF0PegVNK vR4KXHzUvuM67r5w9nOL16URpQ3mkjt7OoXwFzThd70kRN90+vdRL74Gt++/QWpEUdiG 8F9Q== X-Gm-Message-State: AOAM5306uiYc7gzKABm9qqwVHYmudkhL06tsTQgkdCxNwt/vGoOsSkWJ HS0xdRwgjTknOWjRVeFrvnUHe+aShQo= X-Google-Smtp-Source: ABdhPJwtcQCPcKsL836HtM01oq2yIQgGLMDDuGFlt+YMDNatGfk2+3+5qkxyAbDIEI4tZzagXmhdrw== X-Received: by 2002:adf:f784:: with SMTP id q4mr11456241wrp.397.1594822030385; Wed, 15 Jul 2020 07:07:10 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id l67sm4195339wml.13.2020.07.15.07.07.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jul 2020 07:07:09 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 15 Jul 2020 15:06:54 +0100 Message-Id: <20200715140703.566298-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200715140703.566298-1-naush@raspberrypi.com> References: <20200715140703.566298-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 01/10] libcamera: pipeline: raspberrypi: Move RPiStream into a separate file X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Put RPiStream into the RPi namespace and add a new log category (RPISTREAM) Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/meson.build | 1 + .../pipeline/raspberrypi/raspberrypi.cpp | 193 ++---------------- .../pipeline/raspberrypi/rpi_stream.cpp | 116 +++++++++++ .../pipeline/raspberrypi/rpi_stream.h | 97 +++++++++ 4 files changed, 233 insertions(+), 174 deletions(-) create mode 100644 src/libcamera/pipeline/raspberrypi/rpi_stream.cpp create mode 100644 src/libcamera/pipeline/raspberrypi/rpi_stream.h diff --git a/src/libcamera/pipeline/raspberrypi/meson.build b/src/libcamera/pipeline/raspberrypi/meson.build index dcfe07c5..6d6b893e 100644 --- a/src/libcamera/pipeline/raspberrypi/meson.build +++ b/src/libcamera/pipeline/raspberrypi/meson.build @@ -2,5 +2,6 @@ libcamera_sources += files([ 'raspberrypi.cpp', + 'rpi_stream.cpp', 'staggered_ctrl.cpp', ]) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index a08ad6a8..a80db2eb 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -17,7 +17,6 @@ #include #include #include -#include #include @@ -30,6 +29,7 @@ #include "libcamera/internal/v4l2_controls.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "rpi_stream.h" #include "staggered_ctrl.h" #include "vcsm.h" @@ -121,165 +121,10 @@ V4L2DeviceFormat findBestMode(V4L2PixFmtMap &formatsMap, const Size &req) return bestMode; } -} /* namespace */ - -/* - * Device stream abstraction for either an internal or external stream. - * Used for both Unicam and the ISP. - */ -class RPiStream : public Stream -{ -public: - RPiStream() - { - } - - RPiStream(const char *name, MediaEntity *dev, bool importOnly = false) - : external_(false), importOnly_(importOnly), name_(name), - dev_(std::make_unique(dev)) - { - } - - V4L2VideoDevice *dev() const - { - return dev_.get(); - } - - void setExternal(bool external) - { - external_ = external; - } - - bool isExternal() const - { - /* - * Import streams cannot be external. - * - * RAW capture is a special case where we simply copy the RAW - * buffer out of the request. All other buffer handling happens - * as if the stream is internal. - */ - return external_ && !importOnly_; - } - - bool isImporter() const - { - return importOnly_; - } - - void reset() - { - external_ = false; - internalBuffers_.clear(); - } - - std::string name() const - { - return name_; - } - - void setExternalBuffers(std::vector> *buffers) - { - externalBuffers_ = buffers; - } - - const std::vector> *getBuffers() const - { - return external_ ? externalBuffers_ : &internalBuffers_; - } - - void releaseBuffers() - { - dev_->releaseBuffers(); - if (!external_ && !importOnly_) - internalBuffers_.clear(); - } - - int importBuffers(unsigned int count) - { - return dev_->importBuffers(count); - } - - int allocateBuffers(unsigned int count) - { - return dev_->allocateBuffers(count, &internalBuffers_); - } - - int queueBuffers() - { - if (external_) - return 0; - - for (auto &b : internalBuffers_) { - int ret = dev_->queueBuffer(b.get()); - if (ret) { - LOG(RPI, Error) << "Failed to queue buffers for " - << name_; - return ret; - } - } - - return 0; - } - - bool findFrameBuffer(FrameBuffer *buffer) const - { - auto start = external_ ? externalBuffers_->begin() : internalBuffers_.begin(); - auto end = external_ ? externalBuffers_->end() : internalBuffers_.end(); - - if (importOnly_) - return false; - - if (std::find_if(start, end, - [buffer](std::unique_ptr const &ref) { return ref.get() == buffer; }) != end) - return true; - - return false; - } - -private: - /* - * Indicates that this stream is active externally, i.e. the buffers - * are provided by the application. - */ - bool external_; - /* Indicates that this stream only imports buffers, e.g. ISP input. */ - bool importOnly_; - /* Stream name identifier. */ - std::string name_; - /* The actual device stream. */ - std::unique_ptr dev_; - /* Internally allocated framebuffers associated with this device stream. */ - std::vector> internalBuffers_; - /* Externally allocated framebuffers associated with this device stream. */ - std::vector> *externalBuffers_; -}; - -/* - * The following class is just a convenient (and typesafe) array of device - * streams indexed with an enum class. - */ enum class Unicam : unsigned int { Image, Embedded }; enum class Isp : unsigned int { Input, Output0, Output1, Stats }; -template -class RPiDevice : public std::array -{ -private: - constexpr auto index(E e) const noexcept - { - return static_cast>(e); - } -public: - RPiStream &operator[](E e) - { - return std::array::operator[](index(e)); - } - const RPiStream &operator[](E e) const - { - return std::array::operator[](index(e)); - } -}; +} /* namespace */ class RPiCameraData : public CameraData { @@ -318,15 +163,15 @@ public: void ispOutputDequeue(FrameBuffer *buffer); void clearIncompleteRequests(); - void handleStreamBuffer(FrameBuffer *buffer, const RPiStream *stream); + void handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream); void handleState(); CameraSensor *sensor_; /* Array of Unicam and ISP device streams and associated buffers/streams. */ - RPiDevice unicam_; - RPiDevice isp_; + RPi::RPiDevice unicam_; + RPi::RPiDevice isp_; /* The vector below is just for convenience when iterating over all streams. */ - std::vector streams_; + std::vector streams_; /* Buffers passed to the IPA. */ std::vector ipaBuffers_; @@ -782,7 +627,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) int PipelineHandlerRPi::exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) { - RPiStream *s = static_cast(stream); + RPi::RPiStream *s = static_cast(stream); unsigned int count = stream->configuration().bufferCount; int ret = s->dev()->exportBuffers(count, buffers); @@ -914,14 +759,14 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) std::unique_ptr data = std::make_unique(this); /* Locate and open the unicam video streams. */ - data->unicam_[Unicam::Embedded] = RPiStream("Unicam Embedded", unicam_->getEntityByName("unicam-embedded")); - data->unicam_[Unicam::Image] = RPiStream("Unicam Image", unicam_->getEntityByName("unicam-image")); + data->unicam_[Unicam::Embedded] = RPi::RPiStream("Unicam Embedded", unicam_->getEntityByName("unicam-embedded")); + data->unicam_[Unicam::Image] = RPi::RPiStream("Unicam Image", unicam_->getEntityByName("unicam-image")); /* Tag the ISP input stream as an import stream. */ - data->isp_[Isp::Input] = RPiStream("ISP Input", isp_->getEntityByName("bcm2835-isp0-output0"), true); - data->isp_[Isp::Output0] = RPiStream("ISP Output0", isp_->getEntityByName("bcm2835-isp0-capture1")); - data->isp_[Isp::Output1] = RPiStream("ISP Output1", isp_->getEntityByName("bcm2835-isp0-capture2")); - data->isp_[Isp::Stats] = RPiStream("ISP Stats", isp_->getEntityByName("bcm2835-isp0-capture3")); + data->isp_[Isp::Input] = RPi::RPiStream("ISP Input", isp_->getEntityByName("bcm2835-isp0-output0"), true); + data->isp_[Isp::Output0] = RPi::RPiStream("ISP Output0", isp_->getEntityByName("bcm2835-isp0-capture1")); + data->isp_[Isp::Output1] = RPi::RPiStream("ISP Output1", isp_->getEntityByName("bcm2835-isp0-capture2")); + data->isp_[Isp::Stats] = RPi::RPiStream("ISP Stats", isp_->getEntityByName("bcm2835-isp0-capture3")); /* This is just for convenience so that we can easily iterate over all streams. */ for (auto &stream : data->unicam_) @@ -1070,7 +915,7 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) */ unsigned int maxBuffers = 0; for (const Stream *s : camera->streams()) - if (static_cast(s)->isExternal()) + if (static_cast(s)->isExternal()) maxBuffers = std::max(maxBuffers, s->configuration().bufferCount); for (auto const stream : data->streams_) { @@ -1271,12 +1116,12 @@ done: void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) { - const RPiStream *stream = nullptr; + const RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPiStream const &s : unicam_) { + for (RPi::RPiStream const &s : unicam_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1332,12 +1177,12 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) { - const RPiStream *stream = nullptr; + const RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPiStream const &s : isp_) { + for (RPi::RPiStream const &s : isp_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1418,7 +1263,7 @@ void RPiCameraData::clearIncompleteRequests() } } -void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPiStream *stream) +void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream) { if (stream->isExternal()) { if (!dropFrame_) { diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp new file mode 100644 index 00000000..57e5cf72 --- /dev/null +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -0,0 +1,116 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi (Trading) Ltd. + * + * rpi_stream.cpp - Raspberry Pi device stream abstraction class. + */ +#include "libcamera/internal/log.h" + +#include "rpi_stream.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(RPISTREAM) + +namespace RPi { + +V4L2VideoDevice *RPiStream::dev() const +{ + return dev_.get(); +} + +void RPiStream::setExternal(bool external) +{ + external_ = external; +} + +bool RPiStream::isExternal() const +{ + /* + * Import streams cannot be external. + * + * RAW capture is a special case where we simply copy the RAW + * buffer out of the request. All other buffer handling happens + * as if the stream is internal. + */ + return external_ && !importOnly_; +} + +bool RPiStream::isImporter() const +{ + return importOnly_; +} + +void RPiStream::reset() +{ + external_ = false; + internalBuffers_.clear(); +} + +std::string RPiStream::name() const +{ + return name_; +} + +void RPiStream::setExternalBuffers(std::vector> *buffers) +{ + externalBuffers_ = buffers; +} + +const std::vector> *RPiStream::getBuffers() const +{ + return external_ ? externalBuffers_ : &internalBuffers_; +} + +void RPiStream::releaseBuffers() +{ + dev_->releaseBuffers(); + if (!external_ && !importOnly_) + internalBuffers_.clear(); +} + +int RPiStream::importBuffers(unsigned int count) +{ + return dev_->importBuffers(count); +} + +int RPiStream::allocateBuffers(unsigned int count) +{ + return dev_->allocateBuffers(count, &internalBuffers_); +} + +int RPiStream::queueBuffers() +{ + if (external_) + return 0; + + for (auto &b : internalBuffers_) { + int ret = dev_->queueBuffer(b.get()); + if (ret) { + LOG(RPISTREAM, Error) << "Failed to queue buffers for " + << name_; + return ret; + } + } + + return 0; +} + +bool RPiStream::findFrameBuffer(FrameBuffer *buffer) const +{ + auto start = external_ ? externalBuffers_->begin() : internalBuffers_.begin(); + auto end = external_ ? externalBuffers_->end() : internalBuffers_.end(); + + if (importOnly_) + return false; + + if (std::find_if(start, end, + [buffer](std::unique_ptr const &ref) { return ref.get() == buffer; }) != end) + return true; + + return false; +} + +} /* namespace RPi */ + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h new file mode 100644 index 00000000..40fff81d --- /dev/null +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -0,0 +1,97 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi (Trading) Ltd. + * + * rpi_stream.h - Raspberry Pi device stream abstraction class. + */ +#ifndef __LIBCAMERA_PIPELINE_RPI_STREAM_H__ +#define __LIBCAMERA_PIPELINE_RPI_STREAM_H__ + +#include +#include +#include + +#include "libcamera/internal/v4l2_videodevice.h" +#include + +namespace libcamera { + +namespace RPi { + +/* + * Device stream abstraction for either an internal or external stream. + * Used for both Unicam and the ISP. + */ +class RPiStream : public Stream +{ +public: + RPiStream() + { + } + + RPiStream(const char *name, MediaEntity *dev, bool importOnly = false) + : external_(false), importOnly_(importOnly), name_(name), + dev_(std::make_unique(dev)) + { + } + + V4L2VideoDevice *dev() const; + void setExternal(bool external); + bool isExternal() const; + bool isImporter() const; + void reset(); + std::string name() const; + void setExternalBuffers(std::vector> *buffers); + const std::vector> *getBuffers() const; + void releaseBuffers(); + int importBuffers(unsigned int count); + int allocateBuffers(unsigned int count); + int queueBuffers(); + bool findFrameBuffer(FrameBuffer *buffer) const; + +private: + /* + * Indicates that this stream is active externally, i.e. the buffers + * are provided by the application. + */ + bool external_; + /* Indicates that this stream only imports buffers, e.g. ISP input. */ + bool importOnly_; + /* Stream name identifier. */ + std::string name_; + /* The actual device stream. */ + std::unique_ptr dev_; + /* Internally allocated framebuffers associated with this device stream. */ + std::vector> internalBuffers_; + /* Externally allocated framebuffers associated with this device stream. */ + std::vector> *externalBuffers_; +}; + +/* + * The following class is just a convenient (and typesafe) array of device + * streams indexed with an enum class. + */ +template +class RPiDevice : public std::array +{ +private: + constexpr auto index(E e) const noexcept + { + return static_cast>(e); + } +public: + RPiStream &operator[](E e) + { + return std::array::operator[](index(e)); + } + const RPiStream &operator[](E e) const + { + return std::array::operator[](index(e)); + } +}; + +} /* namespace RPi */ + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_PIPELINE_RPI_STREAM_H__ */ From patchwork Wed Jul 15 14:06:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8825 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E44C2BD790 for ; Wed, 15 Jul 2020 14:07:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B1D9D609C7; Wed, 15 Jul 2020 16:07:16 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Zy6CfmDa"; dkim-atps=neutral Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7E27160579 for ; Wed, 15 Jul 2020 16:07:12 +0200 (CEST) Received: by mail-wm1-x32d.google.com with SMTP id f18so5974725wml.3 for ; Wed, 15 Jul 2020 07:07:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=m4D5LJKKW8ExK8OFbShXoKjTF+Hcogn926zMCP2Xrxg=; b=Zy6CfmDauQla8kAFz6XdgkGuv3iCtkSQEYjZrN2HIxTg1HA7JA3og7Jti8AI2cU7t4 4DEmPpI3bIcElwayLeE8ua8fK6cFgQTU9NR8AB5g0z7Ne5GCN3wzMyzoIXsg2r0yB1Pt Y7eeHPxN94VeSssCwJoZZgbVinPl+X2b4YkG53ZIIWYI7RW3cQJDc5RKGy/FbQcH2gO5 d3pDEFlYzomiIKt3bRp8zQSKm0MV1W8sR3DzhSgBt4qVfjcx/78wHY9CjR/c3OBUj36M tiUeK4w2rQh7sn5Ok9zXTG8Up0xfIUrjfjtyFREImaCftNXzFdSOoE9JZrBF4Qa2ZVdR Bj4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=m4D5LJKKW8ExK8OFbShXoKjTF+Hcogn926zMCP2Xrxg=; b=JS8YHk7b3+Syf4Nvu7w1MMyVHR9gFDt83b2OP2/NSOxD0vW5xAQMg2R3N1GgPmgiKb HIYoI53Tc4oCeCxCRvO2iOswS2HFzOTHsqjWbJoHo0VCRm6SNnPrSofJciJSDtkG09Xn NgpwlD9nbuFyyHgz2DUsfvad3ABttUS850vDc2hlh+PvcugtiKUoBvQBC9Q1GQDdxeAU KGWAZBXNKF35zbakjzh3W0tUqjCmbJVVNmd5tv7FDNR5k1WU1PzX5ZO1acJvhZ16zFZS PSkT1O/pOOlAwKFfOzzHOHyxxpgwKL8Yb2X4R4ZGZO180/p0yth1vmauwaHrA4kHhKDG HqYA== X-Gm-Message-State: AOAM532P6ZRaAg81/DQN7o2Kw+J2WoH7g9t0L3963tFAs2i8bs19gcU2 Ly4oCBVTqJH+ZfDGmrBVV8NGSvE5+IU= X-Google-Smtp-Source: ABdhPJzPVufRiq4S26q02mlfGoaz2bF1YVoBY3QAPJaF4uObShVc8XKHwD9gGkfpjefeUDLfgCYo8g== X-Received: by 2002:a1c:1bc4:: with SMTP id b187mr9427041wmb.105.1594822031590; Wed, 15 Jul 2020 07:07:11 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id l67sm4195339wml.13.2020.07.15.07.07.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jul 2020 07:07:10 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 15 Jul 2020 15:06:55 +0100 Message-Id: <20200715140703.566298-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200715140703.566298-1-naush@raspberrypi.com> References: <20200715140703.566298-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 02/10] libcamera: pipeline: ipa: raspberrypi: Rework drop frame signalling X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The IPA now signals up front how many frames it wants the pipeline handler to drop. This makes it easier to handle up-coming changes to the buffer handling for import/export buffers. Signed-off-by: Naushir Patuck --- include/libcamera/ipa/raspberrypi.h | 2 +- src/ipa/raspberrypi/raspberrypi.cpp | 21 +++++++------- .../pipeline/raspberrypi/raspberrypi.cpp | 29 ++++++++++++------- 3 files changed, 30 insertions(+), 22 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index a18ce9a8..2dab2b00 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -15,8 +15,8 @@ enum RPiOperations { RPI_IPA_ACTION_V4L2_SET_ISP, RPI_IPA_ACTION_STATS_METADATA_COMPLETE, RPI_IPA_ACTION_RUN_ISP, - RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME, RPI_IPA_ACTION_SET_SENSOR_CONFIG, + RPI_IPA_ACTION_SET_DROP_FRAME_COUNT, RPI_IPA_ACTION_EMBEDDED_COMPLETE, RPI_IPA_EVENT_SIGNAL_STAT_READY, RPI_IPA_EVENT_SIGNAL_ISP_PREPARE, diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index b1f27861..f3568882 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -64,8 +64,8 @@ class IPARPi : public IPAInterface public: IPARPi() : lastMode_({}), controller_(), controllerInit_(false), - frame_count_(0), check_count_(0), hide_count_(0), - mistrust_count_(0), lsTableHandle_(0), lsTable_(nullptr) + frame_count_(0), check_count_(0), mistrust_count_(0), + lsTableHandle_(0), lsTable_(nullptr) { } @@ -132,8 +132,6 @@ private: uint64_t frame_count_; /* For checking the sequencing of Prepare/Process calls. */ uint64_t check_count_; - /* How many frames the pipeline handler should hide, or "drop". */ - unsigned int hide_count_; /* How many frames we should avoid running control algos on. */ unsigned int mistrust_count_; /* LS table allocation passed in from the pipeline handler. */ @@ -241,14 +239,19 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, */ frame_count_ = 0; check_count_ = 0; + int drop_frame = 0; if (controllerInit_) { - hide_count_ = helper_->HideFramesModeSwitch(); + drop_frame = helper_->HideFramesModeSwitch(); mistrust_count_ = helper_->MistrustFramesModeSwitch(); } else { - hide_count_ = helper_->HideFramesStartup(); + drop_frame = helper_->HideFramesStartup(); mistrust_count_ = helper_->MistrustFramesStartup(); } + IPAOperationData op; + op.operation = RPI_IPA_ACTION_SET_DROP_FRAME_COUNT; + op.data.push_back(drop_frame); + struct AgcStatus agcStatus; /* These zero values mean not program anything (unless overwritten). */ agcStatus.shutter_time = 0.0; @@ -339,13 +342,11 @@ void IPARPi::processEvent(const IPAOperationData &event) * they are "unreliable". */ prepareISP(embeddedbufferId); + frame_count_++; /* Ready to push the input buffer into the ISP. */ IPAOperationData op; - if (++frame_count_ > hide_count_) - op.operation = RPI_IPA_ACTION_RUN_ISP; - else - op.operation = RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME; + op.operation = RPI_IPA_ACTION_RUN_ISP; op.data = { bayerbufferId & RPiIpaMask::ID }; queueFrameAction.emit(0, op); break; diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index a80db2eb..8b6f578f 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -131,7 +131,7 @@ class RPiCameraData : public CameraData public: RPiCameraData(PipelineHandler *pipe) : CameraData(pipe), sensor_(nullptr), lsTable_(nullptr), - state_(State::Stopped), dropFrame_(false), ispOutputCount_(0) + state_(State::Stopped), dropFrameCount_(0), ispOutputCount_(0) { } @@ -194,14 +194,15 @@ public: std::queue embeddedQueue_; std::deque requestQueue_; + unsigned int dropFrameCount_; + private: void checkRequestCompleted(); void tryRunPipeline(); void tryFlushQueues(); FrameBuffer *updateQueue(std::queue &q, uint64_t timestamp, V4L2VideoDevice *dev); - bool dropFrame_; - int ispOutputCount_; + unsigned int ispOutputCount_; }; class RPiCameraConfiguration : public CameraConfiguration @@ -1058,6 +1059,11 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData goto done; } + case RPI_IPA_ACTION_SET_DROP_FRAME_COUNT: { + dropFrameCount_ = action.data[0]; + goto done; + } + case RPI_IPA_ACTION_V4L2_SET_ISP: { ControlList controls = action.controls[0]; isp_[Isp::Input].dev()->setControls(&controls); @@ -1091,7 +1097,6 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData break; } - case RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME: case RPI_IPA_ACTION_RUN_ISP: { unsigned int bufferId = action.data[0]; FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers()->at(bufferId).get(); @@ -1100,7 +1105,6 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData << ", timestamp: " << buffer->metadata().timestamp; isp_[Isp::Input].dev()->queueBuffer(buffer); - dropFrame_ = (action.operation == RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME) ? true : false; ispOutputCount_ = 0; break; } @@ -1266,7 +1270,7 @@ void RPiCameraData::clearIncompleteRequests() void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream) { if (stream->isExternal()) { - if (!dropFrame_) { + if (!dropFrameCount_) { Request *request = buffer->request(); pipe_->completeBuffer(camera_, request, buffer); } @@ -1278,7 +1282,7 @@ void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream * simply memcpy to the Request buffer and requeue back to the * device. */ - if (stream == &unicam_[Unicam::Image] && !dropFrame_) { + if (stream == &unicam_[Unicam::Image] && !dropFrameCount_) { const Stream *rawStream = static_cast(&isp_[Isp::Input]); Request *request = requestQueue_.front(); FrameBuffer *raw = request->findBuffer(const_cast(rawStream)); @@ -1323,7 +1327,7 @@ void RPiCameraData::checkRequestCompleted() * If we are dropping this frame, do not touch the request, simply * change the state to IDLE when ready. */ - if (!dropFrame_) { + if (!dropFrameCount_) { Request *request = requestQueue_.front(); if (request->hasPendingBuffers()) return; @@ -1342,10 +1346,13 @@ void RPiCameraData::checkRequestCompleted() * frame. */ if (state_ == State::IpaComplete && - ((ispOutputCount_ == 3 && dropFrame_) || requestCompleted)) { + ((ispOutputCount_ == 3 && dropFrameCount_) || requestCompleted)) { state_ = State::Idle; - if (dropFrame_) - LOG(RPI, Info) << "Dropping frame at the request of the IPA"; + if (dropFrameCount_) { + dropFrameCount_--; + LOG(RPI, Info) << "Dropping frame at the request of the IPA ( " + << dropFrameCount_ << " left)"; + } } } From patchwork Wed Jul 15 14:06:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8826 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 65216BD790 for ; Wed, 15 Jul 2020 14:07:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1D42D609A0; Wed, 15 Jul 2020 16:07:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="HmfkcLQH"; dkim-atps=neutral Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9E0F2609A0 for ; Wed, 15 Jul 2020 16:07:14 +0200 (CEST) Received: by mail-wm1-x343.google.com with SMTP id o8so5730065wmh.4 for ; Wed, 15 Jul 2020 07:07:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kTU4Ekw107BBgDT7sZ1t6Vbk53bJFssFJ+kUcM6X3B8=; b=HmfkcLQHdEDwJiVrvqhalLLTj0rSMA0GuOCQ0ONagN2zXzv/2UqprJ7/l5lS21vAZE qod2cxDe+PFDPzlxx7cgFD4ehrF3m4bqcwIElOyq+cjAcSZzyK6fKqx6vyzB/YbW/N5z 41K4MnM7JztQL230QpFjkI4nfnyGtLxJ8Lv6Xithekp6KNaZAwSIVSo6JmlcS1nZ4bNM 5koI5JpBgGDITz/ZPNH1Z7DvYU8xMx49SkvDMp4OvgeBA5HARpZDf+9ELIHBysb6bRi4 6I+k/ZcivqYxigcgaFjJswif/S0k8JoKSwlqebwNrnKvbWRW12XHTmr1Z2ZoXKnVXxn6 ptBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kTU4Ekw107BBgDT7sZ1t6Vbk53bJFssFJ+kUcM6X3B8=; b=YAz8Lhj6b9f1352C7rA5879es6TDAJLTa+7FJHblEc4LCE8x2YoRQLkJqtMSuROC7A Z+qV5/TBnOFMHzjU5ul/0LLNbbgPgAChtiEB9C/DhTzPw3HHWXlR/4hJbw2I4a/n0S+4 ie6HH95DwnN0jT/NCcmYZiEh0rwkB8rDwM3Hnmdfxewkffglt9UYQSGTqiDfiL9aQe3r 7ad3jcXopVAzOGc7JI4uP0EfAfIFRZyPaZdRBWSmXzQ1dSQ/kkwl3hvbhrY88m+SHGM/ ltmcy3IATLjXvSDfoy3dkPa9NS8Bq6VELLfsXFGXoZkL54g0gc++aDNSRDTPdBld/7tm l16w== X-Gm-Message-State: AOAM533XRhd/INvbNDIyYqU5/sfMTeXQ7bgO632jZGHtdseezba+TXOn 8fUUJKeTZVHWgit8yqxi6C6WfrdlwHo= X-Google-Smtp-Source: ABdhPJy56VfZhqA2qVi7nsQOsJb5D1SgplBgCGJcJB12homlac6G0PpCzhc20Q1zVd+P+8dQCTNUDA== X-Received: by 2002:a1c:790c:: with SMTP id l12mr8766811wme.50.1594822033627; Wed, 15 Jul 2020 07:07:13 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id l67sm4195339wml.13.2020.07.15.07.07.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jul 2020 07:07:11 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 15 Jul 2020 15:06:56 +0100 Message-Id: <20200715140703.566298-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200715140703.566298-1-naush@raspberrypi.com> References: <20200715140703.566298-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 03/10] libcamera: pipeline: raspberrypi: Add some debug logging X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" No functional changes, only added some more trace points. Signed-off-by: Naushir Patuck --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 8b6f578f..65c8557d 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1175,6 +1175,11 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) if (state_ == State::Stopped) return; + LOG(RPI, Debug) << "Stream ISP Input buffer complete" + << ", buffer id " << buffer->cookie() + << ", timestamp: " << buffer->metadata().timestamp; + + /* The ISP input buffer gets re-queued into Unicam. */ handleStreamBuffer(buffer, &unicam_[Unicam::Image]); handleState(); } @@ -1339,6 +1344,8 @@ void RPiCameraData::checkRequestCompleted() pipe_->completeRequest(camera_, request); requestQueue_.pop_front(); requestCompleted = true; + + LOG(RPI, Debug) << "Request is complete"; } /* @@ -1477,7 +1484,7 @@ FrameBuffer *RPiCameraData::updateQueue(std::queue &q, uint64_t t if (b->metadata().timestamp < timestamp) { q.pop(); dev->queueBuffer(b); - LOG(RPI, Error) << "Dropping input frame!"; + LOG(RPI, Warning) << "Dropping input frame!"; } else if (b->metadata().timestamp == timestamp) { /* The calling function will pop the item from the queue. */ return b; From patchwork Wed Jul 15 14:06:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8827 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DD74FBD790 for ; Wed, 15 Jul 2020 14:07:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A1D37609A5; Wed, 15 Jul 2020 16:07:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="NzKZfcTU"; dkim-atps=neutral Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4210F60487 for ; Wed, 15 Jul 2020 16:07:16 +0200 (CEST) Received: by mail-wr1-x42c.google.com with SMTP id k6so2879206wrn.3 for ; Wed, 15 Jul 2020 07:07:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LZGD0uLcsL+C84A+8e4/kmbzXQMygMHVTfZe4dBWDLo=; b=NzKZfcTUrTIx1FaIXSndiZz3NNorocrT0olZBLdWMeq2QYbZ4zva9FLEJ6FG4qMbcZ 41riHzcpvEd1hTFeYCHAYWI7MbTohwanUnFh/HMRe1nyFlJcwWjDE2qRRNc4fShcl96C CIAU0pn/DbrIbl2M0yhK5D4kQ3/k53iSXwbGyWEunfs2wiEiHKLUQ38+Gc5U7ufoDU6I zYLRg3OaWZz6JiQGXPpYyVdfymDuSRFlSKQgcrU9ozm6dKGTE5jy0feqfV9yjUtrYI9W PtwLFO/JkziureOmINTRmgPZZz9rrjuzEJKWGzBhKbDFVrqCAQEdopzEs9SE1UAs89IO Z9iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LZGD0uLcsL+C84A+8e4/kmbzXQMygMHVTfZe4dBWDLo=; b=IwbiFy3Pyxt8yV2HtB16YVFYX9+koUhnfT9NBVa2JygqkfgB/GyMlhLGr+1tGfY9YY WWaTRCesMy8mIectnU+oaH01AkkrR6loIqu8g8tVZd3y1DpLFp9rv9f0C7zU9IG3HQF4 1mYjMgTwUz5buaWfkfHrjbAhKjyoXztriNLsKp3OYz14IxvKprocScUpr4nXLymiBWcW 2ZxzCcRvf1oVmLzUJ6nBrpP0q6Eh0MMEClXRncSLz78//HjbRPQJwQg/f6mmKhbmfeCu qht9PKtZYop12zvsUFUXeSvZ4WENYYrEQbSygvA1ia1n2Ug3g2KDr8ZKr9XUlPwDw2ZQ AAfQ== X-Gm-Message-State: AOAM5309o4Y/bHdmqqJ7Ihk6AM6cG40hnUZ+V5POIKXhM8/tfGmXvhy+ 4Aw7GuLIYoeBFE7k3u+XKd+LRY+r5h8= X-Google-Smtp-Source: ABdhPJygsGzfiM94Sx5UFG7AGtCduevdkAVgj0L4OYxlEs1hlGODO6kML1ewgWTiT/4h2+oIUvwu4w== X-Received: by 2002:a5d:4e50:: with SMTP id r16mr11460765wrt.274.1594822035511; Wed, 15 Jul 2020 07:07:15 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id l67sm4195339wml.13.2020.07.15.07.07.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jul 2020 07:07:13 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 15 Jul 2020 15:06:57 +0100 Message-Id: <20200715140703.566298-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200715140703.566298-1-naush@raspberrypi.com> References: <20200715140703.566298-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 04/10] libcamera: pipeline: raspberrypi: Increase the number of RAW buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Increase the number of expected RAW buffers in the stream configuration to 2. This will avoid dropping Unicam frames when exporting RAW streams. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 65c8557d..a613ce65 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -394,7 +394,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, sensorFormat = findBestMode(fmts, size); pixelFormat = sensorFormat.fourcc.toPixelFormat(); ASSERT(pixelFormat.isValid()); - bufferCount = 1; + bufferCount = 2; rawCount++; break; From patchwork Wed Jul 15 14:06:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8828 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4BC36BD790 for ; Wed, 15 Jul 2020 14:07:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 180C16093A; Wed, 15 Jul 2020 16:07:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="rP3jMW6i"; dkim-atps=neutral Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 152BA6093A for ; Wed, 15 Jul 2020 16:07:19 +0200 (CEST) Received: by mail-wm1-x332.google.com with SMTP id j18so5744798wmi.3 for ; Wed, 15 Jul 2020 07:07:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NC23Yh0Jo9Xk9clV400Nvp3F4rLVrpwh1znNskXO+qY=; b=rP3jMW6i1C07YmcXRE+euAic5pps2SF+Uu661PAik0Ifdo9/kUOH2Dw/SidQJeOfGS vANVvN4wePi2NCApWrA0Ge/mgUzZ7bcQoopkHcy+04tHEV9bzZjrSAu1IppF8ltSZqAw vjH8yXsTijBSnzxTW4gpig/CNCx5rU8N8m9+abvEq9df0P+kDDLN4WevsOv8aQMN6hCM Od7g2/Fd7B96k1aRXZxMjeH4RD7QZzntnkS79SHQhg0MGwKSHQUxJwqaXmUotnpEQWOq Qw0yiZ4AptaA93++VeFfxSfy0L4cVfQm3OkzJij/+CNqyawUlbjweChDnccmsZ0iW7A4 V7VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NC23Yh0Jo9Xk9clV400Nvp3F4rLVrpwh1znNskXO+qY=; b=lkldCZ1Es62ccrFLU5RogWA0V7W/csvU3CyAfkX80xTYYmqsrS8fesvMR11XYAc//4 /rmtbOVgnasUKsTeURoRYpvIPXqJYjETRA15BN2cam1yonF9ahX4Pyi37qZIT3PUaEeA kjtsarEYlUY4WAMEd/z1jXDeXKqRAKOGH+ZE3xa4i8oAtDyAtH6W1nJK+CXgLIaYY1Qf yHBzMwLErzfFn/zazMHNssLdvshAqjG+Gy44elPcQU7WKJLyuq1BZVCfB0oGDqzjf60+ 9U9PIAVmzbUpqOHEmD4laXX1BxTyuAqgpIPcRNv3THPvJX1UX4+h2mp24p442nW1qCPE msww== X-Gm-Message-State: AOAM533GKo+bCWd32QFi2ZAR5jf17NSLde1W0fZaVGOeQyP2+bhtQvNb f8dPLG6IyEtb5PeMxhsVqB/HuC3WfbM= X-Google-Smtp-Source: ABdhPJxC+XnUt13GydW9lAm5uNGV+h6vriV9pnm9Ch51RWwOKWbWCzRnOofDzwydPuGJFZccb111Cw== X-Received: by 2002:a1c:a90d:: with SMTP id s13mr8466186wme.184.1594822037531; Wed, 15 Jul 2020 07:07:17 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id l67sm4195339wml.13.2020.07.15.07.07.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jul 2020 07:07:15 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 15 Jul 2020 15:06:58 +0100 Message-Id: <20200715140703.566298-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200715140703.566298-1-naush@raspberrypi.com> References: <20200715140703.566298-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 05/10] libcamera: pipeline: raspberrypi: Remove const qualifier from RPiStream X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For the zero-copy RAW capture feature, the RPiStream will have to be modified. Remove the const qualifier in anticipation of the future commits for this feature. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index a613ce65..2f4255a4 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -163,7 +163,7 @@ public: void ispOutputDequeue(FrameBuffer *buffer); void clearIncompleteRequests(); - void handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream); + void handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream); void handleState(); CameraSensor *sensor_; @@ -1120,12 +1120,12 @@ done: void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) { - const RPi::RPiStream *stream = nullptr; + RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPi::RPiStream const &s : unicam_) { + for (RPi::RPiStream &s : unicam_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1186,12 +1186,12 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) { - const RPi::RPiStream *stream = nullptr; + RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPi::RPiStream const &s : isp_) { + for (RPi::RPiStream &s : isp_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1272,7 +1272,7 @@ void RPiCameraData::clearIncompleteRequests() } } -void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream) +void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream) { if (stream->isExternal()) { if (!dropFrameCount_) { From patchwork Wed Jul 15 14:06:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8829 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A3BC6BD790 for ; Wed, 15 Jul 2020 14:07:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6C5C960BF1; Wed, 15 Jul 2020 16:07:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="QOSfgVib"; dkim-atps=neutral Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DDF13609A3 for ; Wed, 15 Jul 2020 16:07:21 +0200 (CEST) Received: by mail-wr1-x434.google.com with SMTP id o11so2847581wrv.9 for ; Wed, 15 Jul 2020 07:07:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Pcha+JQRT7YTGtnvxjZmmChxkQ9Rdh9h3mGIavmMW1w=; b=QOSfgVibosdCDwbp8t8gMYJHOTE7M4uyYMELlbSEwxyVt4yiT7nUynEY4rIMYMDUmH qiEi+jo4PAbcAb7VgwalqQ07ujwmbhmNsrsFIdNSNwgOKuj/J2X5OWI/4hNaBsDhmC+N h6bV5SejVQZa0sIXqcggIfOgJlwLYAFONOzi6s50PV2dY0zQ1bUiYyeffH44p7jYLgkx 471YGFi44sR+MrNf5q2Hw0ZSqsu59roeWeF23l+/GYU2iO3zcRLLN1kNM1lgg7zFQXrV ePmuRXRJWqMA97Ax9mnvXdhaeLBL7UucO3fHdFHvgQmWrjsEHl+d4tEOWkShIBGpBrZc BZlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Pcha+JQRT7YTGtnvxjZmmChxkQ9Rdh9h3mGIavmMW1w=; b=gZqXvTkSwisQZKJW6zF5fS8e7eQ70TdmPIXEPnDjagOOU1xTnj0n12WqZSXUDF/w6c Mn7yNFEpDk+VrbovQ5YrMkmuMzqqzTD2nRqPrtV6tu+CnEWon1r7WRtiU3QpSL2Lm8br 3RUQsWWiJFF8KDXJb9OlYB20MFZunDyButbfyqhtn551KPp94RYktgQC64GlMaTBAyWC RJSJ0tNQBBmIvOTyrI8NvH8UeOnTxbM/zK5NGP25Ijiu9h/SGA1V9TVyE7zzWz+gp21v L37l4ewN1hVW1eG2xXFFD2sl/Tvfs+VTgNow6f/koWpya++CuEypdn1X4cLmoIp+UJ80 7vRQ== X-Gm-Message-State: AOAM530yRvWZz88NYdR8ysTr/RCSKDSKnBdGAsWw+ZYiCVwqyQRsPo1J I0Lt6Fu+CKVZ0KEAeyq0tynDFMWV9Wo= X-Google-Smtp-Source: ABdhPJxAjIBLSxGWg9HQt1f8PwnkWyQHXzkCmlI7vQajX5gyEbHufjVBvapxNZsHnngU60cZxi4Icw== X-Received: by 2002:adf:ef46:: with SMTP id c6mr11576035wrp.34.1594822040007; Wed, 15 Jul 2020 07:07:20 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id l67sm4195339wml.13.2020.07.15.07.07.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jul 2020 07:07:18 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 15 Jul 2020 15:06:59 +0100 Message-Id: <20200715140703.566298-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200715140703.566298-1-naush@raspberrypi.com> References: <20200715140703.566298-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 06/10] libcamera: pipeline: raspberrypi: Rework stream buffer logic for zero-copy X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Stop using v4l2_videodevice::allocateBuffer() for internal buffers and instead export/import all buffers. This allows the pipeline to return any stream buffer requested by the application as zero-copy. Advertise the Unicam Image stream as the RAW capture stream now. The RPiStream object now maintains a new list of buffers that are available to queue into a device. This is needed to distinguish between FrameBuffers allocated for internal use vs externally provided buffers. When a Request comes in, if a buffer is not provided for an exported stream, we re-use a buffer from this list. Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 201 +++++++++--------- .../pipeline/raspberrypi/rpi_stream.cpp | 119 +++++++---- .../pipeline/raspberrypi/rpi_stream.h | 28 ++- 3 files changed, 199 insertions(+), 149 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 2f4255a4..cd635eb3 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -200,7 +200,8 @@ private: void checkRequestCompleted(); void tryRunPipeline(); void tryFlushQueues(); - FrameBuffer *updateQueue(std::queue &q, uint64_t timestamp, V4L2VideoDevice *dev); + FrameBuffer *updateQueue(std::queue &q, uint64_t timestamp, + RPi::RPiStream *stream); unsigned int ispOutputCount_; }; @@ -523,8 +524,15 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) StreamConfiguration &cfg = config->at(i); if (isRaw(cfg.pixelFormat)) { - cfg.setStream(&data->isp_[Isp::Input]); - data->isp_[Isp::Input].setExternal(true); + cfg.setStream(&data->unicam_[Unicam::Image]); + /* + * We must set both Unicam streams as external, even + * though the application may only request RAW frames. + * This is because we match timestamps on both streams + * to synchronise buffers. + */ + data->unicam_[Unicam::Image].setExternal(true); + data->unicam_[Unicam::Embedded].setExternal(true); continue; } @@ -718,14 +726,21 @@ int PipelineHandlerRPi::queueRequestDevice(Camera *camera, Request *request) if (data->state_ == RPiCameraData::State::Stopped) return -EINVAL; - /* Ensure all external streams have associated buffers! */ - for (auto &stream : data->isp_) { - if (!stream.isExternal()) - continue; + LOG(RPI, Debug) << "queueRequestDevice: New request."; - if (!request->findBuffer(&stream)) { - LOG(RPI, Error) << "Attempt to queue request with invalid stream."; - return -ENOENT; + /* Push all buffers supplied in the Request to the respective streams. */ + for (auto stream : data->streams_) { + if (stream->isExternal()) { + FrameBuffer *buffer = request->findBuffer(stream); + /* + * If we have been asked to drop frames by the IPA, passing + * in a nullptr here will queue an internally allocated buffer. + * The Request buffer will be stored in another queue to be + * queued to the device - in sync with the Request. + */ + int ret = stream->queueBuffer(data->dropFrameCount_ ? nullptr : buffer); + if (ret) + return ret; } } @@ -814,12 +829,10 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); - /* - * List the available output streams. - * Currently cannot do Unicam streams! - */ + /*List the available streams an application may request. */ std::set streams; - streams.insert(&data->isp_[Isp::Input]); + streams.insert(&data->unicam_[Unicam::Image]); + streams.insert(&data->unicam_[Unicam::Embedded]); streams.insert(&data->isp_[Isp::Output0]); streams.insert(&data->isp_[Isp::Output1]); streams.insert(&data->isp_[Isp::Stats]); @@ -896,7 +909,7 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera) int ret; for (auto const stream : data->streams_) { - ret = stream->queueBuffers(); + ret = stream->queueAllBuffers(); if (ret < 0) return ret; } @@ -912,7 +925,8 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) /* * Decide how many internal buffers to allocate. For now, simply look * at how many external buffers will be provided. Will need to improve - * this logic. + * this logic. However, we really must have all stream allocate the same + * number of buffers to simplify error handling in queueRequestDevice(). */ unsigned int maxBuffers = 0; for (const Stream *s : camera->streams()) @@ -920,33 +934,9 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) maxBuffers = std::max(maxBuffers, s->configuration().bufferCount); for (auto const stream : data->streams_) { - if (stream->isExternal() || stream->isImporter()) { - /* - * If a stream is marked as external reserve memory to - * prepare to import as many buffers are requested in - * the stream configuration. - * - * If a stream is an internal stream with importer - * role, reserve as many buffers as possible. - */ - unsigned int count = stream->isExternal() - ? stream->configuration().bufferCount - : maxBuffers; - ret = stream->importBuffers(count); - if (ret < 0) - return ret; - } else { - /* - * If the stream is an internal exporter allocate and - * export as many buffers as possible to its internal - * pool. - */ - ret = stream->allocateBuffers(maxBuffers); - if (ret < 0) { - freeBuffers(camera); - return ret; - } - } + ret = stream->prepareBuffers(maxBuffers); + if (ret < 0) + return ret; } /* @@ -954,7 +944,7 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * the IPA and RPI_IPA_EVENT_SIGNAL_ISP_PREPARE event. */ count = 0; - for (auto const &b : *data->unicam_[Unicam::Image].getBuffers()) { + for (auto const &b : data->unicam_[Unicam::Image].getBuffers()) { b->setCookie(count++); } @@ -963,14 +953,14 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * the IPA. */ count = 0; - for (auto const &b : *data->isp_[Isp::Stats].getBuffers()) { + for (auto const &b : data->isp_[Isp::Stats].getBuffers()) { b->setCookie(count++); data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | b->cookie(), .planes = b->planes() }); } count = 0; - for (auto const &b : *data->unicam_[Unicam::Embedded].getBuffers()) { + for (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) { b->setCookie(count++); data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | b->cookie(), .planes = b->planes() }); @@ -1081,7 +1071,7 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData switch (action.operation) { case RPI_IPA_ACTION_STATS_METADATA_COMPLETE: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = isp_[Isp::Stats].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = isp_[Isp::Stats].getBuffers().at(bufferId); handleStreamBuffer(buffer, &isp_[Isp::Stats]); /* Fill the Request metadata buffer with what the IPA has provided */ @@ -1092,19 +1082,19 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData case RPI_IPA_ACTION_EMBEDDED_COMPLETE: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = unicam_[Unicam::Embedded].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = unicam_[Unicam::Embedded].getBuffers().at(bufferId); handleStreamBuffer(buffer, &unicam_[Unicam::Embedded]); break; } case RPI_IPA_ACTION_RUN_ISP: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers().at(bufferId); LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << buffer->cookie() << ", timestamp: " << buffer->metadata().timestamp; - isp_[Isp::Input].dev()->queueBuffer(buffer); + isp_[Isp::Input].queueBuffer(buffer); ispOutputCount_ = 0; break; } @@ -1205,22 +1195,27 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) << ", buffer id " << buffer->cookie() << ", timestamp: " << buffer->metadata().timestamp; - handleStreamBuffer(buffer, stream); - /* - * Increment the number of ISP outputs generated. - * This is needed to track dropped frames. + * ISP statistics buffer must not be re-queued or sent back to the + * application until after the IPA signals so. */ - ispOutputCount_++; - - /* If this is a stats output, hand it to the IPA now. */ if (stream == &isp_[Isp::Stats]) { IPAOperationData op; op.operation = RPI_IPA_EVENT_SIGNAL_STAT_READY; op.data = { RPiIpaMask::STATS | buffer->cookie() }; ipa_->processEvent(op); + } else { + /* Any other ISP output can be handed back to the application now. */ + handleStreamBuffer(buffer, stream); } + /* + * Increment the number of ISP outputs generated. + * This is needed to track dropped frames. + */ + ispOutputCount_++; + + handleState(); } @@ -1233,8 +1228,12 @@ void RPiCameraData::clearIncompleteRequests() */ for (auto const request : requestQueue_) { for (auto const stream : streams_) { - if (stream->isExternal()) - stream->dev()->queueBuffer(request->findBuffer(stream)); + if (!stream->isExternal()) + continue; + + FrameBuffer *buffer = request->findBuffer(stream); + if (buffer) + stream->queueBuffer(buffer); } } @@ -1263,7 +1262,7 @@ void RPiCameraData::clearIncompleteRequests() * Has the buffer already been handed back to the * request? If not, do so now. */ - if (buffer->request()) + if (buffer && buffer->request()) pipe_->completeBuffer(camera_, request, buffer); } @@ -1275,30 +1274,24 @@ void RPiCameraData::clearIncompleteRequests() void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream) { if (stream->isExternal()) { - if (!dropFrameCount_) { - Request *request = buffer->request(); + Request *request = requestQueue_.front(); + + if (!dropFrameCount_ && request->findBuffer(stream) == buffer) { + /* + * Tag the buffer as completed, returning it to the + * application. + */ pipe_->completeBuffer(camera_, request, buffer); + } else { + /* + * This buffer was not part of the Request, so we can + * recycle it. + */ + stream->returnBuffer(buffer); } } else { - /* Special handling for RAW buffer Requests. - * - * The ISP input stream is alway an import stream, but if the - * current Request has been made for a buffer on the stream, - * simply memcpy to the Request buffer and requeue back to the - * device. - */ - if (stream == &unicam_[Unicam::Image] && !dropFrameCount_) { - const Stream *rawStream = static_cast(&isp_[Isp::Input]); - Request *request = requestQueue_.front(); - FrameBuffer *raw = request->findBuffer(const_cast(rawStream)); - if (raw) { - raw->copyFrom(buffer); - pipe_->completeBuffer(camera_, request, raw); - } - } - - /* Simply requeue the buffer. */ - stream->dev()->queueBuffer(buffer); + /* Simply re-queue the buffer to the requested stream. */ + stream->queueBuffer(buffer); } } @@ -1382,7 +1375,7 @@ void RPiCameraData::tryRunPipeline() * current bayer buffer will be removed and re-queued to the driver. */ embeddedBuffer = updateQueue(embeddedQueue_, bayerBuffer->metadata().timestamp, - unicam_[Unicam::Embedded].dev()); + &unicam_[Unicam::Embedded]); if (!embeddedBuffer) { LOG(RPI, Debug) << "Could not find matching embedded buffer"; @@ -1401,7 +1394,7 @@ void RPiCameraData::tryRunPipeline() embeddedBuffer = embeddedQueue_.front(); bayerBuffer = updateQueue(bayerQueue_, embeddedBuffer->metadata().timestamp, - unicam_[Unicam::Image].dev()); + &unicam_[Unicam::Image]); if (!bayerBuffer) { LOG(RPI, Debug) << "Could not find matching bayer buffer - ending."; @@ -1409,11 +1402,7 @@ void RPiCameraData::tryRunPipeline() } } - /* - * Take the first request from the queue and action the IPA. - * Unicam buffers for the request have already been queued as they come - * in. - */ + /* Take the first request from the queue and action the IPA. */ Request *request = requestQueue_.front(); /* @@ -1425,12 +1414,6 @@ void RPiCameraData::tryRunPipeline() op.controls = { request->controls() }; ipa_->processEvent(op); - /* Queue up any ISP buffers passed into the request. */ - for (auto &stream : isp_) { - if (stream.isExternal()) - stream.dev()->queueBuffer(request->findBuffer(&stream)); - } - /* Ready to use the buffers, pop them off the queue. */ bayerQueue_.pop(); embeddedQueue_.pop(); @@ -1460,32 +1443,42 @@ void RPiCameraData::tryFlushQueues() * and give a chance for the hardware to return to lock-step. We do have * to drop all interim frames. */ - if (unicam_[Unicam::Image].getBuffers()->size() == bayerQueue_.size() && - unicam_[Unicam::Embedded].getBuffers()->size() == embeddedQueue_.size()) { + if (unicam_[Unicam::Image].getBuffers().size() == bayerQueue_.size() && + unicam_[Unicam::Embedded].getBuffers().size() == embeddedQueue_.size()) { + /* This cannot happen when Unicam streams are external. */ + assert(!unicam_[Unicam::Image].isExternal()); + LOG(RPI, Warning) << "Flushing all buffer queues!"; while (!bayerQueue_.empty()) { - unicam_[Unicam::Image].dev()->queueBuffer(bayerQueue_.front()); + unicam_[Unicam::Image].queueBuffer(bayerQueue_.front()); bayerQueue_.pop(); } while (!embeddedQueue_.empty()) { - unicam_[Unicam::Embedded].dev()->queueBuffer(embeddedQueue_.front()); + unicam_[Unicam::Embedded].queueBuffer(embeddedQueue_.front()); embeddedQueue_.pop(); } } } FrameBuffer *RPiCameraData::updateQueue(std::queue &q, uint64_t timestamp, - V4L2VideoDevice *dev) + RPi::RPiStream *stream) { + /* + * If the unicam streams are external (both have to the same), then we + * can only return out the top buffer in the queue, and assume they have + * been synced by queuing at the same time. We cannot drop these frames, + * as they may have been provided externally. + */ while (!q.empty()) { FrameBuffer *b = q.front(); - if (b->metadata().timestamp < timestamp) { + if (!stream->isExternal() && b->metadata().timestamp < timestamp) { q.pop(); - dev->queueBuffer(b); - LOG(RPI, Warning) << "Dropping input frame!"; - } else if (b->metadata().timestamp == timestamp) { + stream->queueBuffer(b); + LOG(RPI, Warning) << "Dropping unmatched input frame in stream " + << stream->name(); + } else if (stream->isExternal() || b->metadata().timestamp == timestamp) { /* The calling function will pop the item from the queue. */ return b; } else { diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index 57e5cf72..73111b95 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -21,30 +21,20 @@ V4L2VideoDevice *RPiStream::dev() const void RPiStream::setExternal(bool external) { + /* Import streams cannot be external. */ + assert(!external || !importOnly_); external_ = external; } bool RPiStream::isExternal() const { - /* - * Import streams cannot be external. - * - * RAW capture is a special case where we simply copy the RAW - * buffer out of the request. All other buffer handling happens - * as if the stream is internal. - */ - return external_ && !importOnly_; -} - -bool RPiStream::isImporter() const -{ - return importOnly_; + return external_; } void RPiStream::reset() { external_ = false; - internalBuffers_.clear(); + clearBuffers(); } std::string RPiStream::name() const @@ -54,63 +44,120 @@ std::string RPiStream::name() const void RPiStream::setExternalBuffers(std::vector> *buffers) { - externalBuffers_ = buffers; + std::transform(buffers->begin(), buffers->end(), std::back_inserter(bufferList_), + [](std::unique_ptr &b) { return b.get(); }); } -const std::vector> *RPiStream::getBuffers() const +const std::vector &RPiStream::getBuffers() const { - return external_ ? externalBuffers_ : &internalBuffers_; + return bufferList_; } void RPiStream::releaseBuffers() { dev_->releaseBuffers(); - if (!external_ && !importOnly_) - internalBuffers_.clear(); + clearBuffers(); } -int RPiStream::importBuffers(unsigned int count) +int RPiStream::prepareBuffers(unsigned int count) { + int ret; + + if (!importOnly_) { + if (count) { + /* Export some frame buffers for internal use. */ + ret = dev_->exportBuffers(count, &internalBuffers_); + if (ret < 0) + return ret; + + /* Add these exported buffers to the internal/external buffer list. */ + std::transform(internalBuffers_.begin(), internalBuffers_.end(), + std::back_inserter(bufferList_), + [](std::unique_ptr &b) { return b.get(); }); + + /* Add these buffers to the queue of internal usable buffers. */ + for (auto const &buffer : internalBuffers_) + availableBuffers_.push(buffer.get()); + } + + /* We must import all internal/external exported buffers. */ + count = bufferList_.size(); + } + return dev_->importBuffers(count); } -int RPiStream::allocateBuffers(unsigned int count) +int RPiStream::queueAllBuffers() { - return dev_->allocateBuffers(count, &internalBuffers_); -} + int ret; -int RPiStream::queueBuffers() -{ if (external_) return 0; - for (auto &b : internalBuffers_) { - int ret = dev_->queueBuffer(b.get()); - if (ret) { - LOG(RPISTREAM, Error) << "Failed to queue buffers for " - << name_; + while (!availableBuffers_.empty()) { + ret = queueBuffer(availableBuffers_.front()); + if (ret < 0) return ret; - } + + availableBuffers_.pop(); } return 0; } -bool RPiStream::findFrameBuffer(FrameBuffer *buffer) const +int RPiStream::queueBuffer(FrameBuffer *buffer) +{ + /* + * A nullptr buffer implies an external stream, but no external + * buffer has been supplied. So, pick one from the availableBuffers_ + * queue. + */ + if (!buffer) { + if (availableBuffers_.empty()) { + LOG(RPISTREAM, Warning) << "No buffers available for " + << name_; + return -EINVAL; + } + + buffer = availableBuffers_.front(); + availableBuffers_.pop(); + } + + LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + << " for " << name_; + + int ret = dev_->queueBuffer(buffer); + if (ret) { + LOG(RPISTREAM, Error) << "Failed to queue buffer for " + << name_; + } + + return ret; +} + +void RPiStream::returnBuffer(FrameBuffer *buffer) { - auto start = external_ ? externalBuffers_->begin() : internalBuffers_.begin(); - auto end = external_ ? externalBuffers_->end() : internalBuffers_.end(); + availableBuffers_.push(buffer); +} +bool RPiStream::findFrameBuffer(FrameBuffer *buffer) const +{ if (importOnly_) return false; - if (std::find_if(start, end, - [buffer](std::unique_ptr const &ref) { return ref.get() == buffer; }) != end) + if (std::find(bufferList_.begin(), bufferList_.end(), buffer) != bufferList_.end()) return true; return false; } +void RPiStream::clearBuffers() +{ + availableBuffers_ = std::queue{}; + internalBuffers_.clear(); + bufferList_.clear(); +} + } /* namespace RPi */ } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index 40fff81d..dc65e5d5 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -38,21 +38,22 @@ public: V4L2VideoDevice *dev() const; void setExternal(bool external); bool isExternal() const; - bool isImporter() const; void reset(); std::string name() const; void setExternalBuffers(std::vector> *buffers); - const std::vector> *getBuffers() const; + const std::vector &getBuffers() const; void releaseBuffers(); - int importBuffers(unsigned int count); - int allocateBuffers(unsigned int count); - int queueBuffers(); + int prepareBuffers(unsigned int count); + int queueAllBuffers(); + int queueBuffer(FrameBuffer *buffer); + void returnBuffer(FrameBuffer *buffer); bool findFrameBuffer(FrameBuffer *buffer) const; private: + void clearBuffers(); /* * Indicates that this stream is active externally, i.e. the buffers - * are provided by the application. + * might be provided by (and returned to) the application. */ bool external_; /* Indicates that this stream only imports buffers, e.g. ISP input. */ @@ -61,10 +62,19 @@ private: std::string name_; /* The actual device stream. */ std::unique_ptr dev_; - /* Internally allocated framebuffers associated with this device stream. */ + /* All framebuffers associated with this device stream. */ + std::vector bufferList_; + /* + * List of frame buffer that we can use if none have been provided by + * the application for external streams. This is populated by the + * buffers exported internally. + */ + std::queue availableBuffers_; + /* + * This is a list of buffers exported internally. Need to keep this around + * as the stream needs to maintain ownership of these buffers. + */ std::vector> internalBuffers_; - /* Externally allocated framebuffers associated with this device stream. */ - std::vector> *externalBuffers_; }; /* From patchwork Wed Jul 15 14:07:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8830 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 11CA9BDB1C for ; Wed, 15 Jul 2020 14:07:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CD01460C50; Wed, 15 Jul 2020 16:07:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="jCUHMPex"; dkim-atps=neutral Received: from mail-wr1-x444.google.com (mail-wr1-x444.google.com [IPv6:2a00:1450:4864:20::444]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 61E6F60487 for ; Wed, 15 Jul 2020 16:07:22 +0200 (CEST) Received: by mail-wr1-x444.google.com with SMTP id k6so2879560wrn.3 for ; Wed, 15 Jul 2020 07:07:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ygtNeMxA39WHsaHOmt9Zi7Yr2keJJ8hfWsz1Zzbb8Qs=; b=jCUHMPex3LXWugepHSNLCymkCKIAGljCpBjOEI3s7PFI5VJkhgCtnlHXRvApTXqVS/ hM6vZdAF7VzDTY2ymGGLz5tzTZQHtM1S+wDTX/7wd6Wx9ZNY0pRhFneS6GvA3QyhuST2 ijTjpQZTstKePOs25dOngqyAT+U+F1W7dPP2N26aXXDN+SYOomsoV9dxAkbZd8kwon9m ife5tsvvIvQMQ+dzE8XwnM52+fMLEphj/PLOmYqmX8Rc7deS2gibslKiqGYEskvA231n dH3SDIQv0yunUeD2Sq+ZUPJwaX+amtbX9dR8pZdATm/04MUW56YunyAl1HoHT7icQeVA wbuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ygtNeMxA39WHsaHOmt9Zi7Yr2keJJ8hfWsz1Zzbb8Qs=; b=NKy5pDh2tK1gFLocW1ha8f4dSQIRsSqwPAgF1ZqaEqk4VbXqJ8CUeMF1zYBjj+0Cff MyF5+y9MSDeFkU8N1ItOREVWMbSs9exzcOKsi8LLgJtc6+doQqVuIOrXURWAah+ZxvkN NJTGdZ9wi6KpNdkheN4UeewImclgCDfFC5UsGZjxNZalB67gRW4Qp84qqGrkJXSn17io mXBaNd01zKsmtMIzyb85EDQk/SewYWeU+ThveV4vjuceknHquTJ3UDLjnG2nJhUvpYf7 Cao09kCU2dU874JCb0kkek/W6HZHAhTvnfuXR7nV+bSeMBqwDsZ4/kepGQ/cRRsGsd9l xBqQ== X-Gm-Message-State: AOAM532BBnIN/CJJdRu6EwUWy8PAgLgg5I7duii3/VzKUVoQBjmT3lfs 1hzibeV56JteFPBQsfq3QRnW8OcXAjY= X-Google-Smtp-Source: ABdhPJwZW42VnIJC9Y7LG8qdvsgQTWym2taewSCJzf3BtBOZk/iMUbtacWzVlFOtNMCDsadzaPmlfQ== X-Received: by 2002:a5d:6990:: with SMTP id g16mr11369540wru.131.1594822041732; Wed, 15 Jul 2020 07:07:21 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id l67sm4195339wml.13.2020.07.15.07.07.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jul 2020 07:07:20 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 15 Jul 2020 15:07:00 +0100 Message-Id: <20200715140703.566298-8-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200715140703.566298-1-naush@raspberrypi.com> References: <20200715140703.566298-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 07/10] libcamera: pipeline: raspberrypi: Fix bug in passing configuration to IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The counter was not incremented, so multiple streams would only pass the last stream config to the IPA. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index cd635eb3..b91f031c 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -856,12 +856,11 @@ int PipelineHandlerRPi::configureIPA(Camera *camera) /* Inform IPA of stream configuration and sensor controls. */ unsigned int i = 0; for (auto const &stream : data->isp_) { - if (stream.isExternal()) { - streamConfig[i] = { + if (stream.isExternal()) + streamConfig[i++] = { .pixelFormat = stream.configuration().pixelFormat, .size = stream.configuration().size }; - } } entityControls.emplace(0, data->unicam_[Unicam::Image].dev()->controls()); entityControls.emplace(1, data->isp_[Isp::Input].dev()->controls()); From patchwork Wed Jul 15 14:07:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8831 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C1B1ABD790 for ; Wed, 15 Jul 2020 14:07:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8B02D60BEA; Wed, 15 Jul 2020 16:07:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="UHcIoGol"; dkim-atps=neutral Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 21FED60BEA for ; Wed, 15 Jul 2020 16:07:24 +0200 (CEST) Received: by mail-wr1-x42a.google.com with SMTP id z15so2841996wrl.8 for ; Wed, 15 Jul 2020 07:07:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/KXVsZT/hUhQq4nWWSzSgN5H5zYfjcF3lv06viQPH5Y=; b=UHcIoGolBPRR6kx+byyOBq1IoaXSUeMIRE8CaX6s8tz6Ip0dNzisTgnPSYkcoomTPG utGZRAXwFyQj3sAZbXcZaQA3yV1aJ3pd408j6cWMKa4yyn09E4ZaxOUjfQIRc0w7jdR6 UlUQrOn8zXaTiFv+/xLp8BPIxajoF41hmkHzJfYYS62lGErLwgDDI+G8MiquJw/irJ9q 8lX4lerhPyGLk17kVTsfrG3pbSOjy04jHiw1Bo1OwmB9Hhij/GI1KnPPtlqGoFS9xTij ATpy0Xl0WMnmazGaYGsDaY7sIEQYBQcpCfZR+MisPoFPSndWv4KUEk01igXagkFUGhsF CQwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/KXVsZT/hUhQq4nWWSzSgN5H5zYfjcF3lv06viQPH5Y=; b=ahJPkauvgei37xP44A4PQHH/0IbeGibs/MMowtcJweL4L5VWqiv4X7EHnq0EPTASMu fylBhFPM9rT4Q2SggRmAfLnMhBQ3rlpasEAikwWr1AmmtxaNMN8z5M0YpoLcSqIWmi3B rykkYr1/Qd02ZuL9HpWtnZQq69qbedWozrErX/sKJmg+8fclLlgQRUC3HqTADFKCO4Wx Man7In6ADPM8auH92O5/gsI4bQAkueq9loTmRDYCaGUgLz94usMcuth2JkWdJfR1//f3 dmCqJ9SxkDAvS1iuuNrhEgtcYP7H0JT/Mx1q1Bf2qqOwIZFsESzI7z7REuC6RKw8nFdl +SJQ== X-Gm-Message-State: AOAM531GesEdd14qhF3k4S3bDOdL7ICwCdYj/WBNAkHyl81OJByQzrW3 y2J1DsJe4alIcIMix6fL0shJ2ZpnNuo= X-Google-Smtp-Source: ABdhPJx1msNgp4C44vzmcxuiOv+wszDMEml5Bjv6z+zd6jSvVOQ3Wl4KrjMFE3Wa4DUk+skUUvYSSw== X-Received: by 2002:a5d:540d:: with SMTP id g13mr10854440wrv.380.1594822043319; Wed, 15 Jul 2020 07:07:23 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id l67sm4195339wml.13.2020.07.15.07.07.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jul 2020 07:07:22 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 15 Jul 2020 15:07:01 +0100 Message-Id: <20200715140703.566298-9-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200715140703.566298-1-naush@raspberrypi.com> References: <20200715140703.566298-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 08/10] libcamera: pipeline: raspberrypi: Add more robust stream buffer logic X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add further queueing into the RPiStream object to ensure that we always follow the buffer ordering (be it internal or external) given by incoming Requests. This is essential, otherwise we risk dropping frames that are meant to be part of a Request, and can cause the pipeline to stall indefinitely. This also prevents any possibility of mismatched frame buffers going through the pipeline and out to the application. Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/rpi_stream.cpp | 69 ++++++++++++++++--- .../pipeline/raspberrypi/rpi_stream.h | 12 +++- 2 files changed, 70 insertions(+), 11 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index 73111b95..d9eea3ed 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -116,28 +116,64 @@ int RPiStream::queueBuffer(FrameBuffer *buffer) if (availableBuffers_.empty()) { LOG(RPISTREAM, Warning) << "No buffers available for " << name_; - return -EINVAL; + /* + * Note that we need to requeue an internal buffer as soon + * as one becomes available. + */ + requeueBuffers_.push(nullptr); + return -ENOMEM; } buffer = availableBuffers_.front(); availableBuffers_.pop(); } - LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() - << " for " << name_; - - int ret = dev_->queueBuffer(buffer); - if (ret) { - LOG(RPISTREAM, Error) << "Failed to queue buffer for " - << name_; + if (requeueBuffers_.empty()) { + /* + * No earlier requests are pending to be queued, so we can + * go ahead and queue the buffer into the device. + */ + return queueToDevice(buffer); + } else { + /* + * There are earlier buffers to be queued, so this buffer + * must go on the waiting list. + */ + requeueBuffers_.push(buffer); + return 0; } - - return ret; } void RPiStream::returnBuffer(FrameBuffer *buffer) { + /* Push this buffer back into the queue to be used again. */ availableBuffers_.push(buffer); + + /* + * Do we have any buffers that are waiting to be queued? + * If so, do it now as availableBuffers_ will not be empty. + */ + while (!requeueBuffers_.empty()) { + FrameBuffer *buffer = requeueBuffers_.front(); + requeueBuffers_.pop(); + + if (!buffer && !availableBuffers_.empty()) { + /* + * We want to queue an internal buffer, and at + * least one is available. + */ + buffer = availableBuffers_.front(); + availableBuffers_.pop(); + } else if (!buffer && availableBuffers_.empty()) { + /* + * We want to queue an internal buffer, but none + * are available. Can't do anything, quit the loop. + */ + break; + } + + queueToDevice(buffer); + } } bool RPiStream::findFrameBuffer(FrameBuffer *buffer) const @@ -154,10 +190,23 @@ bool RPiStream::findFrameBuffer(FrameBuffer *buffer) const void RPiStream::clearBuffers() { availableBuffers_ = std::queue{}; + requeueBuffers_ = std::queue{}; internalBuffers_.clear(); bufferList_.clear(); } +int RPiStream::queueToDevice(FrameBuffer *buffer) +{ + LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + << " for " << name_; + + int ret = dev_->queueBuffer(buffer); + if (ret) + LOG(RPISTREAM, Error) << "Failed to queue buffer for " + << name_; + return ret; +} + } /* namespace RPi */ } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index dc65e5d5..0e58d261 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -51,6 +51,7 @@ public: private: void clearBuffers(); + int queueToDevice(FrameBuffer *buffer); /* * Indicates that this stream is active externally, i.e. the buffers * might be provided by (and returned to) the application. @@ -62,7 +63,7 @@ private: std::string name_; /* The actual device stream. */ std::unique_ptr dev_; - /* All framebuffers associated with this device stream. */ + /* All frame buffers associated with this device stream. */ std::vector bufferList_; /* * List of frame buffer that we can use if none have been provided by @@ -70,6 +71,15 @@ private: * buffers exported internally. */ std::queue availableBuffers_; + /* + * List of frame buffer that are to be re-queued into the device. + * A nullptr indicates any internal buffer can be used (from availableBuffers_), + * whereas a valid pointer indicates an external buffer to be queued. + * + * Ordering buffers to be queued is important here as it must match the + * requests coming from the application. + */ + std::queue requeueBuffers_; /* * This is a list of buffers exported internally. Need to keep this around * as the stream needs to maintain ownership of these buffers. From patchwork Wed Jul 15 14:07:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8832 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 28D61BD790 for ; Wed, 15 Jul 2020 14:07:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E2D28609C7; Wed, 15 Jul 2020 16:07:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="q/I+J3dN"; dkim-atps=neutral Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B0BA860BF0 for ; Wed, 15 Jul 2020 16:07:25 +0200 (CEST) Received: by mail-wm1-x32b.google.com with SMTP id o2so5994906wmh.2 for ; Wed, 15 Jul 2020 07:07:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=89XDUmOOweY/74VwbWfQnqVU0d55kKtBU9X4qHiUPMI=; b=q/I+J3dN5YyiBlXKWGqlWoitpIZT0foo1xy15JcvAoZxzBjTwY0ZhZpCEQ+fT1Ea14 qu8U1rakU6b92fCVJuWRIfNV6yHxIOq9oNKRFUwMw7adxteajFfKpeTZ0fD0102TpzMe WdBowN7zhK7Wl+iEJpwZu46OI8y9/yVlCm7mqeO8IdNzk+IVlojtBxhOkO/3EGWpfXuu MqMvJbP6kCvE23JwhcSYq1yWKHsOejYbBtkMlOsEolrcT/Wz6Xjr352Zkoil0fZsf0eQ xa/DlL1c8iAJVB9HiGn5164iIWgYKqb30E0kvesg0KECg2lauYftgzg92bHdi4DPSBM8 jP9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=89XDUmOOweY/74VwbWfQnqVU0d55kKtBU9X4qHiUPMI=; b=XIvpwSNIBHBJ8FPsuLWqlxiG1s1YH1p2H1l2ZckdL/7/Y/Ru7KCzq92cNAIMKyPIU2 t9OPrfhFCVqPNDR0cRwA7kT5DIDoHHiw5N3RzkCDgL6Lov3kvOh1p0jBBMDcPn81xRta GUmMhsCKvcUGHNspfdRGuB7svW6YltGBvDcNTkjtcC9YqNKpn0MK7in71cFxef4grH0+ 7k0R1kbijJAZ4dyiq0cSCY7UnaotGka/vmuN0uNCUXjq4j6ziQcwYhrL2O00dFOM63W2 6BDSmi3b9L2FXUdvzR0qXDc2OhUOa9/rHgn++hUspMSvn0k1NLafhLTOaxZBhIb3isEV XmlA== X-Gm-Message-State: AOAM532BxelV6csdzFkm7UwSCpR9ZUKx88UJsm6S8YKhY2QRono6qL/4 CFba1yqBZ1zasaUrSo+bRGkwD42Aqpc= X-Google-Smtp-Source: ABdhPJxOEvVcWrSVgHqPz4wWeEOm+mOc05Cc/doaGWa+MMk86yJFzBDN2n51bUOVR535nJtBVqITLA== X-Received: by 2002:a1c:80d3:: with SMTP id b202mr9139053wmd.111.1594822044814; Wed, 15 Jul 2020 07:07:24 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id l67sm4195339wml.13.2020.07.15.07.07.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jul 2020 07:07:23 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 15 Jul 2020 15:07:02 +0100 Message-Id: <20200715140703.566298-10-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200715140703.566298-1-naush@raspberrypi.com> References: <20200715140703.566298-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 09/10] libcamera: pipeline: ipa: raspberrypi: Remove use of FrameBuffer cookie X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The FrameBuffer cookie may be set by the application, so this cannot be set by the pipeline handler as well. Revert to using a simple index into the buffer list to identify buffers passing to and from the IPA. Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 60 ++++++++++--------- .../pipeline/raspberrypi/rpi_stream.cpp | 14 +++-- .../pipeline/raspberrypi/rpi_stream.h | 2 +- 3 files changed, 40 insertions(+), 36 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index b91f031c..fc6fbfc3 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -919,7 +919,8 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera) int PipelineHandlerRPi::prepareBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera); - int count, ret; + unsigned int index; + int ret; /* * Decide how many internal buffers to allocate. For now, simply look @@ -939,30 +940,24 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) } /* - * Add cookies to the ISP Input buffers so that we can link them with - * the IPA and RPI_IPA_EVENT_SIGNAL_ISP_PREPARE event. - */ - count = 0; - for (auto const &b : data->unicam_[Unicam::Image].getBuffers()) { - b->setCookie(count++); - } - - /* - * Add cookies to the stats and embedded data buffers and link them with - * the IPA. + * Link the FrameBuffers with the index of their position in the vector + * stored in the RPi stream object. + * + * This will allow us to identify buffers passed between the pipeline + * handler and the IPA. */ - count = 0; + index = 0; for (auto const &b : data->isp_[Isp::Stats].getBuffers()) { - b->setCookie(count++); - data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | b->cookie(), + data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | index, .planes = b->planes() }); + index++; } - count = 0; + index = 0; for (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) { - b->setCookie(count++); - data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | b->cookie(), + data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | index, .planes = b->planes() }); + index++; } data->ipa_->mapBuffers(data->ipaBuffers_); @@ -1090,7 +1085,7 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData unsigned int bufferId = action.data[0]; FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers().at(bufferId); - LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << buffer->cookie() + LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << bufferId << ", timestamp: " << buffer->metadata().timestamp; isp_[Isp::Input].queueBuffer(buffer); @@ -1110,12 +1105,14 @@ done: void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) { RPi::RPiStream *stream = nullptr; + int index; if (state_ == State::Stopped) return; for (RPi::RPiStream &s : unicam_) { - if (s.findFrameBuffer(buffer)) { + index = s.getBufferIndex(buffer); + if (index != -1) { stream = &s; break; } @@ -1125,7 +1122,7 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) ASSERT(stream); LOG(RPI, Debug) << "Stream " << stream->name() << " buffer dequeue" - << ", buffer id " << buffer->cookie() + << ", buffer id " << index << ", timestamp: " << buffer->metadata().timestamp; if (stream == &unicam_[Unicam::Image]) { @@ -1165,7 +1162,7 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) return; LOG(RPI, Debug) << "Stream ISP Input buffer complete" - << ", buffer id " << buffer->cookie() + << ", buffer id " << unicam_[Unicam::Image].getBufferIndex(buffer) << ", timestamp: " << buffer->metadata().timestamp; /* The ISP input buffer gets re-queued into Unicam. */ @@ -1176,12 +1173,14 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) { RPi::RPiStream *stream = nullptr; + int index; if (state_ == State::Stopped) return; for (RPi::RPiStream &s : isp_) { - if (s.findFrameBuffer(buffer)) { + index = s.getBufferIndex(buffer); + if (index != -1) { stream = &s; break; } @@ -1191,7 +1190,7 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) ASSERT(stream); LOG(RPI, Debug) << "Stream " << stream->name() << " buffer complete" - << ", buffer id " << buffer->cookie() + << ", buffer id " << index << ", timestamp: " << buffer->metadata().timestamp; /* @@ -1201,7 +1200,7 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) if (stream == &isp_[Isp::Stats]) { IPAOperationData op; op.operation = RPI_IPA_EVENT_SIGNAL_STAT_READY; - op.data = { RPiIpaMask::STATS | buffer->cookie() }; + op.data = { RPiIpaMask::STATS | static_cast(index) }; ipa_->processEvent(op); } else { /* Any other ISP output can be handed back to the application now. */ @@ -1420,13 +1419,16 @@ void RPiCameraData::tryRunPipeline() /* Set our state to say the pipeline is active. */ state_ = State::Busy; + unsigned int bayerIndex = unicam_[Unicam::Image].getBufferIndex(bayerBuffer); + unsigned int embeddedIndex = unicam_[Unicam::Embedded].getBufferIndex(embeddedBuffer); + LOG(RPI, Debug) << "Signalling RPI_IPA_EVENT_SIGNAL_ISP_PREPARE:" - << " Bayer buffer id: " << bayerBuffer->cookie() - << " Embedded buffer id: " << embeddedBuffer->cookie(); + << " Bayer buffer id: " << bayerIndex + << " Embedded buffer id: " << embeddedIndex; op.operation = RPI_IPA_EVENT_SIGNAL_ISP_PREPARE; - op.data = { RPiIpaMask::EMBEDDED_DATA | embeddedBuffer->cookie(), - RPiIpaMask::BAYER_DATA | bayerBuffer->cookie() }; + op.data = { RPiIpaMask::EMBEDDED_DATA | embeddedIndex, + RPiIpaMask::BAYER_DATA | bayerIndex }; ipa_->processEvent(op); } diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index d9eea3ed..97f87ad7 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -176,15 +176,17 @@ void RPiStream::returnBuffer(FrameBuffer *buffer) } } -bool RPiStream::findFrameBuffer(FrameBuffer *buffer) const +int RPiStream::getBufferIndex(FrameBuffer *buffer) const { if (importOnly_) - return false; + return -1; - if (std::find(bufferList_.begin(), bufferList_.end(), buffer) != bufferList_.end()) - return true; + /* Find the buffer in the list, and return the index position. */ + auto it = std::find(bufferList_.begin(), bufferList_.end(), buffer); + if (it != bufferList_.end()) + return std::distance(bufferList_.begin(), it); - return false; + return -1; } void RPiStream::clearBuffers() @@ -197,7 +199,7 @@ void RPiStream::clearBuffers() int RPiStream::queueToDevice(FrameBuffer *buffer) { - LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + LOG(RPISTREAM, Debug) << "Queuing buffer " << getBufferIndex(buffer) << " for " << name_; int ret = dev_->queueBuffer(buffer); diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index 0e58d261..16b90fac 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -47,7 +47,7 @@ public: int queueAllBuffers(); int queueBuffer(FrameBuffer *buffer); void returnBuffer(FrameBuffer *buffer); - bool findFrameBuffer(FrameBuffer *buffer) const; + int getBufferIndex(FrameBuffer *buffer) const; private: void clearBuffers(); From patchwork Wed Jul 15 14:07:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8833 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8962DBD790 for ; Wed, 15 Jul 2020 14:07:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4D52F60BF0; Wed, 15 Jul 2020 16:07:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Rm1m7xWy"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7BBCF60BEA for ; Wed, 15 Jul 2020 16:07:27 +0200 (CEST) Received: by mail-wm1-x336.google.com with SMTP id l2so6002744wmf.0 for ; Wed, 15 Jul 2020 07:07:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6zW1E+HopE1BSrWRdHQY+/uzS0skXS5tqcgLuihqK6M=; b=Rm1m7xWycW/1y6hTb//8eN8NvRATLZAcq0b+Fp1j6tgDuwGiIkdGIEvx9L8MHATfpi P3t+gt+YJBmCoANBAFE2vKtXyK3iscddPEJQTBeYkL3j4eOenY7N8ePFdp3LdN9xTCIs jhXvE6PvCcCkjPy1yxL1XoIAerhK0YWVgG+maOtR0q7w3o1HDnMIhZ2jopZ47WxfpjX2 S1c6a/KVI76fnIuJi+QroO/bdajpljoJw4VleDlNB0FiYl6p/5+5DT0jsgZgG1SP8VSi bAcTvuZqnRrsk6/TyXtJIhYqzUuJjlfDpsNPwXfPsL/Grpbp7zb7k8HJesJoV49i2cvS O+sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6zW1E+HopE1BSrWRdHQY+/uzS0skXS5tqcgLuihqK6M=; b=dITEiWQXZRYfj2Tk7V/U3jwYxwVnAb8DcjolEuT98x+FVZ6YfWB+4aGrEskFLPuQ/G EKwMu7YkIW3jdzX2TNL+bgOwxXFID3/12vD1NzuqNNvmIzP0VqVEXiLD7PeIAuJFTmKN Pf3YzO7lGR5IRVh7PbjJo9nP2G5aU0PDqqCLKF+A4cmhlnVBInbzgnHgKYJ9n9c6d/Ye PabuR2QqRAQNNI/Z8NVzVoBjnqJY/qohpTlfQd2+g3ojsWVkUdV2M67/A56O2nazGk35 nKyjfmAOQ0DU2+TePO9RD30H9NxTJLnRZwquU58v4/G4VGx9R1yrxh9Ec7N5wZbCyThi gOuQ== X-Gm-Message-State: AOAM532DPKMK3I73vps6qwZb5uXyKnvZDWm2UCg7ZuxSoypHSdu7+EOy 8rm19WQit6xQw5cjfHKkKc19MocEUSE= X-Google-Smtp-Source: ABdhPJyOCV2s9wkKE/tLHILo1yMThiQwPlSHTv0dKtUtARUja/GuGl4UMELH32wE2nCRR5ExMPaXNg== X-Received: by 2002:a1c:2905:: with SMTP id p5mr9318955wmp.91.1594822046632; Wed, 15 Jul 2020 07:07:26 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id l67sm4195339wml.13.2020.07.15.07.07.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jul 2020 07:07:25 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 15 Jul 2020 15:07:03 +0100 Message-Id: <20200715140703.566298-11-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200715140703.566298-1-naush@raspberrypi.com> References: <20200715140703.566298-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 10/10] libcamera: pipeline: raspberrypi: Handle any externally allocated FrameBuffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Handle the case where a FrameBuffer that has been externally allocated (i.e. not through the v4l2 video device) is passed into a Request. We must store the buffer pointer in our internal buffer list to identify when used, as well as mmap the buffer in the IPA if needed. Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 90 +++++++++++-------- .../pipeline/raspberrypi/rpi_stream.cpp | 5 ++ .../pipeline/raspberrypi/rpi_stream.h | 1 + 3 files changed, 61 insertions(+), 35 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index fc6fbfc3..a32ec687 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -172,8 +172,8 @@ public: RPi::RPiDevice isp_; /* The vector below is just for convenience when iterating over all streams. */ std::vector streams_; - /* Buffers passed to the IPA. */ - std::vector ipaBuffers_; + /* Stores the ids of the buffers mapped in the IPA. */ + std::vector ipaBufferIds_; /* VCSM allocation helper. */ ::RPi::Vcsm vcsm_; @@ -193,7 +193,6 @@ public: std::queue bayerQueue_; std::queue embeddedQueue_; std::deque requestQueue_; - unsigned int dropFrameCount_; private: @@ -246,6 +245,8 @@ private: int queueAllBuffers(Camera *camera); int prepareBuffers(Camera *camera); void freeBuffers(Camera *camera); + void mapBuffers(Camera *camera, const std::vector &buffers, + unsigned int startId); MediaDevice *unicam_; MediaDevice *isp_; @@ -730,18 +731,36 @@ int PipelineHandlerRPi::queueRequestDevice(Camera *camera, Request *request) /* Push all buffers supplied in the Request to the respective streams. */ for (auto stream : data->streams_) { - if (stream->isExternal()) { - FrameBuffer *buffer = request->findBuffer(stream); + if (!stream->isExternal()) + continue; + + FrameBuffer *buffer = request->findBuffer(stream); + if (stream->getBufferIndex(buffer) == -1) { /* - * If we have been asked to drop frames by the IPA, passing - * in a nullptr here will queue an internally allocated buffer. - * The Request buffer will be stored in another queue to be - * queued to the device - in sync with the Request. + * This buffer is not recognised, so it must have been allocated + * outside the v4l2 device. Store it in the stream buffer list + * so we can track it. */ - int ret = stream->queueBuffer(data->dropFrameCount_ ? nullptr : buffer); - if (ret) - return ret; + stream->setExternalBuffer(buffer); + + /* Also get the IPA to mmap it if needed. */ + if (stream == &data->unicam_[Unicam::Embedded]) { + mapBuffers(camera, { buffer }, + RPiIpaMask::EMBEDDED_DATA | (stream->getBuffers().size() - 1)); + } else if (stream == &data->isp_[Isp::Stats]) { + mapBuffers(camera, { buffer }, + RPiIpaMask::STATS | (stream->getBuffers().size() - 1)); + } } + /* + * If we have been asked to drop frames by the IPA, passing + * in a nullptr here will queue an internally allocated buffer. + * The Request buffer will be stored in another queue to be + * queued to the device - in sync with the Request. + */ + int ret = stream->queueBuffer(data->dropFrameCount_ ? nullptr : buffer); + if (ret) + return ret; } /* Push the request to the back of the queue. */ @@ -919,7 +938,6 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera) int PipelineHandlerRPi::prepareBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera); - unsigned int index; int ret; /* @@ -939,42 +957,44 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) return ret; } + /* + * Pass the stats and embedded data buffers to the IPA. No other + * buffers need to be passed. + */ + mapBuffers(camera, data->isp_[Isp::Stats].getBuffers(), RPiIpaMask::STATS); + mapBuffers(camera, data->unicam_[Unicam::Embedded].getBuffers(), RPiIpaMask::EMBEDDED_DATA); + + return 0; +} + +void PipelineHandlerRPi::mapBuffers(Camera *camera, const std::vector &buffers, + unsigned int startId) +{ + RPiCameraData *data = cameraData(camera); + std::vector ipaBuffers; + unsigned int id = startId; /* * Link the FrameBuffers with the index of their position in the vector - * stored in the RPi stream object. + * stored in the RPi stream object - along with an identifer mask. * * This will allow us to identify buffers passed between the pipeline * handler and the IPA. */ - index = 0; - for (auto const &b : data->isp_[Isp::Stats].getBuffers()) { - data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | index, - .planes = b->planes() }); - index++; + for (auto const &b : buffers) { + ipaBuffers.push_back({ .id = id, .planes = b->planes() }); + data->ipaBufferIds_.push_back(id); + id++; } - index = 0; - for (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) { - data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | index, - .planes = b->planes() }); - index++; - } - - data->ipa_->mapBuffers(data->ipaBuffers_); - - return 0; + data->ipa_->mapBuffers(ipaBuffers); } void PipelineHandlerRPi::freeBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera); - std::vector ids; - for (IPABuffer &ipabuf : data->ipaBuffers_) - ids.push_back(ipabuf.id); - - data->ipa_->unmapBuffers(ids); - data->ipaBuffers_.clear(); + data->ipa_->unmapBuffers(data->ipaBufferIds_); + data->ipaBufferIds_.clear(); for (auto const stream : data->streams_) stream->releaseBuffers(); diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index 97f87ad7..862a094a 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -48,6 +48,11 @@ void RPiStream::setExternalBuffers(std::vector> *bu [](std::unique_ptr &b) { return b.get(); }); } +void RPiStream::setExternalBuffer(FrameBuffer *buffer) +{ + bufferList_.push_back(buffer); +} + const std::vector &RPiStream::getBuffers() const { return bufferList_; diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index 16b90fac..266441df 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -41,6 +41,7 @@ public: void reset(); std::string name() const; void setExternalBuffers(std::vector> *buffers); + void setExternalBuffer(FrameBuffer *buffer); const std::vector &getBuffers() const; void releaseBuffers(); int prepareBuffers(unsigned int count);