From patchwork Mon Jul 13 08:47:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8751 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6E2E7BD790 for ; Mon, 13 Jul 2020 08:48:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 36986605BA; Mon, 13 Jul 2020 10:48:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="UwL47plZ"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6B3926055F for ; Mon, 13 Jul 2020 10:47:59 +0200 (CEST) Received: by mail-wm1-x32c.google.com with SMTP id f18so12348445wml.3 for ; Mon, 13 Jul 2020 01:47:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4x9yj0Ay38Zs+Dv7yZpSnZKHaCbb0rxGoqp9XCIUguY=; b=UwL47plZcOpvrBtWPDYke/gAyzFPcdnNDbxUYFL+Z+Q3NdT+cdI7asTRfVeuzYiX7l ILVzxqAfp5xZpRSAG4R57F1HREGdusqtjBuZ/84HQI1IBIynXh41J1V1sKIeCw7kVP7B weQ+oPSbdbo1JkYt5gfRoBu32EQN+RNdMAY6cG2oZ2p8f4sPiYwB69s8yQeBs2W5v2CN THxKEOZ+nZueZyZNBHdSYeCJxzaUG/v3t1rfz7dDMpnseh39Hmej3xv83wxclytogCUy NQ3y3MSdlzxb3oVi3orTBYDKioACHG47GQmce8P7hBJDpomtH7UrXHLitMA638Ua6h0N 3T7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4x9yj0Ay38Zs+Dv7yZpSnZKHaCbb0rxGoqp9XCIUguY=; b=H0VGTtyXYhpCypgn23K4UBrguJQ8NkimIddr5GIN4r6bV0pHIipPtZH5WEkaKB/LNH W0nlmCFtr4wFN6fM2NKeyk06ifm/vcBT+PksbB1XZBOH0a3y2d663uY5Ol2MWkQ7B2YU VAA84OhplTDWSj9HmHcfWa8Jrk4ffOvwPl2701w2KQTZ8E8De5RNRqOA1FyJ7R3/5cJG Aw6qmEkL7BuDshjMG9odw4IfXVsM9+OxwVmeJ7yt+Dy1aw3raNCSd3Z5oqiGca7pTQYR ORFJeBh44JVsbeDJqa6tcwvjvgYxYOECds07ycm909E3naXvuSFkLkBfAbmEzjWvWgxo mo4Q== X-Gm-Message-State: AOAM533w/0auQJ+TI5KP30nhZb4u3yel2gLDfe8EwVV7sD7+n4i27NdN teMgwWqFbSSfxCVmhrIhFBvMS+JJz/M= X-Google-Smtp-Source: ABdhPJyd66fwglIztn6LJev8rDOr4654DOLwatkYT6Au0ESS2drFZPxXdtJTkNJC7XxaCQoUFYEhXg== X-Received: by 2002:a1c:238d:: with SMTP id j135mr17407050wmj.71.1594630078522; Mon, 13 Jul 2020 01:47:58 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id g14sm23203357wrm.93.2020.07.13.01.47.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2020 01:47:57 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 13 Jul 2020 09:47:20 +0100 Message-Id: <20200713084727.232422-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200713084727.232422-1-naush@raspberrypi.com> References: <20200713084727.232422-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/9] libcamera: pipeline: raspberrypi: Move RPiStream into a separate header X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Put RPiStream into the RPi namespace and add a new log category (RPISTREAM) Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 189 ++---------------- .../pipeline/raspberrypi/rpi_stream.h | 182 +++++++++++++++++ 2 files changed, 199 insertions(+), 172 deletions(-) create mode 100644 src/libcamera/pipeline/raspberrypi/rpi_stream.h diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index a08ad6a8..7bae72f9 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -17,7 +17,6 @@ #include #include #include -#include #include @@ -30,6 +29,7 @@ #include "libcamera/internal/v4l2_controls.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "rpi_stream.h" #include "staggered_ctrl.h" #include "vcsm.h" @@ -121,165 +121,10 @@ V4L2DeviceFormat findBestMode(V4L2PixFmtMap &formatsMap, const Size &req) return bestMode; } -} /* namespace */ - -/* - * Device stream abstraction for either an internal or external stream. - * Used for both Unicam and the ISP. - */ -class RPiStream : public Stream -{ -public: - RPiStream() - { - } - - RPiStream(const char *name, MediaEntity *dev, bool importOnly = false) - : external_(false), importOnly_(importOnly), name_(name), - dev_(std::make_unique(dev)) - { - } - - V4L2VideoDevice *dev() const - { - return dev_.get(); - } - - void setExternal(bool external) - { - external_ = external; - } - - bool isExternal() const - { - /* - * Import streams cannot be external. - * - * RAW capture is a special case where we simply copy the RAW - * buffer out of the request. All other buffer handling happens - * as if the stream is internal. - */ - return external_ && !importOnly_; - } - - bool isImporter() const - { - return importOnly_; - } - - void reset() - { - external_ = false; - internalBuffers_.clear(); - } - - std::string name() const - { - return name_; - } - - void setExternalBuffers(std::vector> *buffers) - { - externalBuffers_ = buffers; - } - - const std::vector> *getBuffers() const - { - return external_ ? externalBuffers_ : &internalBuffers_; - } - - void releaseBuffers() - { - dev_->releaseBuffers(); - if (!external_ && !importOnly_) - internalBuffers_.clear(); - } - - int importBuffers(unsigned int count) - { - return dev_->importBuffers(count); - } - - int allocateBuffers(unsigned int count) - { - return dev_->allocateBuffers(count, &internalBuffers_); - } - - int queueBuffers() - { - if (external_) - return 0; - - for (auto &b : internalBuffers_) { - int ret = dev_->queueBuffer(b.get()); - if (ret) { - LOG(RPI, Error) << "Failed to queue buffers for " - << name_; - return ret; - } - } - - return 0; - } - - bool findFrameBuffer(FrameBuffer *buffer) const - { - auto start = external_ ? externalBuffers_->begin() : internalBuffers_.begin(); - auto end = external_ ? externalBuffers_->end() : internalBuffers_.end(); - - if (importOnly_) - return false; - - if (std::find_if(start, end, - [buffer](std::unique_ptr const &ref) { return ref.get() == buffer; }) != end) - return true; - - return false; - } - -private: - /* - * Indicates that this stream is active externally, i.e. the buffers - * are provided by the application. - */ - bool external_; - /* Indicates that this stream only imports buffers, e.g. ISP input. */ - bool importOnly_; - /* Stream name identifier. */ - std::string name_; - /* The actual device stream. */ - std::unique_ptr dev_; - /* Internally allocated framebuffers associated with this device stream. */ - std::vector> internalBuffers_; - /* Externally allocated framebuffers associated with this device stream. */ - std::vector> *externalBuffers_; -}; - -/* - * The following class is just a convenient (and typesafe) array of device - * streams indexed with an enum class. - */ enum class Unicam : unsigned int { Image, Embedded }; enum class Isp : unsigned int { Input, Output0, Output1, Stats }; -template -class RPiDevice : public std::array -{ -private: - constexpr auto index(E e) const noexcept - { - return static_cast>(e); - } -public: - RPiStream &operator[](E e) - { - return std::array::operator[](index(e)); - } - const RPiStream &operator[](E e) const - { - return std::array::operator[](index(e)); - } -}; +} /* namespace */ class RPiCameraData : public CameraData { @@ -318,7 +163,7 @@ public: void ispOutputDequeue(FrameBuffer *buffer); void clearIncompleteRequests(); - void handleStreamBuffer(FrameBuffer *buffer, const RPiStream *stream); + void handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream); void handleState(); CameraSensor *sensor_; @@ -326,7 +171,7 @@ public: RPiDevice unicam_; RPiDevice isp_; /* The vector below is just for convenience when iterating over all streams. */ - std::vector streams_; + std::vector streams_; /* Buffers passed to the IPA. */ std::vector ipaBuffers_; @@ -782,7 +627,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) int PipelineHandlerRPi::exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) { - RPiStream *s = static_cast(stream); + RPi::RPiStream *s = static_cast(stream); unsigned int count = stream->configuration().bufferCount; int ret = s->dev()->exportBuffers(count, buffers); @@ -914,14 +759,14 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) std::unique_ptr data = std::make_unique(this); /* Locate and open the unicam video streams. */ - data->unicam_[Unicam::Embedded] = RPiStream("Unicam Embedded", unicam_->getEntityByName("unicam-embedded")); - data->unicam_[Unicam::Image] = RPiStream("Unicam Image", unicam_->getEntityByName("unicam-image")); + data->unicam_[Unicam::Embedded] = RPi::RPiStream("Unicam Embedded", unicam_->getEntityByName("unicam-embedded")); + data->unicam_[Unicam::Image] = RPi::RPiStream("Unicam Image", unicam_->getEntityByName("unicam-image")); /* Tag the ISP input stream as an import stream. */ - data->isp_[Isp::Input] = RPiStream("ISP Input", isp_->getEntityByName("bcm2835-isp0-output0"), true); - data->isp_[Isp::Output0] = RPiStream("ISP Output0", isp_->getEntityByName("bcm2835-isp0-capture1")); - data->isp_[Isp::Output1] = RPiStream("ISP Output1", isp_->getEntityByName("bcm2835-isp0-capture2")); - data->isp_[Isp::Stats] = RPiStream("ISP Stats", isp_->getEntityByName("bcm2835-isp0-capture3")); + data->isp_[Isp::Input] = RPi::RPiStream("ISP Input", isp_->getEntityByName("bcm2835-isp0-output0"), true); + data->isp_[Isp::Output0] = RPi::RPiStream("ISP Output0", isp_->getEntityByName("bcm2835-isp0-capture1")); + data->isp_[Isp::Output1] = RPi::RPiStream("ISP Output1", isp_->getEntityByName("bcm2835-isp0-capture2")); + data->isp_[Isp::Stats] = RPi::RPiStream("ISP Stats", isp_->getEntityByName("bcm2835-isp0-capture3")); /* This is just for convenience so that we can easily iterate over all streams. */ for (auto &stream : data->unicam_) @@ -1070,7 +915,7 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) */ unsigned int maxBuffers = 0; for (const Stream *s : camera->streams()) - if (static_cast(s)->isExternal()) + if (static_cast(s)->isExternal()) maxBuffers = std::max(maxBuffers, s->configuration().bufferCount); for (auto const stream : data->streams_) { @@ -1271,12 +1116,12 @@ done: void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) { - const RPiStream *stream = nullptr; + const RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPiStream const &s : unicam_) { + for (RPi::RPiStream const &s : unicam_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1332,12 +1177,12 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) { - const RPiStream *stream = nullptr; + const RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPiStream const &s : isp_) { + for (RPi::RPiStream const &s : isp_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1418,7 +1263,7 @@ void RPiCameraData::clearIncompleteRequests() } } -void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPiStream *stream) +void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream) { if (stream->isExternal()) { if (!dropFrame_) { diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h new file mode 100644 index 00000000..ed4f4987 --- /dev/null +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -0,0 +1,182 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi (Trading) Ltd. + * + * rpi_stream.h - Raspberry Pi device stream abstraction class. + */ +#ifndef __LIBCAMERA_PIPELINE_RPI_STREAM_H__ +#define __LIBCAMERA_PIPELINE_RPI_STREAM_H__ + +#include +#include +#include + +#include "libcamera/internal/v4l2_videodevice.h" +#include + +namespace libcamera { + +LOG_DEFINE_CATEGORY(RPISTREAM) + +namespace RPi { + +/* + * Device stream abstraction for either an internal or external stream. + * Used for both Unicam and the ISP. + */ +class RPiStream : public Stream +{ +public: + RPiStream() + { + } + + RPiStream(const char *name, MediaEntity *dev, bool importOnly = false) + : external_(false), importOnly_(importOnly), name_(name), + dev_(std::make_unique(dev)) + { + } + + V4L2VideoDevice *dev() const + { + return dev_.get(); + } + + void setExternal(bool external) + { + external_ = external; + } + + bool isExternal() const + { + /* + * Import streams cannot be external. + * + * RAW capture is a special case where we simply copy the RAW + * buffer out of the request. All other buffer handling happens + * as if the stream is internal. + */ + return external_ && !importOnly_; + } + + bool isImporter() const + { + return importOnly_; + } + + void reset() + { + external_ = false; + internalBuffers_.clear(); + } + + std::string name() const + { + return name_; + } + + void setExternalBuffers(std::vector> *buffers) + { + externalBuffers_ = buffers; + } + + const std::vector> *getBuffers() const + { + return external_ ? externalBuffers_ : &internalBuffers_; + } + + void releaseBuffers() + { + dev_->releaseBuffers(); + if (!external_ && !importOnly_) + internalBuffers_.clear(); + } + + int importBuffers(unsigned int count) + { + return dev_->importBuffers(count); + } + + int allocateBuffers(unsigned int count) + { + return dev_->allocateBuffers(count, &internalBuffers_); + } + + int queueBuffers() + { + if (external_) + return 0; + + for (auto &b : internalBuffers_) { + int ret = dev_->queueBuffer(b.get()); + if (ret) { + LOG(RPISTREAM, Error) << "Failed to queue buffers for " + << name_; + return ret; + } + } + + return 0; + } + + bool findFrameBuffer(FrameBuffer *buffer) const + { + auto start = external_ ? externalBuffers_->begin() : internalBuffers_.begin(); + auto end = external_ ? externalBuffers_->end() : internalBuffers_.end(); + + if (importOnly_) + return false; + + if (std::find_if(start, end, + [buffer](std::unique_ptr const &ref) { return ref.get() == buffer; }) != end) + return true; + + return false; + } + +private: + /* + * Indicates that this stream is active externally, i.e. the buffers + * are provided by the application. + */ + bool external_; + /* Indicates that this stream only imports buffers, e.g. ISP input. */ + bool importOnly_; + /* Stream name identifier. */ + std::string name_; + /* The actual device stream. */ + std::unique_ptr dev_; + /* Internally allocated framebuffers associated with this device stream. */ + std::vector> internalBuffers_; + /* Externally allocated framebuffers associated with this device stream. */ + std::vector> *externalBuffers_; +}; + +/* + * The following class is just a convenient (and typesafe) array of device + * streams indexed with an enum class. + */ +template +class RPiDevice : public std::array +{ +private: + constexpr auto index(E e) const noexcept + { + return static_cast>(e); + } +public: + RPiStream &operator[](E e) + { + return std::array::operator[](index(e)); + } + const RPiStream &operator[](E e) const + { + return std::array::operator[](index(e)); + } +}; + +} /* namespace RPi */ + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_PIPELINE_RPI_STREAM_H__ */ From patchwork Mon Jul 13 08:47:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8752 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F36D2BD790 for ; Mon, 13 Jul 2020 08:48:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BE7F6605B2; Mon, 13 Jul 2020 10:48:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="mYfyydaV"; dkim-atps=neutral Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8AE87605B9 for ; Mon, 13 Jul 2020 10:48:00 +0200 (CEST) Received: by mail-wm1-x331.google.com with SMTP id j18so12486114wmi.3 for ; Mon, 13 Jul 2020 01:48:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WG8+s91IeG7E6WymCCDki2FQYSW7oGpUVoQKd4yQDQk=; b=mYfyydaVVzIxnn0Vwi7KuvO9+9dTYmcGm0dLNjdz829SQ1faABpMk2i6HVH8Fdg4yH UKX0NAMSVk8KBZsSYFzLcrM1DxnTwXvf3gbTuJfbi8VUVsl9M4ii5ksC6MO+InAPVYCW BBeNNUUlRx8yeE3XhPSPXBwQsQkzJT/yzqnW4HkFCdY9ZXeoE55AJm3RhrSq7WEPU2Ch d2DZ/wQb2umukgv5fC5GULaRVyJ9Sdt/bNlpo6/lk2TCJIcVGzPA5SpOPzXbrHhKmRzM wZEqm6rRrAtyBtGmEU8zM8pPRS3zR4rZ8rFiYl3yDIdVOd2hERO0grfdSyxi27FrJIIF vSdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WG8+s91IeG7E6WymCCDki2FQYSW7oGpUVoQKd4yQDQk=; b=Ch7MvDB2ZyghHK1tuSezgzz4yKUUVDmqznCH5HCeY7RAE8XhC1IXHRGYu5a2QropaP SAR6pd8An6Z14DgovXKWdhwtLOpbpia5hlO2A1bGwdn7ZLV4rYunuTPdIHkaKWIglVx1 +BGLMhfaK2FatSVzxGbY5UQqYIOKwLGEa62zlMDOWTkrY2yZmLAU1sOIxB7c4J2kSXTD 5bvRAHbu4kBbjx/BaQFqIi/PGoEWpmLiwx+jv5UUikq0O5xpX5/FO+dvo97aMUyxkR5I uNAcQtbTJlF2kDXxHH+/SdUbQGN76oLTQxnXTIiPFklaurQ9hMv/hzz9ZbzRzlUmlgtE OHtw== X-Gm-Message-State: AOAM533Byjx3vb1OC0A1tWWQmS8C59XLiW5kZN+icyVT1/vVcUmLAFbu 617rUrleyhscQF4VbQ4X4cvYzyIlMhc= X-Google-Smtp-Source: ABdhPJxEMbETdpTyfRi9kP7ABf0z6CB5Zkik8BA5yUA6sm180eFsHBUhz5tvQNhBqrDCqxRkkJTs/w== X-Received: by 2002:a1c:9d96:: with SMTP id g144mr18924145wme.163.1594630079317; Mon, 13 Jul 2020 01:47:59 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id g14sm23203357wrm.93.2020.07.13.01.47.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2020 01:47:58 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 13 Jul 2020 09:47:21 +0100 Message-Id: <20200713084727.232422-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200713084727.232422-1-naush@raspberrypi.com> References: <20200713084727.232422-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/9] libcamera: pipeline: ipa: raspberrypi: Rework drop frame signalling X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The IPA now signals up front how many frames it wants the pipeline handler to drop. This makes it easier to handle up-coming changes to the buffer handling for import/export buffers. Signed-off-by: Naushir Patuck --- include/libcamera/ipa/raspberrypi.h | 2 +- src/ipa/raspberrypi/raspberrypi.cpp | 21 ++++++------ .../pipeline/raspberrypi/raspberrypi.cpp | 33 +++++++++++-------- 3 files changed, 32 insertions(+), 24 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index a18ce9a8..2dab2b00 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -15,8 +15,8 @@ enum RPiOperations { RPI_IPA_ACTION_V4L2_SET_ISP, RPI_IPA_ACTION_STATS_METADATA_COMPLETE, RPI_IPA_ACTION_RUN_ISP, - RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME, RPI_IPA_ACTION_SET_SENSOR_CONFIG, + RPI_IPA_ACTION_SET_DROP_FRAME_COUNT, RPI_IPA_ACTION_EMBEDDED_COMPLETE, RPI_IPA_EVENT_SIGNAL_STAT_READY, RPI_IPA_EVENT_SIGNAL_ISP_PREPARE, diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index b1f27861..f3568882 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -64,8 +64,8 @@ class IPARPi : public IPAInterface public: IPARPi() : lastMode_({}), controller_(), controllerInit_(false), - frame_count_(0), check_count_(0), hide_count_(0), - mistrust_count_(0), lsTableHandle_(0), lsTable_(nullptr) + frame_count_(0), check_count_(0), mistrust_count_(0), + lsTableHandle_(0), lsTable_(nullptr) { } @@ -132,8 +132,6 @@ private: uint64_t frame_count_; /* For checking the sequencing of Prepare/Process calls. */ uint64_t check_count_; - /* How many frames the pipeline handler should hide, or "drop". */ - unsigned int hide_count_; /* How many frames we should avoid running control algos on. */ unsigned int mistrust_count_; /* LS table allocation passed in from the pipeline handler. */ @@ -241,14 +239,19 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, */ frame_count_ = 0; check_count_ = 0; + int drop_frame = 0; if (controllerInit_) { - hide_count_ = helper_->HideFramesModeSwitch(); + drop_frame = helper_->HideFramesModeSwitch(); mistrust_count_ = helper_->MistrustFramesModeSwitch(); } else { - hide_count_ = helper_->HideFramesStartup(); + drop_frame = helper_->HideFramesStartup(); mistrust_count_ = helper_->MistrustFramesStartup(); } + IPAOperationData op; + op.operation = RPI_IPA_ACTION_SET_DROP_FRAME_COUNT; + op.data.push_back(drop_frame); + struct AgcStatus agcStatus; /* These zero values mean not program anything (unless overwritten). */ agcStatus.shutter_time = 0.0; @@ -339,13 +342,11 @@ void IPARPi::processEvent(const IPAOperationData &event) * they are "unreliable". */ prepareISP(embeddedbufferId); + frame_count_++; /* Ready to push the input buffer into the ISP. */ IPAOperationData op; - if (++frame_count_ > hide_count_) - op.operation = RPI_IPA_ACTION_RUN_ISP; - else - op.operation = RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME; + op.operation = RPI_IPA_ACTION_RUN_ISP; op.data = { bayerbufferId & RPiIpaMask::ID }; queueFrameAction.emit(0, op); break; diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 7bae72f9..8b6f578f 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -131,7 +131,7 @@ class RPiCameraData : public CameraData public: RPiCameraData(PipelineHandler *pipe) : CameraData(pipe), sensor_(nullptr), lsTable_(nullptr), - state_(State::Stopped), dropFrame_(false), ispOutputCount_(0) + state_(State::Stopped), dropFrameCount_(0), ispOutputCount_(0) { } @@ -168,8 +168,8 @@ public: CameraSensor *sensor_; /* Array of Unicam and ISP device streams and associated buffers/streams. */ - RPiDevice unicam_; - RPiDevice isp_; + RPi::RPiDevice unicam_; + RPi::RPiDevice isp_; /* The vector below is just for convenience when iterating over all streams. */ std::vector streams_; /* Buffers passed to the IPA. */ @@ -194,14 +194,15 @@ public: std::queue embeddedQueue_; std::deque requestQueue_; + unsigned int dropFrameCount_; + private: void checkRequestCompleted(); void tryRunPipeline(); void tryFlushQueues(); FrameBuffer *updateQueue(std::queue &q, uint64_t timestamp, V4L2VideoDevice *dev); - bool dropFrame_; - int ispOutputCount_; + unsigned int ispOutputCount_; }; class RPiCameraConfiguration : public CameraConfiguration @@ -1058,6 +1059,11 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData goto done; } + case RPI_IPA_ACTION_SET_DROP_FRAME_COUNT: { + dropFrameCount_ = action.data[0]; + goto done; + } + case RPI_IPA_ACTION_V4L2_SET_ISP: { ControlList controls = action.controls[0]; isp_[Isp::Input].dev()->setControls(&controls); @@ -1091,7 +1097,6 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData break; } - case RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME: case RPI_IPA_ACTION_RUN_ISP: { unsigned int bufferId = action.data[0]; FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers()->at(bufferId).get(); @@ -1100,7 +1105,6 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData << ", timestamp: " << buffer->metadata().timestamp; isp_[Isp::Input].dev()->queueBuffer(buffer); - dropFrame_ = (action.operation == RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME) ? true : false; ispOutputCount_ = 0; break; } @@ -1266,7 +1270,7 @@ void RPiCameraData::clearIncompleteRequests() void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream) { if (stream->isExternal()) { - if (!dropFrame_) { + if (!dropFrameCount_) { Request *request = buffer->request(); pipe_->completeBuffer(camera_, request, buffer); } @@ -1278,7 +1282,7 @@ void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream * simply memcpy to the Request buffer and requeue back to the * device. */ - if (stream == &unicam_[Unicam::Image] && !dropFrame_) { + if (stream == &unicam_[Unicam::Image] && !dropFrameCount_) { const Stream *rawStream = static_cast(&isp_[Isp::Input]); Request *request = requestQueue_.front(); FrameBuffer *raw = request->findBuffer(const_cast(rawStream)); @@ -1323,7 +1327,7 @@ void RPiCameraData::checkRequestCompleted() * If we are dropping this frame, do not touch the request, simply * change the state to IDLE when ready. */ - if (!dropFrame_) { + if (!dropFrameCount_) { Request *request = requestQueue_.front(); if (request->hasPendingBuffers()) return; @@ -1342,10 +1346,13 @@ void RPiCameraData::checkRequestCompleted() * frame. */ if (state_ == State::IpaComplete && - ((ispOutputCount_ == 3 && dropFrame_) || requestCompleted)) { + ((ispOutputCount_ == 3 && dropFrameCount_) || requestCompleted)) { state_ = State::Idle; - if (dropFrame_) - LOG(RPI, Info) << "Dropping frame at the request of the IPA"; + if (dropFrameCount_) { + dropFrameCount_--; + LOG(RPI, Info) << "Dropping frame at the request of the IPA ( " + << dropFrameCount_ << " left)"; + } } } From patchwork Mon Jul 13 08:47:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8753 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 98F77BD790 for ; Mon, 13 Jul 2020 08:48:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1EC9B605C1; Mon, 13 Jul 2020 10:48:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="ZLaYTAem"; dkim-atps=neutral Received: from mail-wr1-x443.google.com (mail-wr1-x443.google.com [IPv6:2a00:1450:4864:20::443]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0C53D605B9 for ; Mon, 13 Jul 2020 10:48:01 +0200 (CEST) Received: by mail-wr1-x443.google.com with SMTP id f2so14986035wrp.7 for ; Mon, 13 Jul 2020 01:48:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kTU4Ekw107BBgDT7sZ1t6Vbk53bJFssFJ+kUcM6X3B8=; b=ZLaYTAemSo/xfKk3VgEJrWrPfsPaSiIVlnfVQrawlb7gr6vPlmBW1QBGG6YE20jTGh ytM/tzUIwxoHJo20kIZY3k0UbXuK1cTQ8fWUazd5mynGn6OsOm3rL/mbMMCYz8/teriW md9j5N72aF4Y5VZH+UyG9Y2uX7ooLGn/qw0ODfVoqXDV+gce9PKwKgjDSUGqER/w7ayS y7g2dXhiFwCjF54XxlCXYwH9e/XyaAefcOYb6ZAT4XK86Nud8517ayBT2n713gznuxs7 m7EfceM8HlNY/9/4N3R5iCwYJIDve4kGnCobBNm3vZa5Lsu4F97eWhCESIS6Tbi6xvK7 MuDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kTU4Ekw107BBgDT7sZ1t6Vbk53bJFssFJ+kUcM6X3B8=; b=DV4k3vN3CRiK9GgvVNSTSPSHDX+tPwKe6eC7UET9DaFn+rdkDM/3B19BLoAS0lEO6y vS8+4NYFwrUVdMEq7m/baHkbJpZOSMDz/k9CvS+a4PN6yXPcskq2na0P7EqbC+esVnH6 ZVX3GrxJqg5+GYmG31UeLcecsJBNPJA62YTA30BFQd7TN2r8cY/5JIolvJ0PVVszPFSk csq2gUkXKofZ5K4XrWp8HdoJRR6wbHOcsIrmoXjGJuLk3+/nP4nG4h3VHp3AAyN8qW1z EWiKw7A7Jwxbbrm/pmHh7oL1fqUq3kQGCIZtwsngbA59CuqyTDaxBQpiMfTMMBuw2Kva 621A== X-Gm-Message-State: AOAM532cF0rHfcaH3k7ABO1KjnfPhYWnoeOHLnwbe0xId3awIT5tLQzl X9XZLXoxJfvDcjLfo0lWQExhZHF1/Ak= X-Google-Smtp-Source: ABdhPJwZy44QySL7ZtEhEmpYxkzqWpdcYpGdZMJWzp+gKsbKKQ0bEI6ifmAb93b9bmNjZyKXRLLOrg== X-Received: by 2002:a5d:5084:: with SMTP id a4mr81230998wrt.191.1594630080378; Mon, 13 Jul 2020 01:48:00 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id g14sm23203357wrm.93.2020.07.13.01.47.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2020 01:47:59 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 13 Jul 2020 09:47:22 +0100 Message-Id: <20200713084727.232422-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200713084727.232422-1-naush@raspberrypi.com> References: <20200713084727.232422-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/9] libcamera: pipeline: raspberrypi: Add some debug logging X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" No functional changes, only added some more trace points. Signed-off-by: Naushir Patuck --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 8b6f578f..65c8557d 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1175,6 +1175,11 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) if (state_ == State::Stopped) return; + LOG(RPI, Debug) << "Stream ISP Input buffer complete" + << ", buffer id " << buffer->cookie() + << ", timestamp: " << buffer->metadata().timestamp; + + /* The ISP input buffer gets re-queued into Unicam. */ handleStreamBuffer(buffer, &unicam_[Unicam::Image]); handleState(); } @@ -1339,6 +1344,8 @@ void RPiCameraData::checkRequestCompleted() pipe_->completeRequest(camera_, request); requestQueue_.pop_front(); requestCompleted = true; + + LOG(RPI, Debug) << "Request is complete"; } /* @@ -1477,7 +1484,7 @@ FrameBuffer *RPiCameraData::updateQueue(std::queue &q, uint64_t t if (b->metadata().timestamp < timestamp) { q.pop(); dev->queueBuffer(b); - LOG(RPI, Error) << "Dropping input frame!"; + LOG(RPI, Warning) << "Dropping input frame!"; } else if (b->metadata().timestamp == timestamp) { /* The calling function will pop the item from the queue. */ return b; From patchwork Mon Jul 13 08:47:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8754 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2546CBD790 for ; Mon, 13 Jul 2020 08:48:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 64655605C5; Mon, 13 Jul 2020 10:48:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="nqzM1Od1"; dkim-atps=neutral Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F1DB9605B4 for ; Mon, 13 Jul 2020 10:48:01 +0200 (CEST) Received: by mail-wr1-x42f.google.com with SMTP id f2so14986136wrp.7 for ; Mon, 13 Jul 2020 01:48:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SzxleyS2L5G8oRnbBxFxOqT8fwhmy6L0kBRX2aN6Q4c=; b=nqzM1Od11c4ASESNB68kxs4Vg0B0odL/9jCD5tkJmLygMaKIErAYGELJIrQ89JuK+O GfB4rMzp1TS6ZKiiVI05Bj3B7Zd7WMIAHCpVYgqAk9OZbApSNINm6ahnbk9rignwaUUH fbxLZIbnPf0XYkuZDNTH0LX8UAowGUfr+srM7D9tnEmqmSnempMxZmGxkkugaT2xJ5jl xgqk1bLt1h3B7bxOnc3xWG+nK5HQhVqCk+fhkSDDPFDhyFyHwoh9hIh5bVNIVDxX9Np3 4eqayAtjOE5t1/ZMdmraCwvtwjWP5ZizO82DZJ2hue3sE2wJFFoxF7qJLf3EEbF6GgIG 811g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SzxleyS2L5G8oRnbBxFxOqT8fwhmy6L0kBRX2aN6Q4c=; b=g5Vu2Cz6PpfS0PYKaocFO3n3Pp8VTe5uvxOWk3mqlOgS2uTzHFCaqZaF2Q/+jB6S5p ZQsQXirPS4xcb3tn4SE9POD1ttJYYvWvmiUxBFqFZRPP2urNGg7Wt4v+TfViOEXv/wKp hw0ztkoUVxOWAa0vcPuhCZ6FRJqhIOAgyWi3F6ScHZb6fyvZyZ5UNqplTc+ix2iADQ9x kgZlT7wkNdlreaQEPdZVBy1PXlQvixg25MHHtI6nxp5JMdz7e98WNNQcqjFaW4I5jHyR Yu3vpsBLdQwmz33V31SPx08/zDLjqpUpXboQa0Nnb2AJvsH0bX2wdoNplK5zKgiklePk GxZw== X-Gm-Message-State: AOAM530Pjs3Ur/ImyYHm82VvlnxzBMZYIe3oyn2MMNHfAixt0NmnxG+H +yEzIgz56GyySFo+XCz3hPoXfmdgvxE= X-Google-Smtp-Source: ABdhPJyJ3zGKAZ42huAv7j8IpDVtPEUKQMoozpasqZQs/v1cjwNQC5BQZooC6XQfRbcHM4GBbS7Epw== X-Received: by 2002:a5d:4a45:: with SMTP id v5mr79446110wrs.228.1594630081459; Mon, 13 Jul 2020 01:48:01 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id g14sm23203357wrm.93.2020.07.13.01.48.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2020 01:48:00 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 13 Jul 2020 09:47:23 +0100 Message-Id: <20200713084727.232422-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200713084727.232422-1-naush@raspberrypi.com> References: <20200713084727.232422-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 4/9] libcamera: pipeline: raspberrypi: Increase the number of RAW buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Increase the number of expected RAW buffers in the stream configuration to 2. This will avoid dropping Unicam frames when exporting RAW streams. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 65c8557d..a613ce65 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -394,7 +394,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, sensorFormat = findBestMode(fmts, size); pixelFormat = sensorFormat.fourcc.toPixelFormat(); ASSERT(pixelFormat.isValid()); - bufferCount = 1; + bufferCount = 2; rawCount++; break; From patchwork Mon Jul 13 08:47:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8755 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EF3E8BD790 for ; Mon, 13 Jul 2020 08:48:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BBA126055F; Mon, 13 Jul 2020 10:48:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="dLhfHdn/"; dkim-atps=neutral Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 23075605C2 for ; Mon, 13 Jul 2020 10:48:03 +0200 (CEST) Received: by mail-wm1-x32e.google.com with SMTP id 22so12491504wmg.1 for ; Mon, 13 Jul 2020 01:48:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XqHsiNLBAY2ZmnuKeHpy0MW/ly4AmD1Jxn/qN65J8x8=; b=dLhfHdn/MrkyueYAD2ydUhhIFDcagtqZLEnkDQOolzhHh3+vd/LHX33BaVp3snQc96 MCa0rT1TijVZmIzxaiufRYobpNwSxnXPjHEdYRhdaIVwOBI2V8M77W1MoUWi4lNJ+SXr ZJyL5ZC4vl6ufUYktkUqMikLbImXl4OHyYdBGUAR9YUR65VRi8Cms10UjrlqKs3OV2me OvDHl1IWaFw6jhgbGX0zGJMsgveoGS+rmOb8ke1gPl98blEjcSnyWNHTnjnMBZEL2M7d mYJZi9pQSGwdNs5VJpS+5xN3YD+x4j/PQkQRMuEyQ7UrZJrm97Yr8tkFwSeiuDQ4siaQ YgmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XqHsiNLBAY2ZmnuKeHpy0MW/ly4AmD1Jxn/qN65J8x8=; b=QxucBJRa57xFmOXT7hvpoNJSIuTj0aZ4ql49z1JzkuiU7D5grLna5semQocqhuWYWQ kpvPXyO9xW9ZFSbxRw31/GAafoHE1WBFYvwbgjKkBuXAILigxFUUyAkQW1/wfR06zn3N 0l1zJ0UBKELiLAhAdChXzHrIVdM53osOAhvpLhAH0NvTeTDpZ6cE/58Ir7PWAWtURu0Y 2YvVPGhhHS1Fcsefz8NsswMAFCgrpuusgAJ1UG11xcFNZuh00YVFxkHfBum6LpwYWyap wqtEtn0gjyCNjVgUkpr/Di2zwIbCQIs0+iovTyrvHUDBtusqwEnMLPmO6X6/rBLeCu7V aNTA== X-Gm-Message-State: AOAM530cvJbvSwzJIH3jI0PdfdVmF/4tygTLgsxJitA/isFQMqkvZHf6 O4FtnPkkz/2ez7uz2rStUy+4Ng4XWxM= X-Google-Smtp-Source: ABdhPJwfyr+7+vTejQ6MLw3KlDnD87XN3B1JxSqGUAt1KUORs78gqOZiHtb9GeMtW8wVZ84oOOG5Og== X-Received: by 2002:a05:600c:218f:: with SMTP id e15mr16636207wme.63.1594630082474; Mon, 13 Jul 2020 01:48:02 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id g14sm23203357wrm.93.2020.07.13.01.48.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2020 01:48:01 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 13 Jul 2020 09:47:24 +0100 Message-Id: <20200713084727.232422-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200713084727.232422-1-naush@raspberrypi.com> References: <20200713084727.232422-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 5/9] libcamera: pipeline: raspberrypi: Remove const qualifier from RPiStream X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For the zero-copy RAW capture feature, the RPiStream will have to be modified. Remove the const qualifier in anticipation of the future commits for this feature. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index a613ce65..2f4255a4 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -163,7 +163,7 @@ public: void ispOutputDequeue(FrameBuffer *buffer); void clearIncompleteRequests(); - void handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream); + void handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream); void handleState(); CameraSensor *sensor_; @@ -1120,12 +1120,12 @@ done: void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) { - const RPi::RPiStream *stream = nullptr; + RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPi::RPiStream const &s : unicam_) { + for (RPi::RPiStream &s : unicam_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1186,12 +1186,12 @@ void RPiCameraData::ispInputDequeue(FrameBuffer *buffer) void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) { - const RPi::RPiStream *stream = nullptr; + RPi::RPiStream *stream = nullptr; if (state_ == State::Stopped) return; - for (RPi::RPiStream const &s : isp_) { + for (RPi::RPiStream &s : isp_) { if (s.findFrameBuffer(buffer)) { stream = &s; break; @@ -1272,7 +1272,7 @@ void RPiCameraData::clearIncompleteRequests() } } -void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, const RPi::RPiStream *stream) +void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream) { if (stream->isExternal()) { if (!dropFrameCount_) { From patchwork Mon Jul 13 08:47:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8756 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6128EBDB1C for ; Mon, 13 Jul 2020 08:48:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2B18D605BD; Mon, 13 Jul 2020 10:48:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="dyz7GIuw"; dkim-atps=neutral Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D5B98605B2 for ; Mon, 13 Jul 2020 10:48:04 +0200 (CEST) Received: by mail-wr1-x430.google.com with SMTP id r12so14993621wrj.13 for ; Mon, 13 Jul 2020 01:48:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Flrmqvsok2P/t2kYp3nlCT1qzUXZizkhVcFIG6UCXWs=; b=dyz7GIuwxfXq2AZjgSQGsHq65SmCnAP1kbU7wMhSiw2gPa7CDvtRkzi3xlWV3hQPRa 1M9tKVWQ1OtVOK57g9BZZdWUxo5TS7k+BlGUbUaO/zb4a4JLCkJb6FS77sgVpB3srxgh XbwDE4DR8QzlvquG5eetmKaVifvUZf/KdjQP6BkKNzFTm1wWQggvNCaL6YSsL2llEIsi eTVTTbk73aGNY418bR9281oHt9soub8ZF/2EC+0f6vjeQN2jJ6N5/CBgHqjyfofhkNI8 6oR5cM0HYi+GGx1sen48g5VOWRzbp8rG4EgbeZPLrUGCv1MLM3/jXdcOYJpCBmIHZI0H DxDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Flrmqvsok2P/t2kYp3nlCT1qzUXZizkhVcFIG6UCXWs=; b=PXscPWLIgMY8BS0XCysgnDGcp/V0SI3s3MN0F9KWkTwah1iohB+73Zchf133zq5O2t KOeDqFjOx5XkVPsl4bNHBCLSWEnGCV65dyS4dbKUvb2jzAkR9wJd8QrTc00mdogEeCi6 YAQ4eAhCDzXnoVchIrFlDwuYf+UHo0vkeRIVxxYtcaWn/VI4V+tZJsoB8zKxrGXBFzjy 5uTq29d7MdXrneXSr55QlT7RWfs0S4WoBmpTQeyJh4vUUuLEcn+84w4k8M05Hz/NSD66 lpSS/a88NSO39Mw8PNpz+//F63B+RNhoE2IE2EwA7yP1rqM80YxqMe1YkS+3+igpbAEM Drmw== X-Gm-Message-State: AOAM530Jl3F+KJk+u6GqwXkJ17krXXMH5rsUIFR80AxaRW4wznkdEvva giasAinDreHlbuH1AMz0uUbb66Fuq2I= X-Google-Smtp-Source: ABdhPJzg4Yo5Axoxy6CEaLZza4poGevnNs7BoTLeXr5VrekpGnegjMO9ZOnzc8rVSevPPsjZfiLdUg== X-Received: by 2002:adf:9148:: with SMTP id j66mr75538756wrj.311.1594630083585; Mon, 13 Jul 2020 01:48:03 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id g14sm23203357wrm.93.2020.07.13.01.48.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2020 01:48:02 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 13 Jul 2020 09:47:25 +0100 Message-Id: <20200713084727.232422-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200713084727.232422-1-naush@raspberrypi.com> References: <20200713084727.232422-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 6/9] libcamera: pipeline: raspberrypi: Rework stream buffer logic for zero-copy X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Stop using v4l2_videodevice::allocateBuffer() for internal buffers and instead export/import all buffers. This allows the pipeline to return any stream buffer requested by the application as zero-copy. Advertise the Unicam Image stream as the RAW capture stream now. The RPiStream object now maintains a new list of buffers that are available to queue into a device. This is needed to distinguish between FrameBuffers allocated for internal use vs externally provided buffers. When a Request comes in, if a buffer is not provided for an exported stream, we re-use a buffer from this list. Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 181 ++++++++---------- .../pipeline/raspberrypi/rpi_stream.h | 136 +++++++++---- 2 files changed, 179 insertions(+), 138 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 2f4255a4..f7e16a6c 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -200,7 +200,8 @@ private: void checkRequestCompleted(); void tryRunPipeline(); void tryFlushQueues(); - FrameBuffer *updateQueue(std::queue &q, uint64_t timestamp, V4L2VideoDevice *dev); + FrameBuffer *updateQueue(std::queue &q, uint64_t timestamp, + RPi::RPiStream *stream); unsigned int ispOutputCount_; }; @@ -523,8 +524,15 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) StreamConfiguration &cfg = config->at(i); if (isRaw(cfg.pixelFormat)) { - cfg.setStream(&data->isp_[Isp::Input]); - data->isp_[Isp::Input].setExternal(true); + cfg.setStream(&data->unicam_[Unicam::Image]); + /* + * We must set both Unicam streams as external, even + * though the application may only request RAW frames. + * This is because we match timestamps on both streams + * to synchronise buffers. + */ + data->unicam_[Unicam::Image].setExternal(true); + data->unicam_[Unicam::Embedded].setExternal(true); continue; } @@ -718,14 +726,13 @@ int PipelineHandlerRPi::queueRequestDevice(Camera *camera, Request *request) if (data->state_ == RPiCameraData::State::Stopped) return -EINVAL; - /* Ensure all external streams have associated buffers! */ - for (auto &stream : data->isp_) { - if (!stream.isExternal()) - continue; + LOG(RPI, Debug) << "queueRequestDevice: New request."; - if (!request->findBuffer(&stream)) { - LOG(RPI, Error) << "Attempt to queue request with invalid stream."; - return -ENOENT; + /* Push all buffers supplied in the Request to the respective streams. */ + for (auto stream : data->streams_) { + if (stream->isExternal()) { + FrameBuffer *buffer = request->findBuffer(stream); + stream->queueBuffer(data->dropFrameCount_ ? nullptr : buffer); } } @@ -814,12 +821,10 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); - /* - * List the available output streams. - * Currently cannot do Unicam streams! - */ + /*List the available streams an application may request. */ std::set streams; - streams.insert(&data->isp_[Isp::Input]); + streams.insert(&data->unicam_[Unicam::Image]); + streams.insert(&data->unicam_[Unicam::Embedded]); streams.insert(&data->isp_[Isp::Output0]); streams.insert(&data->isp_[Isp::Output1]); streams.insert(&data->isp_[Isp::Stats]); @@ -896,7 +901,7 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera) int ret; for (auto const stream : data->streams_) { - ret = stream->queueBuffers(); + ret = stream->queueAllBuffers(); if (ret < 0) return ret; } @@ -912,7 +917,8 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) /* * Decide how many internal buffers to allocate. For now, simply look * at how many external buffers will be provided. Will need to improve - * this logic. + * this logic. However, we really must have all stream allocate the same + * number of buffers to simplify error handling in queueRequestDevice(). */ unsigned int maxBuffers = 0; for (const Stream *s : camera->streams()) @@ -920,33 +926,9 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) maxBuffers = std::max(maxBuffers, s->configuration().bufferCount); for (auto const stream : data->streams_) { - if (stream->isExternal() || stream->isImporter()) { - /* - * If a stream is marked as external reserve memory to - * prepare to import as many buffers are requested in - * the stream configuration. - * - * If a stream is an internal stream with importer - * role, reserve as many buffers as possible. - */ - unsigned int count = stream->isExternal() - ? stream->configuration().bufferCount - : maxBuffers; - ret = stream->importBuffers(count); - if (ret < 0) - return ret; - } else { - /* - * If the stream is an internal exporter allocate and - * export as many buffers as possible to its internal - * pool. - */ - ret = stream->allocateBuffers(maxBuffers); - if (ret < 0) { - freeBuffers(camera); - return ret; - } - } + ret = stream->prepareBuffers(maxBuffers); + if (ret < 0) + return ret; } /* @@ -954,7 +936,7 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * the IPA and RPI_IPA_EVENT_SIGNAL_ISP_PREPARE event. */ count = 0; - for (auto const &b : *data->unicam_[Unicam::Image].getBuffers()) { + for (auto const &b : data->unicam_[Unicam::Image].getBuffers()) { b->setCookie(count++); } @@ -963,14 +945,14 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * the IPA. */ count = 0; - for (auto const &b : *data->isp_[Isp::Stats].getBuffers()) { + for (auto const &b : data->isp_[Isp::Stats].getBuffers()) { b->setCookie(count++); data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | b->cookie(), .planes = b->planes() }); } count = 0; - for (auto const &b : *data->unicam_[Unicam::Embedded].getBuffers()) { + for (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) { b->setCookie(count++); data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | b->cookie(), .planes = b->planes() }); @@ -1081,7 +1063,7 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData switch (action.operation) { case RPI_IPA_ACTION_STATS_METADATA_COMPLETE: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = isp_[Isp::Stats].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = isp_[Isp::Stats].getBuffers().at(bufferId); handleStreamBuffer(buffer, &isp_[Isp::Stats]); /* Fill the Request metadata buffer with what the IPA has provided */ @@ -1092,19 +1074,19 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData case RPI_IPA_ACTION_EMBEDDED_COMPLETE: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = unicam_[Unicam::Embedded].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = unicam_[Unicam::Embedded].getBuffers().at(bufferId); handleStreamBuffer(buffer, &unicam_[Unicam::Embedded]); break; } case RPI_IPA_ACTION_RUN_ISP: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers().at(bufferId); LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << buffer->cookie() << ", timestamp: " << buffer->metadata().timestamp; - isp_[Isp::Input].dev()->queueBuffer(buffer); + isp_[Isp::Input].queueBuffer(buffer); ispOutputCount_ = 0; break; } @@ -1205,7 +1187,12 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) << ", buffer id " << buffer->cookie() << ", timestamp: " << buffer->metadata().timestamp; - handleStreamBuffer(buffer, stream); + /* + * ISP statistics buffer must not be re-queued or sent back to the + * application until after the IPA signals so. + */ + if (stream != &isp_[Isp::Stats]) + handleStreamBuffer(buffer, stream); /* * Increment the number of ISP outputs generated. @@ -1233,8 +1220,12 @@ void RPiCameraData::clearIncompleteRequests() */ for (auto const request : requestQueue_) { for (auto const stream : streams_) { - if (stream->isExternal()) - stream->dev()->queueBuffer(request->findBuffer(stream)); + if (!stream->isExternal()) + continue; + + FrameBuffer *buffer = request->findBuffer(stream); + if (buffer) + stream->queueBuffer(buffer); } } @@ -1263,7 +1254,7 @@ void RPiCameraData::clearIncompleteRequests() * Has the buffer already been handed back to the * request? If not, do so now. */ - if (buffer->request()) + if (buffer && buffer->request()) pipe_->completeBuffer(camera_, request, buffer); } @@ -1275,30 +1266,24 @@ void RPiCameraData::clearIncompleteRequests() void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream) { if (stream->isExternal()) { - if (!dropFrameCount_) { - Request *request = buffer->request(); + Request *request = requestQueue_.front(); + + if (!dropFrameCount_ && request->findBuffer(stream) == buffer) { + /* + * Tag the buffer as completed, returning it to the + * application. + */ pipe_->completeBuffer(camera_, request, buffer); + } else { + /* + * This buffer was not part of the Request, so we can + * recycle it. + */ + stream->returnBuffer(buffer); } } else { - /* Special handling for RAW buffer Requests. - * - * The ISP input stream is alway an import stream, but if the - * current Request has been made for a buffer on the stream, - * simply memcpy to the Request buffer and requeue back to the - * device. - */ - if (stream == &unicam_[Unicam::Image] && !dropFrameCount_) { - const Stream *rawStream = static_cast(&isp_[Isp::Input]); - Request *request = requestQueue_.front(); - FrameBuffer *raw = request->findBuffer(const_cast(rawStream)); - if (raw) { - raw->copyFrom(buffer); - pipe_->completeBuffer(camera_, request, raw); - } - } - - /* Simply requeue the buffer. */ - stream->dev()->queueBuffer(buffer); + /* Simply re-queue the buffer to the requested stream. */ + stream->queueBuffer(buffer); } } @@ -1382,7 +1367,7 @@ void RPiCameraData::tryRunPipeline() * current bayer buffer will be removed and re-queued to the driver. */ embeddedBuffer = updateQueue(embeddedQueue_, bayerBuffer->metadata().timestamp, - unicam_[Unicam::Embedded].dev()); + &unicam_[Unicam::Embedded]); if (!embeddedBuffer) { LOG(RPI, Debug) << "Could not find matching embedded buffer"; @@ -1401,7 +1386,7 @@ void RPiCameraData::tryRunPipeline() embeddedBuffer = embeddedQueue_.front(); bayerBuffer = updateQueue(bayerQueue_, embeddedBuffer->metadata().timestamp, - unicam_[Unicam::Image].dev()); + &unicam_[Unicam::Image]); if (!bayerBuffer) { LOG(RPI, Debug) << "Could not find matching bayer buffer - ending."; @@ -1409,11 +1394,7 @@ void RPiCameraData::tryRunPipeline() } } - /* - * Take the first request from the queue and action the IPA. - * Unicam buffers for the request have already been queued as they come - * in. - */ + /* Take the first request from the queue and action the IPA. */ Request *request = requestQueue_.front(); /* @@ -1425,12 +1406,6 @@ void RPiCameraData::tryRunPipeline() op.controls = { request->controls() }; ipa_->processEvent(op); - /* Queue up any ISP buffers passed into the request. */ - for (auto &stream : isp_) { - if (stream.isExternal()) - stream.dev()->queueBuffer(request->findBuffer(&stream)); - } - /* Ready to use the buffers, pop them off the queue. */ bayerQueue_.pop(); embeddedQueue_.pop(); @@ -1460,32 +1435,42 @@ void RPiCameraData::tryFlushQueues() * and give a chance for the hardware to return to lock-step. We do have * to drop all interim frames. */ - if (unicam_[Unicam::Image].getBuffers()->size() == bayerQueue_.size() && - unicam_[Unicam::Embedded].getBuffers()->size() == embeddedQueue_.size()) { + if (unicam_[Unicam::Image].getBuffers().size() == bayerQueue_.size() && + unicam_[Unicam::Embedded].getBuffers().size() == embeddedQueue_.size()) { + /* This cannot happen when Unicam streams are external. */ + assert(!unicam_[Unicam::Image].isExternal()); + LOG(RPI, Warning) << "Flushing all buffer queues!"; while (!bayerQueue_.empty()) { - unicam_[Unicam::Image].dev()->queueBuffer(bayerQueue_.front()); + unicam_[Unicam::Image].queueBuffer(bayerQueue_.front()); bayerQueue_.pop(); } while (!embeddedQueue_.empty()) { - unicam_[Unicam::Embedded].dev()->queueBuffer(embeddedQueue_.front()); + unicam_[Unicam::Embedded].queueBuffer(embeddedQueue_.front()); embeddedQueue_.pop(); } } } FrameBuffer *RPiCameraData::updateQueue(std::queue &q, uint64_t timestamp, - V4L2VideoDevice *dev) + RPi::RPiStream *stream) { + /* + * If the unicam streams are external (both have to the same), then we + * can only return out the top buffer in the queue, and assume they have + * been synced by queuing at the same time. We cannot drop these frames, + * as they may have been provided externally. + */ while (!q.empty()) { FrameBuffer *b = q.front(); - if (b->metadata().timestamp < timestamp) { + if (!stream->isExternal() && b->metadata().timestamp < timestamp) { q.pop(); - dev->queueBuffer(b); - LOG(RPI, Warning) << "Dropping input frame!"; - } else if (b->metadata().timestamp == timestamp) { + stream->queueBuffer(b); + LOG(RPI, Warning) << "Dropping unmatched input frame in stream " + << stream->name(); + } else if (stream->isExternal() || b->metadata().timestamp == timestamp) { /* The calling function will pop the item from the queue. */ return b; } else { diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index ed4f4987..a6f573fa 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -44,30 +44,20 @@ public: void setExternal(bool external) { + /* Import streams cannot be external. */ + assert(!external || !importOnly_); external_ = external; } bool isExternal() const { - /* - * Import streams cannot be external. - * - * RAW capture is a special case where we simply copy the RAW - * buffer out of the request. All other buffer handling happens - * as if the stream is internal. - */ - return external_ && !importOnly_; - } - - bool isImporter() const - { - return importOnly_; + return external_; } void reset() { external_ = false; - internalBuffers_.clear(); + clearBuffers(); } std::string name() const @@ -77,67 +67,124 @@ public: void setExternalBuffers(std::vector> *buffers) { - externalBuffers_ = buffers; + std::transform(buffers->begin(), buffers->end(), std::back_inserter(bufferList_), + [](std::unique_ptr &b) { return b.get(); }); } - const std::vector> *getBuffers() const + const std::vector &getBuffers() const { - return external_ ? externalBuffers_ : &internalBuffers_; + return bufferList_; } void releaseBuffers() { dev_->releaseBuffers(); - if (!external_ && !importOnly_) - internalBuffers_.clear(); + clearBuffers(); } - int importBuffers(unsigned int count) + int prepareBuffers(unsigned int count) { + int ret; + + if (!importOnly_) { + if (count) { + /* Export some frame buffers for internal use. */ + ret = dev_->exportBuffers(count, &internalBuffers_); + if (ret < 0) + return ret; + + /* Add these exported buffers to the internal/external buffer list. */ + std::transform(internalBuffers_.begin(), internalBuffers_.end(), + std::back_inserter(bufferList_), + [](std::unique_ptr &b) { return b.get(); }); + + /* Add these buffers to the queue of internal usable buffers. */ + for (auto const &buffer : internalBuffers_) + availableBuffers_.push(buffer.get()); + } + + /* We must import all internal/external exported buffers. */ + count = bufferList_.size(); + } + return dev_->importBuffers(count); } - int allocateBuffers(unsigned int count) + int queueAllBuffers() { - return dev_->allocateBuffers(count, &internalBuffers_); - } + int ret; - int queueBuffers() - { if (external_) return 0; - for (auto &b : internalBuffers_) { - int ret = dev_->queueBuffer(b.get()); - if (ret) { - LOG(RPISTREAM, Error) << "Failed to queue buffers for " - << name_; + while (!availableBuffers_.empty()) { + ret = queueBuffer(availableBuffers_.front()); + if (ret < 0) return ret; - } + + availableBuffers_.pop(); } return 0; } - bool findFrameBuffer(FrameBuffer *buffer) const + int queueBuffer(FrameBuffer *buffer) { - auto start = external_ ? externalBuffers_->begin() : internalBuffers_.begin(); - auto end = external_ ? externalBuffers_->end() : internalBuffers_.end(); + /* + * A nullptr buffer implies an external stream, but no external + * buffer has been supplied. So, pick one from the availableBuffers_ + * queue. + */ + if (!buffer) { + if (availableBuffers_.empty()) { + LOG(RPISTREAM, Warning) << "No buffers available for " + << name_; + return -EINVAL; + } + buffer = availableBuffers_.front(); + availableBuffers_.pop(); + } + + LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + << " for " << name_; + + int ret = dev_->queueBuffer(buffer); + if (ret) { + LOG(RPISTREAM, Error) << "Failed to queue buffer for " + << name_; + } + + return ret; + } + + void returnBuffer(FrameBuffer *buffer) + { + availableBuffers_.push(buffer); + } + + bool findFrameBuffer(FrameBuffer *buffer) const + { if (importOnly_) return false; - if (std::find_if(start, end, - [buffer](std::unique_ptr const &ref) { return ref.get() == buffer; }) != end) + if (std::find(bufferList_.begin(), bufferList_.end(), buffer) != bufferList_.end()) return true; return false; } private: + void clearBuffers() + { + availableBuffers_ = std::queue{}; + internalBuffers_.clear(); + bufferList_.clear(); + } + /* * Indicates that this stream is active externally, i.e. the buffers - * are provided by the application. + * might be provided by (and returned to) the application. */ bool external_; /* Indicates that this stream only imports buffers, e.g. ISP input. */ @@ -146,10 +193,19 @@ private: std::string name_; /* The actual device stream. */ std::unique_ptr dev_; - /* Internally allocated framebuffers associated with this device stream. */ + /* All framebuffers associated with this device stream. */ + std::vector bufferList_; + /* + * List of frame buffer that we can use if none have been provided by + * the application for external streams. This is populated by the + * buffers exported internally. + */ + std::queue availableBuffers_; + /* + * This is a list of buffers exported internally. Need to keep this around + * as the stream needs to maintain ownership of these buffers. + */ std::vector> internalBuffers_; - /* Externally allocated framebuffers associated with this device stream. */ - std::vector> *externalBuffers_; }; /* From patchwork Mon Jul 13 08:47:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8757 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C920CBD790 for ; Mon, 13 Jul 2020 08:48:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8B97C605C1; Mon, 13 Jul 2020 10:48:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="o+mLV7RT"; dkim-atps=neutral Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 167F46055F for ; Mon, 13 Jul 2020 10:48:05 +0200 (CEST) Received: by mail-wm1-x343.google.com with SMTP id q15so12483443wmj.2 for ; Mon, 13 Jul 2020 01:48:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uvNMA7md3GEsF7CgFbQDovALSUX1eX1X5TxmnWVUmYg=; b=o+mLV7RTU+w1vO00nHATSv2+cMmlc85bZccjRARF+D/qu5buVEtLrF8nu2iS6n9+Mu X4CLzITU2PMXUw6LX8hgjXE67yySzSeil2YLIFoTQ0QnN1EYzhIdDX2BG3TyhZS8ed2g MFNWI05Z9o8bwGLTWe+bX/HnEQgCnc+UprC0fyesKiLwF5sM9KsK4o5IaG9/UyuWtGJ7 oSBx52+KswEaUEdpDZTBim7ncHK8UWrVSVSYxvoHV1HaRgEU22Niaz4vvxijBejdJJik BaAaYaCplpf6ibu89Dc1hczMXuSc3qArr8FudZUWC4pd/7iTV5LWYpm++4fJRNUloZUA TTgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uvNMA7md3GEsF7CgFbQDovALSUX1eX1X5TxmnWVUmYg=; b=f5Mw9ynFrRS+xezE6j0ETPEpy33OmVvubOmqWlX/hLTKVJdWihgVuJipooto4gKyxX PKe+NfHXqKY/AyF7bSWk1TnaNZLGLzUDPrclgi9/iS2lIR9m7ssgzC59sf1zgvtIns7i 9Vl2WMwvsOaHLAVIFPIzYgsEAr5GRDXT1AoA5020vR4F4J+zWvyxdc9iC+myLejxN8BW WBGdJzFRSK/i//a7fMdh4XAYq2qQ4YmV8Ikauu6wc6nB3b0rSvIcHWtF/pGhSgqZ6B8n LTV+FtbmZMcMngqBAF518zXKs6mtpNoOeAYKPjt/yFKC+FKJOJbEFfQ3rGFSQ4QSlKJF FwfQ== X-Gm-Message-State: AOAM532ZUF1h+WGdT5/cvaHVJAbSSLkbr6hzwWUklX8b02ohPo4y11AA /VMxuebVe+gOAstRfdFRfQgfkV4a5I8= X-Google-Smtp-Source: ABdhPJyEXf2Bjcrygmrss6VyD4sq9SiduIMUBXvrh13+6Sfbl4BKH1jhmrrsEjfdOx2HwmqwpB5qKw== X-Received: by 2002:a7b:c14a:: with SMTP id z10mr17215192wmi.19.1594630084436; Mon, 13 Jul 2020 01:48:04 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id g14sm23203357wrm.93.2020.07.13.01.48.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2020 01:48:03 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 13 Jul 2020 09:47:26 +0100 Message-Id: <20200713084727.232422-8-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200713084727.232422-1-naush@raspberrypi.com> References: <20200713084727.232422-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 7/9] libcamera: pipeline: raspberrypi: Fix bug in passing configuration to IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The counter was not incremented, so multiple streams would only pass the last stream config to the IPA. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index f7e16a6c..e1a74d89 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -848,12 +848,11 @@ int PipelineHandlerRPi::configureIPA(Camera *camera) /* Inform IPA of stream configuration and sensor controls. */ unsigned int i = 0; for (auto const &stream : data->isp_) { - if (stream.isExternal()) { - streamConfig[i] = { + if (stream.isExternal()) + streamConfig[i++] = { .pixelFormat = stream.configuration().pixelFormat, .size = stream.configuration().size }; - } } entityControls.emplace(0, data->unicam_[Unicam::Image].dev()->controls()); entityControls.emplace(1, data->isp_[Isp::Input].dev()->controls()); From patchwork Mon Jul 13 08:47:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8758 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 55B09BD790 for ; Mon, 13 Jul 2020 08:48:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1F7C4605BD; Mon, 13 Jul 2020 10:48:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="pLt+a5it"; dkim-atps=neutral Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8E9EB605C4 for ; Mon, 13 Jul 2020 10:48:06 +0200 (CEST) Received: by mail-wr1-x434.google.com with SMTP id b6so15028196wrs.11 for ; Mon, 13 Jul 2020 01:48:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uWihPdm75vu/VFOlLQkZP5/xs711ROqPH6Z4lRafXUc=; b=pLt+a5itpEHAGzasMGkA/eOE5GADK/ppi/0oRIFDPWWvtlTD8NQqN9p6MioyDKyuuF m1AvXANsjvGwAHCexke32q1uf9MEmLnbezDWAaMj2LqXOIv5tpIR7ZcMLJDDduKLWGNr q5G/zwYIpoxdXKuhwpURTnrR4ofcKV9K+tiglpVo6bTjz5S6MbDA96ITtvsLMzGjB9jW /U/5aHzY0hcucSn867NY24hEc25p43Ds+RIqMe7MotrcFTKWPPzzBMQLrCSQhRbGIEoo M9CgMCKDjNqvyx3Z/DX4lVJVcOYT2cPumRAcGnQzIMwXLtVi9uWhH9mbpPQxnQc+NHA3 uiHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uWihPdm75vu/VFOlLQkZP5/xs711ROqPH6Z4lRafXUc=; b=KX4nZArJAuAv73AgiAY7b9nZNbvocaPwk1hhIcV2jiUE5ymEA1WCMNBv6znaHizy7G RSzSqKCAo3Llb9+0bKSoLbW+dUOKoSZjFxxmF3qEK9TOT6NnJl8SqNG6jxs7J3qRG8aS hU3rMxrnfU1TRe/264WUpl7ottaPZtlEw8Q5adeVL7+nsVTx1Ifar0G5z5ravuRlZCuP 9OzfFgF3X6aNgqzWiHFS8kGSaauvIHyA4+hvDE7WiNMz+6+M8CN7fLMXJObLkYPNwv/2 L7yNLfHoYhs3Ve8dA6cEx3XqQLxlxgo+1ce7TDigqq01i3+IBinmYMg8E3knK4qeSlYz hNDA== X-Gm-Message-State: AOAM532+wcLE1N3G6dcZc+aXHPye32miU7JBIO4rSJKlp+aJjjupdqGp KFBTk24HXekYQa3A3JPxg21XOPFcnUE= X-Google-Smtp-Source: ABdhPJzuELv2LVcXxczijUpD4dfjnpFTa3Vb+73Lw7KMr2raHl3v0qVKkkV4Wqw+3tX/M+PTcj/JKA== X-Received: by 2002:a5d:420b:: with SMTP id n11mr78307170wrq.91.1594630085632; Mon, 13 Jul 2020 01:48:05 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id g14sm23203357wrm.93.2020.07.13.01.48.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2020 01:48:05 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 13 Jul 2020 09:47:27 +0100 Message-Id: <20200713084727.232422-9-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200713084727.232422-1-naush@raspberrypi.com> References: <20200713084727.232422-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 8/9] libcamera: pipeline: raspberrypi: Add more robust stream buffer logic X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add further queueing into the RPiStream object to ensure that we always follow the buffer ordering (be it internal or external) given by incoming Requests. This is essential, otherwise we risk dropping frames that are meant to be part of a Request, and can cause the pipeline to stall indefinitely. This also prevents any possibility of mismatched frame buffers going through the pipeline and out to the application. Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 4 +- .../pipeline/raspberrypi/rpi_stream.h | 81 ++++++++++++++++--- 2 files changed, 73 insertions(+), 12 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index e1a74d89..2c5af05b 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -732,7 +732,9 @@ int PipelineHandlerRPi::queueRequestDevice(Camera *camera, Request *request) for (auto stream : data->streams_) { if (stream->isExternal()) { FrameBuffer *buffer = request->findBuffer(stream); - stream->queueBuffer(data->dropFrameCount_ ? nullptr : buffer); + int ret = stream->queueBuffer(data->dropFrameCount_ ? nullptr : buffer); + if (ret) + return ret; } } diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index a6f573fa..3309edfb 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -139,28 +139,77 @@ public: if (availableBuffers_.empty()) { LOG(RPISTREAM, Warning) << "No buffers available for " << name_; - return -EINVAL; + /* + * Note that we need to requeue an internal buffer as soon + * as one becomes available. + */ + requeueBuffers_.push(nullptr); + return -ENOMEM; } buffer = availableBuffers_.front(); availableBuffers_.pop(); } - LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() - << " for " << name_; - - int ret = dev_->queueBuffer(buffer); - if (ret) { - LOG(RPISTREAM, Error) << "Failed to queue buffer for " - << name_; + if (requeueBuffers_.empty()) { + /* + * No earlier requests are pending to be queued, so we can + * go ahead and queue the buffer into the device. + */ + LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + << " for " << name_; + + int ret = dev_->queueBuffer(buffer); + if (ret) + LOG(RPISTREAM, Error) << "Failed to queue buffer for " + << name_; + return ret; + } else { + /* + * There are earlier buffers to be queued, so this buffer + * must go on the waiting list. + */ + requeueBuffers_.push(buffer); + return 0; } - - return ret; } void returnBuffer(FrameBuffer *buffer) { + /* Push this buffer back into the queue to be used again. */ availableBuffers_.push(buffer); + + /* + * Do we have any buffers that are waiting to be queued? + * If so, do it now as availableBuffers_ will not be empty. + */ + while (!requeueBuffers_.empty()) { + FrameBuffer *buffer = requeueBuffers_.front(); + requeueBuffers_.pop(); + + if (!buffer && !availableBuffers_.empty()) { + /* + * We want to queue an internal buffer, and at + * least one is available. + */ + buffer = availableBuffers_.front(); + availableBuffers_.pop(); + } else if (!buffer && !availableBuffers_.empty()) { + /* + * We want to queue an internal buffer, but none + * are available. + */ + break; + } + + LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + << " for " << name_ << " from returnBuffer"; + + int ret = dev_->queueBuffer(buffer); + if (ret) + LOG(RPISTREAM, Error) << "Failed to queue buffer for " + << name_ << " from returnBuffer"; + } } bool findFrameBuffer(FrameBuffer *buffer) const @@ -178,6 +227,7 @@ private: void clearBuffers() { availableBuffers_ = std::queue{}; + requeueBuffers_ = std::queue{}; internalBuffers_.clear(); bufferList_.clear(); } @@ -193,7 +243,7 @@ private: std::string name_; /* The actual device stream. */ std::unique_ptr dev_; - /* All framebuffers associated with this device stream. */ + /* All frame buffers associated with this device stream. */ std::vector bufferList_; /* * List of frame buffer that we can use if none have been provided by @@ -201,6 +251,15 @@ private: * buffers exported internally. */ std::queue availableBuffers_; + /* + * List of frame buffer that are to be re-queued into the device. + * A nullptr indicates any internal buffer can be used (from availableBuffers_), + * whereas a valid pointer indicates an external buffer to be queued. + * + * Ordering buffers to be queued is important here as it must match the + * requests coming from the application. + */ + std::queue requeueBuffers_; /* * This is a list of buffers exported internally. Need to keep this around * as the stream needs to maintain ownership of these buffers. From patchwork Mon Jul 13 08:47:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8759 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B12BEBDB1C for ; Mon, 13 Jul 2020 08:48:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7F45A605B4; Mon, 13 Jul 2020 10:48:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="i6Rw1VbW"; dkim-atps=neutral Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 57D60605B4 for ; Mon, 13 Jul 2020 10:48:07 +0200 (CEST) Received: by mail-wm1-x32a.google.com with SMTP id q15so12483576wmj.2 for ; Mon, 13 Jul 2020 01:48:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=by0PDKAUD7j60C7WR+Teh6crYrQuh/f0Gxq6tC3Qaks=; b=i6Rw1VbWKAjivS5YZ3X240LGgVOdr8qknqaJ3goISljbo+liLMSV7eN5SeaJeG6nwa qaizeHjdlOMM54RpZnTGHE/fk2dh6FJhEkqXcWga4FUg2AzSuKUVodju4Y5v7F183giW TSBGd80naUzkWOvXz5Cy0OilKa8qNZDDWBuo85GfD2Z1QoodykV6VFsH2RYeG3417SRD 2SvrE129hqDuWFay2xe2yzUEvGPmxfuCn5p+UGcrGo/wcuJTQRJY9AQXDGVK4yJhyqWJ InJyvrkx4kAYZF4jJrokYYpbXg4LzekM0CSGu2+vEvt0SKJe5y/UYk1xaGJ0RrjpLqyj BcOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=by0PDKAUD7j60C7WR+Teh6crYrQuh/f0Gxq6tC3Qaks=; b=q5CQj8M9/q8KeGP58TkBuObNC0QMY80iP+Pq2rIrkePdDmB2TFsmtoJn9s8mvQyCEQ JMaQM6Zehn6k4AnTOUb0gZsGHbKEzSxFAMeSCgPaIDMuG9lQEuYs8Cy30D8wSpkpkYJ8 0t9nwR9MDC4eMrfiTBjP4jnTwYyavdQMYHm/truF59/BC27NxBCdqqX1JU11/xW7yR+P NATTugnwpSm2ATdTFNyoQbounVlxU4sUC2s5YY8d/KB2jn6uaZY7jQMzfCSzKYNkfKEI HgQaVs4R9CXBKN0gRgiZcPSC7oQDt2ehgEbp8jH6IR7tTE7Lew8Z+RTrWAPfDMlgyfdb eesQ== X-Gm-Message-State: AOAM531MN4GnAs/dnPLResNshwsPDrqQ7bRP5l3L0gZyaA6opN4EFJEw /yXH7CSeEgAnWz7mw2xzGPzcaLc39wQ= X-Google-Smtp-Source: ABdhPJz6M6n2CqznAgrr0LbViX/Lw4t5cOgo+P5aTw5QvBvN0aGqT8nSJim2Yupf9nfIuQAbeIx/uQ== X-Received: by 2002:a1c:bc54:: with SMTP id m81mr17599599wmf.22.1594630086451; Mon, 13 Jul 2020 01:48:06 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id g14sm23203357wrm.93.2020.07.13.01.48.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2020 01:48:05 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 13 Jul 2020 09:47:28 +0100 Message-Id: <20200713084727.232422-10-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200713084727.232422-1-naush@raspberrypi.com> References: <20200713084727.232422-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 9/9] libcamera: pipeline: raspberrypi: Move RPiStream methods to a cpp file X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Some of the RPiStream methods have grown considerably in code size. Move those functions into a cpp file to stop inlining. No other functional changes in this commit. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund --- .../pipeline/raspberrypi/meson.build | 1 + .../pipeline/raspberrypi/rpi_stream.cpp | 167 ++++++++++++++++++ .../pipeline/raspberrypi/rpi_stream.h | 155 +--------------- 3 files changed, 174 insertions(+), 149 deletions(-) create mode 100644 src/libcamera/pipeline/raspberrypi/rpi_stream.cpp diff --git a/src/libcamera/pipeline/raspberrypi/meson.build b/src/libcamera/pipeline/raspberrypi/meson.build index dcfe07c5..4cdfd8e1 100644 --- a/src/libcamera/pipeline/raspberrypi/meson.build +++ b/src/libcamera/pipeline/raspberrypi/meson.build @@ -3,4 +3,5 @@ libcamera_sources += files([ 'raspberrypi.cpp', 'staggered_ctrl.cpp', + 'rpi_stream.cpp' ]) diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp new file mode 100644 index 00000000..4eaa0686 --- /dev/null +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -0,0 +1,167 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Raspberry Pi (Trading) Ltd. + * + * rpi_stream.cpp - Raspberry Pi device stream abstraction class. + */ + +#include "rpi_stream.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(RPISTREAM) + +namespace RPi { + +int RPiStream::prepareBuffers(unsigned int count) +{ + int ret; + + if (!importOnly_) { + if (count) { + /* Export some frame buffers for internal use. */ + ret = dev_->exportBuffers(count, &internalBuffers_); + if (ret < 0) + return ret; + + /* Add these exported buffers to the internal/external buffer list. */ + std::transform(internalBuffers_.begin(), internalBuffers_.end(), + std::back_inserter(bufferList_), + [](std::unique_ptr &b) { return b.get(); }); + + /* Add these buffers to the queue of internal usable buffers. */ + for (auto const &buffer : internalBuffers_) + availableBuffers_.push(buffer.get()); + } + + /* We must import all internal/external exported buffers. */ + count = bufferList_.size(); + } + + return dev_->importBuffers(count); +} + +int RPiStream::queueAllBuffers() +{ + int ret; + + if (external_) + return 0; + + while (!availableBuffers_.empty()) { + ret = queueBuffer(availableBuffers_.front()); + if (ret < 0) + return ret; + + availableBuffers_.pop(); + } + + return 0; +} + +int RPiStream::queueBuffer(FrameBuffer *buffer) +{ + /* + * A nullptr buffer implies an external stream, but no external + * buffer has been supplied. So, pick one from the availableBuffers_ + * queue. + */ + if (!buffer) { + if (availableBuffers_.empty()) { + LOG(RPISTREAM, Warning) << "No buffers available for " + << name_; + /* + * Note that we need to requeue an internal buffer as soon + * as one becomes available. + */ + requeueBuffers_.push(nullptr); + return -ENOMEM; + } + + buffer = availableBuffers_.front(); + availableBuffers_.pop(); + } + + if (requeueBuffers_.empty()) { + /* + * No earlier requests are pending to be queued, so we can + * go ahead and queue the buffer into the device. + */ + LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + << " for " << name_; + + int ret = dev_->queueBuffer(buffer); + if (ret) + LOG(RPISTREAM, Error) << "Failed to queue buffer for " + << name_; + return ret; + } else { + /* + * There are earlier buffers to be queued, so this buffer must + * go on the waiting list. + */ + requeueBuffers_.push(buffer); + return 0; + } +} + +void RPiStream::returnBuffer(FrameBuffer *buffer) +{ + /* Push this buffer back into the queue to be used again. */ + availableBuffers_.push(buffer); + + /* + * Do we have any buffers that are waiting to be queued? + * If so, do it now as availableBuffers_ will not be empty. + */ + while (!requeueBuffers_.empty()) { + FrameBuffer *buffer = requeueBuffers_.front(); + requeueBuffers_.pop(); + + if (!buffer && !availableBuffers_.empty()) { + /* + * We want to queue an internal buffer, and at + * least one is available. + */ + buffer = availableBuffers_.front(); + availableBuffers_.pop(); + } else if (!buffer && !availableBuffers_.empty()) { + /* + * We want to queue an internal buffer, but none + * are available. + */ + break; + } + + LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + << " for " << name_ << " from returnBuffer"; + + int ret = dev_->queueBuffer(buffer); + if (ret) + LOG(RPISTREAM, Error) << "Failed to queue buffer for " + << name_ << " from returnBuffer"; + } +} + +bool RPiStream::findFrameBuffer(FrameBuffer *buffer) const +{ + if (importOnly_) + return false; + + if (std::find(bufferList_.begin(), bufferList_.end(), buffer) != bufferList_.end()) + return true; + + return false; +} + +void RPiStream::clearBuffers() +{ + availableBuffers_ = std::queue{}; + requeueBuffers_ = std::queue{}; + internalBuffers_.clear(); + bufferList_.clear(); +} + +} /* namespace RPi */ + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index 3309edfb..dbbff70b 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -16,8 +16,6 @@ namespace libcamera { -LOG_DEFINE_CATEGORY(RPISTREAM) - namespace RPi { /* @@ -82,155 +80,14 @@ public: clearBuffers(); } - int prepareBuffers(unsigned int count) - { - int ret; - - if (!importOnly_) { - if (count) { - /* Export some frame buffers for internal use. */ - ret = dev_->exportBuffers(count, &internalBuffers_); - if (ret < 0) - return ret; - - /* Add these exported buffers to the internal/external buffer list. */ - std::transform(internalBuffers_.begin(), internalBuffers_.end(), - std::back_inserter(bufferList_), - [](std::unique_ptr &b) { return b.get(); }); - - /* Add these buffers to the queue of internal usable buffers. */ - for (auto const &buffer : internalBuffers_) - availableBuffers_.push(buffer.get()); - } - - /* We must import all internal/external exported buffers. */ - count = bufferList_.size(); - } - - return dev_->importBuffers(count); - } - - int queueAllBuffers() - { - int ret; - - if (external_) - return 0; - - while (!availableBuffers_.empty()) { - ret = queueBuffer(availableBuffers_.front()); - if (ret < 0) - return ret; - - availableBuffers_.pop(); - } - - return 0; - } - - int queueBuffer(FrameBuffer *buffer) - { - /* - * A nullptr buffer implies an external stream, but no external - * buffer has been supplied. So, pick one from the availableBuffers_ - * queue. - */ - if (!buffer) { - if (availableBuffers_.empty()) { - LOG(RPISTREAM, Warning) << "No buffers available for " - << name_; - /* - * Note that we need to requeue an internal buffer as soon - * as one becomes available. - */ - requeueBuffers_.push(nullptr); - return -ENOMEM; - } - - buffer = availableBuffers_.front(); - availableBuffers_.pop(); - } - - if (requeueBuffers_.empty()) { - /* - * No earlier requests are pending to be queued, so we can - * go ahead and queue the buffer into the device. - */ - LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() - << " for " << name_; - - int ret = dev_->queueBuffer(buffer); - if (ret) - LOG(RPISTREAM, Error) << "Failed to queue buffer for " - << name_; - return ret; - } else { - /* - * There are earlier buffers to be queued, so this buffer - * must go on the waiting list. - */ - requeueBuffers_.push(buffer); - return 0; - } - } - - void returnBuffer(FrameBuffer *buffer) - { - /* Push this buffer back into the queue to be used again. */ - availableBuffers_.push(buffer); - - /* - * Do we have any buffers that are waiting to be queued? - * If so, do it now as availableBuffers_ will not be empty. - */ - while (!requeueBuffers_.empty()) { - FrameBuffer *buffer = requeueBuffers_.front(); - requeueBuffers_.pop(); - - if (!buffer && !availableBuffers_.empty()) { - /* - * We want to queue an internal buffer, and at - * least one is available. - */ - buffer = availableBuffers_.front(); - availableBuffers_.pop(); - } else if (!buffer && !availableBuffers_.empty()) { - /* - * We want to queue an internal buffer, but none - * are available. - */ - break; - } - - LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() - << " for " << name_ << " from returnBuffer"; - - int ret = dev_->queueBuffer(buffer); - if (ret) - LOG(RPISTREAM, Error) << "Failed to queue buffer for " - << name_ << " from returnBuffer"; - } - } - - bool findFrameBuffer(FrameBuffer *buffer) const - { - if (importOnly_) - return false; - - if (std::find(bufferList_.begin(), bufferList_.end(), buffer) != bufferList_.end()) - return true; - - return false; - } + int prepareBuffers(unsigned int count); + int queueAllBuffers(); + int queueBuffer(FrameBuffer *buffer); + void returnBuffer(FrameBuffer *buffer); + bool findFrameBuffer(FrameBuffer *buffer) const; private: - void clearBuffers() - { - availableBuffers_ = std::queue{}; - requeueBuffers_ = std::queue{}; - internalBuffers_.clear(); - bufferList_.clear(); - } + void clearBuffers(); /* * Indicates that this stream is active externally, i.e. the buffers