From patchwork Wed Jul 15 14:07:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 8833 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8962DBD790 for ; Wed, 15 Jul 2020 14:07:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4D52F60BF0; Wed, 15 Jul 2020 16:07:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Rm1m7xWy"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7BBCF60BEA for ; Wed, 15 Jul 2020 16:07:27 +0200 (CEST) Received: by mail-wm1-x336.google.com with SMTP id l2so6002744wmf.0 for ; Wed, 15 Jul 2020 07:07:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6zW1E+HopE1BSrWRdHQY+/uzS0skXS5tqcgLuihqK6M=; b=Rm1m7xWycW/1y6hTb//8eN8NvRATLZAcq0b+Fp1j6tgDuwGiIkdGIEvx9L8MHATfpi P3t+gt+YJBmCoANBAFE2vKtXyK3iscddPEJQTBeYkL3j4eOenY7N8ePFdp3LdN9xTCIs jhXvE6PvCcCkjPy1yxL1XoIAerhK0YWVgG+maOtR0q7w3o1HDnMIhZ2jopZ47WxfpjX2 S1c6a/KVI76fnIuJi+QroO/bdajpljoJw4VleDlNB0FiYl6p/5+5DT0jsgZgG1SP8VSi bAcTvuZqnRrsk6/TyXtJIhYqzUuJjlfDpsNPwXfPsL/Grpbp7zb7k8HJesJoV49i2cvS O+sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6zW1E+HopE1BSrWRdHQY+/uzS0skXS5tqcgLuihqK6M=; b=dITEiWQXZRYfj2Tk7V/U3jwYxwVnAb8DcjolEuT98x+FVZ6YfWB+4aGrEskFLPuQ/G EKwMu7YkIW3jdzX2TNL+bgOwxXFID3/12vD1NzuqNNvmIzP0VqVEXiLD7PeIAuJFTmKN Pf3YzO7lGR5IRVh7PbjJo9nP2G5aU0PDqqCLKF+A4cmhlnVBInbzgnHgKYJ9n9c6d/Ye PabuR2QqRAQNNI/Z8NVzVoBjnqJY/qohpTlfQd2+g3ojsWVkUdV2M67/A56O2nazGk35 nKyjfmAOQ0DU2+TePO9RD30H9NxTJLnRZwquU58v4/G4VGx9R1yrxh9Ec7N5wZbCyThi gOuQ== X-Gm-Message-State: AOAM532DPKMK3I73vps6qwZb5uXyKnvZDWm2UCg7ZuxSoypHSdu7+EOy 8rm19WQit6xQw5cjfHKkKc19MocEUSE= X-Google-Smtp-Source: ABdhPJyOCV2s9wkKE/tLHILo1yMThiQwPlSHTv0dKtUtARUja/GuGl4UMELH32wE2nCRR5ExMPaXNg== X-Received: by 2002:a1c:2905:: with SMTP id p5mr9318955wmp.91.1594822046632; Wed, 15 Jul 2020 07:07:26 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id l67sm4195339wml.13.2020.07.15.07.07.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jul 2020 07:07:25 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 15 Jul 2020 15:07:03 +0100 Message-Id: <20200715140703.566298-11-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200715140703.566298-1-naush@raspberrypi.com> References: <20200715140703.566298-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 10/10] libcamera: pipeline: raspberrypi: Handle any externally allocated FrameBuffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Handle the case where a FrameBuffer that has been externally allocated (i.e. not through the v4l2 video device) is passed into a Request. We must store the buffer pointer in our internal buffer list to identify when used, as well as mmap the buffer in the IPA if needed. Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 90 +++++++++++-------- .../pipeline/raspberrypi/rpi_stream.cpp | 5 ++ .../pipeline/raspberrypi/rpi_stream.h | 1 + 3 files changed, 61 insertions(+), 35 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index fc6fbfc3..a32ec687 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -172,8 +172,8 @@ public: RPi::RPiDevice isp_; /* The vector below is just for convenience when iterating over all streams. */ std::vector streams_; - /* Buffers passed to the IPA. */ - std::vector ipaBuffers_; + /* Stores the ids of the buffers mapped in the IPA. */ + std::vector ipaBufferIds_; /* VCSM allocation helper. */ ::RPi::Vcsm vcsm_; @@ -193,7 +193,6 @@ public: std::queue bayerQueue_; std::queue embeddedQueue_; std::deque requestQueue_; - unsigned int dropFrameCount_; private: @@ -246,6 +245,8 @@ private: int queueAllBuffers(Camera *camera); int prepareBuffers(Camera *camera); void freeBuffers(Camera *camera); + void mapBuffers(Camera *camera, const std::vector &buffers, + unsigned int startId); MediaDevice *unicam_; MediaDevice *isp_; @@ -730,18 +731,36 @@ int PipelineHandlerRPi::queueRequestDevice(Camera *camera, Request *request) /* Push all buffers supplied in the Request to the respective streams. */ for (auto stream : data->streams_) { - if (stream->isExternal()) { - FrameBuffer *buffer = request->findBuffer(stream); + if (!stream->isExternal()) + continue; + + FrameBuffer *buffer = request->findBuffer(stream); + if (stream->getBufferIndex(buffer) == -1) { /* - * If we have been asked to drop frames by the IPA, passing - * in a nullptr here will queue an internally allocated buffer. - * The Request buffer will be stored in another queue to be - * queued to the device - in sync with the Request. + * This buffer is not recognised, so it must have been allocated + * outside the v4l2 device. Store it in the stream buffer list + * so we can track it. */ - int ret = stream->queueBuffer(data->dropFrameCount_ ? nullptr : buffer); - if (ret) - return ret; + stream->setExternalBuffer(buffer); + + /* Also get the IPA to mmap it if needed. */ + if (stream == &data->unicam_[Unicam::Embedded]) { + mapBuffers(camera, { buffer }, + RPiIpaMask::EMBEDDED_DATA | (stream->getBuffers().size() - 1)); + } else if (stream == &data->isp_[Isp::Stats]) { + mapBuffers(camera, { buffer }, + RPiIpaMask::STATS | (stream->getBuffers().size() - 1)); + } } + /* + * If we have been asked to drop frames by the IPA, passing + * in a nullptr here will queue an internally allocated buffer. + * The Request buffer will be stored in another queue to be + * queued to the device - in sync with the Request. + */ + int ret = stream->queueBuffer(data->dropFrameCount_ ? nullptr : buffer); + if (ret) + return ret; } /* Push the request to the back of the queue. */ @@ -919,7 +938,6 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera) int PipelineHandlerRPi::prepareBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera); - unsigned int index; int ret; /* @@ -939,42 +957,44 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) return ret; } + /* + * Pass the stats and embedded data buffers to the IPA. No other + * buffers need to be passed. + */ + mapBuffers(camera, data->isp_[Isp::Stats].getBuffers(), RPiIpaMask::STATS); + mapBuffers(camera, data->unicam_[Unicam::Embedded].getBuffers(), RPiIpaMask::EMBEDDED_DATA); + + return 0; +} + +void PipelineHandlerRPi::mapBuffers(Camera *camera, const std::vector &buffers, + unsigned int startId) +{ + RPiCameraData *data = cameraData(camera); + std::vector ipaBuffers; + unsigned int id = startId; /* * Link the FrameBuffers with the index of their position in the vector - * stored in the RPi stream object. + * stored in the RPi stream object - along with an identifer mask. * * This will allow us to identify buffers passed between the pipeline * handler and the IPA. */ - index = 0; - for (auto const &b : data->isp_[Isp::Stats].getBuffers()) { - data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | index, - .planes = b->planes() }); - index++; + for (auto const &b : buffers) { + ipaBuffers.push_back({ .id = id, .planes = b->planes() }); + data->ipaBufferIds_.push_back(id); + id++; } - index = 0; - for (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) { - data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | index, - .planes = b->planes() }); - index++; - } - - data->ipa_->mapBuffers(data->ipaBuffers_); - - return 0; + data->ipa_->mapBuffers(ipaBuffers); } void PipelineHandlerRPi::freeBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera); - std::vector ids; - for (IPABuffer &ipabuf : data->ipaBuffers_) - ids.push_back(ipabuf.id); - - data->ipa_->unmapBuffers(ids); - data->ipaBuffers_.clear(); + data->ipa_->unmapBuffers(data->ipaBufferIds_); + data->ipaBufferIds_.clear(); for (auto const stream : data->streams_) stream->releaseBuffers(); diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index 97f87ad7..862a094a 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -48,6 +48,11 @@ void RPiStream::setExternalBuffers(std::vector> *bu [](std::unique_ptr &b) { return b.get(); }); } +void RPiStream::setExternalBuffer(FrameBuffer *buffer) +{ + bufferList_.push_back(buffer); +} + const std::vector &RPiStream::getBuffers() const { return bufferList_; diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index 16b90fac..266441df 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -41,6 +41,7 @@ public: void reset(); std::string name() const; void setExternalBuffers(std::vector> *buffers); + void setExternalBuffer(FrameBuffer *buffer); const std::vector &getBuffers() const; void releaseBuffers(); int prepareBuffers(unsigned int count);