From patchwork Tue Aug 4 09:58:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9171 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 82717BD86F for ; Tue, 4 Aug 2020 09:59:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4F97B6049B; Tue, 4 Aug 2020 11:59:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Tcmg2+tI"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 892DC60495 for ; Tue, 4 Aug 2020 11:59:03 +0200 (CEST) Received: by mail-wr1-x429.google.com with SMTP id a15so36787508wrh.10 for ; Tue, 04 Aug 2020 02:59:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WyUMgdVpAn4EKQU1q0Py72yq7qPLwGImGumfb+AouDg=; b=Tcmg2+tIW7FfFbCB0OyEGJLGNUIe8oLzYZpRuecDs988ggiBHcEaHDE7N9gbBoY8F1 XbjP6WzAAgw+0J8O6dQZjZRFhGyLOwhUzcXddk4k4GkuigJ6Z6FrLSDOkYkcW1Je52Ih KKZc+6GBf4BApc5CTjG/vSW4sodd7AX7Avt1ukCAXj3oIHn/anoBjG9cFU4vTIM2klVT YHbuZbSC7OJkC7tm3vqRc9rHyEaSu0NXx3ZbT1PtB9tQJjbbsLvAMN90yDsLjvFBJfVP 5jLrEfMzCfaTDyEpXhFUXItDNt+U6N5utO7hf0AVc+uYAyAwq2QbRERISZvy1fGFy6Ze 5N8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WyUMgdVpAn4EKQU1q0Py72yq7qPLwGImGumfb+AouDg=; b=mD1iQJavLZm9H27slZXl3jMNU5Y+UHVUfowk2YHpPya8L1gHdyfxg2FWoQBhY2x2bI kXIUIiKJngmCmBI2SaeDTwgc40frIVMA6Lr9a8et1ft1dqQNFAahiIobSwEobO+TY77B x0yIKybtNjG8O8io76s+BBjEpROltPaSbWkBGNgjCG/J2Q3Zt9b67gopS2fz5nc34U01 +4rcd+QHs4utD5VxNx/uHRxH8wb/90vkch8RDfY7sJyVP6UCPqk1MkD/99344RwtrWqA WFmIuecy6C07ib9EEDJUm68RqRjGcmPejfwT1y12d83C6S/oYbVq29FLZVpn151STsvK 8AvQ== X-Gm-Message-State: AOAM532+Kc4JTynfo5nKubQY8LmHiH7jNe4U+KK8Jpl68lhZx5rn00wS C5z+a5U/IpGQyd6Hr9DIz3dUDTanXmVmEQ== X-Google-Smtp-Source: ABdhPJze8duR/zfeUbKw1Fz3GL8A4BAR9wQyyLCvreMtfvIZA+ngSb5sjc/WnhxtvOV5jWQytvHFMw== X-Received: by 2002:adf:bc13:: with SMTP id s19mr1822764wrg.412.1596535142208; Tue, 04 Aug 2020 02:59:02 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id 32sm34047234wrh.18.2020.08.04.02.59.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 02:59:01 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Aug 2020 10:58:44 +0100 Message-Id: <20200804095850.275499-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200804095850.275499-1-naush@raspberrypi.com> References: <20200804095850.275499-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 06/12] libcamera: pipeline: raspberrypi: Rework stream buffer logic for zero-copy X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Stop using v4l2_videodevice::allocateBuffer() for internal buffers and instead export/import all buffers. This allows the pipeline to return any stream buffer requested by the application as zero-copy. Advertise the Unicam Image stream as the RAW capture stream now. The RPiStream object now maintains a new list of buffers that are available to queue into a device. This is needed to distinguish between FrameBuffers allocated for internal use vs externally provided buffers. When a Request comes in, if a buffer is not provided for an exported stream, we re-use a buffer from this list. Signed-off-by: Naushir Patuck Reviewed-by: Niklas Söderlund Reviewed-by: Kieran Bingham Tested-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 231 ++++++++++-------- .../pipeline/raspberrypi/rpi_stream.cpp | 118 ++++++--- .../pipeline/raspberrypi/rpi_stream.h | 33 ++- 3 files changed, 231 insertions(+), 151 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 53a07361..d4791a16 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -186,7 +186,8 @@ private: void checkRequestCompleted(); void tryRunPipeline(); void tryFlushQueues(); - FrameBuffer *updateQueue(std::queue &q, uint64_t timestamp, V4L2VideoDevice *dev); + FrameBuffer *updateQueue(std::queue &q, uint64_t timestamp, + RPi::RPiStream *stream); unsigned int ispOutputCount_; }; @@ -507,8 +508,15 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) StreamConfiguration &cfg = config->at(i); if (isRaw(cfg.pixelFormat)) { - cfg.setStream(&data->isp_[Isp::Input]); - data->isp_[Isp::Input].setExternal(true); + cfg.setStream(&data->unicam_[Unicam::Image]); + /* + * We must set both Unicam streams as external, even + * though the application may only request RAW frames. + * This is because we match timestamps on both streams + * to synchronise buffers. + */ + data->unicam_[Unicam::Image].setExternal(true); + data->unicam_[Unicam::Embedded].setExternal(true); continue; } @@ -611,7 +619,7 @@ int PipelineHandlerRPi::exportFrameBuffers(Camera *camera, Stream *stream, unsigned int count = stream->configuration().bufferCount; int ret = s->dev()->exportBuffers(count, buffers); - s->setExternalBuffers(buffers); + s->setExportedBuffers(buffers); return ret; } @@ -625,6 +633,7 @@ int PipelineHandlerRPi::start(Camera *camera) ret = prepareBuffers(camera); if (ret) { LOG(RPI, Error) << "Failed to allocate buffers"; + stop(camera); return ret; } @@ -711,14 +720,26 @@ int PipelineHandlerRPi::queueRequestDevice(Camera *camera, Request *request) if (data->state_ == RPiCameraData::State::Stopped) return -EINVAL; - /* Ensure all external streams have associated buffers! */ - for (auto &stream : data->isp_) { - if (!stream.isExternal()) - continue; + LOG(RPI, Debug) << "queueRequestDevice: New request."; - if (!request->findBuffer(&stream)) { - LOG(RPI, Error) << "Attempt to queue request with invalid stream."; - return -ENOENT; + /* Push all buffers supplied in the Request to the respective streams. */ + for (auto stream : data->streams_) { + if (stream->isExternal()) { + FrameBuffer *buffer = request->findBuffer(stream); + /* + * If no buffer is provided by the request for this stream, we + * queue a nullptr to the stream to signify that it must use an + * internally allocated buffer for this capture request. This + * buffer will not be given back to the application, but is used + * to support the internal pipeline flow. + * + * The below queueBuffer() call will do nothing if there are not + * enough internal buffers allocated, but this will be handled by + * queuing the request for buffers in the RPiStream object. + */ + int ret = stream->queueBuffer(buffer); + if (ret) + return ret; } } @@ -808,14 +829,14 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) data->properties_ = data->sensor_->properties(); /* - * List the available output streams. - * Currently cannot do Unicam streams! + * List the available streams an application may request. At present, we + * do not advertise Unicam Embedded and ISP Statistics streams, as there + * is no mechanism for the application to request non-image buffer formats. */ std::set streams; - streams.insert(&data->isp_[Isp::Input]); + streams.insert(&data->unicam_[Unicam::Image]); streams.insert(&data->isp_[Isp::Output0]); streams.insert(&data->isp_[Isp::Output1]); - streams.insert(&data->isp_[Isp::Stats]); /* Create and register the camera. */ std::shared_ptr camera = Camera::create(this, data->sensor_->model(), streams); @@ -830,9 +851,28 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera) int ret; for (auto const stream : data->streams_) { - ret = stream->queueBuffers(); - if (ret < 0) - return ret; + if (!stream->isExternal()) { + ret = stream->queueAllBuffers(); + if (ret < 0) + return ret; + } else { + /* + * For external streams, we must queue up a set of internal + * buffers to handle the number of drop frames requested by + * the IPA. This is done by passing nullptr in queueBuffer(). + * + * The below queueBuffer() call will do nothing if there + * are not enough internal buffers allocated, but this will + * be handled by queuing the request for buffers in the + * RPiStream object. + */ + unsigned int i; + for (i = 0; i < data->dropFrameCount_; i++) { + int ret = stream->queueBuffer(nullptr); + if (ret) + return ret; + } + } } return 0; @@ -846,7 +886,8 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) /* * Decide how many internal buffers to allocate. For now, simply look * at how many external buffers will be provided. Will need to improve - * this logic. + * this logic. However, we really must have all streams allocate the same + * number of buffers to simplify error handling in queueRequestDevice(). */ unsigned int maxBuffers = 0; for (const Stream *s : camera->streams()) @@ -854,33 +895,9 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) maxBuffers = std::max(maxBuffers, s->configuration().bufferCount); for (auto const stream : data->streams_) { - if (stream->isExternal() || stream->isImporter()) { - /* - * If a stream is marked as external reserve memory to - * prepare to import as many buffers are requested in - * the stream configuration. - * - * If a stream is an internal stream with importer - * role, reserve as many buffers as possible. - */ - unsigned int count = stream->isExternal() - ? stream->configuration().bufferCount - : maxBuffers; - ret = stream->importBuffers(count); - if (ret < 0) - return ret; - } else { - /* - * If the stream is an internal exporter allocate and - * export as many buffers as possible to its internal - * pool. - */ - ret = stream->allocateBuffers(maxBuffers); - if (ret < 0) { - freeBuffers(camera); - return ret; - } - } + ret = stream->prepareBuffers(maxBuffers); + if (ret < 0) + return ret; } /* @@ -888,7 +905,7 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * the IPA and RPI_IPA_EVENT_SIGNAL_ISP_PREPARE event. */ count = 0; - for (auto const &b : *data->unicam_[Unicam::Image].getBuffers()) { + for (auto const &b : data->unicam_[Unicam::Image].getBuffers()) { b->setCookie(count++); } @@ -897,14 +914,14 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * the IPA. */ count = 0; - for (auto const &b : *data->isp_[Isp::Stats].getBuffers()) { + for (auto const &b : data->isp_[Isp::Stats].getBuffers()) { b->setCookie(count++); data->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | b->cookie(), .planes = b->planes() }); } count = 0; - for (auto const &b : *data->unicam_[Unicam::Embedded].getBuffers()) { + for (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) { b->setCookie(count++); data->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | b->cookie(), .planes = b->planes() }); @@ -1065,7 +1082,7 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData switch (action.operation) { case RPI_IPA_ACTION_STATS_METADATA_COMPLETE: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = isp_[Isp::Stats].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = isp_[Isp::Stats].getBuffers().at(bufferId); handleStreamBuffer(buffer, &isp_[Isp::Stats]); /* Fill the Request metadata buffer with what the IPA has provided */ @@ -1076,19 +1093,19 @@ void RPiCameraData::queueFrameAction(unsigned int frame, const IPAOperationData case RPI_IPA_ACTION_EMBEDDED_COMPLETE: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = unicam_[Unicam::Embedded].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = unicam_[Unicam::Embedded].getBuffers().at(bufferId); handleStreamBuffer(buffer, &unicam_[Unicam::Embedded]); break; } case RPI_IPA_ACTION_RUN_ISP: { unsigned int bufferId = action.data[0]; - FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers()->at(bufferId).get(); + FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers().at(bufferId); LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << buffer->cookie() << ", timestamp: " << buffer->metadata().timestamp; - isp_[Isp::Input].dev()->queueBuffer(buffer); + isp_[Isp::Input].queueBuffer(buffer); ispOutputCount_ = 0; break; } @@ -1189,22 +1206,26 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) << ", buffer id " << buffer->cookie() << ", timestamp: " << buffer->metadata().timestamp; - handleStreamBuffer(buffer, stream); - /* - * Increment the number of ISP outputs generated. - * This is needed to track dropped frames. + * ISP statistics buffer must not be re-queued or sent back to the + * application until after the IPA signals so. */ - ispOutputCount_++; - - /* If this is a stats output, hand it to the IPA now. */ if (stream == &isp_[Isp::Stats]) { IPAOperationData op; op.operation = RPI_IPA_EVENT_SIGNAL_STAT_READY; op.data = { RPiIpaMask::STATS | buffer->cookie() }; ipa_->processEvent(op); + } else { + /* Any other ISP output can be handed back to the application now. */ + handleStreamBuffer(buffer, stream); } + /* + * Increment the number of ISP outputs generated. + * This is needed to track dropped frames. + */ + ispOutputCount_++; + handleState(); } @@ -1217,8 +1238,12 @@ void RPiCameraData::clearIncompleteRequests() */ for (auto const request : requestQueue_) { for (auto const stream : streams_) { - if (stream->isExternal()) - stream->dev()->queueBuffer(request->findBuffer(stream)); + if (!stream->isExternal()) + continue; + + FrameBuffer *buffer = request->findBuffer(stream); + if (buffer) + stream->queueBuffer(buffer); } } @@ -1247,7 +1272,7 @@ void RPiCameraData::clearIncompleteRequests() * Has the buffer already been handed back to the * request? If not, do so now. */ - if (buffer->request()) + if (buffer && buffer->request()) pipe_->completeBuffer(camera_, request, buffer); } @@ -1259,30 +1284,24 @@ void RPiCameraData::clearIncompleteRequests() void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream) { if (stream->isExternal()) { - if (!dropFrameCount_) { - Request *request = buffer->request(); + Request *request = requestQueue_.front(); + + if (!dropFrameCount_ && request->findBuffer(stream) == buffer) { + /* + * Tag the buffer as completed, returning it to the + * application. + */ pipe_->completeBuffer(camera_, request, buffer); + } else { + /* + * This buffer was not part of the Request, so we can + * recycle it. + */ + stream->returnBuffer(buffer); } } else { - /* Special handling for RAW buffer Requests. - * - * The ISP input stream is alway an import stream, but if the - * current Request has been made for a buffer on the stream, - * simply memcpy to the Request buffer and requeue back to the - * device. - */ - if (stream == &unicam_[Unicam::Image] && !dropFrameCount_) { - const Stream *rawStream = static_cast(&isp_[Isp::Input]); - Request *request = requestQueue_.front(); - FrameBuffer *raw = request->findBuffer(const_cast(rawStream)); - if (raw) { - raw->copyFrom(buffer); - pipe_->completeBuffer(camera_, request, raw); - } - } - - /* Simply requeue the buffer. */ - stream->dev()->queueBuffer(buffer); + /* Simply re-queue the buffer to the requested stream. */ + stream->queueBuffer(buffer); } } @@ -1364,7 +1383,7 @@ void RPiCameraData::tryRunPipeline() * current bayer buffer will be removed and re-queued to the driver. */ embeddedBuffer = updateQueue(embeddedQueue_, bayerBuffer->metadata().timestamp, - unicam_[Unicam::Embedded].dev()); + &unicam_[Unicam::Embedded]); if (!embeddedBuffer) { LOG(RPI, Debug) << "Could not find matching embedded buffer"; @@ -1383,7 +1402,7 @@ void RPiCameraData::tryRunPipeline() embeddedBuffer = embeddedQueue_.front(); bayerBuffer = updateQueue(bayerQueue_, embeddedBuffer->metadata().timestamp, - unicam_[Unicam::Image].dev()); + &unicam_[Unicam::Image]); if (!bayerBuffer) { LOG(RPI, Debug) << "Could not find matching bayer buffer - ending."; @@ -1391,11 +1410,7 @@ void RPiCameraData::tryRunPipeline() } } - /* - * Take the first request from the queue and action the IPA. - * Unicam buffers for the request have already been queued as they come - * in. - */ + /* Take the first request from the queue and action the IPA. */ Request *request = requestQueue_.front(); /* @@ -1407,12 +1422,6 @@ void RPiCameraData::tryRunPipeline() op.controls = { request->controls() }; ipa_->processEvent(op); - /* Queue up any ISP buffers passed into the request. */ - for (auto &stream : isp_) { - if (stream.isExternal()) - stream.dev()->queueBuffer(request->findBuffer(&stream)); - } - /* Ready to use the buffers, pop them off the queue. */ bayerQueue_.pop(); embeddedQueue_.pop(); @@ -1442,32 +1451,42 @@ void RPiCameraData::tryFlushQueues() * and give a chance for the hardware to return to lock-step. We do have * to drop all interim frames. */ - if (unicam_[Unicam::Image].getBuffers()->size() == bayerQueue_.size() && - unicam_[Unicam::Embedded].getBuffers()->size() == embeddedQueue_.size()) { + if (unicam_[Unicam::Image].getBuffers().size() == bayerQueue_.size() && + unicam_[Unicam::Embedded].getBuffers().size() == embeddedQueue_.size()) { + /* This cannot happen when Unicam streams are external. */ + assert(!unicam_[Unicam::Image].isExternal()); + LOG(RPI, Warning) << "Flushing all buffer queues!"; while (!bayerQueue_.empty()) { - unicam_[Unicam::Image].dev()->queueBuffer(bayerQueue_.front()); + unicam_[Unicam::Image].queueBuffer(bayerQueue_.front()); bayerQueue_.pop(); } while (!embeddedQueue_.empty()) { - unicam_[Unicam::Embedded].dev()->queueBuffer(embeddedQueue_.front()); + unicam_[Unicam::Embedded].queueBuffer(embeddedQueue_.front()); embeddedQueue_.pop(); } } } FrameBuffer *RPiCameraData::updateQueue(std::queue &q, uint64_t timestamp, - V4L2VideoDevice *dev) + RPi::RPiStream *stream) { + /* + * If the unicam streams are external (both have be to the same), then we + * can only return out the top buffer in the queue, and assume they have + * been synced by queuing at the same time. We cannot drop these frames, + * as they may have been provided externally. + */ while (!q.empty()) { FrameBuffer *b = q.front(); - if (b->metadata().timestamp < timestamp) { + if (!stream->isExternal() && b->metadata().timestamp < timestamp) { q.pop(); - dev->queueBuffer(b); - LOG(RPI, Warning) << "Dropping input frame!"; - } else if (b->metadata().timestamp == timestamp) { + stream->queueBuffer(b); + LOG(RPI, Warning) << "Dropping unmatched input frame in stream " + << stream->name(); + } else if (stream->isExternal() || b->metadata().timestamp == timestamp) { /* The calling function will pop the item from the queue. */ return b; } else { diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index 2e52bd5c..b4d737db 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -27,78 +27,120 @@ std::string RPiStream::name() const void RPiStream::reset() { external_ = false; - internalBuffers_.clear(); -} - -bool RPiStream::isImporter() const -{ - return importOnly_; + clearBuffers(); } void RPiStream::setExternal(bool external) { + /* Import streams cannot be external. */ + ASSERT(!external || !importOnly_); external_ = external; } bool RPiStream::isExternal() const { - /* - * Import streams cannot be external. - * - * RAW capture is a special case where we simply copy the RAW - * buffer out of the request. All other buffer handling happens - * as if the stream is internal. - */ - return external_ && !importOnly_; + return external_; } -void RPiStream::setExternalBuffers(std::vector> *buffers) +void RPiStream::setExportedBuffers(std::vector> *buffers) { - externalBuffers_ = buffers; + std::transform(buffers->begin(), buffers->end(), std::back_inserter(bufferList_), + [](std::unique_ptr &b) { return b.get(); }); } -const std::vector> *RPiStream::getBuffers() const +const std::vector &RPiStream::getBuffers() const { - return external_ ? externalBuffers_ : &internalBuffers_; + return bufferList_; } bool RPiStream::findFrameBuffer(FrameBuffer *buffer) const { - auto start = external_ ? externalBuffers_->begin() : internalBuffers_.begin(); - auto end = external_ ? externalBuffers_->end() : internalBuffers_.end(); - if (importOnly_) return false; - if (std::find_if(start, end, - [buffer](std::unique_ptr const &ref) { return ref.get() == buffer; }) != end) + if (std::find(bufferList_.begin(), bufferList_.end(), buffer) != bufferList_.end()) return true; return false; } -int RPiStream::importBuffers(unsigned int count) +int RPiStream::prepareBuffers(unsigned int count) { + int ret; + + if (!importOnly_) { + if (count) { + /* Export some frame buffers for internal use. */ + ret = dev_->exportBuffers(count, &internalBuffers_); + if (ret < 0) + return ret; + + /* Add these exported buffers to the internal/external buffer list. */ + setExportedBuffers(&internalBuffers_); + + /* Add these buffers to the queue of internal usable buffers. */ + for (auto const &buffer : internalBuffers_) + availableBuffers_.push(buffer.get()); + } + + /* We must import all internal/external exported buffers. */ + count = bufferList_.size(); + } + return dev_->importBuffers(count); } -int RPiStream::allocateBuffers(unsigned int count) +int RPiStream::queueBuffer(FrameBuffer *buffer) +{ + /* + * A nullptr buffer implies an external stream, but no external + * buffer has been supplied. So, pick one from the availableBuffers_ + * queue. + */ + if (!buffer) { + if (availableBuffers_.empty()) { + LOG(RPISTREAM, Warning) << "No buffers available for " + << name_; + return -EINVAL; + } + + buffer = availableBuffers_.front(); + availableBuffers_.pop(); + } + + LOG(RPISTREAM, Debug) << "Queuing buffer " << buffer->cookie() + << " for " << name_; + + int ret = dev_->queueBuffer(buffer); + if (ret) { + LOG(RPISTREAM, Error) << "Failed to queue buffer for " + << name_; + } + + return ret; +} + +void RPiStream::returnBuffer(FrameBuffer *buffer) { - return dev_->allocateBuffers(count, &internalBuffers_); + /* This can only be called for external streams. */ + assert(external_); + + availableBuffers_.push(buffer); } -int RPiStream::queueBuffers() +int RPiStream::queueAllBuffers() { + int ret; + if (external_) return 0; - for (auto &b : internalBuffers_) { - int ret = dev_->queueBuffer(b.get()); - if (ret) { - LOG(RPISTREAM, Error) << "Failed to queue buffers for " - << name_; + while (!availableBuffers_.empty()) { + ret = queueBuffer(availableBuffers_.front()); + if (ret < 0) return ret; - } + + availableBuffers_.pop(); } return 0; @@ -107,8 +149,14 @@ int RPiStream::queueBuffers() void RPiStream::releaseBuffers() { dev_->releaseBuffers(); - if (!external_ && !importOnly_) - internalBuffers_.clear(); + clearBuffers(); +} + +void RPiStream::clearBuffers() +{ + availableBuffers_ = std::queue{}; + internalBuffers_.clear(); + bufferList_.clear(); } } /* namespace RPi */ diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index 8fcf522b..73954ec7 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -44,20 +44,23 @@ public: void setExternal(bool external); bool isExternal() const; - void setExternalBuffers(std::vector> *buffers); - const std::vector> *getBuffers() const; + void setExportedBuffers(std::vector> *buffers); + const std::vector &getBuffers() const; bool findFrameBuffer(FrameBuffer *buffer) const; - int importBuffers(unsigned int count); - int allocateBuffers(unsigned int count); + int prepareBuffers(unsigned int count); + int queueBuffer(FrameBuffer *buffer); + void returnBuffer(FrameBuffer *buffer); - int queueBuffers(); + int queueAllBuffers(); void releaseBuffers(); private: + void clearBuffers(); + /* * Indicates that this stream is active externally, i.e. the buffers - * are provided by the application. + * might be provided by (and returned to) the application. */ bool external_; @@ -70,11 +73,21 @@ private: /* The actual device stream. */ std::unique_ptr dev_; - /* Internally allocated framebuffers associated with this device stream. */ - std::vector> internalBuffers_; + /* All framebuffers associated with this device stream. */ + std::vector bufferList_; + + /* + * List of frame buffers that we can use if none have been provided by + * the application for external streams. This is populated by the + * buffers exported internally. + */ + std::queue availableBuffers_; - /* Externally allocated framebuffers associated with this device stream. */ - std::vector> *externalBuffers_; + /* + * This is a list of buffers exported internally. Need to keep this around + * as the stream needs to maintain ownership of these buffers. + */ + std::vector> internalBuffers_; }; /*