From patchwork Tue Aug 4 09:58:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 9177 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7956DBD86F for ; Tue, 4 Aug 2020 09:59:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 430EB6049B; Tue, 4 Aug 2020 11:59:12 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="aS/ATfxi"; dkim-atps=neutral Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1005B60550 for ; Tue, 4 Aug 2020 11:59:09 +0200 (CEST) Received: by mail-wr1-x42c.google.com with SMTP id f7so36827481wrw.1 for ; Tue, 04 Aug 2020 02:59:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VrnZVRzAFcTlfpKwg6N0sIpXHNob2AVqtYJvsi6G/f4=; b=aS/ATfxiQDBlhTPdF9BqRoymrGLfHs3LwduRyuAy2pzxZQ29PPd7q9ReZwdMwncv1e Yr+5tH/B+dH3Uxau/+jvQTTtmW2vdk1oa7sdTG17eChzlHzVLItp0h48bV/0zNyS2Ae6 mWRyNZ77TWZlgAOSOTJeedSZD/EedBL2cgRt0WU+/vYpn7VRt37sVJSJfZm4F246zg2S wfuNs0xUVCKAUsGr4IQENJ3i7LjU93xlTsJiDljGOB2EC8z93g3D1nQMxlP0YVeN4/4j nrGTLDXzDvJGl8P+L2gaCYw8CIwplenJLLWLzRMyOUqHGVd4batplq4NafEsxrPl9qfZ KA9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VrnZVRzAFcTlfpKwg6N0sIpXHNob2AVqtYJvsi6G/f4=; b=TL1nqVZKiR80uAGzWq/EEH+OAw6Pl2rb8GZZltM1Y5s5a76jeDa0SGiMbOU31j3o7y GvBnQXB7L5T65CP4cPxQaR+6rWwWHILECtqPTmeLupPRIzrIJ1V82zyEHECT5jBWnrIW naaNC4AjHFahylFg5nllFUEFilOVV52ouzf+N4qIzNbW0cjxC86A9J7DVmpILxTMp7H9 fby9VDeCbxKxmGx5RUfF+h6eXH5b8uGnB7/7OohJ9kXvv4P8DT1VA8hKmgG/EJy+MXvt 9hzx9v19AltkF39DC2oPjQO9j5sFQxaLAaNIHwie9Osi9AoBqwN3hN74ekxOe+qHcjQ1 wUnA== X-Gm-Message-State: AOAM533S9YO49Hz+TuBQQtTsoQAwIuBSmovCXrOOIJKBRARUUp4Jiv1B okkCokFSK2lgcdf94g4urAxIwpT3QMBGBA== X-Google-Smtp-Source: ABdhPJzt9EiSGL8oe6ZjQxDaB5G4zJ7FMr4/lK6xqmEA2+Let7409S1qLH/AqHlCu2S6Ax+ew4TRAw== X-Received: by 2002:a5d:4802:: with SMTP id l2mr17877581wrq.183.1596535148230; Tue, 04 Aug 2020 02:59:08 -0700 (PDT) Received: from naushir-VirtualBox.patuck.local ([88.97.76.4]) by smtp.gmail.com with ESMTPSA id 32sm34047234wrh.18.2020.08.04.02.59.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 02:59:07 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Aug 2020 10:58:50 +0100 Message-Id: <20200804095850.275499-13-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200804095850.275499-1-naush@raspberrypi.com> References: <20200804095850.275499-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 12/12] pipeline: ipa: raspberrypi: Handle any externally allocated FrameBuffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Handle the case where a FrameBuffer that has been externally allocated (i.e. not through the v4l2 video device) is passed into a Request. We must store the buffer pointer in the stream internal buffer list to identify when used, as well as mmap the buffer in the IPA if needed. Signed-off-by: Naushir Patuck --- include/libcamera/ipa/raspberrypi.h | 11 +-- src/ipa/raspberrypi/raspberrypi.cpp | 6 +- .../pipeline/raspberrypi/raspberrypi.cpp | 82 ++++++++++++++----- .../pipeline/raspberrypi/rpi_stream.cpp | 30 ++++++- .../pipeline/raspberrypi/rpi_stream.h | 3 + 5 files changed, 103 insertions(+), 29 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index 262fc6f3..dd6ebeac 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -28,11 +28,12 @@ enum RPiOperations { RPI_IPA_EVENT_QUEUE_REQUEST, }; -enum RPiIpaMask { - ID = 0x0ffff, - STATS = 0x10000, - EMBEDDED_DATA = 0x20000, - BAYER_DATA = 0x40000 +enum RPiBufferMask { + ID = 0x00ffff, + STATS = 0x010000, + EMBEDDED_DATA = 0x020000, + BAYER_DATA = 0x040000, + EXTERNAL_BUFFER = 0x100000, }; /* Size of the LS grid allocation. */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index cd49de80..93dfef27 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -352,7 +352,7 @@ void IPARPi::processEvent(const IPAOperationData &event) IPAOperationData op; op.operation = RPI_IPA_ACTION_STATS_METADATA_COMPLETE; - op.data = { bufferId & RPiIpaMask::ID }; + op.data = { bufferId & RPiBufferMask::ID }; op.controls = { libcameraMetadata_ }; queueFrameAction.emit(0, op); break; @@ -373,7 +373,7 @@ void IPARPi::processEvent(const IPAOperationData &event) /* Ready to push the input buffer into the ISP. */ IPAOperationData op; op.operation = RPI_IPA_ACTION_RUN_ISP; - op.data = { bayerbufferId & RPiIpaMask::ID }; + op.data = { bayerbufferId & RPiBufferMask::ID }; queueFrameAction.emit(0, op); break; } @@ -700,7 +700,7 @@ void IPARPi::returnEmbeddedBuffer(unsigned int bufferId) { IPAOperationData op; op.operation = RPI_IPA_ACTION_EMBEDDED_COMPLETE; - op.data = { bufferId & RPiIpaMask::ID }; + op.data = { bufferId & RPiBufferMask::ID }; queueFrameAction.emit(0, op); } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index b059c582..e3240914 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -151,6 +151,7 @@ public: void clearIncompleteRequests(); void handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stream); + void handleExternalBuffer(FrameBuffer *buffer, RPi::RPiStream *stream); void handleState(); CameraSensor *sensor_; @@ -726,23 +727,41 @@ int PipelineHandlerRPi::queueRequestDevice(Camera *camera, Request *request) /* Push all buffers supplied in the Request to the respective streams. */ for (auto stream : data->streams_) { - if (stream->isExternal()) { - FrameBuffer *buffer = request->findBuffer(stream); + if (!stream->isExternal()) + continue; + + FrameBuffer *buffer = request->findBuffer(stream); + if (buffer && stream->getBufferId(buffer) == -1) { /* - * If no buffer is provided by the request for this stream, we - * queue a nullptr to the stream to signify that it must use an - * internally allocated buffer for this capture request. This - * buffer will not be given back to the application, but is used - * to support the internal pipeline flow. - * - * The below queueBuffer() call will do nothing if there are not - * enough internal buffers allocated, but this will be handled by - * queuing the request for buffers in the RPiStream object. + * This buffer is not recognised, so it must have been allocated + * outside the v4l2 device. Store it in the stream buffer list + * so we can track it. */ - int ret = stream->queueBuffer(buffer); - if (ret) - return ret; + stream->setExternalBuffer(buffer); + + /* Also get the IPA to mmap it if needed. */ + if (stream == &data->unicam_[Unicam::Embedded] || + stream == &data->isp_[Isp::Stats]) { + unsigned int id = stream->getBufferId(buffer); + unsigned int mask = stream == &data->unicam_[Unicam::Embedded] ? + RPiBufferMask::EMBEDDED_DATA : RPiBufferMask::STATS; + mapBuffers(camera, {{ id, buffer }}, mask); + } } + /* + * If no buffer is provided by the request for this stream, we + * queue a nullptr to the stream to signify that it must use an + * internally allocated buffer for this capture request. This + * buffer will not be given back to the application, but is used + * to support the internal pipeline flow. + * + * The below queueBuffer() call will do nothing if there are not + * enough internal buffers allocated, but this will be handled by + * queuing the request for buffers in the RPiStream object. + */ + int ret = stream->queueBuffer(buffer); + if (ret) + return ret; } /* Push the request to the back of the queue. */ @@ -906,8 +925,8 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera) * Pass the stats and embedded data buffers to the IPA. No other * buffers need to be passed. */ - mapBuffers(camera, data->isp_[Isp::Stats].getBuffers(), RPiIpaMask::STATS); - mapBuffers(camera, data->unicam_[Unicam::Embedded].getBuffers(), RPiIpaMask::EMBEDDED_DATA); + mapBuffers(camera, data->isp_[Isp::Stats].getBuffers(), RPiBufferMask::STATS); + mapBuffers(camera, data->unicam_[Unicam::Embedded].getBuffers(), RPiBufferMask::EMBEDDED_DATA); return 0; } @@ -1216,7 +1235,7 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) if (stream == &isp_[Isp::Stats]) { IPAOperationData op; op.operation = RPI_IPA_EVENT_SIGNAL_STAT_READY; - op.data = { RPiIpaMask::STATS | static_cast(index) }; + op.data = { RPiBufferMask::STATS | static_cast(index) }; ipa_->processEvent(op); } else { /* Any other ISP output can be handed back to the application now. */ @@ -1290,6 +1309,12 @@ void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stre Request *request = requestQueue_.front(); if (!dropFrameCount_ && request->findBuffer(stream) == buffer) { + /* + * Check if this is an externally provided buffer, and if + * so, we must get the IPA to unmap it and stop tracking + * it in the pipeline handler. + */ + handleExternalBuffer(buffer, stream); /* * Tag the buffer as completed, returning it to the * application. @@ -1308,6 +1333,25 @@ void RPiCameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::RPiStream *stre } } +void RPiCameraData::handleExternalBuffer(FrameBuffer *buffer, RPi::RPiStream *stream) +{ + unsigned int id = stream->getBufferId(buffer); + + if (!(id & RPiBufferMask::EXTERNAL_BUFFER)) + return; + + if (stream == &unicam_[Unicam::Embedded] || stream == &isp_[Isp::Stats]) { + unsigned int mask = stream == &unicam_[Unicam::Embedded] ? + RPiBufferMask::EMBEDDED_DATA : RPiBufferMask::STATS; + /* Ask the IPA to unmap the buffer. */ + ipa_->unmapBuffers({ mask | id }); + ipaBuffers_.erase(mask | id); + } + + /* Stop the Stream object from tracking the buffer. */ + stream->removeExternalBuffer(buffer); +} + void RPiCameraData::handleState() { switch (state_) { @@ -1440,8 +1484,8 @@ void RPiCameraData::tryRunPipeline() << " Embedded buffer id: " << embeddedIndex; op.operation = RPI_IPA_EVENT_SIGNAL_ISP_PREPARE; - op.data = { RPiIpaMask::EMBEDDED_DATA | embeddedIndex, - RPiIpaMask::BAYER_DATA | bayerIndex }; + op.data = { RPiBufferMask::EMBEDDED_DATA | embeddedIndex, + RPiBufferMask::BAYER_DATA | bayerIndex }; ipa_->processEvent(op); } diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp index 879e25ba..c09f14c9 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp @@ -6,6 +6,8 @@ */ #include "rpi_stream.h" +#include + #include "libcamera/internal/log.h" namespace libcamera { @@ -44,8 +46,13 @@ bool RPiStream::isExternal() const void RPiStream::setExportedBuffers(std::vector> *buffers) { - for (auto const &buffer : *buffers) - bufferMap_.emplace(id_++, buffer.get()); + /* Ensure we are using a sensible number of buffers. */ + ASSERT(id_ < RPiBufferMask::ID); + + for (auto const &buffer : *buffers) { + bufferMap_.emplace(id_ & RPiBufferMask::ID, buffer.get()); + id_++; + } } const BufferMap &RPiStream::getBuffers() const @@ -68,6 +75,25 @@ int RPiStream::getBufferId(FrameBuffer *buffer) const return it->first; } +void RPiStream::setExternalBuffer(FrameBuffer *buffer) +{ + /* Ensure we are using a sensible number of buffers. */ + ASSERT(id_ < RPiBufferMask::ID); + + bufferMap_.emplace(RPiBufferMask::EXTERNAL_BUFFER | (id_ & RPiBufferMask::ID), + buffer); + id_++; +} + +void RPiStream::removeExternalBuffer(FrameBuffer *buffer) +{ + int id = getBufferId(buffer); + + /* Ensure we have this buffer in the stream, and it is marked external. */ + ASSERT(id != -1 && (id & RPiBufferMask::EXTERNAL_BUFFER)); + bufferMap_.erase(id); +} + int RPiStream::prepareBuffers(unsigned int count) { int ret; diff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h index ed517c22..8b23c4b2 100644 --- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h +++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h @@ -51,6 +51,9 @@ public: const BufferMap &getBuffers() const; int getBufferId(FrameBuffer *buffer) const; + void setExternalBuffer(FrameBuffer *buffer); + void removeExternalBuffer(FrameBuffer *buffer); + int prepareBuffers(unsigned int count); int queueBuffer(FrameBuffer *buffer); void returnBuffer(FrameBuffer *buffer);