From patchwork Mon Aug 21 13:10:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gabrielle George X-Patchwork-Id: 18947 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0FE42BF415 for ; Mon, 21 Aug 2023 13:10:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A5BE3628D2; Mon, 21 Aug 2023 15:10:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1692623451; bh=rPAqqsyOSUamSnOAgOEB1nNiDlbbH8F5K4TXQGSNDmc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=sUo540ObJSr8FW6SD6b3QyA0zh6jXmUCkLqnv0iRVKSa+cHBOOS2FiNIyImHQD2pu 11r+KhB8LeuUKo04cTo8pNkguFoMmKk/ia94/fNTDsM6hkaYGgLA0vEIWerP49Dq05 +vfqwkJjBZeiDG6oei6YiwvYKH2rSKXo2KaC2zplIwO0XisXzzO3bKhtRRiDi8n88/ yhwvXMjW9Lly/g3G+Q0tAErCAYSfs44On1c4taVgXRvK4z6E4x6uup68zg5r4eOFgZ rL1sCSx1S5iARcJwtUWyCSIg4VaLfXjTeHn4J/A3to41gTrhrr7XcjE9u6zQpX1Nb/ e/XBBcf1HX0Ow== Received: from mail-oa1-x2f.google.com (mail-oa1-x2f.google.com [IPv6:2001:4860:4864:20::2f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1D70661E09 for ; Mon, 21 Aug 2023 15:10:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="XAz0LZjj"; dkim-atps=neutral Received: by mail-oa1-x2f.google.com with SMTP id 586e51a60fabf-1c4b4c40281so2370819fac.1 for ; Mon, 21 Aug 2023 06:10:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692623445; x=1693228245; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=EU6t+fl/QqCK94Dhs7LxVZPCFNQK/h1wuTeKXag0jbU=; b=XAz0LZjj4dTHE/R1BR7lULrWmujiUM98d4ZkfHUU3t/tvnzGJ4I5Pa7EQibsBw63Bg 3mSD1gBz4uh7R5+M/GsDexOmIzTmdFJU8ORzC9/yC0nMUXYpDOkvjfGQ4BqX2INIfuEo yFTZevWacZ51RcgViD1EZxJFausOglLubc/JtOl41wi8M/VvBJzfKel4/B81ieaLGXD7 /VYvVwBFLbkQgJtEQi540DktT+2Y0v9/ckXoLt/OCzn9DtxlHYIhhrCfvrHfDGFl0Ldr YAu7Ol7NTa+KM5VqsSBK/Nu7MG67phcq+ke8VN8EjHzMpsyZQ2Xt3qHZdLnVEPi843Mp FM2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692623445; x=1693228245; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EU6t+fl/QqCK94Dhs7LxVZPCFNQK/h1wuTeKXag0jbU=; b=GSm+YBU69C583HUJaMm/ZXgTgniGKHJUGfpaQ75Glapu1/KGwztgDZUN4Sc9XhgYeT 6FE1SF7c3vnGE+XoBXLoZXv3XNmGuAugWDeNMzeS4LKFFimz6e6SvFohSzdXGE+B1CFu cYk/xbwErW5Ka3UcqYqXEqppUDYVQLZ6hi6pkv95sy//J1HW16vIBkfpHwVVT9m+RfbJ Jks7UHRf9qMYzSiwZYdh3Wj+Q3IK1wrq4KhGSvtDGI6p4A/9IERdVLRGUg0rt+zI/B/9 EPmzHpumku9upv41F+kBbcpZQUfpByXwxvA5Z9JXwicvG/yD9IUliq0Q3DocZ/QvNa3p ha0Q== X-Gm-Message-State: AOJu0YyuOfnvSPLslurOkzjIPer0E5Pb2/TeF7RDcrbWjNKgII80TQju OP/hPrGlvYDb+6vjdyAEgjt52CvMyTc= X-Google-Smtp-Source: AGHT+IGpoqNewMqbPmpHwQMEdvahBu2zXq1QIDCtHpvenaN8k9yyPlgf6IpgEGPAyaWHPWHOyGEO0g== X-Received: by 2002:a05:6870:2198:b0:1b7:273d:780a with SMTP id l24-20020a056870219800b001b7273d780amr4653240oae.22.1692623445541; Mon, 21 Aug 2023 06:10:45 -0700 (PDT) Received: from localhost.localdomain (97-115-76-16.ptld.qwest.net. [97.115.76.16]) by smtp.gmail.com with ESMTPSA id ci1-20020a056871c48100b001bb51450d85sm4160283oac.4.2023.08.21.06.10.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 06:10:45 -0700 (PDT) To: libcamera-devel@lists.libcamera.org, kieran.bingham@ideasonboard.com, vedantparanjape160201@gmail.com, gabbymg94@gmail.com Date: Mon, 21 Aug 2023 06:10:38 -0700 Message-Id: <20230821131039.127370-5-gabbymg94@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821131039.127370-1-gabbymg94@gmail.com> References: <20230821131039.127370-1-gabbymg94@gmail.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH v2 4/5] libcamera: pipeline: uvcvideo: Allocate metadata buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Gabby George via libcamera-devel From: Gabrielle George Reply-To: Gabby George Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Perform the allocation and mapping of metadata buffers into libcamera's virtual address space. UVC metadata buffers cannot be exported as DMA buffer file descriptors, so use the MappedFrameBuffer class to map them into memory directly. This will give the UVC pipeline access to buffer data to extract timestamp information. Metadata buffers are internal to the UVC pipeline, so buffer memory should not be exposed to the user. Signed-off-by: Gabby George --- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 98 +++++++++++++++++++- 1 file changed, 96 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index dbe0cc8c..c8d6633f 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -24,6 +24,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/mapped_framebuffer.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/sysfs.h" @@ -51,6 +52,9 @@ public: std::unique_ptr video_; std::unique_ptr metadata_; Stream stream_; + std::vector> metadataBuffers_; + std::map mappedMetadataBuffers_; + std::map> formats_; private: @@ -96,6 +100,10 @@ private: const ControlValue &value); int processControls(UVCCameraData *data, Request *request); + int createMetadataBuffers(Camera *camera, unsigned int count); + int cleanupMetadataBuffers(Camera *camera); + int cleanup(Camera *camera); + UVCCameraData *cameraData(Camera *camera) { return static_cast(camera->_d()); @@ -225,10 +233,67 @@ int PipelineHandlerUVC::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; cfg.setStream(&data->stream_); + return 0; +} + +int PipelineHandlerUVC::cleanupMetadataBuffers(Camera *camera) +{ + int ret = 0; + UVCCameraData *data = cameraData(camera); + + if (data->metadata_) + data->metadata_->releaseBuffers(); + data->metadataBuffers_.clear(); + data->mappedMetadataBuffers_.clear(); + data->metadata_ = nullptr; + return ret; +} + +int PipelineHandlerUVC::cleanup(Camera *camera) +{ + UVCCameraData *data = cameraData(camera); + cleanupMetadataBuffers(camera); + data->video_->releaseBuffers(); return 0; } +/** + * UVC Metadata stream does not support exporting buffers via EXPBUF, + * so it is necessary to create and store mmap-ed addresses. + * Metadata buffers are internal to libcamera. They are not, and + * cannot be, exposed to the user. + * + * Returns the number of buffers allocated and mapped. + * + * \return The number of buffers allocated, or a negative error code if + * the number of buffers allocated was not equal to "count" + * \retval -EINVAL if "count" buffers were not successfully allocated. + * \retval MappedFrameBuffer::error() + */ +int PipelineHandlerUVC::createMetadataBuffers(Camera *camera, unsigned int count) +{ + UVCCameraData *data = cameraData(camera); + int ret = data->metadata_->allocateBuffers(count, &data->metadataBuffers_); + if (ret < 0) + return -EINVAL; + + for (unsigned int i = 0; i < count; i++) { + MappedFrameBuffer mappedBuffer(data->metadataBuffers_[i].get(), + MappedFrameBuffer::MapFlag::Read, true); + if (!mappedBuffer.isValid()) { + LOG(UVC, Warning) + << "Failed to mmap metadata buffer: " + << strerror(mappedBuffer.error()); + return mappedBuffer.error(); + } + + data->mappedMetadataBuffers_.emplace(i, std::move(mappedBuffer)); + data->metadataBuffers_[i]->setCookie(i); + } + return ret; +} + int PipelineHandlerUVC::exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) { @@ -249,18 +314,47 @@ int PipelineHandlerUVC::start(Camera *camera, [[maybe_unused]] const ControlList ret = data->video_->streamOn(); if (ret < 0) { - data->video_->releaseBuffers(); + cleanup(camera); return ret; } + /* + * If metadata allocation fails, exit this function but + * do not return a failure as video started successfully. + * Fall back on using driver timestamps. + */ + if (data->metadata_) { + if (createMetadataBuffers(camera, count) < 0 || + data->metadata_->streamOn()) { + LOG(UVC, Warning) + << "Metadata stream unavailable. Using driver timestamps."; + data->metadata_ = nullptr; + return 0; + } + + for (std::unique_ptr &buf : data->metadataBuffers_) { + ret = data->metadata_->queueBuffer(buf.get()); + if (ret < 0) { + LOG(UVC, Warning) + << "Metadata stream unavailable. Using driver timestamps."; + cleanupMetadataBuffers(camera); + return 0; + } + } + } return 0; } void PipelineHandlerUVC::stopDevice(Camera *camera) { UVCCameraData *data = cameraData(camera); + data->video_->streamOff(); - data->video_->releaseBuffers(); + + if (data->metadata_) + data->metadata_->streamOff(); + + cleanup(camera); } int PipelineHandlerUVC::processControl(ControlList *controls, unsigned int id,