From patchwork Mon Aug 14 11:28:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gabrielle George X-Patchwork-Id: 18941 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 56798C32B1 for ; Mon, 14 Aug 2023 11:29:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2541E628DD; Mon, 14 Aug 2023 13:29:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1692012541; bh=szpRlwvfHuwJi9e/cV+qIhsj0OHhxLHDghQNsEK/8f4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=SfK9cHj4V1t5fFQlKzRcROAnFVTPYCYz4Wtg3WmRfMmY8nKhU/Q2fvH9RXGp9/jOU qrZMbbozEJ4lOqeZv+d6Q/HUsB96GnJJRovTfdYBRmdq3Ani7fAXzAJY/YaujGmJbT DKJzptm5ROqGErQ/sz/y55oPg3QshEP97+CpXQ8ZSiIC3x85CJ9mmmuPjpfZlL92rk o7/iP7rtiXg/ydePRtfHdMih2hYbmyKMnLY5c2qQI3Hzivr7LOiIB4BNslTTZbBae0 WNjYP2yJbu1+FzHS0BaF50A0Hm9ZK2WPrWhkKTd70Kw6jt8Cavebs2Mcak9WvPbesY 9dpz8/z58w+kQ== Received: from mail-ot1-x32f.google.com (mail-ot1-x32f.google.com [IPv6:2607:f8b0:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5A8C7628DB for ; Mon, 14 Aug 2023 13:28:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Not+KZnU"; dkim-atps=neutral Received: by mail-ot1-x32f.google.com with SMTP id 46e09a7af769-6bca66e6c44so3735614a34.0 for ; Mon, 14 Aug 2023 04:28:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692012536; x=1692617336; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=2paCm8n5nk6/G3ElxO9IuZP3Z4eBx88uZvY2ElGxhDk=; b=Not+KZnUubJMvjuTOcwvhfnSmib04EyzzXYOXH5+hNJpnH7/Fwbn5RDspj3r5knCrx 6c1wWfb9AUYV3ICVNNbl4fKTBZ0ORdYYvYF72wBEsOrTiEh1Yb/D+eS5pwq5Fc3eviE4 Uxt1Saodr1uFX5161fJOXkjNu1TT939Wy/9Ovzl8rjHMdDGwVxnl+b2B2k9iRB6e+2uA Bb1sPUu2MTG+apDsTM/m7YK3VyE2DLspxl+JcWiUmNObTpkGX0rFDIsd8tUq7ACVOHd3 O63s6Yh0cdlKsRBInGzPyIbfCult/SHprM29UC1p+Up6H7hJOSbq85yFlEFHhPqrYwug BdqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692012536; x=1692617336; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2paCm8n5nk6/G3ElxO9IuZP3Z4eBx88uZvY2ElGxhDk=; b=FxEu7rJvqKojAaRX89pZ3s9hMU+J7Ckcj4KXNlyAxYwjaaBfdFQmuTJBaMk/kuwN97 3twNjjiHHUNSgQasOKo/3xiVYn38al2/rPEMtabCJBN3epBLeYgobcp4OoygPuFFvHnd zywzgJBDpXxdTmVh0ISBLWXCorpO7vKIdgcNBDjg8I6IARiMIqGNpsnFcF9GzDhvR0my shrVVAyz99miZmVafuE+xV66PJlWE/f5UZ6xT9HDEVQXB/GqBepWJhRN0CbxF7ocTqUp RJp9wlOjgiBXP/3WD03M1Os/NDGbuWrG++C2ymzFUPpRw26x0+n7ow2AeVs55AenYuxm omhQ== X-Gm-Message-State: AOJu0YzX3o3TjJDmA5jZuqSfsF4Ym6ANNNkhTv+yNquESeoLR/xgFvuw jjI8lE8xx09ajDTrm3x1sJ5W/iprOAOYuQ== X-Google-Smtp-Source: AGHT+IHjrbpcs3YSyLE8L1EbRDdGRPuif37IATGwiWpMmjU596ZQCgTiT7+Uf+SEaF7M8L/RYAFczg== X-Received: by 2002:a9d:7585:0:b0:6b5:f457:adaa with SMTP id s5-20020a9d7585000000b006b5f457adaamr8688818otk.29.1692012535912; Mon, 14 Aug 2023 04:28:55 -0700 (PDT) Received: from localhost.localdomain (97-115-76-16.ptld.qwest.net. [97.115.76.16]) by smtp.gmail.com with ESMTPSA id c3-20020a9d6c83000000b006b92509e76esm4163817otr.32.2023.08.14.04.28.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Aug 2023 04:28:55 -0700 (PDT) To: libcamera-devel@lists.libcamera.org, kieran.bingham@ideasonboard.com, vedantparanjape160201@gmail.com, gabbymg94@gmail.com Date: Mon, 14 Aug 2023 04:28:48 -0700 Message-Id: <20230814112849.176943-5-gabbymg94@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230814112849.176943-1-gabbymg94@gmail.com> References: <20230814112849.176943-1-gabbymg94@gmail.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 4/5] libcamera: pipeline: uvcvideo: Allocate metadata buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Gabby George via libcamera-devel From: Gabrielle George Reply-To: Gabby George Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Perform the allocation and mapping of metadata buffers into libcamera's virtual address space. UVC metadata buffers cannot be exported as DMA buffer file descriptors, so use the MappedFrameBuffer class to map them into memory directly. This will give the UVC pipeline access to buffer data to extract timestamp information. Metadata buffers are internal to the UVC pipeline, so buffer memory should not be exposed to the user. Signed-off-by: Gabby George --- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 97 +++++++++++++++++++- 1 file changed, 95 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index 4470d8a2..51f30187 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -24,6 +24,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/mapped_framebuffer.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/sysfs.h" @@ -51,11 +52,17 @@ public: std::unique_ptr video_; std::unique_ptr metadata_; Stream stream_; + std::vector> metadataBuffers_; + std::map mappedMetadataBuffers_; + bool useMetadataStream_; + std::map> formats_; private: int initMetadata(MediaDevice *media); + const unsigned int minLengthHeaderBuf_ = 10; + bool generateId(); std::string id_; @@ -96,6 +103,10 @@ private: const ControlValue &value); int processControls(UVCCameraData *data, Request *request); + int createMetadataBuffers(Camera *camera, unsigned int count); + int cleanupMetadataBuffers(Camera *camera); + int cleanup(Camera *camera); + UVCCameraData *cameraData(Camera *camera) { return static_cast(camera->_d()); @@ -225,10 +236,66 @@ int PipelineHandlerUVC::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; cfg.setStream(&data->stream_); + return 0; +} +int PipelineHandlerUVC::cleanupMetadataBuffers(Camera *camera) +{ + int ret = 0; + UVCCameraData *data = cameraData(camera); + + ret = data->metadata_->releaseBuffers(); + data->metadataBuffers_.clear(); //call the destructor for the frame buffers + data->mappedMetadataBuffers_.clear(); + data->useMetadataStream_ = false; + + return ret; +} + +int PipelineHandlerUVC::cleanup(Camera *camera) +{ + UVCCameraData *data = cameraData(camera); + cleanupMetadataBuffers(camera); + data->video_->releaseBuffers(); return 0; } +/* + * UVC Metadata stream does not support exporting buffers via EXPBUF, + * so it is necessary to create and store mmap-ed addresses. + * Metadata buffers are internal to libcamera. They are not, and + * cannot be, exposed to the user. + * + * Returns the number of buffers allocated and mapped. + * + * \return The number of buffers allocated, or a negative error code if + * the number of buffers allocated was not equal to "count" + * \retval -EINVAL if "count" buffers were not successfully allocated. + * \retval -ENOMEM if mmap failed. + */ +int PipelineHandlerUVC::createMetadataBuffers(Camera *camera, unsigned int count) +{ + UVCCameraData *data = cameraData(camera); + int ret = data->metadata_->allocateBuffers(count, &data->metadataBuffers_); + if (ret < 0) + return -EINVAL; + + for (unsigned int i = 0; i < count; i++) { + MappedFrameBuffer mappedBuffer(data->metadataBuffers_[i].get(), + MappedFrameBuffer::MapFlag::Read, true); + if (!mappedBuffer.isValid()) { + LOG(UVC, Error) + << "Failed to mmap metadata buffer: " + << strerror(mappedBuffer.error()); + return mappedBuffer.error(); + } + + data->mappedMetadataBuffers_.emplace(i, std::move(mappedBuffer)); + data->metadataBuffers_[i]->setCookie(i); + } + return ret; +} + int PipelineHandlerUVC::exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) { @@ -247,20 +314,46 @@ int PipelineHandlerUVC::start(Camera *camera, [[maybe_unused]] const ControlList if (ret < 0) return ret; + if (data->useMetadataStream_) { + if (createMetadataBuffers(camera, count) < 0) { + LOG(UVC, Error) << "Unable to allocate buffers for UVC metadata stream."; + data->useMetadataStream_ = false; + } + } + ret = data->video_->streamOn(); if (ret < 0) { - data->video_->releaseBuffers(); + cleanup(camera); return ret; } + if (data->useMetadataStream_) { + ret = data->metadata_->streamOn(); + if (ret) { + LOG(UVC, Error) << "Failed to start metadata stream"; + return ret; + } + + for (std::unique_ptr &buf : data->metadataBuffers_) { + ret = data->metadata_->queueBuffer(buf.get()); + if (ret < 0) { + cleanupMetadataBuffers(camera); + return ret; + } + } + } return 0; } void PipelineHandlerUVC::stopDevice(Camera *camera) { UVCCameraData *data = cameraData(camera); + data->video_->streamOff(); - data->video_->releaseBuffers(); + + data->metadata_->streamOff(); + + cleanup(camera); } int PipelineHandlerUVC::processControl(ControlList *controls, unsigned int id,