{"id":21387,"url":"https://patchwork.libcamera.org/api/1.1/patches/21387/?format=json","web_url":"https://patchwork.libcamera.org/patch/21387/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240926163141.1593026-4-chenghaoyang@google.com>","date":"2024-09-26T16:29:52","name":"[v13,3/7] libcamera: virtual: Add VirtualPipelineHandler","commit_ref":null,"pull_url":null,"state":"superseded","archived":true,"hash":"d58339ab936d4531fad10767a516a4a2c8954dfb","submitter":{"id":117,"url":"https://patchwork.libcamera.org/api/1.1/people/117/?format=json","name":"Cheng-Hao Yang","email":"chenghaoyang@chromium.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/21387/mbox/","series":[{"id":4632,"url":"https://patchwork.libcamera.org/api/1.1/series/4632/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4632","date":"2024-09-26T16:29:49","name":"Add VirtualPipelineHandler","version":13,"mbox":"https://patchwork.libcamera.org/series/4632/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/21387/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/21387/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 6F8CFC3257\n\tfor ;\n\tThu, 26 Sep 2024 16:31:57 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A497A63514;\n\tThu, 26 Sep 2024 18:31:55 +0200 (CEST)","from mail-pl1-x634.google.com (mail-pl1-x634.google.com\n\t[IPv6:2607:f8b0:4864:20::634])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id EE7C963502\n\tfor ;\n\tThu, 26 Sep 2024 18:31:51 +0200 (CEST)","by mail-pl1-x634.google.com with SMTP id\n\td9443c01a7336-2055136b612so14679115ad.0\n\tfor ;\n\tThu, 26 Sep 2024 09:31:51 -0700 (PDT)","from chenghaoyang-low.c.googlers.com.com\n\t(208.158.221.35.bc.googleusercontent.com. [35.221.158.208])\n\tby smtp.gmail.com with ESMTPSA id\n\td9443c01a7336-20b37e20dffsm469035ad.170.2024.09.26.09.31.48\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tThu, 26 Sep 2024 09:31:49 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"UGCPeaPD\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=chromium.org; s=google; t=1727368310; x=1727973110;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=wSXvEpeAq88KA7GGtwleKftwpSAO3e16Y5ff6zIFlLY=;\n\tb=UGCPeaPDWkjY3F07oMZPCfQj+OXBHOHQM6LmwAESR3LUTbiduF2A/cR0oGPoQPNNmL\n\tPj1G92Vx57wBATKSYQAKBntsgk/OridmAUk686iwDvfK3M30wO6DtwMRx0eNMesaYuyR\n\twSs9YGWCRSptzxYbosQ0x6MrsiAOP3tm15AnQ=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1727368310; x=1727973110;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc\n\t:subject:date:message-id:reply-to;\n\tbh=wSXvEpeAq88KA7GGtwleKftwpSAO3e16Y5ff6zIFlLY=;\n\tb=MVqVcr3ppRTiU6A/HF56ksLGvbIny/3L+sHiaDO0B+2yTCwsMAEnGy7S7C/apkaVmj\n\ttfUYyMB0hBbBmXVNXn/uTiPq163UepbXICCBIshVGaxh1osST6U2wJqYhIA1tf39LFeS\n\tDuHWvjEjMwBzTdw6utJtPv9RhzTwNuIMS3U9T/dqsYoXD2qnwTFSypL1BWpAG5jSrM88\n\tMHujBuImHEYIPOFnFqpGkIMMmJ0sTQki88uczIDt3KTefko8yC5ehHOHGzhVOlWNbjV2\n\teV5Gp94ef04lgStEAJoUxazGaCFCm70zlT5oWeUk/q4Q1D/TENjDifqznGCY1aGWFUi3\n\tuMPQ==","X-Gm-Message-State":"AOJu0YyBNwTM6A3Sgjds8TIDR6eTqSuZXbkyj8gajI+lNKSsVbArnsCF\n\tXxHdcnsEyAA3MLsbaPgty8fuYjYwvzK3OQ/GQ2rrw4RAQIK35NXLvk0DoIQovyoVIZpq/SqwYbc\n\t=","X-Google-Smtp-Source":"AGHT+IEo34OBfTgUk9jeA+Dr5uI5znODPn4fVn1dM3l4x1HfDx6x8COrs5XWGGvN7otW5B8evF9ufA==","X-Received":"by 2002:a17:903:2302:b0:207:1709:dbe with SMTP id\n\td9443c01a7336-20b37b942c2mr4156835ad.50.1727368310079; \n\tThu, 26 Sep 2024 09:31:50 -0700 (PDT)","From":"Harvey Yang ","X-Google-Original-From":"Harvey Yang ","To":"libcamera-devel@lists.libcamera.org","Cc":"Harvey Yang ,\n\tJacopo Mondi ","Subject":"[PATCH v13 3/7] libcamera: virtual: Add VirtualPipelineHandler","Date":"Thu, 26 Sep 2024 16:29:52 +0000","Message-ID":"<20240926163141.1593026-4-chenghaoyang@google.com>","X-Mailer":"git-send-email 2.46.1.824.gd892dcdcdd-goog","In-Reply-To":"<20240926163141.1593026-1-chenghaoyang@google.com>","References":"<20240926163141.1593026-1-chenghaoyang@google.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"From: Harvey Yang \n\nAdd VirtualPipelineHandler for more unit tests and verfiy libcamera\ninfrastructure works on devices without using hardware cameras.\n\nSigned-off-by: Harvey Yang \nReviewed-by: Jacopo Mondi \n---\n meson.build | 1 +\n meson_options.txt | 3 +-\n src/libcamera/pipeline/virtual/meson.build | 5 +\n src/libcamera/pipeline/virtual/virtual.cpp | 317 +++++++++++++++++++++\n src/libcamera/pipeline/virtual/virtual.h | 45 +++\n 5 files changed, 370 insertions(+), 1 deletion(-)\n create mode 100644 src/libcamera/pipeline/virtual/meson.build\n create mode 100644 src/libcamera/pipeline/virtual/virtual.cpp\n create mode 100644 src/libcamera/pipeline/virtual/virtual.h","diff":"diff --git a/meson.build b/meson.build\nindex 63e45465..5e533b0c 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -214,6 +214,7 @@ pipelines_support = {\n 'simple': ['any'],\n 'uvcvideo': ['any'],\n 'vimc': ['test'],\n+ 'virtual': ['test'],\n }\n \n if pipelines.contains('all')\ndiff --git a/meson_options.txt b/meson_options.txt\nindex 7aa41249..c91cd241 100644\n--- a/meson_options.txt\n+++ b/meson_options.txt\n@@ -53,7 +53,8 @@ option('pipelines',\n 'rpi/vc4',\n 'simple',\n 'uvcvideo',\n- 'vimc'\n+ 'vimc',\n+ 'virtual'\n ],\n description : 'Select which pipeline handlers to build. If this is set to \"auto\", all the pipelines applicable to the target architecture will be built. If this is set to \"all\", all the pipelines will be built. If both are selected then \"all\" will take precedence.')\n \ndiff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build\nnew file mode 100644\nindex 00000000..ada1b335\n--- /dev/null\n+++ b/src/libcamera/pipeline/virtual/meson.build\n@@ -0,0 +1,5 @@\n+# SPDX-License-Identifier: CC0-1.0\n+\n+libcamera_internal_sources += files([\n+ 'virtual.cpp',\n+])\ndiff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp\nnew file mode 100644\nindex 00000000..d1584f59\n--- /dev/null\n+++ b/src/libcamera/pipeline/virtual/virtual.cpp\n@@ -0,0 +1,317 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2024, Google Inc.\n+ *\n+ * virtual.cpp - Pipeline handler for virtual cameras\n+ */\n+\n+#include \"virtual.h\"\n+\n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+\n+#include \n+#include \n+\n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+\n+#include \"libcamera/internal/camera.h\"\n+#include \"libcamera/internal/dma_buf_allocator.h\"\n+#include \"libcamera/internal/formats.h\"\n+#include \"libcamera/internal/pipeline_handler.h\"\n+\n+namespace libcamera {\n+\n+LOG_DEFINE_CATEGORY(Virtual)\n+\n+namespace {\n+\n+uint64_t currentTimestamp()\n+{\n+\tconst auto now = std::chrono::steady_clock::now();\n+\tauto nsecs = std::chrono::duration_cast(\n+\t\tnow.time_since_epoch());\n+\n+\treturn nsecs.count();\n+}\n+\n+} /* namespace */\n+\n+class VirtualCameraConfiguration : public CameraConfiguration\n+{\n+public:\n+\tstatic constexpr unsigned int kBufferCount = 4;\n+\n+\tVirtualCameraConfiguration(VirtualCameraData *data);\n+\n+\tStatus validate() override;\n+\n+private:\n+\tconst VirtualCameraData *data_;\n+};\n+\n+class PipelineHandlerVirtual : public PipelineHandler\n+{\n+public:\n+\tPipelineHandlerVirtual(CameraManager *manager);\n+\n+\tstd::unique_ptr generateConfiguration(Camera *camera,\n+\t\t\t\t\t\t\t\t Span roles) override;\n+\tint configure(Camera *camera, CameraConfiguration *config) override;\n+\n+\tint exportFrameBuffers(Camera *camera, Stream *stream,\n+\t\t\t std::vector> *buffers) override;\n+\n+\tint start(Camera *camera, const ControlList *controls) override;\n+\tvoid stopDevice(Camera *camera) override;\n+\n+\tint queueRequestDevice(Camera *camera, Request *request) override;\n+\n+\tbool match(DeviceEnumerator *enumerator) override;\n+\n+private:\n+\tstatic bool created_;\n+\n+\tVirtualCameraData *cameraData(Camera *camera)\n+\t{\n+\t\treturn static_cast(camera->_d());\n+\t}\n+\n+\tDmaBufAllocator dmaBufAllocator_;\n+};\n+\n+/* static */\n+bool PipelineHandlerVirtual::created_ = false;\n+\n+VirtualCameraData::VirtualCameraData(PipelineHandler *pipe,\n+\t\t\t\t const std::vector &supportedResolutions)\n+\t: Camera::Private(pipe), supportedResolutions_(supportedResolutions)\n+{\n+\tfor (const auto &resolution : supportedResolutions_) {\n+\t\tif (minResolutionSize_.isNull() || minResolutionSize_ > resolution.size)\n+\t\t\tminResolutionSize_ = resolution.size;\n+\n+\t\tmaxResolutionSize_ = std::max(maxResolutionSize_, resolution.size);\n+\t}\n+\n+\t/* \\todo Support multiple streams and pass multi_stream_test */\n+\tstreamConfigs_.resize(kMaxStream);\n+}\n+\n+VirtualCameraConfiguration::VirtualCameraConfiguration(VirtualCameraData *data)\n+\t: CameraConfiguration(), data_(data)\n+{\n+}\n+\n+CameraConfiguration::Status VirtualCameraConfiguration::validate()\n+{\n+\tStatus status = Valid;\n+\n+\tif (config_.empty()) {\n+\t\tLOG(Virtual, Error) << \"Empty config\";\n+\t\treturn Invalid;\n+\t}\n+\n+\t/* Only one stream is supported */\n+\tif (config_.size() > VirtualCameraData::kMaxStream) {\n+\t\tconfig_.resize(VirtualCameraData::kMaxStream);\n+\t\tstatus = Adjusted;\n+\t}\n+\n+\tfor (StreamConfiguration &cfg : config_) {\n+\t\tbool found = false;\n+\t\tfor (const auto &resolution : data_->supportedResolutions_) {\n+\t\t\tif (resolution.size.width == cfg.size.width &&\n+\t\t\t resolution.size.height == cfg.size.height) {\n+\t\t\t\tfound = true;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (!found) {\n+\t\t\t/*\n+\t\t\t * \\todo It's a pipeline's decision to choose a\n+\t\t\t * resolution when the exact one is not supported.\n+\t\t\t * Defining the default logic in PipelineHandler to\n+\t\t\t * find the closest resolution would be nice.\n+\t\t\t */\n+\t\t\tcfg.size = data_->maxResolutionSize_;\n+\t\t\tstatus = Adjusted;\n+\t\t}\n+\n+\t\tif (cfg.pixelFormat != formats::NV12) {\n+\t\t\tcfg.pixelFormat = formats::NV12;\n+\t\t\tLOG(Virtual, Debug)\n+\t\t\t\t<< \"Stream configuration adjusted to \" << cfg.toString();\n+\t\t\tstatus = Adjusted;\n+\t\t}\n+\n+\t\tconst PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat);\n+\t\tcfg.stride = info.stride(cfg.size.width, 0, 1);\n+\t\tcfg.frameSize = info.frameSize(cfg.size, 1);\n+\n+\t\tcfg.bufferCount = VirtualCameraConfiguration::kBufferCount;\n+\t}\n+\n+\treturn status;\n+}\n+\n+PipelineHandlerVirtual::PipelineHandlerVirtual(CameraManager *manager)\n+\t: PipelineHandler(manager),\n+\t dmaBufAllocator_(DmaBufAllocator::DmaBufAllocatorFlag::CmaHeap |\n+\t\t\t DmaBufAllocator::DmaBufAllocatorFlag::SystemHeap |\n+\t\t\t DmaBufAllocator::DmaBufAllocatorFlag::UDmaBuf)\n+{\n+}\n+\n+std::unique_ptr\n+PipelineHandlerVirtual::generateConfiguration(Camera *camera,\n+\t\t\t\t\t Span roles)\n+{\n+\tVirtualCameraData *data = cameraData(camera);\n+\tauto config = std::make_unique(data);\n+\n+\tif (roles.empty())\n+\t\treturn config;\n+\n+\tfor (const StreamRole role : roles) {\n+\t\tswitch (role) {\n+\t\tcase StreamRole::StillCapture:\n+\t\tcase StreamRole::VideoRecording:\n+\t\tcase StreamRole::Viewfinder:\n+\t\t\tbreak;\n+\n+\t\tcase StreamRole::Raw:\n+\t\tdefault:\n+\t\t\tLOG(Virtual, Error)\n+\t\t\t\t<< \"Requested stream role not supported: \" << role;\n+\t\t\tconfig.reset();\n+\t\t\treturn config;\n+\t\t}\n+\n+\t\tstd::map> streamFormats;\n+\t\tPixelFormat pixelFormat = formats::NV12;\n+\t\tstreamFormats[pixelFormat] = { { data->minResolutionSize_,\n+\t\t\t\t\t\t data->maxResolutionSize_ } };\n+\t\tStreamFormats formats(streamFormats);\n+\t\tStreamConfiguration cfg(formats);\n+\t\tcfg.pixelFormat = pixelFormat;\n+\t\tcfg.size = data->maxResolutionSize_;\n+\t\tcfg.bufferCount = VirtualCameraConfiguration::kBufferCount;\n+\n+\t\tconfig->addConfiguration(cfg);\n+\t}\n+\n+\tASSERT(config->validate() != CameraConfiguration::Invalid);\n+\n+\treturn config;\n+}\n+\n+int PipelineHandlerVirtual::configure(Camera *camera,\n+\t\t\t\t CameraConfiguration *config)\n+{\n+\tVirtualCameraData *data = cameraData(camera);\n+\tfor (auto [i, c] : utils::enumerate(*config))\n+\t\tc.setStream(&data->streamConfigs_[i].stream);\n+\n+\treturn 0;\n+}\n+\n+int PipelineHandlerVirtual::exportFrameBuffers([[maybe_unused]] Camera *camera,\n+\t\t\t\t\t Stream *stream,\n+\t\t\t\t\t std::vector> *buffers)\n+{\n+\tif (!dmaBufAllocator_.isValid())\n+\t\treturn -ENOBUFS;\n+\n+\tconst StreamConfiguration &config = stream->configuration();\n+\n+\tauto info = PixelFormatInfo::info(config.pixelFormat);\n+\n+\tstd::vector planeSizes;\n+\tfor (size_t i = 0; i < info.planes.size(); ++i)\n+\t\tplaneSizes.push_back(info.planeSize(config.size, i));\n+\n+\treturn dmaBufAllocator_.exportBuffers(config.bufferCount, planeSizes, buffers);\n+}\n+\n+int PipelineHandlerVirtual::start([[maybe_unused]] Camera *camera,\n+\t\t\t\t [[maybe_unused]] const ControlList *controls)\n+{\n+\treturn 0;\n+}\n+\n+void PipelineHandlerVirtual::stopDevice([[maybe_unused]] Camera *camera)\n+{\n+}\n+\n+int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera,\n+\t\t\t\t\t Request *request)\n+{\n+\tfor (auto it : request->buffers())\n+\t\tcompleteBuffer(request, it.second);\n+\n+\trequest->metadata().set(controls::SensorTimestamp, currentTimestamp());\n+\tcompleteRequest(request);\n+\n+\treturn 0;\n+}\n+\n+bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator)\n+{\n+\tif (created_)\n+\t\treturn false;\n+\n+\tcreated_ = true;\n+\n+\t/* \\todo Add virtual cameras according to a config file. */\n+\n+\tstd::vector supportedResolutions;\n+\tsupportedResolutions.resize(2);\n+\tsupportedResolutions[0] = { .size = Size(1920, 1080), .frameRates = { 30 } };\n+\tsupportedResolutions[1] = { .size = Size(1280, 720), .frameRates = { 30 } };\n+\n+\tstd::unique_ptr data =\n+\t\tstd::make_unique(this, supportedResolutions);\n+\n+\tdata->properties_.set(properties::Location, properties::CameraLocationFront);\n+\tdata->properties_.set(properties::Model, \"Virtual Video Device\");\n+\tdata->properties_.set(properties::PixelArrayActiveAreas, { Rectangle(Size(1920, 1080)) });\n+\n+\t/* \\todo Set FrameDurationLimits based on config. */\n+\tControlInfoMap::Map controls;\n+\tint64_t min_frame_duration = 33333, max_frame_duration = 33333;\n+\tcontrols[&controls::FrameDurationLimits] = ControlInfo(min_frame_duration, max_frame_duration);\n+\tdata->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls);\n+\n+\t/* Create and register the camera. */\n+\tstd::set streams;\n+\tfor (auto &streamConfig : data->streamConfigs_)\n+\t\tstreams.insert(&streamConfig.stream);\n+\n+\tconst std::string id = \"Virtual0\";\n+\tstd::shared_ptr camera = Camera::create(std::move(data), id, streams);\n+\tregisterCamera(std::move(camera));\n+\n+\treturn true;\n+}\n+\n+REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual, \"virtual\")\n+\n+} /* namespace libcamera */\ndiff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h\nnew file mode 100644\nindex 00000000..4df70a13\n--- /dev/null\n+++ b/src/libcamera/pipeline/virtual/virtual.h\n@@ -0,0 +1,45 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2024, Google Inc.\n+ *\n+ * virtual.h - Pipeline handler for virtual cameras\n+ */\n+\n+#pragma once\n+\n+#include \n+\n+#include \n+#include \n+\n+#include \"libcamera/internal/camera.h\"\n+#include \"libcamera/internal/pipeline_handler.h\"\n+\n+namespace libcamera {\n+\n+class VirtualCameraData : public Camera::Private\n+{\n+public:\n+\tconst static unsigned int kMaxStream = 1;\n+\n+\tstruct Resolution {\n+\t\tSize size;\n+\t\tstd::vector frameRates;\n+\t};\n+\tstruct StreamConfig {\n+\t\tStream stream;\n+\t};\n+\n+\tVirtualCameraData(PipelineHandler *pipe,\n+\t\t\t const std::vector &supportedResolutions);\n+\n+\t~VirtualCameraData() = default;\n+\n+\tconst std::vector supportedResolutions_;\n+\tSize maxResolutionSize_;\n+\tSize minResolutionSize_;\n+\n+\tstd::vector streamConfigs_;\n+};\n+\n+} /* namespace libcamera */\n","prefixes":["v13","3/7"]}