{"id":21727,"url":"https://patchwork.libcamera.org/api/1.1/patches/21727/?format=json","web_url":"https://patchwork.libcamera.org/patch/21727/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20241022074544.3790451-4-chenghaoyang@chromium.org>","date":"2024-10-22T07:43:39","name":"[v16,3/7] libcamera: virtual: Add VirtualPipelineHandler","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"5e48d047bbc96d4151dad7421a24f888899ee75d","submitter":{"id":117,"url":"https://patchwork.libcamera.org/api/1.1/people/117/?format=json","name":"Cheng-Hao Yang","email":"chenghaoyang@chromium.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/21727/mbox/","series":[{"id":4733,"url":"https://patchwork.libcamera.org/api/1.1/series/4733/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4733","date":"2024-10-22T07:43:36","name":"Add VirtualPipelineHandler","version":16,"mbox":"https://patchwork.libcamera.org/series/4733/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/21727/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/21727/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id D949FC330B\n\tfor ;\n\tTue, 22 Oct 2024 07:46:03 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9D20565398;\n\tTue, 22 Oct 2024 09:46:02 +0200 (CEST)","from mail-pg1-x52e.google.com (mail-pg1-x52e.google.com\n\t[IPv6:2607:f8b0:4864:20::52e])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id DD10165391\n\tfor ;\n\tTue, 22 Oct 2024 09:45:56 +0200 (CEST)","by mail-pg1-x52e.google.com with SMTP id\n\t41be03b00d2f7-7ea79711fd4so3663531a12.0\n\tfor ;\n\tTue, 22 Oct 2024 00:45:56 -0700 (PDT)","from chenghaoyang-low.c.googlers.com.com\n\t(27.247.221.35.bc.googleusercontent.com. [35.221.247.27])\n\tby smtp.gmail.com with ESMTPSA id\n\td2e1a72fcca58-71ec13d75aesm4091024b3a.124.2024.10.22.00.45.53\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tTue, 22 Oct 2024 00:45:54 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"fIQb6OT7\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=chromium.org; s=google; t=1729583155; x=1730187955;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=MWtBQ9pknojJ8FNDwwaZSxktSCdoqhRmg9wuL32stmk=;\n\tb=fIQb6OT7zP5XKU3BMUt25C/KKalATBwf6eAaHX4r/9p5ZlhlzihrC75pQYleQnKjso\n\tKFrCRCnDpw2CZmsMOisQW42gdylGYB9X5hJCGScxddPXluwCxyPeuGvmbyUZBKHdda5n\n\teoXL8hhTNrTpiuRs+C1u28kDenPjQ946XR5/c=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1729583155; x=1730187955;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc\n\t:subject:date:message-id:reply-to;\n\tbh=MWtBQ9pknojJ8FNDwwaZSxktSCdoqhRmg9wuL32stmk=;\n\tb=jW5EqTyouZ72AriyRN+rgiAvxzfpnCVs+B5voBIeCVEFuE1IzF9F6it6A6ViJ1yVCg\n\twoluZFuVBpRcGTXNnMGc2ctl6GyQW7IDmRUKbkdGd5b8GZuJVdsPAnUpGcLV+4tXX84m\n\teLPY8ECvh+ksrYYJCtkR+0GTSPXgXgFkvZW3+Pd1eKprGXOtiUr/D8f3UbBMJpKI/sMY\n\t/IFTj4n73s5khO4EGbon27rUK1TnXFwot+1k19t5aOlb8j5xY5V5KRXlre8dz0x9ljox\n\tBr2o4hnBK9fAE3b+J0rJYWfQb8pQEDr64EIvxWNf903KRQgmNTZxdN5xq3Phl2hnlQgk\n\tS7Jw==","X-Gm-Message-State":"AOJu0YwSTEv21pi4NilYh00sCGiIpyHW8Dt/4cOD9CtYFQHtYDQhBP7M\n\tn1M8aTViFsFvoxHOlzgEDH7xILfHR8btyaYy8ofr3pJ/JHnggxs3keL9ZCC2ndtrexGj8+w/+2w\n\t=","X-Google-Smtp-Source":"AGHT+IEJDdo4uO7EUo8l+gmTL8ht3njecj531QCAjK/hYs1MsgC5Pu9WUr/XjHKBx4Shnuc3/Bvnkw==","X-Received":"by 2002:a05:6300:668a:b0:1d4:fcd0:5bea with SMTP id\n\tadf61e73a8af0-1d92c4baddbmr19498403637.6.1729583154813; \n\tTue, 22 Oct 2024 00:45:54 -0700 (PDT)","From":"Harvey Yang ","To":"libcamera-devel@lists.libcamera.org","Cc":"Harvey Yang ,\n\tJacopo Mondi ,\n\tKieran Bingham ","Subject":"[PATCH v16 3/7] libcamera: virtual: Add VirtualPipelineHandler","Date":"Tue, 22 Oct 2024 07:43:39 +0000","Message-ID":"<20241022074544.3790451-4-chenghaoyang@chromium.org>","X-Mailer":"git-send-email 2.47.0.105.g07ac214952-goog","In-Reply-To":"<20241022074544.3790451-1-chenghaoyang@chromium.org>","References":"<20241022074544.3790451-1-chenghaoyang@chromium.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Add VirtualPipelineHandler for more unit tests and verfiy libcamera\ninfrastructure works on devices without using hardware cameras.\n\nSigned-off-by: Harvey Yang \nReviewed-by: Jacopo Mondi \nReviewed-by: Kieran Bingham \n---\n meson.build | 1 +\n meson_options.txt | 3 +-\n src/libcamera/pipeline/virtual/meson.build | 5 +\n src/libcamera/pipeline/virtual/virtual.cpp | 337 +++++++++++++++++++++\n src/libcamera/pipeline/virtual/virtual.h | 45 +++\n 5 files changed, 390 insertions(+), 1 deletion(-)\n create mode 100644 src/libcamera/pipeline/virtual/meson.build\n create mode 100644 src/libcamera/pipeline/virtual/virtual.cpp\n create mode 100644 src/libcamera/pipeline/virtual/virtual.h","diff":"diff --git a/meson.build b/meson.build\nindex 63e45465d..5e533b0c3 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -214,6 +214,7 @@ pipelines_support = {\n 'simple': ['any'],\n 'uvcvideo': ['any'],\n 'vimc': ['test'],\n+ 'virtual': ['test'],\n }\n \n if pipelines.contains('all')\ndiff --git a/meson_options.txt b/meson_options.txt\nindex 7aa412491..c91cd241a 100644\n--- a/meson_options.txt\n+++ b/meson_options.txt\n@@ -53,7 +53,8 @@ option('pipelines',\n 'rpi/vc4',\n 'simple',\n 'uvcvideo',\n- 'vimc'\n+ 'vimc',\n+ 'virtual'\n ],\n description : 'Select which pipeline handlers to build. If this is set to \"auto\", all the pipelines applicable to the target architecture will be built. If this is set to \"all\", all the pipelines will be built. If both are selected then \"all\" will take precedence.')\n \ndiff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build\nnew file mode 100644\nindex 000000000..ada1b3358\n--- /dev/null\n+++ b/src/libcamera/pipeline/virtual/meson.build\n@@ -0,0 +1,5 @@\n+# SPDX-License-Identifier: CC0-1.0\n+\n+libcamera_internal_sources += files([\n+ 'virtual.cpp',\n+])\ndiff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp\nnew file mode 100644\nindex 000000000..13107874a\n--- /dev/null\n+++ b/src/libcamera/pipeline/virtual/virtual.cpp\n@@ -0,0 +1,337 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2024, Google Inc.\n+ *\n+ * Pipeline handler for virtual cameras\n+ */\n+\n+#include \"virtual.h\"\n+\n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+\n+#include \n+#include \n+\n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+\n+#include \"libcamera/internal/camera.h\"\n+#include \"libcamera/internal/dma_buf_allocator.h\"\n+#include \"libcamera/internal/formats.h\"\n+#include \"libcamera/internal/pipeline_handler.h\"\n+\n+namespace libcamera {\n+\n+LOG_DEFINE_CATEGORY(Virtual)\n+\n+namespace {\n+\n+uint64_t currentTimestamp()\n+{\n+\tconst auto now = std::chrono::steady_clock::now();\n+\tauto nsecs = std::chrono::duration_cast(\n+\t\tnow.time_since_epoch());\n+\n+\treturn nsecs.count();\n+}\n+\n+} /* namespace */\n+\n+class VirtualCameraConfiguration : public CameraConfiguration\n+{\n+public:\n+\tstatic constexpr unsigned int kBufferCount = 4;\n+\n+\tVirtualCameraConfiguration(VirtualCameraData *data);\n+\n+\tStatus validate() override;\n+\n+private:\n+\tconst VirtualCameraData *data_;\n+};\n+\n+class PipelineHandlerVirtual : public PipelineHandler\n+{\n+public:\n+\tPipelineHandlerVirtual(CameraManager *manager);\n+\t~PipelineHandlerVirtual();\n+\n+\tstd::unique_ptr generateConfiguration(Camera *camera,\n+\t\t\t\t\t\t\t\t Span roles) override;\n+\tint configure(Camera *camera, CameraConfiguration *config) override;\n+\n+\tint exportFrameBuffers(Camera *camera, Stream *stream,\n+\t\t\t std::vector> *buffers) override;\n+\n+\tint start(Camera *camera, const ControlList *controls) override;\n+\tvoid stopDevice(Camera *camera) override;\n+\n+\tint queueRequestDevice(Camera *camera, Request *request) override;\n+\n+\tbool match(DeviceEnumerator *enumerator) override;\n+\n+private:\n+\tstatic bool created_;\n+\n+\tVirtualCameraData *cameraData(Camera *camera)\n+\t{\n+\t\treturn static_cast(camera->_d());\n+\t}\n+\n+\tDmaBufAllocator dmaBufAllocator_;\n+\n+\tbool resetCreated_ = false;\n+};\n+\n+VirtualCameraData::VirtualCameraData(PipelineHandler *pipe,\n+\t\t\t\t const std::vector &supportedResolutions)\n+\t: Camera::Private(pipe), supportedResolutions_(supportedResolutions)\n+{\n+\tfor (const auto &resolution : supportedResolutions_) {\n+\t\tif (minResolutionSize_.isNull() || minResolutionSize_ > resolution.size)\n+\t\t\tminResolutionSize_ = resolution.size;\n+\n+\t\tmaxResolutionSize_ = std::max(maxResolutionSize_, resolution.size);\n+\t}\n+\n+\t/* \\todo Support multiple streams and pass multi_stream_test */\n+\tstreamConfigs_.resize(kMaxStream);\n+}\n+\n+VirtualCameraConfiguration::VirtualCameraConfiguration(VirtualCameraData *data)\n+\t: CameraConfiguration(), data_(data)\n+{\n+}\n+\n+CameraConfiguration::Status VirtualCameraConfiguration::validate()\n+{\n+\tStatus status = Valid;\n+\n+\tif (config_.empty()) {\n+\t\tLOG(Virtual, Error) << \"Empty config\";\n+\t\treturn Invalid;\n+\t}\n+\n+\t/* Only one stream is supported */\n+\tif (config_.size() > VirtualCameraData::kMaxStream) {\n+\t\tconfig_.resize(VirtualCameraData::kMaxStream);\n+\t\tstatus = Adjusted;\n+\t}\n+\n+\tfor (StreamConfiguration &cfg : config_) {\n+\t\tbool adjusted = false;\n+\t\tbool found = false;\n+\t\tfor (const auto &resolution : data_->supportedResolutions_) {\n+\t\t\tif (resolution.size.width == cfg.size.width &&\n+\t\t\t resolution.size.height == cfg.size.height) {\n+\t\t\t\tfound = true;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (!found) {\n+\t\t\t/*\n+\t\t\t * \\todo It's a pipeline's decision to choose a\n+\t\t\t * resolution when the exact one is not supported.\n+\t\t\t * Defining the default logic in PipelineHandler to\n+\t\t\t * find the closest resolution would be nice.\n+\t\t\t */\n+\t\t\tcfg.size = data_->maxResolutionSize_;\n+\t\t\tstatus = Adjusted;\n+\t\t\tadjusted = true;\n+\t\t}\n+\n+\t\tif (cfg.pixelFormat != formats::NV12) {\n+\t\t\tcfg.pixelFormat = formats::NV12;\n+\t\t\tstatus = Adjusted;\n+\t\t\tadjusted = true;\n+\t\t}\n+\n+\t\tif (adjusted)\n+\t\t\tLOG(Virtual, Info)\n+\t\t\t\t<< \"Stream configuration adjusted to \" << cfg.toString();\n+\n+\t\tconst PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat);\n+\t\tcfg.stride = info.stride(cfg.size.width, 0, 1);\n+\t\tcfg.frameSize = info.frameSize(cfg.size, 1);\n+\n+\t\tcfg.bufferCount = VirtualCameraConfiguration::kBufferCount;\n+\t}\n+\n+\treturn status;\n+}\n+\n+/* static */\n+bool PipelineHandlerVirtual::created_ = false;\n+\n+PipelineHandlerVirtual::PipelineHandlerVirtual(CameraManager *manager)\n+\t: PipelineHandler(manager),\n+\t dmaBufAllocator_(DmaBufAllocator::DmaBufAllocatorFlag::CmaHeap |\n+\t\t\t DmaBufAllocator::DmaBufAllocatorFlag::SystemHeap |\n+\t\t\t DmaBufAllocator::DmaBufAllocatorFlag::UDmaBuf)\n+{\n+}\n+\n+PipelineHandlerVirtual::~PipelineHandlerVirtual()\n+{\n+\tif (resetCreated_)\n+\t\tcreated_ = false;\n+}\n+\n+std::unique_ptr\n+PipelineHandlerVirtual::generateConfiguration(Camera *camera,\n+\t\t\t\t\t Span roles)\n+{\n+\tVirtualCameraData *data = cameraData(camera);\n+\tauto config = std::make_unique(data);\n+\n+\tif (roles.empty())\n+\t\treturn config;\n+\n+\tfor (const StreamRole role : roles) {\n+\t\tswitch (role) {\n+\t\tcase StreamRole::StillCapture:\n+\t\tcase StreamRole::VideoRecording:\n+\t\tcase StreamRole::Viewfinder:\n+\t\t\tbreak;\n+\n+\t\tcase StreamRole::Raw:\n+\t\tdefault:\n+\t\t\tLOG(Virtual, Error)\n+\t\t\t\t<< \"Requested stream role not supported: \" << role;\n+\t\t\tconfig.reset();\n+\t\t\treturn config;\n+\t\t}\n+\n+\t\tstd::map> streamFormats;\n+\t\tPixelFormat pixelFormat = formats::NV12;\n+\t\tstreamFormats[pixelFormat] = { { data->minResolutionSize_,\n+\t\t\t\t\t\t data->maxResolutionSize_ } };\n+\t\tStreamFormats formats(streamFormats);\n+\t\tStreamConfiguration cfg(formats);\n+\t\tcfg.pixelFormat = pixelFormat;\n+\t\tcfg.size = data->maxResolutionSize_;\n+\t\tcfg.bufferCount = VirtualCameraConfiguration::kBufferCount;\n+\n+\t\tconfig->addConfiguration(cfg);\n+\t}\n+\n+\tASSERT(config->validate() != CameraConfiguration::Invalid);\n+\n+\treturn config;\n+}\n+\n+int PipelineHandlerVirtual::configure(Camera *camera,\n+\t\t\t\t CameraConfiguration *config)\n+{\n+\tVirtualCameraData *data = cameraData(camera);\n+\tfor (auto [i, c] : utils::enumerate(*config))\n+\t\tc.setStream(&data->streamConfigs_[i].stream);\n+\n+\treturn 0;\n+}\n+\n+int PipelineHandlerVirtual::exportFrameBuffers([[maybe_unused]] Camera *camera,\n+\t\t\t\t\t Stream *stream,\n+\t\t\t\t\t std::vector> *buffers)\n+{\n+\tif (!dmaBufAllocator_.isValid())\n+\t\treturn -ENOBUFS;\n+\n+\tconst StreamConfiguration &config = stream->configuration();\n+\n+\tauto info = PixelFormatInfo::info(config.pixelFormat);\n+\n+\tstd::vector planeSizes;\n+\tfor (size_t i = 0; i < info.planes.size(); ++i)\n+\t\tplaneSizes.push_back(info.planeSize(config.size, i));\n+\n+\treturn dmaBufAllocator_.exportBuffers(config.bufferCount, planeSizes, buffers);\n+}\n+\n+int PipelineHandlerVirtual::start([[maybe_unused]] Camera *camera,\n+\t\t\t\t [[maybe_unused]] const ControlList *controls)\n+{\n+\treturn 0;\n+}\n+\n+void PipelineHandlerVirtual::stopDevice([[maybe_unused]] Camera *camera)\n+{\n+}\n+\n+int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera,\n+\t\t\t\t\t Request *request)\n+{\n+\tfor (auto it : request->buffers())\n+\t\tcompleteBuffer(request, it.second);\n+\n+\trequest->metadata().set(controls::SensorTimestamp, currentTimestamp());\n+\tcompleteRequest(request);\n+\n+\treturn 0;\n+}\n+\n+bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator)\n+{\n+\tif (created_)\n+\t\treturn false;\n+\n+\tcreated_ = true;\n+\n+\t/* \\todo Add virtual cameras according to a config file. */\n+\n+\tstd::vector supportedResolutions;\n+\tsupportedResolutions.resize(2);\n+\tsupportedResolutions[0] = { .size = Size(1920, 1080), .frameRates = { 30 } };\n+\tsupportedResolutions[1] = { .size = Size(1280, 720), .frameRates = { 30 } };\n+\n+\tstd::unique_ptr data =\n+\t\tstd::make_unique(this, supportedResolutions);\n+\n+\tdata->properties_.set(properties::Location, properties::CameraLocationFront);\n+\tdata->properties_.set(properties::Model, \"Virtual Video Device\");\n+\tdata->properties_.set(properties::PixelArrayActiveAreas, { Rectangle(Size(1920, 1080)) });\n+\n+\t/* \\todo Set FrameDurationLimits based on config. */\n+\tControlInfoMap::Map controls;\n+\tint64_t min_frame_duration = 33333, max_frame_duration = 33333;\n+\tcontrols[&controls::FrameDurationLimits] = ControlInfo(min_frame_duration, max_frame_duration);\n+\tstd::vector supportedFaceDetectModes{\n+\t\tstatic_cast(controls::draft::FaceDetectModeOff),\n+\t};\n+\tcontrols[&controls::draft::FaceDetectMode] = ControlInfo(supportedFaceDetectModes);\n+\tdata->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls);\n+\n+\t/* Create and register the camera. */\n+\tstd::set streams;\n+\tfor (auto &streamConfig : data->streamConfigs_)\n+\t\tstreams.insert(&streamConfig.stream);\n+\n+\tconst std::string id = \"Virtual0\";\n+\tstd::shared_ptr camera = Camera::create(std::move(data), id, streams);\n+\tregisterCamera(std::move(camera));\n+\n+\tresetCreated_ = true;\n+\n+\treturn true;\n+}\n+\n+REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual, \"virtual\")\n+\n+} /* namespace libcamera */\ndiff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h\nnew file mode 100644\nindex 000000000..f6cacd277\n--- /dev/null\n+++ b/src/libcamera/pipeline/virtual/virtual.h\n@@ -0,0 +1,45 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2024, Google Inc.\n+ *\n+ * Pipeline handler for virtual cameras\n+ */\n+\n+#pragma once\n+\n+#include \n+\n+#include \n+#include \n+\n+#include \"libcamera/internal/camera.h\"\n+#include \"libcamera/internal/pipeline_handler.h\"\n+\n+namespace libcamera {\n+\n+class VirtualCameraData : public Camera::Private\n+{\n+public:\n+\tconst static unsigned int kMaxStream = 3;\n+\n+\tstruct Resolution {\n+\t\tSize size;\n+\t\tstd::vector frameRates;\n+\t};\n+\tstruct StreamConfig {\n+\t\tStream stream;\n+\t};\n+\n+\tVirtualCameraData(PipelineHandler *pipe,\n+\t\t\t const std::vector &supportedResolutions);\n+\n+\t~VirtualCameraData() = default;\n+\n+\tconst std::vector supportedResolutions_;\n+\tSize maxResolutionSize_;\n+\tSize minResolutionSize_;\n+\n+\tstd::vector streamConfigs_;\n+};\n+\n+} /* namespace libcamera */\n","prefixes":["v16","3/7"]}