{"id":20704,"url":"https://patchwork.libcamera.org/api/1.1/patches/20704/?format=json","web_url":"https://patchwork.libcamera.org/patch/20704/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240726095626.3632402-4-chenghaoyang@chromium.org>","date":"2024-07-26T09:54:24","name":"[v6,3/7] libcamera: pipeline: Add VirtualPipelineHandler","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"975e8fd85f8f918e5d4f834068588724ba20b220","submitter":{"id":117,"url":"https://patchwork.libcamera.org/api/1.1/people/117/?format=json","name":"Cheng-Hao Yang","email":"chenghaoyang@chromium.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/20704/mbox/","series":[{"id":4461,"url":"https://patchwork.libcamera.org/api/1.1/series/4461/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4461","date":"2024-07-26T09:54:21","name":"Add VirtualPipelineHandler","version":6,"mbox":"https://patchwork.libcamera.org/series/4461/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/20704/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/20704/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id BABE3BDC71\n\tfor ;\n\tFri, 26 Jul 2024 09:56:44 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 291426337A;\n\tFri, 26 Jul 2024 11:56:44 +0200 (CEST)","from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com\n\t[IPv6:2607:f8b0:4864:20::62a])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id EBA7A63375\n\tfor ;\n\tFri, 26 Jul 2024 11:56:36 +0200 (CEST)","by mail-pl1-x62a.google.com with SMTP id\n\td9443c01a7336-1fc611a0f8cso3973675ad.2\n\tfor ;\n\tFri, 26 Jul 2024 02:56:36 -0700 (PDT)","from chenghaoyang-low.c.googlers.com.com\n\t(148.175.199.104.bc.googleusercontent.com. [104.199.175.148])\n\tby smtp.gmail.com with ESMTPSA id\n\td9443c01a7336-1fed7c7f63asm28396665ad.46.2024.07.26.02.56.33\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tFri, 26 Jul 2024 02:56:34 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"Kaufsj4x\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=chromium.org; s=google; t=1721987795; x=1722592595;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=U7UDgPSResdxH60CPhSb69JHSSLCWSM4C/bJvUhU+YA=;\n\tb=Kaufsj4xkzlwsTmR0gBH/POup+Q67o3wgigFhBZIFoqC6GfnaNgWy+2Dz5Dtwgm1hl\n\tN1GG5EeWQuD1l4kWYW1VIGEaVX8v81bpjQou0hoTOWvqa4Fbpldd9ERmeSrdt5LJsn+5\n\tcD0BW06CBWTOlr4xXRb71loy9An85K48f8aRw=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1721987795; x=1722592595;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc\n\t:subject:date:message-id:reply-to;\n\tbh=U7UDgPSResdxH60CPhSb69JHSSLCWSM4C/bJvUhU+YA=;\n\tb=GSaUZqlntHe1jfFd17V+xylVevMPaj1fvv5h3IBlRGRH6Lg/O3AEmeWUQQbogedbnl\n\t8m/1uDeSFznhENRspW2Noh+NsWjsmJkUkpe/OlqDEM6oCK1iYKg+dlle6f5AJcD8Xi9y\n\tCNtRzjcB0pO+z2RwV6Mcqkx7D+ydfUzk7W4cnqK5U2kO7+PZocuKEHVBvF4DmmsDmYaB\n\tMdOH69+QukS32Ppl3s5dEJDZ8HOvsw8rTtoAHw0k/lllv82FVUkzV6rdmBx/VAkuTcDB\n\tfrQGiW+2S4uCudmbNZHXAovxBkFQrM34vD78wcdzNk5A5Gi6r19sBybPDk7w3VY87dRt\n\tVfdA==","X-Gm-Message-State":"AOJu0Yyx4eXl+eZ8dQ6+PYolm59+pxhBXMvLcLYZQ67b04aOTmAFx3Gh\n\tuKzdRkh6DUV1QmiDDAtFRRqzlR8sivCU4pPGPFuoMqcJJlEm0DrYmstiNwhqT6TxyVsJQSZYxSP\n\t/Zg==","X-Google-Smtp-Source":"AGHT+IGseH6Sbg8cSj+TKnUD9oMS6+zGDQyhQeQNn6Zc2IuiLDMZ5dJwzZG1EC0xesnaF1xvNeyF4w==","X-Received":"by 2002:a17:902:d2cc:b0:1fd:a980:5b77 with SMTP id\n\td9443c01a7336-1fed9280406mr60411475ad.36.1721987794944; \n\tFri, 26 Jul 2024 02:56:34 -0700 (PDT)","From":"Harvey Yang ","To":"libcamera-devel@lists.libcamera.org","Cc":"Harvey Yang ","Subject":"[PATCH v6 3/7] libcamera: pipeline: Add VirtualPipelineHandler","Date":"Fri, 26 Jul 2024 09:54:24 +0000","Message-ID":"<20240726095626.3632402-4-chenghaoyang@chromium.org>","X-Mailer":"git-send-email 2.46.0.rc1.232.g9752f9e123-goog","In-Reply-To":"<20240726095626.3632402-1-chenghaoyang@chromium.org>","References":"<20240726095626.3632402-1-chenghaoyang@chromium.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Add VirtualPipelineHandler for more unit tests and verfiy libcamera\ninfrastructure works on devices without using hardware cameras.\n\nSigned-off-by: Harvey Yang \n\nChange-Id: I0850d67e0a36bd3d1ba66f46edeac333633df6f7\n---\n meson.build | 1 +\n meson_options.txt | 3 +-\n src/libcamera/pipeline/virtual/meson.build | 5 +\n src/libcamera/pipeline/virtual/virtual.cpp | 243 +++++++++++++++++++++\n src/libcamera/pipeline/virtual/virtual.h | 78 +++++++\n 5 files changed, 329 insertions(+), 1 deletion(-)\n create mode 100644 src/libcamera/pipeline/virtual/meson.build\n create mode 100644 src/libcamera/pipeline/virtual/virtual.cpp\n create mode 100644 src/libcamera/pipeline/virtual/virtual.h","diff":"diff --git a/meson.build b/meson.build\nindex 2acd8c3e8..b5475fc74 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -213,6 +213,7 @@ pipelines_support = {\n 'simple': arch_arm,\n 'uvcvideo': ['any'],\n 'vimc': ['test'],\n+ 'virtual': ['test'],\n }\n \n if pipelines.contains('all')\ndiff --git a/meson_options.txt b/meson_options.txt\nindex 7aa412491..c91cd241a 100644\n--- a/meson_options.txt\n+++ b/meson_options.txt\n@@ -53,7 +53,8 @@ option('pipelines',\n 'rpi/vc4',\n 'simple',\n 'uvcvideo',\n- 'vimc'\n+ 'vimc',\n+ 'virtual'\n ],\n description : 'Select which pipeline handlers to build. If this is set to \"auto\", all the pipelines applicable to the target architecture will be built. If this is set to \"all\", all the pipelines will be built. If both are selected then \"all\" will take precedence.')\n \ndiff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build\nnew file mode 100644\nindex 000000000..ba7ff754e\n--- /dev/null\n+++ b/src/libcamera/pipeline/virtual/meson.build\n@@ -0,0 +1,5 @@\n+# SPDX-License-Identifier: CC0-1.0\n+\n+libcamera_sources += files([\n+ 'virtual.cpp',\n+])\ndiff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp\nnew file mode 100644\nindex 000000000..8abb74b48\n--- /dev/null\n+++ b/src/libcamera/pipeline/virtual/virtual.cpp\n@@ -0,0 +1,243 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2023, Google Inc.\n+ *\n+ * virtual.cpp - Pipeline handler for virtual cameras\n+ */\n+\n+#include \"virtual.h\"\n+\n+#include \n+\n+#include \n+#include \n+#include \n+#include \n+#include \n+\n+#include \"libcamera/internal/camera.h\"\n+#include \"libcamera/internal/formats.h\"\n+#include \"libcamera/internal/pipeline_handler.h\"\n+\n+namespace libcamera {\n+\n+LOG_DEFINE_CATEGORY(Virtual)\n+\n+namespace {\n+\n+uint64_t currentTimestamp()\n+{\n+\tstruct timespec ts;\n+\tif (clock_gettime(CLOCK_MONOTONIC, &ts) < 0) {\n+\t\tLOG(Virtual, Error) << \"Get clock time fails\";\n+\t\treturn 0;\n+\t}\n+\n+\treturn ts.tv_sec * 1'000'000'000LL + ts.tv_nsec;\n+}\n+\n+} // namespace\n+\n+VirtualCameraConfiguration::VirtualCameraConfiguration(VirtualCameraData *data)\n+\t: CameraConfiguration(), data_(data)\n+{\n+}\n+\n+CameraConfiguration::Status VirtualCameraConfiguration::validate()\n+{\n+\tStatus status = Valid;\n+\n+\tif (config_.empty()) {\n+\t\tLOG(Virtual, Error) << \"Empty config\";\n+\t\treturn Invalid;\n+\t}\n+\n+\t/* Currently only one stream is supported */\n+\tif (config_.size() > 1) {\n+\t\tconfig_.resize(1);\n+\t\tstatus = Adjusted;\n+\t}\n+\n+\tSize maxSize;\n+\tfor (const auto &resolution : data_->supportedResolutions_)\n+\t\tmaxSize = std::max(maxSize, resolution.size);\n+\n+\tfor (StreamConfiguration &cfg : config_) {\n+\t\tbool found = false;\n+\t\tfor (const auto &resolution : data_->supportedResolutions_) {\n+\t\t\tif (resolution.size.width == cfg.size.width &&\n+\t\t\t resolution.size.height == cfg.size.height) {\n+\t\t\t\tfound = true;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (!found) {\n+\t\t\tcfg.size = maxSize;\n+\t\t\tstatus = Adjusted;\n+\t\t}\n+\n+\t\tconst PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat);\n+\t\tcfg.stride = info.stride(cfg.size.width, 0, 1);\n+\t\tcfg.frameSize = info.frameSize(cfg.size, 1);\n+\n+\t\tcfg.setStream(const_cast(&data_->stream_));\n+\n+\t\tcfg.bufferCount = VirtualCameraConfiguration::kBufferCount;\n+\t}\n+\n+\treturn status;\n+}\n+\n+PipelineHandlerVirtual::PipelineHandlerVirtual(CameraManager *manager)\n+\t: PipelineHandler(manager)\n+{\n+}\n+\n+std::unique_ptr\n+PipelineHandlerVirtual::generateConfiguration(Camera *camera,\n+\t\t\t\t\t Span roles)\n+{\n+\tVirtualCameraData *data = cameraData(camera);\n+\tauto config =\n+\t\tstd::make_unique(data);\n+\n+\tif (roles.empty())\n+\t\treturn config;\n+\n+\tSize minSize, sensorResolution;\n+\tfor (const auto &resolution : data->supportedResolutions_) {\n+\t\tif (minSize.isNull() || minSize > resolution.size)\n+\t\t\tminSize = resolution.size;\n+\n+\t\tsensorResolution = std::max(sensorResolution, resolution.size);\n+\t}\n+\n+\tfor (const StreamRole role : roles) {\n+\t\tstd::map> streamFormats;\n+\t\tunsigned int bufferCount;\n+\t\tPixelFormat pixelFormat;\n+\n+\t\tswitch (role) {\n+\t\tcase StreamRole::StillCapture:\n+\t\t\tpixelFormat = formats::NV12;\n+\t\t\tbufferCount = VirtualCameraConfiguration::kBufferCount;\n+\t\t\tstreamFormats[pixelFormat] = { { minSize, sensorResolution } };\n+\n+\t\t\tbreak;\n+\n+\t\tcase StreamRole::Raw: {\n+\t\t\t/* \\todo check */\n+\t\t\tpixelFormat = formats::SBGGR10;\n+\t\t\tbufferCount = VirtualCameraConfiguration::kBufferCount;\n+\t\t\tstreamFormats[pixelFormat] = { { minSize, sensorResolution } };\n+\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tcase StreamRole::Viewfinder:\n+\t\tcase StreamRole::VideoRecording: {\n+\t\t\tpixelFormat = formats::NV12;\n+\t\t\tbufferCount = VirtualCameraConfiguration::kBufferCount;\n+\t\t\tstreamFormats[pixelFormat] = { { minSize, sensorResolution } };\n+\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tdefault:\n+\t\t\tLOG(Virtual, Error)\n+\t\t\t\t<< \"Requested stream role not supported: \" << role;\n+\t\t\tconfig.reset();\n+\t\t\treturn config;\n+\t\t}\n+\n+\t\tStreamFormats formats(streamFormats);\n+\t\tStreamConfiguration cfg(formats);\n+\t\tcfg.size = sensorResolution;\n+\t\tcfg.pixelFormat = pixelFormat;\n+\t\tcfg.bufferCount = bufferCount;\n+\t\tconfig->addConfiguration(cfg);\n+\t}\n+\n+\tif (config->validate() == CameraConfiguration::Invalid)\n+\t\tconfig.reset();\n+\n+\treturn config;\n+}\n+\n+int PipelineHandlerVirtual::configure(\n+\t[[maybe_unused]] Camera *camera,\n+\t[[maybe_unused]] CameraConfiguration *config)\n+{\n+\t// Nothing to be done.\n+\treturn 0;\n+}\n+\n+int PipelineHandlerVirtual::exportFrameBuffers(\n+\t[[maybe_unused]] Camera *camera,\n+\tStream *stream,\n+\tstd::vector> *buffers)\n+{\n+\tif (!dmaBufAllocator_.isValid())\n+\t\treturn -ENOBUFS;\n+\n+\treturn dmaBufAllocator_.exportFrameBuffers(stream->configuration(), buffers);\n+}\n+\n+int PipelineHandlerVirtual::start([[maybe_unused]] Camera *camera,\n+\t\t\t\t [[maybe_unused]] const ControlList *controls)\n+{\n+\t/* \\todo Start reading the virtual video if any. */\n+\treturn 0;\n+}\n+\n+void PipelineHandlerVirtual::stopDevice([[maybe_unused]] Camera *camera)\n+{\n+\t/* \\todo Reset the virtual video if any. */\n+}\n+\n+int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera,\n+\t\t\t\t\t Request *request)\n+{\n+\t/* \\todo Read from the virtual video if any. */\n+\tfor (auto it : request->buffers())\n+\t\tcompleteBuffer(request, it.second);\n+\n+\trequest->metadata().set(controls::SensorTimestamp, currentTimestamp());\n+\tcompleteRequest(request);\n+\n+\treturn 0;\n+}\n+\n+bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator)\n+{\n+\t/* \\todo Add virtual cameras according to a config file. */\n+\n+\tstd::unique_ptr data = std::make_unique(this);\n+\n+\tdata->supportedResolutions_.resize(2);\n+\tdata->supportedResolutions_[0] = { .size = Size(1920, 1080), .frame_rates = { 30 } };\n+\tdata->supportedResolutions_[1] = { .size = Size(1280, 720), .frame_rates = { 30, 60 } };\n+\n+\tdata->properties_.set(properties::Location, properties::CameraLocationFront);\n+\tdata->properties_.set(properties::Model, \"Virtual Video Device\");\n+\tdata->properties_.set(properties::PixelArrayActiveAreas, { Rectangle(Size(1920, 1080)) });\n+\n+\t/* \\todo Set FrameDurationLimits based on config. */\n+\tControlInfoMap::Map controls;\n+\tint64_t min_frame_duration = 30, max_frame_duration = 60;\n+\tcontrols[&controls::FrameDurationLimits] = ControlInfo(min_frame_duration, max_frame_duration);\n+\tdata->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls);\n+\n+\t/* Create and register the camera. */\n+\tstd::set streams{ &data->stream_ };\n+\tconst std::string id = \"Virtual0\";\n+\tstd::shared_ptr camera = Camera::create(std::move(data), id, streams);\n+\tregisterCamera(std::move(camera));\n+\n+\treturn false; // Prevent infinite loops for now\n+}\n+\n+REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual, \"virtual\")\n+\n+} /* namespace libcamera */\ndiff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h\nnew file mode 100644\nindex 000000000..6fc6b34d8\n--- /dev/null\n+++ b/src/libcamera/pipeline/virtual/virtual.h\n@@ -0,0 +1,78 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2023, Google Inc.\n+ *\n+ * virtual.h - Pipeline handler for virtual cameras\n+ */\n+\n+#pragma once\n+\n+#include \n+\n+#include \"libcamera/internal/camera.h\"\n+#include \"libcamera/internal/dma_buf_allocator.h\"\n+#include \"libcamera/internal/pipeline_handler.h\"\n+\n+namespace libcamera {\n+\n+class VirtualCameraData : public Camera::Private\n+{\n+public:\n+\tstruct Resolution {\n+\t\tSize size;\n+\t\tstd::vector frame_rates;\n+\t};\n+\tVirtualCameraData(PipelineHandler *pipe)\n+\t\t: Camera::Private(pipe)\n+\t{\n+\t}\n+\n+\t~VirtualCameraData() = default;\n+\n+\tstd::vector supportedResolutions_;\n+\n+\tStream stream_;\n+};\n+\n+class VirtualCameraConfiguration : public CameraConfiguration\n+{\n+public:\n+\tstatic constexpr unsigned int kBufferCount = 4;\n+\n+\tVirtualCameraConfiguration(VirtualCameraData *data);\n+\n+\tStatus validate() override;\n+\n+private:\n+\tconst VirtualCameraData *data_;\n+};\n+\n+class PipelineHandlerVirtual : public PipelineHandler\n+{\n+public:\n+\tPipelineHandlerVirtual(CameraManager *manager);\n+\n+\tstd::unique_ptr generateConfiguration(Camera *camera,\n+\t\t\t\t\t\t\t\t Span roles) override;\n+\tint configure(Camera *camera, CameraConfiguration *config) override;\n+\n+\tint exportFrameBuffers(Camera *camera, Stream *stream,\n+\t\t\t std::vector> *buffers) override;\n+\n+\tint start(Camera *camera, const ControlList *controls) override;\n+\tvoid stopDevice(Camera *camera) override;\n+\n+\tint queueRequestDevice(Camera *camera, Request *request) override;\n+\n+\tbool match(DeviceEnumerator *enumerator) override;\n+\n+private:\n+\tVirtualCameraData *cameraData(Camera *camera)\n+\t{\n+\t\treturn static_cast(camera->_d());\n+\t}\n+\n+\tDmaBufAllocator dmaBufAllocator_;\n+};\n+\n+} // namespace libcamera\n","prefixes":["v6","3/7"]}