{"id":18533,"url":"https://patchwork.libcamera.org/api/1.1/patches/18533/?format=json","web_url":"https://patchwork.libcamera.org/patch/18533/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20230407102050.17537-3-harveyycyang@gmail.com>","date":"2023-04-07T10:20:50","name":"[libcamera-devel,v5,2/2] libcamera: pipeline: Add VirtualPipelineHandler","commit_ref":null,"pull_url":null,"state":"superseded","archived":true,"hash":"5c0b268bee60245ef9f0e1b2802ae0bbe51f03bc","submitter":{"id":117,"url":"https://patchwork.libcamera.org/api/1.1/people/117/?format=json","name":"Cheng-Hao Yang","email":"chenghaoyang@chromium.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/18533/mbox/","series":[{"id":3838,"url":"https://patchwork.libcamera.org/api/1.1/series/3838/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3838","date":"2023-04-07T10:20:48","name":"Virtual Pipeline Handler","version":5,"mbox":"https://patchwork.libcamera.org/series/3838/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/18533/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/18533/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 40A86C32A4\n\tfor ;\n\tFri, 7 Apr 2023 10:21:01 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D2E87627C1;\n\tFri, 7 Apr 2023 12:20:59 +0200 (CEST)","from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com\n\t[IPv6:2607:f8b0:4864:20::1041])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id DB9E0627BC\n\tfor ;\n\tFri, 7 Apr 2023 12:20:57 +0200 (CEST)","by mail-pj1-x1041.google.com with SMTP id\n\t60-20020a17090a09c200b0023fcc8ce113so960611pjo.4\n\tfor ;\n\tFri, 07 Apr 2023 03:20:57 -0700 (PDT)","from harveyyang.localdomain (1-163-45-206.dynamic-ip.hinet.net.\n\t[1.163.45.206]) by smtp.gmail.com with ESMTPSA id\n\tl11-20020a17090270cb00b0019a997bca5csm2641409plt.121.2023.04.07.03.20.55\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tFri, 07 Apr 2023 03:20:55 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1680862859;\n\tbh=MNC1s1nBxLLFybgFqpNuc5X05ScTmUDIaPPuUj2NjK4=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=YcxdMthZAtYVbVupTotxHXDpPGMwdWjoCV7K8wmtgoKzdcidvsdDbGdul4DsamE6w\n\tU2c/e5ffJVV41cIToVZXhNxVYi7GrD0ggQvLQz8Py/HJkwPkGdAXZ9DeAht0hBoYEO\n\thBm8457FWp+fUMt6rG43IhsirswKeJTJ/Qq7OIYhHVFHw48vzX4PLKrVzb9b2h0Eb0\n\t+znuuQNdsLrkajdGGW1Eh00qoekSVu5ypoeAUj0rhvSFSUPLUxiehdnwYQq1iqXYA8\n\taW8ULiwSSU7cpGKgMrkmM6VZfGmKPtfZqcwAN8rNMI5MK90wiUEyomWQpe1c63Kw7F\n\tZhf95Sfi0vmKQ==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=chromium.org; s=google; t=1680862856; x=1683454856;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=ddUZ8heIfq3Er1vENB+PGHr8XTBkzsPqI+xFpEEnc30=;\n\tb=Otm6RBzLAstsxK7zCdSX1M/u0Arx2OFM3ORpUv/R35n/sln62Kwn61WzVwF3NBpMZL\n\tzCBHsOKdLQNhiuIRgZ3yaoZIMGuwWQm6rDCL5QebwtaP3f9KhyOfIk0bkjISPRLQcmUa\n\teKF4CAuJy7sljzqXWTi2EFWJ4KcjdT6mS+LEk="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=chromium.org\n\theader.i=@chromium.org header.b=\"Otm6RBzL\"; \n\tdkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112; t=1680862856; x=1683454856;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc\n\t:subject:date:message-id:reply-to;\n\tbh=ddUZ8heIfq3Er1vENB+PGHr8XTBkzsPqI+xFpEEnc30=;\n\tb=spNEO8/7aQy77miHDLsa+DzaEq4wn11otZo2+E4IUdzyFTk9A8bq3t4FNkFozfcBZh\n\tJQt+V6t2l2zMj1rmC5fY9QjnH+xvcZGJUgnQzu+vYswZR/G/IrxXZ0s/vCX2PNoQv04h\n\txL55YctFIM3pt3xQ4wz5XyNo87Od+FABFeuJeettuDxTdFoy38citMXzKbsKGy35+up1\n\t6Y75WhomdpMZ3a637ypWer5xPGiFmcUMvfuElOYaJFh9WD5f5HbpQnTKMs6MnYe3Yioo\n\tkuJib6Y0Vb/rZlHsuuq1DYeiOr6EsBJ/zm0OSVLh/rLTkx1HzG6I5xOFvuavyJwI0xeL\n\tI7TQ==","X-Gm-Message-State":"AAQBX9eeYb/o91sOOv/PM9h7reEb5SbBX7gdz3+yLwKl4Z+t6xRUlHMy\n\t7pkZR1B3pe7Aii+K1NQTeKMg+NVB3ECJm7JCN0Vj8mnc","X-Google-Smtp-Source":"AKy350Z58PW9/Q8LlS0OxJIx7Jw6JleJveGJ10zqipNWtifotxCGlCa2Vbs0D9gEJ8GiKmp1NpwZpw==","X-Received":"by 2002:a17:903:189:b0:1a1:b11d:6af5 with SMTP id\n\tz9-20020a170903018900b001a1b11d6af5mr2377379plg.52.1680862856024; \n\tFri, 07 Apr 2023 03:20:56 -0700 (PDT)","X-Google-Original-From":"Harvey Yang ","To":"libcamera-devel@lists.libcamera.org","Date":"Fri, 7 Apr 2023 18:20:50 +0800","Message-Id":"<20230407102050.17537-3-harveyycyang@gmail.com>","X-Mailer":"git-send-email 2.40.0","In-Reply-To":"<20230407102050.17537-1-harveyycyang@gmail.com>","References":"<20230407102050.17537-1-harveyycyang@gmail.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v5 2/2] libcamera: pipeline: Add\n\tVirtualPipelineHandler","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Harvey Yang via libcamera-devel ","Reply-To":"Harvey Yang ","Cc":"Harvey Yang ,\n\tHarvey Yang ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Add VirtualPipelineHandler for more unit tests and verfiy libcamera\ninfrastructure works on devices without using hardware cameras.\n\nUpdating `/etc/camera/libcamera/camera_hal.yaml` on a chromebook DUT is\nrequired to find the virtual camera with id `Virtual0`.\n\n[todo Read frames from the virtual video if any]\n\nTest the configurations can be generated and reported with cam -I:\n\"\"\"\nbuild/src/apps/cam/cam -c 1 -I\n[45:19:28.901456135] [2611530] INFO IPAManager ipa_manager.cpp:143\nlibcamera is not installed. Adding\n'/usr/local/google/home/chenghaoyang/cros2/src/third_party/libcamera/build/src\n/ipa' to the IPA search path\n[45:19:28.904364465] [2611530] INFO Camera camera_manager.cpp:293\nlibcamera v0.0.1+56-4f4554fa-dirty (2022-12-07T06:55:04+00:00)\nUsing camera Virtual0 as cam0\n0: 1920x1080-NV12\n * Pixelformat: NV12 (1280x720)-(1920x1080)/(+1,+1)\n - 1280x720\n - 1280x800\n - 1360x768\n - 1366x768\n - 1440x900\n - 1280x1024\n - 1536x864\n - 1280x1080\n - 1600x900\n - 1400x1050\n - 1680x1050\n - 1920x1080\n\"\"\"\n\n\"\"\"\nbuild/src/apps/cam/cam -l\n[550:47:04.505850647] [1969734] INFO IPAManager ipa_manager.cpp:143\nlibcamera is not installed. Adding ... to the IPA search path\n[550:47:04.509307667] [1969734] INFO Camera camera_manager.cpp:293\nlibcamera v0.0.1+54-55fecb4d-dirty (2022-12-01T05:47:11+00:00)\nAvailable cameras:\n1: (Virtual0)\n\"\"\"\n\nUsing qcam should receive blank (all green) frames:\n\"\"\"\nbuild/src/apps/qcam/qcam -c Virtual0\n\"\"\"\n\nSigned-off-by: Harvey Yang \n---\n meson.build | 1 +\n meson_options.txt | 3 +-\n src/libcamera/pipeline/virtual/meson.build | 5 +\n src/libcamera/pipeline/virtual/virtual.cpp | 302 +++++++++++++++++++++\n 4 files changed, 310 insertions(+), 1 deletion(-)\n create mode 100644 src/libcamera/pipeline/virtual/meson.build\n create mode 100644 src/libcamera/pipeline/virtual/virtual.cpp","diff":"diff --git a/meson.build b/meson.build\nindex 0f89b45a..c2c4ba5f 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -177,6 +177,7 @@ pipelines_support = {\n 'simple': arch_arm,\n 'uvcvideo': ['any'],\n 'vimc': ['test'],\n+ 'virtual': ['test'],\n }\n \n if pipelines.contains('all')\ndiff --git a/meson_options.txt b/meson_options.txt\nindex 78a78b72..a0a75e7f 100644\n--- a/meson_options.txt\n+++ b/meson_options.txt\n@@ -47,7 +47,8 @@ option('pipelines',\n 'rkisp1',\n 'simple',\n 'uvcvideo',\n- 'vimc'\n+ 'vimc',\n+ 'virtual'\n ],\n description : 'Select which pipeline handlers to build. If this is set to \"auto\", all the pipelines applicable to the target architecture will be built. If this is set to \"all\", all the pipelines will be built. If both are selected then \"all\" will take precedence.')\n \ndiff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build\nnew file mode 100644\nindex 00000000..ba7ff754\n--- /dev/null\n+++ b/src/libcamera/pipeline/virtual/meson.build\n@@ -0,0 +1,5 @@\n+# SPDX-License-Identifier: CC0-1.0\n+\n+libcamera_sources += files([\n+ 'virtual.cpp',\n+])\ndiff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp\nnew file mode 100644\nindex 00000000..78616fab\n--- /dev/null\n+++ b/src/libcamera/pipeline/virtual/virtual.cpp\n@@ -0,0 +1,302 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2022, Google Inc.\n+ *\n+ * fake.cpp - Pipeline handler for fake cameras\n+ */\n+\n+#include \n+\n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+\n+#include \"libcamera/internal/camera.h\"\n+#include \"libcamera/internal/formats.h\"\n+#include \"libcamera/internal/pipeline_handler.h\"\n+\n+namespace libcamera {\n+\n+LOG_DEFINE_CATEGORY(VIRTUAL)\n+\n+static const ControlInfoMap::Map VirtualControls = {\n+\t{ &controls::draft::PipelineDepth, ControlInfo(2, 3) },\n+};\n+\n+uint64_t CurrentTimestamp()\n+{\n+\tstruct timespec ts;\n+\tif (clock_gettime(CLOCK_MONOTONIC, &ts) < 0) {\n+\t\tLOG(VIRTUAL, Error) << \"Get clock time fails\";\n+\t\treturn 0;\n+\t}\n+\n+\treturn ts.tv_sec * 1'000'000'000LL + ts.tv_nsec;\n+}\n+\n+class VirtualCameraData : public Camera::Private\n+{\n+public:\n+\tstruct Resolution {\n+\t\tSize size;\n+\t\tstd::vector frame_rates;\n+\t\tstd::vector formats;\n+\t};\n+\tVirtualCameraData(PipelineHandler *pipe)\n+\t\t: Camera::Private(pipe)\n+\t{\n+\t}\n+\n+\t~VirtualCameraData() = default;\n+\n+\tstd::vector supportedResolutions_;\n+\n+\tStream stream_;\n+};\n+\n+class VirtualCameraConfiguration : public CameraConfiguration\n+{\n+public:\n+\tstatic constexpr unsigned int kBufferCount = 4; // 4~6\n+\n+\tVirtualCameraConfiguration(VirtualCameraData *data);\n+\n+\tStatus validate() override;\n+\n+private:\n+\tconst VirtualCameraData *data_;\n+};\n+\n+class PipelineHandlerVirtual : public PipelineHandler\n+{\n+public:\n+\tPipelineHandlerVirtual(CameraManager *manager);\n+\n+\tstd::unique_ptr generateConfiguration(Camera *camera,\n+\t\t\t\t\t\t\t\t const StreamRoles &roles) override;\n+\tint configure(Camera *camera, CameraConfiguration *config) override;\n+\n+\tint exportFrameBuffers(Camera *camera, Stream *stream,\n+\t\t\t std::vector> *buffers) override;\n+\n+\tint start(Camera *camera, const ControlList *controls) override;\n+\tvoid stopDevice(Camera *camera) override;\n+\n+\tint queueRequestDevice(Camera *camera, Request *request) override;\n+\n+\tbool match(DeviceEnumerator *enumerator) override;\n+\n+private:\n+\tVirtualCameraData *cameraData(Camera *camera)\n+\t{\n+\t\treturn static_cast(camera->_d());\n+\t}\n+\n+\tHeapAllocator heapAllocator_;\n+};\n+\n+VirtualCameraConfiguration::VirtualCameraConfiguration(VirtualCameraData *data)\n+\t: CameraConfiguration(), data_(data)\n+{\n+}\n+\n+CameraConfiguration::Status VirtualCameraConfiguration::validate()\n+{\n+\tStatus status = Valid;\n+\n+\tif (config_.empty()) {\n+\t\tLOG(VIRTUAL, Error) << \"Empty config\";\n+\t\treturn Invalid;\n+\t}\n+\n+\t// TODO: check if we should limit |config_.size()|\n+\n+\tSize maxSize;\n+\tfor (const auto &resolution : data_->supportedResolutions_)\n+\t\tmaxSize = std::max(maxSize, resolution.size);\n+\n+\tfor (StreamConfiguration &cfg : config_) {\n+\t\tbool found = false;\n+\t\tfor (const auto &resolution : data_->supportedResolutions_) {\n+\t\t\tif (resolution.size.width >= cfg.size.width && resolution.size.height >= cfg.size.height) {\n+\t\t\t\tfound = true;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (!found) {\n+\t\t\tcfg.size = maxSize;\n+\t\t\tstatus = Adjusted;\n+\t\t}\n+\n+\t\tconst PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat);\n+\t\tcfg.stride = info.stride(cfg.size.width, 0, 1);\n+\t\tcfg.frameSize = info.frameSize(cfg.size, 1);\n+\n+\t\tcfg.setStream(const_cast(&data_->stream_));\n+\n+\t\tcfg.bufferCount = VirtualCameraConfiguration::kBufferCount;\n+\t}\n+\n+\treturn status;\n+}\n+\n+PipelineHandlerVirtual::PipelineHandlerVirtual(CameraManager *manager)\n+\t: PipelineHandler(manager)\n+{\n+}\n+\n+std::unique_ptr PipelineHandlerVirtual::generateConfiguration(Camera *camera,\n+\t\t\t\t\t\t\t\t\t\t const StreamRoles &roles)\n+{\n+\tVirtualCameraData *data = cameraData(camera);\n+\tauto config =\n+\t\tstd::make_unique(data);\n+\n+\tif (roles.empty())\n+\t\treturn config;\n+\n+\tSize minSize, sensorResolution;\n+\tfor (const auto &resolution : data->supportedResolutions_) {\n+\t\tif (minSize.isNull() || minSize > resolution.size)\n+\t\t\tminSize = resolution.size;\n+\n+\t\tsensorResolution = std::max(sensorResolution, resolution.size);\n+\t}\n+\n+\tfor (const StreamRole role : roles) {\n+\t\tstd::map> streamFormats;\n+\t\tunsigned int bufferCount;\n+\t\tPixelFormat pixelFormat;\n+\n+\t\tswitch (role) {\n+\t\tcase StreamRole::StillCapture:\n+\t\t\tpixelFormat = formats::NV12;\n+\t\t\tbufferCount = VirtualCameraConfiguration::kBufferCount;\n+\t\t\tstreamFormats[pixelFormat] = { { minSize, sensorResolution } };\n+\n+\t\t\tbreak;\n+\n+\t\tcase StreamRole::Raw: {\n+\t\t\t// TODO: check\n+\t\t\tpixelFormat = formats::SBGGR10;\n+\t\t\tbufferCount = VirtualCameraConfiguration::kBufferCount;\n+\t\t\tstreamFormats[pixelFormat] = { { minSize, sensorResolution } };\n+\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tcase StreamRole::Viewfinder:\n+\t\tcase StreamRole::VideoRecording: {\n+\t\t\tpixelFormat = formats::NV12;\n+\t\t\tbufferCount = VirtualCameraConfiguration::kBufferCount;\n+\t\t\tstreamFormats[pixelFormat] = { { minSize, sensorResolution } };\n+\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tdefault:\n+\t\t\tLOG(VIRTUAL, Error)\n+\t\t\t\t<< \"Requested stream role not supported: \" << role;\n+\t\t\tconfig.reset();\n+\t\t\treturn config;\n+\t\t}\n+\n+\t\tStreamFormats formats(streamFormats);\n+\t\tStreamConfiguration cfg(formats);\n+\t\tcfg.size = sensorResolution;\n+\t\tcfg.pixelFormat = pixelFormat;\n+\t\tcfg.bufferCount = bufferCount;\n+\t\tconfig->addConfiguration(cfg);\n+\t}\n+\n+\tif (config->validate() == CameraConfiguration::Invalid) {\n+\t\tconfig.reset();\n+\t\treturn config;\n+\t}\n+\n+\treturn config;\n+}\n+\n+int PipelineHandlerVirtual::configure(Camera *camera, CameraConfiguration *config)\n+{\n+\t(void)camera;\n+\t(void)config;\n+\t// Nothing to be done.\n+\treturn 0;\n+}\n+\n+int PipelineHandlerVirtual::exportFrameBuffers(Camera *camera, Stream *stream,\n+\t\t\t\t\t std::vector> *buffers)\n+{\n+\tif (!heapAllocator_.isValid())\n+\t\treturn -ENOBUFS;\n+\n+\treturn heapAllocator_.exportFrameBuffers(camera, stream, buffers);\n+}\n+\n+int PipelineHandlerVirtual::start(Camera *camera, const ControlList *controls)\n+{\n+\t(void)camera;\n+\t(void)controls;\n+\t// TODO: Start reading the virtual video if any.\n+\treturn 0;\n+}\n+\n+void PipelineHandlerVirtual::stopDevice(Camera *camera)\n+{\n+\t(void)camera;\n+\t// TODO: Reset the virtual video if any.\n+}\n+\n+int PipelineHandlerVirtual::queueRequestDevice(Camera *camera, Request *request)\n+{\n+\t(void)camera;\n+\n+\t// TODO: Read from the virtual video if any.\n+\tfor (auto it : request->buffers())\n+\t\tcompleteBuffer(request, it.second);\n+\n+\trequest->metadata().set(controls::SensorTimestamp, CurrentTimestamp());\n+\tcompleteRequest(request);\n+\n+\treturn 0;\n+}\n+\n+bool PipelineHandlerVirtual::match(DeviceEnumerator *enumerator)\n+{\n+\t(void)enumerator;\n+\n+\t// TODO: Add virtual cameras according to a config file.\n+\n+\tstd::unique_ptr data = std::make_unique(this);\n+\n+\tdata->supportedResolutions_.resize(2);\n+\tdata->supportedResolutions_[0] = { .size = Size(1920, 1080), .frame_rates = { 30 }, .formats = { \"YCbCr_420_888\" } };\n+\tdata->supportedResolutions_[1] = { .size = Size(1280, 720), .frame_rates = { 30, 60 }, .formats = { \"YCbCr_420_888\" } };\n+\n+\tdata->properties_.set(properties::Location, properties::CameraLocationFront);\n+\tdata->properties_.set(properties::Model, \"Virtual Video Device\");\n+\tdata->properties_.set(properties::PixelArrayActiveAreas, { Rectangle(Size(1920, 1080)) });\n+\n+\t// TODO: Set FrameDurationLimits based on config.\n+\tControlInfoMap::Map controls = VirtualControls;\n+\tint64_t min_frame_duration = 30, max_frame_duration = 60;\n+\tcontrols[&controls::FrameDurationLimits] = ControlInfo(min_frame_duration, max_frame_duration);\n+\tdata->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls);\n+\n+\t/* Create and register the camera. */\n+\tstd::set streams{ &data->stream_ };\n+\tconst std::string id = \"Virtual0\";\n+\tstd::shared_ptr camera = Camera::create(std::move(data), id, streams);\n+\tregisterCamera(std::move(camera));\n+\n+\treturn false; // Prevent infinite loops for now\n+}\n+\n+REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual)\n+\n+} /* namespace libcamera */\n","prefixes":["libcamera-devel","v5","2/2"]}