From patchwork Fri Apr 7 10:20:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 18533 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 40A86C32A4 for ; Fri, 7 Apr 2023 10:21:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D2E87627C1; Fri, 7 Apr 2023 12:20:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1680862859; bh=MNC1s1nBxLLFybgFqpNuc5X05ScTmUDIaPPuUj2NjK4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=YcxdMthZAtYVbVupTotxHXDpPGMwdWjoCV7K8wmtgoKzdcidvsdDbGdul4DsamE6w U2c/e5ffJVV41cIToVZXhNxVYi7GrD0ggQvLQz8Py/HJkwPkGdAXZ9DeAht0hBoYEO hBm8457FWp+fUMt6rG43IhsirswKeJTJ/Qq7OIYhHVFHw48vzX4PLKrVzb9b2h0Eb0 +znuuQNdsLrkajdGGW1Eh00qoekSVu5ypoeAUj0rhvSFSUPLUxiehdnwYQq1iqXYA8 aW8ULiwSSU7cpGKgMrkmM6VZfGmKPtfZqcwAN8rNMI5MK90wiUEyomWQpe1c63Kw7F Zhf95Sfi0vmKQ== Received: from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com [IPv6:2607:f8b0:4864:20::1041]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DB9E0627BC for ; Fri, 7 Apr 2023 12:20:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Otm6RBzL"; dkim-atps=neutral Received: by mail-pj1-x1041.google.com with SMTP id 60-20020a17090a09c200b0023fcc8ce113so960611pjo.4 for ; Fri, 07 Apr 2023 03:20:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1680862856; x=1683454856; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ddUZ8heIfq3Er1vENB+PGHr8XTBkzsPqI+xFpEEnc30=; b=Otm6RBzLAstsxK7zCdSX1M/u0Arx2OFM3ORpUv/R35n/sln62Kwn61WzVwF3NBpMZL zCBHsOKdLQNhiuIRgZ3yaoZIMGuwWQm6rDCL5QebwtaP3f9KhyOfIk0bkjISPRLQcmUa eKF4CAuJy7sljzqXWTi2EFWJ4KcjdT6mS+LEk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680862856; x=1683454856; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ddUZ8heIfq3Er1vENB+PGHr8XTBkzsPqI+xFpEEnc30=; b=spNEO8/7aQy77miHDLsa+DzaEq4wn11otZo2+E4IUdzyFTk9A8bq3t4FNkFozfcBZh JQt+V6t2l2zMj1rmC5fY9QjnH+xvcZGJUgnQzu+vYswZR/G/IrxXZ0s/vCX2PNoQv04h xL55YctFIM3pt3xQ4wz5XyNo87Od+FABFeuJeettuDxTdFoy38citMXzKbsKGy35+up1 6Y75WhomdpMZ3a637ypWer5xPGiFmcUMvfuElOYaJFh9WD5f5HbpQnTKMs6MnYe3Yioo kuJib6Y0Vb/rZlHsuuq1DYeiOr6EsBJ/zm0OSVLh/rLTkx1HzG6I5xOFvuavyJwI0xeL I7TQ== X-Gm-Message-State: AAQBX9eeYb/o91sOOv/PM9h7reEb5SbBX7gdz3+yLwKl4Z+t6xRUlHMy 7pkZR1B3pe7Aii+K1NQTeKMg+NVB3ECJm7JCN0Vj8mnc X-Google-Smtp-Source: AKy350Z58PW9/Q8LlS0OxJIx7Jw6JleJveGJ10zqipNWtifotxCGlCa2Vbs0D9gEJ8GiKmp1NpwZpw== X-Received: by 2002:a17:903:189:b0:1a1:b11d:6af5 with SMTP id z9-20020a170903018900b001a1b11d6af5mr2377379plg.52.1680862856024; Fri, 07 Apr 2023 03:20:56 -0700 (PDT) Received: from harveyyang.localdomain (1-163-45-206.dynamic-ip.hinet.net. [1.163.45.206]) by smtp.gmail.com with ESMTPSA id l11-20020a17090270cb00b0019a997bca5csm2641409plt.121.2023.04.07.03.20.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Apr 2023 03:20:55 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Fri, 7 Apr 2023 18:20:50 +0800 Message-Id: <20230407102050.17537-3-harveyycyang@gmail.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230407102050.17537-1-harveyycyang@gmail.com> References: <20230407102050.17537-1-harveyycyang@gmail.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 2/2] libcamera: pipeline: Add VirtualPipelineHandler X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add VirtualPipelineHandler for more unit tests and verfiy libcamera infrastructure works on devices without using hardware cameras. Updating `/etc/camera/libcamera/camera_hal.yaml` on a chromebook DUT is required to find the virtual camera with id `Virtual0`. [todo Read frames from the virtual video if any] Test the configurations can be generated and reported with cam -I: """ build/src/apps/cam/cam -c 1 -I [45:19:28.901456135] [2611530] INFO IPAManager ipa_manager.cpp:143 libcamera is not installed. Adding '/usr/local/google/home/chenghaoyang/cros2/src/third_party/libcamera/build/src /ipa' to the IPA search path [45:19:28.904364465] [2611530] INFO Camera camera_manager.cpp:293 libcamera v0.0.1+56-4f4554fa-dirty (2022-12-07T06:55:04+00:00) Using camera Virtual0 as cam0 0: 1920x1080-NV12 * Pixelformat: NV12 (1280x720)-(1920x1080)/(+1,+1) - 1280x720 - 1280x800 - 1360x768 - 1366x768 - 1440x900 - 1280x1024 - 1536x864 - 1280x1080 - 1600x900 - 1400x1050 - 1680x1050 - 1920x1080 """ """ build/src/apps/cam/cam -l [550:47:04.505850647] [1969734] INFO IPAManager ipa_manager.cpp:143 libcamera is not installed. Adding ... to the IPA search path [550:47:04.509307667] [1969734] INFO Camera camera_manager.cpp:293 libcamera v0.0.1+54-55fecb4d-dirty (2022-12-01T05:47:11+00:00) Available cameras: 1: (Virtual0) """ Using qcam should receive blank (all green) frames: """ build/src/apps/qcam/qcam -c Virtual0 """ Signed-off-by: Harvey Yang --- meson.build | 1 + meson_options.txt | 3 +- src/libcamera/pipeline/virtual/meson.build | 5 + src/libcamera/pipeline/virtual/virtual.cpp | 302 +++++++++++++++++++++ 4 files changed, 310 insertions(+), 1 deletion(-) create mode 100644 src/libcamera/pipeline/virtual/meson.build create mode 100644 src/libcamera/pipeline/virtual/virtual.cpp diff --git a/meson.build b/meson.build index 0f89b45a..c2c4ba5f 100644 --- a/meson.build +++ b/meson.build @@ -177,6 +177,7 @@ pipelines_support = { 'simple': arch_arm, 'uvcvideo': ['any'], 'vimc': ['test'], + 'virtual': ['test'], } if pipelines.contains('all') diff --git a/meson_options.txt b/meson_options.txt index 78a78b72..a0a75e7f 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -47,7 +47,8 @@ option('pipelines', 'rkisp1', 'simple', 'uvcvideo', - 'vimc' + 'vimc', + 'virtual' ], description : 'Select which pipeline handlers to build. If this is set to "auto", all the pipelines applicable to the target architecture will be built. If this is set to "all", all the pipelines will be built. If both are selected then "all" will take precedence.') diff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build new file mode 100644 index 00000000..ba7ff754 --- /dev/null +++ b/src/libcamera/pipeline/virtual/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +libcamera_sources += files([ + 'virtual.cpp', +]) diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp new file mode 100644 index 00000000..78616fab --- /dev/null +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -0,0 +1,302 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2022, Google Inc. + * + * fake.cpp - Pipeline handler for fake cameras + */ + +#include + +#include +#include +#include +#include +#include +#include + +#include "libcamera/internal/camera.h" +#include "libcamera/internal/formats.h" +#include "libcamera/internal/pipeline_handler.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(VIRTUAL) + +static const ControlInfoMap::Map VirtualControls = { + { &controls::draft::PipelineDepth, ControlInfo(2, 3) }, +}; + +uint64_t CurrentTimestamp() +{ + struct timespec ts; + if (clock_gettime(CLOCK_MONOTONIC, &ts) < 0) { + LOG(VIRTUAL, Error) << "Get clock time fails"; + return 0; + } + + return ts.tv_sec * 1'000'000'000LL + ts.tv_nsec; +} + +class VirtualCameraData : public Camera::Private +{ +public: + struct Resolution { + Size size; + std::vector frame_rates; + std::vector formats; + }; + VirtualCameraData(PipelineHandler *pipe) + : Camera::Private(pipe) + { + } + + ~VirtualCameraData() = default; + + std::vector supportedResolutions_; + + Stream stream_; +}; + +class VirtualCameraConfiguration : public CameraConfiguration +{ +public: + static constexpr unsigned int kBufferCount = 4; // 4~6 + + VirtualCameraConfiguration(VirtualCameraData *data); + + Status validate() override; + +private: + const VirtualCameraData *data_; +}; + +class PipelineHandlerVirtual : public PipelineHandler +{ +public: + PipelineHandlerVirtual(CameraManager *manager); + + std::unique_ptr generateConfiguration(Camera *camera, + const StreamRoles &roles) override; + int configure(Camera *camera, CameraConfiguration *config) override; + + int exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) override; + + int start(Camera *camera, const ControlList *controls) override; + void stopDevice(Camera *camera) override; + + int queueRequestDevice(Camera *camera, Request *request) override; + + bool match(DeviceEnumerator *enumerator) override; + +private: + VirtualCameraData *cameraData(Camera *camera) + { + return static_cast(camera->_d()); + } + + HeapAllocator heapAllocator_; +}; + +VirtualCameraConfiguration::VirtualCameraConfiguration(VirtualCameraData *data) + : CameraConfiguration(), data_(data) +{ +} + +CameraConfiguration::Status VirtualCameraConfiguration::validate() +{ + Status status = Valid; + + if (config_.empty()) { + LOG(VIRTUAL, Error) << "Empty config"; + return Invalid; + } + + // TODO: check if we should limit |config_.size()| + + Size maxSize; + for (const auto &resolution : data_->supportedResolutions_) + maxSize = std::max(maxSize, resolution.size); + + for (StreamConfiguration &cfg : config_) { + bool found = false; + for (const auto &resolution : data_->supportedResolutions_) { + if (resolution.size.width >= cfg.size.width && resolution.size.height >= cfg.size.height) { + found = true; + break; + } + } + + if (!found) { + cfg.size = maxSize; + status = Adjusted; + } + + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + cfg.stride = info.stride(cfg.size.width, 0, 1); + cfg.frameSize = info.frameSize(cfg.size, 1); + + cfg.setStream(const_cast(&data_->stream_)); + + cfg.bufferCount = VirtualCameraConfiguration::kBufferCount; + } + + return status; +} + +PipelineHandlerVirtual::PipelineHandlerVirtual(CameraManager *manager) + : PipelineHandler(manager) +{ +} + +std::unique_ptr PipelineHandlerVirtual::generateConfiguration(Camera *camera, + const StreamRoles &roles) +{ + VirtualCameraData *data = cameraData(camera); + auto config = + std::make_unique(data); + + if (roles.empty()) + return config; + + Size minSize, sensorResolution; + for (const auto &resolution : data->supportedResolutions_) { + if (minSize.isNull() || minSize > resolution.size) + minSize = resolution.size; + + sensorResolution = std::max(sensorResolution, resolution.size); + } + + for (const StreamRole role : roles) { + std::map> streamFormats; + unsigned int bufferCount; + PixelFormat pixelFormat; + + switch (role) { + case StreamRole::StillCapture: + pixelFormat = formats::NV12; + bufferCount = VirtualCameraConfiguration::kBufferCount; + streamFormats[pixelFormat] = { { minSize, sensorResolution } }; + + break; + + case StreamRole::Raw: { + // TODO: check + pixelFormat = formats::SBGGR10; + bufferCount = VirtualCameraConfiguration::kBufferCount; + streamFormats[pixelFormat] = { { minSize, sensorResolution } }; + + break; + } + + case StreamRole::Viewfinder: + case StreamRole::VideoRecording: { + pixelFormat = formats::NV12; + bufferCount = VirtualCameraConfiguration::kBufferCount; + streamFormats[pixelFormat] = { { minSize, sensorResolution } }; + + break; + } + + default: + LOG(VIRTUAL, Error) + << "Requested stream role not supported: " << role; + config.reset(); + return config; + } + + StreamFormats formats(streamFormats); + StreamConfiguration cfg(formats); + cfg.size = sensorResolution; + cfg.pixelFormat = pixelFormat; + cfg.bufferCount = bufferCount; + config->addConfiguration(cfg); + } + + if (config->validate() == CameraConfiguration::Invalid) { + config.reset(); + return config; + } + + return config; +} + +int PipelineHandlerVirtual::configure(Camera *camera, CameraConfiguration *config) +{ + (void)camera; + (void)config; + // Nothing to be done. + return 0; +} + +int PipelineHandlerVirtual::exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) +{ + if (!heapAllocator_.isValid()) + return -ENOBUFS; + + return heapAllocator_.exportFrameBuffers(camera, stream, buffers); +} + +int PipelineHandlerVirtual::start(Camera *camera, const ControlList *controls) +{ + (void)camera; + (void)controls; + // TODO: Start reading the virtual video if any. + return 0; +} + +void PipelineHandlerVirtual::stopDevice(Camera *camera) +{ + (void)camera; + // TODO: Reset the virtual video if any. +} + +int PipelineHandlerVirtual::queueRequestDevice(Camera *camera, Request *request) +{ + (void)camera; + + // TODO: Read from the virtual video if any. + for (auto it : request->buffers()) + completeBuffer(request, it.second); + + request->metadata().set(controls::SensorTimestamp, CurrentTimestamp()); + completeRequest(request); + + return 0; +} + +bool PipelineHandlerVirtual::match(DeviceEnumerator *enumerator) +{ + (void)enumerator; + + // TODO: Add virtual cameras according to a config file. + + std::unique_ptr data = std::make_unique(this); + + data->supportedResolutions_.resize(2); + data->supportedResolutions_[0] = { .size = Size(1920, 1080), .frame_rates = { 30 }, .formats = { "YCbCr_420_888" } }; + data->supportedResolutions_[1] = { .size = Size(1280, 720), .frame_rates = { 30, 60 }, .formats = { "YCbCr_420_888" } }; + + data->properties_.set(properties::Location, properties::CameraLocationFront); + data->properties_.set(properties::Model, "Virtual Video Device"); + data->properties_.set(properties::PixelArrayActiveAreas, { Rectangle(Size(1920, 1080)) }); + + // TODO: Set FrameDurationLimits based on config. + ControlInfoMap::Map controls = VirtualControls; + int64_t min_frame_duration = 30, max_frame_duration = 60; + controls[&controls::FrameDurationLimits] = ControlInfo(min_frame_duration, max_frame_duration); + data->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); + + /* Create and register the camera. */ + std::set streams{ &data->stream_ }; + const std::string id = "Virtual0"; + std::shared_ptr camera = Camera::create(std::move(data), id, streams); + registerCamera(std::move(camera)); + + return false; // Prevent infinite loops for now +} + +REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual) + +} /* namespace libcamera */