From patchwork Mon Aug 5 13:48:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 20780 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0EDA5BE173 for ; Mon, 5 Aug 2024 13:51:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F145363381; Mon, 5 Aug 2024 15:51:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Rn5k6Fmf"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 10A1963369 for ; Mon, 5 Aug 2024 15:51:08 +0200 (CEST) Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-4280c55e488so33122285e9.0 for ; Mon, 05 Aug 2024 06:51:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1722865867; x=1723470667; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=urDAndi6oox0tDq7i1gXEYZuOknup8uTDy94++8STp0=; b=Rn5k6FmfDDReR9UVvpnw3GUlUftVNrU8nVc9wiSJ5egyhiha+9t36uKyR4Vo4ix3PD JisxewthhOMXISwapbuP+GbrmahPWgMZbbjFip10G+CYpAY4FQ3Yu3zM5EDRoQj/ygee eo3sVtODMPC9AVRHAD64NCpStHhjtFaMl8G38= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722865867; x=1723470667; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=urDAndi6oox0tDq7i1gXEYZuOknup8uTDy94++8STp0=; b=USCdtFcFDSJEXufnmd0UGh92i7SVXDGIYq8xCEckpf5LAcIwXmog6s3L0dCz9gvY8a wpot4HuTfaJhlo3apg8Fyc/LayqMOeTe6k1uYOwzxafLukhvVUNoO75LGMtmIx8M1QmU VuCGXkSu6dkO/mD+usvFiX+wNwpxN5PiNCCy+yuMArsrIOxM6WHqfucVr+mGb+gijdJT uQE7d7dsYoLpC0p5M4FS6Htz6h67m1W0h7MbXm2wlsQ0p1pOX228Se3bVpLMESy/VWKf V6LaeN4uuHJuX/Xg+0kg9n59WXuiz9glWDGzKsUfBzo/ZP9bC9uVgKYbKxKRhhfugzJU hXpA== X-Gm-Message-State: AOJu0YwZRLCWshC6l8i6tF2/r1mgsGFHKqJNToK26XcJZR2CpSMLOuIP oxfy/3WoL+eLqBbCFz9pyVAoSGzamPI38p4dqfOhpotoUahDbf7KJD1sWIEMOyhsOLOs9W3dKtl H41lSti4= X-Google-Smtp-Source: AGHT+IGfNxAjSuSsiSQ6LSCTw+rHBrVmg84KAxJtGKV4lSFyED5R7rFebuiepdY5qjwHgFdnAg5Kpg== X-Received: by 2002:a05:600c:1c8b:b0:428:e820:37dc with SMTP id 5b1f17b1804b1-428e8203a6bmr88031715e9.7.1722865867178; Mon, 05 Aug 2024 06:51:07 -0700 (PDT) Received: from chenghaoyang-germany.c.googlers.com.com (49.222.77.34.bc.googleusercontent.com. [34.77.222.49]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-428e8d6555fsm137146675e9.26.2024.08.05.06.51.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Aug 2024 06:51:06 -0700 (PDT) From: Harvey Yang X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang Subject: [PATCH v8 3/8] libcamera: pipeline: Add VirtualPipelineHandler Date: Mon, 5 Aug 2024 13:48:34 +0000 Message-ID: <20240805135104.139932-4-chenghaoyang@google.com> X-Mailer: git-send-email 2.46.0.rc2.264.g509ed76dc8-goog In-Reply-To: <20240805135104.139932-1-chenghaoyang@google.com> References: <20240805135104.139932-1-chenghaoyang@google.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang Add VirtualPipelineHandler for more unit tests and verfiy libcamera infrastructure works on devices without using hardware cameras. Signed-off-by: Harvey Yang --- meson.build | 1 + meson_options.txt | 3 +- src/libcamera/pipeline/virtual/meson.build | 5 + src/libcamera/pipeline/virtual/virtual.cpp | 251 +++++++++++++++++++++ src/libcamera/pipeline/virtual/virtual.h | 78 +++++++ 5 files changed, 337 insertions(+), 1 deletion(-) create mode 100644 src/libcamera/pipeline/virtual/meson.build create mode 100644 src/libcamera/pipeline/virtual/virtual.cpp create mode 100644 src/libcamera/pipeline/virtual/virtual.h diff --git a/meson.build b/meson.build index f946eba9..3cad3249 100644 --- a/meson.build +++ b/meson.build @@ -222,6 +222,7 @@ pipelines_support = { 'simple': arch_arm, 'uvcvideo': ['any'], 'vimc': ['test'], + 'virtual': ['test'], } if pipelines.contains('all') diff --git a/meson_options.txt b/meson_options.txt index 7aa41249..c91cd241 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -53,7 +53,8 @@ option('pipelines', 'rpi/vc4', 'simple', 'uvcvideo', - 'vimc' + 'vimc', + 'virtual' ], description : 'Select which pipeline handlers to build. If this is set to "auto", all the pipelines applicable to the target architecture will be built. If this is set to "all", all the pipelines will be built. If both are selected then "all" will take precedence.') diff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build new file mode 100644 index 00000000..ba7ff754 --- /dev/null +++ b/src/libcamera/pipeline/virtual/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +libcamera_sources += files([ + 'virtual.cpp', +]) diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp new file mode 100644 index 00000000..74eb8c7a --- /dev/null +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -0,0 +1,251 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Google Inc. + * + * virtual.cpp - Pipeline handler for virtual cameras + */ + +#include "virtual.h" + +#include + +#include +#include +#include +#include +#include + +#include "libcamera/internal/camera.h" +#include "libcamera/internal/formats.h" +#include "libcamera/internal/pipeline_handler.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(Virtual) + +namespace { + +uint64_t currentTimestamp() +{ + struct timespec ts; + if (clock_gettime(CLOCK_MONOTONIC, &ts) < 0) { + LOG(Virtual, Error) << "Get clock time fails"; + return 0; + } + + return ts.tv_sec * 1'000'000'000LL + ts.tv_nsec; +} + +} // namespace + +VirtualCameraConfiguration::VirtualCameraConfiguration(VirtualCameraData *data) + : CameraConfiguration(), data_(data) +{ +} + +CameraConfiguration::Status VirtualCameraConfiguration::validate() +{ + Status status = Valid; + + if (config_.empty()) { + LOG(Virtual, Error) << "Empty config"; + return Invalid; + } + + /* Currently only one stream is supported */ + if (config_.size() > 1) { + config_.resize(1); + status = Adjusted; + } + + Size maxSize; + for (const auto &resolution : data_->supportedResolutions_) + maxSize = std::max(maxSize, resolution.size); + + for (StreamConfiguration &cfg : config_) { + bool found = false; + for (const auto &resolution : data_->supportedResolutions_) { + if (resolution.size.width == cfg.size.width && + resolution.size.height == cfg.size.height) { + found = true; + break; + } + } + + if (!found) { + cfg.size = maxSize; + status = Adjusted; + } + + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + cfg.stride = info.stride(cfg.size.width, 0, 1); + cfg.frameSize = info.frameSize(cfg.size, 1); + + cfg.setStream(const_cast(&data_->stream_)); + + cfg.bufferCount = VirtualCameraConfiguration::kBufferCount; + } + + return status; +} + +PipelineHandlerVirtual::PipelineHandlerVirtual(CameraManager *manager) + : PipelineHandler(manager) +{ +} + +std::unique_ptr +PipelineHandlerVirtual::generateConfiguration(Camera *camera, + Span roles) +{ + VirtualCameraData *data = cameraData(camera); + auto config = + std::make_unique(data); + + if (roles.empty()) + return config; + + Size minSize, sensorResolution; + for (const auto &resolution : data->supportedResolutions_) { + if (minSize.isNull() || minSize > resolution.size) + minSize = resolution.size; + + sensorResolution = std::max(sensorResolution, resolution.size); + } + + for (const StreamRole role : roles) { + std::map> streamFormats; + unsigned int bufferCount; + PixelFormat pixelFormat; + + switch (role) { + case StreamRole::StillCapture: + pixelFormat = formats::NV12; + bufferCount = VirtualCameraConfiguration::kBufferCount; + streamFormats[pixelFormat] = { { minSize, sensorResolution } }; + + break; + + case StreamRole::Raw: { + /* \todo check */ + pixelFormat = formats::SBGGR10; + bufferCount = VirtualCameraConfiguration::kBufferCount; + streamFormats[pixelFormat] = { { minSize, sensorResolution } }; + + break; + } + + case StreamRole::Viewfinder: + case StreamRole::VideoRecording: { + pixelFormat = formats::NV12; + bufferCount = VirtualCameraConfiguration::kBufferCount; + streamFormats[pixelFormat] = { { minSize, sensorResolution } }; + + break; + } + + default: + LOG(Virtual, Error) + << "Requested stream role not supported: " << role; + config.reset(); + return config; + } + + StreamFormats formats(streamFormats); + StreamConfiguration cfg(formats); + cfg.size = sensorResolution; + cfg.pixelFormat = pixelFormat; + cfg.bufferCount = bufferCount; + config->addConfiguration(cfg); + } + + if (config->validate() == CameraConfiguration::Invalid) + config.reset(); + + return config; +} + +int PipelineHandlerVirtual::configure( + [[maybe_unused]] Camera *camera, + [[maybe_unused]] CameraConfiguration *config) +{ + // Nothing to be done. + return 0; +} + +int PipelineHandlerVirtual::exportFrameBuffers( + [[maybe_unused]] Camera *camera, + Stream *stream, + std::vector> *buffers) +{ + if (!dmaBufAllocator_.isValid()) + return -ENOBUFS; + + const StreamConfiguration &config = stream->configuration(); + + auto info = PixelFormatInfo::info(config.pixelFormat); + + std::vector planeSizes; + for (size_t i = 0; i < info.planes.size(); ++i) + planeSizes.push_back(info.planeSize(config.size, i)); + + return dmaBufAllocator_.exportBuffers(config.bufferCount, planeSizes, buffers); +} + +int PipelineHandlerVirtual::start([[maybe_unused]] Camera *camera, + [[maybe_unused]] const ControlList *controls) +{ + /* \todo Start reading the virtual video if any. */ + return 0; +} + +void PipelineHandlerVirtual::stopDevice([[maybe_unused]] Camera *camera) +{ + /* \todo Reset the virtual video if any. */ +} + +int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera, + Request *request) +{ + /* \todo Read from the virtual video if any. */ + for (auto it : request->buffers()) + completeBuffer(request, it.second); + + request->metadata().set(controls::SensorTimestamp, currentTimestamp()); + completeRequest(request); + + return 0; +} + +bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator) +{ + /* \todo Add virtual cameras according to a config file. */ + + std::unique_ptr data = std::make_unique(this); + + data->supportedResolutions_.resize(2); + data->supportedResolutions_[0] = { .size = Size(1920, 1080), .frame_rates = { 30 } }; + data->supportedResolutions_[1] = { .size = Size(1280, 720), .frame_rates = { 30, 60 } }; + + data->properties_.set(properties::Location, properties::CameraLocationFront); + data->properties_.set(properties::Model, "Virtual Video Device"); + data->properties_.set(properties::PixelArrayActiveAreas, { Rectangle(Size(1920, 1080)) }); + + /* \todo Set FrameDurationLimits based on config. */ + ControlInfoMap::Map controls; + int64_t min_frame_duration = 30, max_frame_duration = 60; + controls[&controls::FrameDurationLimits] = ControlInfo(min_frame_duration, max_frame_duration); + data->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); + + /* Create and register the camera. */ + std::set streams{ &data->stream_ }; + const std::string id = "Virtual0"; + std::shared_ptr camera = Camera::create(std::move(data), id, streams); + registerCamera(std::move(camera)); + + return false; // Prevent infinite loops for now +} + +REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual, "virtual") + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h new file mode 100644 index 00000000..6fc6b34d --- /dev/null +++ b/src/libcamera/pipeline/virtual/virtual.h @@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Google Inc. + * + * virtual.h - Pipeline handler for virtual cameras + */ + +#pragma once + +#include + +#include "libcamera/internal/camera.h" +#include "libcamera/internal/dma_buf_allocator.h" +#include "libcamera/internal/pipeline_handler.h" + +namespace libcamera { + +class VirtualCameraData : public Camera::Private +{ +public: + struct Resolution { + Size size; + std::vector frame_rates; + }; + VirtualCameraData(PipelineHandler *pipe) + : Camera::Private(pipe) + { + } + + ~VirtualCameraData() = default; + + std::vector supportedResolutions_; + + Stream stream_; +}; + +class VirtualCameraConfiguration : public CameraConfiguration +{ +public: + static constexpr unsigned int kBufferCount = 4; + + VirtualCameraConfiguration(VirtualCameraData *data); + + Status validate() override; + +private: + const VirtualCameraData *data_; +}; + +class PipelineHandlerVirtual : public PipelineHandler +{ +public: + PipelineHandlerVirtual(CameraManager *manager); + + std::unique_ptr generateConfiguration(Camera *camera, + Span roles) override; + int configure(Camera *camera, CameraConfiguration *config) override; + + int exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) override; + + int start(Camera *camera, const ControlList *controls) override; + void stopDevice(Camera *camera) override; + + int queueRequestDevice(Camera *camera, Request *request) override; + + bool match(DeviceEnumerator *enumerator) override; + +private: + VirtualCameraData *cameraData(Camera *camera) + { + return static_cast(camera->_d()); + } + + DmaBufAllocator dmaBufAllocator_; +}; + +} // namespace libcamera