From patchwork Thu Aug 29 19:47:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 21054 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A2E91C32C2 for ; Thu, 29 Aug 2024 19:57:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BB8C5634BA; Thu, 29 Aug 2024 21:57:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="bpE+Q0sK"; dkim-atps=neutral Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 32CED6345A for ; Thu, 29 Aug 2024 21:57:12 +0200 (CEST) Received: by mail-wm1-x32b.google.com with SMTP id 5b1f17b1804b1-428e1915e18so8908655e9.1 for ; Thu, 29 Aug 2024 12:57:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1724961431; x=1725566231; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IwwVi+OxRq9k7DPUlRzVijKxuzbzPMxBlot3oQWGpN8=; b=bpE+Q0sKzs4ucqFAQGcX4bFRC/43PnBIIgcuEM+exd8dVSqKhJuGL/jKSIpIgTeopc BfwURL/bSnkQuru88mGaIYblXWSUkqokO84e5hVHJLD8hz1U677MnlDvSgCNO+sfm5hP yWRAhMGplVvqMgCEamF3QtiKkMoSpIU+lsNGg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724961431; x=1725566231; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IwwVi+OxRq9k7DPUlRzVijKxuzbzPMxBlot3oQWGpN8=; b=iJnTlQTIzkEiWwDDtmiOeXo1KYiBe2W6EKrkljxYGg3Zc3xpoRlgTNl9N5qn1A1Mai teqAxaVQCPxnXI9l302JF0geac2F/FDQyHwyafxmGWYFkgCttGfXtVplcEKYb1us0oL5 jjy5bMJJpF4ikAeyIySko+idOvNeD7eXSGn998eVF89g2wtexkh1oCIDdQfubRMI3U/J ejpFMQe6hd6VQFMT4yrI0oJhcTo+5110UpEx7XhnGlgYkCbSy7mqyPOCmxKXykQND08P E0H2KtOB83Wz/mDLrEUzxaO0Dky4pgEP7qA/8S/kj716W6OlpTjE91W66u8uaj32ghba MChw== X-Gm-Message-State: AOJu0Ywb7TJwR+e9sHv/7LasgKVbXmZ3FjTu34KlixcM0BwlNw4Ywq5/ UJSzbC/yiD+QAME2yp/m8wcGx7w3J+KA8FShWwatxwluVKhrXdMCzvw6p1g+Mp4cIZ1NzWRa5Ma 3Hw== X-Google-Smtp-Source: AGHT+IHcEh6R9AFrH+DSvxPylhkNPlHjd1y5V0Oo01Q14yM26FrmOEnRCvfLcQE9umYc1jl7mjOu5g== X-Received: by 2002:a05:600c:3c8f:b0:423:791:f446 with SMTP id 5b1f17b1804b1-42bb02c185emr30916885e9.7.1724961431315; Thu, 29 Aug 2024 12:57:11 -0700 (PDT) Received: from chenghaoyang-germany.c.googlers.com.com (161.126.77.34.bc.googleusercontent.com. [34.77.126.161]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42ba7b4271fsm54602805e9.29.2024.08.29.12.57.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Aug 2024 12:57:10 -0700 (PDT) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Konami Shu , Harvey Yang , Yunke Cao , Tomasz Figa Subject: [PATCH v10 6/7] libcamera: virtual: Add ImageFrameGenerator Date: Thu, 29 Aug 2024 19:47:45 +0000 Message-ID: <20240829195703.1456614-7-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog In-Reply-To: <20240829195703.1456614-1-chenghaoyang@chromium.org> References: <20240829195703.1456614-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Konami Shu Besides TestPatternGenerator, this patch adds ImageFrameGenerator that loads real images (jpg / jpeg for now) as the source and generates scaled frames. Signed-off-by: Konami Shu Co-developed-by: Harvey Yang Co-developed-by: Yunke Cao Co-developed-by: Tomasz Figa --- src/libcamera/pipeline/virtual/README.md | 9 +- .../virtual/image_frame_generator.cpp | 179 ++++++++++++++++++ .../pipeline/virtual/image_frame_generator.h | 54 ++++++ src/libcamera/pipeline/virtual/meson.build | 4 + src/libcamera/pipeline/virtual/parser.cpp | 76 +++++++- src/libcamera/pipeline/virtual/parser.h | 2 + src/libcamera/pipeline/virtual/utils.h | 17 ++ src/libcamera/pipeline/virtual/virtual.cpp | 60 ++++-- src/libcamera/pipeline/virtual/virtual.h | 24 ++- 9 files changed, 390 insertions(+), 35 deletions(-) create mode 100644 src/libcamera/pipeline/virtual/image_frame_generator.cpp create mode 100644 src/libcamera/pipeline/virtual/image_frame_generator.h create mode 100644 src/libcamera/pipeline/virtual/utils.h diff --git a/src/libcamera/pipeline/virtual/README.md b/src/libcamera/pipeline/virtual/README.md index ff1e8a5f9..847b8eb87 100644 --- a/src/libcamera/pipeline/virtual/README.md +++ b/src/libcamera/pipeline/virtual/README.md @@ -16,7 +16,13 @@ Each camera block is a dictionary, containing the following keys: - `width` (`unsigned int`, default=1920): Width of the window resolution. This needs to be even. - `height` (`unsigned int`, default=1080): Height of the window resolution. - `frame_rates` (list of `int`, default=`[30,60]` ): Range of the frame rate (per second). The list has to be two values of the lower bound and the upper bound of the frame rate. -- `test_pattern` (`string`): Which test pattern to use as frames. The options are "bars", "lines". +- `test_pattern` (`string`): Which test pattern to use as frames. The options are "bars", "lines". Cannot be set with `frames`. +- `frames` (dictionary): + - `path` (`string`): Path to an image, or path to a directory of a series of images. Cannot be set with `test_pattern`. + - The test patterns are "bars" which means color bars, and "lines" which means diagonal lines. + - The path to an image has ".jpg" extension. + - The path to a directory ends with "/". The name of the images in the directory are "{n}.jpg" with {n} is the sequence of images starting with 0. + - `scale_mode`(`string`, default="fill"): Scale mode when the frames are images. The only scale mode supported now is "fill". This does not affect the scale mode for now. - `location` (`string`, default="front"): The location of the camera. Support "front" and "back". This is displayed in qcam camera selection window but this does not change the output. - `model` (`string`, default="Unknown"): The model name of the camera. This is displayed in qcam camera selection window but this does not change the output. @@ -37,6 +43,7 @@ This is the procedure of the Parser class: 3. Parse each property and register the data. - `parseSupportedFormats()`: Parses `supported_formats` in the config, which contains resolutions and frame rates. - `parseTestPattern()`: Parses `test_pattern` in the config. + - `parseFrame()`: Parses `frames` in the config. - `parseLocation()`: Parses `location` in the config. - `parseModel()`: Parses `model` in the config. 4. Back to `parseConfigFile()` and append the camera configuration. diff --git a/src/libcamera/pipeline/virtual/image_frame_generator.cpp b/src/libcamera/pipeline/virtual/image_frame_generator.cpp new file mode 100644 index 000000000..072e55620 --- /dev/null +++ b/src/libcamera/pipeline/virtual/image_frame_generator.cpp @@ -0,0 +1,179 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * image_frame_generator.cpp - Derived class of FrameGenerator for + * generating frames from images + */ + +#include "image_frame_generator.h" + +#include +#include +#include + +#include +#include + +#include + +#include "libcamera/internal/mapped_framebuffer.h" + +#include "libyuv/convert.h" +#include "libyuv/scale.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Virtual) + +/* + * Factory function to create an ImageFrameGenerator object. + * Read the images and convert them to buffers in NV12 format. + * Store the pointers to the buffers to a list (imageFrameDatas) + */ +std::unique_ptr +ImageFrameGenerator::create(ImageFrames &imageFrames) +{ + std::unique_ptr imageFrameGenerator = + std::make_unique(); + imageFrameGenerator->imageFrames_ = &imageFrames; + + /* + * For each file in the directory, load the image, + * convert it to NV12, and store the pointer. + */ + for (unsigned int i = 0; i < imageFrames.number.value_or(1); i++) { + std::filesystem::path path; + if (!imageFrames.number) { + /* If the path is to an image */ + path = imageFrames.path; + } else { + /* If the path is to a directory */ + path = imageFrames.path / (std::to_string(i) + ".jpg"); + } + + File file(path); + if (!file.open(File::OpenModeFlag::ReadOnly)) { + LOG(Virtual, Error) << "Failed to open image file " << file.fileName() + << ": " << strerror(file.error()); + return nullptr; + } + + /* Read the image file to data */ + auto fileSize = file.size(); + auto buffer = std::make_unique(file.size()); + if (file.read({ buffer.get(), static_cast(fileSize) }) != fileSize) { + LOG(Virtual, Error) << "Failed to read file " << file.fileName() + << ": " << strerror(file.error()); + return nullptr; + } + + /* Get the width and height of the image */ + int width, height; + if (libyuv::MJPGSize(buffer.get(), fileSize, &width, &height)) { + LOG(Virtual, Error) << "Failed to get the size of the image file: " + << file.fileName(); + return nullptr; + } + + /* Convert to NV12 and write the data to tmpY and tmpUV */ + unsigned int halfWidth = (width + 1) / 2; + unsigned int halfHeight = (height + 1) / 2; + std::unique_ptr dstY = + std::make_unique(width * height); + std::unique_ptr dstUV = + std::make_unique(halfWidth * halfHeight * 2); + int ret = libyuv::MJPGToNV12(buffer.get(), fileSize, + dstY.get(), width, dstUV.get(), + width, width, height, width, height); + if (ret != 0) + LOG(Virtual, Error) << "MJPGToNV12() failed with " << ret; + + imageFrameGenerator->imageFrameDatas_.emplace_back( + ImageFrameData{ std::move(dstY), std::move(dstUV), + Size(width, height) }); + } + + return imageFrameGenerator; +} + +/* Scale the buffers for image frames. */ +void ImageFrameGenerator::configure(const Size &size) +{ + /* Reset the source images to prevent multiple configuration calls */ + scaledFrameDatas_.clear(); + frameCount_ = 0; + parameter_ = 0; + + for (unsigned int i = 0; i < imageFrames_->number.value_or(1); i++) { + /* Scale the imageFrameDatas_ to scaledY and scaledUV */ + unsigned int halfSizeWidth = (size.width + 1) / 2; + unsigned int halfSizeHeight = (size.height + 1) / 2; + std::unique_ptr scaledY = + std::make_unique(size.width * size.height); + std::unique_ptr scaledUV = + std::make_unique(halfSizeWidth * halfSizeHeight * 2); + auto &src = imageFrameDatas_[i]; + + /* + * \todo Some platforms might enforce stride due to GPU, like + * ChromeOS ciri (64). The weight needs to be a multiple of + * the stride to work properly for now. + */ + libyuv::NV12Scale(src.Y.get(), src.size.width, + src.UV.get(), src.size.width, + src.size.width, src.size.height, + scaledY.get(), size.width, scaledUV.get(), size.width, + size.width, size.height, libyuv::FilterMode::kFilterBilinear); + + scaledFrameDatas_.emplace_back( + ImageFrameData{ std::move(scaledY), std::move(scaledUV), size }); + } +} + +void ImageFrameGenerator::generateFrame(const Size &size, const FrameBuffer *buffer) +{ + /* Don't do anything when the list of buffers is empty*/ + ASSERT(!scaledFrameDatas_.empty()); + + MappedFrameBuffer mappedFrameBuffer(buffer, MappedFrameBuffer::MapFlag::Write); + + auto planes = mappedFrameBuffer.planes(); + + /* Make sure the frameCount does not over the number of images */ + frameCount_ %= imageFrames_->number.value_or(1); + + /* Write the scaledY and scaledUV to the mapped frame buffer */ + libyuv::NV12Copy(scaledFrameDatas_[frameCount_].Y.get(), size.width, + scaledFrameDatas_[frameCount_].UV.get(), size.width, planes[0].begin(), + size.width, planes[1].begin(), size.width, + size.width, size.height); + + /* proceed an image every 4 frames */ + /* \todo read the parameter_ from the configuration file? */ + parameter_++; + if (parameter_ % 4 == 0) + frameCount_++; +} + +/** + * \var ImageFrameGenerator::imageFrameDatas_ + * \brief List of pointers to the not scaled image buffers + */ + +/** + * \var ImageFrameGenerator::scaledFrameDatas_ + * \brief List of pointers to the scaled image buffers + */ + +/** + * \var ImageFrameGenerator::imageFrames_ + * \brief Pointer to the imageFrames_ in VirtualCameraData + */ + +/** + * \var ImageFrameGenerator::parameter_ + * \brief Speed parameter. Change to the next image every parameter_ frames + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/image_frame_generator.h b/src/libcamera/pipeline/virtual/image_frame_generator.h new file mode 100644 index 000000000..4ad8aad24 --- /dev/null +++ b/src/libcamera/pipeline/virtual/image_frame_generator.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * image_frame_generator.h - Derived class of FrameGenerator for + * generating frames from images + */ + +#pragma once + +#include +#include +#include +#include +#include + +#include "frame_generator.h" + +namespace libcamera { + +enum class ScaleMode : char { + Fill = 0, +}; + +/* Frame configuration provided by the config file */ +struct ImageFrames { + std::filesystem::path path; + ScaleMode scaleMode; + std::optional number; +}; + +class ImageFrameGenerator : public FrameGenerator +{ +public: + static std::unique_ptr create(ImageFrames &imageFrames); + +private: + struct ImageFrameData { + std::unique_ptr Y; + std::unique_ptr UV; + Size size; + }; + + void configure(const Size &size) override; + void generateFrame(const Size &size, const FrameBuffer *buffer) override; + + std::vector imageFrameDatas_; + std::vector scaledFrameDatas_; + ImageFrames *imageFrames_; + unsigned int frameCount_; + unsigned int parameter_; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build index d72ac5be7..395919b39 100644 --- a/src/libcamera/pipeline/virtual/meson.build +++ b/src/libcamera/pipeline/virtual/meson.build @@ -1,9 +1,13 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'image_frame_generator.cpp', 'parser.cpp', 'test_pattern_generator.cpp', 'virtual.cpp', ]) +libjpeg = dependency('libjpeg', required : false) + libcamera_deps += [libyuv_dep] +libcamera_deps += [libjpeg] diff --git a/src/libcamera/pipeline/virtual/parser.cpp b/src/libcamera/pipeline/virtual/parser.cpp index e579a3f7c..f0797fc7e 100644 --- a/src/libcamera/pipeline/virtual/parser.cpp +++ b/src/libcamera/pipeline/virtual/parser.cpp @@ -52,12 +52,12 @@ Parser::parseConfigFile(File &file, PipelineHandler *pipe) continue; } - data->id_ = cameraId; + data->config_.id = cameraId; ControlInfoMap::Map controls; /* todo: Check which resolution's frame rate to be reported */ controls[&controls::FrameDurationLimits] = - ControlInfo(int64_t(1000000 / data->supportedResolutions_[0].frameRates[1]), - int64_t(1000000 / data->supportedResolutions_[0].frameRates[0])); + ControlInfo(int64_t(1000000 / data->config_.resolutions[0].frameRates[1]), + int64_t(1000000 / data->config_.resolutions[0].frameRates[0])); data->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); configurations.push_back(std::move(data)); } @@ -75,7 +75,8 @@ Parser::parseCameraConfigData(const YamlObject &cameraConfigData, std::unique_ptr data = std::make_unique(pipe, resolutions); - if (parseTestPattern(cameraConfigData, data.get())) + if (parseTestPattern(cameraConfigData, data.get()) && + parseFrame(cameraConfigData, data.get())) return nullptr; if (parseLocation(cameraConfigData, data.get())) @@ -142,16 +143,75 @@ int Parser::parseTestPattern(const YamlObject &cameraConfigData, VirtualCameraDa { std::string testPattern = cameraConfigData["test_pattern"].get(""); - /* Default value is "bars" */ if (testPattern == "bars") { - data->testPattern_ = TestPattern::ColorBars; + data->config_.frame = TestPattern::ColorBars; } else if (testPattern == "lines") { - data->testPattern_ = TestPattern::DiagonalLines; + data->config_.frame = TestPattern::DiagonalLines; } else { - LOG(Virtual, Error) << "Test pattern: " << testPattern + LOG(Virtual, Debug) << "Test pattern: " << testPattern << "is not supported"; return -EINVAL; } + + return 0; +} + +int Parser::parseFrame(const YamlObject &cameraConfigData, VirtualCameraData *data) +{ + const YamlObject &frames = cameraConfigData["frames"]; + + /* When there is no frames provided in the config file, use color bar test pattern */ + if (frames.size() == 0) { + data->config_.frame = TestPattern::ColorBars; + return 0; + } + + if (!frames.isDictionary()) { + LOG(Virtual, Error) << "'frames' is not a dictionary."; + return -EINVAL; + } + + std::string path = frames["path"].get(""); + + ScaleMode scaleMode; + if (auto ext = std::filesystem::path(path).extension(); + ext == ".jpg" || ext == ".jpeg") { + if (parseScaleMode(frames, &scaleMode)) + return -EINVAL; + data->config_.frame = ImageFrames{ path, scaleMode, std::nullopt }; + } else if (std::filesystem::is_directory(std::filesystem::symlink_status(path))) { + if (parseScaleMode(frames, &scaleMode)) + return -EINVAL; + + using std::filesystem::directory_iterator; + unsigned int numOfFiles = std::distance(directory_iterator(path), directory_iterator{}); + if (numOfFiles == 0) { + LOG(Virtual, Error) << "Empty directory"; + return -EINVAL; + } + data->config_.frame = ImageFrames{ path, scaleMode, numOfFiles }; + } else { + LOG(Virtual, Error) << "Frame: " << path << " is not supported"; + return -EINVAL; + } + + return 0; +} + +int Parser::parseScaleMode( + const YamlObject &framesConfigData, ScaleMode *scaleMode) +{ + std::string mode = framesConfigData["scale_mode"].get(""); + + /* Default value is fill */ + if (mode == "fill" || mode == "") { + *scaleMode = ScaleMode::Fill; + } else { + LOG(Virtual, Error) << "scaleMode: " << mode + << " is not supported"; + return -EINVAL; + } + return 0; } diff --git a/src/libcamera/pipeline/virtual/parser.h b/src/libcamera/pipeline/virtual/parser.h index 09c3c56b8..f65616e33 100644 --- a/src/libcamera/pipeline/virtual/parser.h +++ b/src/libcamera/pipeline/virtual/parser.h @@ -35,8 +35,10 @@ private: int parseSupportedFormats(const YamlObject &cameraConfigData, std::vector *resolutions); int parseTestPattern(const YamlObject &cameraConfigData, VirtualCameraData *data); + int parseFrame(const YamlObject &cameraConfigData, VirtualCameraData *data); int parseLocation(const YamlObject &cameraConfigData, VirtualCameraData *data); int parseModel(const YamlObject &cameraConfigData, VirtualCameraData *data); + int parseScaleMode(const YamlObject &framesConfigData, ScaleMode *scaleMode); }; } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/utils.h b/src/libcamera/pipeline/virtual/utils.h new file mode 100644 index 000000000..43a14d4b5 --- /dev/null +++ b/src/libcamera/pipeline/virtual/utils.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * utils.h - Utility types for Virtual Pipeline Handler + */ + +namespace libcamera { + +template +struct overloaded : Ts... { + using Ts::operator()...; +}; +template +overloaded(Ts...) -> overloaded; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index c196a56aa..e79e6c095 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -27,6 +27,7 @@ #include "libcamera/internal/yaml_parser.h" #include "parser.h" +#include "utils.h" namespace libcamera { @@ -49,17 +50,18 @@ uint64_t currentTimestamp() VirtualCameraData::VirtualCameraData(PipelineHandler *pipe, std::vector supportedResolutions) - : Camera::Private(pipe), supportedResolutions_(std::move(supportedResolutions)) + : Camera::Private(pipe) { - for (const auto &resolution : supportedResolutions_) { - if (minResolutionSize_.isNull() || minResolutionSize_ > resolution.size) - minResolutionSize_ = resolution.size; + config_.resolutions = std::move(supportedResolutions); + for (const auto &resolution : config_.resolutions) { + if (config_.minResolutionSize.isNull() || config_.minResolutionSize > resolution.size) + config_.minResolutionSize = resolution.size; - maxResolutionSize_ = std::max(maxResolutionSize_, resolution.size); + config_.maxResolutionSize = std::max(config_.maxResolutionSize, resolution.size); } properties_.set(properties::PixelArrayActiveAreas, - { Rectangle(maxResolutionSize_) }); + { Rectangle(config_.maxResolutionSize) }); /* \todo Support multiple streams and pass multi_stream_test */ streamConfigs_.resize(kMaxStream); @@ -87,7 +89,7 @@ CameraConfiguration::Status VirtualCameraConfiguration::validate() for (StreamConfiguration &cfg : config_) { bool found = false; - for (const auto &resolution : data_->supportedResolutions_) { + for (const auto &resolution : data_->config_.resolutions) { if (resolution.size.width == cfg.size.width && resolution.size.height == cfg.size.height) { found = true; @@ -96,7 +98,7 @@ CameraConfiguration::Status VirtualCameraConfiguration::validate() } if (!found) { - cfg.size = data_->maxResolutionSize_; + cfg.size = data_->config_.maxResolutionSize; status = Adjusted; } @@ -139,11 +141,11 @@ PipelineHandlerVirtual::generateConfiguration(Camera *camera, for (const StreamRole role : roles) { std::map> streamFormats; PixelFormat pixelFormat = formats::NV12; - streamFormats[pixelFormat] = { { data->minResolutionSize_, data->maxResolutionSize_ } }; + streamFormats[pixelFormat] = { { data->config_.minResolutionSize, data->config_.maxResolutionSize } }; StreamFormats formats(streamFormats); StreamConfiguration cfg(formats); cfg.pixelFormat = pixelFormat; - cfg.size = data->maxResolutionSize_; + cfg.size = data->config_.maxResolutionSize; cfg.bufferCount = VirtualCameraConfiguration::kBufferCount; switch (role) { @@ -175,6 +177,7 @@ int PipelineHandlerVirtual::configure(Camera *camera, VirtualCameraData *data = cameraData(camera); for (size_t i = 0; i < config->size(); ++i) { config->at(i).setStream(&data->streamConfigs_[i].stream); + /* Start reading the images/generating test patterns */ data->streamConfigs_[i].frameGenerator->configure( data->streamConfigs_[i].stream.configuration().size); } @@ -268,10 +271,14 @@ bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator std::set streams; for (auto &streamConfig : data->streamConfigs_) streams.insert(&streamConfig.stream); - std::string id = data->id_; + std::string id = data->config_.id; std::shared_ptr camera = Camera::create(std::move(data), id, streams); - initFrameGenerator(camera.get()); + if (!initFrameGenerator(camera.get())) { + LOG(Virtual, Error) << "Failed to initialize frame " + << "generator for camera: " << id; + continue; + } registerCamera(std::move(camera)); } @@ -279,15 +286,30 @@ bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator return true; } -void PipelineHandlerVirtual::initFrameGenerator(Camera *camera) +bool PipelineHandlerVirtual::initFrameGenerator(Camera *camera) { auto data = cameraData(camera); - for (auto &streamConfig : data->streamConfigs_) { - if (data->testPattern_ == TestPattern::DiagonalLines) - streamConfig.frameGenerator = std::make_unique(); - else - streamConfig.frameGenerator = std::make_unique(); - } + auto &frame = data->config_.frame; + std::visit(overloaded{ + [&](TestPattern &testPattern) { + for (auto &streamConfig : data->streamConfigs_) { + if (testPattern == TestPattern::DiagonalLines) + streamConfig.frameGenerator = std::make_unique(); + else + streamConfig.frameGenerator = std::make_unique(); + } + }, + [&](ImageFrames &imageFrames) { + for (auto &streamConfig : data->streamConfigs_) + streamConfig.frameGenerator = ImageFrameGenerator::create(imageFrames); + } }, + frame); + + for (auto &streamConfig : data->streamConfigs_) + if (!streamConfig.frameGenerator) + return false; + + return true; } REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual, "virtual") diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h index 5daa960b1..efa97e889 100644 --- a/src/libcamera/pipeline/virtual/virtual.h +++ b/src/libcamera/pipeline/virtual/virtual.h @@ -8,6 +8,8 @@ #pragma once #include +#include +#include #include #include @@ -16,10 +18,14 @@ #include "libcamera/internal/dma_buf_allocator.h" #include "libcamera/internal/pipeline_handler.h" +#include "frame_generator.h" +#include "image_frame_generator.h" #include "test_pattern_generator.h" namespace libcamera { +using VirtualFrame = std::variant; + class VirtualCameraData : public Camera::Private { public: @@ -33,18 +39,22 @@ public: Stream stream; std::unique_ptr frameGenerator; }; + /* The config file is parsed to the Configuration struct */ + struct Configuration { + std::string id; + std::vector resolutions; + VirtualFrame frame; + + Size maxResolutionSize; + Size minResolutionSize; + }; VirtualCameraData(PipelineHandler *pipe, std::vector supportedResolutions); ~VirtualCameraData() = default; - std::string id_; - TestPattern testPattern_; - - const std::vector supportedResolutions_; - Size maxResolutionSize_; - Size minResolutionSize_; + Configuration config_; std::vector streamConfigs_; }; @@ -89,7 +99,7 @@ private: return static_cast(camera->_d()); } - void initFrameGenerator(Camera *camera); + bool initFrameGenerator(Camera *camera); DmaBufAllocator dmaBufAllocator_; };