From patchwork Thu Sep 26 16:29:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 21388 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0E723C3257 for ; Thu, 26 Sep 2024 16:32:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9669863513; Thu, 26 Sep 2024 18:32:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="SkRCDjU2"; dkim-atps=neutral Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 36CBF6350C for ; Thu, 26 Sep 2024 18:31:54 +0200 (CEST) Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-20afdad26f5so14951475ad.1 for ; Thu, 26 Sep 2024 09:31:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1727368312; x=1727973112; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VuU4cGIN+faXvAvU9Paf7WKxATqLyOfcBxu4BApn4rU=; b=SkRCDjU21YNszzlOTr4N10aSBycULDO31nJO+txd7YVZPjVfY2RYRnwMA6gcpXajTw iFJ9ilfqrJ5VAY1mx9igoZcRT8MpsThmeWFx15uyL/7D4FwfpJVnQ4aMUPJqpvNXhDfY V0l/+P9LpfMO5512/zkW1eBjLJjc3xJs1gZM0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727368312; x=1727973112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VuU4cGIN+faXvAvU9Paf7WKxATqLyOfcBxu4BApn4rU=; b=mjfhYelN9qTyzb3msR91Dxbv0FTXv2ikyhtEjrbEcbF6tLphb2icAexJlem/bd4m/8 YxAjWjWGgKoRoaBfAE9Wq21jL5bIwyX5gsknajp2o5nvZglxBREKgIXnnc8SGKPc6oIw MqvTQZU9RwL2BlCtPKJqyuQcnETW6D7ksWZzLxSu/YaZYnLHOO4Z72w5qEwBEpfmCf5p te7O6fMHx78a87YOy/mF/KW4F6Jos3FgVzrrxdx9QWBp54j0AhuJ79rJD0gzibjFzY1/ +gsg3F0utWbHggqqDrwggQhhZJ2mKdaB4OObhq73y7+HN+wf4yBnH61f3W2MQNf+RXDw qfrQ== X-Gm-Message-State: AOJu0YxcUacdN6/vppWwbx16D+6xP3b4rZAqvXZjBlKa09AVQV+7o3wU Vl7vdJqaamZdY50kT3DsmVHtzN4qsX9LJyam9wu5E3guWg5vmJn1PH4ns84FPJPZQxfrgKbn4P0 = X-Google-Smtp-Source: AGHT+IHrOwDVUHiVrvyreFqQoTuo5yRTScNRd52erM2gQu/5iQYWWgstomKoOCp7YEN1fQ7ckWk47A== X-Received: by 2002:a17:903:22c1:b0:207:1720:3aa2 with SMTP id d9443c01a7336-20b37b7a0aamr4177355ad.41.1727368312043; Thu, 26 Sep 2024 09:31:52 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (208.158.221.35.bc.googleusercontent.com. [35.221.158.208]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20b37e20dffsm469035ad.170.2024.09.26.09.31.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2024 09:31:51 -0700 (PDT) From: Harvey Yang X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Konami Shu , Harvey Yang , Yunke Cao , Tomasz Figa , Jacopo Mondi Subject: [PATCH v13 4/7] libcamera: pipeline: Add test pattern for VirtualPipelineHandler Date: Thu, 26 Sep 2024 16:29:53 +0000 Message-ID: <20240926163141.1593026-5-chenghaoyang@google.com> X-Mailer: git-send-email 2.46.1.824.gd892dcdcdd-goog In-Reply-To: <20240926163141.1593026-1-chenghaoyang@google.com> References: <20240926163141.1593026-1-chenghaoyang@google.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Konami Shu Add a test pattern generator class hierarchy for the Virtual pipeline handler. Implement two types of test patterns: color bars and diagonal lines generator and use them in the Virtual pipeline handler. A shifting mechanism is enabled. For each frame, the image is shifted to the left by 1 pixel. It drops FPS though. Add a dependency for libyuv to the build system to generate images in NV12 format from the test pattern. Signed-off-by: Konami Shu Co-developed-by: Harvey Yang Co-developed-by: Yunke Cao Co-developed-by: Tomasz Figa Reviewed-by: Jacopo Mondi --- src/android/meson.build | 19 --- .../pipeline/virtual/frame_generator.h | 29 ++++ src/libcamera/pipeline/virtual/meson.build | 3 + .../virtual/test_pattern_generator.cpp | 137 ++++++++++++++++++ .../pipeline/virtual/test_pattern_generator.h | 53 +++++++ src/libcamera/pipeline/virtual/virtual.cpp | 41 +++++- src/libcamera/pipeline/virtual/virtual.h | 5 + src/meson.build | 19 +++ 8 files changed, 284 insertions(+), 22 deletions(-) create mode 100644 src/libcamera/pipeline/virtual/frame_generator.h create mode 100644 src/libcamera/pipeline/virtual/test_pattern_generator.cpp create mode 100644 src/libcamera/pipeline/virtual/test_pattern_generator.h diff --git a/src/android/meson.build b/src/android/meson.build index 68646120..6341ee8b 100644 --- a/src/android/meson.build +++ b/src/android/meson.build @@ -15,25 +15,6 @@ foreach dep : android_deps endif endforeach -libyuv_dep = dependency('libyuv', required : false) - -# Fallback to a subproject if libyuv isn't found, as it's typically not -# provided by distributions. -if not libyuv_dep.found() - cmake = import('cmake') - - libyuv_vars = cmake.subproject_options() - libyuv_vars.add_cmake_defines({'CMAKE_POSITION_INDEPENDENT_CODE': 'ON'}) - libyuv_vars.set_override_option('cpp_std', 'c++17') - libyuv_vars.append_compile_args('cpp', - '-Wno-sign-compare', - '-Wno-unused-variable', - '-Wno-unused-parameter') - libyuv_vars.append_link_args('-ljpeg') - libyuv = cmake.subproject('libyuv', options : libyuv_vars) - libyuv_dep = libyuv.dependency('yuv') -endif - android_deps += [libyuv_dep] android_hal_sources = files([ diff --git a/src/libcamera/pipeline/virtual/frame_generator.h b/src/libcamera/pipeline/virtual/frame_generator.h new file mode 100644 index 00000000..4ff41dd8 --- /dev/null +++ b/src/libcamera/pipeline/virtual/frame_generator.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * frame_generator.h - Virtual cameras helper to generate frames + */ + +#pragma once + +#include +#include + +namespace libcamera { + +class FrameGenerator +{ +public: + virtual ~FrameGenerator() = default; + + virtual void configure(const Size &size) = 0; + + virtual int generateFrame(const Size &size, + const FrameBuffer *buffer) = 0; + +protected: + FrameGenerator() {} +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build index ada1b335..0c537777 100644 --- a/src/libcamera/pipeline/virtual/meson.build +++ b/src/libcamera/pipeline/virtual/meson.build @@ -1,5 +1,8 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'test_pattern_generator.cpp', 'virtual.cpp', ]) + +libcamera_deps += [libyuv_dep] diff --git a/src/libcamera/pipeline/virtual/test_pattern_generator.cpp b/src/libcamera/pipeline/virtual/test_pattern_generator.cpp new file mode 100644 index 00000000..c5a93d37 --- /dev/null +++ b/src/libcamera/pipeline/virtual/test_pattern_generator.cpp @@ -0,0 +1,137 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * test_pattern_generator.cpp - Derived class of FrameGenerator for + * generating test patterns + */ + +#include "test_pattern_generator.h" + +#include + +#include "libcamera/internal/mapped_framebuffer.h" + +#include + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Virtual) + +static const unsigned int kARGBSize = 4; + +int TestPatternGenerator::generateFrame(const Size &size, + const FrameBuffer *buffer) +{ + MappedFrameBuffer mappedFrameBuffer(buffer, + MappedFrameBuffer::MapFlag::Write); + + auto planes = mappedFrameBuffer.planes(); + + shiftLeft(size); + + /* Convert the template_ to the frame buffer */ + int ret = libyuv::ARGBToNV12(template_.get(), size.width * kARGBSize, + planes[0].begin(), size.width, + planes[1].begin(), size.width, + size.width, size.height); + if (ret != 0) + LOG(Virtual, Error) << "ARGBToNV12() failed with " << ret; + + return ret; +} + +void TestPatternGenerator::shiftLeft(const Size &size) +{ + /* Store the first column temporarily */ + auto firstColumn = std::make_unique(size.height * kARGBSize); + for (size_t h = 0; h < size.height; h++) { + unsigned int index = h * size.width * kARGBSize; + unsigned int index1 = h * kARGBSize; + firstColumn[index1] = template_[index]; + firstColumn[index1 + 1] = template_[index + 1]; + firstColumn[index1 + 2] = template_[index + 2]; + firstColumn[index1 + 3] = 0x00; + } + + /* Overwrite template_ */ + uint8_t *buf = template_.get(); + for (size_t h = 0; h < size.height; h++) { + for (size_t w = 0; w < size.width - 1; w++) { + /* Overwrite with the pixel on the right */ + unsigned int index = (h * size.width + w + 1) * kARGBSize; + *buf++ = template_[index]; /* B */ + *buf++ = template_[index + 1]; /* G */ + *buf++ = template_[index + 2]; /* R */ + *buf++ = 0x00; /* A */ + } + /* Overwrite the new last column with the original first column */ + unsigned int index1 = h * kARGBSize; + *buf++ = firstColumn[index1]; /* B */ + *buf++ = firstColumn[index1 + 1]; /* G */ + *buf++ = firstColumn[index1 + 2]; /* R */ + *buf++ = 0x00; /* A */ + } +} + +void ColorBarsGenerator::configure(const Size &size) +{ + constexpr uint8_t kColorBar[8][3] = { + /* R, G, B */ + { 0xff, 0xff, 0xff }, /* White */ + { 0xff, 0xff, 0x00 }, /* Yellow */ + { 0x00, 0xff, 0xff }, /* Cyan */ + { 0x00, 0xff, 0x00 }, /* Green */ + { 0xff, 0x00, 0xff }, /* Magenta */ + { 0xff, 0x00, 0x00 }, /* Red */ + { 0x00, 0x00, 0xff }, /* Blue */ + { 0x00, 0x00, 0x00 }, /* Black */ + }; + + template_ = std::make_unique( + size.width * size.height * kARGBSize); + + unsigned int colorBarWidth = size.width / std::size(kColorBar); + + uint8_t *buf = template_.get(); + for (size_t h = 0; h < size.height; h++) { + for (size_t w = 0; w < size.width; w++) { + /* Repeat when the width is exceed */ + unsigned int index = (w / colorBarWidth) % std::size(kColorBar); + + *buf++ = kColorBar[index][2]; /* B */ + *buf++ = kColorBar[index][1]; /* G */ + *buf++ = kColorBar[index][0]; /* R */ + *buf++ = 0x00; /* A */ + } + } +} + +void DiagonalLinesGenerator::configure(const Size &size) +{ + constexpr uint8_t kColorBar[8][3] = { + /* R, G, B */ + { 0xff, 0xff, 0xff }, /* White */ + { 0x00, 0x00, 0x00 }, /* Black */ + }; + + template_ = std::make_unique( + size.width * size.height * kARGBSize); + + unsigned int lineWidth = size.width / 10; + + uint8_t *buf = template_.get(); + for (size_t h = 0; h < size.height; h++) { + for (size_t w = 0; w < size.width; w++) { + /* Repeat when the width is exceed */ + int index = ((w + h) / lineWidth) % 2; + + *buf++ = kColorBar[index][2]; /* B */ + *buf++ = kColorBar[index][1]; /* G */ + *buf++ = kColorBar[index][0]; /* R */ + *buf++ = 0x00; /* A */ + } + } +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/test_pattern_generator.h b/src/libcamera/pipeline/virtual/test_pattern_generator.h new file mode 100644 index 00000000..2ff1e40e --- /dev/null +++ b/src/libcamera/pipeline/virtual/test_pattern_generator.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * test_pattern_generator.h - Derived class of FrameGenerator for + * generating test patterns + */ + +#pragma once + +#include + +#include +#include + +#include "frame_generator.h" + +namespace libcamera { + +enum class TestPattern : char { + ColorBars = 0, + DiagonalLines = 1, +}; + +class TestPatternGenerator : public FrameGenerator +{ +public: + int generateFrame(const Size &size, const FrameBuffer *buffer) override; + +protected: + /* Buffer of test pattern template */ + std::unique_ptr template_; + +private: + /* Shift the buffer by 1 pixel left each frame */ + void shiftLeft(const Size &size); +}; + +class ColorBarsGenerator : public TestPatternGenerator +{ +public: + /* Generate a template buffer of the color bar test pattern. */ + void configure(const Size &size) override; +}; + +class DiagonalLinesGenerator : public TestPatternGenerator +{ +public: + /* Generate a template buffer of the diagonal lines test pattern. */ + void configure(const Size &size) override; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index d1584f59..2b258492 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -34,6 +34,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/dma_buf_allocator.h" #include "libcamera/internal/formats.h" +#include "libcamera/internal/framebuffer.h" #include "libcamera/internal/pipeline_handler.h" namespace libcamera { @@ -93,6 +94,8 @@ private: return static_cast(camera->_d()); } + void initFrameGenerator(Camera *camera); + DmaBufAllocator dmaBufAllocator_; }; @@ -227,8 +230,10 @@ int PipelineHandlerVirtual::configure(Camera *camera, CameraConfiguration *config) { VirtualCameraData *data = cameraData(camera); - for (auto [i, c] : utils::enumerate(*config)) + for (auto [i, c] : utils::enumerate(*config)) { c.setStream(&data->streamConfigs_[i].stream); + data->streamConfigs_[i].frameGenerator->configure(c.size); + } return 0; } @@ -264,8 +269,24 @@ void PipelineHandlerVirtual::stopDevice([[maybe_unused]] Camera *camera) int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera, Request *request) { - for (auto it : request->buffers()) - completeBuffer(request, it.second); + VirtualCameraData *data = cameraData(camera); + + for (auto const &[stream, buffer] : request->buffers()) { + bool found = false; + /* map buffer and fill test patterns */ + for (auto &streamConfig : data->streamConfigs_) { + if (stream == &streamConfig.stream) { + found = true; + if (streamConfig.frameGenerator->generateFrame( + stream->configuration().size, buffer)) + buffer->_d()->cancel(); + + completeBuffer(request, buffer); + break; + } + } + ASSERT(found); + } request->metadata().set(controls::SensorTimestamp, currentTimestamp()); completeRequest(request); @@ -307,11 +328,25 @@ bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator const std::string id = "Virtual0"; std::shared_ptr camera = Camera::create(std::move(data), id, streams); + + initFrameGenerator(camera.get()); + registerCamera(std::move(camera)); return true; } +void PipelineHandlerVirtual::initFrameGenerator(Camera *camera) +{ + auto data = cameraData(camera); + for (auto &streamConfig : data->streamConfigs_) { + if (data->testPattern_ == TestPattern::DiagonalLines) + streamConfig.frameGenerator = std::make_unique(); + else + streamConfig.frameGenerator = std::make_unique(); + } +} + REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual, "virtual") } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h index 4df70a13..9b79ac09 100644 --- a/src/libcamera/pipeline/virtual/virtual.h +++ b/src/libcamera/pipeline/virtual/virtual.h @@ -15,6 +15,8 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/pipeline_handler.h" +#include "test_pattern_generator.h" + namespace libcamera { class VirtualCameraData : public Camera::Private @@ -28,6 +30,7 @@ public: }; struct StreamConfig { Stream stream; + std::unique_ptr frameGenerator; }; VirtualCameraData(PipelineHandler *pipe, @@ -35,6 +38,8 @@ public: ~VirtualCameraData() = default; + TestPattern testPattern_ = TestPattern::ColorBars; + const std::vector supportedResolutions_; Size maxResolutionSize_; Size minResolutionSize_; diff --git a/src/meson.build b/src/meson.build index 165a77bb..91bea775 100644 --- a/src/meson.build +++ b/src/meson.build @@ -27,6 +27,25 @@ else ipa_sign_module = false endif +libyuv_dep = dependency('libyuv', required : false) + +# Fallback to a subproject if libyuv isn't found, as it's typically not +# provided by distributions. +if not libyuv_dep.found() + cmake = import('cmake') + + libyuv_vars = cmake.subproject_options() + libyuv_vars.add_cmake_defines({'CMAKE_POSITION_INDEPENDENT_CODE': 'ON'}) + libyuv_vars.set_override_option('cpp_std', 'c++17') + libyuv_vars.append_compile_args('cpp', + '-Wno-sign-compare', + '-Wno-unused-variable', + '-Wno-unused-parameter') + libyuv_vars.append_link_args('-ljpeg') + libyuv = cmake.subproject('libyuv', options : libyuv_vars) + libyuv_dep = libyuv.dependency('yuv') +endif + # libcamera must be built first as a dependency to the other components. subdir('libcamera')