From patchwork Fri Oct 4 09:59:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 21506 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 29517C3257 for ; Fri, 4 Oct 2024 10:00:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F164463530; Fri, 4 Oct 2024 12:00:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="As/0YhUx"; dkim-atps=neutral Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 31F7463535 for ; Fri, 4 Oct 2024 12:00:00 +0200 (CEST) Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-71dae4fc4c9so1644691b3a.0 for ; Fri, 04 Oct 2024 03:00:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1728035998; x=1728640798; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ejd5Ijcf3gvFoIR1J3BXtiQ5eRvIdMpIrVwtHLm3RQE=; b=As/0YhUx6ozf/LKAtuzWHIvtw1Ij49sVftGNLeLD4VMQ5LqO6jSuJAOoOD07V1ViyR I4IGtVnW6uiibm0E5f0Fv4Tp233MM1r8ojxUH4w+Qj5iUBPi3uz6ZO7tglzgc9tMhrnQ XyDfxz2Dl3jovJk5HOPihe9gjYwiwpzNnLclk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728035998; x=1728640798; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ejd5Ijcf3gvFoIR1J3BXtiQ5eRvIdMpIrVwtHLm3RQE=; b=XJGV1/CxkIEx5WFKhjdcNWWeZWTeMZwuV281lORe8MCZNUXz2KVB+thBLiEpwGW0ld TM4WvhL3CUwtn406w5CE/chDB6qCtNqt31M73OekduAWJ2HuR/6+a4lN35366TqmiSG9 zhzD9oLg/TnYFEA7ZRBtr7yFhnxI9xhTjqOQ6g5eNj18yUtOz1oIWCJPD4IZ6fSXpEnZ ICi7pv9KmIfoCgT9dE0JI0CIDUjGaiNSnUAVwPPdbN8Xbbr6+XhysPmN95OhafZmeeB9 tB9JSYlYHgmqRZOaXkVWSx6ttMNnantts/cNySmxPkaZuwgVNfgmiBEU/mGPxUBGmo7x +83A== X-Gm-Message-State: AOJu0Yz4JmlvUHQzqTLIhkobUEeCgtUZ6DjLNQxDJ4Y0cUwY4XAsyelj 2FvBCzMbnIJy7zNNEhTv5WVFsdZy0yHysqUty2H1D2ripLjCaI8s5+qaWU0xUWqnZa2xZZjQGZY jng== X-Google-Smtp-Source: AGHT+IEwaDbsRXFC9fg8tMPwxdZyWOdFBGyKTC3ag4cDGwPZBeRs2PDY7eFkTuvPrB9otl7db8C71A== X-Received: by 2002:a05:6a20:d807:b0:1cf:2d62:8584 with SMTP id adf61e73a8af0-1d6dfa35ecbmr3133463637.11.1728035997951; Fri, 04 Oct 2024 02:59:57 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (27.247.221.35.bc.googleusercontent.com. [35.221.247.27]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ded2ad931sm316911b3a.41.2024.10.04.02.59.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Oct 2024 02:59:57 -0700 (PDT) From: Harvey Yang X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Konami Shu , Harvey Yang , Yunke Cao , Tomasz Figa , Jacopo Mondi , Kieran Bingham Subject: [PATCH 15 4/7] libcamera: pipeline: Add test pattern for VirtualPipelineHandler Date: Fri, 4 Oct 2024 09:59:17 +0000 Message-ID: <20241004095946.264537-5-chenghaoyang@google.com> X-Mailer: git-send-email 2.47.0.rc0.187.ge670bccf7e-goog In-Reply-To: <20241004095946.264537-1-chenghaoyang@google.com> References: <20241004095946.264537-1-chenghaoyang@google.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Konami Shu Add a test pattern generator class hierarchy for the Virtual pipeline handler. Implement two types of test patterns: color bars and diagonal lines generator and use them in the Virtual pipeline handler. A shifting mechanism is enabled. For each frame, the image is shifted to the left by 1 pixel. It drops FPS though. Add a dependency for libyuv to the build system to generate images in NV12 format from the test pattern. Signed-off-by: Konami Shu Co-developed-by: Harvey Yang Co-developed-by: Yunke Cao Co-developed-by: Tomasz Figa Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- src/android/meson.build | 19 --- .../pipeline/virtual/frame_generator.h | 29 ++++ src/libcamera/pipeline/virtual/meson.build | 3 + .../virtual/test_pattern_generator.cpp | 137 ++++++++++++++++++ .../pipeline/virtual/test_pattern_generator.h | 53 +++++++ src/libcamera/pipeline/virtual/virtual.cpp | 41 +++++- src/libcamera/pipeline/virtual/virtual.h | 5 + src/meson.build | 19 +++ 8 files changed, 284 insertions(+), 22 deletions(-) create mode 100644 src/libcamera/pipeline/virtual/frame_generator.h create mode 100644 src/libcamera/pipeline/virtual/test_pattern_generator.cpp create mode 100644 src/libcamera/pipeline/virtual/test_pattern_generator.h diff --git a/src/android/meson.build b/src/android/meson.build index 68646120a..6341ee8b0 100644 --- a/src/android/meson.build +++ b/src/android/meson.build @@ -15,25 +15,6 @@ foreach dep : android_deps endif endforeach -libyuv_dep = dependency('libyuv', required : false) - -# Fallback to a subproject if libyuv isn't found, as it's typically not -# provided by distributions. -if not libyuv_dep.found() - cmake = import('cmake') - - libyuv_vars = cmake.subproject_options() - libyuv_vars.add_cmake_defines({'CMAKE_POSITION_INDEPENDENT_CODE': 'ON'}) - libyuv_vars.set_override_option('cpp_std', 'c++17') - libyuv_vars.append_compile_args('cpp', - '-Wno-sign-compare', - '-Wno-unused-variable', - '-Wno-unused-parameter') - libyuv_vars.append_link_args('-ljpeg') - libyuv = cmake.subproject('libyuv', options : libyuv_vars) - libyuv_dep = libyuv.dependency('yuv') -endif - android_deps += [libyuv_dep] android_hal_sources = files([ diff --git a/src/libcamera/pipeline/virtual/frame_generator.h b/src/libcamera/pipeline/virtual/frame_generator.h new file mode 100644 index 000000000..4ff41dd8d --- /dev/null +++ b/src/libcamera/pipeline/virtual/frame_generator.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * frame_generator.h - Virtual cameras helper to generate frames + */ + +#pragma once + +#include +#include + +namespace libcamera { + +class FrameGenerator +{ +public: + virtual ~FrameGenerator() = default; + + virtual void configure(const Size &size) = 0; + + virtual int generateFrame(const Size &size, + const FrameBuffer *buffer) = 0; + +protected: + FrameGenerator() {} +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build index ada1b3358..0c537777f 100644 --- a/src/libcamera/pipeline/virtual/meson.build +++ b/src/libcamera/pipeline/virtual/meson.build @@ -1,5 +1,8 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'test_pattern_generator.cpp', 'virtual.cpp', ]) + +libcamera_deps += [libyuv_dep] diff --git a/src/libcamera/pipeline/virtual/test_pattern_generator.cpp b/src/libcamera/pipeline/virtual/test_pattern_generator.cpp new file mode 100644 index 000000000..3600bd5ed --- /dev/null +++ b/src/libcamera/pipeline/virtual/test_pattern_generator.cpp @@ -0,0 +1,137 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * test_pattern_generator.cpp - Derived class of FrameGenerator for + * generating test patterns + */ + +#include "test_pattern_generator.h" + +#include + +#include "libcamera/internal/mapped_framebuffer.h" + +#include + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Virtual) + +static const unsigned int kARGBSize = 4; + +int TestPatternGenerator::generateFrame(const Size &size, + const FrameBuffer *buffer) +{ + MappedFrameBuffer mappedFrameBuffer(buffer, + MappedFrameBuffer::MapFlag::Write); + + auto planes = mappedFrameBuffer.planes(); + + shiftLeft(size); + + /* Convert the template_ to the frame buffer */ + int ret = libyuv::ARGBToNV12(template_.get(), size.width * kARGBSize, + planes[0].begin(), size.width, + planes[1].begin(), size.width, + size.width, size.height); + if (ret != 0) + LOG(Virtual, Error) << "ARGBToNV12() failed with " << ret; + + return ret; +} + +void TestPatternGenerator::shiftLeft(const Size &size) +{ + /* Store the first column temporarily */ + auto firstColumn = std::make_unique(size.height * kARGBSize); + for (size_t h = 0; h < size.height; h++) { + unsigned int index = h * size.width * kARGBSize; + unsigned int index1 = h * kARGBSize; + firstColumn[index1] = template_[index]; + firstColumn[index1 + 1] = template_[index + 1]; + firstColumn[index1 + 2] = template_[index + 2]; + firstColumn[index1 + 3] = 0x00; + } + + /* Overwrite template_ */ + uint8_t *buf = template_.get(); + for (size_t h = 0; h < size.height; h++) { + for (size_t w = 0; w < size.width - 1; w++) { + /* Overwrite with the pixel on the right */ + unsigned int index = (h * size.width + w + 1) * kARGBSize; + *buf++ = template_[index]; /* B */ + *buf++ = template_[index + 1]; /* G */ + *buf++ = template_[index + 2]; /* R */ + *buf++ = 0x00; /* A */ + } + /* Overwrite the new last column with the original first column */ + unsigned int index1 = h * kARGBSize; + *buf++ = firstColumn[index1]; /* B */ + *buf++ = firstColumn[index1 + 1]; /* G */ + *buf++ = firstColumn[index1 + 2]; /* R */ + *buf++ = 0x00; /* A */ + } +} + +void ColorBarsGenerator::configure(const Size &size) +{ + constexpr uint8_t kColorBar[8][3] = { + /* R, G, B */ + { 0xff, 0xff, 0xff }, /* White */ + { 0xff, 0xff, 0x00 }, /* Yellow */ + { 0x00, 0xff, 0xff }, /* Cyan */ + { 0x00, 0xff, 0x00 }, /* Green */ + { 0xff, 0x00, 0xff }, /* Magenta */ + { 0xff, 0x00, 0x00 }, /* Red */ + { 0x00, 0x00, 0xff }, /* Blue */ + { 0x00, 0x00, 0x00 }, /* Black */ + }; + + template_ = std::make_unique( + size.width * size.height * kARGBSize); + + unsigned int colorBarWidth = size.width / std::size(kColorBar); + + uint8_t *buf = template_.get(); + for (size_t h = 0; h < size.height; h++) { + for (size_t w = 0; w < size.width; w++) { + /* Repeat when the width is exceed */ + unsigned int index = (w / colorBarWidth) % std::size(kColorBar); + + *buf++ = kColorBar[index][2]; /* B */ + *buf++ = kColorBar[index][1]; /* G */ + *buf++ = kColorBar[index][0]; /* R */ + *buf++ = 0x00; /* A */ + } + } +} + +void DiagonalLinesGenerator::configure(const Size &size) +{ + constexpr uint8_t kColorBar[2][3] = { + /* R, G, B */ + { 0xff, 0xff, 0xff }, /* White */ + { 0x00, 0x00, 0x00 }, /* Black */ + }; + + template_ = std::make_unique( + size.width * size.height * kARGBSize); + + unsigned int lineWidth = size.width / 10; + + uint8_t *buf = template_.get(); + for (size_t h = 0; h < size.height; h++) { + for (size_t w = 0; w < size.width; w++) { + /* Repeat when the width is exceed */ + int index = ((w + h) / lineWidth) % 2; + + *buf++ = kColorBar[index][2]; /* B */ + *buf++ = kColorBar[index][1]; /* G */ + *buf++ = kColorBar[index][0]; /* R */ + *buf++ = 0x00; /* A */ + } + } +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/test_pattern_generator.h b/src/libcamera/pipeline/virtual/test_pattern_generator.h new file mode 100644 index 000000000..2ff1e40ec --- /dev/null +++ b/src/libcamera/pipeline/virtual/test_pattern_generator.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * test_pattern_generator.h - Derived class of FrameGenerator for + * generating test patterns + */ + +#pragma once + +#include + +#include +#include + +#include "frame_generator.h" + +namespace libcamera { + +enum class TestPattern : char { + ColorBars = 0, + DiagonalLines = 1, +}; + +class TestPatternGenerator : public FrameGenerator +{ +public: + int generateFrame(const Size &size, const FrameBuffer *buffer) override; + +protected: + /* Buffer of test pattern template */ + std::unique_ptr template_; + +private: + /* Shift the buffer by 1 pixel left each frame */ + void shiftLeft(const Size &size); +}; + +class ColorBarsGenerator : public TestPatternGenerator +{ +public: + /* Generate a template buffer of the color bar test pattern. */ + void configure(const Size &size) override; +}; + +class DiagonalLinesGenerator : public TestPatternGenerator +{ +public: + /* Generate a template buffer of the diagonal lines test pattern. */ + void configure(const Size &size) override; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index 5cac0e364..1ad7417c7 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -34,6 +34,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/dma_buf_allocator.h" #include "libcamera/internal/formats.h" +#include "libcamera/internal/framebuffer.h" #include "libcamera/internal/pipeline_handler.h" namespace libcamera { @@ -94,6 +95,8 @@ private: return static_cast(camera->_d()); } + void initFrameGenerator(Camera *camera); + DmaBufAllocator dmaBufAllocator_; bool resetCreated_ = false; @@ -241,8 +244,10 @@ int PipelineHandlerVirtual::configure(Camera *camera, CameraConfiguration *config) { VirtualCameraData *data = cameraData(camera); - for (auto [i, c] : utils::enumerate(*config)) + for (auto [i, c] : utils::enumerate(*config)) { c.setStream(&data->streamConfigs_[i].stream); + data->streamConfigs_[i].frameGenerator->configure(c.size); + } return 0; } @@ -278,8 +283,24 @@ void PipelineHandlerVirtual::stopDevice([[maybe_unused]] Camera *camera) int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera, Request *request) { - for (auto it : request->buffers()) - completeBuffer(request, it.second); + VirtualCameraData *data = cameraData(camera); + + for (auto const &[stream, buffer] : request->buffers()) { + bool found = false; + /* map buffer and fill test patterns */ + for (auto &streamConfig : data->streamConfigs_) { + if (stream == &streamConfig.stream) { + found = true; + if (streamConfig.frameGenerator->generateFrame( + stream->configuration().size, buffer)) + buffer->_d()->cancel(); + + completeBuffer(request, buffer); + break; + } + } + ASSERT(found); + } request->metadata().set(controls::SensorTimestamp, currentTimestamp()); completeRequest(request); @@ -325,6 +346,9 @@ bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator const std::string id = "Virtual0"; std::shared_ptr camera = Camera::create(std::move(data), id, streams); + + initFrameGenerator(camera.get()); + registerCamera(std::move(camera)); resetCreated_ = true; @@ -332,6 +356,17 @@ bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator return true; } +void PipelineHandlerVirtual::initFrameGenerator(Camera *camera) +{ + auto data = cameraData(camera); + for (auto &streamConfig : data->streamConfigs_) { + if (data->testPattern_ == TestPattern::DiagonalLines) + streamConfig.frameGenerator = std::make_unique(); + else + streamConfig.frameGenerator = std::make_unique(); + } +} + REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual, "virtual") } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h index bddd3db5a..0b15f8d9c 100644 --- a/src/libcamera/pipeline/virtual/virtual.h +++ b/src/libcamera/pipeline/virtual/virtual.h @@ -15,6 +15,8 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/pipeline_handler.h" +#include "test_pattern_generator.h" + namespace libcamera { class VirtualCameraData : public Camera::Private @@ -28,6 +30,7 @@ public: }; struct StreamConfig { Stream stream; + std::unique_ptr frameGenerator; }; VirtualCameraData(PipelineHandler *pipe, @@ -35,6 +38,8 @@ public: ~VirtualCameraData() = default; + TestPattern testPattern_ = TestPattern::ColorBars; + const std::vector supportedResolutions_; Size maxResolutionSize_; Size minResolutionSize_; diff --git a/src/meson.build b/src/meson.build index 165a77bb9..91bea7753 100644 --- a/src/meson.build +++ b/src/meson.build @@ -27,6 +27,25 @@ else ipa_sign_module = false endif +libyuv_dep = dependency('libyuv', required : false) + +# Fallback to a subproject if libyuv isn't found, as it's typically not +# provided by distributions. +if not libyuv_dep.found() + cmake = import('cmake') + + libyuv_vars = cmake.subproject_options() + libyuv_vars.add_cmake_defines({'CMAKE_POSITION_INDEPENDENT_CODE': 'ON'}) + libyuv_vars.set_override_option('cpp_std', 'c++17') + libyuv_vars.append_compile_args('cpp', + '-Wno-sign-compare', + '-Wno-unused-variable', + '-Wno-unused-parameter') + libyuv_vars.append_link_args('-ljpeg') + libyuv = cmake.subproject('libyuv', options : libyuv_vars) + libyuv_dep = libyuv.dependency('yuv') +endif + # libcamera must be built first as a dependency to the other components. subdir('libcamera')