From patchwork Thu Aug 29 19:47:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 21049 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 66B51C323E for ; Thu, 29 Aug 2024 19:57:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A1815634A6; Thu, 29 Aug 2024 21:57:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="OJ9Bvj3n"; dkim-atps=neutral Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DA7DD63458 for ; Thu, 29 Aug 2024 21:57:07 +0200 (CEST) Received: by mail-wm1-x32b.google.com with SMTP id 5b1f17b1804b1-428e0d184b4so8872615e9.2 for ; Thu, 29 Aug 2024 12:57:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1724961427; x=1725566227; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ELIT8Vdp9VvtK30ZWCYSufAPpOhd7DlPXyMWDCSWtCM=; b=OJ9Bvj3nx1/rXaoliq+pScgrQJh5TRUL6xgWwcgD54gbX7/RhOltGYUPBTAzZ3vlEu p9WzAedaYhkmm6SUvHrduBy6Gf726aWb/guZc8G9YfomzOueoExfTDGvyEphbCunwxYo NHrXjvwoIX3FaJ/a+YYQpvhjpGnZFTQBnMRDQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724961427; x=1725566227; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ELIT8Vdp9VvtK30ZWCYSufAPpOhd7DlPXyMWDCSWtCM=; b=iJutcvOw+8qAE7h0izeteGOgy5tT/vc/yp+JuPISkJlhjE9Sxpjp3o8gUyXVd0vj8j HOMTAo2Js+5tklc1x6otw2wVgojfmuKPrOT3Vg76pXjWTtHInIvwjTL1qCMsTpQRyauq +alz7i279gpbklHV4/b3LjelUnm/SYJbANRKr3X/Hd1VuZQp4pzTa66PjfadUTi8vBcH NOXdXE+meIk/jpGsyRjAVl80eJPlJB9i36x2IkBB2Zm1yPnBDWXTmEWLSgaKMICmlUxF qTrhqg0rZ5+d36to/DqfHe0SJydzA+XIL9D443unmRpSf0NcXdqX8oXpTHH68CMGzBLV Jphw== X-Gm-Message-State: AOJu0YxGVszGM1F5wlr+BiUZ3pyMrxfEbi3ZQ82RATdwsyDYuYRxQg5n U8C6HTBgG/9zPA3mU/v6iw+d7PhvlgsFPDIfVGq7MdyMlTAY/qkYvE2008m8zIEy9rAeSIH3MeM oIw== X-Google-Smtp-Source: AGHT+IEBQa9Gv3TwceuE+Gsm8JoUNIrLMatcIEzg31y8yIyoy1tTunT5GdhWbJBe4uQrPto/z7F38A== X-Received: by 2002:adf:f242:0:b0:360:791c:aff2 with SMTP id ffacd0b85a97d-3749b58e3bdmr2530741f8f.47.1724961426882; Thu, 29 Aug 2024 12:57:06 -0700 (PDT) Received: from chenghaoyang-germany.c.googlers.com.com (161.126.77.34.bc.googleusercontent.com. [34.77.126.161]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42ba7b4271fsm54602805e9.29.2024.08.29.12.57.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Aug 2024 12:57:06 -0700 (PDT) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Harvey Yang Subject: [PATCH v10 1/7] libcamera: add DmaBufAllocator::exportBuffers() Date: Thu, 29 Aug 2024 19:47:40 +0000 Message-ID: <20240829195703.1456614-2-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog In-Reply-To: <20240829195703.1456614-1-chenghaoyang@chromium.org> References: <20240829195703.1456614-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang Add a helper function exportBuffers in DmaBufAllocator to make it easier to use. It'll be used in Virtual Pipeline Handler and SoftwareIsp. Signed-off-by: Harvey Yang --- .../libcamera/internal/dma_buf_allocator.h | 12 ++++ src/libcamera/dma_buf_allocator.cpp | 55 +++++++++++++++++++ 2 files changed, 67 insertions(+) diff --git a/include/libcamera/internal/dma_buf_allocator.h b/include/libcamera/internal/dma_buf_allocator.h index 36ec1696b..49122ed95 100644 --- a/include/libcamera/internal/dma_buf_allocator.h +++ b/include/libcamera/internal/dma_buf_allocator.h @@ -7,13 +7,18 @@ #pragma once +#include #include +#include +#include #include #include namespace libcamera { +class FrameBuffer; + class DmaBufAllocator { public: @@ -30,7 +35,14 @@ public: bool isValid() const { return providerHandle_.isValid(); } UniqueFD alloc(const char *name, std::size_t size); + int exportBuffers(unsigned int count, + const std::vector &frameSize, + std::vector> *buffers); + private: + std::unique_ptr createBuffer( + std::string name, const std::vector &frameSizes); + UniqueFD allocFromHeap(const char *name, std::size_t size); UniqueFD allocFromUDmaBuf(const char *name, std::size_t size); UniqueFD providerHandle_; diff --git a/src/libcamera/dma_buf_allocator.cpp b/src/libcamera/dma_buf_allocator.cpp index be6efb89f..d2ac175f0 100644 --- a/src/libcamera/dma_buf_allocator.cpp +++ b/src/libcamera/dma_buf_allocator.cpp @@ -23,6 +23,10 @@ #include #include +#include + +#include "libcamera/internal/formats.h" + /** * \file dma_buf_allocator.cpp * \brief dma-buf allocator @@ -205,4 +209,55 @@ UniqueFD DmaBufAllocator::alloc(const char *name, std::size_t size) return allocFromHeap(name, size); } +/** + * \brief Allocate and export buffers from the DmaBufAllocator + * \param[in] count The number of requested FrameBuffers + * \param[in] frameSizes The sizes of planes in each FrameBuffer + * \param[out] buffers Array of buffers successfully allocated + * + * \return The number of allocated buffers on success or a negative error code + * otherwise + */ +int DmaBufAllocator::exportBuffers(unsigned int count, + const std::vector &frameSizes, + std::vector> *buffers) +{ + for (unsigned int i = 0; i < count; ++i) { + std::unique_ptr buffer = + createBuffer("frame-" + std::to_string(i), frameSizes); + if (!buffer) { + LOG(DmaBufAllocator, Error) << "Unable to create buffer"; + + buffers->clear(); + return -EINVAL; + } + + buffers->push_back(std::move(buffer)); + } + + return count; +} + +std::unique_ptr +DmaBufAllocator::createBuffer(std::string name, + const std::vector &frameSizes) +{ + std::vector planes; + + unsigned int bufferSize = 0, offset = 0; + for (auto frameSize : frameSizes) + bufferSize += frameSize; + + SharedFD fd(alloc(name.c_str(), bufferSize)); + if (!fd.isValid()) + return nullptr; + + for (auto frameSize : frameSizes) { + planes.emplace_back(FrameBuffer::Plane{ fd, offset, frameSize }); + offset += frameSize; + } + + return std::make_unique(planes); +} + } /* namespace libcamera */ From patchwork Thu Aug 29 19:47:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 21050 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BC212C323E for ; Thu, 29 Aug 2024 19:57:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 261D3634A6; Thu, 29 Aug 2024 21:57:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="eGn8zxfS"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2DCF463469 for ; Thu, 29 Aug 2024 21:57:09 +0200 (CEST) Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-4281faefea9so8964235e9.2 for ; Thu, 29 Aug 2024 12:57:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1724961428; x=1725566228; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=op9M+Tjc1Fx/nSmxqXpHdIQcWLZASgXBAijpK3rPZHg=; b=eGn8zxfS2cUHnhepQSLFGZFyRTIQueFv85wc4WLWdAxEPu0AibNzC1u1jA0wx1qVru fn5GqWa5biibqDu+aoQuz6WPrJnFr4/8Ia5vIleF1SQXOaXfmGxa9kEIzSmDWcRO1iMI eRo/Tydctjh0+okpSJ87YBqHPqXacOJot6Gxw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724961428; x=1725566228; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=op9M+Tjc1Fx/nSmxqXpHdIQcWLZASgXBAijpK3rPZHg=; b=ohoxIdrRMJHHJxWbX5pRJ8KiQwq26G8Qq5ycGmt8K1EbItCO6e9zH+6LJ4lr7kIHY9 rnCJhpVtoQgLIaU/Aj8FR+Gp6zx6QMBMWlCSMrh0EjfmP8/9CfbXZx9BJCLVz5pb5nW4 7TxQb56TVp6/fEMa5NVjy2IhgAC8XsuQ8BvXDyW6dfBSBkwMAqLrb5MvIN6NxqICuvNl H3VOQrS89LJFibIEjucvdaANlZfnMxOv1kcQaKZmB9rq2u5ylhf+VZEiFKr4nA2pD/iO DT9jsUSzGCRf7GJnAWtK9Z/uhHy1Al+V7aCHgL0G0/A8vbgjxISGfJOjD7nVgC9IX0oQ 0OyQ== X-Gm-Message-State: AOJu0Yy8YlT+CJ76HHB5Y53ch/ip5usHnW9UWJM67hUT/5GMxkDlVRYO X4UPOaGJs+LGLcID3p0FMeGkc5O/Ir52Ohr9/7Iznv8yQoWoIkviyYi85+8m7JgD6RMARr6YHvd hpA== X-Google-Smtp-Source: AGHT+IFE/30fVhVcEWYoalr0ME0rS9QEJMrbtT55uRH6sTntisparhpUwUxKrZEm15efbhHSzx6tGw== X-Received: by 2002:a05:600c:46c9:b0:42b:8a35:1acf with SMTP id 5b1f17b1804b1-42bbb436e9amr145025e9.25.1724961427654; Thu, 29 Aug 2024 12:57:07 -0700 (PDT) Received: from chenghaoyang-germany.c.googlers.com.com (161.126.77.34.bc.googleusercontent.com. [34.77.126.161]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42ba7b4271fsm54602805e9.29.2024.08.29.12.57.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Aug 2024 12:57:07 -0700 (PDT) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Harvey Yang , Laurent Pinchart Subject: [PATCH v10 2/7] libcamera: Remove PipelineHandler Fatal check of non-empty MediaDevices Date: Thu, 29 Aug 2024 19:47:41 +0000 Message-ID: <20240829195703.1456614-3-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog In-Reply-To: <20240829195703.1456614-1-chenghaoyang@chromium.org> References: <20240829195703.1456614-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang The Fatal check of having at least one MediaDevice was to prevent pipeline handler implementations searching and owning media devices with custom conventions, instead of using the base function |acquireMediaDevice|. It also has the assumption that there's at least one media device to make a camera work. Now that the assumption will be broken by the virtual pipeline handler added in the following patches, and developers should be aware of the available functions in the base class to handle media devices, the Fatal check is no longer needed. Signed-off-by: Harvey Yang Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline_handler.cpp | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index 5a6de685b..2cb1a4bd7 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -605,9 +605,14 @@ void PipelineHandler::registerCamera(std::shared_ptr camera) { cameras_.push_back(camera); - if (mediaDevices_.empty()) - LOG(Pipeline, Fatal) - << "Registering camera with no media devices!"; + if (mediaDevices_.empty()) { + /* + * For virtual devices with no MediaDevice, there are no system + * devices to register. + */ + manager_->_d()->addCamera(std::move(camera)); + return; + } /* * Walk the entity list and map the devnums of all capture video nodes From patchwork Thu Aug 29 19:47:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 21051 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 42F17C32C2 for ; Thu, 29 Aug 2024 19:57:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 09C0A634D2; Thu, 29 Aug 2024 21:57:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="HbMYEWWi"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6A82A6347E for ; Thu, 29 Aug 2024 21:57:09 +0200 (CEST) Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-4280bca3960so9236085e9.3 for ; Thu, 29 Aug 2024 12:57:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1724961428; x=1725566228; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ezM3n0mE+r+MM5FN9/A8FK5e+VugHHICwCh/6CBa0wg=; b=HbMYEWWijC/unzZD+0/FpyOOQpaSvuOgrz1n8jv6PaqBsC0UiSY0UaLQKFKASlnd5p roNB3HO6yAbAHRFfstUp5qbu7xoZkS7X3aO46LmLJQk1UY+Bm/uAoKjvKczYMjX8U57f eJ/rBzgbkt/7eGRE8OpUn+QL0M/FytRrgBhxs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724961428; x=1725566228; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ezM3n0mE+r+MM5FN9/A8FK5e+VugHHICwCh/6CBa0wg=; b=ZWV/HuIAVFZAAOvypzdQPKNR6I2yt4N2VnnHa833j6UAgLE6t2e34N/NA8JKwJPUdY CTxe27+X7JAUzdQR9gF9pLqHinZ/3bsh6DNbEvPs77mw5mh8hAIK5vt6nDDWozAgc0sv 3DOC7CEHIpsYHolaNRo42BgjNvOLZ4UPvn58CBRZ6BdvLsk6IKDe9vEUXeobuWMIOthc t4+54v83/wckEVOjiWvO7JNoy95pBoPDaV2S60/fRp2E2hTN/HsonLMirQMknd3NHUmP 66HxjTznDHKNCtL1hW48VEIST3rZs+Qe+vXiraUMW99XJ3mfBZRhWb8hJ9X19GqyVehJ OVMg== X-Gm-Message-State: AOJu0YznRFlNepu00wB1aWrnvBfWN2OfTPf6tKliExntfZAjs8n6zCjK JKSPuagsTEOlHYQaRYxrQDOkP9Qr9wHmlDLD8KfVJkWBNXpBikEfPGfZ0yAjQOybzU2Kjyv4j1b ZqQ== X-Google-Smtp-Source: AGHT+IGYnMSrbU1ZaHILfhaMcadO9RPJ47OCAXZjQbglniS48HLmG9NxzRi7Pe/g2AOCRODCR40cQg== X-Received: by 2002:a05:600c:1391:b0:426:5546:71a with SMTP id 5b1f17b1804b1-42bb023cdcbmr32488515e9.2.1724961428294; Thu, 29 Aug 2024 12:57:08 -0700 (PDT) Received: from chenghaoyang-germany.c.googlers.com.com (161.126.77.34.bc.googleusercontent.com. [34.77.126.161]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42ba7b4271fsm54602805e9.29.2024.08.29.12.57.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Aug 2024 12:57:07 -0700 (PDT) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang Subject: [PATCH v10 3/7] libcamera: virtual: Add VirtualPipelineHandler Date: Thu, 29 Aug 2024 19:47:42 +0000 Message-ID: <20240829195703.1456614-4-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog In-Reply-To: <20240829195703.1456614-1-chenghaoyang@chromium.org> References: <20240829195703.1456614-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add VirtualPipelineHandler for more unit tests and verfiy libcamera infrastructure works on devices without using hardware cameras. Signed-off-by: Harvey Yang --- meson.build | 1 + meson_options.txt | 3 +- src/libcamera/pipeline/virtual/meson.build | 5 + src/libcamera/pipeline/virtual/virtual.cpp | 261 +++++++++++++++++++++ src/libcamera/pipeline/virtual/virtual.h | 89 +++++++ 5 files changed, 358 insertions(+), 1 deletion(-) create mode 100644 src/libcamera/pipeline/virtual/meson.build create mode 100644 src/libcamera/pipeline/virtual/virtual.cpp create mode 100644 src/libcamera/pipeline/virtual/virtual.h diff --git a/meson.build b/meson.build index 432ae1337..ff9a70cf8 100644 --- a/meson.build +++ b/meson.build @@ -214,6 +214,7 @@ pipelines_support = { 'simple': ['any'], 'uvcvideo': ['any'], 'vimc': ['test'], + 'virtual': ['test'], } if pipelines.contains('all') diff --git a/meson_options.txt b/meson_options.txt index 7aa412491..c91cd241a 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -53,7 +53,8 @@ option('pipelines', 'rpi/vc4', 'simple', 'uvcvideo', - 'vimc' + 'vimc', + 'virtual' ], description : 'Select which pipeline handlers to build. If this is set to "auto", all the pipelines applicable to the target architecture will be built. If this is set to "all", all the pipelines will be built. If both are selected then "all" will take precedence.') diff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build new file mode 100644 index 000000000..ada1b3358 --- /dev/null +++ b/src/libcamera/pipeline/virtual/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +libcamera_internal_sources += files([ + 'virtual.cpp', +]) diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp new file mode 100644 index 000000000..d4e3f60ad --- /dev/null +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -0,0 +1,261 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * virtual.cpp - Pipeline handler for virtual cameras + */ + +#include "virtual.h" + +#include +#include +#include +#include +#include +#include + +#include + +#include +#include +#include +#include + +#include "libcamera/internal/camera.h" +#include "libcamera/internal/formats.h" +#include "libcamera/internal/pipeline_handler.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(Virtual) + +namespace { + +uint64_t currentTimestamp() +{ + struct timespec ts; + if (clock_gettime(CLOCK_MONOTONIC, &ts) < 0) { + LOG(Virtual, Error) << "Get clock time fails"; + return 0; + } + + return ts.tv_sec * 1'000'000'000LL + ts.tv_nsec; +} + +} /* namespace */ + +VirtualCameraData::VirtualCameraData(PipelineHandler *pipe, + std::vector supportedResolutions) + : Camera::Private(pipe), supportedResolutions_(std::move(supportedResolutions)) +{ + for (const auto &resolution : supportedResolutions_) { + if (minResolutionSize_.isNull() || minResolutionSize_ > resolution.size) + minResolutionSize_ = resolution.size; + + maxResolutionSize_ = std::max(maxResolutionSize_, resolution.size); + } + + /* \todo Support multiple streams and pass multi_stream_test */ + streamConfigs_.resize(kMaxStream); +} + +VirtualCameraConfiguration::VirtualCameraConfiguration(VirtualCameraData *data) + : CameraConfiguration(), data_(data) +{ +} + +CameraConfiguration::Status VirtualCameraConfiguration::validate() +{ + Status status = Valid; + + if (config_.empty()) { + LOG(Virtual, Error) << "Empty config"; + return Invalid; + } + + /* Only one stream is supported */ + if (config_.size() > VirtualCameraData::kMaxStream) { + config_.resize(VirtualCameraData::kMaxStream); + status = Adjusted; + } + + for (StreamConfiguration &cfg : config_) { + bool found = false; + for (const auto &resolution : data_->supportedResolutions_) { + if (resolution.size.width == cfg.size.width && + resolution.size.height == cfg.size.height) { + found = true; + break; + } + } + + if (!found) { + cfg.size = data_->maxResolutionSize_; + status = Adjusted; + } + + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + cfg.stride = info.stride(cfg.size.width, 0, 1); + cfg.frameSize = info.frameSize(cfg.size, 1); + + cfg.bufferCount = VirtualCameraConfiguration::kBufferCount; + + if (cfg.pixelFormat != formats::NV12) { + cfg.pixelFormat = formats::NV12; + LOG(Virtual, Debug) + << "Stream configuration adjusted to " << cfg.toString(); + status = Adjusted; + } + } + + return status; +} + +PipelineHandlerVirtual::PipelineHandlerVirtual(CameraManager *manager) + : PipelineHandler(manager), + dmaBufAllocator_(DmaBufAllocator::DmaBufAllocatorFlag::CmaHeap | + DmaBufAllocator::DmaBufAllocatorFlag::SystemHeap | + DmaBufAllocator::DmaBufAllocatorFlag::UDmaBuf) +{ +} + +std::unique_ptr +PipelineHandlerVirtual::generateConfiguration(Camera *camera, + Span roles) +{ + VirtualCameraData *data = cameraData(camera); + auto config = + std::make_unique(data); + + if (roles.empty()) + return config; + + for (const StreamRole role : roles) { + std::map> streamFormats; + PixelFormat pixelFormat = formats::NV12; + streamFormats[pixelFormat] = { { data->minResolutionSize_, data->maxResolutionSize_ } }; + StreamFormats formats(streamFormats); + StreamConfiguration cfg(formats); + cfg.pixelFormat = pixelFormat; + cfg.size = data->maxResolutionSize_; + cfg.bufferCount = VirtualCameraConfiguration::kBufferCount; + + switch (role) { + case StreamRole::StillCapture: + case StreamRole::VideoRecording: + case StreamRole::Viewfinder: + break; + + case StreamRole::Raw: + default: + LOG(Virtual, Error) + << "Requested stream role not supported: " << role; + config.reset(); + return config; + } + + config->addConfiguration(cfg); + } + + if (config->validate() == CameraConfiguration::Invalid) + config.reset(); + + return config; +} + +int PipelineHandlerVirtual::configure(Camera *camera, + CameraConfiguration *config) +{ + VirtualCameraData *data = cameraData(camera); + for (size_t i = 0; i < config->size(); ++i) + config->at(i).setStream(&data->streamConfigs_[i].stream); + + return 0; +} + +int PipelineHandlerVirtual::exportFrameBuffers([[maybe_unused]] Camera *camera, + Stream *stream, + std::vector> *buffers) +{ + if (!dmaBufAllocator_.isValid()) + return -ENOBUFS; + + const StreamConfiguration &config = stream->configuration(); + + auto info = PixelFormatInfo::info(config.pixelFormat); + + std::vector planeSizes; + for (size_t i = 0; i < info.planes.size(); ++i) + planeSizes.push_back(info.planeSize(config.size, i)); + + return dmaBufAllocator_.exportBuffers(config.bufferCount, planeSizes, buffers); +} + +int PipelineHandlerVirtual::start([[maybe_unused]] Camera *camera, + [[maybe_unused]] const ControlList *controls) +{ + return 0; +} + +void PipelineHandlerVirtual::stopDevice([[maybe_unused]] Camera *camera) +{ +} + +int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera, + Request *request) +{ + /* \todo Read from the virtual video if any. */ + for (auto it : request->buffers()) + completeBuffer(request, it.second); + + request->metadata().set(controls::SensorTimestamp, currentTimestamp()); + completeRequest(request); + + return 0; +} + +// static +bool PipelineHandlerVirtual::created_ = false; + +bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator) +{ + if (created_) + return false; + + created_ = true; + + /* \todo Add virtual cameras according to a config file. */ + + std::vector supportedResolutions; + supportedResolutions.resize(2); + supportedResolutions[0] = { .size = Size(1920, 1080), .frameRates = { 30 } }; + supportedResolutions[1] = { .size = Size(1280, 720), .frameRates = { 30 } }; + + std::unique_ptr data = + std::make_unique(this, supportedResolutions); + + data->properties_.set(properties::Location, properties::CameraLocationFront); + data->properties_.set(properties::Model, "Virtual Video Device"); + data->properties_.set(properties::PixelArrayActiveAreas, { Rectangle(Size(1920, 1080)) }); + + /* \todo Set FrameDurationLimits based on config. */ + ControlInfoMap::Map controls; + int64_t min_frame_duration = 33333, max_frame_duration = 33333; + controls[&controls::FrameDurationLimits] = ControlInfo(min_frame_duration, max_frame_duration); + data->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); + + /* Create and register the camera. */ + std::set streams; + for (auto &streamConfig : data->streamConfigs_) + streams.insert(&streamConfig.stream); + + const std::string id = "Virtual0"; + std::shared_ptr camera = Camera::create(std::move(data), id, streams); + registerCamera(std::move(camera)); + + return true; +} + +REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual, "virtual") + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h new file mode 100644 index 000000000..fb3dbcad3 --- /dev/null +++ b/src/libcamera/pipeline/virtual/virtual.h @@ -0,0 +1,89 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * virtual.h - Pipeline handler for virtual cameras + */ + +#pragma once + +#include +#include + +#include + +#include "libcamera/internal/camera.h" +#include "libcamera/internal/dma_buf_allocator.h" +#include "libcamera/internal/pipeline_handler.h" + +namespace libcamera { + +class VirtualCameraData : public Camera::Private +{ +public: + const static unsigned int kMaxStream = 1; + + struct Resolution { + Size size; + std::vector frameRates; + }; + struct StreamConfig { + Stream stream; + }; + + VirtualCameraData(PipelineHandler *pipe, + std::vector supportedResolutions); + + ~VirtualCameraData() = default; + + const std::vector supportedResolutions_; + Size maxResolutionSize_; + Size minResolutionSize_; + + std::vector streamConfigs_; +}; + +class VirtualCameraConfiguration : public CameraConfiguration +{ +public: + static constexpr unsigned int kBufferCount = 4; + + VirtualCameraConfiguration(VirtualCameraData *data); + + Status validate() override; + +private: + const VirtualCameraData *data_; +}; + +class PipelineHandlerVirtual : public PipelineHandler +{ +public: + PipelineHandlerVirtual(CameraManager *manager); + + std::unique_ptr generateConfiguration(Camera *camera, + Span roles) override; + int configure(Camera *camera, CameraConfiguration *config) override; + + int exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) override; + + int start(Camera *camera, const ControlList *controls) override; + void stopDevice(Camera *camera) override; + + int queueRequestDevice(Camera *camera, Request *request) override; + + bool match(DeviceEnumerator *enumerator) override; + +private: + static bool created_; + + VirtualCameraData *cameraData(Camera *camera) + { + return static_cast(camera->_d()); + } + + DmaBufAllocator dmaBufAllocator_; +}; + +} /* namespace libcamera */ From patchwork Thu Aug 29 19:47:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 21052 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7CC1DC323E for ; Thu, 29 Aug 2024 19:57:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 21E3163486; Thu, 29 Aug 2024 21:57:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="OYtKxLMM"; dkim-atps=neutral Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4406063456 for ; Thu, 29 Aug 2024 21:57:10 +0200 (CEST) Received: by mail-wr1-x42f.google.com with SMTP id ffacd0b85a97d-37186c2278bso1176600f8f.1 for ; Thu, 29 Aug 2024 12:57:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1724961429; x=1725566229; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eDdOJNBfN1Q9u3EgerR9sPOU2V44OMCOmwM8jfeXUn8=; b=OYtKxLMMc0xgJeYio6UZrZ5NnfMu8x28AwcmMNNxvkM+2iyHyrEAkLpQOCbrBwEHT5 5xXnOLN4DcHnEehphcRM7uxWMrsBp91tgxb86owulRaVF0SchQpQkrqPYk9WzLBP/8R1 4+dD37Vrga5jhRB6HlbignSFETQf9zuxBkVYo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724961429; x=1725566229; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eDdOJNBfN1Q9u3EgerR9sPOU2V44OMCOmwM8jfeXUn8=; b=qixGOLnLuRkhCqyfNc/B7gcRKdBmC3vouyBc3Kf9T0IoyAiQ/4VSVy/coQzfGtQUN8 bx/R0+KuOtvXhkp2LxbR03I7h+/ijVOgoYvysDQBKo4DRY3OfVSm67qM9qWe41XZ82Jj xI16twSmUQjwkxSvixlPvT5mMnkceqWdqcg84rAl07rZBeAglXb74D8xgrFR+/92zVaz RFeeiB7JA3PIs86MN4FHyTbBfY6Zt1srI8lLdLKRid5v+zxQrWy7Zg3yWE0U5nUyIACB HW9xt54kXcmV2xYJ4Jw8AX0bot4o3MxFWfu51HGaQupssZbMCiRtrPV7VJ2Qs7GCmfEv zP/A== X-Gm-Message-State: AOJu0YyG3MmqyifReFSdi/E5HgDOQ117V/N9ao2fpMF8h1YbWp6MiVOd x/RrhfieYfVaKfR4wC+Hw7VoAfMPIb102iB/FLrcJ0e2ss4hW8725dNuDVmjIoqPrvF9+Pxv0dS /mQ== X-Google-Smtp-Source: AGHT+IFPbx9BoKh5oQPHsp77QUmBhNSMOuZ2M1esTd74IO0C1eC67dXzfM9wOgKkpL663Ou8TVqZqw== X-Received: by 2002:adf:fd0a:0:b0:371:6fcf:5256 with SMTP id ffacd0b85a97d-3749c1de10fmr2012175f8f.18.1724961429410; Thu, 29 Aug 2024 12:57:09 -0700 (PDT) Received: from chenghaoyang-germany.c.googlers.com.com (161.126.77.34.bc.googleusercontent.com. [34.77.126.161]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42ba7b4271fsm54602805e9.29.2024.08.29.12.57.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Aug 2024 12:57:08 -0700 (PDT) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Konami Shu , Harvey Yang , Yunke Cao , Tomasz Figa Subject: [PATCH v10 4/7] libcamera: pipeline: Add test pattern for VirtualPipelineHandler Date: Thu, 29 Aug 2024 19:47:43 +0000 Message-ID: <20240829195703.1456614-5-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog In-Reply-To: <20240829195703.1456614-1-chenghaoyang@chromium.org> References: <20240829195703.1456614-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Konami Shu Add a test pattern generator class hierarchy for the Virtual pipeline handler. Implement two types of test patterns: color bars and diagonal lines generator and use them in the Virtual pipeline handler. Add a dependency for libyuv to the build system to generate images in NV12 format from the test pattern. Signed-off-by: Konami Shu Co-developed-by: Harvey Yang Co-developed-by: Yunke Cao Co-developed-by: Tomasz Figa libcamera: pipeline: Shift test pattern by 1 pixel left every frame - This write the buffer every frame - Shifting makes the frame rate dropped from about 160 to 40 Patchset1->2 - Use constant instead of using a magic number Patchset2->3 - Make shiftLeft() private Signed-off-by: Konami Shu Co-developed-by: Harvey Yang Co-developed-by: Yunke Cao Co-developed-by: Tomasz Figa --- src/android/meson.build | 19 --- .../pipeline/virtual/frame_generator.h | 29 ++++ src/libcamera/pipeline/virtual/meson.build | 3 + .../virtual/test_pattern_generator.cpp | 140 ++++++++++++++++++ .../pipeline/virtual/test_pattern_generator.h | 57 +++++++ src/libcamera/pipeline/virtual/virtual.cpp | 38 ++++- src/libcamera/pipeline/virtual/virtual.h | 7 + src/meson.build | 19 +++ 8 files changed, 290 insertions(+), 22 deletions(-) create mode 100644 src/libcamera/pipeline/virtual/frame_generator.h create mode 100644 src/libcamera/pipeline/virtual/test_pattern_generator.cpp create mode 100644 src/libcamera/pipeline/virtual/test_pattern_generator.h diff --git a/src/android/meson.build b/src/android/meson.build index 68646120a..6341ee8b0 100644 --- a/src/android/meson.build +++ b/src/android/meson.build @@ -15,25 +15,6 @@ foreach dep : android_deps endif endforeach -libyuv_dep = dependency('libyuv', required : false) - -# Fallback to a subproject if libyuv isn't found, as it's typically not -# provided by distributions. -if not libyuv_dep.found() - cmake = import('cmake') - - libyuv_vars = cmake.subproject_options() - libyuv_vars.add_cmake_defines({'CMAKE_POSITION_INDEPENDENT_CODE': 'ON'}) - libyuv_vars.set_override_option('cpp_std', 'c++17') - libyuv_vars.append_compile_args('cpp', - '-Wno-sign-compare', - '-Wno-unused-variable', - '-Wno-unused-parameter') - libyuv_vars.append_link_args('-ljpeg') - libyuv = cmake.subproject('libyuv', options : libyuv_vars) - libyuv_dep = libyuv.dependency('yuv') -endif - android_deps += [libyuv_dep] android_hal_sources = files([ diff --git a/src/libcamera/pipeline/virtual/frame_generator.h b/src/libcamera/pipeline/virtual/frame_generator.h new file mode 100644 index 000000000..d8727b8f3 --- /dev/null +++ b/src/libcamera/pipeline/virtual/frame_generator.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * frame_generator.h - Virtual cameras helper to generate frames + */ + +#pragma once + +#include +#include + +namespace libcamera { + +class FrameGenerator +{ +public: + virtual ~FrameGenerator() = default; + + virtual void configure(const Size &size) = 0; + + virtual void generateFrame(const Size &size, + const FrameBuffer *buffer) = 0; + +protected: + FrameGenerator() {} +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build index ada1b3358..0c537777f 100644 --- a/src/libcamera/pipeline/virtual/meson.build +++ b/src/libcamera/pipeline/virtual/meson.build @@ -1,5 +1,8 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'test_pattern_generator.cpp', 'virtual.cpp', ]) + +libcamera_deps += [libyuv_dep] diff --git a/src/libcamera/pipeline/virtual/test_pattern_generator.cpp b/src/libcamera/pipeline/virtual/test_pattern_generator.cpp new file mode 100644 index 000000000..7094818e5 --- /dev/null +++ b/src/libcamera/pipeline/virtual/test_pattern_generator.cpp @@ -0,0 +1,140 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * test_pattern_generator.cpp - Derived class of FrameGenerator for + * generating test patterns + */ + +#include "test_pattern_generator.h" + +#include + +#include "libcamera/internal/mapped_framebuffer.h" + +#include "libyuv/convert_from_argb.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Virtual) + +static const unsigned int kARGBSize = 4; + +void TestPatternGenerator::generateFrame(const Size &size, + const FrameBuffer *buffer) +{ + MappedFrameBuffer mappedFrameBuffer(buffer, + MappedFrameBuffer::MapFlag::Write); + + auto planes = mappedFrameBuffer.planes(); + + shiftLeft(size); + + /* Convert the template_ to the frame buffer */ + int ret = libyuv::ARGBToNV12( + template_.get(), size.width * kARGBSize, + planes[0].begin(), size.width, + planes[1].begin(), size.width, + size.width, size.height); + if (ret != 0) + LOG(Virtual, Error) << "ARGBToNV12() failed with " << ret; +} + +void TestPatternGenerator::shiftLeft(const Size &size) +{ + /* Store the first column temporarily */ + auto firstColumn = std::make_unique(size.height * kARGBSize); + for (size_t h = 0; h < size.height; h++) { + unsigned int index = h * size.width * kARGBSize; + unsigned int index1 = h * kARGBSize; + firstColumn[index1] = template_[index]; + firstColumn[index1 + 1] = template_[index + 1]; + firstColumn[index1 + 2] = template_[index + 2]; + firstColumn[index1 + 3] = 0x00; + } + + /* Overwrite template_ */ + uint8_t *buf = template_.get(); + for (size_t h = 0; h < size.height; h++) { + for (size_t w = 0; w < size.width - 1; w++) { + /* Overwrite with the pixel on the right */ + unsigned int index = (h * size.width + w + 1) * kARGBSize; + *buf++ = template_[index]; // B + *buf++ = template_[index + 1]; // G + *buf++ = template_[index + 2]; // R + *buf++ = 0x00; // A + } + /* Overwrite the new last column with the original first column */ + unsigned int index1 = h * kARGBSize; + *buf++ = firstColumn[index1]; // B + *buf++ = firstColumn[index1 + 1]; // G + *buf++ = firstColumn[index1 + 2]; // R + *buf++ = 0x00; // A + } +} + +ColorBarsGenerator::ColorBarsGenerator() = default; + +void ColorBarsGenerator::configure(const Size &size) +{ + constexpr uint8_t kColorBar[8][3] = { + // R, G, B + { 0xff, 0xff, 0xff }, // White + { 0xff, 0xff, 0x00 }, // Yellow + { 0x00, 0xff, 0xff }, // Cyan + { 0x00, 0xff, 0x00 }, // Green + { 0xff, 0x00, 0xff }, // Magenta + { 0xff, 0x00, 0x00 }, // Red + { 0x00, 0x00, 0xff }, // Blue + { 0x00, 0x00, 0x00 }, // Black + }; + + template_ = std::make_unique( + size.width * size.height * kARGBSize); + + unsigned int colorBarWidth = size.width / std::size(kColorBar); + + uint8_t *buf = template_.get(); + for (size_t h = 0; h < size.height; h++) { + for (size_t w = 0; w < size.width; w++) { + /* Repeat when the width is exceed */ + unsigned int index = (w / colorBarWidth) % std::size(kColorBar); + + *buf++ = kColorBar[index][2]; /* B */ + *buf++ = kColorBar[index][1]; /* G */ + *buf++ = kColorBar[index][0]; /* R */ + *buf++ = 0x00; /* A */ + } + } +} + +DiagonalLinesGenerator::DiagonalLinesGenerator() = default; + +void DiagonalLinesGenerator::configure(const Size &size) +{ + constexpr uint8_t kColorBar[8][3] = { + /* R, G, B */ + { 0xff, 0xff, 0xff }, /* White */ + { 0x00, 0x00, 0x00 }, /* Black */ + }; + + template_ = std::make_unique( + size.width * size.height * kARGBSize); + + unsigned int lineWidth = size.width / 10; + + uint8_t *buf = template_.get(); + for (size_t h = 0; h < size.height; h++) { + for (size_t w = 0; w < size.width; w++) { + /* Repeat when the width is exceed */ + int index = ((w + h) / lineWidth) % 2; + + *buf++ = kColorBar[index][2]; /* B */ + *buf++ = kColorBar[index][1]; /* G */ + *buf++ = kColorBar[index][0]; /* R */ + *buf++ = 0x00; /* A */ + } + } +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/test_pattern_generator.h b/src/libcamera/pipeline/virtual/test_pattern_generator.h new file mode 100644 index 000000000..11bcb5f04 --- /dev/null +++ b/src/libcamera/pipeline/virtual/test_pattern_generator.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * test_pattern_generator.h - Derived class of FrameGenerator for + * generating test patterns + */ + +#pragma once + +#include + +#include +#include + +#include "frame_generator.h" + +namespace libcamera { + +enum class TestPattern : char { + ColorBars = 0, + DiagonalLines = 1, +}; + +class TestPatternGenerator : public FrameGenerator +{ +public: + void generateFrame(const Size &size, const FrameBuffer *buffer) override; + +protected: + /* Buffer of test pattern template */ + std::unique_ptr template_; + +private: + /* Shift the buffer by 1 pixel left each frame */ + void shiftLeft(const Size &size); +}; + +class ColorBarsGenerator : public TestPatternGenerator +{ +public: + ColorBarsGenerator(); + + /* Generate a template buffer of the color bar test pattern. */ + void configure(const Size &size) override; +}; + +class DiagonalLinesGenerator : public TestPatternGenerator +{ +public: + DiagonalLinesGenerator(); + + /* Generate a template buffer of the diagonal lines test pattern. */ + void configure(const Size &size) override; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index d4e3f60ad..690d72a22 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -167,8 +167,11 @@ int PipelineHandlerVirtual::configure(Camera *camera, CameraConfiguration *config) { VirtualCameraData *data = cameraData(camera); - for (size_t i = 0; i < config->size(); ++i) + for (size_t i = 0; i < config->size(); ++i) { config->at(i).setStream(&data->streamConfigs_[i].stream); + data->streamConfigs_[i].frameGenerator->configure( + data->streamConfigs_[i].stream.configuration().size); + } return 0; } @@ -204,9 +207,24 @@ void PipelineHandlerVirtual::stopDevice([[maybe_unused]] Camera *camera) int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera, Request *request) { + VirtualCameraData *data = cameraData(camera); + /* \todo Read from the virtual video if any. */ - for (auto it : request->buffers()) - completeBuffer(request, it.second); + for (auto const &[stream, buffer] : request->buffers()) { + bool found = false; + /* map buffer and fill test patterns */ + for (auto &streamConfig : data->streamConfigs_) { + if (stream == &streamConfig.stream) { + found = true; + streamConfig.frameGenerator->generateFrame( + stream->configuration().size, buffer); + completeBuffer(request, buffer); + break; + } + } + if (!found) + LOG(Virtual, Fatal) << "Stream not defined"; + } request->metadata().set(controls::SensorTimestamp, currentTimestamp()); completeRequest(request); @@ -251,11 +269,25 @@ bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator const std::string id = "Virtual0"; std::shared_ptr camera = Camera::create(std::move(data), id, streams); + + initFrameGenerator(camera.get()); + registerCamera(std::move(camera)); return true; } +void PipelineHandlerVirtual::initFrameGenerator(Camera *camera) +{ + auto data = cameraData(camera); + for (auto &streamConfig : data->streamConfigs_) { + if (data->testPattern_ == TestPattern::DiagonalLines) + streamConfig.frameGenerator = std::make_unique(); + else + streamConfig.frameGenerator = std::make_unique(); + } +} + REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual, "virtual") } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h index fb3dbcad3..79c951a06 100644 --- a/src/libcamera/pipeline/virtual/virtual.h +++ b/src/libcamera/pipeline/virtual/virtual.h @@ -16,6 +16,8 @@ #include "libcamera/internal/dma_buf_allocator.h" #include "libcamera/internal/pipeline_handler.h" +#include "test_pattern_generator.h" + namespace libcamera { class VirtualCameraData : public Camera::Private @@ -29,6 +31,7 @@ public: }; struct StreamConfig { Stream stream; + std::unique_ptr frameGenerator; }; VirtualCameraData(PipelineHandler *pipe, @@ -36,6 +39,8 @@ public: ~VirtualCameraData() = default; + TestPattern testPattern_; + const std::vector supportedResolutions_; Size maxResolutionSize_; Size minResolutionSize_; @@ -83,6 +88,8 @@ private: return static_cast(camera->_d()); } + void initFrameGenerator(Camera *camera); + DmaBufAllocator dmaBufAllocator_; }; diff --git a/src/meson.build b/src/meson.build index 165a77bb9..91bea7753 100644 --- a/src/meson.build +++ b/src/meson.build @@ -27,6 +27,25 @@ else ipa_sign_module = false endif +libyuv_dep = dependency('libyuv', required : false) + +# Fallback to a subproject if libyuv isn't found, as it's typically not +# provided by distributions. +if not libyuv_dep.found() + cmake = import('cmake') + + libyuv_vars = cmake.subproject_options() + libyuv_vars.add_cmake_defines({'CMAKE_POSITION_INDEPENDENT_CODE': 'ON'}) + libyuv_vars.set_override_option('cpp_std', 'c++17') + libyuv_vars.append_compile_args('cpp', + '-Wno-sign-compare', + '-Wno-unused-variable', + '-Wno-unused-parameter') + libyuv_vars.append_link_args('-ljpeg') + libyuv = cmake.subproject('libyuv', options : libyuv_vars) + libyuv_dep = libyuv.dependency('yuv') +endif + # libcamera must be built first as a dependency to the other components. subdir('libcamera') From patchwork Thu Aug 29 19:47:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 21053 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9EF22C32D5 for ; Thu, 29 Aug 2024 19:57:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DCBC7634D5; Thu, 29 Aug 2024 21:57:22 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="b7PYxXoA"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 20B0763461 for ; Thu, 29 Aug 2024 21:57:11 +0200 (CEST) Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-42bb8c6e250so5816135e9.1 for ; Thu, 29 Aug 2024 12:57:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1724961430; x=1725566230; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZZmdvzRgpzfypR6+SodM748cVYFeImRRBgndtuYIC78=; b=b7PYxXoAY/akwdyNOzG6hgXH3jwUslk+7ZNaAZfPYJdrXiUXYPe5B1cikT2pKsnGt+ DP9HixWcqvkQLHrUF5JUrOkwRQTRmrh8JjMcw+i7gsC9BINJ0aUFfjRNkkXF+oYLvMYk 1VdDLqs8gANeJgaQkTXqFjmczz0ZSfAMVf7Wg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724961430; x=1725566230; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZZmdvzRgpzfypR6+SodM748cVYFeImRRBgndtuYIC78=; b=CdAVpw9wUiWpEzik3XHvnPyFqs6WAb9RJR5d7Wrt9v5N9BZtYD71Hvs7EBH4v5oGw6 kNa6IvtXLZeje1ScR/+xhil8wHbZcjUwkVxX1UTuXY+noB27O5cQSj1z1gw+mERA3cHf S2vBFMN+kQdjaZNIiaKYtsYWjbzjE/c3Chw1bg2DntVNtfM4IQyzcg0tQdYAwhw/WEQr 2HL+TXOJI0vedfaRQ6ksV5/s1hse4KxlvSkg/fqSDUdmNSJl3KpxQ1kBIFYNvu7nFR6R Bv2mGdTRSk1EcMTD1iWN+zKBns4FANJRxHzuByKpZ8hD6jQ6EuuEkxmDn3PislMsQG3o Cdfg== X-Gm-Message-State: AOJu0YxbQWzYyC57kJo8qKm5gvquLgNwNNTvEcE6hsUl5EqlHbhsJJZY Ui8nugaMEop0acEaC32C9blYnr9zd+QvLXecp3Oa7amHskQBJPeieDLEGdJ4E5oZGSWTbBOVNV7 kog== X-Google-Smtp-Source: AGHT+IFgCBXtpdsldoM4G6XLcRH54Ztxl6ON7o3OSHG001dT8LeueGXQnE6E919AMOKYizLCLVTXOA== X-Received: by 2002:a05:600c:4586:b0:42a:b62c:8c86 with SMTP id 5b1f17b1804b1-42bb02f95b6mr33567215e9.32.1724961430361; Thu, 29 Aug 2024 12:57:10 -0700 (PDT) Received: from chenghaoyang-germany.c.googlers.com.com (161.126.77.34.bc.googleusercontent.com. [34.77.126.161]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42ba7b4271fsm54602805e9.29.2024.08.29.12.57.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Aug 2024 12:57:09 -0700 (PDT) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Konami Shu , Harvey Yang , Yunke Cao , Tomasz Figa Subject: [PATCH v10 5/7] libcamera: virtual: Read config and register cameras based on the config Date: Thu, 29 Aug 2024 19:47:44 +0000 Message-ID: <20240829195703.1456614-6-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog In-Reply-To: <20240829195703.1456614-1-chenghaoyang@chromium.org> References: <20240829195703.1456614-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Konami Shu This patch introduces the configuration file for Virtual Pipeline Handler. The config file is written in yaml, and the format is documented in `README.md`. The config file will define the camera with IDs, supported formats and image sources, etc. Only Test Patterns are supported now, while the next patches will add generators to read from real images. Signed-off-by: Konami Shu Co-developed-by: Harvey Yang Co-developed-by: Yunke Cao Co-developed-by: Tomasz Figa --- src/libcamera/pipeline/virtual/README.md | 43 ++++ .../pipeline/virtual/data/virtual.yaml | 37 ++++ .../pipeline/virtual/frame_generator.h | 3 +- src/libcamera/pipeline/virtual/meson.build | 1 + src/libcamera/pipeline/virtual/parser.cpp | 191 ++++++++++++++++++ src/libcamera/pipeline/virtual/parser.h | 42 ++++ src/libcamera/pipeline/virtual/virtual.cpp | 56 ++--- src/libcamera/pipeline/virtual/virtual.h | 1 + 8 files changed, 345 insertions(+), 29 deletions(-) create mode 100644 src/libcamera/pipeline/virtual/README.md create mode 100644 src/libcamera/pipeline/virtual/data/virtual.yaml create mode 100644 src/libcamera/pipeline/virtual/parser.cpp create mode 100644 src/libcamera/pipeline/virtual/parser.h diff --git a/src/libcamera/pipeline/virtual/README.md b/src/libcamera/pipeline/virtual/README.md new file mode 100644 index 000000000..ff1e8a5f9 --- /dev/null +++ b/src/libcamera/pipeline/virtual/README.md @@ -0,0 +1,43 @@ +# Virtual Pipeline Handler + +Virtual pipeline handler emulates fake external camera(s) on ChromeOS for testing. + +## Parse config file and register cameras + +- The config file is located at `src/libcamera/pipeline/virtual/data/virtual.yaml` + +### Config File Format +The config file contains the information about cameras' properties to register. +The config file should be a yaml file with dictionary of the cameraIds +associated with their properties as top level. The default value will be applied when any property is empty. + +Each camera block is a dictionary, containing the following keys: +- `supported_formats` (list of `VirtualCameraData::Resolution`, optional) : List of supported resolution and frame rates of the emulated camera + - `width` (`unsigned int`, default=1920): Width of the window resolution. This needs to be even. + - `height` (`unsigned int`, default=1080): Height of the window resolution. + - `frame_rates` (list of `int`, default=`[30,60]` ): Range of the frame rate (per second). The list has to be two values of the lower bound and the upper bound of the frame rate. +- `test_pattern` (`string`): Which test pattern to use as frames. The options are "bars", "lines". +- `location` (`string`, default="front"): The location of the camera. Support "front" and "back". This is displayed in qcam camera selection window but this does not change the output. +- `model` (`string`, default="Unknown"): The model name of the camera. This is displayed in qcam camera selection window but this does not change the output. + +Check `data/virtual.yaml` as the sample config file. + +### Implementation + +`Parser` class provides methods to parse the config file to register cameras +in Virtual Pipeline Handler. `parseConfigFile()` is exposed to use in +Virtual Pipeline Handler. + +This is the procedure of the Parser class: +1. `parseConfigFile()` parses the config file to `YamlObject` using `YamlParser::parse()`. + - Parse the top level of config file which are the camera ids and look into each camera properties. +2. For each camera, `parseCameraConfigData()` returns a camera with the configuration. + - The methods in the next step fill the data with the pointer to the Camera object. + - If the config file contains invalid configuration, this method returns nullptr. The camera will be skipped. +3. Parse each property and register the data. + - `parseSupportedFormats()`: Parses `supported_formats` in the config, which contains resolutions and frame rates. + - `parseTestPattern()`: Parses `test_pattern` in the config. + - `parseLocation()`: Parses `location` in the config. + - `parseModel()`: Parses `model` in the config. +4. Back to `parseConfigFile()` and append the camera configuration. +5. Returns a list of camera configurations. diff --git a/src/libcamera/pipeline/virtual/data/virtual.yaml b/src/libcamera/pipeline/virtual/data/virtual.yaml new file mode 100644 index 000000000..317beb828 --- /dev/null +++ b/src/libcamera/pipeline/virtual/data/virtual.yaml @@ -0,0 +1,37 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +"Virtual0": + supported_formats: + - width: 1920 + height: 1080 + frame_rates: + - 30 + - 60 + - width: 1680 + height: 1050 + frame_rates: + - 70 + - 80 + test_pattern: "lines" + location: "front" + model: "Virtual Video Device" +"Virtual1": + supported_formats: + - width: 800 + height: 600 + frame_rates: + - 30 + - 60 + test_pattern: "bars" + location: "back" + model: "Virtual Video Device1" +"Virtual2": + supported_formats: + - width: 400 + height: 300 + test_pattern: "lines" + location: "front" + model: "Virtual Video Device2" +"Virtual3": + test_pattern: "bars" diff --git a/src/libcamera/pipeline/virtual/frame_generator.h b/src/libcamera/pipeline/virtual/frame_generator.h index d8727b8f3..f68a795fb 100644 --- a/src/libcamera/pipeline/virtual/frame_generator.h +++ b/src/libcamera/pipeline/virtual/frame_generator.h @@ -19,8 +19,7 @@ public: virtual void configure(const Size &size) = 0; - virtual void generateFrame(const Size &size, - const FrameBuffer *buffer) = 0; + virtual void generateFrame(const Size &size, const FrameBuffer *buffer) = 0; protected: FrameGenerator() {} diff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build index 0c537777f..d72ac5be7 100644 --- a/src/libcamera/pipeline/virtual/meson.build +++ b/src/libcamera/pipeline/virtual/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'parser.cpp', 'test_pattern_generator.cpp', 'virtual.cpp', ]) diff --git a/src/libcamera/pipeline/virtual/parser.cpp b/src/libcamera/pipeline/virtual/parser.cpp new file mode 100644 index 000000000..e579a3f7c --- /dev/null +++ b/src/libcamera/pipeline/virtual/parser.cpp @@ -0,0 +1,191 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * parser.cpp - Virtual cameras helper to parse config file + */ + +#include "parser.h" + +#include +#include + +#include + +#include +#include + +#include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/yaml_parser.h" + +#include "virtual.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Virtual) + +std::vector> +Parser::parseConfigFile(File &file, PipelineHandler *pipe) +{ + std::vector> configurations; + + std::unique_ptr cameras = YamlParser::parse(file); + if (!cameras) { + LOG(Virtual, Error) << "Failed to pass config file."; + return configurations; + } + + if (!cameras->isDictionary()) { + LOG(Virtual, Error) << "Config file is not a dictionary at the top level."; + return configurations; + } + + /* Look into the configuration of each camera */ + for (const auto &[cameraId, cameraConfigData] : cameras->asDict()) { + std::unique_ptr data = + parseCameraConfigData(cameraConfigData, pipe); + /* Parse configData to data*/ + if (!data) { + /* Skip the camera if it has invalid config */ + LOG(Virtual, Error) << "Failed to parse config of the camera: " + << cameraId; + continue; + } + + data->id_ = cameraId; + ControlInfoMap::Map controls; + /* todo: Check which resolution's frame rate to be reported */ + controls[&controls::FrameDurationLimits] = + ControlInfo(int64_t(1000000 / data->supportedResolutions_[0].frameRates[1]), + int64_t(1000000 / data->supportedResolutions_[0].frameRates[0])); + data->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); + configurations.push_back(std::move(data)); + } + return configurations; +} + +std::unique_ptr +Parser::parseCameraConfigData(const YamlObject &cameraConfigData, + PipelineHandler *pipe) +{ + std::vector resolutions; + if (parseSupportedFormats(cameraConfigData, &resolutions)) + return nullptr; + + std::unique_ptr data = + std::make_unique(pipe, resolutions); + + if (parseTestPattern(cameraConfigData, data.get())) + return nullptr; + + if (parseLocation(cameraConfigData, data.get())) + return nullptr; + + if (parseModel(cameraConfigData, data.get())) + return nullptr; + + return data; +} + +int Parser::parseSupportedFormats(const YamlObject &cameraConfigData, + std::vector *resolutions) +{ + if (cameraConfigData.contains("supported_formats")) { + const YamlObject &supportedResolutions = cameraConfigData["supported_formats"]; + + for (const YamlObject &supportedResolution : supportedResolutions.asList()) { + unsigned int width = supportedResolution["width"].get(1920); + unsigned int height = supportedResolution["height"].get(1080); + if (width == 0 || height == 0) { + LOG(Virtual, Error) << "Invalid width or/and height"; + return -EINVAL; + } + if (width % 2 != 0) { + LOG(Virtual, Error) << "Invalid width: width needs to be even"; + return -EINVAL; + } + + std::vector frameRates; + if (supportedResolution.contains("frame_rates")) { + auto frameRatesList = + supportedResolution["frame_rates"].getList(); + if (!frameRatesList || frameRatesList->size() != 2) { + LOG(Virtual, Error) << "frame_rates needs to be the two edge values of a range"; + return -EINVAL; + } + if (frameRatesList.value()[0] > frameRatesList.value()[1]) { + LOG(Virtual, Error) << "frame_rates's first value(lower bound)" + << " is higher than the second value(upper bound)"; + return -EINVAL; + } + frameRates.push_back(frameRatesList.value()[0]); + frameRates.push_back(frameRatesList.value()[1]); + } else { + frameRates.push_back(30); + frameRates.push_back(60); + } + + resolutions->emplace_back( + VirtualCameraData::Resolution{ Size{ width, height }, + frameRates }); + } + } else { + resolutions->emplace_back( + VirtualCameraData::Resolution{ Size{ 1920, 1080 }, + { 30, 60 } }); + } + + return 0; +} + +int Parser::parseTestPattern(const YamlObject &cameraConfigData, VirtualCameraData *data) +{ + std::string testPattern = cameraConfigData["test_pattern"].get(""); + + /* Default value is "bars" */ + if (testPattern == "bars") { + data->testPattern_ = TestPattern::ColorBars; + } else if (testPattern == "lines") { + data->testPattern_ = TestPattern::DiagonalLines; + } else { + LOG(Virtual, Error) << "Test pattern: " << testPattern + << "is not supported"; + return -EINVAL; + } + return 0; +} + +int Parser::parseLocation(const YamlObject &cameraConfigData, VirtualCameraData *data) +{ + std::string location = cameraConfigData["location"].get(""); + + /* Default value is properties::CameraLocationFront */ + if (location == "front" || location == "") { + data->properties_.set(properties::Location, + properties::CameraLocationFront); + } else if (location == "back") { + data->properties_.set(properties::Location, + properties::CameraLocationBack); + } else { + LOG(Virtual, Error) << "location: " << location + << " is not supported"; + return -EINVAL; + } + + return 0; +} + +int Parser::parseModel(const YamlObject &cameraConfigData, VirtualCameraData *data) +{ + std::string model = cameraConfigData["model"].get(""); + + /* Default value is "Unknown" */ + if (model == "") + data->properties_.set(properties::Model, "Unknown"); + else + data->properties_.set(properties::Model, model); + + return 0; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/parser.h b/src/libcamera/pipeline/virtual/parser.h new file mode 100644 index 000000000..09c3c56b8 --- /dev/null +++ b/src/libcamera/pipeline/virtual/parser.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * parser.h - Virtual cameras helper to parse config file + */ + +#pragma once + +#include +#include + +#include + +#include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/yaml_parser.h" + +#include "virtual.h" + +namespace libcamera { + +class Parser +{ +public: + Parser() {} + ~Parser() = default; + + std::vector> + parseConfigFile(File &file, PipelineHandler *pipe); + +private: + std::unique_ptr + parseCameraConfigData(const YamlObject &cameraConfigData, PipelineHandler *pipe); + + int parseSupportedFormats(const YamlObject &cameraConfigData, + std::vector *resolutions); + int parseTestPattern(const YamlObject &cameraConfigData, VirtualCameraData *data); + int parseLocation(const YamlObject &cameraConfigData, VirtualCameraData *data); + int parseModel(const YamlObject &cameraConfigData, VirtualCameraData *data); +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index 690d72a22..c196a56aa 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -24,6 +24,9 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/formats.h" #include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/yaml_parser.h" + +#include "parser.h" namespace libcamera { @@ -55,6 +58,9 @@ VirtualCameraData::VirtualCameraData(PipelineHandler *pipe, maxResolutionSize_ = std::max(maxResolutionSize_, resolution.size); } + properties_.set(properties::PixelArrayActiveAreas, + { Rectangle(maxResolutionSize_) }); + /* \todo Support multiple streams and pass multi_stream_test */ streamConfigs_.resize(kMaxStream); } @@ -242,37 +248,33 @@ bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator created_ = true; - /* \todo Add virtual cameras according to a config file. */ - - std::vector supportedResolutions; - supportedResolutions.resize(2); - supportedResolutions[0] = { .size = Size(1920, 1080), .frameRates = { 30 } }; - supportedResolutions[1] = { .size = Size(1280, 720), .frameRates = { 30 } }; - - std::unique_ptr data = - std::make_unique(this, supportedResolutions); - - data->properties_.set(properties::Location, properties::CameraLocationFront); - data->properties_.set(properties::Model, "Virtual Video Device"); - data->properties_.set(properties::PixelArrayActiveAreas, { Rectangle(Size(1920, 1080)) }); - - /* \todo Set FrameDurationLimits based on config. */ - ControlInfoMap::Map controls; - int64_t min_frame_duration = 33333, max_frame_duration = 33333; - controls[&controls::FrameDurationLimits] = ControlInfo(min_frame_duration, max_frame_duration); - data->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); + File file(configurationFile("virtual", "virtual.yaml")); + bool isOpen = file.open(File::OpenModeFlag::ReadOnly); + if (!isOpen) { + LOG(Virtual, Error) << "Failed to open config file: " << file.fileName(); + return false; + } - /* Create and register the camera. */ - std::set streams; - for (auto &streamConfig : data->streamConfigs_) - streams.insert(&streamConfig.stream); + Parser parser; + auto configData = parser.parseConfigFile(file, this); + if (configData.size() == 0) { + LOG(Virtual, Error) << "Failed to parse any cameras from the config file: " + << file.fileName(); + return false; + } - const std::string id = "Virtual0"; - std::shared_ptr camera = Camera::create(std::move(data), id, streams); + /* Configure and register cameras with configData */ + for (auto &data : configData) { + std::set streams; + for (auto &streamConfig : data->streamConfigs_) + streams.insert(&streamConfig.stream); + std::string id = data->id_; + std::shared_ptr camera = Camera::create(std::move(data), id, streams); - initFrameGenerator(camera.get()); + initFrameGenerator(camera.get()); - registerCamera(std::move(camera)); + registerCamera(std::move(camera)); + } return true; } diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h index 79c951a06..5daa960b1 100644 --- a/src/libcamera/pipeline/virtual/virtual.h +++ b/src/libcamera/pipeline/virtual/virtual.h @@ -39,6 +39,7 @@ public: ~VirtualCameraData() = default; + std::string id_; TestPattern testPattern_; const std::vector supportedResolutions_; From patchwork Thu Aug 29 19:47:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 21054 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A2E91C32C2 for ; Thu, 29 Aug 2024 19:57:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BB8C5634BA; Thu, 29 Aug 2024 21:57:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="bpE+Q0sK"; dkim-atps=neutral Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 32CED6345A for ; Thu, 29 Aug 2024 21:57:12 +0200 (CEST) Received: by mail-wm1-x32b.google.com with SMTP id 5b1f17b1804b1-428e1915e18so8908655e9.1 for ; Thu, 29 Aug 2024 12:57:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1724961431; x=1725566231; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IwwVi+OxRq9k7DPUlRzVijKxuzbzPMxBlot3oQWGpN8=; b=bpE+Q0sKzs4ucqFAQGcX4bFRC/43PnBIIgcuEM+exd8dVSqKhJuGL/jKSIpIgTeopc BfwURL/bSnkQuru88mGaIYblXWSUkqokO84e5hVHJLD8hz1U677MnlDvSgCNO+sfm5hP yWRAhMGplVvqMgCEamF3QtiKkMoSpIU+lsNGg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724961431; x=1725566231; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IwwVi+OxRq9k7DPUlRzVijKxuzbzPMxBlot3oQWGpN8=; b=iJnTlQTIzkEiWwDDtmiOeXo1KYiBe2W6EKrkljxYGg3Zc3xpoRlgTNl9N5qn1A1Mai teqAxaVQCPxnXI9l302JF0geac2F/FDQyHwyafxmGWYFkgCttGfXtVplcEKYb1us0oL5 jjy5bMJJpF4ikAeyIySko+idOvNeD7eXSGn998eVF89g2wtexkh1oCIDdQfubRMI3U/J ejpFMQe6hd6VQFMT4yrI0oJhcTo+5110UpEx7XhnGlgYkCbSy7mqyPOCmxKXykQND08P E0H2KtOB83Wz/mDLrEUzxaO0Dky4pgEP7qA/8S/kj716W6OlpTjE91W66u8uaj32ghba MChw== X-Gm-Message-State: AOJu0Ywb7TJwR+e9sHv/7LasgKVbXmZ3FjTu34KlixcM0BwlNw4Ywq5/ UJSzbC/yiD+QAME2yp/m8wcGx7w3J+KA8FShWwatxwluVKhrXdMCzvw6p1g+Mp4cIZ1NzWRa5Ma 3Hw== X-Google-Smtp-Source: AGHT+IHcEh6R9AFrH+DSvxPylhkNPlHjd1y5V0Oo01Q14yM26FrmOEnRCvfLcQE9umYc1jl7mjOu5g== X-Received: by 2002:a05:600c:3c8f:b0:423:791:f446 with SMTP id 5b1f17b1804b1-42bb02c185emr30916885e9.7.1724961431315; Thu, 29 Aug 2024 12:57:11 -0700 (PDT) Received: from chenghaoyang-germany.c.googlers.com.com (161.126.77.34.bc.googleusercontent.com. [34.77.126.161]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42ba7b4271fsm54602805e9.29.2024.08.29.12.57.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Aug 2024 12:57:10 -0700 (PDT) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Konami Shu , Harvey Yang , Yunke Cao , Tomasz Figa Subject: [PATCH v10 6/7] libcamera: virtual: Add ImageFrameGenerator Date: Thu, 29 Aug 2024 19:47:45 +0000 Message-ID: <20240829195703.1456614-7-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog In-Reply-To: <20240829195703.1456614-1-chenghaoyang@chromium.org> References: <20240829195703.1456614-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Konami Shu Besides TestPatternGenerator, this patch adds ImageFrameGenerator that loads real images (jpg / jpeg for now) as the source and generates scaled frames. Signed-off-by: Konami Shu Co-developed-by: Harvey Yang Co-developed-by: Yunke Cao Co-developed-by: Tomasz Figa --- src/libcamera/pipeline/virtual/README.md | 9 +- .../virtual/image_frame_generator.cpp | 179 ++++++++++++++++++ .../pipeline/virtual/image_frame_generator.h | 54 ++++++ src/libcamera/pipeline/virtual/meson.build | 4 + src/libcamera/pipeline/virtual/parser.cpp | 76 +++++++- src/libcamera/pipeline/virtual/parser.h | 2 + src/libcamera/pipeline/virtual/utils.h | 17 ++ src/libcamera/pipeline/virtual/virtual.cpp | 60 ++++-- src/libcamera/pipeline/virtual/virtual.h | 24 ++- 9 files changed, 390 insertions(+), 35 deletions(-) create mode 100644 src/libcamera/pipeline/virtual/image_frame_generator.cpp create mode 100644 src/libcamera/pipeline/virtual/image_frame_generator.h create mode 100644 src/libcamera/pipeline/virtual/utils.h diff --git a/src/libcamera/pipeline/virtual/README.md b/src/libcamera/pipeline/virtual/README.md index ff1e8a5f9..847b8eb87 100644 --- a/src/libcamera/pipeline/virtual/README.md +++ b/src/libcamera/pipeline/virtual/README.md @@ -16,7 +16,13 @@ Each camera block is a dictionary, containing the following keys: - `width` (`unsigned int`, default=1920): Width of the window resolution. This needs to be even. - `height` (`unsigned int`, default=1080): Height of the window resolution. - `frame_rates` (list of `int`, default=`[30,60]` ): Range of the frame rate (per second). The list has to be two values of the lower bound and the upper bound of the frame rate. -- `test_pattern` (`string`): Which test pattern to use as frames. The options are "bars", "lines". +- `test_pattern` (`string`): Which test pattern to use as frames. The options are "bars", "lines". Cannot be set with `frames`. +- `frames` (dictionary): + - `path` (`string`): Path to an image, or path to a directory of a series of images. Cannot be set with `test_pattern`. + - The test patterns are "bars" which means color bars, and "lines" which means diagonal lines. + - The path to an image has ".jpg" extension. + - The path to a directory ends with "/". The name of the images in the directory are "{n}.jpg" with {n} is the sequence of images starting with 0. + - `scale_mode`(`string`, default="fill"): Scale mode when the frames are images. The only scale mode supported now is "fill". This does not affect the scale mode for now. - `location` (`string`, default="front"): The location of the camera. Support "front" and "back". This is displayed in qcam camera selection window but this does not change the output. - `model` (`string`, default="Unknown"): The model name of the camera. This is displayed in qcam camera selection window but this does not change the output. @@ -37,6 +43,7 @@ This is the procedure of the Parser class: 3. Parse each property and register the data. - `parseSupportedFormats()`: Parses `supported_formats` in the config, which contains resolutions and frame rates. - `parseTestPattern()`: Parses `test_pattern` in the config. + - `parseFrame()`: Parses `frames` in the config. - `parseLocation()`: Parses `location` in the config. - `parseModel()`: Parses `model` in the config. 4. Back to `parseConfigFile()` and append the camera configuration. diff --git a/src/libcamera/pipeline/virtual/image_frame_generator.cpp b/src/libcamera/pipeline/virtual/image_frame_generator.cpp new file mode 100644 index 000000000..072e55620 --- /dev/null +++ b/src/libcamera/pipeline/virtual/image_frame_generator.cpp @@ -0,0 +1,179 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * image_frame_generator.cpp - Derived class of FrameGenerator for + * generating frames from images + */ + +#include "image_frame_generator.h" + +#include +#include +#include + +#include +#include + +#include + +#include "libcamera/internal/mapped_framebuffer.h" + +#include "libyuv/convert.h" +#include "libyuv/scale.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Virtual) + +/* + * Factory function to create an ImageFrameGenerator object. + * Read the images and convert them to buffers in NV12 format. + * Store the pointers to the buffers to a list (imageFrameDatas) + */ +std::unique_ptr +ImageFrameGenerator::create(ImageFrames &imageFrames) +{ + std::unique_ptr imageFrameGenerator = + std::make_unique(); + imageFrameGenerator->imageFrames_ = &imageFrames; + + /* + * For each file in the directory, load the image, + * convert it to NV12, and store the pointer. + */ + for (unsigned int i = 0; i < imageFrames.number.value_or(1); i++) { + std::filesystem::path path; + if (!imageFrames.number) { + /* If the path is to an image */ + path = imageFrames.path; + } else { + /* If the path is to a directory */ + path = imageFrames.path / (std::to_string(i) + ".jpg"); + } + + File file(path); + if (!file.open(File::OpenModeFlag::ReadOnly)) { + LOG(Virtual, Error) << "Failed to open image file " << file.fileName() + << ": " << strerror(file.error()); + return nullptr; + } + + /* Read the image file to data */ + auto fileSize = file.size(); + auto buffer = std::make_unique(file.size()); + if (file.read({ buffer.get(), static_cast(fileSize) }) != fileSize) { + LOG(Virtual, Error) << "Failed to read file " << file.fileName() + << ": " << strerror(file.error()); + return nullptr; + } + + /* Get the width and height of the image */ + int width, height; + if (libyuv::MJPGSize(buffer.get(), fileSize, &width, &height)) { + LOG(Virtual, Error) << "Failed to get the size of the image file: " + << file.fileName(); + return nullptr; + } + + /* Convert to NV12 and write the data to tmpY and tmpUV */ + unsigned int halfWidth = (width + 1) / 2; + unsigned int halfHeight = (height + 1) / 2; + std::unique_ptr dstY = + std::make_unique(width * height); + std::unique_ptr dstUV = + std::make_unique(halfWidth * halfHeight * 2); + int ret = libyuv::MJPGToNV12(buffer.get(), fileSize, + dstY.get(), width, dstUV.get(), + width, width, height, width, height); + if (ret != 0) + LOG(Virtual, Error) << "MJPGToNV12() failed with " << ret; + + imageFrameGenerator->imageFrameDatas_.emplace_back( + ImageFrameData{ std::move(dstY), std::move(dstUV), + Size(width, height) }); + } + + return imageFrameGenerator; +} + +/* Scale the buffers for image frames. */ +void ImageFrameGenerator::configure(const Size &size) +{ + /* Reset the source images to prevent multiple configuration calls */ + scaledFrameDatas_.clear(); + frameCount_ = 0; + parameter_ = 0; + + for (unsigned int i = 0; i < imageFrames_->number.value_or(1); i++) { + /* Scale the imageFrameDatas_ to scaledY and scaledUV */ + unsigned int halfSizeWidth = (size.width + 1) / 2; + unsigned int halfSizeHeight = (size.height + 1) / 2; + std::unique_ptr scaledY = + std::make_unique(size.width * size.height); + std::unique_ptr scaledUV = + std::make_unique(halfSizeWidth * halfSizeHeight * 2); + auto &src = imageFrameDatas_[i]; + + /* + * \todo Some platforms might enforce stride due to GPU, like + * ChromeOS ciri (64). The weight needs to be a multiple of + * the stride to work properly for now. + */ + libyuv::NV12Scale(src.Y.get(), src.size.width, + src.UV.get(), src.size.width, + src.size.width, src.size.height, + scaledY.get(), size.width, scaledUV.get(), size.width, + size.width, size.height, libyuv::FilterMode::kFilterBilinear); + + scaledFrameDatas_.emplace_back( + ImageFrameData{ std::move(scaledY), std::move(scaledUV), size }); + } +} + +void ImageFrameGenerator::generateFrame(const Size &size, const FrameBuffer *buffer) +{ + /* Don't do anything when the list of buffers is empty*/ + ASSERT(!scaledFrameDatas_.empty()); + + MappedFrameBuffer mappedFrameBuffer(buffer, MappedFrameBuffer::MapFlag::Write); + + auto planes = mappedFrameBuffer.planes(); + + /* Make sure the frameCount does not over the number of images */ + frameCount_ %= imageFrames_->number.value_or(1); + + /* Write the scaledY and scaledUV to the mapped frame buffer */ + libyuv::NV12Copy(scaledFrameDatas_[frameCount_].Y.get(), size.width, + scaledFrameDatas_[frameCount_].UV.get(), size.width, planes[0].begin(), + size.width, planes[1].begin(), size.width, + size.width, size.height); + + /* proceed an image every 4 frames */ + /* \todo read the parameter_ from the configuration file? */ + parameter_++; + if (parameter_ % 4 == 0) + frameCount_++; +} + +/** + * \var ImageFrameGenerator::imageFrameDatas_ + * \brief List of pointers to the not scaled image buffers + */ + +/** + * \var ImageFrameGenerator::scaledFrameDatas_ + * \brief List of pointers to the scaled image buffers + */ + +/** + * \var ImageFrameGenerator::imageFrames_ + * \brief Pointer to the imageFrames_ in VirtualCameraData + */ + +/** + * \var ImageFrameGenerator::parameter_ + * \brief Speed parameter. Change to the next image every parameter_ frames + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/image_frame_generator.h b/src/libcamera/pipeline/virtual/image_frame_generator.h new file mode 100644 index 000000000..4ad8aad24 --- /dev/null +++ b/src/libcamera/pipeline/virtual/image_frame_generator.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * image_frame_generator.h - Derived class of FrameGenerator for + * generating frames from images + */ + +#pragma once + +#include +#include +#include +#include +#include + +#include "frame_generator.h" + +namespace libcamera { + +enum class ScaleMode : char { + Fill = 0, +}; + +/* Frame configuration provided by the config file */ +struct ImageFrames { + std::filesystem::path path; + ScaleMode scaleMode; + std::optional number; +}; + +class ImageFrameGenerator : public FrameGenerator +{ +public: + static std::unique_ptr create(ImageFrames &imageFrames); + +private: + struct ImageFrameData { + std::unique_ptr Y; + std::unique_ptr UV; + Size size; + }; + + void configure(const Size &size) override; + void generateFrame(const Size &size, const FrameBuffer *buffer) override; + + std::vector imageFrameDatas_; + std::vector scaledFrameDatas_; + ImageFrames *imageFrames_; + unsigned int frameCount_; + unsigned int parameter_; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build index d72ac5be7..395919b39 100644 --- a/src/libcamera/pipeline/virtual/meson.build +++ b/src/libcamera/pipeline/virtual/meson.build @@ -1,9 +1,13 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'image_frame_generator.cpp', 'parser.cpp', 'test_pattern_generator.cpp', 'virtual.cpp', ]) +libjpeg = dependency('libjpeg', required : false) + libcamera_deps += [libyuv_dep] +libcamera_deps += [libjpeg] diff --git a/src/libcamera/pipeline/virtual/parser.cpp b/src/libcamera/pipeline/virtual/parser.cpp index e579a3f7c..f0797fc7e 100644 --- a/src/libcamera/pipeline/virtual/parser.cpp +++ b/src/libcamera/pipeline/virtual/parser.cpp @@ -52,12 +52,12 @@ Parser::parseConfigFile(File &file, PipelineHandler *pipe) continue; } - data->id_ = cameraId; + data->config_.id = cameraId; ControlInfoMap::Map controls; /* todo: Check which resolution's frame rate to be reported */ controls[&controls::FrameDurationLimits] = - ControlInfo(int64_t(1000000 / data->supportedResolutions_[0].frameRates[1]), - int64_t(1000000 / data->supportedResolutions_[0].frameRates[0])); + ControlInfo(int64_t(1000000 / data->config_.resolutions[0].frameRates[1]), + int64_t(1000000 / data->config_.resolutions[0].frameRates[0])); data->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); configurations.push_back(std::move(data)); } @@ -75,7 +75,8 @@ Parser::parseCameraConfigData(const YamlObject &cameraConfigData, std::unique_ptr data = std::make_unique(pipe, resolutions); - if (parseTestPattern(cameraConfigData, data.get())) + if (parseTestPattern(cameraConfigData, data.get()) && + parseFrame(cameraConfigData, data.get())) return nullptr; if (parseLocation(cameraConfigData, data.get())) @@ -142,16 +143,75 @@ int Parser::parseTestPattern(const YamlObject &cameraConfigData, VirtualCameraDa { std::string testPattern = cameraConfigData["test_pattern"].get(""); - /* Default value is "bars" */ if (testPattern == "bars") { - data->testPattern_ = TestPattern::ColorBars; + data->config_.frame = TestPattern::ColorBars; } else if (testPattern == "lines") { - data->testPattern_ = TestPattern::DiagonalLines; + data->config_.frame = TestPattern::DiagonalLines; } else { - LOG(Virtual, Error) << "Test pattern: " << testPattern + LOG(Virtual, Debug) << "Test pattern: " << testPattern << "is not supported"; return -EINVAL; } + + return 0; +} + +int Parser::parseFrame(const YamlObject &cameraConfigData, VirtualCameraData *data) +{ + const YamlObject &frames = cameraConfigData["frames"]; + + /* When there is no frames provided in the config file, use color bar test pattern */ + if (frames.size() == 0) { + data->config_.frame = TestPattern::ColorBars; + return 0; + } + + if (!frames.isDictionary()) { + LOG(Virtual, Error) << "'frames' is not a dictionary."; + return -EINVAL; + } + + std::string path = frames["path"].get(""); + + ScaleMode scaleMode; + if (auto ext = std::filesystem::path(path).extension(); + ext == ".jpg" || ext == ".jpeg") { + if (parseScaleMode(frames, &scaleMode)) + return -EINVAL; + data->config_.frame = ImageFrames{ path, scaleMode, std::nullopt }; + } else if (std::filesystem::is_directory(std::filesystem::symlink_status(path))) { + if (parseScaleMode(frames, &scaleMode)) + return -EINVAL; + + using std::filesystem::directory_iterator; + unsigned int numOfFiles = std::distance(directory_iterator(path), directory_iterator{}); + if (numOfFiles == 0) { + LOG(Virtual, Error) << "Empty directory"; + return -EINVAL; + } + data->config_.frame = ImageFrames{ path, scaleMode, numOfFiles }; + } else { + LOG(Virtual, Error) << "Frame: " << path << " is not supported"; + return -EINVAL; + } + + return 0; +} + +int Parser::parseScaleMode( + const YamlObject &framesConfigData, ScaleMode *scaleMode) +{ + std::string mode = framesConfigData["scale_mode"].get(""); + + /* Default value is fill */ + if (mode == "fill" || mode == "") { + *scaleMode = ScaleMode::Fill; + } else { + LOG(Virtual, Error) << "scaleMode: " << mode + << " is not supported"; + return -EINVAL; + } + return 0; } diff --git a/src/libcamera/pipeline/virtual/parser.h b/src/libcamera/pipeline/virtual/parser.h index 09c3c56b8..f65616e33 100644 --- a/src/libcamera/pipeline/virtual/parser.h +++ b/src/libcamera/pipeline/virtual/parser.h @@ -35,8 +35,10 @@ private: int parseSupportedFormats(const YamlObject &cameraConfigData, std::vector *resolutions); int parseTestPattern(const YamlObject &cameraConfigData, VirtualCameraData *data); + int parseFrame(const YamlObject &cameraConfigData, VirtualCameraData *data); int parseLocation(const YamlObject &cameraConfigData, VirtualCameraData *data); int parseModel(const YamlObject &cameraConfigData, VirtualCameraData *data); + int parseScaleMode(const YamlObject &framesConfigData, ScaleMode *scaleMode); }; } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/utils.h b/src/libcamera/pipeline/virtual/utils.h new file mode 100644 index 000000000..43a14d4b5 --- /dev/null +++ b/src/libcamera/pipeline/virtual/utils.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * utils.h - Utility types for Virtual Pipeline Handler + */ + +namespace libcamera { + +template +struct overloaded : Ts... { + using Ts::operator()...; +}; +template +overloaded(Ts...) -> overloaded; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index c196a56aa..e79e6c095 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -27,6 +27,7 @@ #include "libcamera/internal/yaml_parser.h" #include "parser.h" +#include "utils.h" namespace libcamera { @@ -49,17 +50,18 @@ uint64_t currentTimestamp() VirtualCameraData::VirtualCameraData(PipelineHandler *pipe, std::vector supportedResolutions) - : Camera::Private(pipe), supportedResolutions_(std::move(supportedResolutions)) + : Camera::Private(pipe) { - for (const auto &resolution : supportedResolutions_) { - if (minResolutionSize_.isNull() || minResolutionSize_ > resolution.size) - minResolutionSize_ = resolution.size; + config_.resolutions = std::move(supportedResolutions); + for (const auto &resolution : config_.resolutions) { + if (config_.minResolutionSize.isNull() || config_.minResolutionSize > resolution.size) + config_.minResolutionSize = resolution.size; - maxResolutionSize_ = std::max(maxResolutionSize_, resolution.size); + config_.maxResolutionSize = std::max(config_.maxResolutionSize, resolution.size); } properties_.set(properties::PixelArrayActiveAreas, - { Rectangle(maxResolutionSize_) }); + { Rectangle(config_.maxResolutionSize) }); /* \todo Support multiple streams and pass multi_stream_test */ streamConfigs_.resize(kMaxStream); @@ -87,7 +89,7 @@ CameraConfiguration::Status VirtualCameraConfiguration::validate() for (StreamConfiguration &cfg : config_) { bool found = false; - for (const auto &resolution : data_->supportedResolutions_) { + for (const auto &resolution : data_->config_.resolutions) { if (resolution.size.width == cfg.size.width && resolution.size.height == cfg.size.height) { found = true; @@ -96,7 +98,7 @@ CameraConfiguration::Status VirtualCameraConfiguration::validate() } if (!found) { - cfg.size = data_->maxResolutionSize_; + cfg.size = data_->config_.maxResolutionSize; status = Adjusted; } @@ -139,11 +141,11 @@ PipelineHandlerVirtual::generateConfiguration(Camera *camera, for (const StreamRole role : roles) { std::map> streamFormats; PixelFormat pixelFormat = formats::NV12; - streamFormats[pixelFormat] = { { data->minResolutionSize_, data->maxResolutionSize_ } }; + streamFormats[pixelFormat] = { { data->config_.minResolutionSize, data->config_.maxResolutionSize } }; StreamFormats formats(streamFormats); StreamConfiguration cfg(formats); cfg.pixelFormat = pixelFormat; - cfg.size = data->maxResolutionSize_; + cfg.size = data->config_.maxResolutionSize; cfg.bufferCount = VirtualCameraConfiguration::kBufferCount; switch (role) { @@ -175,6 +177,7 @@ int PipelineHandlerVirtual::configure(Camera *camera, VirtualCameraData *data = cameraData(camera); for (size_t i = 0; i < config->size(); ++i) { config->at(i).setStream(&data->streamConfigs_[i].stream); + /* Start reading the images/generating test patterns */ data->streamConfigs_[i].frameGenerator->configure( data->streamConfigs_[i].stream.configuration().size); } @@ -268,10 +271,14 @@ bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator std::set streams; for (auto &streamConfig : data->streamConfigs_) streams.insert(&streamConfig.stream); - std::string id = data->id_; + std::string id = data->config_.id; std::shared_ptr camera = Camera::create(std::move(data), id, streams); - initFrameGenerator(camera.get()); + if (!initFrameGenerator(camera.get())) { + LOG(Virtual, Error) << "Failed to initialize frame " + << "generator for camera: " << id; + continue; + } registerCamera(std::move(camera)); } @@ -279,15 +286,30 @@ bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator return true; } -void PipelineHandlerVirtual::initFrameGenerator(Camera *camera) +bool PipelineHandlerVirtual::initFrameGenerator(Camera *camera) { auto data = cameraData(camera); - for (auto &streamConfig : data->streamConfigs_) { - if (data->testPattern_ == TestPattern::DiagonalLines) - streamConfig.frameGenerator = std::make_unique(); - else - streamConfig.frameGenerator = std::make_unique(); - } + auto &frame = data->config_.frame; + std::visit(overloaded{ + [&](TestPattern &testPattern) { + for (auto &streamConfig : data->streamConfigs_) { + if (testPattern == TestPattern::DiagonalLines) + streamConfig.frameGenerator = std::make_unique(); + else + streamConfig.frameGenerator = std::make_unique(); + } + }, + [&](ImageFrames &imageFrames) { + for (auto &streamConfig : data->streamConfigs_) + streamConfig.frameGenerator = ImageFrameGenerator::create(imageFrames); + } }, + frame); + + for (auto &streamConfig : data->streamConfigs_) + if (!streamConfig.frameGenerator) + return false; + + return true; } REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual, "virtual") diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h index 5daa960b1..efa97e889 100644 --- a/src/libcamera/pipeline/virtual/virtual.h +++ b/src/libcamera/pipeline/virtual/virtual.h @@ -8,6 +8,8 @@ #pragma once #include +#include +#include #include #include @@ -16,10 +18,14 @@ #include "libcamera/internal/dma_buf_allocator.h" #include "libcamera/internal/pipeline_handler.h" +#include "frame_generator.h" +#include "image_frame_generator.h" #include "test_pattern_generator.h" namespace libcamera { +using VirtualFrame = std::variant; + class VirtualCameraData : public Camera::Private { public: @@ -33,18 +39,22 @@ public: Stream stream; std::unique_ptr frameGenerator; }; + /* The config file is parsed to the Configuration struct */ + struct Configuration { + std::string id; + std::vector resolutions; + VirtualFrame frame; + + Size maxResolutionSize; + Size minResolutionSize; + }; VirtualCameraData(PipelineHandler *pipe, std::vector supportedResolutions); ~VirtualCameraData() = default; - std::string id_; - TestPattern testPattern_; - - const std::vector supportedResolutions_; - Size maxResolutionSize_; - Size minResolutionSize_; + Configuration config_; std::vector streamConfigs_; }; @@ -89,7 +99,7 @@ private: return static_cast(camera->_d()); } - void initFrameGenerator(Camera *camera); + bool initFrameGenerator(Camera *camera); DmaBufAllocator dmaBufAllocator_; }; From patchwork Thu Aug 29 19:47:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 21055 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 03916C323E for ; Thu, 29 Aug 2024 19:57:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6552E634DD; Thu, 29 Aug 2024 21:57:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="jVrTQgtA"; dkim-atps=neutral Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 85FB3634C8 for ; Thu, 29 Aug 2024 21:57:12 +0200 (CEST) Received: by mail-wm1-x32b.google.com with SMTP id 5b1f17b1804b1-428178fc07eso8860405e9.3 for ; Thu, 29 Aug 2024 12:57:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1724961432; x=1725566232; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OlGML7j698JA+GNnOHiuVJg0H9KbzljNqElhQR8sHZI=; b=jVrTQgtAHthQ1/P7MPx/CqSR3ulzKaJe6xC/y4CWIzjb3lkU8yKuEslOUlUz+Q64NR NK5CVsIWEpdyRpYuwZ1ahQ3ixisqIXfUV4lr4Fqa+uCTM+vuzjoajjsoIawcnpF6HjNy g03et6v5lgYhawqr9wPk5zln/sRmR8fQoYGUg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724961432; x=1725566232; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OlGML7j698JA+GNnOHiuVJg0H9KbzljNqElhQR8sHZI=; b=OH7WGWOlL664schGz4HzPsf3Da2dxVh4EHBrEGi9M4IjdqWT2rXkV2NixxQDYL4aLS 9nXpnazq/nc7P7nybYfMp+lpd4nqxKlfFEQ9wxNnKaMYoTprhYABUxCBevKPWG/DLWrv 8ilQ7BKAtmK5JUe6Dg6hOkA0WhfgoQMgI5qEJ7Z37omyS09P1qiutUoAF25KRM7sKgBu z3BWI3MNfHyoge9SaNAKWgqyMndCaooHcPNNjuiVOoI5huU6RbLMXh79bf/RxzBnaKoh U/wKrCP3P526Zy47Mjpy6nGv/Gd4jwgKkEZHHbO9FX0fiu2HxgE2cQU3L9q/GtkCZe8C B15A== X-Gm-Message-State: AOJu0Yzw6vMzy8b7Fpv5Ty2Dfmb6ZrbBUClafgPq15pazTB+4iOFl5bd VGlznM4MBHvJ6FUmrE2eoIfhPdpCxQ05iyvBiyCflkgzxdYH5qrcaog1q8tDckatvyGjoe4TFwg Pww== X-Google-Smtp-Source: AGHT+IHD/0QRG6hwOUzthlgTth5XOn29XO9d+FgYVvmVxjVsWgFolx5rfLxGzNdAJk+iiPEaDqYHpw== X-Received: by 2002:a7b:c449:0:b0:42b:aecb:d280 with SMTP id 5b1f17b1804b1-42bb27a1021mr33170805e9.27.1724961431808; Thu, 29 Aug 2024 12:57:11 -0700 (PDT) Received: from chenghaoyang-germany.c.googlers.com.com (161.126.77.34.bc.googleusercontent.com. [34.77.126.161]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42ba7b4271fsm54602805e9.29.2024.08.29.12.57.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Aug 2024 12:57:11 -0700 (PDT) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Harvey Yang , Milan Zamazal Subject: [PATCH v10 7/7] libcamera: software_isp: Refactor SoftwareIsp to use DmaBufAllocator::exportBuffers Date: Thu, 29 Aug 2024 19:47:46 +0000 Message-ID: <20240829195703.1456614-8-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog In-Reply-To: <20240829195703.1456614-1-chenghaoyang@chromium.org> References: <20240829195703.1456614-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang As the helper function DmaBufAllocator::exportBuffers is added, we can avoid some code duplication in SoftwareIsp as well. Signed-off-by: Harvey Yang Reviewed-by: Milan Zamazal --- src/libcamera/software_isp/software_isp.cpp | 20 +------------------- 1 file changed, 1 insertion(+), 19 deletions(-) diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 1140372ce..3455f13aa 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -255,25 +255,7 @@ int SoftwareIsp::exportBuffers(const Stream *stream, unsigned int count, if (stream == nullptr) return -EINVAL; - for (unsigned int i = 0; i < count; i++) { - const std::string name = "frame-" + std::to_string(i); - const size_t frameSize = debayer_->frameSize(); - - FrameBuffer::Plane outPlane; - outPlane.fd = SharedFD(dmaHeap_.alloc(name.c_str(), frameSize)); - if (!outPlane.fd.isValid()) { - LOG(SoftwareIsp, Error) - << "failed to allocate a dma_buf"; - return -ENOMEM; - } - outPlane.offset = 0; - outPlane.length = frameSize; - - std::vector planes{ outPlane }; - buffers->emplace_back(std::make_unique(std::move(planes))); - } - - return count; + return dmaHeap_.exportBuffers(count, { debayer_->frameSize() }, buffers); } /**