From patchwork Fri Oct 4 09:59:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 21503 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 86C40C3257 for ; Fri, 4 Oct 2024 09:59:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4255D63531; Fri, 4 Oct 2024 11:59:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="RvXS40aD"; dkim-atps=neutral Received: from mail-ot1-x333.google.com (mail-ot1-x333.google.com [IPv6:2607:f8b0:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3D6466352B for ; Fri, 4 Oct 2024 11:59:54 +0200 (CEST) Received: by mail-ot1-x333.google.com with SMTP id 46e09a7af769-7124395ca86so859025a34.0 for ; Fri, 04 Oct 2024 02:59:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1728035993; x=1728640793; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=llkXwXEbrhhVjPAPlCwKt6WULyQC7Ghp18EkJKUOl4E=; b=RvXS40aDpcMWq9tzdVsBn45WjJjoz3DDXIVKgEbkaJP2VUrFWDsZAEhgmnZBiSdx/t AUHEs39ZqPmb1jmWwRgp7Vb5VisPkJEHUJS+n6EcoXhCmaaKgu4Ibm7ZdyJcSJiTQ2Xc Tm9Tw50BcOkIcboD9nz7Q0XbQd6BBnnKivDhs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728035993; x=1728640793; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=llkXwXEbrhhVjPAPlCwKt6WULyQC7Ghp18EkJKUOl4E=; b=kSskorlwl6lNe9k4d4BIHNFlhWGBUA6owmL74scFBgRpl7JFU3jp4INR1cQjITcCzk Z8aLPCkbVDFCRjprKmMjlZjZxfZbShhAz+VTaNtLdeHQVExlVlnRKdM8hASKxrmjVAin b7e9vW0Wi30618XruErM+j8QLZ00YsG3gyKJ7F2Qhps5R8aT2TKDP8SGMt6CgMIGd2ca ogixZzR9Rb1XLTZ0spiKwYFKBMaFSaZox4VZsRZf5pMFOLLcsd7yxrjPCX+OaIJmx9ON 4wJtJGEw/MMl26ipd9eImc30pKl8fcYdXCuQ1PMVcczWbO1JBXpiefNs9RDGXmxOQB1k VZwA== X-Gm-Message-State: AOJu0YzfHXFxdysYm5gG1gcDQmL4p5yREdQmiZNtDUPme8AHWQVOAiUu MLx5TSOGf7L7wWAYXAGE1Bz8Rz6kDOP/MrCe9yrN3ibr3Ljeqc5oWbs40TBuUWFS5ojoxScOHYM = X-Google-Smtp-Source: AGHT+IFkbKL6kbVX3lRTibASKtGE5BLBYAa6qvB3WwAPvANnE9m7QikXA51G1K/+kg1SQ0awvIhXtA== X-Received: by 2002:a05:6830:4983:b0:713:d15b:6cb7 with SMTP id 46e09a7af769-7154e81a831mr2286204a34.4.1728035992690; Fri, 04 Oct 2024 02:59:52 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (27.247.221.35.bc.googleusercontent.com. [35.221.247.27]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ded2ad931sm316911b3a.41.2024.10.04.02.59.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Oct 2024 02:59:52 -0700 (PDT) From: Harvey Yang X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Harvey Yang , Kieran Bingham , Jacopo Mondi Subject: [PATCH 15 1/7] libcamera: add DmaBufAllocator::exportBuffers() Date: Fri, 4 Oct 2024 09:59:14 +0000 Message-ID: <20241004095946.264537-2-chenghaoyang@google.com> X-Mailer: git-send-email 2.47.0.rc0.187.ge670bccf7e-goog In-Reply-To: <20241004095946.264537-1-chenghaoyang@google.com> References: <20241004095946.264537-1-chenghaoyang@google.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a helper function exportBuffers in DmaBufAllocator to make it easier to use. It'll be used in Virtual Pipeline Handler and SoftwareIsp. Signed-off-by: Harvey Yang Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- .../libcamera/internal/dma_buf_allocator.h | 13 +++++ src/libcamera/dma_buf_allocator.cpp | 57 +++++++++++++++++++ 2 files changed, 70 insertions(+) diff --git a/include/libcamera/internal/dma_buf_allocator.h b/include/libcamera/internal/dma_buf_allocator.h index d2a0a0d19..9211d4837 100644 --- a/include/libcamera/internal/dma_buf_allocator.h +++ b/include/libcamera/internal/dma_buf_allocator.h @@ -7,11 +7,17 @@ #pragma once +#include +#include +#include + #include #include namespace libcamera { +class FrameBuffer; + class DmaBufAllocator { public: @@ -28,7 +34,14 @@ public: bool isValid() const { return providerHandle_.isValid(); } UniqueFD alloc(const char *name, std::size_t size); + int exportBuffers(unsigned int count, + const std::vector &planeSizes, + std::vector> *buffers); + private: + std::unique_ptr createBuffer( + std::string name, const std::vector &planeSizes); + UniqueFD allocFromHeap(const char *name, std::size_t size); UniqueFD allocFromUDmaBuf(const char *name, std::size_t size); UniqueFD providerHandle_; diff --git a/src/libcamera/dma_buf_allocator.cpp b/src/libcamera/dma_buf_allocator.cpp index be6efb89f..262eb53a0 100644 --- a/src/libcamera/dma_buf_allocator.cpp +++ b/src/libcamera/dma_buf_allocator.cpp @@ -22,6 +22,9 @@ #include #include +#include + +#include /** * \file dma_buf_allocator.cpp @@ -205,4 +208,58 @@ UniqueFD DmaBufAllocator::alloc(const char *name, std::size_t size) return allocFromHeap(name, size); } +/** + * \brief Allocate and export buffers from the DmaBufAllocator + * \param[in] count The number of requested FrameBuffers + * \param[in] planeSizes The sizes of planes in each FrameBuffer + * \param[out] buffers Array of buffers successfully allocated + * + * Planes in a FrameBuffer are allocated with a single dma buf. + * \todo Add the option to allocate each plane with a dma buf respectively. + * + * \return The number of allocated buffers on success or a negative error code + * otherwise + */ +int DmaBufAllocator::exportBuffers(unsigned int count, + const std::vector &planeSizes, + std::vector> *buffers) +{ + for (unsigned int i = 0; i < count; ++i) { + std::unique_ptr buffer = + createBuffer("frame-" + std::to_string(i), planeSizes); + if (!buffer) { + LOG(DmaBufAllocator, Error) << "Unable to create buffer"; + + buffers->clear(); + return -EINVAL; + } + + buffers->push_back(std::move(buffer)); + } + + return count; +} + +std::unique_ptr +DmaBufAllocator::createBuffer(std::string name, + const std::vector &planeSizes) +{ + std::vector planes; + + unsigned int frameSize = 0, offset = 0; + for (auto planeSize : planeSizes) + frameSize += planeSize; + + SharedFD fd(alloc(name.c_str(), frameSize)); + if (!fd.isValid()) + return nullptr; + + for (auto planeSize : planeSizes) { + planes.emplace_back(FrameBuffer::Plane{ fd, offset, planeSize }); + offset += planeSize; + } + + return std::make_unique(planes); +} + } /* namespace libcamera */ From patchwork Fri Oct 4 09:59:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 21504 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1BAFBC3257 for ; Fri, 4 Oct 2024 10:00:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2E06863532; Fri, 4 Oct 2024 12:00:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="NXXaqPqY"; dkim-atps=neutral Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8861E6352D for ; Fri, 4 Oct 2024 11:59:56 +0200 (CEST) Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-71b20ffd809so1731357b3a.0 for ; Fri, 04 Oct 2024 02:59:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1728035994; x=1728640794; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WjgHep/iOhTVamI1PFk3bNKyYMctjnfcztOa4A+1UwI=; b=NXXaqPqYJGLTGl28AkF6dh3U2z/JmssMxzP7vNNjVmLZ9d3VEr3huRi8GhQ6ilhVc8 EDWLBL7H2L/mzygimzolx+eJa+B03XJjrQx7BqkhV+DJ/I3nmxOvBO9su6MxnwNxicW6 tzxIYtOsnbfT7uVvjI/JEHkZ9xssNPJGm4kWU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728035994; x=1728640794; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WjgHep/iOhTVamI1PFk3bNKyYMctjnfcztOa4A+1UwI=; b=oQ94/e9EGlWNdj7HkFHyTpBxeTSPBmYYFvTcRVLNIsj/G2CsjKvfv1u+Qr3D7mtxkB s2+zrTeXmmgWidnwenRqeAh3ASr6TCpDIKj4tqeQloOjJCC8/qjH357FoOtkmQiBlrnc cfavTQiP10HqjTNXm0RZqELANchQ1JGIAZouSbBi4cK65mCGct0xjt1O+EO/+ZrZQTxi fJvKXDP3AuFv2q2cXxpsJHIxXCOHaCnlOD6jCkQXpPtGIHR+hb+X+0dVh9BrAOl8zuhJ kYnifCMOuOtA+bOdX5ycuItVsH+sVGGVI6AVTj7JtgDcEpXoWf/j9QS/WivJAO1d9G0k O6wg== X-Gm-Message-State: AOJu0YzQnhbmyLkZZklROvr+INV5oRvvXtTIzipjFlpUJa7o52/h9aWK aHIurGwxDpcF50IWqI78SBnv8pE6FdcUpinQ+kSE7N6KSVcChkxTsxZOUwvC//nt0/92p1OItJM sGA== X-Google-Smtp-Source: AGHT+IEtSLVkyF7BnKe+cL0clSWF6Z65+cOKTH+4VgAPQXAPZFKBNpCPOid8o73nu8ujktBz7iHULg== X-Received: by 2002:a05:6a00:886:b0:714:340c:b9ee with SMTP id d2e1a72fcca58-71de23a5576mr3081486b3a.1.1728035994292; Fri, 04 Oct 2024 02:59:54 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (27.247.221.35.bc.googleusercontent.com. [35.221.247.27]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ded2ad931sm316911b3a.41.2024.10.04.02.59.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Oct 2024 02:59:53 -0700 (PDT) From: Harvey Yang X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Harvey Yang , Laurent Pinchart , Kieran Bingham Subject: [PATCH 15 2/7] libcamera: Remove PipelineHandler Fatal check of non-empty MediaDevices Date: Fri, 4 Oct 2024 09:59:15 +0000 Message-ID: <20241004095946.264537-3-chenghaoyang@google.com> X-Mailer: git-send-email 2.47.0.rc0.187.ge670bccf7e-goog In-Reply-To: <20241004095946.264537-1-chenghaoyang@google.com> References: <20241004095946.264537-1-chenghaoyang@google.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Fatal check of having at least one MediaDevice was to prevent pipeline handler implementations searching and owning media devices with custom conventions, instead of using the base function |acquireMediaDevice|. It also has the assumption that there's at least one media device to make a camera work. Now that the assumption will be broken by the virtual pipeline handler added in the following patches, and developers should be aware of the available functions in the base class to handle media devices, the Fatal check is no longer needed. Signed-off-by: Harvey Yang Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/libcamera/pipeline_handler.cpp | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index e59404691..89a513e46 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -637,9 +637,14 @@ void PipelineHandler::registerCamera(std::shared_ptr camera) { cameras_.push_back(camera); - if (mediaDevices_.empty()) - LOG(Pipeline, Fatal) - << "Registering camera with no media devices!"; + if (mediaDevices_.empty()) { + /* + * For virtual devices with no MediaDevice, there are no system + * devices to register. + */ + manager_->_d()->addCamera(std::move(camera)); + return; + } /* * Walk the entity list and map the devnums of all capture video nodes From patchwork Fri Oct 4 09:59:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 21505 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EDE89C3257 for ; Fri, 4 Oct 2024 10:00:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6CA3D63534; Fri, 4 Oct 2024 12:00:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="JjkiV0QB"; dkim-atps=neutral Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D9EC36352D for ; Fri, 4 Oct 2024 11:59:57 +0200 (CEST) Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-71b20ffd809so1731366b3a.0 for ; Fri, 04 Oct 2024 02:59:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1728035996; x=1728640796; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tdGShL1KMVo+t9gKFgngI3lAI2ROd+yBJsBooaTBsuQ=; b=JjkiV0QBt2mYaFrd36qYkrSW392f7m5oCNruTs5ck5uJsLyLAd7LH1Dmn2C+DWERJC F/K+PVkmblYg4U/+srrKk2FcwvDKE7mBqgtrPrQyKDwsomOkB8cWAza+xtpUjJ8aQur0 vwJfun+pKU7Y0SSvWTabvzygpZmzmMFMC02WY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728035996; x=1728640796; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tdGShL1KMVo+t9gKFgngI3lAI2ROd+yBJsBooaTBsuQ=; b=vO6OluYOti/GMpY60evynC5IJocgLPcMrF92mKD7eEu/hSZ7ih+3e3yhVjw33xTkEc PlnILuY/1lwhUHVTA4YGYDYyDJHLa0xMMvnmblTK4kTgHUNZI5GUaQsbAzaWs23mLI7A 3vIxdTQ7Y3OGiLgPa3AW86cHu35Gnsl9sdyIdTdoj2vYvO8d+gIDK19c6mLTsmVNSlIh 9pW4ToNaBIIo5MQ/gx6XE5PEzZvqItszZAGhySRtKBWwJRU1LOUDB2t0Be7+NDxhZOys FkERd6PKKszfMChAu2HzKB3yMIX5Uid0L0LQon1AqJBmSsH5HocuPS+1hR4O6gnkP06t SkZg== X-Gm-Message-State: AOJu0YxUXtFhgCVSURQlWcYzoqbGi61mJgIr5HxIbiAlZRmE4HJfcuTV a9cXhOXVesvpdEUPfq7Z41rDbm3SRkxnHEupQHv1vaUnhIoeLnJKS00hRYdIbArubkoJp/JMLpe jeQ== X-Google-Smtp-Source: AGHT+IF3pMNVsHNPRLWfVaC0Dv2JpeoN8lS8n3UZlumBypGFzxDVh9knLELgfSk3iVk8HYERod/o/A== X-Received: by 2002:a05:6a00:3d03:b0:717:8d5d:94d6 with SMTP id d2e1a72fcca58-71de23c7079mr2980743b3a.12.1728035995859; Fri, 04 Oct 2024 02:59:55 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (27.247.221.35.bc.googleusercontent.com. [35.221.247.27]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ded2ad931sm316911b3a.41.2024.10.04.02.59.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Oct 2024 02:59:55 -0700 (PDT) From: Harvey Yang X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Jacopo Mondi , Kieran Bingham Subject: [PATCH 15 3/7] libcamera: virtual: Add VirtualPipelineHandler Date: Fri, 4 Oct 2024 09:59:16 +0000 Message-ID: <20241004095946.264537-4-chenghaoyang@google.com> X-Mailer: git-send-email 2.47.0.rc0.187.ge670bccf7e-goog In-Reply-To: <20241004095946.264537-1-chenghaoyang@google.com> References: <20241004095946.264537-1-chenghaoyang@google.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang Add VirtualPipelineHandler for more unit tests and verfiy libcamera infrastructure works on devices without using hardware cameras. Signed-off-by: Harvey Yang Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- meson.build | 1 + meson_options.txt | 3 +- src/libcamera/pipeline/virtual/meson.build | 5 + src/libcamera/pipeline/virtual/virtual.cpp | 337 +++++++++++++++++++++ src/libcamera/pipeline/virtual/virtual.h | 45 +++ 5 files changed, 390 insertions(+), 1 deletion(-) create mode 100644 src/libcamera/pipeline/virtual/meson.build create mode 100644 src/libcamera/pipeline/virtual/virtual.cpp create mode 100644 src/libcamera/pipeline/virtual/virtual.h diff --git a/meson.build b/meson.build index 63e45465d..5e533b0c3 100644 --- a/meson.build +++ b/meson.build @@ -214,6 +214,7 @@ pipelines_support = { 'simple': ['any'], 'uvcvideo': ['any'], 'vimc': ['test'], + 'virtual': ['test'], } if pipelines.contains('all') diff --git a/meson_options.txt b/meson_options.txt index 7aa412491..c91cd241a 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -53,7 +53,8 @@ option('pipelines', 'rpi/vc4', 'simple', 'uvcvideo', - 'vimc' + 'vimc', + 'virtual' ], description : 'Select which pipeline handlers to build. If this is set to "auto", all the pipelines applicable to the target architecture will be built. If this is set to "all", all the pipelines will be built. If both are selected then "all" will take precedence.') diff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build new file mode 100644 index 000000000..ada1b3358 --- /dev/null +++ b/src/libcamera/pipeline/virtual/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +libcamera_internal_sources += files([ + 'virtual.cpp', +]) diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp new file mode 100644 index 000000000..5cac0e364 --- /dev/null +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -0,0 +1,337 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * virtual.cpp - Pipeline handler for virtual cameras + */ + +#include "virtual.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include +#include +#include +#include +#include +#include + +#include "libcamera/internal/camera.h" +#include "libcamera/internal/dma_buf_allocator.h" +#include "libcamera/internal/formats.h" +#include "libcamera/internal/pipeline_handler.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(Virtual) + +namespace { + +uint64_t currentTimestamp() +{ + const auto now = std::chrono::steady_clock::now(); + auto nsecs = std::chrono::duration_cast( + now.time_since_epoch()); + + return nsecs.count(); +} + +} /* namespace */ + +class VirtualCameraConfiguration : public CameraConfiguration +{ +public: + static constexpr unsigned int kBufferCount = 4; + + VirtualCameraConfiguration(VirtualCameraData *data); + + Status validate() override; + +private: + const VirtualCameraData *data_; +}; + +class PipelineHandlerVirtual : public PipelineHandler +{ +public: + PipelineHandlerVirtual(CameraManager *manager); + ~PipelineHandlerVirtual(); + + std::unique_ptr generateConfiguration(Camera *camera, + Span roles) override; + int configure(Camera *camera, CameraConfiguration *config) override; + + int exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) override; + + int start(Camera *camera, const ControlList *controls) override; + void stopDevice(Camera *camera) override; + + int queueRequestDevice(Camera *camera, Request *request) override; + + bool match(DeviceEnumerator *enumerator) override; + +private: + static bool created_; + + VirtualCameraData *cameraData(Camera *camera) + { + return static_cast(camera->_d()); + } + + DmaBufAllocator dmaBufAllocator_; + + bool resetCreated_ = false; +}; + +VirtualCameraData::VirtualCameraData(PipelineHandler *pipe, + const std::vector &supportedResolutions) + : Camera::Private(pipe), supportedResolutions_(supportedResolutions) +{ + for (const auto &resolution : supportedResolutions_) { + if (minResolutionSize_.isNull() || minResolutionSize_ > resolution.size) + minResolutionSize_ = resolution.size; + + maxResolutionSize_ = std::max(maxResolutionSize_, resolution.size); + } + + /* \todo Support multiple streams and pass multi_stream_test */ + streamConfigs_.resize(kMaxStream); +} + +VirtualCameraConfiguration::VirtualCameraConfiguration(VirtualCameraData *data) + : CameraConfiguration(), data_(data) +{ +} + +CameraConfiguration::Status VirtualCameraConfiguration::validate() +{ + Status status = Valid; + + if (config_.empty()) { + LOG(Virtual, Error) << "Empty config"; + return Invalid; + } + + /* Only one stream is supported */ + if (config_.size() > VirtualCameraData::kMaxStream) { + config_.resize(VirtualCameraData::kMaxStream); + status = Adjusted; + } + + for (StreamConfiguration &cfg : config_) { + bool adjusted = false; + bool found = false; + for (const auto &resolution : data_->supportedResolutions_) { + if (resolution.size.width == cfg.size.width && + resolution.size.height == cfg.size.height) { + found = true; + break; + } + } + + if (!found) { + /* + * \todo It's a pipeline's decision to choose a + * resolution when the exact one is not supported. + * Defining the default logic in PipelineHandler to + * find the closest resolution would be nice. + */ + cfg.size = data_->maxResolutionSize_; + status = Adjusted; + adjusted = true; + } + + if (cfg.pixelFormat != formats::NV12) { + cfg.pixelFormat = formats::NV12; + status = Adjusted; + adjusted = true; + } + + if (adjusted) + LOG(Virtual, Info) + << "Stream configuration adjusted to " << cfg.toString(); + + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + cfg.stride = info.stride(cfg.size.width, 0, 1); + cfg.frameSize = info.frameSize(cfg.size, 1); + + cfg.bufferCount = VirtualCameraConfiguration::kBufferCount; + } + + return status; +} + +/* static */ +bool PipelineHandlerVirtual::created_ = false; + +PipelineHandlerVirtual::PipelineHandlerVirtual(CameraManager *manager) + : PipelineHandler(manager), + dmaBufAllocator_(DmaBufAllocator::DmaBufAllocatorFlag::CmaHeap | + DmaBufAllocator::DmaBufAllocatorFlag::SystemHeap | + DmaBufAllocator::DmaBufAllocatorFlag::UDmaBuf) +{ +} + +PipelineHandlerVirtual::~PipelineHandlerVirtual() +{ + if (resetCreated_) + created_ = false; +} + +std::unique_ptr +PipelineHandlerVirtual::generateConfiguration(Camera *camera, + Span roles) +{ + VirtualCameraData *data = cameraData(camera); + auto config = std::make_unique(data); + + if (roles.empty()) + return config; + + for (const StreamRole role : roles) { + switch (role) { + case StreamRole::StillCapture: + case StreamRole::VideoRecording: + case StreamRole::Viewfinder: + break; + + case StreamRole::Raw: + default: + LOG(Virtual, Error) + << "Requested stream role not supported: " << role; + config.reset(); + return config; + } + + std::map> streamFormats; + PixelFormat pixelFormat = formats::NV12; + streamFormats[pixelFormat] = { { data->minResolutionSize_, + data->maxResolutionSize_ } }; + StreamFormats formats(streamFormats); + StreamConfiguration cfg(formats); + cfg.pixelFormat = pixelFormat; + cfg.size = data->maxResolutionSize_; + cfg.bufferCount = VirtualCameraConfiguration::kBufferCount; + + config->addConfiguration(cfg); + } + + ASSERT(config->validate() != CameraConfiguration::Invalid); + + return config; +} + +int PipelineHandlerVirtual::configure(Camera *camera, + CameraConfiguration *config) +{ + VirtualCameraData *data = cameraData(camera); + for (auto [i, c] : utils::enumerate(*config)) + c.setStream(&data->streamConfigs_[i].stream); + + return 0; +} + +int PipelineHandlerVirtual::exportFrameBuffers([[maybe_unused]] Camera *camera, + Stream *stream, + std::vector> *buffers) +{ + if (!dmaBufAllocator_.isValid()) + return -ENOBUFS; + + const StreamConfiguration &config = stream->configuration(); + + auto info = PixelFormatInfo::info(config.pixelFormat); + + std::vector planeSizes; + for (size_t i = 0; i < info.planes.size(); ++i) + planeSizes.push_back(info.planeSize(config.size, i)); + + return dmaBufAllocator_.exportBuffers(config.bufferCount, planeSizes, buffers); +} + +int PipelineHandlerVirtual::start([[maybe_unused]] Camera *camera, + [[maybe_unused]] const ControlList *controls) +{ + return 0; +} + +void PipelineHandlerVirtual::stopDevice([[maybe_unused]] Camera *camera) +{ +} + +int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera, + Request *request) +{ + for (auto it : request->buffers()) + completeBuffer(request, it.second); + + request->metadata().set(controls::SensorTimestamp, currentTimestamp()); + completeRequest(request); + + return 0; +} + +bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator) +{ + if (created_) + return false; + + created_ = true; + + /* \todo Add virtual cameras according to a config file. */ + + std::vector supportedResolutions; + supportedResolutions.resize(2); + supportedResolutions[0] = { .size = Size(1920, 1080), .frameRates = { 30 } }; + supportedResolutions[1] = { .size = Size(1280, 720), .frameRates = { 30 } }; + + std::unique_ptr data = + std::make_unique(this, supportedResolutions); + + data->properties_.set(properties::Location, properties::CameraLocationFront); + data->properties_.set(properties::Model, "Virtual Video Device"); + data->properties_.set(properties::PixelArrayActiveAreas, { Rectangle(Size(1920, 1080)) }); + + /* \todo Set FrameDurationLimits based on config. */ + ControlInfoMap::Map controls; + int64_t min_frame_duration = 33333, max_frame_duration = 33333; + controls[&controls::FrameDurationLimits] = ControlInfo(min_frame_duration, max_frame_duration); + std::vector supportedFaceDetectModes{ + static_cast(controls::draft::FaceDetectModeOff), + }; + controls[&controls::draft::FaceDetectMode] = ControlInfo(supportedFaceDetectModes); + data->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); + + /* Create and register the camera. */ + std::set streams; + for (auto &streamConfig : data->streamConfigs_) + streams.insert(&streamConfig.stream); + + const std::string id = "Virtual0"; + std::shared_ptr camera = Camera::create(std::move(data), id, streams); + registerCamera(std::move(camera)); + + resetCreated_ = true; + + return true; +} + +REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual, "virtual") + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h new file mode 100644 index 000000000..bddd3db5a --- /dev/null +++ b/src/libcamera/pipeline/virtual/virtual.h @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * virtual.h - Pipeline handler for virtual cameras + */ + +#pragma once + +#include + +#include +#include + +#include "libcamera/internal/camera.h" +#include "libcamera/internal/pipeline_handler.h" + +namespace libcamera { + +class VirtualCameraData : public Camera::Private +{ +public: + const static unsigned int kMaxStream = 3; + + struct Resolution { + Size size; + std::vector frameRates; + }; + struct StreamConfig { + Stream stream; + }; + + VirtualCameraData(PipelineHandler *pipe, + const std::vector &supportedResolutions); + + ~VirtualCameraData() = default; + + const std::vector supportedResolutions_; + Size maxResolutionSize_; + Size minResolutionSize_; + + std::vector streamConfigs_; +}; + +} /* namespace libcamera */ From patchwork Fri Oct 4 09:59:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 21506 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 29517C3257 for ; Fri, 4 Oct 2024 10:00:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F164463530; Fri, 4 Oct 2024 12:00:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="As/0YhUx"; dkim-atps=neutral Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 31F7463535 for ; Fri, 4 Oct 2024 12:00:00 +0200 (CEST) Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-71dae4fc4c9so1644691b3a.0 for ; Fri, 04 Oct 2024 03:00:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1728035998; x=1728640798; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ejd5Ijcf3gvFoIR1J3BXtiQ5eRvIdMpIrVwtHLm3RQE=; b=As/0YhUx6ozf/LKAtuzWHIvtw1Ij49sVftGNLeLD4VMQ5LqO6jSuJAOoOD07V1ViyR I4IGtVnW6uiibm0E5f0Fv4Tp233MM1r8ojxUH4w+Qj5iUBPi3uz6ZO7tglzgc9tMhrnQ XyDfxz2Dl3jovJk5HOPihe9gjYwiwpzNnLclk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728035998; x=1728640798; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ejd5Ijcf3gvFoIR1J3BXtiQ5eRvIdMpIrVwtHLm3RQE=; b=XJGV1/CxkIEx5WFKhjdcNWWeZWTeMZwuV281lORe8MCZNUXz2KVB+thBLiEpwGW0ld TM4WvhL3CUwtn406w5CE/chDB6qCtNqt31M73OekduAWJ2HuR/6+a4lN35366TqmiSG9 zhzD9oLg/TnYFEA7ZRBtr7yFhnxI9xhTjqOQ6g5eNj18yUtOz1oIWCJPD4IZ6fSXpEnZ ICi7pv9KmIfoCgT9dE0JI0CIDUjGaiNSnUAVwPPdbN8Xbbr6+XhysPmN95OhafZmeeB9 tB9JSYlYHgmqRZOaXkVWSx6ttMNnantts/cNySmxPkaZuwgVNfgmiBEU/mGPxUBGmo7x +83A== X-Gm-Message-State: AOJu0Yz4JmlvUHQzqTLIhkobUEeCgtUZ6DjLNQxDJ4Y0cUwY4XAsyelj 2FvBCzMbnIJy7zNNEhTv5WVFsdZy0yHysqUty2H1D2ripLjCaI8s5+qaWU0xUWqnZa2xZZjQGZY jng== X-Google-Smtp-Source: AGHT+IEwaDbsRXFC9fg8tMPwxdZyWOdFBGyKTC3ag4cDGwPZBeRs2PDY7eFkTuvPrB9otl7db8C71A== X-Received: by 2002:a05:6a20:d807:b0:1cf:2d62:8584 with SMTP id adf61e73a8af0-1d6dfa35ecbmr3133463637.11.1728035997951; Fri, 04 Oct 2024 02:59:57 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (27.247.221.35.bc.googleusercontent.com. [35.221.247.27]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ded2ad931sm316911b3a.41.2024.10.04.02.59.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Oct 2024 02:59:57 -0700 (PDT) From: Harvey Yang X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Konami Shu , Harvey Yang , Yunke Cao , Tomasz Figa , Jacopo Mondi , Kieran Bingham Subject: [PATCH 15 4/7] libcamera: pipeline: Add test pattern for VirtualPipelineHandler Date: Fri, 4 Oct 2024 09:59:17 +0000 Message-ID: <20241004095946.264537-5-chenghaoyang@google.com> X-Mailer: git-send-email 2.47.0.rc0.187.ge670bccf7e-goog In-Reply-To: <20241004095946.264537-1-chenghaoyang@google.com> References: <20241004095946.264537-1-chenghaoyang@google.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Konami Shu Add a test pattern generator class hierarchy for the Virtual pipeline handler. Implement two types of test patterns: color bars and diagonal lines generator and use them in the Virtual pipeline handler. A shifting mechanism is enabled. For each frame, the image is shifted to the left by 1 pixel. It drops FPS though. Add a dependency for libyuv to the build system to generate images in NV12 format from the test pattern. Signed-off-by: Konami Shu Co-developed-by: Harvey Yang Co-developed-by: Yunke Cao Co-developed-by: Tomasz Figa Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- src/android/meson.build | 19 --- .../pipeline/virtual/frame_generator.h | 29 ++++ src/libcamera/pipeline/virtual/meson.build | 3 + .../virtual/test_pattern_generator.cpp | 137 ++++++++++++++++++ .../pipeline/virtual/test_pattern_generator.h | 53 +++++++ src/libcamera/pipeline/virtual/virtual.cpp | 41 +++++- src/libcamera/pipeline/virtual/virtual.h | 5 + src/meson.build | 19 +++ 8 files changed, 284 insertions(+), 22 deletions(-) create mode 100644 src/libcamera/pipeline/virtual/frame_generator.h create mode 100644 src/libcamera/pipeline/virtual/test_pattern_generator.cpp create mode 100644 src/libcamera/pipeline/virtual/test_pattern_generator.h diff --git a/src/android/meson.build b/src/android/meson.build index 68646120a..6341ee8b0 100644 --- a/src/android/meson.build +++ b/src/android/meson.build @@ -15,25 +15,6 @@ foreach dep : android_deps endif endforeach -libyuv_dep = dependency('libyuv', required : false) - -# Fallback to a subproject if libyuv isn't found, as it's typically not -# provided by distributions. -if not libyuv_dep.found() - cmake = import('cmake') - - libyuv_vars = cmake.subproject_options() - libyuv_vars.add_cmake_defines({'CMAKE_POSITION_INDEPENDENT_CODE': 'ON'}) - libyuv_vars.set_override_option('cpp_std', 'c++17') - libyuv_vars.append_compile_args('cpp', - '-Wno-sign-compare', - '-Wno-unused-variable', - '-Wno-unused-parameter') - libyuv_vars.append_link_args('-ljpeg') - libyuv = cmake.subproject('libyuv', options : libyuv_vars) - libyuv_dep = libyuv.dependency('yuv') -endif - android_deps += [libyuv_dep] android_hal_sources = files([ diff --git a/src/libcamera/pipeline/virtual/frame_generator.h b/src/libcamera/pipeline/virtual/frame_generator.h new file mode 100644 index 000000000..4ff41dd8d --- /dev/null +++ b/src/libcamera/pipeline/virtual/frame_generator.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * frame_generator.h - Virtual cameras helper to generate frames + */ + +#pragma once + +#include +#include + +namespace libcamera { + +class FrameGenerator +{ +public: + virtual ~FrameGenerator() = default; + + virtual void configure(const Size &size) = 0; + + virtual int generateFrame(const Size &size, + const FrameBuffer *buffer) = 0; + +protected: + FrameGenerator() {} +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build index ada1b3358..0c537777f 100644 --- a/src/libcamera/pipeline/virtual/meson.build +++ b/src/libcamera/pipeline/virtual/meson.build @@ -1,5 +1,8 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'test_pattern_generator.cpp', 'virtual.cpp', ]) + +libcamera_deps += [libyuv_dep] diff --git a/src/libcamera/pipeline/virtual/test_pattern_generator.cpp b/src/libcamera/pipeline/virtual/test_pattern_generator.cpp new file mode 100644 index 000000000..3600bd5ed --- /dev/null +++ b/src/libcamera/pipeline/virtual/test_pattern_generator.cpp @@ -0,0 +1,137 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * test_pattern_generator.cpp - Derived class of FrameGenerator for + * generating test patterns + */ + +#include "test_pattern_generator.h" + +#include + +#include "libcamera/internal/mapped_framebuffer.h" + +#include + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Virtual) + +static const unsigned int kARGBSize = 4; + +int TestPatternGenerator::generateFrame(const Size &size, + const FrameBuffer *buffer) +{ + MappedFrameBuffer mappedFrameBuffer(buffer, + MappedFrameBuffer::MapFlag::Write); + + auto planes = mappedFrameBuffer.planes(); + + shiftLeft(size); + + /* Convert the template_ to the frame buffer */ + int ret = libyuv::ARGBToNV12(template_.get(), size.width * kARGBSize, + planes[0].begin(), size.width, + planes[1].begin(), size.width, + size.width, size.height); + if (ret != 0) + LOG(Virtual, Error) << "ARGBToNV12() failed with " << ret; + + return ret; +} + +void TestPatternGenerator::shiftLeft(const Size &size) +{ + /* Store the first column temporarily */ + auto firstColumn = std::make_unique(size.height * kARGBSize); + for (size_t h = 0; h < size.height; h++) { + unsigned int index = h * size.width * kARGBSize; + unsigned int index1 = h * kARGBSize; + firstColumn[index1] = template_[index]; + firstColumn[index1 + 1] = template_[index + 1]; + firstColumn[index1 + 2] = template_[index + 2]; + firstColumn[index1 + 3] = 0x00; + } + + /* Overwrite template_ */ + uint8_t *buf = template_.get(); + for (size_t h = 0; h < size.height; h++) { + for (size_t w = 0; w < size.width - 1; w++) { + /* Overwrite with the pixel on the right */ + unsigned int index = (h * size.width + w + 1) * kARGBSize; + *buf++ = template_[index]; /* B */ + *buf++ = template_[index + 1]; /* G */ + *buf++ = template_[index + 2]; /* R */ + *buf++ = 0x00; /* A */ + } + /* Overwrite the new last column with the original first column */ + unsigned int index1 = h * kARGBSize; + *buf++ = firstColumn[index1]; /* B */ + *buf++ = firstColumn[index1 + 1]; /* G */ + *buf++ = firstColumn[index1 + 2]; /* R */ + *buf++ = 0x00; /* A */ + } +} + +void ColorBarsGenerator::configure(const Size &size) +{ + constexpr uint8_t kColorBar[8][3] = { + /* R, G, B */ + { 0xff, 0xff, 0xff }, /* White */ + { 0xff, 0xff, 0x00 }, /* Yellow */ + { 0x00, 0xff, 0xff }, /* Cyan */ + { 0x00, 0xff, 0x00 }, /* Green */ + { 0xff, 0x00, 0xff }, /* Magenta */ + { 0xff, 0x00, 0x00 }, /* Red */ + { 0x00, 0x00, 0xff }, /* Blue */ + { 0x00, 0x00, 0x00 }, /* Black */ + }; + + template_ = std::make_unique( + size.width * size.height * kARGBSize); + + unsigned int colorBarWidth = size.width / std::size(kColorBar); + + uint8_t *buf = template_.get(); + for (size_t h = 0; h < size.height; h++) { + for (size_t w = 0; w < size.width; w++) { + /* Repeat when the width is exceed */ + unsigned int index = (w / colorBarWidth) % std::size(kColorBar); + + *buf++ = kColorBar[index][2]; /* B */ + *buf++ = kColorBar[index][1]; /* G */ + *buf++ = kColorBar[index][0]; /* R */ + *buf++ = 0x00; /* A */ + } + } +} + +void DiagonalLinesGenerator::configure(const Size &size) +{ + constexpr uint8_t kColorBar[2][3] = { + /* R, G, B */ + { 0xff, 0xff, 0xff }, /* White */ + { 0x00, 0x00, 0x00 }, /* Black */ + }; + + template_ = std::make_unique( + size.width * size.height * kARGBSize); + + unsigned int lineWidth = size.width / 10; + + uint8_t *buf = template_.get(); + for (size_t h = 0; h < size.height; h++) { + for (size_t w = 0; w < size.width; w++) { + /* Repeat when the width is exceed */ + int index = ((w + h) / lineWidth) % 2; + + *buf++ = kColorBar[index][2]; /* B */ + *buf++ = kColorBar[index][1]; /* G */ + *buf++ = kColorBar[index][0]; /* R */ + *buf++ = 0x00; /* A */ + } + } +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/test_pattern_generator.h b/src/libcamera/pipeline/virtual/test_pattern_generator.h new file mode 100644 index 000000000..2ff1e40ec --- /dev/null +++ b/src/libcamera/pipeline/virtual/test_pattern_generator.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * test_pattern_generator.h - Derived class of FrameGenerator for + * generating test patterns + */ + +#pragma once + +#include + +#include +#include + +#include "frame_generator.h" + +namespace libcamera { + +enum class TestPattern : char { + ColorBars = 0, + DiagonalLines = 1, +}; + +class TestPatternGenerator : public FrameGenerator +{ +public: + int generateFrame(const Size &size, const FrameBuffer *buffer) override; + +protected: + /* Buffer of test pattern template */ + std::unique_ptr template_; + +private: + /* Shift the buffer by 1 pixel left each frame */ + void shiftLeft(const Size &size); +}; + +class ColorBarsGenerator : public TestPatternGenerator +{ +public: + /* Generate a template buffer of the color bar test pattern. */ + void configure(const Size &size) override; +}; + +class DiagonalLinesGenerator : public TestPatternGenerator +{ +public: + /* Generate a template buffer of the diagonal lines test pattern. */ + void configure(const Size &size) override; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index 5cac0e364..1ad7417c7 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -34,6 +34,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/dma_buf_allocator.h" #include "libcamera/internal/formats.h" +#include "libcamera/internal/framebuffer.h" #include "libcamera/internal/pipeline_handler.h" namespace libcamera { @@ -94,6 +95,8 @@ private: return static_cast(camera->_d()); } + void initFrameGenerator(Camera *camera); + DmaBufAllocator dmaBufAllocator_; bool resetCreated_ = false; @@ -241,8 +244,10 @@ int PipelineHandlerVirtual::configure(Camera *camera, CameraConfiguration *config) { VirtualCameraData *data = cameraData(camera); - for (auto [i, c] : utils::enumerate(*config)) + for (auto [i, c] : utils::enumerate(*config)) { c.setStream(&data->streamConfigs_[i].stream); + data->streamConfigs_[i].frameGenerator->configure(c.size); + } return 0; } @@ -278,8 +283,24 @@ void PipelineHandlerVirtual::stopDevice([[maybe_unused]] Camera *camera) int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera, Request *request) { - for (auto it : request->buffers()) - completeBuffer(request, it.second); + VirtualCameraData *data = cameraData(camera); + + for (auto const &[stream, buffer] : request->buffers()) { + bool found = false; + /* map buffer and fill test patterns */ + for (auto &streamConfig : data->streamConfigs_) { + if (stream == &streamConfig.stream) { + found = true; + if (streamConfig.frameGenerator->generateFrame( + stream->configuration().size, buffer)) + buffer->_d()->cancel(); + + completeBuffer(request, buffer); + break; + } + } + ASSERT(found); + } request->metadata().set(controls::SensorTimestamp, currentTimestamp()); completeRequest(request); @@ -325,6 +346,9 @@ bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator const std::string id = "Virtual0"; std::shared_ptr camera = Camera::create(std::move(data), id, streams); + + initFrameGenerator(camera.get()); + registerCamera(std::move(camera)); resetCreated_ = true; @@ -332,6 +356,17 @@ bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator return true; } +void PipelineHandlerVirtual::initFrameGenerator(Camera *camera) +{ + auto data = cameraData(camera); + for (auto &streamConfig : data->streamConfigs_) { + if (data->testPattern_ == TestPattern::DiagonalLines) + streamConfig.frameGenerator = std::make_unique(); + else + streamConfig.frameGenerator = std::make_unique(); + } +} + REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual, "virtual") } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h index bddd3db5a..0b15f8d9c 100644 --- a/src/libcamera/pipeline/virtual/virtual.h +++ b/src/libcamera/pipeline/virtual/virtual.h @@ -15,6 +15,8 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/pipeline_handler.h" +#include "test_pattern_generator.h" + namespace libcamera { class VirtualCameraData : public Camera::Private @@ -28,6 +30,7 @@ public: }; struct StreamConfig { Stream stream; + std::unique_ptr frameGenerator; }; VirtualCameraData(PipelineHandler *pipe, @@ -35,6 +38,8 @@ public: ~VirtualCameraData() = default; + TestPattern testPattern_ = TestPattern::ColorBars; + const std::vector supportedResolutions_; Size maxResolutionSize_; Size minResolutionSize_; diff --git a/src/meson.build b/src/meson.build index 165a77bb9..91bea7753 100644 --- a/src/meson.build +++ b/src/meson.build @@ -27,6 +27,25 @@ else ipa_sign_module = false endif +libyuv_dep = dependency('libyuv', required : false) + +# Fallback to a subproject if libyuv isn't found, as it's typically not +# provided by distributions. +if not libyuv_dep.found() + cmake = import('cmake') + + libyuv_vars = cmake.subproject_options() + libyuv_vars.add_cmake_defines({'CMAKE_POSITION_INDEPENDENT_CODE': 'ON'}) + libyuv_vars.set_override_option('cpp_std', 'c++17') + libyuv_vars.append_compile_args('cpp', + '-Wno-sign-compare', + '-Wno-unused-variable', + '-Wno-unused-parameter') + libyuv_vars.append_link_args('-ljpeg') + libyuv = cmake.subproject('libyuv', options : libyuv_vars) + libyuv_dep = libyuv.dependency('yuv') +endif + # libcamera must be built first as a dependency to the other components. subdir('libcamera') From patchwork Fri Oct 4 09:59:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 21507 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 74F35C3257 for ; Fri, 4 Oct 2024 10:00:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ED89663537; Fri, 4 Oct 2024 12:00:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="PnKCrs1T"; dkim-atps=neutral Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5573063533 for ; Fri, 4 Oct 2024 12:00:02 +0200 (CEST) Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-71dba8b05cbso1930626b3a.3 for ; Fri, 04 Oct 2024 03:00:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1728036000; x=1728640800; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3JyVolO8XjOiqTX/jZ0em4R/BrPOODb1SlyVieKwbNY=; b=PnKCrs1TLAWO8AtLYHrqX4qAkPPpvFrtp5GRoNPJjUoGvmhHlK1HGbdywv+LyVDXyd xzL2Wqb1ana3lnmxZ+oMcWwmte5At/DZglQhMGizMBDh7eF/aNYKpbZaNGMO5tEU7OuV C5URvJje+NGkZk38TdCnAgc0wgV0kBgugZMj8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728036000; x=1728640800; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3JyVolO8XjOiqTX/jZ0em4R/BrPOODb1SlyVieKwbNY=; b=rrXRQMd9iJveMgVtocOX2AMr1mr37WVWMpoCbqgl8KIhGEFlgbrX27aGYrYe3UY8+A CzujG+yCP1aSJJH1Ov8Ro4jMxUbQbB6YeRFZVVTwvFVoCcNXN0eXdrka5bvwHcF74ysU 8BV35myuVDQIZGdhXXnN1LjKOYwgeHPMSfntNi7epcNglCfUp6DDup8XA1qUNTndBKiG ASr8WvsBIPv2cvDmq0ioW9bQtuzJMqHM8/1DkTZS4dHYsDQytl5X2hkAQ0e+I/wABsUQ lkcoiIvRKEWHlNyVAaCJfdiAutNr5Svi0mO9U14QEp5+6EWcU74hVbYZOveShZyiRupH S3eg== X-Gm-Message-State: AOJu0Yx4gfqLAHkMkbrY5P4bVwr9g4LETE4FkPQsJWpuqLlZWOqy8HWi jqTepP7+0B68Yi/nBE0hNnW2KbWCJpYsnypjLD5FXnLruRufTX0GOEX7R8rQxAxH4TxBbBdMCSi FTw== X-Google-Smtp-Source: AGHT+IHtzlSSeVn24RMNWsuuP4JpmyDUU8MP+jJ7KjzwEt4jw/5xCVuSKKx0ORA4br/UyEZtmy8eIg== X-Received: by 2002:a05:6a00:14ca:b0:70d:22b5:5420 with SMTP id d2e1a72fcca58-71de23e7fe9mr3582048b3a.15.1728035999886; Fri, 04 Oct 2024 02:59:59 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (27.247.221.35.bc.googleusercontent.com. [35.221.247.27]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ded2ad931sm316911b3a.41.2024.10.04.02.59.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Oct 2024 02:59:59 -0700 (PDT) From: Harvey Yang X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Konami Shu , Harvey Yang , Yunke Cao , Tomasz Figa , Jacopo Mondi Subject: [PATCH 15 5/7] libcamera: virtual: Add ImageFrameGenerator Date: Fri, 4 Oct 2024 09:59:18 +0000 Message-ID: <20241004095946.264537-6-chenghaoyang@google.com> X-Mailer: git-send-email 2.47.0.rc0.187.ge670bccf7e-goog In-Reply-To: <20241004095946.264537-1-chenghaoyang@google.com> References: <20241004095946.264537-1-chenghaoyang@google.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Besides TestPatternGenerator, this patch adds ImageFrameGenerator that loads real images (jpg / jpeg for now) as the source and generates scaled frames. Signed-off-by: Konami Shu Co-developed-by: Harvey Yang Co-developed-by: Yunke Cao Co-developed-by: Tomasz Figa Reviewed-by: Jacopo Mondi --- .../virtual/image_frame_generator.cpp | 179 ++++++++++++++++++ .../pipeline/virtual/image_frame_generator.h | 51 +++++ src/libcamera/pipeline/virtual/meson.build | 4 + 3 files changed, 234 insertions(+) create mode 100644 src/libcamera/pipeline/virtual/image_frame_generator.cpp create mode 100644 src/libcamera/pipeline/virtual/image_frame_generator.h diff --git a/src/libcamera/pipeline/virtual/image_frame_generator.cpp b/src/libcamera/pipeline/virtual/image_frame_generator.cpp new file mode 100644 index 000000000..04382beb7 --- /dev/null +++ b/src/libcamera/pipeline/virtual/image_frame_generator.cpp @@ -0,0 +1,179 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * image_frame_generator.cpp - Derived class of FrameGenerator for + * generating frames from images + */ + +#include "image_frame_generator.h" + +#include + +#include +#include + +#include + +#include "libcamera/internal/mapped_framebuffer.h" + +#include "libyuv/convert.h" +#include "libyuv/scale.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Virtual) + +/* + * Factory function to create an ImageFrameGenerator object. + * Read the images and convert them to buffers in NV12 format. + * Store the pointers to the buffers to a list (imageFrameDatas) + */ +std::unique_ptr +ImageFrameGenerator::create(ImageFrames &imageFrames) +{ + std::unique_ptr imageFrameGenerator = + std::make_unique(); + imageFrameGenerator->imageFrames_ = &imageFrames; + + /* + * For each file in the directory, load the image, + * convert it to NV12, and store the pointer. + */ + for (unsigned int i = 0; i < imageFrames.number.value_or(1); i++) { + std::filesystem::path path; + if (!imageFrames.number) + /* If the path is to an image */ + path = imageFrames.path; + else + /* If the path is to a directory */ + path = imageFrames.path / (std::to_string(i) + ".jpg"); + + File file(path); + if (!file.open(File::OpenModeFlag::ReadOnly)) { + LOG(Virtual, Error) << "Failed to open image file " << file.fileName() + << ": " << strerror(file.error()); + return nullptr; + } + + /* Read the image file to data */ + auto fileSize = file.size(); + auto buffer = std::make_unique(fileSize); + if (file.read({ buffer.get(), static_cast(fileSize) }) != fileSize) { + LOG(Virtual, Error) << "Failed to read file " << file.fileName() + << ": " << strerror(file.error()); + return nullptr; + } + + /* Get the width and height of the image */ + int width, height; + if (libyuv::MJPGSize(buffer.get(), fileSize, &width, &height)) { + LOG(Virtual, Error) << "Failed to get the size of the image file: " + << file.fileName(); + return nullptr; + } + + std::unique_ptr dstY = + std::make_unique(width * height); + std::unique_ptr dstUV = + std::make_unique(width * height / 2); + int ret = libyuv::MJPGToNV12(buffer.get(), fileSize, + dstY.get(), width, dstUV.get(), + width, width, height, width, height); + if (ret != 0) + LOG(Virtual, Error) << "MJPGToNV12() failed with " << ret; + + imageFrameGenerator->imageFrameDatas_.emplace_back( + ImageFrameData{ std::move(dstY), std::move(dstUV), + Size(width, height) }); + } + + return imageFrameGenerator; +} + +/* + * \var ImageFrameGenerator::frameRepeat + * \brief Number of frames to repeat before proceeding to the next frame + */ + +/* Scale the buffers for image frames. */ +void ImageFrameGenerator::configure(const Size &size) +{ + /* Reset the source images to prevent multiple configuration calls */ + scaledFrameDatas_.clear(); + frameIndex_ = 0; + parameter_ = 0; + + for (unsigned int i = 0; i < imageFrames_->number.value_or(1); i++) { + /* Scale the imageFrameDatas_ to scaledY and scaledUV */ + unsigned int halfSizeWidth = (size.width + 1) / 2; + unsigned int halfSizeHeight = (size.height + 1) / 2; + std::unique_ptr scaledY = + std::make_unique(size.width * size.height); + std::unique_ptr scaledUV = + std::make_unique(halfSizeWidth * halfSizeHeight * 2); + auto &src = imageFrameDatas_[i]; + + /* + * \todo Some platforms might enforce stride due to GPU. + * The width needs to be a multiple of the stride to work + * properly for now. + */ + libyuv::NV12Scale(src.Y.get(), src.size.width, + src.UV.get(), src.size.width, + src.size.width, src.size.height, + scaledY.get(), size.width, scaledUV.get(), size.width, + size.width, size.height, libyuv::FilterMode::kFilterBilinear); + + scaledFrameDatas_.emplace_back( + ImageFrameData{ std::move(scaledY), std::move(scaledUV), size }); + } +} + +int ImageFrameGenerator::generateFrame(const Size &size, const FrameBuffer *buffer) +{ + ASSERT(!scaledFrameDatas_.empty()); + + MappedFrameBuffer mappedFrameBuffer(buffer, MappedFrameBuffer::MapFlag::Write); + + auto planes = mappedFrameBuffer.planes(); + + /* Loop only around the number of images available */ + frameIndex_ %= imageFrames_->number.value_or(1); + + /* Write the scaledY and scaledUV to the mapped frame buffer */ + libyuv::NV12Copy(scaledFrameDatas_[frameIndex_].Y.get(), size.width, + scaledFrameDatas_[frameIndex_].UV.get(), size.width, planes[0].begin(), + size.width, planes[1].begin(), size.width, + size.width, size.height); + + /* Proceed to the next image every 4 frames */ + /* \todo Consider setting the frameRepeat in the config file */ + parameter_++; + if (parameter_ % frameRepeat == 0) + frameIndex_++; + + return 0; +} + +/* + * \var ImageFrameGenerator::imageFrameDatas_ + * \brief List of pointers to the not scaled image buffers + */ + +/* + * \var ImageFrameGenerator::scaledFrameDatas_ + * \brief List of pointers to the scaled image buffers + */ + +/* + * \var ImageFrameGenerator::imageFrames_ + * \brief Pointer to the imageFrames_ in VirtualCameraData + */ + +/* + * \var ImageFrameGenerator::parameter_ + * \brief Speed parameter. Change to the next image every parameter_ frames + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/image_frame_generator.h b/src/libcamera/pipeline/virtual/image_frame_generator.h new file mode 100644 index 000000000..a68094a66 --- /dev/null +++ b/src/libcamera/pipeline/virtual/image_frame_generator.h @@ -0,0 +1,51 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * image_frame_generator.h - Derived class of FrameGenerator for + * generating frames from images + */ + +#pragma once + +#include +#include +#include +#include +#include + +#include "frame_generator.h" + +namespace libcamera { + +/* Frame configuration provided by the config file */ +struct ImageFrames { + std::filesystem::path path; + std::optional number; +}; + +class ImageFrameGenerator : public FrameGenerator +{ +public: + static std::unique_ptr create(ImageFrames &imageFrames); + +private: + static constexpr unsigned int frameRepeat = 4; + + struct ImageFrameData { + std::unique_ptr Y; + std::unique_ptr UV; + Size size; + }; + + void configure(const Size &size) override; + int generateFrame(const Size &size, const FrameBuffer *buffer) override; + + std::vector imageFrameDatas_; + std::vector scaledFrameDatas_; + ImageFrames *imageFrames_; + unsigned int frameIndex_; + unsigned int parameter_; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build index 0c537777f..bb38c193c 100644 --- a/src/libcamera/pipeline/virtual/meson.build +++ b/src/libcamera/pipeline/virtual/meson.build @@ -1,8 +1,12 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'image_frame_generator.cpp', 'test_pattern_generator.cpp', 'virtual.cpp', ]) +libjpeg = dependency('libjpeg', required : true) + libcamera_deps += [libyuv_dep] +libcamera_deps += [libjpeg] From patchwork Fri Oct 4 09:59:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 21508 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3194DC3257 for ; Fri, 4 Oct 2024 10:00:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BD97163537; Fri, 4 Oct 2024 12:00:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Q3Dy4BLf"; dkim-atps=neutral Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2670F6352D for ; Fri, 4 Oct 2024 12:00:04 +0200 (CEST) Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-71de776bc69so286380b3a.1 for ; Fri, 04 Oct 2024 03:00:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1728036002; x=1728640802; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ewx9Da/rZEtCO2SvT1OinnkZN0t86Pz4pHb1XX8yOj0=; b=Q3Dy4BLfpyLjWLdCocOUSqnrJ52WaIg3bodpvB1gHSer778l8ENmkd0kl79fxD0H09 iEmZlIN+2ZJslRAqy87YcgjBwQwrH9zLQDI88JeK2wOKfcJS/Nete4HyHgPAPmPGggh7 tn4vQA57GrAcqT4/7LSgQpedLJrdtYBqTgDWU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728036002; x=1728640802; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ewx9Da/rZEtCO2SvT1OinnkZN0t86Pz4pHb1XX8yOj0=; b=Yl7iN5mjYQjdl/yX8rw6iGZnqIPYzwvat1v+oxDCHwCA5HLzE/halgMtrGEZLdFYL+ HUvUKbCVaSt41GADOswO3KT/SEKMpiKds1HVsuwuEQAir57CWhnjO5aS8RarNUviLKP0 ae7wxJsdKjN+MQUgDtkHP7kAA2pz3up7IeXMNtWIVBpHH5Z1SQ7OygqSpWWz0Fk1nL6j 2v4798UeLY4zzmXzFSbqC7HidcN10+XN/oXTM0uLU/PijwDHo0nznu0XshyI/5b4DH9+ zhyIpAfjPxJEGI6I6HsVsn4IPjNHTF1rx2wSDmsznAV4kZe2ZCKRbwQHNJQ3hoqgl1v3 rHew== X-Gm-Message-State: AOJu0Yxdf3CgsbZ7ecSpZ5ET/G1KWzajYuN2sDCJ40uSTtJ8w45gMckp V7KRBKBk0QSHJfTLpkPoAEFaKEUiVuzI83USYitue7A5/4xi2GUuhF/Iykqad+SRd6tsA9p4zxy 1Jw== X-Google-Smtp-Source: AGHT+IGqCNIu+TF3ddqrncp3hVK9gXRiYCpjx6ulzUAwgezySF4WW6Zs74U0q5fjwsqcck1cIHz0ZQ== X-Received: by 2002:a05:6a00:886:b0:702:3e36:b7c4 with SMTP id d2e1a72fcca58-71de23a6620mr3113927b3a.5.1728036001682; Fri, 04 Oct 2024 03:00:01 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (27.247.221.35.bc.googleusercontent.com. [35.221.247.27]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ded2ad931sm316911b3a.41.2024.10.04.03.00.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Oct 2024 03:00:01 -0700 (PDT) From: Harvey Yang X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Konami Shu , Harvey Yang , Yunke Cao , Tomasz Figa Subject: [PATCH 15 6/7] libcamera: virtual: Read config and register cameras based on the config Date: Fri, 4 Oct 2024 09:59:19 +0000 Message-ID: <20241004095946.264537-7-chenghaoyang@google.com> X-Mailer: git-send-email 2.47.0.rc0.187.ge670bccf7e-goog In-Reply-To: <20241004095946.264537-1-chenghaoyang@google.com> References: <20241004095946.264537-1-chenghaoyang@google.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch introduces the configuration file for Virtual Pipeline Handler. The config file is written in yaml, and the format is documented in `README.md`. The config file will define the camera with IDs, supported formats and image sources, etc. In the default config file, only Test Patterns are used. Developers can use real images loading if desired. Signed-off-by: Konami Shu Co-developed-by: Harvey Yang Co-developed-by: Yunke Cao Co-developed-by: Tomasz Figa Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/virtual/README.md | 65 +++++ .../pipeline/virtual/config_parser.cpp | 263 ++++++++++++++++++ .../pipeline/virtual/config_parser.h | 39 +++ .../pipeline/virtual/data/virtual.yaml | 36 +++ .../virtual/image_frame_generator.cpp | 16 +- .../pipeline/virtual/image_frame_generator.h | 5 +- src/libcamera/pipeline/virtual/meson.build | 1 + src/libcamera/pipeline/virtual/virtual.cpp | 126 +++++---- src/libcamera/pipeline/virtual/virtual.h | 23 +- 9 files changed, 504 insertions(+), 70 deletions(-) create mode 100644 src/libcamera/pipeline/virtual/README.md create mode 100644 src/libcamera/pipeline/virtual/config_parser.cpp create mode 100644 src/libcamera/pipeline/virtual/config_parser.h create mode 100644 src/libcamera/pipeline/virtual/data/virtual.yaml diff --git a/src/libcamera/pipeline/virtual/README.md b/src/libcamera/pipeline/virtual/README.md new file mode 100644 index 000000000..5801e79b5 --- /dev/null +++ b/src/libcamera/pipeline/virtual/README.md @@ -0,0 +1,65 @@ +# Virtual Pipeline Handler + +Virtual pipeline handler emulates fake external camera(s) on ChromeOS for testing. + +## Parse config file and register cameras + +- The config file is located at `src/libcamera/pipeline/virtual/data/virtual.yaml` + +### Config File Format +The config file contains the information about cameras' properties to register. +The config file should be a yaml file with dictionary of the cameraIds +associated with their properties as top level. The default value will be applied +when any property is empty. + +Each camera block is a dictionary, containing the following keys: +- `supported_formats` (list of `VirtualCameraData::Resolution`, optional) : + List of supported resolution and frame rates of the emulated camera + - `width` (`unsigned int`, default=1920): Width of the window resolution. + This needs to be even. + - `height` (`unsigned int`, default=1080): Height of the window resolution. + - `frame_rates` (list of `int`, default=`[30,60]` ): Range of the frame + rate (per second). If the list contains one value, it's the lower bound + and the upper bound. If the list contains two values, the first is the + lower bound and the second is the upper bound. No other number of values + is allowed. +- `test_pattern` (`string`): Which test pattern to use as frames. The options + are "bars", "lines". Cannot be set with `frames`. +- `frames` (dictionary): + - `path` (`string`): Path to an image, or path to a directory of a series of + images. Cannot be set with `test_pattern`. + - The test patterns are "bars" which means color bars, and "lines" which + means diagonal lines. + - The path to an image has ".jpg" extension. + - The path to a directory ends with "/". The name of the images in the + directory are "{n}.jpg" with {n} is the sequence of images starting with 0. +- `location` (`string`, default="front"): The location of the camera. Support + "front" and "back". This is displayed in qcam camera selection window but + this does not change the output. +- `model` (`string`, default="Unknown"): The model name of the camera. + This is displayed in qcam camera selection window but this does not change the output. + +Check `data/virtual.yaml` as the sample config file. + +### Implementation + +`Parser` class provides methods to parse the config file to register cameras +in Virtual Pipeline Handler. `parseConfigFile()` is exposed to use in +Virtual Pipeline Handler. + +This is the procedure of the Parser class: +1. `parseConfigFile()` parses the config file to `YamlObject` using `YamlParser::parse()`. + - Parse the top level of config file which are the camera ids and look into + each camera properties. +2. For each camera, `parseCameraConfigData()` returns a camera with the configuration. + - The methods in the next step fill the data with the pointer to the Camera object. + - If the config file contains invalid configuration, this method returns + nullptr. The camera will be skipped. +3. Parse each property and register the data. + - `parseSupportedFormats()`: Parses `supported_formats` in the config, which + contains resolutions and frame rates. + - `parseFrameGenerator()`: Parses `test_pattern` or `frames` in the config. + - `parseLocation()`: Parses `location` in the config. + - `parseModel()`: Parses `model` in the config. +4. Back to `parseConfigFile()` and append the camera configuration. +5. Returns a list of camera configurations. diff --git a/src/libcamera/pipeline/virtual/config_parser.cpp b/src/libcamera/pipeline/virtual/config_parser.cpp new file mode 100644 index 000000000..467dde485 --- /dev/null +++ b/src/libcamera/pipeline/virtual/config_parser.cpp @@ -0,0 +1,263 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * config_parser.cpp - Virtual cameras helper to parse config file + */ + +#include "config_parser.h" + +#include +#include + +#include + +#include +#include + +#include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/yaml_parser.h" + +#include "virtual.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(Virtual) + +std::vector> +ConfigParser::parseConfigFile(File &file, PipelineHandler *pipe) +{ + std::vector> configurations; + + std::unique_ptr cameras = YamlParser::parse(file); + if (!cameras) { + LOG(Virtual, Error) << "Failed to pass config file."; + return configurations; + } + + if (!cameras->isDictionary()) { + LOG(Virtual, Error) << "Config file is not a dictionary at the top level."; + return configurations; + } + + /* Look into the configuration of each camera */ + for (const auto &[cameraId, cameraConfigData] : cameras->asDict()) { + std::unique_ptr data = + parseCameraConfigData(cameraConfigData, pipe); + /* Parse configData to data */ + if (!data) { + /* Skip the camera if it has invalid config */ + LOG(Virtual, Error) << "Failed to parse config of the camera: " + << cameraId; + continue; + } + + data->config_.id = cameraId; + ControlInfoMap::Map controls; + /* todo: Check which resolution's frame rate to be reported */ + controls[&controls::FrameDurationLimits] = + ControlInfo(1000000 / data->config_.resolutions[0].frameRates[1], + 1000000 / data->config_.resolutions[0].frameRates[0]); + + std::vector supportedFaceDetectModes{ + static_cast(controls::draft::FaceDetectModeOff), + }; + controls[&controls::draft::FaceDetectMode] = ControlInfo(supportedFaceDetectModes); + + data->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); + configurations.push_back(std::move(data)); + } + + return configurations; +} + +std::unique_ptr +ConfigParser::parseCameraConfigData(const YamlObject &cameraConfigData, + PipelineHandler *pipe) +{ + std::vector resolutions; + if (parseSupportedFormats(cameraConfigData, &resolutions)) + return nullptr; + + std::unique_ptr data = + std::make_unique(pipe, resolutions); + + if (parseFrameGenerator(cameraConfigData, data.get())) + return nullptr; + + if (parseLocation(cameraConfigData, data.get())) + return nullptr; + + if (parseModel(cameraConfigData, data.get())) + return nullptr; + + return data; +} + +int ConfigParser::parseSupportedFormats(const YamlObject &cameraConfigData, + std::vector *resolutions) +{ + if (cameraConfigData.contains("supported_formats")) { + const YamlObject &supportedResolutions = cameraConfigData["supported_formats"]; + + for (const YamlObject &supportedResolution : supportedResolutions.asList()) { + unsigned int width = supportedResolution["width"].get(1920); + unsigned int height = supportedResolution["height"].get(1080); + if (width == 0 || height == 0) { + LOG(Virtual, Error) << "Invalid width or/and height"; + return -EINVAL; + } + if (width % 2 != 0) { + LOG(Virtual, Error) << "Invalid width: width needs to be even"; + return -EINVAL; + } + + std::vector frameRates; + if (supportedResolution.contains("frame_rates")) { + auto frameRatesList = + supportedResolution["frame_rates"].getList(); + if (!frameRatesList || (frameRatesList->size() != 1 && + frameRatesList->size() != 2)) { + LOG(Virtual, Error) << "Invalid frame_rates: either one or two values"; + return -EINVAL; + } + + if (frameRatesList->size() == 2 && + frameRatesList.value()[0] > frameRatesList.value()[1]) { + LOG(Virtual, Error) << "frame_rates's first value(lower bound)" + << " is higher than the second value(upper bound)"; + return -EINVAL; + } + /* + * Push the min and max framerates. A + * single rate is duplicated. + */ + frameRates.push_back(frameRatesList.value().front()); + frameRates.push_back(frameRatesList.value().back()); + } else { + frameRates.push_back(30); + frameRates.push_back(60); + } + + resolutions->emplace_back( + VirtualCameraData::Resolution{ Size{ width, height }, + frameRates }); + } + } else { + resolutions->emplace_back( + VirtualCameraData::Resolution{ Size{ 1920, 1080 }, + { 30, 60 } }); + } + + return 0; +} + +int ConfigParser::parseFrameGenerator(const YamlObject &cameraConfigData, VirtualCameraData *data) +{ + const std::string testPatternKey = "test_pattern"; + const std::string framesKey = "frames"; + if (cameraConfigData.contains(testPatternKey)) { + if (cameraConfigData.contains(framesKey)) { + LOG(Virtual, Error) << "A camera should use either " + << testPatternKey << " or " << framesKey; + return -EINVAL; + } + + auto testPattern = cameraConfigData[testPatternKey].get(""); + + if (testPattern == "bars") { + data->config_.frame = TestPattern::ColorBars; + } else if (testPattern == "lines") { + data->config_.frame = TestPattern::DiagonalLines; + } else { + LOG(Virtual, Debug) << "Test pattern: " << testPattern + << " is not supported"; + return -EINVAL; + } + + return 0; + } + + const YamlObject &frames = cameraConfigData[framesKey]; + + /* When there is no frames provided in the config file, use color bar test pattern */ + if (!frames) { + data->config_.frame = TestPattern::ColorBars; + return 0; + } + + if (!frames.isDictionary()) { + LOG(Virtual, Error) << "'frames' is not a dictionary."; + return -EINVAL; + } + + auto path = frames["path"].get(); + + if (!path) { + LOG(Virtual, Error) << "Test pattern or path should be specified."; + return -EINVAL; + } + + std::vector files; + + switch (std::filesystem::symlink_status(*path).type()) { + case std::filesystem::file_type::regular: + files.push_back(*path); + break; + + case std::filesystem::file_type::directory: + for (const auto &dentry : std::filesystem::directory_iterator{ *path }) { + if (dentry.is_regular_file()) + files.push_back(dentry.path()); + } + + std::sort(files.begin(), files.end(), [](const auto &a, const auto &b) { + return ::strverscmp(a.c_str(), b.c_str()) < 0; + }); + + if (files.empty()) { + LOG(Virtual, Error) << "Directory has no files: " << *path; + return -EINVAL; + } + break; + + default: + LOG(Virtual, Error) << "Frame: " << *path << " is not supported"; + return -EINVAL; + } + + data->config_.frame = ImageFrames{ std::move(files) }; + + return 0; +} + +int ConfigParser::parseLocation(const YamlObject &cameraConfigData, VirtualCameraData *data) +{ + std::string location = cameraConfigData["location"].get("front"); + + /* Default value is properties::CameraLocationFront */ + if (location == "front") { + data->properties_.set(properties::Location, + properties::CameraLocationFront); + } else if (location == "back") { + data->properties_.set(properties::Location, + properties::CameraLocationBack); + } else { + LOG(Virtual, Error) << "location: " << location + << " is not supported"; + return -EINVAL; + } + + return 0; +} + +int ConfigParser::parseModel(const YamlObject &cameraConfigData, VirtualCameraData *data) +{ + std::string model = cameraConfigData["model"].get("Unknown"); + + data->properties_.set(properties::Model, model); + + return 0; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/config_parser.h b/src/libcamera/pipeline/virtual/config_parser.h new file mode 100644 index 000000000..9abba62d0 --- /dev/null +++ b/src/libcamera/pipeline/virtual/config_parser.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Google Inc. + * + * config_parser.h - Virtual cameras helper to parse config file + */ + +#pragma once + +#include +#include + +#include + +#include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/yaml_parser.h" + +#include "virtual.h" + +namespace libcamera { + +class ConfigParser +{ +public: + std::vector> + parseConfigFile(File &file, PipelineHandler *pipe); + +private: + std::unique_ptr + parseCameraConfigData(const YamlObject &cameraConfigData, PipelineHandler *pipe); + + int parseSupportedFormats(const YamlObject &cameraConfigData, + std::vector *resolutions); + int parseFrameGenerator(const YamlObject &cameraConfigData, VirtualCameraData *data); + int parseLocation(const YamlObject &cameraConfigData, VirtualCameraData *data); + int parseModel(const YamlObject &cameraConfigData, VirtualCameraData *data); +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/virtual/data/virtual.yaml b/src/libcamera/pipeline/virtual/data/virtual.yaml new file mode 100644 index 000000000..6b73ddf2b --- /dev/null +++ b/src/libcamera/pipeline/virtual/data/virtual.yaml @@ -0,0 +1,36 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +"Virtual0": + supported_formats: + - width: 1920 + height: 1080 + frame_rates: + - 30 + - 60 + - width: 1680 + height: 1050 + frame_rates: + - 70 + - 80 + test_pattern: "lines" + location: "front" + model: "Virtual Video Device" +"Virtual1": + supported_formats: + - width: 800 + height: 600 + frame_rates: + - 60 + test_pattern: "bars" + location: "back" + model: "Virtual Video Device1" +"Virtual2": + supported_formats: + - width: 400 + height: 300 + test_pattern: "lines" + location: "front" + model: "Virtual Video Device2" +"Virtual3": + test_pattern: "bars" diff --git a/src/libcamera/pipeline/virtual/image_frame_generator.cpp b/src/libcamera/pipeline/virtual/image_frame_generator.cpp index 04382beb7..36bdc20e5 100644 --- a/src/libcamera/pipeline/virtual/image_frame_generator.cpp +++ b/src/libcamera/pipeline/virtual/image_frame_generator.cpp @@ -40,15 +40,7 @@ ImageFrameGenerator::create(ImageFrames &imageFrames) * For each file in the directory, load the image, * convert it to NV12, and store the pointer. */ - for (unsigned int i = 0; i < imageFrames.number.value_or(1); i++) { - std::filesystem::path path; - if (!imageFrames.number) - /* If the path is to an image */ - path = imageFrames.path; - else - /* If the path is to a directory */ - path = imageFrames.path / (std::to_string(i) + ".jpg"); - + for (std::filesystem::path path : imageFrames.files) { File file(path); if (!file.open(File::OpenModeFlag::ReadOnly)) { LOG(Virtual, Error) << "Failed to open image file " << file.fileName() @@ -88,6 +80,8 @@ ImageFrameGenerator::create(ImageFrames &imageFrames) Size(width, height) }); } + ASSERT(!imageFrameGenerator->imageFrameDatas_.empty()); + return imageFrameGenerator; } @@ -104,7 +98,7 @@ void ImageFrameGenerator::configure(const Size &size) frameIndex_ = 0; parameter_ = 0; - for (unsigned int i = 0; i < imageFrames_->number.value_or(1); i++) { + for (unsigned int i = 0; i < imageFrameDatas_.size(); i++) { /* Scale the imageFrameDatas_ to scaledY and scaledUV */ unsigned int halfSizeWidth = (size.width + 1) / 2; unsigned int halfSizeHeight = (size.height + 1) / 2; @@ -139,7 +133,7 @@ int ImageFrameGenerator::generateFrame(const Size &size, const FrameBuffer *buff auto planes = mappedFrameBuffer.planes(); /* Loop only around the number of images available */ - frameIndex_ %= imageFrames_->number.value_or(1); + frameIndex_ %= imageFrameDatas_.size(); /* Write the scaledY and scaledUV to the mapped frame buffer */ libyuv::NV12Copy(scaledFrameDatas_[frameIndex_].Y.get(), size.width, diff --git a/src/libcamera/pipeline/virtual/image_frame_generator.h b/src/libcamera/pipeline/virtual/image_frame_generator.h index a68094a66..8554d647d 100644 --- a/src/libcamera/pipeline/virtual/image_frame_generator.h +++ b/src/libcamera/pipeline/virtual/image_frame_generator.h @@ -10,9 +10,9 @@ #include #include -#include #include #include +#include #include "frame_generator.h" @@ -20,8 +20,7 @@ namespace libcamera { /* Frame configuration provided by the config file */ struct ImageFrames { - std::filesystem::path path; - std::optional number; + std::vector files; }; class ImageFrameGenerator : public FrameGenerator diff --git a/src/libcamera/pipeline/virtual/meson.build b/src/libcamera/pipeline/virtual/meson.build index bb38c193c..4786fe2e0 100644 --- a/src/libcamera/pipeline/virtual/meson.build +++ b/src/libcamera/pipeline/virtual/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ + 'config_parser.cpp', 'image_frame_generator.cpp', 'test_pattern_generator.cpp', 'virtual.cpp', diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index 1ad7417c7..cafd395c3 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -36,6 +36,9 @@ #include "libcamera/internal/formats.h" #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/yaml_parser.h" + +#include "pipeline/virtual/config_parser.h" namespace libcamera { @@ -54,6 +57,13 @@ uint64_t currentTimestamp() } /* namespace */ +template +struct overloaded : Ts... { + using Ts::operator()...; +}; +template +overloaded(Ts...) -> overloaded; + class VirtualCameraConfiguration : public CameraConfiguration { public: @@ -95,7 +105,7 @@ private: return static_cast(camera->_d()); } - void initFrameGenerator(Camera *camera); + bool initFrameGenerator(Camera *camera); DmaBufAllocator dmaBufAllocator_; @@ -104,15 +114,19 @@ private: VirtualCameraData::VirtualCameraData(PipelineHandler *pipe, const std::vector &supportedResolutions) - : Camera::Private(pipe), supportedResolutions_(supportedResolutions) + : Camera::Private(pipe) { - for (const auto &resolution : supportedResolutions_) { - if (minResolutionSize_.isNull() || minResolutionSize_ > resolution.size) - minResolutionSize_ = resolution.size; + config_.resolutions = supportedResolutions; + for (const auto &resolution : config_.resolutions) { + if (config_.minResolutionSize.isNull() || config_.minResolutionSize > resolution.size) + config_.minResolutionSize = resolution.size; - maxResolutionSize_ = std::max(maxResolutionSize_, resolution.size); + config_.maxResolutionSize = std::max(config_.maxResolutionSize, resolution.size); } + properties_.set(properties::PixelArrayActiveAreas, + { Rectangle(config_.maxResolutionSize) }); + /* \todo Support multiple streams and pass multi_stream_test */ streamConfigs_.resize(kMaxStream); } @@ -140,7 +154,7 @@ CameraConfiguration::Status VirtualCameraConfiguration::validate() for (StreamConfiguration &cfg : config_) { bool adjusted = false; bool found = false; - for (const auto &resolution : data_->supportedResolutions_) { + for (const auto &resolution : data_->config_.resolutions) { if (resolution.size.width == cfg.size.width && resolution.size.height == cfg.size.height) { found = true; @@ -155,7 +169,7 @@ CameraConfiguration::Status VirtualCameraConfiguration::validate() * Defining the default logic in PipelineHandler to * find the closest resolution would be nice. */ - cfg.size = data_->maxResolutionSize_; + cfg.size = data_->config_.maxResolutionSize; status = Adjusted; adjusted = true; } @@ -224,12 +238,12 @@ PipelineHandlerVirtual::generateConfiguration(Camera *camera, std::map> streamFormats; PixelFormat pixelFormat = formats::NV12; - streamFormats[pixelFormat] = { { data->minResolutionSize_, - data->maxResolutionSize_ } }; + streamFormats[pixelFormat] = { { data->config_.minResolutionSize, + data->config_.maxResolutionSize } }; StreamFormats formats(streamFormats); StreamConfiguration cfg(formats); cfg.pixelFormat = pixelFormat; - cfg.size = data->maxResolutionSize_; + cfg.size = data->config_.maxResolutionSize; cfg.bufferCount = VirtualCameraConfiguration::kBufferCount; config->addConfiguration(cfg); @@ -246,6 +260,7 @@ int PipelineHandlerVirtual::configure(Camera *camera, VirtualCameraData *data = cameraData(camera); for (auto [i, c] : utils::enumerate(*config)) { c.setStream(&data->streamConfigs_[i].stream); + /* Start reading the images/generating test patterns */ data->streamConfigs_[i].frameGenerator->configure(c.size); } @@ -315,56 +330,67 @@ bool PipelineHandlerVirtual::match([[maybe_unused]] DeviceEnumerator *enumerator created_ = true; - /* \todo Add virtual cameras according to a config file. */ - - std::vector supportedResolutions; - supportedResolutions.resize(2); - supportedResolutions[0] = { .size = Size(1920, 1080), .frameRates = { 30 } }; - supportedResolutions[1] = { .size = Size(1280, 720), .frameRates = { 30 } }; - - std::unique_ptr data = - std::make_unique(this, supportedResolutions); - - data->properties_.set(properties::Location, properties::CameraLocationFront); - data->properties_.set(properties::Model, "Virtual Video Device"); - data->properties_.set(properties::PixelArrayActiveAreas, { Rectangle(Size(1920, 1080)) }); - - /* \todo Set FrameDurationLimits based on config. */ - ControlInfoMap::Map controls; - int64_t min_frame_duration = 33333, max_frame_duration = 33333; - controls[&controls::FrameDurationLimits] = ControlInfo(min_frame_duration, max_frame_duration); - std::vector supportedFaceDetectModes{ - static_cast(controls::draft::FaceDetectModeOff), - }; - controls[&controls::draft::FaceDetectMode] = ControlInfo(supportedFaceDetectModes); - data->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); - - /* Create and register the camera. */ - std::set streams; - for (auto &streamConfig : data->streamConfigs_) - streams.insert(&streamConfig.stream); + File file(configurationFile("virtual", "virtual.yaml")); + bool isOpen = file.open(File::OpenModeFlag::ReadOnly); + if (!isOpen) { + LOG(Virtual, Error) << "Failed to open config file: " << file.fileName(); + return false; + } - const std::string id = "Virtual0"; - std::shared_ptr camera = Camera::create(std::move(data), id, streams); + ConfigParser parser; + auto configData = parser.parseConfigFile(file, this); + if (configData.size() == 0) { + LOG(Virtual, Error) << "Failed to parse any cameras from the config file: " + << file.fileName(); + return false; + } - initFrameGenerator(camera.get()); + /* Configure and register cameras with configData */ + for (auto &data : configData) { + std::set streams; + for (auto &streamConfig : data->streamConfigs_) + streams.insert(&streamConfig.stream); + std::string id = data->config_.id; + std::shared_ptr camera = Camera::create(std::move(data), id, streams); + + if (!initFrameGenerator(camera.get())) { + LOG(Virtual, Error) << "Failed to initialize frame " + << "generator for camera: " << id; + continue; + } - registerCamera(std::move(camera)); + registerCamera(std::move(camera)); + } resetCreated_ = true; return true; } -void PipelineHandlerVirtual::initFrameGenerator(Camera *camera) +bool PipelineHandlerVirtual::initFrameGenerator(Camera *camera) { auto data = cameraData(camera); - for (auto &streamConfig : data->streamConfigs_) { - if (data->testPattern_ == TestPattern::DiagonalLines) - streamConfig.frameGenerator = std::make_unique(); - else - streamConfig.frameGenerator = std::make_unique(); - } + auto &frame = data->config_.frame; + std::visit(overloaded{ + [&](TestPattern &testPattern) { + for (auto &streamConfig : data->streamConfigs_) { + if (testPattern == TestPattern::DiagonalLines) + streamConfig.frameGenerator = std::make_unique(); + else + streamConfig.frameGenerator = std::make_unique(); + } + }, + [&](ImageFrames &imageFrames) { + for (auto &streamConfig : data->streamConfigs_) + streamConfig.frameGenerator = ImageFrameGenerator::create(imageFrames); + } }, + frame); + + for (auto &streamConfig : data->streamConfigs_) + if (!streamConfig.frameGenerator) + return false; + + return true; } REGISTER_PIPELINE_HANDLER(PipelineHandlerVirtual, "virtual") diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h index 0b15f8d9c..8c4d6ae64 100644 --- a/src/libcamera/pipeline/virtual/virtual.h +++ b/src/libcamera/pipeline/virtual/virtual.h @@ -7,6 +7,8 @@ #pragma once +#include +#include #include #include @@ -15,10 +17,14 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/pipeline_handler.h" +#include "frame_generator.h" +#include "image_frame_generator.h" #include "test_pattern_generator.h" namespace libcamera { +using VirtualFrame = std::variant; + class VirtualCameraData : public Camera::Private { public: @@ -26,23 +32,28 @@ public: struct Resolution { Size size; - std::vector frameRates; + std::vector frameRates; }; struct StreamConfig { Stream stream; std::unique_ptr frameGenerator; }; + /* The config file is parsed to the Configuration struct */ + struct Configuration { + std::string id; + std::vector resolutions; + VirtualFrame frame; + + Size maxResolutionSize; + Size minResolutionSize; + }; VirtualCameraData(PipelineHandler *pipe, const std::vector &supportedResolutions); ~VirtualCameraData() = default; - TestPattern testPattern_ = TestPattern::ColorBars; - - const std::vector supportedResolutions_; - Size maxResolutionSize_; - Size minResolutionSize_; + Configuration config_; std::vector streamConfigs_; }; From patchwork Fri Oct 4 09:59:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 21509 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C67B1C3263 for ; Fri, 4 Oct 2024 10:00:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C341163532; Fri, 4 Oct 2024 12:00:16 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="is5ll8ZR"; dkim-atps=neutral Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B468065365 for ; Fri, 4 Oct 2024 12:00:05 +0200 (CEST) Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-71957eb256bso1735957b3a.3 for ; Fri, 04 Oct 2024 03:00:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1728036004; x=1728640804; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7oxGveuNcRHzMqD3QAv03hUxkHXc5hzggXnXt3vxi9U=; b=is5ll8ZRtrAWWJZFQjWMx5GfbdbbmoqdvKqXfTFt5ErWVpQFo/8+bRmYKBreLiShLl WsjbpCojqpCAWDXi0pjQyKiOZCFEIKdIOMO+JhAGlY6tnC68NjFJrGbEyDQq9TRgBw9v mrw7e+EjDdSC6vLfi9tQdj0w1MpDhOobZuZd0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728036004; x=1728640804; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7oxGveuNcRHzMqD3QAv03hUxkHXc5hzggXnXt3vxi9U=; b=oGxZaeTklu3z5+8Rc1VFu4ZFmDmxORL99gVzX0IYrkVNebb3g2jyUb/YOMJaEf5LyX fkpmoLI8t75I55v+Ny+at/2RtYz23CzsDs1sN/mTqSeyLTgnrIM8WXjZZ+wc3BX3aKhE C1Yo1D0ohnbkkTz+6FZdgMm+DmzcMSd8pJrNNqWHgkk8bhGkf7u9p29vKLITt4ldh7tD tcUZE96IPLRlGxSe/nWUI7tYnvQUpiq2j6wgIzYq1vKx52dfvQ1omkkwqt8K+nsvzVYw /8shO3mI6ccr1PpjctvFXl5Ou9RW68lfKgGMFvMmWTV+OHYVjTw+F7T44/giTUcA107D f4SQ== X-Gm-Message-State: AOJu0YxRUCVYVhYvcFd4pYEk43hDWu2jWDPm1My/cgYb4yZyirReBCFK lFXKVAivrFBrZOARpcsYIF/6MfPtEp+E3K4gvawGIepziasrdxg3apHon5EEcA3bmwBmZiWBJjO CSA== X-Google-Smtp-Source: AGHT+IE/QAeFepXHyLksF+1w/7Ao0UB453htpLXPllqzyD+nIPErNIP5r5mU6wiZ06csS63ODlNhcA== X-Received: by 2002:a05:6a00:c94:b0:71d:d1b4:b454 with SMTP id d2e1a72fcca58-71de23a95bemr3084145b3a.2.1728036003508; Fri, 04 Oct 2024 03:00:03 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (27.247.221.35.bc.googleusercontent.com. [35.221.247.27]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ded2ad931sm316911b3a.41.2024.10.04.03.00.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Oct 2024 03:00:03 -0700 (PDT) From: Harvey Yang X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Harvey Yang , Milan Zamazal , Kieran Bingham Subject: [PATCH 15 7/7] libcamera: software_isp: Refactor SoftwareIsp to use DmaBufAllocator::exportBuffers Date: Fri, 4 Oct 2024 09:59:20 +0000 Message-ID: <20241004095946.264537-8-chenghaoyang@google.com> X-Mailer: git-send-email 2.47.0.rc0.187.ge670bccf7e-goog In-Reply-To: <20241004095946.264537-1-chenghaoyang@google.com> References: <20241004095946.264537-1-chenghaoyang@google.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As the helper function DmaBufAllocator::exportBuffers is added, we can avoid some code duplication in SoftwareIsp as well. Signed-off-by: Harvey Yang Reviewed-by: Milan Zamazal Reviewed-by: Kieran Bingham --- src/libcamera/software_isp/software_isp.cpp | 20 +------------------- 1 file changed, 1 insertion(+), 19 deletions(-) diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 47677784d..5f58e5e40 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -256,25 +256,7 @@ int SoftwareIsp::exportBuffers(const Stream *stream, unsigned int count, if (stream == nullptr) return -EINVAL; - for (unsigned int i = 0; i < count; i++) { - const std::string name = "frame-" + std::to_string(i); - const size_t frameSize = debayer_->frameSize(); - - FrameBuffer::Plane outPlane; - outPlane.fd = SharedFD(dmaHeap_.alloc(name.c_str(), frameSize)); - if (!outPlane.fd.isValid()) { - LOG(SoftwareIsp, Error) - << "failed to allocate a dma_buf"; - return -ENOMEM; - } - outPlane.offset = 0; - outPlane.length = frameSize; - - std::vector planes{ outPlane }; - buffers->emplace_back(std::make_unique(std::move(planes))); - } - - return count; + return dmaHeap_.exportBuffers(count, { debayer_->frameSize() }, buffers); } /**