{"id":21557,"url":"https://patchwork.libcamera.org/api/patches/21557/?format=json","web_url":"https://patchwork.libcamera.org/patch/21557/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20241009074642.2965791-6-chenghaoyang@chromium.org>","date":"2024-10-09T07:41:23","name":"[5/5] libcamera: Add InfoFrame implememtation","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"e6efab69893090210d050cd4ce8072f9e5fa6a7f","submitter":{"id":117,"url":"https://patchwork.libcamera.org/api/people/117/?format=json","name":"Cheng-Hao Yang","email":"chenghaoyang@chromium.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/21557/mbox/","series":[{"id":4670,"url":"https://patchwork.libcamera.org/api/series/4670/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4670","date":"2024-10-09T07:41:18","name":"Add InfoFrame","version":1,"mbox":"https://patchwork.libcamera.org/series/4670/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/21557/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/21557/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 1C62CC32DE\n\tfor ;\n\tWed, 9 Oct 2024 07:47:03 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C72A265372;\n\tWed, 9 Oct 2024 09:47:02 +0200 (CEST)","from mail-pl1-x629.google.com (mail-pl1-x629.google.com\n\t[IPv6:2607:f8b0:4864:20::629])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 06CD965372\n\tfor ;\n\tWed, 9 Oct 2024 09:46:57 +0200 (CEST)","by mail-pl1-x629.google.com with SMTP id\n\td9443c01a7336-20c593d6b1cso13713525ad.0\n\tfor ;\n\tWed, 09 Oct 2024 00:46:56 -0700 (PDT)","from chenghaoyang-low.c.googlers.com.com\n\t(199.211.81.34.bc.googleusercontent.com. [34.81.211.199])\n\tby smtp.gmail.com with ESMTPSA id\n\t98e67ed59e1d1-2e2a57077c2sm944414a91.17.2024.10.09.00.46.53\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tWed, 09 Oct 2024 00:46:54 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"Cemb5dEo\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=chromium.org; s=google; t=1728460015; x=1729064815;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=1jCHNJHRlUl6YhNHIyh2oLEZ9RXC+uN9lCYMqUib2tg=;\n\tb=Cemb5dEonj3IEN+YjMNCfHX/6XdFUsWBS1/4rElENluycL9cxuscImQAi9CYTzW1h+\n\tXORyGeuyx7fwGmoHo7pE9nMTzxCi48pujZdSZdZDGx5THUoEmKo8wc41wspZaZ17YdWr\n\tvBRABNNH8eYOjruK2ClLY5l2MTzXSni/W5s+s=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1728460015; x=1729064815;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc\n\t:subject:date:message-id:reply-to;\n\tbh=1jCHNJHRlUl6YhNHIyh2oLEZ9RXC+uN9lCYMqUib2tg=;\n\tb=YMidVPThteqatbB+rpEiYsyNViCxbTrc1QNNlx70/Nxs0PWBcdgsKPLpBYUQAJxqM2\n\ttvckko+o8hFa7Sh+BbCY/2iXxU6WdtLHFN6Qch1xWx6h+K20R0eeDx+/NBLxdI9daLJz\n\tQUTfhIv43y1zfYQmmysjNbNmOXWlkqXU7uWKuVe8Dtz6aLvaiDOKLHFBLCfXxLWLIcue\n\tnaXbW/Y1flIMH3gzn26PQcdibKELchDCtzs1DeiyM1cvuE2D1ICdaIsmUbXX1ts7bvyd\n\tn7/TkWvPLGidZW2XQDhLUWiR2EO7JF3Ik6KXouauR7c55kTUAEs2MMuntSnWlm6S8rl1\n\tx+/A==","X-Gm-Message-State":"AOJu0YzxmtDRHPdFGIyJCK+hUuK1PdzVOpA+Xv327N6ASjVBov3ukWyP\n\tSdfriyvhWBRpJDbvgcJ/LDAe7ETrvrcKiiyJgLyK54IrnxkSzNVCWWGndsTDxCSrl0unppJxpl8\n\tW/A==","X-Google-Smtp-Source":"AGHT+IEZf+5PvCmrrbWy3OGP1dJ6Kf3KUG1/Dh5z8gwbe1pQAGZeDHFQgp8JVnlK9gX4w6uFMFuXDQ==","X-Received":"by 2002:a17:902:f68a:b0:206:aac4:b844 with SMTP id\n\td9443c01a7336-20c636dcbf4mr22252365ad.6.1728460015087; \n\tWed, 09 Oct 2024 00:46:55 -0700 (PDT)","From":"Harvey Yang ","To":"libcamera-devel@lists.libcamera.org","Cc":"Harvey Yang ,\n\tHan-Lin Chen ","Subject":"[PATCH 5/5] libcamera: Add InfoFrame implememtation","Date":"Wed, 9 Oct 2024 07:41:23 +0000","Message-ID":"<20241009074642.2965791-6-chenghaoyang@chromium.org>","X-Mailer":"git-send-email 2.47.0.rc0.187.ge670bccf7e-goog","In-Reply-To":"<20241009074642.2965791-1-chenghaoyang@chromium.org>","References":"<20241009074642.2965791-1-chenghaoyang@chromium.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"InfoFrame consists of an InfoFrame class which is used as an extension\nof FrameBuffer, with extra information of pixel format, size, and memory\naddress if it's mmapped.\n\nInfoFramePool is introduced to handle InfoFrames more easily, which supports\na pool implementation of InfoFrames, creates frames by DmaBufAllocator, and\nmmap/munmap all frames of the pool.\n\nSigned-off-by: Han-Lin Chen \nCo-developed-by: Harvey Yang \nSigned-off-by: Harvey Yang \n---\n include/libcamera/internal/info_frame.h | 105 ++++++++\n include/libcamera/internal/meson.build | 1 +\n src/libcamera/info_frame.cpp | 302 ++++++++++++++++++++++++\n src/libcamera/meson.build | 1 +\n 4 files changed, 409 insertions(+)\n create mode 100644 include/libcamera/internal/info_frame.h\n create mode 100644 src/libcamera/info_frame.cpp","diff":"diff --git a/include/libcamera/internal/info_frame.h b/include/libcamera/internal/info_frame.h\nnew file mode 100644\nindex 000000000..29b445bb3\n--- /dev/null\n+++ b/include/libcamera/internal/info_frame.h\n@@ -0,0 +1,105 @@\n+/*\n+ * Copyright (C) 2023, Google Inc.\n+ *\n+ * info_frame.h - InfoFrame and InfoFramePool\n+ */\n+\n+#pragma once\n+\n+#include \n+#include \n+#include \n+#include \n+\n+#include \n+#include \n+#include \n+\n+#include \"libcamera/internal/dma_buf_allocator.h\"\n+#include \"libcamera/internal/mailbox.h\"\n+#include \"libcamera/internal/pool.h\"\n+\n+namespace libcamera {\n+\n+class InfoFrame\n+{\n+public:\n+\tstruct Plane {\n+\t\tuint8_t *address;\n+\t};\n+\n+\tInfoFrame();\n+\tInfoFrame(const PixelFormat &format, const Size &size, FrameBuffer *buffers,\n+\t\t unsigned int strideAlign = 1, unsigned int scanAlign = 1,\n+\t\t std::array planes = {});\n+\n+\tuint8_t *address(unsigned int plane) const;\n+\n+\tPixelFormat format() const { return format_; }\n+\tSize size() const { return size_; }\n+\tFrameBuffer *buffer() const { return buffer_; }\n+\tunsigned int numPlanes() const { return buffer_ ? buffer_->planes().size() : 0; }\n+\tunsigned int strideAlign() const { return strideAlign_; }\n+\tunsigned int scanAlign() const { return scanAlign_; }\n+\n+private:\n+\tSize size_;\n+\tPixelFormat format_;\n+\tFrameBuffer *buffer_ = nullptr;\n+\n+\tunsigned int strideAlign_ = 1;\n+\tunsigned int scanAlign_ = 1;\n+\n+\tstd::array planes_;\n+};\n+\n+class InfoFramePool\n+{\n+public:\n+\tstruct MappedBufferInfo {\n+\t\tuint8_t *address = nullptr;\n+\t\tsize_t dmabufLength = 0;\n+\t};\n+\n+\tInfoFramePool();\n+\t~InfoFramePool();\n+\n+\tint createBuffers(DmaBufAllocator *dmaHeap, const PixelFormat &format,\n+\t\t\t const Size &size, uint32_t count,\n+\t\t\t unsigned int strideAlign = 1, unsigned scanAlign = 1);\n+\n+\tvoid release();\n+\n+\tvoid fetch(SharedMailBox &mailBox);\n+\n+\tint mmap();\n+\tint munmap();\n+\n+\tbool mapped() const { return 0 != mappedBuffers_.size(); }\n+\n+\tsize_t size() { return pool_.size(); }\n+\tstd::vector> &content()\n+\t{\n+\t\treturn pool_.content();\n+\t}\n+\n+private:\n+\tLIBCAMERA_DISABLE_COPY_AND_MOVE(InfoFramePool)\n+\n+\tvoid setBuffers(const PixelFormat &format, const Size &size,\n+\t\t\tstd::vector> &buffers,\n+\t\t\tunsigned int align, unsigned int scanAlign);\n+\n+\tInfoFrame get();\n+\tvoid put(InfoFrame &frameInfo);\n+\n+\tSize size_;\n+\tPixelFormat format_;\n+\tPool> pool_;\n+\tunsigned int strideAlign_;\n+\tunsigned int scanAlign_;\n+\n+\tstd::unordered_map mappedBuffers_;\n+};\n+\n+} /* namespace libcamera */\ndiff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build\nindex b78e51d0b..8983bcd21 100644\n--- a/include/libcamera/internal/meson.build\n+++ b/include/libcamera/internal/meson.build\n@@ -21,6 +21,7 @@ libcamera_internal_headers = files([\n 'dma_buf_allocator.h',\n 'formats.h',\n 'framebuffer.h',\n+ 'info_frame.h',\n 'ipa_data_serializer.h',\n 'ipa_manager.h',\n 'ipa_module.h',\ndiff --git a/src/libcamera/info_frame.cpp b/src/libcamera/info_frame.cpp\nnew file mode 100644\nindex 000000000..5407fadd9\n--- /dev/null\n+++ b/src/libcamera/info_frame.cpp\n@@ -0,0 +1,302 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2023, Google Inc.\n+ *\n+ * info_frame.cpp - InfoFrame and InfoFramePool\n+ */\n+\n+#include \"libcamera/internal/info_frame.h\"\n+\n+#include \n+#include \n+\n+#include \"libcamera/internal/dma_buf_allocator.h\"\n+#include \"libcamera/internal/formats.h\"\n+#include \"libcamera/internal/pool.h\"\n+\n+namespace libcamera {\n+\n+LOG_DEFINE_CATEGORY(InfoFrame)\n+\n+/**\n+ * \\class InfoFrame\n+ * \\brief A wrapper class that consists of a FrameBuffer and extra information,\n+ * including PixelFormat, size, and memory address if it's mmap'ed.\n+ */\n+\n+/**\n+ * \\class InfoFrame::Plane\n+ * \\brief Per-plane frame mmap'ed address\n+ *\n+ * Frames are stored in memory in one or multiple planes. The\n+ * InfoFrame::Plane structure stores per-plane mmap'ed address.\n+ */\n+\n+/**\n+ * \\var InfoFrame::Plane::address\n+ * \\brief mmap'ed address of a plane\n+ */\n+\n+InfoFrame::InfoFrame() = default;\n+\n+/**\n+ * \\brief Create an InfoFrame instance\n+ * \\param[in] format PixelFormat of the frame\n+ * \\param[in] size Size of the frame\n+ * \\param[in] buffer Pointer to the buffer. InfoFrame doesn't take the ownership\n+ * \\param[in] strideAlign The stride alignment, in bytes (1 for default alignment)\n+ * \\param[in] scanAlign The scanline alignment, in bytes (1 for default alignment)\n+ * \\param[in] planes mmap'ed addresses of planes\n+ */\n+InfoFrame::InfoFrame(const PixelFormat &format, const Size &size,\n+\t\t FrameBuffer *buffer, unsigned int strideAlign,\n+\t\t unsigned int scanAlign, std::array planes)\n+\t: size_(size), format_(format), buffer_(buffer),\n+\t strideAlign_(strideAlign), scanAlign_(scanAlign),\n+\t planes_(planes)\n+\n+{\n+}\n+\n+/**\n+ * \\brief Get the mmap'ed address of \\a plane\n+ * \\param[in] plane The \\a plane'th plane\n+ * \\return Return the mmap'ed address of the plane\n+ */\n+uint8_t *InfoFrame::address(unsigned int plane) const\n+{\n+\treturn (plane < numPlanes()) ? planes_[plane].address : nullptr;\n+}\n+\n+/**\n+ * \\fn InfoFrame::format()\n+ * \\return Return the PixelFormat of the frame\n+ */\n+\n+/**\n+ * \\fn InfoFrame::size()\n+ * \\return Return the size of the frame\n+ */\n+\n+/**\n+ * \\fn InfoFrame::buffer()\n+ * \\return Return the pointer to the buffer\n+ */\n+\n+/**\n+ * \\fn InfoFrame::numPlanes()\n+ * \\return Return the number of planes of the buffer\n+ */\n+\n+/**\n+ * \\fn InfoFrame::strideAlign()\n+ * \\return Return the stride alignment, in bytes (1 for default alignment)\n+ */\n+\n+/**\n+ * \\fn InfoFrame::scanAlign()\n+ * \\return Return the scanline alignment, in bytes (1 for default alignment)\n+ */\n+\n+/**\n+ * \\class InfoFramePool\n+ * \\brief A buffer pool that allows users to allocate and request InfoFrame\n+ * buffers from DmaBufAllocator\n+ */\n+\n+/**\n+ * \\struct InfoFramePool::MappedBufferInfo\n+ * \\brief Contains the information of the mmap'ed buffers, including address\n+ * and length\n+ */\n+\n+/**\n+ * \\var InfoFramePool::MappedBufferInfo::address\n+ * \\brief mmap'ed address of a buffer\n+ */\n+\n+/**\n+ * \\var InfoFramePool::MappedBufferInfo::dmabufLength\n+ * \\brief Length of the DMA buffer\n+ */\n+\n+InfoFramePool::InfoFramePool() = default;\n+\n+InfoFramePool::~InfoFramePool()\n+{\n+\trelease();\n+}\n+\n+/**\n+ * \\brief Create DMA buffers\n+ * \\param[in] dmaHeap The DmaBufAllocator to allocate buffers from\n+ * \\param[in] format PixelFormat of each frame\n+ * \\param[in] size Size of each frame\n+ * \\param[in] count Number of frames to create\n+ * \\param[in] strideAlign The stride alignment, in bytes (1 for default alignment)\n+ * \\param[in] scanAlign The scanline alignment, in bytes (1 for default alignment)\n+ *\n+ * \\return 0 on success or a negative error code otherwise\n+ */\n+int InfoFramePool::createBuffers(DmaBufAllocator *dmaHeap,\n+\t\t\t\t const PixelFormat &format, const Size &size,\n+\t\t\t\t unsigned int count,\n+\t\t\t\t unsigned int strideAlign, unsigned scanAlign)\n+{\n+\trelease();\n+\n+\tconst PixelFormatInfo &info = PixelFormatInfo::info(format);\n+\tuint32_t frameSize = info.frameSize(size, strideAlign, scanAlign);\n+\n+\tstd::vector> buffers;\n+\tbuffers.reserve(count);\n+\tfor (unsigned int i = 0; i < count; i++) {\n+\t\tSharedFD fd(dmaHeap->alloc((\"frame-\" + std::to_string(i)).c_str(),\n+\t\t\t\t\t frameSize));\n+\t\tif (!fd.isValid()) {\n+\t\t\tbuffers.clear();\n+\t\t\treturn -EBUSY;\n+\t\t}\n+\n+\t\tuint32_t offset = 0;\n+\t\tstd::vector planes;\n+\n+\t\tfor (unsigned int j = 0; j < info.numPlanes(); j++) {\n+\t\t\tplanes.emplace_back(FrameBuffer::Plane{\n+\t\t\t\tfd, offset,\n+\t\t\t\tinfo.planeSize(size, j, strideAlign, scanAlign),\n+\t\t\t\tinfo.stride(size.width, j, strideAlign) });\n+\t\t\toffset += planes.back().length;\n+\t\t}\n+\n+\t\tbuffers.emplace_back(std::make_unique(planes));\n+\t}\n+\n+\tsetBuffers(format, size, buffers, strideAlign, scanAlign);\n+\n+\treturn 0;\n+}\n+\n+/**\n+ * \\brief Release all allocated buffers. InfoFramePool can create another set\n+ * of buffers again.\n+ */\n+void InfoFramePool::release()\n+{\n+\tif (munmap())\n+\t\tLOG(InfoFrame, Error) << \"Failed to unmap mapped buffers\";\n+\n+\tpool_.release();\n+}\n+\n+/**\n+ * \\brief Fetch an available InfoFrame from the pool\n+ * \\param[out] mailBox The shared pointer of MailBox to store the fetched\n+ * InfoFrame\n+ *\n+ * When \\a mailBox is reset, the recycler will return the InfoFrame back to the\n+ * InfoFramePool.\n+ */\n+void InfoFramePool::fetch(SharedMailBox &mailBox)\n+{\n+\tauto recycler = [this](InfoFrame &info) {\n+\t\tthis->put(info);\n+\t};\n+\n+\tmailBox->put(get(), recycler);\n+}\n+\n+/**\n+ * \\brief mmap all allocated buffers and set the addresses in `InfoFrame`s\n+ */\n+int InfoFramePool::mmap()\n+{\n+\tif (!mappedBuffers_.empty())\n+\t\treturn 0;\n+\n+\tfor (auto &buffer : pool_.content()) {\n+\t\tfor (const FrameBuffer::Plane &plane : buffer->planes()) {\n+\t\t\tconst int fd = plane.fd.get();\n+\t\t\tif (mappedBuffers_.find(fd) == mappedBuffers_.end()) {\n+\t\t\t\tconst size_t length = lseek(fd, 0, SEEK_END);\n+\t\t\t\tmappedBuffers_[fd] = MappedBufferInfo{ nullptr, length };\n+\t\t\t}\n+\t\t}\n+\t}\n+\n+\tfor (auto &[fd, info] : mappedBuffers_) {\n+\t\tvoid *address = ::mmap(nullptr, info.dmabufLength,\n+\t\t\t\t PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);\n+\t\tinfo.address = static_cast(address);\n+\t}\n+\treturn 0;\n+}\n+\n+/**\n+ * \\brief munmap all allocated buffers and reset addresses\n+ */\n+int InfoFramePool::munmap()\n+{\n+\tif (mappedBuffers_.empty())\n+\t\treturn 0;\n+\n+\tfor (auto &[_, info] : mappedBuffers_) {\n+\t\t::munmap(info.address, info.dmabufLength);\n+\t\tinfo.address = nullptr;\n+\t}\n+\n+\tmappedBuffers_.clear();\n+\treturn 0;\n+}\n+\n+/**\n+ * \\fn InfoFramePool::mapped()\n+ * \\return True if mmap() was called on the allocated buffers, false otherwise\n+ */\n+\n+/**\n+ * \\fn InfoFramePool::size()\n+ * \\return Return the number of `InfoFrame`s allocated\n+ */\n+\n+/**\n+ * \\fn InfoFramePool::content()\n+ * \\return Return frame buffers allocated in the pool\n+ */\n+\n+void InfoFramePool::setBuffers(const PixelFormat &format, const Size &size,\n+\t\t\t std::vector> &buffers,\n+\t\t\t unsigned int strideAlign, unsigned int scanAlign)\n+{\n+\tif (munmap())\n+\t\tLOG(InfoFrame, Error) << \"Failed to unmap buffers\";\n+\n+\tsize_ = size;\n+\tformat_ = format;\n+\tpool_.setData(buffers);\n+\tstrideAlign_ = strideAlign;\n+\tscanAlign_ = scanAlign;\n+}\n+\n+InfoFrame InfoFramePool::get()\n+{\n+\tFrameBuffer *buffer = pool_.get();\n+\n+\tstd::array planes;\n+\tfor (size_t i = 0; i < buffer->planes().size() && i < 3; i++) {\n+\t\tconst int fd = buffer->planes()[i].fd.get();\n+\t\tconst unsigned int offset = buffer->planes()[i].offset;\n+\n+\t\tif (mappedBuffers_.count(fd))\n+\t\t\tplanes[i].address = mappedBuffers_[fd].address + offset;\n+\t}\n+\n+\treturn InfoFrame(format_, size_, buffer, strideAlign_, scanAlign_, planes);\n+}\n+\n+void InfoFramePool::put(InfoFrame &info)\n+{\n+\tpool_.put(info.buffer());\n+}\n+\n+} /* namespace libcamera */\ndiff --git a/src/libcamera/meson.build b/src/libcamera/meson.build\nindex f4403687a..39483b458 100644\n--- a/src/libcamera/meson.build\n+++ b/src/libcamera/meson.build\n@@ -29,6 +29,7 @@ libcamera_internal_sources = files([\n 'device_enumerator_sysfs.cpp',\n 'dma_buf_allocator.cpp',\n 'formats.cpp',\n+ 'info_frame.cpp',\n 'ipa_controls.cpp',\n 'ipa_data_serializer.cpp',\n 'ipa_interface.cpp',\n","prefixes":["5/5"]}