{"id":14726,"url":"https://patchwork.libcamera.org/api/patches/14726/?format=json","web_url":"https://patchwork.libcamera.org/patch/14726/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20211123183947.46839-2-hiroh@chromium.org>","date":"2021-11-23T18:39:47","name":"[libcamera-devel,v3,2/2] android: Introduce PlatformFrameBufferAllocator","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"03b1aece1df2a497825010b3116bedbeb556684f","submitter":{"id":63,"url":"https://patchwork.libcamera.org/api/people/63/?format=json","name":"Hirokazu Honda","email":"hiroh@chromium.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/14726/mbox/","series":[{"id":2747,"url":"https://patchwork.libcamera.org/api/series/2747/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2747","date":"2021-11-23T18:39:46","name":"[libcamera-devel,v3,1/2] libcamera: framebuffer: Enable attaching additional data to FrameBuffer","version":3,"mbox":"https://patchwork.libcamera.org/series/2747/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/14726/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/14726/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id BC5E5BDB13\n\tfor ;\n\tTue, 23 Nov 2021 18:40:00 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 74B1360230;\n\tTue, 23 Nov 2021 19:40:00 +0100 (CET)","from mail-pl1-x636.google.com (mail-pl1-x636.google.com\n\t[IPv6:2607:f8b0:4864:20::636])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7B66E6022F\n\tfor ;\n\tTue, 23 Nov 2021 19:39:57 +0100 (CET)","by mail-pl1-x636.google.com with SMTP id u17so17811486plg.9\n\tfor ;\n\tTue, 23 Nov 2021 10:39:57 -0800 (PST)","from hiroh2.tok.corp.google.com\n\t([2401:fa00:8f:203:cc8f:f5a6:5a9f:7275])\n\tby smtp.gmail.com with ESMTPSA id 66sm5630pgg.63.2021.11.23.10.39.53\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tTue, 23 Nov 2021 10:39:55 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"DfonlJif\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=from:to:cc:subject:date:message-id:in-reply-to:references\n\t:mime-version:content-transfer-encoding;\n\tbh=d34fQ0jqK6GDcCYBmE0lkS7gcEmpbF0JjC1M9c4/6FY=;\n\tb=DfonlJifCM24CNrVuQdM/0r1q2Wo1/o7U+dmKyeITsrYSaE/LQZ9wUrNEJGCQZoVOL\n\tMDSJ/HUbQ6QVcAxPQWngpQPnHuEk8pn48dvXmFRjv6lpNbp0fIyrYROEaHbAtPB5vDpW\n\twScotaS36BtEU+qNTgsnAQNAn6fOnFqJhR258=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references:mime-version:content-transfer-encoding;\n\tbh=d34fQ0jqK6GDcCYBmE0lkS7gcEmpbF0JjC1M9c4/6FY=;\n\tb=UmsShZO9N+AeUPaM0Hi8Zb31b2cZHzINyDCmXXuU+UxamivSU0bmS96XIORgyT037B\n\tT1VlQatEwtVl1CNkCNyuWFzZLaWlVXyKhU8YI4D3aJx7IhxjsGoTCHhVwwYolER/X8HZ\n\tRF03qEa/D/Ln9iLtkwgL6H+nrj/X8nUkbVgN3zXHjQ++NWpziy6EHVApDkP6OTvo8JS9\n\tKeRE2yWeE62kOJI7etX/EUGENKp7EYXJpk0R4pmLwKzEb/YYL/JUpgI/PMX3qnPvIpDi\n\tB5CLiZ1oSC1ilqa9fnqvUlA8/LCZDx0gChnoxj8lXS34mC93JuZSXakdDnVjeWP5SSC8\n\tzaWQ==","X-Gm-Message-State":"AOAM533bZKf7RPsiZ1KqYXN+Di4roQE7/TpExIGZ3IeFlbsu09KZuPge\n\tMGuti+lixcPUO8/aEx42RStkP8JUxI2wvA==","X-Google-Smtp-Source":"ABdhPJwMk/fUME1lrW9AVf6z3XXOkfIXdjTlgFibwVki10ck8PISmXvOMdPEW5xJ4CCp09TnZswR8Q==","X-Received":"by 2002:a17:90a:be0f:: with SMTP id\n\ta15mr5501047pjs.243.1637692795506; \n\tTue, 23 Nov 2021 10:39:55 -0800 (PST)","From":"Hirokazu Honda ","To":"libcamera-devel@lists.libcamera.org","Date":"Wed, 24 Nov 2021 03:39:47 +0900","Message-Id":"<20211123183947.46839-2-hiroh@chromium.org>","X-Mailer":"git-send-email 2.34.0.rc2.393.gf8c9666880-goog","In-Reply-To":"<20211123183947.46839-1-hiroh@chromium.org>","References":"<20211123183947.46839-1-hiroh@chromium.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v3 2/2] android: Introduce\n\tPlatformFrameBufferAllocator","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"The existing FrameBufferAllocator is not allowed to allocate a\nnew buffer while Camera is running. This introduces\nPlatformFrameBufferAllocator. It allocates FrameBuffer using\ncros::CameraBufferManager on ChromeOS and gralloc on non ChromeOS\nplatform. The allocated FrameBuffer owns the underlying buffer\nbut must be destroyed before PlatformFrameBufferAllocator is\ndestroyed.\n\nSigned-off-by: Hirokazu Honda \nReviewed-by: Laurent Pinchart \n---\n src/android/frame_buffer_allocator.h | 54 +++++++\n .../mm/cros_frame_buffer_allocator.cpp | 88 +++++++++++\n .../mm/generic_frame_buffer_allocator.cpp | 142 ++++++++++++++++++\n src/android/mm/meson.build | 6 +-\n 4 files changed, 288 insertions(+), 2 deletions(-)\n create mode 100644 src/android/frame_buffer_allocator.h\n create mode 100644 src/android/mm/cros_frame_buffer_allocator.cpp\n create mode 100644 src/android/mm/generic_frame_buffer_allocator.cpp","diff":"diff --git a/src/android/frame_buffer_allocator.h b/src/android/frame_buffer_allocator.h\nnew file mode 100644\nindex 00000000..41ed4ae1\n--- /dev/null\n+++ b/src/android/frame_buffer_allocator.h\n@@ -0,0 +1,54 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2021, Google Inc.\n+ *\n+ * frame_buffer_allocator.h - Interface definition to allocate Frame buffer in\n+ * platform dependent way.\n+ */\n+#ifndef __ANDROID_FRAME_BUFFER_ALLOCATOR_H__\n+#define __ANDROID_FRAME_BUFFER_ALLOCATOR_H__\n+\n+#include \n+\n+#include \n+#include \n+#include \n+#include \n+\n+class CameraDevice;\n+\n+class PlatformFrameBufferAllocator : libcamera::Extensible\n+{\n+\tLIBCAMERA_DECLARE_PRIVATE()\n+\n+public:\n+\texplicit PlatformFrameBufferAllocator(CameraDevice *const cameraDevice);\n+\t~PlatformFrameBufferAllocator();\n+\n+\t/*\n+\t * FrameBuffer owns the underlying buffer. Returns nullptr on failure.\n+\t * Note: The returned FrameBuffer needs to be destroyed before\n+\t * PlatformFrameBufferAllocator is destroyed.\n+\t */\n+\tstd::unique_ptr allocate(\n+\t\tint halPixelFormat, const libcamera::Size &size, uint32_t usage);\n+};\n+\n+#define PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION\t\t\t\\\n+PlatformFrameBufferAllocator::PlatformFrameBufferAllocator(\t\t\\\n+\tCameraDevice *const cameraDevice)\t\t\t\t\\\n+\t: Extensible(std::make_unique(this, cameraDevice))\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+}\t\t\t\t\t\t\t\t\t\\\n+PlatformFrameBufferAllocator::~PlatformFrameBufferAllocator()\t\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+}\t\t\t\t\t\t\t\t\t\\\n+std::unique_ptr\t\t\t\t\t\\\n+PlatformFrameBufferAllocator::allocate(int halPixelFormat,\t\t\\\n+\t\t\t\t const libcamera::Size& size,\t\\\n+\t\t\t\t uint32_t usage)\t\t\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+\treturn _d()->allocate(halPixelFormat, size, usage);\t\t\\\n+}\n+\n+#endif /* __ANDROID_FRAME_BUFFER_ALLOCATOR_H__ */\ndiff --git a/src/android/mm/cros_frame_buffer_allocator.cpp b/src/android/mm/cros_frame_buffer_allocator.cpp\nnew file mode 100644\nindex 00000000..d6343e53\n--- /dev/null\n+++ b/src/android/mm/cros_frame_buffer_allocator.cpp\n@@ -0,0 +1,88 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2021, Google Inc.\n+ *\n+ * cros_frame_buffer.cpp - Allocate FrameBuffer for Chromium OS using\n+ * CameraBufferManager\n+ */\n+\n+#include \n+#include \n+\n+#include \n+\n+#include \"libcamera/internal/framebuffer.h\"\n+\n+#include \"../camera_device.h\"\n+#include \"../frame_buffer_allocator.h\"\n+#include \"cros-camera/camera_buffer_manager.h\"\n+\n+using namespace libcamera;\n+\n+LOG_DECLARE_CATEGORY(HAL)\n+\n+namespace {\n+class CrosFrameBufferData : public FrameBuffer::Private\n+{\n+\tLIBCAMERA_DECLARE_PUBLIC(FrameBuffer)\n+public:\n+\tCrosFrameBufferData(cros::ScopedBufferHandle scopedHandle)\n+\t\t: FrameBuffer::Private(), scopedHandle_(std::move(scopedHandle))\n+\t{\n+\t}\n+\n+\t~CrosFrameBufferData() override = default;\n+\n+private:\n+\tcros::ScopedBufferHandle scopedHandle_;\n+};\n+} /* namespace */\n+\n+class PlatformFrameBufferAllocator::Private : public Extensible::Private\n+{\n+\tLIBCAMERA_DECLARE_PUBLIC(PlatformFrameBufferAllocator)\n+\n+public:\n+\tPrivate([[maybe_unused]] PlatformFrameBufferAllocator *allocator,\n+\t\t[[maybe_unused]] CameraDevice *const cameraDevice)\n+\t{\n+\t}\n+\n+\tstd::unique_ptr allocate(\n+\t\tint halPixelFormat, const libcamera::Size &size, uint32_t usage);\n+};\n+\n+std::unique_ptr\n+PlatformFrameBufferAllocator::Private::allocate(\n+\tint halPixelFormat, const libcamera::Size &size, uint32_t usage)\n+{\n+\tcros::ScopedBufferHandle scopedHandle =\n+\t\tcros::CameraBufferManager::AllocateScopedBuffer(\n+\t\t\tsize.width, size.height, halPixelFormat, usage);\n+\tif (!scopedHandle) {\n+\t\tLOG(HAL, Error) << \"Failed to allocate buffer handle\";\n+\t\treturn nullptr;\n+\t}\n+\n+\tbuffer_handle_t handle = *scopedHandle;\n+\tconst size_t numPlanes = cros::CameraBufferManager::GetNumPlanes(handle);\n+\tstd::vector planes(numPlanes);\n+\tFileDescriptor fd{ handle->data[0] };\n+\tif (!fd.isValid()) {\n+\t\tLOG(HAL, Fatal) << \"Invalid fd\";\n+\t\treturn nullptr;\n+\t}\n+\n+\t/* This code assumes all the planes are located in the same buffer. */\n+\tfor (auto [i, plane] : utils::enumerate(planes)) {\n+\t\tplane.fd = fd;\n+\t\tplane.offset = cros::CameraBufferManager::GetPlaneOffset(handle, i);\n+\t\tplane.length = cros::CameraBufferManager::GetPlaneSize(handle, i);\n+\t}\n+\n+\treturn std::make_unique(\n+\t\tstd::make_unique(std::move(scopedHandle)),\n+\t\tplanes);\n+}\n+\n+PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION\ndiff --git a/src/android/mm/generic_frame_buffer_allocator.cpp b/src/android/mm/generic_frame_buffer_allocator.cpp\nnew file mode 100644\nindex 00000000..f00fd704\n--- /dev/null\n+++ b/src/android/mm/generic_frame_buffer_allocator.cpp\n@@ -0,0 +1,142 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2021, Google Inc.\n+ *\n+ * generic_camera_buffer.cpp - Allocate FrameBuffer using gralloc API\n+ */\n+\n+#include \n+#include \n+\n+#include \n+\n+#include \"libcamera/internal/formats.h\"\n+#include \"libcamera/internal/framebuffer.h\"\n+\n+#include \n+#include \n+#include \n+\n+#include \"../camera_device.h\"\n+#include \"../frame_buffer_allocator.h\"\n+\n+using namespace libcamera;\n+\n+LOG_DECLARE_CATEGORY(HAL)\n+\n+namespace {\n+class GenericFrameBufferData : public FrameBuffer::Private\n+{\n+\tLIBCAMERA_DECLARE_PUBLIC(FrameBuffer)\n+\n+public:\n+\tGenericFrameBufferData(struct alloc_device_t *allocDevice,\n+\t\t\t buffer_handle_t handle)\n+\t\t: allocDevice_(allocDevice), handle_(handle)\n+\t{\n+\t\tASSERT(allocDevice_);\n+\t\tASSERT(handle_);\n+\t}\n+\n+\t~GenericFrameBufferData() override\n+\t{\n+\t\t/*\n+\t\t * allocDevice_ is used to destroy handle_. allocDevice_ is\n+\t\t * owned by PlatformFrameBufferAllocator::Private.\n+\t\t * GenericFrameBufferData must be destroyed before it is\n+\t\t * destroyed.\n+\t\t *\n+\t\t * \\todo Consider managing alloc_device_t with std::shared_ptr\n+\t\t * if this is difficult to maintain.\n+\t\t *\n+\t\t * \\todo Thread safety against alloc_device_t is not documented.\n+\t\t * Is it no problem to call alloc/free in parallel?\n+\t\t */\n+\t\tallocDevice_->free(allocDevice_, handle_);\n+\t}\n+\n+private:\n+\tstruct alloc_device_t *allocDevice_;\n+\tconst buffer_handle_t handle_;\n+};\n+} /* namespace */\n+\n+class PlatformFrameBufferAllocator::Private : public Extensible::Private\n+{\n+\tLIBCAMERA_DECLARE_PUBLIC(PlatformFrameBufferAllocator)\n+\n+public:\n+\tPrivate([[maybe_unused]] PlatformFrameBufferAllocator *allocator,\n+\t\tCameraDevice *const cameraDevice)\n+\t\t: cameraDevice_(cameraDevice),\n+\t\t hardwareModule_(cameraDevice->camera3Device()->common.module),\n+\t\t allocDevice_(nullptr)\n+\t{\n+\t\tASSERT(hardwareModule_);\n+\t}\n+\n+\t~Private() override;\n+\n+\tstd::unique_ptr allocate(\n+\t\tint halPixelFormat, const libcamera::Size &size, uint32_t usage);\n+\n+private:\n+\tconst CameraDevice *const cameraDevice_;\n+\tstruct hw_module_t *const hardwareModule_;\n+\tstruct alloc_device_t *allocDevice_;\n+};\n+\n+PlatformFrameBufferAllocator::Private::~Private()\n+{\n+\tif (allocDevice_)\n+\t\tgralloc_close(allocDevice_);\n+}\n+\n+std::unique_ptr\n+PlatformFrameBufferAllocator::Private::allocate(\n+\tint halPixelFormat, const libcamera::Size &size, uint32_t usage)\n+{\n+\tif (!allocDevice_) {\n+\t\tint ret = gralloc_open(hardwareModule_, &allocDevice_);\n+\t\tif (ret) {\n+\t\t\tLOG(HAL, Fatal) << \"gralloc_open() failed: \" << ret;\n+\t\t\treturn nullptr;\n+\t\t}\n+\t}\n+\n+\tint stride = 0;\n+\tbuffer_handle_t handle = nullptr;\n+\tint ret = allocDevice_->alloc(allocDevice_, size.width, size.height,\n+\t\t\t\t halPixelFormat, usage, &handle, &stride);\n+\tif (ret) {\n+\t\tLOG(HAL, Error) << \"failed buffer allocating: \" << ret;\n+\t\treturn nullptr;\n+\t}\n+\tif (!handle) {\n+\t\tLOG(HAL, Fatal) << \"buffer_handle_t is empty on success\";\n+\t\treturn nullptr;\n+\t}\n+\n+\tconst libcamera::PixelFormat pixelFormat =\n+\t\tcameraDevice_->capabilities()->toPixelFormat(halPixelFormat);\n+\tconst auto &info = PixelFormatInfo::info(pixelFormat);\n+\tstd::vector planes(info.numPlanes());\n+\n+\t/* This code assumes the planes are mapped consecutively. */\n+\tFileDescriptor fd{ handle->data[0] };\n+\tsize_t offset = 0;\n+\tfor (auto [i, plane] : utils::enumerate(planes)) {\n+\t\tconst size_t planeSize = info.planeSize(size.height, i, stride);\n+\n+\t\tplanes[i].fd = fd;\n+\t\tplanes[i].offset = offset;\n+\t\tplanes[i].length = planeSize;\n+\t\toffset += planeSize;\n+\t}\n+\n+\treturn std::make_unique(\n+\t\tstd::make_unique(allocDevice_, handle),\n+\t\tplanes);\n+}\n+\n+PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION\ndiff --git a/src/android/mm/meson.build b/src/android/mm/meson.build\nindex eeb5cc2e..d40a3b0b 100644\n--- a/src/android/mm/meson.build\n+++ b/src/android/mm/meson.build\n@@ -2,8 +2,10 @@\n \n platform = get_option('android_platform')\n if platform == 'generic'\n- android_hal_sources += files(['generic_camera_buffer.cpp'])\n+ android_hal_sources += files(['generic_camera_buffer.cpp',\n+ 'generic_frame_buffer_allocator.cpp'])\n elif platform == 'cros'\n- android_hal_sources += files(['cros_camera_buffer.cpp'])\n+ android_hal_sources += files(['cros_camera_buffer.cpp',\n+ 'cros_frame_buffer_allocator.cpp'])\n android_deps += [dependency('libcros_camera')]\n endif\n","prefixes":["libcamera-devel","v3","2/2"]}