{"id":14434,"url":"https://patchwork.libcamera.org/api/1.1/patches/14434/?format=json","web_url":"https://patchwork.libcamera.org/patch/14434/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20211101071652.107912-3-hiroh@chromium.org>","date":"2021-11-01T07:16:52","name":"[libcamera-devel,v2,2/2] android: Introduce PlatformFrameBufferAllocator","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"1343be9a0ca5ce219faec762b27c1fbe8f976505","submitter":{"id":63,"url":"https://patchwork.libcamera.org/api/1.1/people/63/?format=json","name":"Hirokazu Honda","email":"hiroh@chromium.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/14434/mbox/","series":[{"id":2683,"url":"https://patchwork.libcamera.org/api/1.1/series/2683/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2683","date":"2021-11-01T07:16:50","name":"Introduce PlatformFrameBufferAllocator","version":2,"mbox":"https://patchwork.libcamera.org/series/2683/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/14434/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/14434/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 75411C324F\n\tfor ;\n\tMon, 1 Nov 2021 07:17:05 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 2B793600C6;\n\tMon, 1 Nov 2021 08:17:05 +0100 (CET)","from mail-pf1-x429.google.com (mail-pf1-x429.google.com\n\t[IPv6:2607:f8b0:4864:20::429])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 9B0E1600B6\n\tfor ;\n\tMon, 1 Nov 2021 08:17:03 +0100 (CET)","by mail-pf1-x429.google.com with SMTP id 127so15571309pfu.1\n\tfor ;\n\tMon, 01 Nov 2021 00:17:03 -0700 (PDT)","from hiroh2.tok.corp.google.com\n\t([2401:fa00:8f:203:3b39:f865:c90:680a])\n\tby smtp.gmail.com with ESMTPSA id\n\tg14sm11910879pgo.88.2021.11.01.00.17.00\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tMon, 01 Nov 2021 00:17:01 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"ZDGho26j\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=from:to:cc:subject:date:message-id:in-reply-to:references\n\t:mime-version:content-transfer-encoding;\n\tbh=Yvdv62NB27u93vblDL2sZj3M1v5wcfeoqzAF6yqzeps=;\n\tb=ZDGho26jISbucNX7GzPj7GioeE8gdrp8edpz3Dv7VX67LujhAk0nclVyG1DRiufBk4\n\tfD+omyBc2EKWQqu4Qinr/SOJEF6XTSYhwAJaoMOPRyU+qf1yFsHjku2Tw9SMpq8D808I\n\tnsDHiJhGbGx0yNi46s5t3qEzdJsF+OeetyrAI=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references:mime-version:content-transfer-encoding;\n\tbh=Yvdv62NB27u93vblDL2sZj3M1v5wcfeoqzAF6yqzeps=;\n\tb=GUDY+0aMpD3V3ZpIBzVufPwqcacO2xaF91WrOGm3Y9jaJAWdYpIIDs1QBOncYnwbjl\n\tlwcddlYzI6jC+MtkXONZSdJUi+aQDgZ5DpTKi1OvwbyeskiybqnQJ5Hm2RDYxey1MB8z\n\tSOOYSrOJ6EnHDz004mbLa6zW8J1+dSVH4p1nN5BH12Zrcw6Kqa2CJ6Qx3RO0uJa1aWR1\n\tKyMvqr85KQIW0tIPTrSnMHjaGycKfZYiYvJp/MkVRagn3bdrEoGRYGh9iQLPrdP62jj4\n\tUfiVFe8SuRwI4bacD29C0OAgHqup9fkaz2AsxUaSIxl0seblU1oMJKwuF1xlvcEXtajo\n\twY3w==","X-Gm-Message-State":"AOAM533YEPx7wZft7HOyRPLuuqgy92lqK4/yJf3WcQeN2OBtJLf7xNta\n\ttu1OQ5luQMtldm+UOCwLDR9MYWaulzlhjg==","X-Google-Smtp-Source":"ABdhPJyNZMpLuAiWEyci6zD0L/Y8Ll3Ra3snK078RdvQTBg4Khjwf2q79QA76fEhbWHJwX2yCeMRRg==","X-Received":"by 2002:a63:ae08:: with SMTP id q8mr20435398pgf.76.1635751021458;\n\tMon, 01 Nov 2021 00:17:01 -0700 (PDT)","From":"Hirokazu Honda ","To":"libcamera-devel@lists.libcamera.org","Date":"Mon, 1 Nov 2021 16:16:52 +0900","Message-Id":"<20211101071652.107912-3-hiroh@chromium.org>","X-Mailer":"git-send-email 2.33.1.1089.g2158813163f-goog","In-Reply-To":"<20211101071652.107912-1-hiroh@chromium.org>","References":"<20211101071652.107912-1-hiroh@chromium.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v2 2/2] android: Introduce\n\tPlatformFrameBufferAllocator","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"After we unify identical streams to one stream configuration, the\ncapture requested by a HAL client can have been resolved as\nCameraStream::Mapped. That is, a buffer to be written by a camera\nis not provided by a client. We would handle this case by\ndynamically allocating FrameBuffer.\n\nThe existing FrameBufferAllocator cannot used because it is not\nallowed to allocate a new buffer while Camera is running.\n\nThis introduces PlatformFrameBufferAllocator. It allocates\nFrameBuffer using cros::CameraBufferManager on ChromeOS and\ngralloc on non ChromeOS platform. The allocated FrameBuffer owns\nthe underlying buffer but must be destroyed before\nPlatformFrameBufferAllocator is destroyed.\n\nSigned-off-by: Hirokazu Honda \n---\n src/android/frame_buffer_allocator.h | 54 +++++++\n .../mm/cros_frame_buffer_allocator.cpp | 90 +++++++++++\n .../mm/generic_frame_buffer_allocator.cpp | 144 ++++++++++++++++++\n src/android/mm/meson.build | 6 +-\n 4 files changed, 292 insertions(+), 2 deletions(-)\n create mode 100644 src/android/frame_buffer_allocator.h\n create mode 100644 src/android/mm/cros_frame_buffer_allocator.cpp\n create mode 100644 src/android/mm/generic_frame_buffer_allocator.cpp","diff":"diff --git a/src/android/frame_buffer_allocator.h b/src/android/frame_buffer_allocator.h\nnew file mode 100644\nindex 00000000..41ed4ae1\n--- /dev/null\n+++ b/src/android/frame_buffer_allocator.h\n@@ -0,0 +1,54 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2021, Google Inc.\n+ *\n+ * frame_buffer_allocator.h - Interface definition to allocate Frame buffer in\n+ * platform dependent way.\n+ */\n+#ifndef __ANDROID_FRAME_BUFFER_ALLOCATOR_H__\n+#define __ANDROID_FRAME_BUFFER_ALLOCATOR_H__\n+\n+#include \n+\n+#include \n+#include \n+#include \n+#include \n+\n+class CameraDevice;\n+\n+class PlatformFrameBufferAllocator : libcamera::Extensible\n+{\n+\tLIBCAMERA_DECLARE_PRIVATE()\n+\n+public:\n+\texplicit PlatformFrameBufferAllocator(CameraDevice *const cameraDevice);\n+\t~PlatformFrameBufferAllocator();\n+\n+\t/*\n+\t * FrameBuffer owns the underlying buffer. Returns nullptr on failure.\n+\t * Note: The returned FrameBuffer needs to be destroyed before\n+\t * PlatformFrameBufferAllocator is destroyed.\n+\t */\n+\tstd::unique_ptr allocate(\n+\t\tint halPixelFormat, const libcamera::Size &size, uint32_t usage);\n+};\n+\n+#define PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION\t\t\t\\\n+PlatformFrameBufferAllocator::PlatformFrameBufferAllocator(\t\t\\\n+\tCameraDevice *const cameraDevice)\t\t\t\t\\\n+\t: Extensible(std::make_unique(this, cameraDevice))\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+}\t\t\t\t\t\t\t\t\t\\\n+PlatformFrameBufferAllocator::~PlatformFrameBufferAllocator()\t\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+}\t\t\t\t\t\t\t\t\t\\\n+std::unique_ptr\t\t\t\t\t\\\n+PlatformFrameBufferAllocator::allocate(int halPixelFormat,\t\t\\\n+\t\t\t\t const libcamera::Size& size,\t\\\n+\t\t\t\t uint32_t usage)\t\t\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+\treturn _d()->allocate(halPixelFormat, size, usage);\t\t\\\n+}\n+\n+#endif /* __ANDROID_FRAME_BUFFER_ALLOCATOR_H__ */\ndiff --git a/src/android/mm/cros_frame_buffer_allocator.cpp b/src/android/mm/cros_frame_buffer_allocator.cpp\nnew file mode 100644\nindex 00000000..9c7e4ea4\n--- /dev/null\n+++ b/src/android/mm/cros_frame_buffer_allocator.cpp\n@@ -0,0 +1,90 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2021, Google Inc.\n+ *\n+ * cros_frame_buffer.cpp - Allocate FrameBuffer for Chromium OS using\n+ * CameraBufferManager\n+ */\n+\n+#include \n+#include \n+\n+#include \n+\n+#include \"libcamera/internal/framebuffer.h\"\n+\n+#include \"../camera_device.h\"\n+#include \"../frame_buffer_allocator.h\"\n+#include \"cros-camera/camera_buffer_manager.h\"\n+\n+using namespace libcamera;\n+\n+LOG_DECLARE_CATEGORY(HAL)\n+\n+namespace {\n+class CrosFrameBufferData : public FrameBuffer::Private\n+{\n+\tLIBCAMERA_DECLARE_PUBLIC(FrameBuffer)\n+public:\n+\tCrosFrameBufferData(cros::ScopedBufferHandle scopedHandle)\n+\t\t: FrameBuffer::Private(), scopedHandle_(std::move(scopedHandle))\n+\t{\n+\t}\n+\n+\t~CrosFrameBufferData() override = default;\n+\n+private:\n+\tcros::ScopedBufferHandle scopedHandle_;\n+};\n+} /* namespace */\n+\n+class PlatformFrameBufferAllocator::Private : public Extensible::Private\n+{\n+\tLIBCAMERA_DECLARE_PUBLIC(PlatformFrameBufferAllocator)\n+\n+public:\n+\tPrivate([[maybe_unused]] PlatformFrameBufferAllocator *allocator,\n+\t\t[[maybe_unused]] CameraDevice *const cameraDevice)\n+\t{\n+\t}\n+\n+\t~Private() override = default;\n+\n+\tstd::unique_ptr allocate(\n+\t\tint halPixelFormat, const libcamera::Size &size, uint32_t usage);\n+};\n+\n+std::unique_ptr\n+PlatformFrameBufferAllocator::Private::allocate(\n+\tint halPixelFormat, const libcamera::Size &size, uint32_t usage)\n+{\n+\tcros::ScopedBufferHandle scopedHandle =\n+\t\tcros::CameraBufferManager::AllocateScopedBuffer(\n+\t\t\tsize.width, size.height, halPixelFormat, usage);\n+\tif (!scopedHandle) {\n+\t\tLOG(HAL, Error) << \"Failed to allocate buffer handle\";\n+\t\treturn nullptr;\n+\t}\n+\n+\tbuffer_handle_t handle = *scopedHandle;\n+\tconst size_t numPlanes = cros::CameraBufferManager::GetNumPlanes(handle);\n+\tstd::vector planes(numPlanes);\n+\tFileDescriptor fd{ handle->data[0] };\n+\tif (!fd.isValid()) {\n+\t\tLOG(HAL, Fatal) << \"Invalid fd\";\n+\t\treturn nullptr;\n+\t}\n+\n+\t/* This code assumes all the planes are located in the same buffer. */\n+\tfor (auto [i, plane] : utils::enumerate(planes)) {\n+\t\tplane.fd = fd;\n+\t\tplane.offset = cros::CameraBufferManager::GetPlaneOffset(handle, i);\n+\t\tplane.length = cros::CameraBufferManager::GetPlaneSize(handle, i);\n+\t}\n+\n+\treturn std::make_unique(\n+\t\tstd::make_unique(std::move(scopedHandle)),\n+\t\tplanes);\n+}\n+\n+PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION\ndiff --git a/src/android/mm/generic_frame_buffer_allocator.cpp b/src/android/mm/generic_frame_buffer_allocator.cpp\nnew file mode 100644\nindex 00000000..167cc1a5\n--- /dev/null\n+++ b/src/android/mm/generic_frame_buffer_allocator.cpp\n@@ -0,0 +1,144 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2021, Google Inc.\n+ *\n+ * generic_camera_buffer.cpp - Allocate FrameBuffer using gralloc API\n+ */\n+\n+#include \n+#include \n+\n+#include \n+\n+#include \"libcamera/internal/formats.h\"\n+#include \"libcamera/internal/framebuffer.h\"\n+\n+#include \n+#include \n+#include \n+\n+#include \"../camera_device.h\"\n+#include \"../frame_buffer_allocator.h\"\n+\n+using namespace libcamera;\n+\n+LOG_DECLARE_CATEGORY(HAL)\n+\n+namespace {\n+class GenericFrameBufferData : public FrameBuffer::Private\n+{\n+\tLIBCAMERA_DECLARE_PUBLIC(FrameBuffer)\n+\n+public:\n+\tGenericFrameBufferData(struct alloc_device_t *allocDevice,\n+\t\t\t buffer_handle_t handle)\n+\t\t: allocDevice_(allocDevice), handle_(handle)\n+\t{\n+\t\tASSERT(allocDevice_);\n+\t\tASSERT(handle_);\n+\t}\n+\n+\t~GenericFrameBufferData() override\n+\t{\n+\t\t/*\n+\t\t * allocDevice_ is used to destroy handle_. allocDevice_ is\n+\t\t * owned by PlatformFrameBufferAllocator::Private.\n+\t\t * GenericFrameBufferData must be destroyed before it is\n+\t\t * destroyed.\n+\t\t *\n+\t\t * \\todo: Consider managing alloc_device_t with std::shared_ptr\n+\t\t * if this is difficult to maintain.\n+\t\t *\n+\t\t * Q: Thread safety against alloc_device_t is not documented.\n+\t\t * Is it no problem to call alloc/free in parallel?\n+\t\t */\n+\t\tASSERT(allocDevice_);\n+\t\tallocDevice_->free(allocDevice_, handle_);\n+\t}\n+\n+private:\n+\tstruct alloc_device_t *allocDevice_;\n+\tconst buffer_handle_t handle_;\n+};\n+} /* namespace */\n+\n+class PlatformFrameBufferAllocator::Private : public Extensible::Private\n+{\n+\tLIBCAMERA_DECLARE_PUBLIC(PlatformFrameBufferAllocator)\n+\n+public:\n+\tPrivate([[maybe_unused]] PlatformFrameBufferAllocator *allocator,\n+\t\tCameraDevice *const cameraDevice)\n+\t\t: cameraDevice_(cameraDevice),\n+\t\t hardwareModule_(cameraDevice->camera3Device()->common.module),\n+\t\t allocDevice_(nullptr)\n+\t{\n+\t\tASSERT(hardwareModule_);\n+\t}\n+\n+\t~Private() override;\n+\n+\tstd::unique_ptr allocate(\n+\t\tint halPixelFormat, const libcamera::Size &size, uint32_t usage);\n+\n+private:\n+\tconst CameraDevice *const cameraDevice_;\n+\tstruct hw_module_t *const hardwareModule_;\n+\tstruct alloc_device_t *allocDevice_;\n+};\n+\n+PlatformFrameBufferAllocator::Private::~Private()\n+{\n+\tif (allocDevice_)\n+\t\tgralloc_close(allocDevice_);\n+}\n+\n+std::unique_ptr\n+PlatformFrameBufferAllocator::Private::allocate(\n+\tint halPixelFormat, const libcamera::Size &size, uint32_t usage)\n+{\n+\tif (!allocDevice_) {\n+\t\tint ret = gralloc_open(hardwareModule_, &allocDevice_);\n+\t\tif (ret) {\n+\t\t\tLOG(HAL, Fatal) << \"gralloc_open() failed: \" << ret;\n+\t\t\treturn nullptr;\n+\t\t}\n+\t}\n+\n+\tint stride = 0;\n+\tbuffer_handle_t handle = nullptr;\n+\tint ret = allocDevice_->alloc(allocDevice_, size.width, size.height,\n+\t\t\t\t halPixelFormat, usage, &handle, &stride);\n+\tif (ret) {\n+\t\tLOG(HAL, Error) << \"failed buffer allocating: \" << ret;\n+\t\treturn nullptr;\n+\t}\n+\tif (!handle) {\n+\t\tLOG(HAL, Fatal) << \"buffer_handle_t is empty on success\";\n+\t\treturn nullptr;\n+\t}\n+\n+\tconst libcamera::PixelFormat pixelFormat =\n+\t\tcameraDevice_->capabilities()->toPixelFormat(halPixelFormat);\n+\tconst auto &info = PixelFormatInfo::info(pixelFormat);\n+\tconst unsigned int numPlanes = info.numPlanes();\n+\tstd::vector planes(numPlanes);\n+\n+\t/* This code assumes the planes are mapped consecutively. */\n+\tFileDescriptor fd{ handle->data[0] };\n+\tsize_t offset = 0;\n+\tfor (auto [i, plane] : utils::enumerate(planes)) {\n+\t\tconst size_t planeSize = info.planeSize(size.height, i, stride);\n+\n+\t\tplanes[i].fd = fd;\n+\t\tplanes[i].offset = offset;\n+\t\tplanes[i].length = planeSize;\n+\t\toffset += planeSize;\n+\t}\n+\n+\treturn std::make_unique(\n+\t\tstd::make_unique(allocDevice_, handle),\n+\t\tplanes);\n+}\n+\n+PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION\ndiff --git a/src/android/mm/meson.build b/src/android/mm/meson.build\nindex eeb5cc2e..d40a3b0b 100644\n--- a/src/android/mm/meson.build\n+++ b/src/android/mm/meson.build\n@@ -2,8 +2,10 @@\n \n platform = get_option('android_platform')\n if platform == 'generic'\n- android_hal_sources += files(['generic_camera_buffer.cpp'])\n+ android_hal_sources += files(['generic_camera_buffer.cpp',\n+ 'generic_frame_buffer_allocator.cpp'])\n elif platform == 'cros'\n- android_hal_sources += files(['cros_camera_buffer.cpp'])\n+ android_hal_sources += files(['cros_camera_buffer.cpp',\n+ 'cros_frame_buffer_allocator.cpp'])\n android_deps += [dependency('libcros_camera')]\n endif\n","prefixes":["libcamera-devel","v2","2/2"]}