{"id":13945,"url":"https://patchwork.libcamera.org/api/1.1/patches/13945/?format=json","web_url":"https://patchwork.libcamera.org/patch/13945/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20210927104821.2526508-3-hiroh@chromium.org>","date":"2021-09-27T10:48:21","name":"[libcamera-devel,RFC,2/2] android: Send alternative stream configuration if no buffer to be sent exists","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"b11624d78b5fe7e8e68128701513c665a290c1a2","submitter":{"id":63,"url":"https://patchwork.libcamera.org/api/1.1/people/63/?format=json","name":"Hirokazu Honda","email":"hiroh@chromium.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/13945/mbox/","series":[{"id":2572,"url":"https://patchwork.libcamera.org/api/1.1/series/2572/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2572","date":"2021-09-27T10:48:19","name":"android: Fix the issue with Type::Mapped only request","version":1,"mbox":"https://patchwork.libcamera.org/series/2572/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/13945/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/13945/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 1163CC3243\n\tfor ;\n\tMon, 27 Sep 2021 10:48:35 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id CE93969196;\n\tMon, 27 Sep 2021 12:48:34 +0200 (CEST)","from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com\n\t[IPv6:2607:f8b0:4864:20::62d])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 221A46012C\n\tfor ;\n\tMon, 27 Sep 2021 12:48:33 +0200 (CEST)","by mail-pl1-x62d.google.com with SMTP id j15so10059946plh.7\n\tfor ;\n\tMon, 27 Sep 2021 03:48:33 -0700 (PDT)","from hiroh2.tok.corp.google.com\n\t([2401:fa00:8f:203:7cac:9967:6683:8845])\n\tby smtp.gmail.com with ESMTPSA id\n\tb11sm18694452pge.57.2021.09.27.03.48.29\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tMon, 27 Sep 2021 03:48:30 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"WHtttgmc\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=from:to:cc:subject:date:message-id:in-reply-to:references\n\t:mime-version:content-transfer-encoding;\n\tbh=9YQvZI94UmReHrH9Fuefm9vag2IyDcleBNQfU6/yotA=;\n\tb=WHtttgmcOVy0rfmMi7zrVMZt9iOkzUC1uvHnVCvO72sXJ8RrjlsrBlv6SpovUy6NKi\n\t5s3LRIuQU1woDTUlvl3ntv9qBL71bDwhFt5GK0JJXw6g6FGUOR9uWDXbQEacYT0xw0nW\n\truFWSq0UQJdxZZtu9HKETZbaXfCnOJWoGDPdA=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references:mime-version:content-transfer-encoding;\n\tbh=9YQvZI94UmReHrH9Fuefm9vag2IyDcleBNQfU6/yotA=;\n\tb=7Qt2IQMVlC0wnGEnvkGsjZ4n6JJM12tyc2ctvI2gdwxGf04YFrWrO1Cn7GRvvsPtG4\n\txt4YSgf7l4MWFOCXnwEo4jmN28niHHixaErvFSzrxbOxmZLVsdZhYwKxVdp97CKk1SDD\n\thXnQk/U+9P+OAzXQ69iRoVAgufAHetFtBuLtkv33iO01NsqbAGmeOtHfopRNlCFM06Jv\n\tRP4AWn0fmBMcEVKPtZ2CPOGehfJIe45z5S9ZrdOJZRINcITtOXwcX6ehoN1D1wrk+mJ+\n\ti8VVfenOeaosKUlVFERuRJU0D7wg6A/c/PMxpkzZMreoeUVsTAddwRbBLuom44Ftnd9O\n\tQX3g==","X-Gm-Message-State":"AOAM533LKuTmE2utDq8jECGsJlpqEbJJzxDADDJ6BWPfhiCUUn2oM5ve\n\tzNvBFuo3fLX67wr5QnyHIsi84vfllFEXuw==","X-Google-Smtp-Source":"ABdhPJxyiVe9vcRrV3Ae/Sm3ql22Jagk/8fSW2hhtWmM71UQbP2fpt9RT6QvCIri9V6dHo6hK2rgVg==","X-Received":"by 2002:a17:90b:4f8a:: with SMTP id\n\tqe10mr19113196pjb.5.1632739711326; \n\tMon, 27 Sep 2021 03:48:31 -0700 (PDT)","From":"Hirokazu Honda ","To":"libcamera-devel@lists.libcamera.org","Date":"Mon, 27 Sep 2021 19:48:21 +0900","Message-Id":"<20210927104821.2526508-3-hiroh@chromium.org>","X-Mailer":"git-send-email 2.33.0.685.g46640cef36-goog","In-Reply-To":"<20210927104821.2526508-1-hiroh@chromium.org>","References":"<20210927104821.2526508-1-hiroh@chromium.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [RFC PATCH 2/2] android: Send alternative stream\n\tconfiguration if no buffer to be sent exists","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"A request given in processCaptureRequest() can contain only streams\nthat are resolved CameraStream::Type::Mapped. In the case, no buffer\nis sent to a native camera. This fixes the issue by looking for the\nstream to be requested and alternatively sending a buffer for the\nstream to a camera.\n\nHowever, the substitute stream is Direct and a buffer needs to be\nprovided by a HAL client. So we have to allocate a buffer in\nAndroid HAL adaptation layer using PlatformFrameBufferAllocator.\n\nSigned-off-by: Hirokazu Honda \n---\n src/android/camera_device.cpp | 50 +++++++++++++++++++++++++++++------\n src/android/camera_device.h | 4 +++\n src/android/camera_stream.cpp | 37 ++++++++++++++++++++++----\n src/android/camera_stream.h | 10 +++++--\n 4 files changed, 86 insertions(+), 15 deletions(-)","diff":"diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\nindex a693dcbe..5887e85e 100644\n--- a/src/android/camera_device.cpp\n+++ b/src/android/camera_device.cpp\n@@ -724,10 +724,14 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)\n \tfor (const auto &streamConfig : streamConfigs) {\n \t\tconfig->addConfiguration(streamConfig.config);\n \n+\t\tCameraStream *mappedStream = nullptr;\n \t\tfor (auto &stream : streamConfig.streams) {\n \t\t\tstreams_.emplace_back(this, config.get(), stream.type,\n-\t\t\t\t\t stream.stream, config->size() - 1);\n+\t\t\t\t\t stream.stream, mappedStream,\n+\t\t\t\t\t config->size() - 1);\n \t\t\tstream.stream->priv = static_cast(&streams_.back());\n+\t\t\tif (stream.type == CameraStream::Type::Direct)\n+\t\t\t\tmappedStream = &streams_.back();\n \t\t}\n \t}\n \n@@ -969,6 +973,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n \n \tLOG(HAL, Debug) << \"Queueing request \" << descriptor.request_->cookie()\n \t\t\t<< \" with \" << descriptor.buffers_.size() << \" streams\";\n+\n+\tstd::set addedStreams;\n \tfor (unsigned int i = 0; i < descriptor.buffers_.size(); ++i) {\n \t\tconst camera3_stream_buffer_t &camera3Buffer = descriptor.buffers_[i];\n \t\tcamera3_stream *camera3Stream = camera3Buffer.stream;\n@@ -1018,6 +1024,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n \t\t\t * once it has been processed.\n \t\t\t */\n \t\t\tbuffer = cameraStream->getBuffer();\n+\t\t\tdescriptor.internalBuffers_[cameraStream] = buffer;\n \t\t\tLOG(HAL, Debug) << ss.str() << \" (internal)\";\n \t\t\tbreak;\n \t\t}\n@@ -1029,6 +1036,37 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n \n \t\tdescriptor.request_->addBuffer(cameraStream->stream(), buffer,\n \t\t\t\t\t\tcamera3Buffer.acquire_fence);\n+\t\taddedStreams.insert(cameraStream->stream());\n+\t}\n+\n+\tfor (unsigned int i = 0; i < descriptor.buffers_.size(); ++i) {\n+\t\tconst camera3_stream_buffer_t &camera3Buffer = descriptor.buffers_[i];\n+\t\tcamera3_stream *camera3Stream = camera3Buffer.stream;\n+\t\tCameraStream *cameraStream = static_cast(camera3Stream->priv);\n+\t\tif (cameraStream->type() != CameraStream::Type::Mapped)\n+\t\t\tcontinue;\n+\t\tif (addedStreams.find(cameraStream->stream()) != addedStreams.end())\n+\t\t\tcontinue;\n+\n+\t\tCameraStream *mappedStream = cameraStream->mappedStream();\n+\t\tif (!mappedStream) {\n+\t\t\tLOG(HAL, Error)\n+\t\t\t\t<< \"Could not find mappedStream for (\"\n+\t\t\t\t<< camera3Stream->width << \"x\"\n+\t\t\t\t<< camera3Stream->height << \")\"\n+\t\t\t\t<< \"[\" << utils::hex(camera3Stream->format) << \"]\";\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\n+\t\tASSERT(mappedStream->type() == CameraStream::Type::Direct);\n+\t\tFrameBuffer *mappedBuffer = mappedStream->getBuffer();\n+\t\tif (!mappedBuffer) {\n+\t\t\tLOG(HAL, Error) << \"Failed getting a buffer\";\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\n+\t\tdescriptor.internalBuffers_[mappedStream] = mappedBuffer;\n+\t\tdescriptor.request_->addBuffer(mappedStream->stream(), mappedBuffer, -1);\n \t}\n \n \t/*\n@@ -1180,13 +1218,6 @@ void CameraDevice::requestComplete(Request *request)\n \t\tint ret = cameraStream->process(*src, *buffer.buffer,\n \t\t\t\t\t\tdescriptor.settings_,\n \t\t\t\t\t\tresultMetadata.get());\n-\t\t/*\n-\t\t * Return the FrameBuffer to the CameraStream now that we're\n-\t\t * done processing it.\n-\t\t */\n-\t\tif (cameraStream->type() == CameraStream::Type::Internal)\n-\t\t\tcameraStream->putBuffer(src);\n-\n \t\tif (ret) {\n \t\t\tbuffer.status = CAMERA3_BUFFER_STATUS_ERROR;\n \t\t\tnotifyError(descriptor.frameNumber_, buffer.stream,\n@@ -1194,6 +1225,9 @@ void CameraDevice::requestComplete(Request *request)\n \t\t}\n \t}\n \n+\tfor (auto &[stream, buffer] : descriptor.internalBuffers_)\n+\t\tstream->putBuffer(buffer);\n+\n \tcaptureResult.result = resultMetadata->get();\n \tcallbacks_->process_capture_result(callbacks_, &captureResult);\n }\ndiff --git a/src/android/camera_device.h b/src/android/camera_device.h\nindex 296c2f18..74d8150b 100644\n--- a/src/android/camera_device.h\n+++ b/src/android/camera_device.h\n@@ -21,6 +21,7 @@\n \n #include \n #include \n+#include \n #include \n #include \n #include \n@@ -84,6 +85,7 @@ private:\n \t\tstd::vector> frameBuffers_;\n \t\tCameraMetadata settings_;\n \t\tstd::unique_ptr request_;\n+\t\tstd::map internalBuffers_;\n \t};\n \n \tenum class State {\n@@ -118,6 +120,8 @@ private:\n \tstd::unique_ptr config_;\n \tCameraCapabilities capabilities_;\n \n+\tstd::unique_ptr frame_buffer_allocator_;\n+\n \tstd::map> requestTemplates_;\n \tconst camera3_callback_ops_t *callbacks_;\n \ndiff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp\nindex e30c7ee4..fc2e82cd 100644\n--- a/src/android/camera_stream.cpp\n+++ b/src/android/camera_stream.cpp\n@@ -18,6 +18,7 @@\n #include \"camera_capabilities.h\"\n #include \"camera_device.h\"\n #include \"camera_metadata.h\"\n+#include \"frame_buffer.h\"\n \n using namespace libcamera;\n \n@@ -46,10 +47,14 @@ LOG_DECLARE_CATEGORY(HAL)\n \n CameraStream::CameraStream(CameraDevice *const cameraDevice,\n \t\t\t CameraConfiguration *config, Type type,\n-\t\t\t camera3_stream_t *camera3Stream, unsigned int index)\n+\t\t\t camera3_stream_t *camera3Stream,\n+\t\t\t CameraStream *const mappedStream,\n+\t\t\t unsigned int index)\n \t: cameraDevice_(cameraDevice), config_(config), type_(type),\n-\t camera3Stream_(camera3Stream), index_(index)\n+\t camera3Stream_(camera3Stream), mappedStream_(mappedStream),\n+\t index_(index)\n {\n+\tASSERT(type_ != Type::Mapped || !!mappedStream);\n }\n \n const StreamConfiguration &CameraStream::configuration() const\n@@ -134,8 +139,30 @@ int CameraStream::process(const FrameBuffer &source,\n \n FrameBuffer *CameraStream::getBuffer()\n {\n-\tif (!allocator_)\n-\t\treturn nullptr;\n+\tif (!mutex_) {\n+\t\tASSERT(type_ == Type::Direct);\n+\t\tASSERT(!allocator_);\n+\t\tmutex_ = std::make_unique();\n+\t\tplatform_allocator_ = std::make_unique(cameraDevice_);\n+\t\tconst StreamConfiguration &config = configuration();\n+\t\tsize_t numBuffers = config.bufferCount;\n+\t\tconst int halPixelFormat = camera3Stream_->format;\n+\t\tconst uint32_t usage = camera3Stream_->usage;\n+\t\tLOG(HAL, Error) << \"getBuffer@@@@@ \" << numBuffers;\n+\t\t// numBuffers is 4 (=config.bufferCount) is not enough sadly...\n+\t\t// Should we dynamically allocate?\n+\t\tnumBuffers = 20;\n+\t\tconst auto &buffers = platform_allocator_->allocate(\n+\t\t\thalPixelFormat, config.size, usage, numBuffers);\n+\t\tif (buffers.empty() || buffers.size() != numBuffers) {\n+\t\t\tLOG(HAL, Error) << \"Failed allocating FrameBuffers\";\n+\t\t\treturn nullptr;\n+\t\t}\n+\n+\t\tbuffers_.reserve(numBuffers);\n+\t\tfor (const auto &frameBuffer : buffers)\n+\t\t\tbuffers_.push_back(frameBuffer.get());\n+\t}\n \n \tstd::lock_guard locker(*mutex_);\n \n@@ -143,7 +170,7 @@ FrameBuffer *CameraStream::getBuffer()\n \t\tLOG(HAL, Error) << \"Buffer underrun\";\n \t\treturn nullptr;\n \t}\n-\n+\tLOG(HAL, Error) << buffers_.size();\n \tFrameBuffer *buffer = buffers_.back();\n \tbuffers_.pop_back();\n \ndiff --git a/src/android/camera_stream.h b/src/android/camera_stream.h\nindex 2dab6c3a..c8eee908 100644\n--- a/src/android/camera_stream.h\n+++ b/src/android/camera_stream.h\n@@ -15,10 +15,11 @@\n \n #include \n #include \n-#include \n #include \n #include \n \n+#include \"frame_buffer.h\"\n+\n class CameraDevice;\n class CameraMetadata;\n class PostProcessor;\n@@ -110,12 +111,14 @@ public:\n \t};\n \tCameraStream(CameraDevice *const cameraDevice,\n \t\t libcamera::CameraConfiguration *config, Type type,\n-\t\t camera3_stream_t *camera3Stream, unsigned int index);\n+\t\t camera3_stream_t *camera3Stream,\n+\t\t CameraStream *const mappedStream, unsigned int index);\n \n \tType type() const { return type_; }\n \tconst camera3_stream_t &camera3Stream() const { return *camera3Stream_; }\n \tconst libcamera::StreamConfiguration &configuration() const;\n \tlibcamera::Stream *stream() const;\n+\tCameraStream *mappedStream() const { return mappedStream_; }\n \n \tint configure();\n \tint process(const libcamera::FrameBuffer &source,\n@@ -130,10 +133,13 @@ private:\n \tconst libcamera::CameraConfiguration *config_;\n \tconst Type type_;\n \tcamera3_stream_t *camera3Stream_;\n+\tCameraStream *const mappedStream_;\n \tconst unsigned int index_;\n \n \tstd::unique_ptr allocator_;\n+\tstd::unique_ptr platform_allocator_;\n \tstd::vector buffers_;\n+\tstd::vector> frameBuffers_;\n \t/*\n \t * The class has to be MoveConstructible as instances are stored in\n \t * an std::vector in CameraDevice.\n","prefixes":["libcamera-devel","RFC","2/2"]}