{"id":15256,"url":"https://patchwork.libcamera.org/api/1.1/patches/15256/?format=json","web_url":"https://patchwork.libcamera.org/patch/15256/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220106094323.1819801-2-hiroh@chromium.org>","date":"2022-01-06T09:43:23","name":"[libcamera-devel,1/1] android: camera_device: Configure one stream for identical stream requests","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"60584e53efd91ae0315c8cfab79624a450c3608b","submitter":{"id":63,"url":"https://patchwork.libcamera.org/api/1.1/people/63/?format=json","name":"Hirokazu Honda","email":"hiroh@chromium.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/15256/mbox/","series":[{"id":2880,"url":"https://patchwork.libcamera.org/api/1.1/series/2880/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2880","date":"2022-01-06T09:43:22","name":"Configure one stream for identical stream requests","version":1,"mbox":"https://patchwork.libcamera.org/series/2880/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/15256/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/15256/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\r\n\t[92.243.16.209])\r\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id F224ABF415\r\n\tfor ;\r\n\tThu, 6 Jan 2022 09:43:35 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\r\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A1C6660928;\r\n\tThu, 6 Jan 2022 10:43:35 +0100 (CET)","from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com\r\n\t[IPv6:2607:f8b0:4864:20::1036])\r\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id BCAF5604F5\r\n\tfor ;\r\n\tThu, 6 Jan 2022 10:43:33 +0100 (CET)","by mail-pj1-x1036.google.com with SMTP id\r\n\trj2-20020a17090b3e8200b001b1944bad25so2573669pjb.5\r\n\tfor ;\r\n\tThu, 06 Jan 2022 01:43:33 -0800 (PST)","from hiroh2.tok.corp.google.com\r\n\t([2401:fa00:8f:203:cb79:fed4:e097:2139])\r\n\tby smtp.gmail.com with ESMTPSA id\r\n\t4sm1844860pfy.191.2022.01.06.01.43.30\r\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\r\n\tThu, 06 Jan 2022 01:43:31 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com;\r\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\r\n\tunprotected) header.d=chromium.org header.i=@chromium.org\r\n\theader.b=\"C/kOC22X\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\r\n\ts=google; \r\n\th=from:to:cc:subject:date:message-id:in-reply-to:references\r\n\t:mime-version:content-transfer-encoding;\r\n\tbh=KyuDPm/zxDMeMdQNU7I6ndUbNpxweiggVhp2DAYM1/4=;\r\n\tb=C/kOC22XlKn9ctJizGH8yZh+Ra7Qb6BVDu+q1/SnMB6sZXyUTdGCsjY8RMh3Lo+BLr\r\n\tzNhCB976mKnkqogqkILy+y4ELyFdMi1I3vOoVt56E5mvLDa+Vu/j5SVQ/UJ0Xm37YbsJ\r\n\toslmVDjbAu0OCCUy8m8QS1zC9i9yOIG/g/SMw=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\r\n\td=1e100.net; s=20210112;\r\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\r\n\t:references:mime-version:content-transfer-encoding;\r\n\tbh=KyuDPm/zxDMeMdQNU7I6ndUbNpxweiggVhp2DAYM1/4=;\r\n\tb=8A0XisMcPJQrJpB5A59lgv7Ga0uBxN/uWMXVYoRybRZps2P5imaSnmoMxAw2WkVf5N\r\n\t2d2LwDlJ8KVatBUoq3rdcT7PRDLxzlVqOKV9c4ckIqtA6uqUlfLHV0JNCf2GGUCn/+n0\r\n\t6z0HbhDHlbx7FjZsYe8dzM3LQBitA+Z1Qt22NJeIMEU8eamGSAfyGHB7FP+B2rKBIEEL\r\n\tRPpnDzasCYwE6ENXj5YJrURNGaySfT0L129Z3bjd0GBQ17L5Ykd0mGr+KLKa0YV/mRmZ\r\n\tPfJ0y7junBVjm4gulK8EhQForB1mGa1j68qZX27pHM9VlkFUIkIr7TezwRK1SmInOx+7\r\n\txP8A==","X-Gm-Message-State":"AOAM531qwf+vyf5AAsLYpGJBWI3+mRe8OfoXEOcZt1SQad22A+gG+Bls\r\n\t/yYBD6MqdBoWiYkfn/4cPPjqE6fXPIFAzw==","X-Google-Smtp-Source":"ABdhPJwgeOX6H3BbI/LPNBfTljMqNkP3l7TJg8HzkYPlRak1iViDQwOQnbyRAAILYe20AV60U0HFYw==","X-Received":"by 2002:a17:90b:3b49:: with SMTP id\r\n\tot9mr9155941pjb.110.1641462211678; \r\n\tThu, 06 Jan 2022 01:43:31 -0800 (PST)","From":"Hirokazu Honda ","To":"libcamera-devel@lists.libcamera.org","Date":"Thu, 6 Jan 2022 18:43:23 +0900","Message-Id":"<20220106094323.1819801-2-hiroh@chromium.org>","X-Mailer":"git-send-email 2.34.1.448.ga2b2bfdf31-goog","In-Reply-To":"<20220106094323.1819801-1-hiroh@chromium.org>","References":"<20220106094323.1819801-1-hiroh@chromium.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH 1/1] android: camera_device: Configure one\r\n\tstream for identical stream requests","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\r\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\r\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"An Android HAL client may request multiple identical streams. It is\nredundant that a native camera device produces a separate stream for\neach of the identical requests. Configure the camera with a single\nstream in that case. The other identical HAL streams will be produced by\nthe YUV post-processor.\n\nThe Android HAL client can provide capture requests that are resolved as\nthey are produced by YUV post-processor. Then a buffer to be filled a\ncamera is not given. So the HAL adaptation layer looks up buffer\ninformation to be passed to a camera and allocate the buffer by using\nPlatformBufferAllocator.\n\nSigned-off-by: Hirokazu Honda \n---\n src/android/camera_device.cpp | 84 ++++++++++++++++++++++++++++++++++-\n src/android/camera_request.h | 2 +\n src/android/camera_stream.cpp | 12 ++++-\n src/android/camera_stream.h | 6 ++-\n 4 files changed, 100 insertions(+), 4 deletions(-)\n\n--\n2.34.1.448.ga2b2bfdf31-goog","diff":"diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\r\nindex 83825736..53533064 100644\r\n--- a/src/android/camera_device.cpp\r\n+++ b/src/android/camera_device.cpp\r\n@@ -9,6 +9,7 @@\r\n\r\n #include \r\n #include \r\n+#include \r\n #include \r\n #include \r\n #include \r\n@@ -604,6 +605,35 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)\r\n \t\t\tcontinue;\r\n \t\t}\r\n\r\n+\t\t/*\r\n+\t\t * While gralloc usage flags are supposed to report usage\r\n+\t\t * patterns to select a suitable buffer allocation strategy, in\r\n+\t\t * practice they're also used to make other decisions, such as\r\n+\t\t * selecting the actual format for the IMPLEMENTATION_DEFINED\r\n+\t\t * HAL pixel format. To avoid issues, we thus have to set the\r\n+\t\t * GRALLOC_USAGE_HW_CAMERA_WRITE flag unconditionally, even for\r\n+\t\t * streams that will be produced in software.\r\n+\t\t */\r\n+\t\tstream->usage |= GRALLOC_USAGE_HW_CAMERA_WRITE;\r\n+\r\n+\t\t/*\r\n+\t\t * If a CameraStream with the same size and format of the\r\n+\t\t * current stream has already been requested, associate the two.\r\n+\t\t */\r\n+\t\tauto iter = std::find_if(\r\n+\t\t\tstreamConfigs.begin(), streamConfigs.end(),\r\n+\t\t\t[size, format](const Camera3StreamConfig &streamConfig) {\r\n+\t\t\t\treturn streamConfig.config.size == size &&\r\n+\t\t\t\t streamConfig.config.pixelFormat == format;\r\n+\t\t\t});\r\n+\t\tif (iter != streamConfigs.end()) {\r\n+\t\t\t/* Add usage to copy the buffer in streams[0] to stream. */\r\n+\t\t\titer->streams[0].stream->usage |= GRALLOC_USAGE_SW_READ_OFTEN;\r\n+\t\t\tstream->usage |= GRALLOC_USAGE_SW_WRITE_OFTEN;\r\n+\t\t\titer->streams.push_back({ stream, CameraStream::Type::Mapped });\r\n+\t\t\tcontinue;\r\n+\t\t}\r\n+\r\n \t\tCamera3StreamConfig streamConfig;\r\n \t\tstreamConfig.streams = { { stream, CameraStream::Type::Direct } };\r\n \t\tstreamConfig.config.size = size;\r\n@@ -681,10 +711,32 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)\r\n \tfor (const auto &streamConfig : streamConfigs) {\r\n \t\tconfig->addConfiguration(streamConfig.config);\r\n\r\n+\t\tCameraStream *directStream = nullptr;\r\n \t\tfor (auto &stream : streamConfig.streams) {\r\n+\t\t\tCameraStream *sourceStream = nullptr;\r\n+\r\n+\t\t\t/*\r\n+\t\t\t * Sets the source stream for Mapped type streams to the\r\n+\t\t\t * same Direct type stream. The Direct type stream is\r\n+\t\t\t * put earlier than Mapped type streams in the current\r\n+\t\t\t * implementation. This might be broken in the future.\r\n+\t\t\t *\r\n+\t\t\t * \\todo Remove this order assumption.\r\n+\t\t\t */\r\n+\t\t\tif (stream.type == CameraStream::Type::Mapped) {\r\n+\t\t\t\tASSERT(directStream);\r\n+\t\t\t\tsourceStream = directStream;\r\n+\t\t\t}\r\n+\r\n \t\t\tstreams_.emplace_back(this, config.get(), stream.type,\r\n-\t\t\t\t\t stream.stream, config->size() - 1);\r\n+\t\t\t\t\t stream.stream,\r\n+\t\t\t\t\t sourceStream,\r\n+\t\t\t\t\t config->size() - 1);\r\n \t\t\tstream.stream->priv = static_cast(&streams_.back());\r\n+\t\t\tif (!directStream &&\r\n+\t\t\t stream.type == CameraStream::Type::Direct) {\r\n+\t\t\t\tdirectStream = &streams_.back();\r\n+\t\t\t}\r\n \t\t}\r\n \t}\r\n\r\n@@ -917,6 +969,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\r\n \tLOG(HAL, Debug) << \"Queueing request \" << descriptor->request_->cookie()\r\n \t\t\t<< \" with \" << descriptor->buffers_.size() << \" streams\";\r\n\r\n+\tstd::set requiredStreams;\r\n+\tstd::set providedStreams;\r\n \tfor (const auto &[i, buffer] : utils::enumerate(descriptor->buffers_)) {\r\n \t\tCameraStream *cameraStream = buffer.stream;\r\n \t\tcamera3_stream_t *camera3Stream = cameraStream->camera3Stream();\r\n@@ -947,6 +1001,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\r\n\r\n \t\t\tdescriptor->pendingStreamsToProcess_.insert(\r\n \t\t\t\t{ cameraStream, &buffer });\r\n+\t\t\tASSERT(cameraStream->sourceStream());\r\n+\t\t\trequiredStreams.insert(cameraStream->sourceStream());\r\n \t\t\tcontinue;\r\n\r\n \t\tcase CameraStream::Type::Direct:\r\n@@ -990,8 +1046,32 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\r\n \t\tauto fence = std::make_unique(std::move(acquireFence));\r\n \t\tdescriptor->request_->addBuffer(cameraStream->stream(),\r\n \t\t\t\t\t\tframeBuffer, std::move(fence));\r\n+\t\tprovidedStreams.insert(cameraStream);\r\n \t}\r\n\r\n+\tfor (CameraStream *requiredStream : requiredStreams) {\r\n+\t\tif (providedStreams.find(requiredStream) != providedStreams.end())\r\n+\t\t\tcontinue;\r\n+\r\n+\t\t/* \\todo Can we Map stream to Internal type stream? */\r\n+\t\tASSERT(requiredStream->type() == CameraStream::Type::Direct);\r\n+\r\n+\t\tFrameBuffer *frameBuffer = requiredStream->getBuffer();\r\n+\t\tif (!frameBuffer) {\r\n+\t\t\tLOG(HAL, Error) << \"Failed to create frame buffer\";\r\n+\t\t\treturn -ENOMEM;\r\n+\t\t}\r\n+\r\n+\t\t/*\r\n+\t\t * addBuffer() lets a buffer for requiredStream be output by\r\n+\t\t * camera. Records frameBuffer with requiredStream to call\r\n+\t\t * putBuffer() after post-processing is complete.\r\n+\t\t */\r\n+\t\tdescriptor->request_->addBuffer(requiredStream->stream(),\r\n+\t\t\t\t\t\tframeBuffer, nullptr);\r\n+\t\tMutexLocker lock(descriptor->streamsProcessMutex_);\r\n+\t\tdescriptor->putBuffers_.push_back({ requiredStream, frameBuffer });\r\n+\t}\r\n \t/*\r\n \t * Translate controls from Android to libcamera and queue the request\r\n \t * to the camera.\r\n@@ -1251,6 +1331,8 @@ void CameraDevice::streamProcessingComplete(Camera3RequestDescriptor::StreamBuff\r\n \t\trequest->pendingStreamsToProcess_.erase(streamBuffer->stream);\r\n \t\tif (!request->pendingStreamsToProcess_.empty())\r\n \t\t\treturn;\r\n+\t\tfor (auto [cameraStream, buffer] : request->putBuffers_)\r\n+\t\t\tcameraStream->putBuffer(buffer);\r\n \t}\r\n\r\n \tcompleteDescriptor(streamBuffer->request);\r\ndiff --git a/src/android/camera_request.h b/src/android/camera_request.h\r\nindex 37b6ae32..7a41c6d9 100644\r\n--- a/src/android/camera_request.h\r\n+++ b/src/android/camera_request.h\r\n@@ -59,6 +59,8 @@ public:\r\n \t/* Keeps track of streams requiring post-processing. */\r\n \tstd::map pendingStreamsToProcess_\r\n \t\tLIBCAMERA_TSA_GUARDED_BY(streamsProcessMutex_);\r\n+\tstd::vector> putBuffers_\r\n+\t\tLIBCAMERA_TSA_GUARDED_BY(streamsProcessMutex_);\r\n \tlibcamera::Mutex streamsProcessMutex_;\r\n\r\n \tCamera3RequestDescriptor(libcamera::Camera *camera,\r\ndiff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp\r\nindex c2157450..634cf319 100644\r\n--- a/src/android/camera_stream.cpp\r\n+++ b/src/android/camera_stream.cpp\r\n@@ -52,9 +52,11 @@ LOG_DECLARE_CATEGORY(HAL)\r\n\r\n CameraStream::CameraStream(CameraDevice *const cameraDevice,\r\n \t\t\t CameraConfiguration *config, Type type,\r\n-\t\t\t camera3_stream_t *camera3Stream, unsigned int index)\r\n+\t\t\t camera3_stream_t *camera3Stream,\r\n+\t\t\t CameraStream *const sourceStream, unsigned int index)\r\n \t: cameraDevice_(cameraDevice), config_(config), type_(type),\r\n-\t camera3Stream_(camera3Stream), index_(index)\r\n+\t camera3Stream_(camera3Stream), sourceStream_(sourceStream),\r\n+\t index_(index)\r\n {\r\n }\r\n\r\n@@ -206,6 +208,12 @@ void CameraStream::flush()\r\n\r\n FrameBuffer *CameraStream::getBuffer()\r\n {\r\n+\tif (type_ == Type::Direct && !allocator_) {\r\n+\t\tallocator_ = std::make_unique(cameraDevice_);\r\n+\t\tASSERT(!mutex_);\r\n+\t\tmutex_ = std::make_unique();\r\n+\t}\r\n+\r\n \tif (!allocator_)\r\n \t\treturn nullptr;\r\n\r\ndiff --git a/src/android/camera_stream.h b/src/android/camera_stream.h\r\nindex f9504462..4c5078b2 100644\r\n--- a/src/android/camera_stream.h\r\n+++ b/src/android/camera_stream.h\r\n@@ -114,7 +114,9 @@ public:\r\n \t};\r\n \tCameraStream(CameraDevice *const cameraDevice,\r\n \t\t libcamera::CameraConfiguration *config, Type type,\r\n-\t\t camera3_stream_t *camera3Stream, unsigned int index);\r\n+\t\t camera3_stream_t *camera3Stream,\r\n+\t\t CameraStream *const sourceStream,\r\n+\t\t unsigned int index);\r\n \tCameraStream(CameraStream &&other);\r\n \t~CameraStream();\r\n\r\n@@ -122,6 +124,7 @@ public:\r\n \tcamera3_stream_t *camera3Stream() const { return camera3Stream_; }\r\n \tconst libcamera::StreamConfiguration &configuration() const;\r\n \tlibcamera::Stream *stream() const;\r\n+\tCameraStream *sourceStream() const { return sourceStream_; }\r\n\r\n \tint configure();\r\n \tint process(Camera3RequestDescriptor::StreamBuffer *streamBuffer);\r\n@@ -167,6 +170,7 @@ private:\r\n \tconst libcamera::CameraConfiguration *config_;\r\n \tconst Type type_;\r\n \tcamera3_stream_t *camera3Stream_;\r\n+\tCameraStream *const sourceStream_;\r\n \tconst unsigned int index_;\r\n\r\n \tstd::unique_ptr allocator_;","prefixes":["libcamera-devel","1/1"]}