{"id":11144,"url":"https://patchwork.libcamera.org/api/1.1/patches/11144/?format=json","web_url":"https://patchwork.libcamera.org/patch/11144/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20210204100541.657503-7-hiroh@chromium.org>","date":"2021-02-04T10:05:41","name":"[libcamera-devel,RFC,6/6] android: camera_device: Support stream mapping","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"3bf4f65090364dd1c67b46736db659fe33c47914","submitter":{"id":63,"url":"https://patchwork.libcamera.org/api/1.1/people/63/?format=json","name":"Hirokazu Honda","email":"hiroh@chromium.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/11144/mbox/","series":[{"id":1650,"url":"https://patchwork.libcamera.org/api/1.1/series/1650/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=1650","date":"2021-02-04T10:05:35","name":"Support stream mapping in Android HAL adaptation layer","version":1,"mbox":"https://patchwork.libcamera.org/series/1650/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/11144/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/11144/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5B8F1BD162\n\tfor ;\n\tThu, 4 Feb 2021 10:06:04 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 2B11861435;\n\tThu, 4 Feb 2021 11:06:04 +0100 (CET)","from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com\n\t[IPv6:2607:f8b0:4864:20::1031])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 549DD61402\n\tfor ;\n\tThu, 4 Feb 2021 11:06:02 +0100 (CET)","by mail-pj1-x1031.google.com with SMTP id q72so1419428pjq.2\n\tfor ;\n\tThu, 04 Feb 2021 02:06:02 -0800 (PST)","from hiroh.c.googlers.com.com\n\t(128.141.236.35.bc.googleusercontent.com. [35.236.141.128])\n\tby smtp.gmail.com with ESMTPSA id\n\tj9sm5808418pgb.47.2021.02.04.02.05.57\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tThu, 04 Feb 2021 02:05:59 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"HhMs6b7F\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=from:to:cc:subject:date:message-id:in-reply-to:references\n\t:mime-version:content-transfer-encoding;\n\tbh=4piughn7EJr6AAA84UOv5h4/uYmAf0s9228/+iPEiJg=;\n\tb=HhMs6b7FtEro1fVhmic20BtTV+zER88RXUl0EDj7sjtAQ3IMR8qKcIIvjaWPsM+iX/\n\t3/DQ9thzd1iOn33T7WR52JKAqgW6/soj87kuAN4y85BTarpMZUDPnmNYf117+32PYQ26\n\tDa6FSfPXtMNzvOY79WnlYtwqi4ysninyjD6bc=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references:mime-version:content-transfer-encoding;\n\tbh=4piughn7EJr6AAA84UOv5h4/uYmAf0s9228/+iPEiJg=;\n\tb=XpWCamwuFVn4UmjQVTEO5GJ7tBL7fucOJjo3AXC+ppR9EFbVJZLLOThur9wujpLp5T\n\t5lsfmQt4k4SbJajGeLht+EUae1xrb8o3/BvBj0NJKJMQCk7+i72YLbOzYBjkvf7hcchS\n\t6oKZ4Q/KD3khIYxS/5176naRBfuq5CP/T/heX2tXN11vWaP5y16t8nVgA5eeQfmhFgXa\n\t6NTQCYPSYCMNww4FbOTZ6/+pYkg78NZNVzYyUUN4l4h7Wdwiy2sBVsgtKhc905Db18nG\n\tbPDWEiBMMSrtcfnCQpY7itS7J7/5DKu6790yrO2hC11ZCH+zaevOa5mKFTlXz0hqObAW\n\tUgZA==","X-Gm-Message-State":"AOAM5329Nrf7EIQOajP1T5DdcM7MnJyJbRi/iSWuk9WYhm5ATCbLpN4o\n\ta96rb0+EZRmy0L7wakWSn3JtRfZlY3D3fauy","X-Google-Smtp-Source":"ABdhPJyD4GEcoRG1ddnRGo7ErQKScmOxv9J5+THseF6PKm3Y0cfaPcPexudId9X+Udf1vBN26DtZeg==","X-Received":"by 2002:a17:902:6b43:b029:df:fb48:aece with SMTP id\n\tg3-20020a1709026b43b02900dffb48aecemr7483343plt.59.1612433160729; \n\tThu, 04 Feb 2021 02:06:00 -0800 (PST)","From":"Hirokazu Honda ","To":"libcamera-devel@lists.libcamera.org","Date":"Thu, 4 Feb 2021 10:05:41 +0000","Message-Id":"<20210204100541.657503-7-hiroh@chromium.org>","X-Mailer":"git-send-email 2.30.0.365.g02bc693789-goog","In-Reply-To":"<20210204100541.657503-1-hiroh@chromium.org>","References":"<20210204100541.657503-1-hiroh@chromium.org>","MIME-Version":"1.0","Subject":"[libcamera-devel] [RFC PATCH 6/6] android: camera_device: Support\n\tstream mapping","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"hanlinchen@chromium.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"This add the support of stream mapping in Android HAL adaptation\nlayer. It helps to produce a stream that cannot be output by\na native camera directly.\n\nSigned-off-by: Hirokazu Honda \n---\n src/android/camera_device.cpp | 143 ++++++++++++++++++++++++++++++++++\n 1 file changed, 143 insertions(+)\n\n--\n2.30.0.365.g02bc693789-goog","diff":"diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\nindex a1ef07b6..d45f83c1 100644\n--- a/src/android/camera_device.cpp\n+++ b/src/android/camera_device.cpp\n@@ -316,6 +316,132 @@ void sortCamera3StreamConfigs(std::vector &unsortedConfigs,\n \tunsortedConfigs = sortedConfigs;\n }\n\n+bool canConvert(const StreamConfiguration &srcConfig,\n+\t\tconst camera3_stream_t &dstStream,\n+\t\tCameraStream::Type &type,\n+\t\tPostProcessorType &postProcessorType)\n+{\n+\t/* NV12 -> JPEG conversion. */\n+\tif (srcConfig.pixelFormat == formats::NV12 &&\n+\t dstStream.format == HAL_PIXEL_FORMAT_BLOB &&\n+\t dstStream.width == srcConfig.size.width &&\n+\t dstStream.height == srcConfig.size.height) {\n+\t\ttype = CameraStream::Type::Mapped;\n+\t\tpostProcessorType = PostProcessorType::Jpeg;\n+\t\treturn true;\n+\t}\n+\n+\t/* NV12 -> NV12 */\n+\tif (srcConfig.pixelFormat == formats::NV12 &&\n+\t (dstStream.format == HAL_PIXEL_FORMAT_YCbCr_420_888 ||\n+\t dstStream.format == HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED)) {\n+\t\tif (dstStream.width == srcConfig.size.width &&\n+\t\t dstStream.height == srcConfig.size.height) {\n+\t\t\t/* Duplicated stream. No post processing is required. */\n+\t\t\ttype = CameraStream::Type::Direct;\n+\t\t\tpostProcessorType = PostProcessorType::None;\n+\t\t\treturn true;\n+\t\t} else if(dstStream.width < srcConfig.size.width &&\n+\t\t\t dstStream.height < srcConfig.size.height) {\n+\t\t\t/* Down scaling. */\n+\t\t\ttype = CameraStream::Type::Mapped;\n+\t\t\tpostProcessorType = PostProcessorType::Yuv;\n+\t\t\treturn true;\n+\t\t}\n+\t}\n+\treturn false;\n+}\n+\n+bool mapConfig(\n+\tstd::vector &streamConfigs,\n+\tconst std::vector> &tryConfigs)\n+{\n+\tconst StreamConfiguration kInvalidStreamConfiguration{};\n+\tassert(!kInvalidStreamConfiguration.pixelFormat.isValid());\n+\tstd::vector dstStreams;\n+\tfor (size_t i = 0; i < tryConfigs.size(); i++) {\n+\t\tLOG(HAL, Debug) << \"StreamConfiguration #\" << i << \": \"\n+\t\t\t\t<< tryConfigs[i].first.toString()\n+\t\t\t\t<< \", tryValidate() result =\"\n+\t\t\t\t<< static_cast(tryConfigs[i].second);\n+\t\tif (tryConfigs[i].second == CameraConfiguration::Valid)\n+\t\t\tcontinue;\n+\n+\t\tfor (auto &stream : streamConfigs[i].streams)\n+\t\t\tdstStreams.push_back(stream.stream);\n+\t\tstreamConfigs[i].streams.clear();\n+\n+\t\tif (tryConfigs[i].second == CameraConfiguration::Adjusted) {\n+\t\t\tstreamConfigs[i].config =\n+\t\t\t\tstd::move(tryConfigs[i].first);\n+\t\t} else {\n+\t\t\t/* Set invalid StreamConfiguration for a\n+\t\t\t * CameraConfiguration::Invalid stream.\n+\t\t\t */\n+\t\t\tstreamConfigs[i].config = kInvalidStreamConfiguration;\n+\t\t}\n+\t}\n+\n+\t/* Remove elements in streamConfigs whose stream is evaluated as\n+\t * CameraConfiguration::Invalid.\n+\t */\n+\tstreamConfigs.erase(\n+\t\tstd::remove_if(streamConfigs.begin(), streamConfigs.end(),\n+\t\t\t [](const auto &cfg) {\n+\t\t\t\t return !cfg.config.pixelFormat.isValid();\n+\t\t\t }),\n+\t\tstreamConfigs.end());\n+\n+\t/* Find a new source StreamConfiguration from which |dstStream| can be\n+\t * produced.\n+\t */\n+\tfor (auto *const dstStream : dstStreams) {\n+\t\tauto bestPostProcessorType = PostProcessorType::None;\n+\t\tauto bestType = CameraStream::Type::Mapped;\n+\t\tCamera3StreamConfig *bestConfig = nullptr;\n+\n+\t\tfor (auto &streamConfig : streamConfigs) {\n+\t\t\tauto type = CameraStream::Type::Mapped;\n+\t\t\tauto postProcessorType = PostProcessorType::None;\n+\n+\t\t\tif (canConvert(streamConfig.config, *dstStream, type,\n+\t\t\t\t postProcessorType) &&\n+\t\t\t static_cast(type) < static_cast(bestType)) {\n+\t\t\t\tbestType = type;\n+\t\t\t\tbestPostProcessorType = postProcessorType;\n+\t\t\t\tbestConfig = &streamConfig;\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (!bestConfig) {\n+\t\t\tLOG(HAL, Error)\n+\t\t\t\t<< \"No convertible native stream for the stream\"\n+\t\t\t\t<< \", width: \" << dstStream->width\n+\t\t\t\t<< \", height: \" << dstStream->height\n+\t\t\t\t<< \", format: \" << utils::hex(dstStream->format);\n+\t\t\treturn false;\n+\t\t}\n+\n+\t\tCamera3StreamConfig::Camera3Stream stream(\n+\t\t\tdstStream, bestType, bestPostProcessorType);\n+\t\tLOG(HAL, Error)\n+\t\t\t<< \" Mapped to StreamConfiguration: \"\n+\t\t\t<< bestConfig->config.toString()\n+\t\t\t<< \" from \" << stream.toString();\n+\t\tbestConfig->streams.push_back(std::move(stream));\n+\t}\n+\n+\t/* Remove elements in streamConfigs whose streams will not be used.*/\n+\tstreamConfigs.erase(\n+\t\tstd::remove_if(streamConfigs.begin(), streamConfigs.end(),\n+\t\t\t [](const auto &cfg) {\n+\t\t\t\t return !cfg.config.pixelFormat.isValid();\n+\t\t\t }),\n+\t\tstreamConfigs.end());\n+\treturn true;\n+}\n+\n } /* namespace */\n\n MappedCamera3Buffer::MappedCamera3Buffer(const buffer_handle_t camera3buffer,\n@@ -1634,6 +1760,23 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)\n\n \tsortCamera3StreamConfigs(streamConfigs, jpegStream);\n\n+\tstd::vector>\n+\t\ttryConfigs(streamConfigs.size());\n+\tfor (size_t i = 0; i < streamConfigs.size(); ++i) {\n+\t\tLOG(HAL, Debug) << \"Camera3StreamConfig #\" << i << \": \"\n+\t\t\t\t<< streamConfigs[i].toString();\n+\t\ttryConfigs[i].first = streamConfigs[i].config;\n+\t\ttryConfigs[i].second = CameraConfiguration::Valid;\n+\t}\n+\tif (!config_->tryValidate(tryConfigs)) {\n+\t\tLOG(HAL, Error) << \"Camera configuration invalid\";\n+\t\treturn -EINVAL;\n+\t}\n+\tif (!mapConfig(streamConfigs, tryConfigs)) {\n+\t\tLOG(HAL, Error) << \"Failed finding a stream mapping\";\n+\t\treturn -EINVAL;\n+\t}\n+\n \tstd::vector postProcessorTypes;\n \tfor (const auto &streamConfig : streamConfigs) {\n \t\tconfig_->addConfiguration(streamConfig.config);\n","prefixes":["libcamera-devel","RFC","6/6"]}