From patchwork Wed Jun 29 10:30:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 16434 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1BF0ABE173 for ; Wed, 29 Jun 2022 10:30:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B459A65646; Wed, 29 Jun 2022 12:30:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656498639; bh=FbJsGszTvkjsFHZa24XQPE191prSV0m7J1vk45/E0Dg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=MnnjLoJSyjhcH6ldf23SQ7WNBDl9idaHlh4JDwvaX6n+R3eemic7L5zX1L4cjh7wT ttF5x+sASkaziscfKtp+ksoQTqZvNCdn7q5alz0JWVCCfSmU4VMCqjmT7LAf+BNC+P GJtSgvkg0aY1NPM2OtkA25ulGOGHT6nspwwjp1kJlNqg+bCcx0Dz0l6QCqkYN173Bo 1A0qMY9LY9B8CUnXl/smTkPFubnohUiyicL2c76YxISpJnU/1/sy6sysbXrJHZ1hEa C3pGKeDNPaV9M16HfEbXDwfN80eB+MZLJ70zgqDrMPgB8/0DZyo/MH/4XOvOuhtzGx EOGnmVLreEy+A== Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 29F7665635 for ; Wed, 29 Jun 2022 12:30:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="CfFfAmYm"; dkim-atps=neutral Received: by mail-pf1-x433.google.com with SMTP id 136so9576186pfy.10 for ; Wed, 29 Jun 2022 03:30:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=umd9+EQTp74CNkn5SYDVVvbPaWnB1mCRZape6APHeiY=; b=CfFfAmYmg7gc/gNFOasafoRl6iqspMH+pSxOdq97vt5UL9rW9xsvshzqQWlikvFjF3 mnXFJ5lF7iaNFSorCCC9rypUzQKrlu3oy4qGd9EltPFtwfx9Bcp4RhHnNJAdk1+9wy9I 9b7zyurJJ+rkJ4Bld58ouI1gvHjmRKCL8VBww= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=umd9+EQTp74CNkn5SYDVVvbPaWnB1mCRZape6APHeiY=; b=Ur+VGzUgWqPd5YDAzBwXIToybhFqmHJzZmcPo9EwwjD2BAYFYPZqW81svug+p+0wMz +3t7jKI+HOMqFqeE0z/SLN7jsrj0JLPSEcK5yD2/B2xT8lcX0pY64YKhFoypbTu5Jd+w GXunnxbZYUtV+v/wbvBPHTpNLiLdoaLX7beoOrN2Av5VQ+TevHEv3L3zoE3xldoujrZU zWCeCwtbDkVlpBU+Dwg7Ff3rq5/okVnoGj5n6YqhbomZJOt7jM8F+5eWtYzIF0WQdcZF aQx2gXtW1FywlrAGbl2LaLUP0EhZxLn0fvSY8e4o4JNsA06cHmYSPVUE6dnss++NsUan NnxA== X-Gm-Message-State: AJIora8xNWkuZuRTtq7l+UL0FF3lGuED+gunP1ASCu9Ie/sWZKvZB+k4 cirzBbqEpWvt9a+BaQROez5dnSMV3RoNlz09 X-Google-Smtp-Source: AGRyM1uUDQYDNm/ktMgxN8iBtRwfTpPggwv/J/lMmL0bW8K7lw3+XIs74UzGC4Dr+yC3/LjJZvpRaQ== X-Received: by 2002:a63:5810:0:b0:40d:77fb:1c25 with SMTP id m16-20020a635810000000b0040d77fb1c25mr2415856pgb.570.1656498632456; Wed, 29 Jun 2022 03:30:32 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (208.158.221.35.bc.googleusercontent.com. [35.221.158.208]) by smtp.gmail.com with ESMTPSA id jb4-20020a170903258400b00169071538a0sm10946059plb.267.2022.06.29.03.30.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Jun 2022 03:30:32 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Wed, 29 Jun 2022 10:30:17 +0000 Message-Id: <20220629103018.4025635-9-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220629103018.4025635-1-chenghaoyang@google.com> References: <20220629103018.4025635-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 8/9] Request StillCapture stream as default in Android adapter X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Harvey Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If the pipeline handler accepts, the Android adapter requests to handle the StillCapture stream separately, instead of being mapped to an existing video/preview stream or creating a new video stream. Signed-off-by: Harvey Yang --- src/android/camera_device.cpp | 235 +++++++++++++++++++--------------- src/android/camera_device.h | 21 +++ 2 files changed, 154 insertions(+), 102 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 00d48471..bbb72615 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -36,23 +36,6 @@ LOG_DECLARE_CATEGORY(HAL) namespace { -/* - * \struct Camera3StreamConfig - * \brief Data to store StreamConfiguration associated with camera3_stream(s) - * \var streams List of the pairs of a stream requested by Android HAL client - * and CameraStream::Type associated with the stream - * \var config StreamConfiguration for streams - */ -struct Camera3StreamConfig { - struct Camera3Stream { - camera3_stream_t *stream; - CameraStream::Type type; - }; - - std::vector streams; - StreamConfiguration config; -}; - /* * Reorder the configurations so that libcamera::Camera can accept them as much * as possible. The sort rule is as follows. @@ -536,16 +519,6 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) return -EINVAL; #endif - /* - * Generate an empty configuration, and construct a StreamConfiguration - * for each camera3_stream to add to it. - */ - std::unique_ptr config = camera_->generateConfiguration(); - if (!config) { - LOG(HAL, Error) << "Failed to generate camera configuration"; - return -EINVAL; - } - /* * Clear and remove any existing configuration from previous calls, and * ensure the required entries are available without further @@ -614,93 +587,102 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) stream->usage |= GRALLOC_USAGE_HW_CAMERA_WRITE; } + std::unique_ptr config; + /* Now handle the MJPEG streams, adding a new stream if required. */ if (jpegStream) { - CameraStream::Type type; - int index = -1; - - /* Search for a compatible stream in the non-JPEG ones. */ - for (size_t i = 0; i < streamConfigs.size(); ++i) { - Camera3StreamConfig &streamConfig = streamConfigs[i]; - const auto &cfg = streamConfig.config; - - /* - * \todo The PixelFormat must also be compatible with - * the encoder. - */ - if (cfg.size.width != jpegStream->width || - cfg.size.height != jpegStream->height) - continue; - - LOG(HAL, Info) - << "Android JPEG stream mapped to libcamera stream " << i; - - type = CameraStream::Type::Mapped; - index = i; - - /* - * The source stream will be read by software to - * produce the JPEG stream. - */ - camera3_stream_t *stream = streamConfig.streams[0].stream; - stream->usage |= GRALLOC_USAGE_SW_READ_OFTEN; - break; - } + /* This stream will be produced by hardware. */ + jpegStream->usage |= GRALLOC_USAGE_HW_CAMERA_WRITE; /* - * Without a compatible match for JPEG encoding we must - * introduce a new stream to satisfy the request requirements. + * \todo The pixelFormat should be a 'best-fit' choice + * and may require a validation cycle. This is not yet + * handled, and should be considered as part of any + * stream configuration reworks. */ - if (index < 0) { + Camera3StreamConfig sConfig; + sConfig.config.streamRole = libcamera::StreamRole::StillCapture; + sConfig.config.size.width = jpegStream->width; + sConfig.config.size.height = jpegStream->height; + sConfig.config.pixelFormat = formats::NV12; + + std::vector streamConfigsCopy(streamConfigs); + streamConfigsCopy.push_back(std::move(sConfig)); + + LOG(HAL, Info) << "Adding " << sConfig.config.toString() + << " for MJPEG support"; + + streamConfigsCopy.back().streams.push_back({ jpegStream, CameraStream::Type::Internal }); + + config = tryValidation(streamConfigsCopy, jpegStream); + if (config) { + streamConfigs = streamConfigsCopy; + } else { + /* The JPEG stream will be produced by software. */ + jpegStream->usage |= GRALLOC_USAGE_SW_WRITE_OFTEN; + + CameraStream::Type type; + int index = -1; + /* Search for a compatible stream in the non-JPEG ones. */ + for (size_t i = 0; i < streamConfigs.size(); ++i) { + Camera3StreamConfig &streamConfig = streamConfigs[i]; + const auto &cfg = streamConfig.config; + + /* + * \todo The PixelFormat must also be compatible with + * the encoder. + */ + if (cfg.size.width != jpegStream->width || + cfg.size.height != jpegStream->height) + continue; + + LOG(HAL, Info) + << "Android JPEG stream mapped to libcamera stream " << i; + index = i; + type = CameraStream::Type::Mapped; + + /* + * The source stream will be read by software to + * produce the JPEG stream. + */ + camera3_stream_t *stream = streamConfig.streams[0].stream; + stream->usage |= GRALLOC_USAGE_SW_READ_OFTEN; + break; + } + /* - * \todo The pixelFormat should be a 'best-fit' choice - * and may require a validation cycle. This is not yet - * handled, and should be considered as part of any - * stream configuration reworks. + * Without a compatible match for JPEG encoding we must + * introduce a new stream to satisfy the request requirements. */ - Camera3StreamConfig streamConfig; - streamConfig.config.size.width = jpegStream->width; - streamConfig.config.size.height = jpegStream->height; - streamConfig.config.pixelFormat = formats::NV12; - streamConfigs.push_back(std::move(streamConfig)); - - LOG(HAL, Info) << "Adding " << streamConfig.config.toString() - << " for MJPEG support"; - - type = CameraStream::Type::Internal; - index = streamConfigs.size() - 1; - } - - /* The JPEG stream will be produced by software. */ - jpegStream->usage |= GRALLOC_USAGE_SW_WRITE_OFTEN; - - streamConfigs[index].streams.push_back({ jpegStream, type }); - } - - sortCamera3StreamConfigs(streamConfigs, jpegStream); - for (const auto &streamConfig : streamConfigs) { - config->addConfiguration(streamConfig.config); + if (index < 0) { + /* + * \todo The pixelFormat should be a 'best-fit' choice + * and may require a validation cycle. This is not yet + * handled, and should be considered as part of any + * stream configuration reworks. + */ + Camera3StreamConfig streamConfig; + streamConfig.config.size.width = jpegStream->width; + streamConfig.config.size.height = jpegStream->height; + streamConfig.config.pixelFormat = formats::NV12; + streamConfigs.push_back(std::move(streamConfig)); + + LOG(HAL, Info) << "Adding " << streamConfig.config.toString() + << " for MJPEG support"; + + type = CameraStream::Type::Internal; + index = streamConfigs.size() - 1; + } - for (auto &stream : streamConfig.streams) { - streams_.emplace_back(this, config.get(), stream.type, - stream.stream, config->size() - 1); - stream.stream->priv = static_cast(&streams_.back()); + streamConfigs[index].streams.push_back({ jpegStream, type }); } } - switch (config->validate()) { - case CameraConfiguration::Valid: - break; - case CameraConfiguration::Adjusted: - LOG(HAL, Info) << "Camera configuration adjusted"; - - for (const StreamConfiguration &cfg : *config) - LOG(HAL, Info) << " - " << cfg.toString(); + if (!config) { + config = tryValidation(streamConfigs, jpegStream); + if (!config) + return -EINVAL; - return -EINVAL; - case CameraConfiguration::Invalid: - LOG(HAL, Info) << "Camera configuration invalid"; - return -EINVAL; } /* @@ -1527,3 +1509,52 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons return resultMetadata; } + +std::unique_ptr CameraDevice::tryValidation( + std::vector &streamConfigs, + const camera3_stream_t *jpegStream) +{ + /* + * Generate an empty configuration, and construct a StreamConfiguration + * for each camera3_stream to add to it. + */ + std::unique_ptr config = camera_->generateConfiguration(); + if (!config) { + LOG(HAL, Error) << "Failed to generate camera configuration"; + return nullptr; + } + + sortCamera3StreamConfigs(streamConfigs, jpegStream); + for (const auto &streamConfig : streamConfigs) { + config->addConfiguration(streamConfig.config); + + for (auto &stream : streamConfig.streams) { + streams_.emplace_back(this, config.get(), stream.type, + stream.stream, config->size() - 1); + stream.stream->priv = static_cast(&streams_.back()); + } + } + + switch (config->validate()) { + case CameraConfiguration::Valid: + break; + case CameraConfiguration::Adjusted: + LOG(HAL, Info) << "Camera configuration adjusted"; + + for (const StreamConfiguration &cfg : *config) + LOG(HAL, Info) << " - " << cfg.toString(); + + config.reset(); + break; + case CameraConfiguration::Invalid: + LOG(HAL, Info) << "Camera configuration invalid"; + + config.reset(); + break; + } + + if (!config) + streams_.clear(); + + return config; +} diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 64050416..8540e3f5 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -34,6 +34,23 @@ class Camera3RequestDescriptor; struct CameraConfigData; +/* + * \struct Camera3StreamConfig + * \brief Data to store StreamConfiguration associated with camera3_stream(s) + * \var streams List of the pairs of a stream requested by Android HAL client + * and CameraStream::Type associated with the stream + * \var config StreamConfiguration for streams + */ +struct Camera3StreamConfig { + struct Camera3Stream { + camera3_stream_t *stream; + CameraStream::Type type; + }; + + std::vector streams; + libcamera::StreamConfiguration config; +}; + class CameraDevice : protected libcamera::Loggable { public: @@ -101,6 +118,10 @@ private: std::unique_ptr getResultMetadata( const Camera3RequestDescriptor &descriptor) const; + std::unique_ptr tryValidation( + std::vector &streamConfigs, + const camera3_stream_t *jpegStream); + unsigned int id_; camera3_device_t camera3Device_;