From patchwork Tue Aug 2 10:29:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16907 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6DA71C3275 for ; Tue, 2 Aug 2022 10:30:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 30E5A6331E; Tue, 2 Aug 2022 12:30:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1659436204; bh=cVUZrqn5rxXptGZixyEE7N62IrSbeb/6ouOj91M4N0Q=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=qY8FLLl0LCFg4IF8lhpYBF1QcxGpP0Y+K481K/oa8pwI6zXN/aMjQejFCP1eih2WO uGDJ8vMk0I1w9IkdmuESVE8D2zKvl2XVcsoyHdi2DSrzLhD3MWo8IR4fshbPWnwTTp QYmsqBN9aPmEsd7H7IbiDE2ZHORwTrAG3/EZvdVDa5PB7cb6zXqpB2YDN2vA6IUqsv D93kv6yIyV0+PoXqyPkB0sNlKZbsdBzpfPVW5f9IYdi8RCN3j1Ll6jCbAkTS4bV1cD 1ap2dGWMXMnm+FfZ6MPd5uhAm38xg8b1RT4q802dTbb2QdpXP1ZW4QVYTmafQm6e6G D8lsO89W3LmtQ== Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6C50263327 for ; Tue, 2 Aug 2022 12:30:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Mup07Rnk"; dkim-atps=neutral Received: by mail-pf1-x429.google.com with SMTP id 17so13214614pfy.0 for ; Tue, 02 Aug 2022 03:30:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=l+ECUe5UPfDYukWy9zAiCTnRz4IAiHnvawL2cEiDqPU=; b=Mup07Rnk9xMNnPzaJaO8dAFvn+91lNXQZRoxdqZGNjKzSjOIwXBw9cRa13lAvXepBV QbOq8H7THm/6vLXai609935V/tl/7RR6t4iPAsYVBWdtDoXGYXA4XQedRNrTwYR+2IXk OuXWtbzkgQKyYSLBEw+WP9W7omKjDAD3AFpoE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l+ECUe5UPfDYukWy9zAiCTnRz4IAiHnvawL2cEiDqPU=; b=KdhRsKPZRbuKFKgRwu+LDfyduJ4EI02py7XLfVvvFW7z6jqhJxufzguZH650eRSgRT j0XOJ1Pe8dQhFHJTFjvRSljBljms97fCdtlfYE3PftprCZ+AlB6t6B11lhsp47t0LkQi Ftjsk9ddj51rTpQW8h3Aunj+pldaM8FgwhhUYmT5pVRLQrwsSbYW/TXhL9UMM0mCzU/q JR44TXoy2HMbKPEjLY14NZt0G8Ev++hrz4q+IvJeBZ21VgRMbqiLpx7i/cBvFVR/dgeA niXzda91F1W/vr+rzXc3+qNEqd9/B/xCUtDVrzWkIDbZSv5FvdDHgnes9wZYNksbsq+k mStQ== X-Gm-Message-State: AJIora91GGua0F7yJd9JVS0d91K5JHVi/G6p+xn1hnHcAZQyuBIDsnId m5TBgJ1rA9d5RG6oywfzoqoyz6rU5fj6qg== X-Google-Smtp-Source: AGRyM1tXdUFcdz+0ZN6ThZPaXkYbFEOKxPiB6bgNl2/A0LkQ3OQQz5wdlCr9ozdIgA16hkTlGhPC+Q== X-Received: by 2002:a63:4913:0:b0:40d:8235:2d1c with SMTP id w19-20020a634913000000b0040d82352d1cmr16259816pga.584.1659436201438; Tue, 02 Aug 2022 03:30:01 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (231.137.80.34.bc.googleusercontent.com. [34.80.137.231]) by smtp.gmail.com with ESMTPSA id t15-20020a170902e84f00b0016db88f69a2sm1955128plg.141.2022.08.02.03.30.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 03:30:00 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Aug 2022 10:29:42 +0000 Message-Id: <20220802102943.3221109-9-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog In-Reply-To: <20220802102943.3221109-1-chenghaoyang@google.com> References: <20220802102943.3221109-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 8/9] Request StillCapture stream as default in Android adapter X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If the pipeline handler accepts, the Android adapter requests to handle the StillCapture stream separately, instead of being mapped to an existing video/preview stream or creating a new video stream. Signed-off-by: Harvey Yang --- src/android/camera_device.cpp | 235 +++++++++++++++++++--------------- src/android/camera_device.h | 21 +++ 2 files changed, 154 insertions(+), 102 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 00d48471..bbb72615 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -36,23 +36,6 @@ LOG_DECLARE_CATEGORY(HAL) namespace { -/* - * \struct Camera3StreamConfig - * \brief Data to store StreamConfiguration associated with camera3_stream(s) - * \var streams List of the pairs of a stream requested by Android HAL client - * and CameraStream::Type associated with the stream - * \var config StreamConfiguration for streams - */ -struct Camera3StreamConfig { - struct Camera3Stream { - camera3_stream_t *stream; - CameraStream::Type type; - }; - - std::vector streams; - StreamConfiguration config; -}; - /* * Reorder the configurations so that libcamera::Camera can accept them as much * as possible. The sort rule is as follows. @@ -536,16 +519,6 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) return -EINVAL; #endif - /* - * Generate an empty configuration, and construct a StreamConfiguration - * for each camera3_stream to add to it. - */ - std::unique_ptr config = camera_->generateConfiguration(); - if (!config) { - LOG(HAL, Error) << "Failed to generate camera configuration"; - return -EINVAL; - } - /* * Clear and remove any existing configuration from previous calls, and * ensure the required entries are available without further @@ -614,93 +587,102 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) stream->usage |= GRALLOC_USAGE_HW_CAMERA_WRITE; } + std::unique_ptr config; + /* Now handle the MJPEG streams, adding a new stream if required. */ if (jpegStream) { - CameraStream::Type type; - int index = -1; - - /* Search for a compatible stream in the non-JPEG ones. */ - for (size_t i = 0; i < streamConfigs.size(); ++i) { - Camera3StreamConfig &streamConfig = streamConfigs[i]; - const auto &cfg = streamConfig.config; - - /* - * \todo The PixelFormat must also be compatible with - * the encoder. - */ - if (cfg.size.width != jpegStream->width || - cfg.size.height != jpegStream->height) - continue; - - LOG(HAL, Info) - << "Android JPEG stream mapped to libcamera stream " << i; - - type = CameraStream::Type::Mapped; - index = i; - - /* - * The source stream will be read by software to - * produce the JPEG stream. - */ - camera3_stream_t *stream = streamConfig.streams[0].stream; - stream->usage |= GRALLOC_USAGE_SW_READ_OFTEN; - break; - } + /* This stream will be produced by hardware. */ + jpegStream->usage |= GRALLOC_USAGE_HW_CAMERA_WRITE; /* - * Without a compatible match for JPEG encoding we must - * introduce a new stream to satisfy the request requirements. + * \todo The pixelFormat should be a 'best-fit' choice + * and may require a validation cycle. This is not yet + * handled, and should be considered as part of any + * stream configuration reworks. */ - if (index < 0) { + Camera3StreamConfig sConfig; + sConfig.config.streamRole = libcamera::StreamRole::StillCapture; + sConfig.config.size.width = jpegStream->width; + sConfig.config.size.height = jpegStream->height; + sConfig.config.pixelFormat = formats::NV12; + + std::vector streamConfigsCopy(streamConfigs); + streamConfigsCopy.push_back(std::move(sConfig)); + + LOG(HAL, Info) << "Adding " << sConfig.config.toString() + << " for MJPEG support"; + + streamConfigsCopy.back().streams.push_back({ jpegStream, CameraStream::Type::Internal }); + + config = tryValidation(streamConfigsCopy, jpegStream); + if (config) { + streamConfigs = streamConfigsCopy; + } else { + /* The JPEG stream will be produced by software. */ + jpegStream->usage |= GRALLOC_USAGE_SW_WRITE_OFTEN; + + CameraStream::Type type; + int index = -1; + /* Search for a compatible stream in the non-JPEG ones. */ + for (size_t i = 0; i < streamConfigs.size(); ++i) { + Camera3StreamConfig &streamConfig = streamConfigs[i]; + const auto &cfg = streamConfig.config; + + /* + * \todo The PixelFormat must also be compatible with + * the encoder. + */ + if (cfg.size.width != jpegStream->width || + cfg.size.height != jpegStream->height) + continue; + + LOG(HAL, Info) + << "Android JPEG stream mapped to libcamera stream " << i; + index = i; + type = CameraStream::Type::Mapped; + + /* + * The source stream will be read by software to + * produce the JPEG stream. + */ + camera3_stream_t *stream = streamConfig.streams[0].stream; + stream->usage |= GRALLOC_USAGE_SW_READ_OFTEN; + break; + } + /* - * \todo The pixelFormat should be a 'best-fit' choice - * and may require a validation cycle. This is not yet - * handled, and should be considered as part of any - * stream configuration reworks. + * Without a compatible match for JPEG encoding we must + * introduce a new stream to satisfy the request requirements. */ - Camera3StreamConfig streamConfig; - streamConfig.config.size.width = jpegStream->width; - streamConfig.config.size.height = jpegStream->height; - streamConfig.config.pixelFormat = formats::NV12; - streamConfigs.push_back(std::move(streamConfig)); - - LOG(HAL, Info) << "Adding " << streamConfig.config.toString() - << " for MJPEG support"; - - type = CameraStream::Type::Internal; - index = streamConfigs.size() - 1; - } - - /* The JPEG stream will be produced by software. */ - jpegStream->usage |= GRALLOC_USAGE_SW_WRITE_OFTEN; - - streamConfigs[index].streams.push_back({ jpegStream, type }); - } - - sortCamera3StreamConfigs(streamConfigs, jpegStream); - for (const auto &streamConfig : streamConfigs) { - config->addConfiguration(streamConfig.config); + if (index < 0) { + /* + * \todo The pixelFormat should be a 'best-fit' choice + * and may require a validation cycle. This is not yet + * handled, and should be considered as part of any + * stream configuration reworks. + */ + Camera3StreamConfig streamConfig; + streamConfig.config.size.width = jpegStream->width; + streamConfig.config.size.height = jpegStream->height; + streamConfig.config.pixelFormat = formats::NV12; + streamConfigs.push_back(std::move(streamConfig)); + + LOG(HAL, Info) << "Adding " << streamConfig.config.toString() + << " for MJPEG support"; + + type = CameraStream::Type::Internal; + index = streamConfigs.size() - 1; + } - for (auto &stream : streamConfig.streams) { - streams_.emplace_back(this, config.get(), stream.type, - stream.stream, config->size() - 1); - stream.stream->priv = static_cast(&streams_.back()); + streamConfigs[index].streams.push_back({ jpegStream, type }); } } - switch (config->validate()) { - case CameraConfiguration::Valid: - break; - case CameraConfiguration::Adjusted: - LOG(HAL, Info) << "Camera configuration adjusted"; - - for (const StreamConfiguration &cfg : *config) - LOG(HAL, Info) << " - " << cfg.toString(); + if (!config) { + config = tryValidation(streamConfigs, jpegStream); + if (!config) + return -EINVAL; - return -EINVAL; - case CameraConfiguration::Invalid: - LOG(HAL, Info) << "Camera configuration invalid"; - return -EINVAL; } /* @@ -1527,3 +1509,52 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons return resultMetadata; } + +std::unique_ptr CameraDevice::tryValidation( + std::vector &streamConfigs, + const camera3_stream_t *jpegStream) +{ + /* + * Generate an empty configuration, and construct a StreamConfiguration + * for each camera3_stream to add to it. + */ + std::unique_ptr config = camera_->generateConfiguration(); + if (!config) { + LOG(HAL, Error) << "Failed to generate camera configuration"; + return nullptr; + } + + sortCamera3StreamConfigs(streamConfigs, jpegStream); + for (const auto &streamConfig : streamConfigs) { + config->addConfiguration(streamConfig.config); + + for (auto &stream : streamConfig.streams) { + streams_.emplace_back(this, config.get(), stream.type, + stream.stream, config->size() - 1); + stream.stream->priv = static_cast(&streams_.back()); + } + } + + switch (config->validate()) { + case CameraConfiguration::Valid: + break; + case CameraConfiguration::Adjusted: + LOG(HAL, Info) << "Camera configuration adjusted"; + + for (const StreamConfiguration &cfg : *config) + LOG(HAL, Info) << " - " << cfg.toString(); + + config.reset(); + break; + case CameraConfiguration::Invalid: + LOG(HAL, Info) << "Camera configuration invalid"; + + config.reset(); + break; + } + + if (!config) + streams_.clear(); + + return config; +} diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 64050416..8540e3f5 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -34,6 +34,23 @@ class Camera3RequestDescriptor; struct CameraConfigData; +/* + * \struct Camera3StreamConfig + * \brief Data to store StreamConfiguration associated with camera3_stream(s) + * \var streams List of the pairs of a stream requested by Android HAL client + * and CameraStream::Type associated with the stream + * \var config StreamConfiguration for streams + */ +struct Camera3StreamConfig { + struct Camera3Stream { + camera3_stream_t *stream; + CameraStream::Type type; + }; + + std::vector streams; + libcamera::StreamConfiguration config; +}; + class CameraDevice : protected libcamera::Loggable { public: @@ -101,6 +118,10 @@ private: std::unique_ptr getResultMetadata( const Camera3RequestDescriptor &descriptor) const; + std::unique_ptr tryValidation( + std::vector &streamConfigs, + const camera3_stream_t *jpegStream); + unsigned int id_; camera3_device_t camera3Device_;