From patchwork Thu Feb 4 10:05:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hirokazu Honda X-Patchwork-Id: 11139 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 45123BD162 for ; Thu, 4 Feb 2021 10:05:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 13CCD6142A; Thu, 4 Feb 2021 11:05:53 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Hj9vuvtL"; dkim-atps=neutral Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 288B061402 for ; Thu, 4 Feb 2021 11:05:51 +0100 (CET) Received: by mail-pj1-x1035.google.com with SMTP id s24so1410042pjp.5 for ; Thu, 04 Feb 2021 02:05:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=c3N37gVuGh2sjzISx2yb690NFKzleVus8ZFnC08+rkU=; b=Hj9vuvtLXgqP0Rv+Jd62DahTfGNdjUNdcsOVpIubyeD7JlwCe+xbO+DCmX8sq2Bejp ahNShZqgkLIQuN/5zYmHzMdR3G43gkUHsdjJYbw5lAKXD9sSQypRjTAOqeF2R85gGUb2 4MaG9PdifeB/2a2uZj4zU9xfn3HzmOkDRS9VU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=c3N37gVuGh2sjzISx2yb690NFKzleVus8ZFnC08+rkU=; b=ehUr9Q2fvHzbI7UGj6WQqTq8Ij1g+DIwqmu8src8Zm8dBjPzvh3EvxIgDuEjOW5cHo kzU6vB69GRPPAxLnYGEKQz8EkStPk5Z5zvWf4VkSC/YD1Pt4BOsuTEpigpa19yVxr+fF Q+dKDtXnynBDiUmsbjgwTs6xuJkbw3SGOcsOKqQ1t6zu1wnQnRdShsmmoeTvRuEA/iYg y04cUXQ/RKXBdpO0fzUKqvXJW3NsetbZxmgwyE2RA8+JScZWWhtKey3EYJbbjmo4PRUc 6NKEfHtODZWPw4xI90fnqeoXI+U04PHbjtDC6WSRDLi+2ZnAUTsJVKALLnCPC+0XPo7W Ykxw== X-Gm-Message-State: AOAM533QE8c+aHvISHJxpsm841Rzpa24b4a+RP94CiOMKbELfPTn4TG/ 4Kf3NsH94Bk86Q3KlyIdcR/D0c0JIQwCzvEV X-Google-Smtp-Source: ABdhPJz5Er5I3t6Oscrq7DymUBU2iIb5R1wZH1F6ut4r+zzc0Oal5QcEtwMKJ2BY2Ij5GGaPcOVriw== X-Received: by 2002:a17:90a:2f83:: with SMTP id t3mr7809433pjd.128.1612433149634; Thu, 04 Feb 2021 02:05:49 -0800 (PST) Received: from hiroh.c.googlers.com.com (128.141.236.35.bc.googleusercontent.com. [35.236.141.128]) by smtp.gmail.com with ESMTPSA id j9sm5808418pgb.47.2021.02.04.02.05.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 02:05:48 -0800 (PST) From: Hirokazu Honda To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Feb 2021 10:05:36 +0000 Message-Id: <20210204100541.657503-2-hiroh@chromium.org> X-Mailer: git-send-email 2.30.0.365.g02bc693789-goog In-Reply-To: <20210204100541.657503-1-hiroh@chromium.org> References: <20210204100541.657503-1-hiroh@chromium.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 1/6] libcamera: camera: Add tryValidate() to CameraConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hanlinchen@chromium.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This adds tryValidate() to CameraConfiguration. The function doesn't change CameraConfiguration status. It dry-runs validate() with passed configurations, so that a caller can know acceptable configurations. Signed-off-by: Hirokazu Honda --- include/libcamera/camera.h | 1 + src/libcamera/camera.cpp | 22 ++++++++++++++++++++++ 2 files changed, 23 insertions(+) -- 2.30.0.365.g02bc693789-goog diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index f94f8599..ac3c7f5a 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -43,6 +43,7 @@ public: void addConfiguration(const StreamConfiguration &cfg); virtual Status validate() = 0; + virtual bool tryValidate(std::vector> &config) const; StreamConfiguration &at(unsigned int index); const StreamConfiguration &at(unsigned int index) const; diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index c9c79b91..6c4ae727 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -171,6 +171,28 @@ void CameraConfiguration::addConfiguration(const StreamConfiguration &cfg) config_.push_back(cfg); } +/** + * \fn CameraConfiguration::tryValidate() + * \brief construct configurations from config, which can be produced + * simultaneously w/o adjust. + * + * This method modifies config to satisfy + * 1.) config[i].second is Invalid or + * 2.) config[i].second is either Valid or Adjust and the config[i].first can be + * accepted by a native camera. validate() returns Valid or Adjust with the + * configurations. + * + * \todo: Remove this tryValidate() in this base class. + */ +bool CameraConfiguration::tryValidate( + std::vector> &config) const +{ + for (auto &c : config) { + c.second = Invalid; + } + return false; +} + /** * \fn CameraConfiguration::validate() * \brief Validate and possibly adjust the camera configuration From patchwork Thu Feb 4 10:05:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hirokazu Honda X-Patchwork-Id: 11140 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A7A33BD162 for ; Thu, 4 Feb 2021 10:05:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 760AC61430; Thu, 4 Feb 2021 11:05:54 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="l0gOpmM0"; dkim-atps=neutral Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2560161402 for ; Thu, 4 Feb 2021 11:05:53 +0100 (CET) Received: by mail-pf1-x42a.google.com with SMTP id w14so1822556pfi.2 for ; Thu, 04 Feb 2021 02:05:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9jWK4b9whbxiv8fx7ANqjpS0eyFsEYZHn6jOiX5aYYA=; b=l0gOpmM0PTpjmnAV2hH5RppMjhSOFh1lp/odhiGY9pewEl1BEkAhQsQPzmRNdMsIOd imiEWddiB+HhrnZCd4l4aJa5GNHRSDhTJbTGB4WlaG0FPWzwcx/iZL6cMxNYhIQlRNbx ZHkAQM4rsHV5/pwg0n4axKfthHASNU0hv442U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9jWK4b9whbxiv8fx7ANqjpS0eyFsEYZHn6jOiX5aYYA=; b=icT2QQIHRUyw8tvZnTG8xMEloPCi7KlkVibpN8Uw6pt9k36qLkiVPz70jSeu9YTi+S e8MAvDl+e4cq6CXKPuGm9aWtKZ6tktKbgvTDwdOw29SgyX+ica/7pYxpauHIviRE0NGX gcW3Gtpt93Qka783B9rWjbpfPLOsjJGlN9tZWIfErhekCuqV1Ga9U4HmX1oOKNR/YzFm nrZa/PFFm9RAhqEjcgZqEcJJ1unDxulyEyD8xaIN4D3x+7AY9TIny2K7PdJK1xUh6KHp 1ocZlXVBMsnHFpG6qjTMhQmIEVnmIKi1xcb8XlLcA6Vqf3HpEryI3dghvE+2ZneXulLi GPvw== X-Gm-Message-State: AOAM530IAaCHu6v5gq4UAb7r4isjLNvQLHk0KJDDpW1vhT3wiYcjatQ1 SMYL2K62WJD9xKkbouMfKQg+LZ4Z7rduoFlf X-Google-Smtp-Source: ABdhPJwu/DDS6qdbLn2wChigESHdwK95GkPJPUtryMsUu5/spgRkdFOcUTOeQUvnmYPKarPdqpsrNg== X-Received: by 2002:a63:3e0c:: with SMTP id l12mr8146527pga.165.1612433151477; Thu, 04 Feb 2021 02:05:51 -0800 (PST) Received: from hiroh.c.googlers.com.com (128.141.236.35.bc.googleusercontent.com. [35.236.141.128]) by smtp.gmail.com with ESMTPSA id j9sm5808418pgb.47.2021.02.04.02.05.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 02:05:50 -0800 (PST) From: Hirokazu Honda To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Feb 2021 10:05:37 +0000 Message-Id: <20210204100541.657503-3-hiroh@chromium.org> X-Mailer: git-send-email 2.30.0.365.g02bc693789-goog In-Reply-To: <20210204100541.657503-1-hiroh@chromium.org> References: <20210204100541.657503-1-hiroh@chromium.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 2/6] libcamera: ipu3: Implement tryValidate() in IPU3CameraConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hanlinchen@chromium.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" IPU3CameraConfiguration::tryValidate() is implemented. It is mostly the same as the validate() implementation. Signed-off-by: Hirokazu Honda --- src/libcamera/pipeline/ipu3/imgu.cpp | 2 +- src/libcamera/pipeline/ipu3/imgu.h | 2 +- src/libcamera/pipeline/ipu3/ipu3.cpp | 160 +++++++++++++++++++++++++++ 3 files changed, 162 insertions(+), 2 deletions(-) -- 2.30.0.365.g02bc693789-goog diff --git a/src/libcamera/pipeline/ipu3/imgu.cpp b/src/libcamera/pipeline/ipu3/imgu.cpp index 5b1c0318..36c5c03c 100644 --- a/src/libcamera/pipeline/ipu3/imgu.cpp +++ b/src/libcamera/pipeline/ipu3/imgu.cpp @@ -377,7 +377,7 @@ int ImgUDevice::init(MediaDevice *media, unsigned int index) * \return An ImgUDevice::PipeConfig instance on success, an empty configuration * otherwise */ -ImgUDevice::PipeConfig ImgUDevice::calculatePipeConfig(Pipe *pipe) +ImgUDevice::PipeConfig ImgUDevice::calculatePipeConfig(Pipe *pipe) const { pipeConfigs.clear(); diff --git a/src/libcamera/pipeline/ipu3/imgu.h b/src/libcamera/pipeline/ipu3/imgu.h index c73ac5a5..4204f50d 100644 --- a/src/libcamera/pipeline/ipu3/imgu.h +++ b/src/libcamera/pipeline/ipu3/imgu.h @@ -43,7 +43,7 @@ public: int init(MediaDevice *media, unsigned int index); - PipeConfig calculatePipeConfig(Pipe *pipe); + PipeConfig calculatePipeConfig(Pipe *pipe) const; int configure(const PipeConfig &pipeConfig, V4L2DeviceFormat *inputFormat); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index db0d6b91..89f1983a 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -73,11 +73,15 @@ public: IPU3CameraConfiguration(IPU3CameraData *data); Status validate() override; + bool tryValidate(std::vector> &config) const override; const StreamConfiguration &cio2Format() const { return cio2Configuration_; } const ImgUDevice::PipeConfig imguConfig() const { return pipeConfig_; } private: + bool tryValidateInternal( + std::vector> &configs, + ImgUDevice::PipeConfig &pipeConfig) const; /* * The IPU3CameraData instance is guaranteed to be valid as long as the * corresponding Camera instance is valid. In order to borrow a @@ -140,6 +144,162 @@ IPU3CameraConfiguration::IPU3CameraConfiguration(IPU3CameraData *data) data_ = data; } + +bool IPU3CameraConfiguration::tryValidateInternal( + std::vector> &configs, + ImgUDevice::PipeConfig &pipeConfig) const +{ + if (configs.empty() || configs.size() > IPU3_MAX_STREAMS) + return false; + + unsigned int rawCount = 0; + unsigned int yuvCount = 0; + Size maxYuvSize; + for (size_t i = 0; i < configs.size(); ++i) { + const auto &cfg = configs[i].first; + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) { + rawCount++; + } else { + yuvCount++; + maxYuvSize.expandTo(cfg.size); + } + } + LOG(IPU3, Debug) << rawCount << ", " << yuvCount; + if (rawCount > 1 || yuvCount > 2) + return false; + + StreamConfiguration cio2Configuration = + data_->cio2_.generateConfiguration({}); + if (!cio2Configuration.pixelFormat.isValid()) + return false; + LOG(IPU3, Debug) << "CIO2 configuration: " << cio2Configuration.toString(); + + ImgUDevice::Pipe pipe{}; + pipe.input = cio2Configuration.size; + + bool mainOutputAvailable = true; + for (auto &[cfg, status] : configs) { + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + const StreamConfiguration originalCfg = cfg; + + LOG(IPU3, Debug) << "Validating " << cfg.toString() + << " to the raw stream"; + if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) { + cfg.size = cio2Configuration.size; + cfg.pixelFormat = cio2Configuration.pixelFormat; + cfg.bufferCount = cio2Configuration.bufferCount; + cfg.stride = info.stride(cfg.size.width, 0, 64); + cfg.frameSize = info.frameSize(cfg.size, 64); + cfg.setStream(const_cast(&data_->rawStream_)); + LOG(IPU3, Debug) << "Assigned " << cfg.toString() + << " to the raw stream"; + } else { + unsigned int limit; + limit = utils::alignDown(cio2Configuration.size.width - 1, + IMGU_OUTPUT_WIDTH_MARGIN); + cfg.size.width = std::clamp(cfg.size.width, + IMGU_OUTPUT_MIN_SIZE.width, + limit); + + limit = utils::alignDown(cio2Configuration.size.height - 1, + IMGU_OUTPUT_HEIGHT_MARGIN); + cfg.size.height = std::clamp(cfg.size.height, + IMGU_OUTPUT_MIN_SIZE.height, + limit); + + cfg.size.alignDownTo(IMGU_OUTPUT_WIDTH_ALIGN, + IMGU_OUTPUT_HEIGHT_ALIGN); + + cfg.pixelFormat = formats::NV12; + cfg.bufferCount = IPU3_BUFFER_COUNT; + cfg.stride = info.stride(cfg.size.width, 0, 1); + cfg.frameSize = info.frameSize(cfg.size, 1); + + if (mainOutputAvailable && + (originalCfg.size == maxYuvSize || yuvCount == 1)) { + cfg.setStream(const_cast(&data_->outStream_)); + mainOutputAvailable = false; + + pipe.main = cfg.size; + if (yuvCount == 1) + pipe.viewfinder = pipe.main; + + LOG(IPU3, Debug) << "Assigned " << cfg.toString() + << " to the main output"; + } else { + cfg.setStream(const_cast(&data_->vfStream_)); + pipe.viewfinder = cfg.size; + + LOG(IPU3, Debug) << "Assigned " << cfg.toString() + << " to the viewfinder output"; + } + } + if (cfg.pixelFormat != originalCfg.pixelFormat || + cfg.size != originalCfg.size) { + LOG(IPU3, Debug) + << "Stream configuration adjusted to " + << cfg.toString(); + + status = Adjusted; + } + } + + if (yuvCount) { + pipeConfig = data_->imgu_->calculatePipeConfig(&pipe); + if (pipeConfig.isNull()) + return false; + } + return true; +} + +bool IPU3CameraConfiguration::tryValidate( + std::vector> &configs) const +{ + if (configs.empty()) + return Invalid; + + struct AcceptedConfig { + StreamConfiguration config; + Status status; + unsigned int index; + }; + std::vector acceptedConfigs; + for (unsigned int i = 0; i < configs.size(); ++i) { + LOG(IPU3, Debug) << "Trying config #" << i + << ": " << configs[i].first.toString(); + std::vector> + tryConfigs(acceptedConfigs.size() + 1); + for (unsigned int j = 0; j < acceptedConfigs.size(); ++j) { + tryConfigs[j].first = acceptedConfigs[j].config; + tryConfigs[j].second = Valid; + } + tryConfigs.back().first = configs[i].first; + tryConfigs.back().second = Valid; + ImgUDevice::PipeConfig tryPipeConfig; + if (!tryValidateInternal(tryConfigs, tryPipeConfig)) + continue; + + for (unsigned int j = 0; j < acceptedConfigs.size(); ++j) { + acceptedConfigs[j].config = tryConfigs[j].first; + acceptedConfigs[j].status = tryConfigs[j].second; + } + acceptedConfigs.push_back( + { tryConfigs[i].first, tryConfigs[i].second, i }); + } + + for (unsigned int i = 0; i < configs.size(); ++i) + configs[i].second = Invalid; + + for (auto &&ac : acceptedConfigs) { + unsigned int i = ac.index; + configs[i].first = std::move(ac.config); + configs[i].second = ac.status; + } + + return !acceptedConfigs.empty(); +} + CameraConfiguration::Status IPU3CameraConfiguration::validate() { Status status = Valid; From patchwork Thu Feb 4 10:05:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hirokazu Honda X-Patchwork-Id: 11141 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 077AEBD162 for ; Thu, 4 Feb 2021 10:05:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CC23F61433; Thu, 4 Feb 2021 11:05:56 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="YN5y9JVg"; dkim-atps=neutral Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2788761402 for ; Thu, 4 Feb 2021 11:05:55 +0100 (CET) Received: by mail-pg1-x52c.google.com with SMTP id z21so1756362pgj.4 for ; Thu, 04 Feb 2021 02:05:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7yERyDLtas5GQ4jIrAxedsZX+9qroAgVoDq37BZqW9A=; b=YN5y9JVgKRLIK+mCIKs4cLBRnsYzwCl0pY+lWoAdcRvjaHUz7oo8Mh32bNeTDWu1dZ HoY0VxwFwZj4o03su1Sj5jt/tz3IyiAJNjwstFpcLPAe1cLxBUMowz7ifWwtEkufsuOp dpDlXGYEV6JnxE93SLyIXuiX6ZLmIHIcnnS+8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7yERyDLtas5GQ4jIrAxedsZX+9qroAgVoDq37BZqW9A=; b=Wyb1bw+d5uPGSkM4who3rvePSJK5+j+WWe04cy8cifQB95GG0z/LK17BdEgcFkO/+H AksQlfO+eM2Wzlbr0epiFJ06R9iCiIQeB4yBJ2tWJ4H2GV378icKIXLYpzWpwRt10WKz 9yaflkm/FsOJwoHznHOYGWmXNN4tgVRQ43BB/s6sVwX8p+EEQ+V6hyJndYlLI0bDtMR+ D5JHyjORE5wYf4HCM30OZmlIO9ALat/mSnbIWiHkauY+IThMGbA7h4Jps4ikVSYlQtDy IioO0iN0HZsyTOX9bbpZ4K+KYRE11+FMwOeNakv3p94O5+Uasg82j5DSXR+c5GDTcRfE 3B5g== X-Gm-Message-State: AOAM531O6StcGdbPwFcGfqLUGzSlko4Dn0o6jT9UMrNxkEYiyAcy4cHC cco/FJ7ixwISsMgYecyDg13EWfOOeb5VPkia X-Google-Smtp-Source: ABdhPJxdRs4jbnDDIKlEXJ2lL3HhxyoJWHCL5ELPWnA7PKmO4dSvMwcS5f1sA1PABkEWe1d2R+9//A== X-Received: by 2002:a63:d5e:: with SMTP id 30mr7428446pgn.178.1612433153227; Thu, 04 Feb 2021 02:05:53 -0800 (PST) Received: from hiroh.c.googlers.com.com (128.141.236.35.bc.googleusercontent.com. [35.236.141.128]) by smtp.gmail.com with ESMTPSA id j9sm5808418pgb.47.2021.02.04.02.05.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 02:05:52 -0800 (PST) From: Hirokazu Honda To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Feb 2021 10:05:38 +0000 Message-Id: <20210204100541.657503-4-hiroh@chromium.org> X-Mailer: git-send-email 2.30.0.365.g02bc693789-goog In-Reply-To: <20210204100541.657503-1-hiroh@chromium.org> References: <20210204100541.657503-1-hiroh@chromium.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 3/6] android: camera_stream: Pass post processor in configure() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hanlinchen@chromium.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" CameraStream creates PostProcessor in its constructor and configures the PostProcessor() in its configure(). The existing code hard-codes them for PostProcessorJpeg. It is difficult to generalize the code for other PostProcessor. A client of CameraStream should know what post processor is needed and how to configure it. Therefore, this changes CameraStream::configure() interface to take a post processor. A client of CameraStream creates and configures a post processor before calling configure(). Signed-off-by: Hirokazu Honda --- src/android/camera_device.cpp | 21 ++++++++++++++++++++- src/android/camera_stream.cpp | 31 ++++++++----------------------- src/android/camera_stream.h | 2 +- 3 files changed, 29 insertions(+), 25 deletions(-) -- 2.30.0.365.g02bc693789-goog diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index a50b0ebf..752e17ba 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -8,6 +8,7 @@ #include "camera_device.h" #include "camera_ops.h" #include "post_processor.h" +#include "jpeg/post_processor_jpeg.h" #include #include @@ -1613,7 +1614,25 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) * the Android camera3_stream_t. */ for (CameraStream &cameraStream : streams_) { - ret = cameraStream.configure(); + std::unique_ptr postProcessor; + + if (cameraStream.type() == CameraStream::Type::Internal || + cameraStream.type() == CameraStream::Type::Mapped) { + postProcessor = + std::make_unique(this); + + auto output = cameraStream.configuration(); + output.pixelFormat = formats::MJPEG; + postProcessor->configure(cameraStream.configuration(), + output); + if (ret) { + LOG(HAL, Error) << "Failed to configure " + << "PostProcessorJpeg"; + return ret; + } + } + + ret = cameraStream.configure(std::move(postProcessor)); if (ret) { LOG(HAL, Error) << "Failed to configure camera stream"; return ret; diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 611ec0d1..ed6c5265 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -9,7 +9,7 @@ #include "camera_device.h" #include "camera_metadata.h" -#include "jpeg/post_processor_jpeg.h" +#include "post_processor.h" #include @@ -38,22 +38,13 @@ LOG_DECLARE_CATEGORY(HAL) * and buffer allocation. */ -CameraStream::CameraStream(CameraDevice *const cameraDevice, Type type, - camera3_stream_t *camera3Stream, unsigned int index) +CameraStream::CameraStream(CameraDevice *const cameraDevice, + Type type, camera3_stream_t *camera3Stream, + unsigned int index) : cameraDevice_(cameraDevice), - config_(cameraDevice->cameraConfiguration()), type_(type), - camera3Stream_(camera3Stream), index_(index) + config_(cameraDevice->cameraConfiguration()), + type_(type), camera3Stream_(camera3Stream), index_(index) { - if (type_ == Type::Internal || type_ == Type::Mapped) { - /* - * \todo There might be multiple post-processors. The logic - * which should be instantiated here, is deferred for the - * future. For now, we only have PostProcessorJpeg and that - * is what we instantiate here. - */ - postProcessor_ = std::make_unique(cameraDevice_); - } - if (type == Type::Internal) { allocator_ = std::make_unique(cameraDevice_->camera()); mutex_ = std::make_unique(); @@ -70,15 +61,9 @@ Stream *CameraStream::stream() const return configuration().stream(); } -int CameraStream::configure() +int CameraStream::configure(std::unique_ptr postProcessor) { - if (postProcessor_) { - StreamConfiguration output = configuration(); - output.pixelFormat = formats::MJPEG; - int ret = postProcessor_->configure(configuration(), output); - if (ret) - return ret; - } + postProcessor_ = std::move(postProcessor); if (allocator_) { int ret = allocator_->allocate(stream()); diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index fc242b2a..faafc934 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -118,7 +118,7 @@ public: const libcamera::StreamConfiguration &configuration() const; libcamera::Stream *stream() const; - int configure(); + int configure(std::unique_ptr postProcessor); int process(const libcamera::FrameBuffer &source, libcamera::MappedBuffer *destination, const CameraMetadata &requestMetadata, From patchwork Thu Feb 4 10:05:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hirokazu Honda X-Patchwork-Id: 11142 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5A702BD162 for ; Thu, 4 Feb 2021 10:05:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 278C86143D; Thu, 4 Feb 2021 11:05:59 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="DKcfUn8Q"; dkim-atps=neutral Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AFFAF61402 for ; Thu, 4 Feb 2021 11:05:56 +0100 (CET) Received: by mail-pg1-x52d.google.com with SMTP id i7so1741538pgc.8 for ; Thu, 04 Feb 2021 02:05:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MXSqpr9JOnO1UjiOWF8Ii2WAsULHhudnBjado3OF8Ys=; b=DKcfUn8Q812/OXWA7nkOjvzi4BTpmhvL285pZ4E3vaF1Vcr1AASSg+nbx7j51D20EY E5vV3QEVuGXS1Or3ZMSeTdncP1QS6ohia+POv0kKDyybK7QIFLya3IBuEjH0j1Lperth LN/pTiaAS4nLN1PNMofE6zaYc3EkJkZk8Vfho= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MXSqpr9JOnO1UjiOWF8Ii2WAsULHhudnBjado3OF8Ys=; b=qPw8nwQ6ddhDAOw/TWbYvP9qCRXeeew9qpLyyE/27qz4FMh3Nv6zUGx/KagqhsECFT r63r4qRvHtyGNEMGppMyJ83nfQtfvvjTdmGrmqWvpNve06FVEmLVel7z6X33hqQV3qBK ckXKR2Cbks37cnvAW0OnZA3xzzAoaalN7YG+ioULZp6VsXU8WKt1CudD0eH2nW3WYPRL kZK2oeGl5jxZnPfLvWaQ9yKBpFFYF4Nsv4NRWcOsubZSRI6ulRd3yOmkxxx8+a/2bNi4 5I8FWsno3rAR8vmofG9yNdEVXa7QrSBVzdPy8JOOwxIpeeCpFEOqnlhYRSCTc8E65Bc4 H1YA== X-Gm-Message-State: AOAM532GebaPcfOL4OHrH33WvMeSbrW2CVHhbk2rApWdgGkXr2df9jB5 MkCBarmn1N8YK+XYzYCHyu7XBvD4ntxbhTtr X-Google-Smtp-Source: ABdhPJzIr0NUkcQA6A9WWJlB5dFzi6W/PgKsJnQa3xaB7JJah6nPuYcxmjCk5uKdcnbMUt5hqCdZHA== X-Received: by 2002:a62:1755:0:b029:1cb:4985:4cf8 with SMTP id 82-20020a6217550000b02901cb49854cf8mr7631415pfx.9.1612433155161; Thu, 04 Feb 2021 02:05:55 -0800 (PST) Received: from hiroh.c.googlers.com.com (128.141.236.35.bc.googleusercontent.com. [35.236.141.128]) by smtp.gmail.com with ESMTPSA id j9sm5808418pgb.47.2021.02.04.02.05.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 02:05:54 -0800 (PST) From: Hirokazu Honda To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Feb 2021 10:05:39 +0000 Message-Id: <20210204100541.657503-5-hiroh@chromium.org> X-Mailer: git-send-email 2.30.0.365.g02bc693789-goog In-Reply-To: <20210204100541.657503-1-hiroh@chromium.org> References: <20210204100541.657503-1-hiroh@chromium.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 4/6] android: camera_device: Implement toString() in Camera3StreamConfig X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hanlinchen@chromium.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This adds toString() function in Camera3StreamConfig and Camera3StreamConfig::Camera3Stream for logging. Signed-off-by: Hirokazu Honda --- src/android/camera_device.cpp | 38 +++++++++++++++++++++++++++++++++-- 1 file changed, 36 insertions(+), 2 deletions(-) -- 2.30.0.365.g02bc693789-goog diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 752e17ba..93dfbcdb 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -140,14 +140,48 @@ const std::map camera3FormatsMap = { */ struct Camera3StreamConfig { struct Camera3Stream { - camera3_stream_t *stream; - CameraStream::Type type; + Camera3Stream(camera3_stream_t *stream, CameraStream::Type type) + : stream(stream), type(type) {} + + std::string toString() const; + + camera3_stream_t *stream = nullptr; + CameraStream::Type type = CameraStream::Type::Direct; }; + std::string toString() const; + std::vector streams; StreamConfiguration config; }; +std::string Camera3StreamConfig::Camera3Stream::toString() const { + std::ostringstream os; + os << "format: " << utils::hex(stream->format) + << ", size:" << stream->width << "x" << stream->height; + os << ", type: "; + switch (type) { + case CameraStream::Type::Direct: + os << "Direct"; break; + case CameraStream::Type::Internal: + os << "Internal"; break; + case CameraStream::Type::Mapped: + os << "Mapped"; break; + } + + return os.str(); +} + +std::string Camera3StreamConfig::toString() const { + std::ostringstream os; + for (size_t i = 0; i < streams.size(); ++i) { + os << "Stream #" << i << ": " << streams[i].toString() << "}, "; + } + + os << "config: " << config.toString(); + return os.str(); +} + /* * Reorder the configurations so that libcamera::Camera can accept them as much * as possible. The sort rule is as follows. From patchwork Thu Feb 4 10:05:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hirokazu Honda X-Patchwork-Id: 11143 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 049DFBD162 for ; Thu, 4 Feb 2021 10:06:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C45286142D; Thu, 4 Feb 2021 11:05:59 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="oQLd+yoN"; dkim-atps=neutral Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B556461438 for ; Thu, 4 Feb 2021 11:05:58 +0100 (CET) Received: by mail-pj1-x1036.google.com with SMTP id l18so1495794pji.3 for ; Thu, 04 Feb 2021 02:05:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=efVFGRlmisrEE/b5fPi2T6th4c/o9G0mdKmW6grdBQg=; b=oQLd+yoNW39WURWHzc5JbuBc9jOfCg/hNVOPuHiCqEoPgTx4d1SB2Te+v1UuaLr6QK b3hxGhNVU/86JKijyV52NwEFPTZbPHVs9QgGSUKhKVMkO/PFZsl2Td7HoRejxPcLRIMv I1N2OSsIF7zzGJLxpHsGLkNlxJcTC9NY+Zsn0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=efVFGRlmisrEE/b5fPi2T6th4c/o9G0mdKmW6grdBQg=; b=uIgy7XImj1sAsdtfygliE/ERmnH8OkEgpBwZvQcicn/KvFF8cq9JvLT/3pzmItCz+Q MV/2Bzztxm2fZ4EwQbdXypyaJG+M6mpYcGatZeh7dSZy6HiY7FjL9Oun0vmrHLspOaEt o1JJ/HDabkCjAAb0K/+cjebh144Xu1ZZxgJI2koUK3IlWtUJ7W9LUXWUR8ZtbHtuFtMu VfKCNhmHuzM7mzJM0IsTtm+4Q66SSdYrGsuLOyx9jZBdNi+ilq7QpRc/2/E3Ufa33+MW KBMndYzER5tLE5YriNoVrC8A9WHJ2ICILZfPjWskuAIcjrp8CyraGJ1TR8PYU8TjLeTy Gn7A== X-Gm-Message-State: AOAM532jzbzSoJVbraF7hgJJaz2OhktvrTnN4IrvFmV0zaM5q4Lf9fvw qfvynTawFR6d2KVdXlnQmqZIdAUSgN6lTLQm X-Google-Smtp-Source: ABdhPJzn5/YFWLWLTJyvEy8zPtlx4CGuGTklpYveBU1/wIcJdvYMrF8i84TRsH9YIYA/Fc4THcTYEw== X-Received: by 2002:a17:90a:df46:: with SMTP id gy6mr7566701pjb.163.1612433157019; Thu, 04 Feb 2021 02:05:57 -0800 (PST) Received: from hiroh.c.googlers.com.com (128.141.236.35.bc.googleusercontent.com. [35.236.141.128]) by smtp.gmail.com with ESMTPSA id j9sm5808418pgb.47.2021.02.04.02.05.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 02:05:56 -0800 (PST) From: Hirokazu Honda To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Feb 2021 10:05:40 +0000 Message-Id: <20210204100541.657503-6-hiroh@chromium.org> X-Mailer: git-send-email 2.30.0.365.g02bc693789-goog In-Reply-To: <20210204100541.657503-1-hiroh@chromium.org> References: <20210204100541.657503-1-hiroh@chromium.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 5/6] android: camera_device: Introduce PostProcessorType X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hanlinchen@chromium.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This introduces PostProcessorType. It tracks the required post processor for CameraStream. Signed-off-by: Hirokazu Honda --- src/android/camera_device.cpp | 80 +++++++++++++++++++++++++++++------ 1 file changed, 68 insertions(+), 12 deletions(-) -- 2.30.0.365.g02bc693789-goog diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 93dfbcdb..a1ef07b6 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -9,6 +9,7 @@ #include "camera_ops.h" #include "post_processor.h" #include "jpeg/post_processor_jpeg.h" +#include "yuv/post_processor_yuv.h" #include #include @@ -131,6 +132,17 @@ const std::map camera3FormatsMap = { }, }; +/* PostProcessorType represents a post processor class. + * None: No post processor + * Jpeg: Jpeg encoding post processor + * Yuv : Yuv post post processor. + */ +enum class PostProcessorType : uint8_t { + None = 0, + Jpeg = 1, + Yuv = 2, +}; + /* * \struct Camera3StreamConfig * \brief Data to store StreamConfiguration associated with camera3_stream(s) @@ -142,11 +154,16 @@ struct Camera3StreamConfig { struct Camera3Stream { Camera3Stream(camera3_stream_t *stream, CameraStream::Type type) : stream(stream), type(type) {} + Camera3Stream(camera3_stream_t *stream, CameraStream::Type type, + PostProcessorType postProcessorType) + : stream(stream), type(type), + postProcessorType(postProcessorType) {} std::string toString() const; camera3_stream_t *stream = nullptr; CameraStream::Type type = CameraStream::Type::Direct; + PostProcessorType postProcessorType = PostProcessorType::None; }; std::string toString() const; @@ -169,6 +186,15 @@ std::string Camera3StreamConfig::Camera3Stream::toString() const { os << "Mapped"; break; } + switch (postProcessorType) { + case PostProcessorType::None: + os << "None"; break; + case PostProcessorType::Jpeg: + os << "Jpeg"; break; + case PostProcessorType::Yuv: + os << "Yuv"; break; + } + return os.str(); } @@ -1554,6 +1580,7 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) /* Now handle the MJPEG streams, adding a new stream if required. */ if (jpegStream) { CameraStream::Type type; + PostProcessorType postProcessorType = PostProcessorType::None; int index = -1; /* Search for a compatible stream in the non-JPEG ones. */ @@ -1573,6 +1600,7 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) type = CameraStream::Type::Mapped; index = i; + postProcessorType = PostProcessorType::Jpeg; break; } @@ -1600,14 +1628,18 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) index = streamConfigs.size() - 1; } - streamConfigs[index].streams.push_back({ jpegStream, type }); + streamConfigs[index].streams.push_back( + { jpegStream, type, postProcessorType }); } sortCamera3StreamConfigs(streamConfigs, jpegStream); + + std::vector postProcessorTypes; for (const auto &streamConfig : streamConfigs) { config_->addConfiguration(streamConfig.config); for (auto &stream : streamConfig.streams) { + postProcessorTypes.push_back(stream.postProcessorType); streams_.emplace_back(this, stream.type, stream.stream, config_->size() - 1); stream.stream->priv = static_cast(&streams_.back()); @@ -1647,21 +1679,46 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) * StreamConfiguration and set the number of required buffers in * the Android camera3_stream_t. */ - for (CameraStream &cameraStream : streams_) { + for (unsigned int i = 0; i < streams_.size(); ++i) { + CameraStream &cameraStream = streams_[i]; std::unique_ptr postProcessor; - if (cameraStream.type() == CameraStream::Type::Internal || - cameraStream.type() == CameraStream::Type::Mapped) { + switch(postProcessorTypes[i]) { + case PostProcessorType::None: + break; + case PostProcessorType::Jpeg: postProcessor = std::make_unique(this); + break; + case PostProcessorType::Yuv: + postProcessor = std::make_unique(); + break; + } + + if (postProcessor) { + const auto &camera3Stream = cameraStream.camera3Stream(); + StreamConfiguration output; + switch (camera3Stream.format) { + case HAL_PIXEL_FORMAT_BLOB: + output.pixelFormat = formats::MJPEG; + break; + case HAL_PIXEL_FORMAT_YCbCr_420_888: + output.pixelFormat = formats::NV12; + break; + default: + LOG(HAL, Error) + << "Unexpected format: " + << utils::hex(camera3Stream.format); + break; + } - auto output = cameraStream.configuration(); - output.pixelFormat = formats::MJPEG; - postProcessor->configure(cameraStream.configuration(), - output); + output.size.width = camera3Stream.width; + output.size.height = camera3Stream.height; + ret = postProcessor->configure( + cameraStream.configuration(), output); if (ret) { - LOG(HAL, Error) << "Failed to configure " - << "PostProcessorJpeg"; + LOG(HAL, Error) + << "Failed to configure post processor"; return ret; } } @@ -1871,12 +1928,11 @@ void CameraDevice::requestComplete(Request *request) uint64_t timestamp = buffers.begin()->second->metadata().timestamp; resultMetadata = getResultMetadata(descriptor, timestamp); - /* Handle any JPEG compression. */ for (unsigned int i = 0; i < descriptor->numBuffers_; ++i) { CameraStream *cameraStream = static_cast(descriptor->buffers_[i].stream->priv); - if (cameraStream->camera3Stream().format != HAL_PIXEL_FORMAT_BLOB) + if (cameraStream->type() == CameraStream::Type::Direct) continue; FrameBuffer *buffer = request->findBuffer(cameraStream->stream()); From patchwork Thu Feb 4 10:05:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hirokazu Honda X-Patchwork-Id: 11144 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5B8F1BD162 for ; Thu, 4 Feb 2021 10:06:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2B11861435; Thu, 4 Feb 2021 11:06:04 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="HhMs6b7F"; dkim-atps=neutral Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 549DD61402 for ; Thu, 4 Feb 2021 11:06:02 +0100 (CET) Received: by mail-pj1-x1031.google.com with SMTP id q72so1419428pjq.2 for ; Thu, 04 Feb 2021 02:06:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4piughn7EJr6AAA84UOv5h4/uYmAf0s9228/+iPEiJg=; b=HhMs6b7FtEro1fVhmic20BtTV+zER88RXUl0EDj7sjtAQ3IMR8qKcIIvjaWPsM+iX/ 3/DQ9thzd1iOn33T7WR52JKAqgW6/soj87kuAN4y85BTarpMZUDPnmNYf117+32PYQ26 Da6FSfPXtMNzvOY79WnlYtwqi4ysninyjD6bc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4piughn7EJr6AAA84UOv5h4/uYmAf0s9228/+iPEiJg=; b=XpWCamwuFVn4UmjQVTEO5GJ7tBL7fucOJjo3AXC+ppR9EFbVJZLLOThur9wujpLp5T 5lsfmQt4k4SbJajGeLht+EUae1xrb8o3/BvBj0NJKJMQCk7+i72YLbOzYBjkvf7hcchS 6oKZ4Q/KD3khIYxS/5176naRBfuq5CP/T/heX2tXN11vWaP5y16t8nVgA5eeQfmhFgXa 6NTQCYPSYCMNww4FbOTZ6/+pYkg78NZNVzYyUUN4l4h7Wdwiy2sBVsgtKhc905Db18nG bPDWEiBMMSrtcfnCQpY7itS7J7/5DKu6790yrO2hC11ZCH+zaevOa5mKFTlXz0hqObAW UgZA== X-Gm-Message-State: AOAM5329Nrf7EIQOajP1T5DdcM7MnJyJbRi/iSWuk9WYhm5ATCbLpN4o a96rb0+EZRmy0L7wakWSn3JtRfZlY3D3fauy X-Google-Smtp-Source: ABdhPJyD4GEcoRG1ddnRGo7ErQKScmOxv9J5+THseF6PKm3Y0cfaPcPexudId9X+Udf1vBN26DtZeg== X-Received: by 2002:a17:902:6b43:b029:df:fb48:aece with SMTP id g3-20020a1709026b43b02900dffb48aecemr7483343plt.59.1612433160729; Thu, 04 Feb 2021 02:06:00 -0800 (PST) Received: from hiroh.c.googlers.com.com (128.141.236.35.bc.googleusercontent.com. [35.236.141.128]) by smtp.gmail.com with ESMTPSA id j9sm5808418pgb.47.2021.02.04.02.05.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 02:05:59 -0800 (PST) From: Hirokazu Honda To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Feb 2021 10:05:41 +0000 Message-Id: <20210204100541.657503-7-hiroh@chromium.org> X-Mailer: git-send-email 2.30.0.365.g02bc693789-goog In-Reply-To: <20210204100541.657503-1-hiroh@chromium.org> References: <20210204100541.657503-1-hiroh@chromium.org> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 6/6] android: camera_device: Support stream mapping X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hanlinchen@chromium.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This add the support of stream mapping in Android HAL adaptation layer. It helps to produce a stream that cannot be output by a native camera directly. Signed-off-by: Hirokazu Honda --- src/android/camera_device.cpp | 143 ++++++++++++++++++++++++++++++++++ 1 file changed, 143 insertions(+) -- 2.30.0.365.g02bc693789-goog diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index a1ef07b6..d45f83c1 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -316,6 +316,132 @@ void sortCamera3StreamConfigs(std::vector &unsortedConfigs, unsortedConfigs = sortedConfigs; } +bool canConvert(const StreamConfiguration &srcConfig, + const camera3_stream_t &dstStream, + CameraStream::Type &type, + PostProcessorType &postProcessorType) +{ + /* NV12 -> JPEG conversion. */ + if (srcConfig.pixelFormat == formats::NV12 && + dstStream.format == HAL_PIXEL_FORMAT_BLOB && + dstStream.width == srcConfig.size.width && + dstStream.height == srcConfig.size.height) { + type = CameraStream::Type::Mapped; + postProcessorType = PostProcessorType::Jpeg; + return true; + } + + /* NV12 -> NV12 */ + if (srcConfig.pixelFormat == formats::NV12 && + (dstStream.format == HAL_PIXEL_FORMAT_YCbCr_420_888 || + dstStream.format == HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED)) { + if (dstStream.width == srcConfig.size.width && + dstStream.height == srcConfig.size.height) { + /* Duplicated stream. No post processing is required. */ + type = CameraStream::Type::Direct; + postProcessorType = PostProcessorType::None; + return true; + } else if(dstStream.width < srcConfig.size.width && + dstStream.height < srcConfig.size.height) { + /* Down scaling. */ + type = CameraStream::Type::Mapped; + postProcessorType = PostProcessorType::Yuv; + return true; + } + } + return false; +} + +bool mapConfig( + std::vector &streamConfigs, + const std::vector> &tryConfigs) +{ + const StreamConfiguration kInvalidStreamConfiguration{}; + assert(!kInvalidStreamConfiguration.pixelFormat.isValid()); + std::vector dstStreams; + for (size_t i = 0; i < tryConfigs.size(); i++) { + LOG(HAL, Debug) << "StreamConfiguration #" << i << ": " + << tryConfigs[i].first.toString() + << ", tryValidate() result =" + << static_cast(tryConfigs[i].second); + if (tryConfigs[i].second == CameraConfiguration::Valid) + continue; + + for (auto &stream : streamConfigs[i].streams) + dstStreams.push_back(stream.stream); + streamConfigs[i].streams.clear(); + + if (tryConfigs[i].second == CameraConfiguration::Adjusted) { + streamConfigs[i].config = + std::move(tryConfigs[i].first); + } else { + /* Set invalid StreamConfiguration for a + * CameraConfiguration::Invalid stream. + */ + streamConfigs[i].config = kInvalidStreamConfiguration; + } + } + + /* Remove elements in streamConfigs whose stream is evaluated as + * CameraConfiguration::Invalid. + */ + streamConfigs.erase( + std::remove_if(streamConfigs.begin(), streamConfigs.end(), + [](const auto &cfg) { + return !cfg.config.pixelFormat.isValid(); + }), + streamConfigs.end()); + + /* Find a new source StreamConfiguration from which |dstStream| can be + * produced. + */ + for (auto *const dstStream : dstStreams) { + auto bestPostProcessorType = PostProcessorType::None; + auto bestType = CameraStream::Type::Mapped; + Camera3StreamConfig *bestConfig = nullptr; + + for (auto &streamConfig : streamConfigs) { + auto type = CameraStream::Type::Mapped; + auto postProcessorType = PostProcessorType::None; + + if (canConvert(streamConfig.config, *dstStream, type, + postProcessorType) && + static_cast(type) < static_cast(bestType)) { + bestType = type; + bestPostProcessorType = postProcessorType; + bestConfig = &streamConfig; + } + } + + if (!bestConfig) { + LOG(HAL, Error) + << "No convertible native stream for the stream" + << ", width: " << dstStream->width + << ", height: " << dstStream->height + << ", format: " << utils::hex(dstStream->format); + return false; + } + + Camera3StreamConfig::Camera3Stream stream( + dstStream, bestType, bestPostProcessorType); + LOG(HAL, Error) + << " Mapped to StreamConfiguration: " + << bestConfig->config.toString() + << " from " << stream.toString(); + bestConfig->streams.push_back(std::move(stream)); + } + + /* Remove elements in streamConfigs whose streams will not be used.*/ + streamConfigs.erase( + std::remove_if(streamConfigs.begin(), streamConfigs.end(), + [](const auto &cfg) { + return !cfg.config.pixelFormat.isValid(); + }), + streamConfigs.end()); + return true; +} + } /* namespace */ MappedCamera3Buffer::MappedCamera3Buffer(const buffer_handle_t camera3buffer, @@ -1634,6 +1760,23 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) sortCamera3StreamConfigs(streamConfigs, jpegStream); + std::vector> + tryConfigs(streamConfigs.size()); + for (size_t i = 0; i < streamConfigs.size(); ++i) { + LOG(HAL, Debug) << "Camera3StreamConfig #" << i << ": " + << streamConfigs[i].toString(); + tryConfigs[i].first = streamConfigs[i].config; + tryConfigs[i].second = CameraConfiguration::Valid; + } + if (!config_->tryValidate(tryConfigs)) { + LOG(HAL, Error) << "Camera configuration invalid"; + return -EINVAL; + } + if (!mapConfig(streamConfigs, tryConfigs)) { + LOG(HAL, Error) << "Failed finding a stream mapping"; + return -EINVAL; + } + std::vector postProcessorTypes; for (const auto &streamConfig : streamConfigs) { config_->addConfiguration(streamConfig.config);