From patchwork Wed Feb 9 07:19:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanlin Chen X-Patchwork-Id: 15353 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2A128C3262 for ; Wed, 9 Feb 2022 07:19:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 79D3D610F9; Wed, 9 Feb 2022 08:19:45 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="QppWawhO"; dkim-atps=neutral Received: from mail-pf1-x430.google.com (mail-pf1-x430.google.com [IPv6:2607:f8b0:4864:20::430]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DC613610F9 for ; Wed, 9 Feb 2022 08:19:42 +0100 (CET) Received: by mail-pf1-x430.google.com with SMTP id e3so1386115pfd.12 for ; Tue, 08 Feb 2022 23:19:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dw278bQHV6XJPfx2edmz6pUTSApQY/UOPFBqZmgNrek=; b=QppWawhO7zG3NhY2F8zdJbHSeQizU9jZYId9dSObubxdIuvh+xHnhH6QOnZ1GM9tke HVphkeeIUq/bG+n94OQ/lqK/GfdwBuCPT5zTsj320CZ+8ZSUOEKR0xMa6l0reLyjB3gv FFpQSe9IPDYAbstcLuCg8lJByZVxdOg/4cLd8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dw278bQHV6XJPfx2edmz6pUTSApQY/UOPFBqZmgNrek=; b=miof8n+0C5qGU5qUaHG2DujqFkcYoWtLUEKkl3zgZZLcGvJLojaAON+ogf/otHEuJP D2kgyIILvAqEDLd82m4Y3F5/OKWJ0edluMTvDep9NQutI3jNyYcYXYWsA/mIHr7OeEwk axGPf0E01pTI2cQjonkjC6SJItbkaHSPTutEBf8i7HzAtj2UFX/zxY9JN149AHD8uH7u tOrtUe5G1BoqoFcFQbmUr9/gsTUS7kf/0PHCbH2QejyTRztyvZp+Lj+IUiywzYVpGtUy /GyswAncKzPL+mk1PqwPV42voiiZIPA5G9IjDPP1bdJsBrk8C7cK0Mp4vYN30RfpUERP Gmbg== X-Gm-Message-State: AOAM531yY4xe3s+rcyvbcJtvr93mVa8+F0z94Yf4YmiMaR/dXU8SeV04 mfxqP132kxa/mSbcvbytYodnqFNKhcknsQ== X-Google-Smtp-Source: ABdhPJxKdJ+mk8jzyU4jNuQymWfZPpjj1JirZsazW+xhGfReubiUqPhhtIVGTycWuSCby8Tp/qFkrw== X-Received: by 2002:a62:b618:: with SMTP id j24mr998678pff.42.1644391180743; Tue, 08 Feb 2022 23:19:40 -0800 (PST) Received: from localhost ([2401:fa00:1:10:a5cf:9a43:1c6d:824]) by smtp.gmail.com with UTF8SMTPSA id my18sm4993033pjb.57.2022.02.08.23.19.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 08 Feb 2022 23:19:40 -0800 (PST) From: Han-Lin Chen To: libcamera-devel@lists.libcamera.org Date: Wed, 9 Feb 2022 15:19:16 +0800 Message-Id: <20220209071917.559993-9-hanlinchen@chromium.org> X-Mailer: git-send-email 2.35.0.263.gb82422642f-goog In-Reply-To: <20220209071917.559993-1-hanlinchen@chromium.org> References: <20220209071917.559993-1-hanlinchen@chromium.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 8/9] libcamera: ipu3: Support PipelineConfigFile option X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add PipelineConfigFile option for ipu3, which supports setting a yaml configuration file which contains preferred ImgU configurations when certain output resolutions are requested. This is a supplement of the current calculation designed by https://github.com/intel/intel-ipu3-pipecfg. This changes the result of IPU3CameraConfiguration::validate() and PipelineHandlerIPU3::generateConfiguration(), and only validates stream configurations defined in the file. The user can cancel the option by calling setOptions() again without the option. Signed-off-by: Han-Lin Chen --- src/libcamera/pipeline/ipu3/ipu3.cpp | 215 ++++++++++++++++++++++++++- 1 file changed, 214 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 6c5617cd..2d43b760 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -16,6 +16,7 @@ #include #include +#include #include #include #include @@ -35,11 +36,16 @@ #include "cio2.h" #include "frames.h" #include "imgu.h" +#include "pipe_config_pref.h" namespace libcamera { LOG_DEFINE_CATEGORY(IPU3) +static const ControlInfoMap::Map IPU3Options = { + { &options::PipelineConfigFile, ControlInfo(0, 512, std::string()) }, +}; + static const ControlInfoMap::Map IPU3Controls = { { &controls::draft::PipelineDepth, ControlInfo(2, 3) }, }; @@ -62,6 +68,11 @@ public: void cancelPendingRequests(); void frameStart(uint32_t sequence); + bool hasPreference() const + { + return pipeConfigPreference_.isValid(); + } + CIO2Device cio2_; ImgUDevice *imgu_; @@ -85,6 +96,8 @@ public: ControlInfoMap ipaControls_; + PipeConfigPreference pipeConfigPreference_; + private: void queueFrameAction(unsigned int id, const ipa::ipu3::IPU3Action &action); @@ -107,6 +120,10 @@ public: Transform combinedTransform_; private: + Status adjustTransform(); + Status validateByCalulation(); + Status validateByPipeConfigPreference(); + /* * The IPU3CameraData instance is guaranteed to be valid as long as the * corresponding Camera instance is valid. In order to borrow a @@ -131,6 +148,8 @@ public: PipelineHandlerIPU3(CameraManager *manager); + int setOptions(Camera *camera, const ControlList *options) override; + CameraConfiguration *generateConfiguration(Camera *camera, const StreamRoles &roles) override; int configure(Camera *camera, CameraConfiguration *config) override; @@ -151,6 +170,7 @@ private: return static_cast(camera->_d()); } + int initOptions(IPU3CameraData *data); int initControls(IPU3CameraData *data); int updateControls(IPU3CameraData *data); int registerCameras(); @@ -158,6 +178,11 @@ private: int allocateBuffers(Camera *camera); int freeBuffers(Camera *camera); + CameraConfiguration *generateConfigurationByCalculation(Camera *camera, + const StreamRoles &roles); + CameraConfiguration *generateConfigurationByPreference(Camera *camera, + const StreamRoles &roles); + ImgUDevice imgu0_; ImgUDevice imgu1_; MediaDevice *cio2MediaDev_; @@ -172,7 +197,7 @@ IPU3CameraConfiguration::IPU3CameraConfiguration(IPU3CameraData *data) data_ = data; } -CameraConfiguration::Status IPU3CameraConfiguration::validate() +CameraConfiguration::Status IPU3CameraConfiguration::adjustTransform() { Status status = Valid; @@ -221,6 +246,25 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() * apply to the sensor to save us working it out again. */ combinedTransform_ = combined; + return status; +} + +CameraConfiguration::Status IPU3CameraConfiguration::validate() +{ + if (data_->hasPreference()) + return validateByPipeConfigPreference(); + + return validateByCalulation(); +} + +CameraConfiguration::Status IPU3CameraConfiguration::validateByCalulation() +{ + Status status = Valid; + + if (config_.empty()) + return Invalid; + + status = adjustTransform(); /* Cap the number of entries to the available streams. */ if (config_.size() > kMaxStreams) { @@ -412,13 +456,118 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() return status; } +CameraConfiguration::Status IPU3CameraConfiguration::validateByPipeConfigPreference() +{ + if (config_.empty() || config_.size() > 2) + return Invalid; + + Status status = adjustTransform(); + + bool mainAvailable = false; + StreamConfiguration *mainStream = nullptr; + StreamConfiguration *viewfinderStream = nullptr; + + /* Assign the mainStream as the stream with the larger size. */ + for (StreamConfiguration &cfg : config_) { + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + if (info.colourEncoding != PixelFormatInfo::ColourEncodingYUV) { + return Invalid; + } + if (!mainAvailable) { + mainStream = &cfg; + } else if (mainStream->size < cfg.size) { + viewfinderStream = mainStream; + mainStream = &cfg; + } + } + + /* + * \todo Query and use the configuration for the still pipe when the + * pipeline handler is refactored to use both ImgU device for a camera. + */ + PipeConfigPreference::PipeConfig videoPipe; + PipeConfigPreference::PipeConfig stillPipe; + + int ret = data_->pipeConfigPreference_.queryPipeConfig( + mainStream->size, + (viewfinderStream) ? viewfinderStream->size : PipeConfigPreference::Disabled, + PipeConfigPreference::Disabled, + PipeConfigPreference::Disabled, + videoPipe, stillPipe); + if (ret) { + LOG(IPU3, Error) + << "Fail to find a valid configuration: " + << " main: " << mainStream->size + << " viewfinder: " + << ((viewfinderStream) ? viewfinderStream->size : PipeConfigPreference::Disabled); + return Invalid; + } + + cio2Configuration_.size = videoPipe.cio2; + cio2Configuration_.pixelFormat = formats::SGRBG10_IPU3; + cio2Configuration_.bufferCount = kBufferCount; + + pipeConfig_.bds = videoPipe.bds; + pipeConfig_.iif = videoPipe.iff; + pipeConfig_.gdc = videoPipe.gdc; + + const PixelFormatInfo &info = PixelFormatInfo::info(formats::NV12); + + mainStream->pixelFormat = formats::NV12; + mainStream->bufferCount = kBufferCount; + mainStream->stride = info.stride(mainStream->size.width, 0, 1); + mainStream->frameSize = info.frameSize(mainStream->size, 1); + mainStream->setStream(const_cast(&data_->outStream_)); + + if (viewfinderStream) { + viewfinderStream->pixelFormat = formats::NV12; + viewfinderStream->bufferCount = kBufferCount; + viewfinderStream->stride = info.stride(viewfinderStream->size.width, 0, 1); + viewfinderStream->frameSize = info.frameSize(viewfinderStream->size, 1); + viewfinderStream->setStream(const_cast(&data_->vfStream_)); + } + + return status; +} + PipelineHandlerIPU3::PipelineHandlerIPU3(CameraManager *manager) : PipelineHandler(manager), cio2MediaDev_(nullptr), imguMediaDev_(nullptr) { } +int PipelineHandlerIPU3::setOptions(Camera *camera, + const ControlList *options) +{ + IPU3CameraData *data = cameraData(camera); + + PipeConfigPreference &preference = data->pipeConfigPreference_; + preference.invalid(); + + if (options && options->contains(options::PipelineConfigFile)) { + std::string path = options->get(options::PipelineConfigFile); + int ret = preference.parsePreferenceFile(path); + if (ret) { + LOG(IPU3, Error) << "Fail to load pipline config " + << "preference file: " << path; + return -EINVAL; + } + } + + return 0; +} + CameraConfiguration *PipelineHandlerIPU3::generateConfiguration(Camera *camera, const StreamRoles &roles) +{ + IPU3CameraData *data = cameraData(camera); + if (data->hasPreference()) + return generateConfigurationByPreference(camera, roles); + + return generateConfigurationByCalculation(camera, roles); +} + +CameraConfiguration *PipelineHandlerIPU3::generateConfigurationByCalculation( + Camera *camera, const StreamRoles &roles) { IPU3CameraData *data = cameraData(camera); IPU3CameraConfiguration *config = new IPU3CameraConfiguration(data); @@ -505,6 +654,57 @@ CameraConfiguration *PipelineHandlerIPU3::generateConfiguration(Camera *camera, return config; } +CameraConfiguration *PipelineHandlerIPU3::generateConfigurationByPreference( + Camera *camera, const StreamRoles &roles) +{ + IPU3CameraData *data = cameraData(camera); + IPU3CameraConfiguration *config = new IPU3CameraConfiguration(data); + PipeConfigPreference &preference = data->pipeConfigPreference_; + + if (roles.empty()) + return config; + + for (const StreamRole role : roles) { + std::map> streamFormats; + PixelFormat pixelFormat = formats::NV12; + Size size; + + switch (role) { + case StreamRole::StillCapture: + size = preference.maxStillResolution(); + streamFormats[pixelFormat] = { { preference.maxStillResolution(), + preference.maxStillResolution() } }; + break; + + case StreamRole::Viewfinder: + case StreamRole::VideoRecording: + size = preference.maxVideoResolution(); + streamFormats[pixelFormat] = { { preference.maxVideoResolution(), + preference.maxVideoResolution() } }; + break; + + case StreamRole::Raw: + default: + LOG(IPU3, Error) + << "Requested stream role not supported: " << role; + delete config; + return nullptr; + } + + StreamFormats formats(streamFormats); + StreamConfiguration cfg(formats); + cfg.size = size; + cfg.pixelFormat = pixelFormat; + cfg.bufferCount = IPU3CameraConfiguration::kBufferCount; + config->addConfiguration(cfg); + } + + if (config->validate() == CameraConfiguration::Invalid) + return {}; + + return config; +} + int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) { IPU3CameraConfiguration *config = @@ -971,6 +1171,15 @@ int PipelineHandlerIPU3::initControls(IPU3CameraData *data) return updateControls(data); } +int PipelineHandlerIPU3::initOptions(IPU3CameraData *data) +{ + /* Expose IPU3 options. */ + ControlInfoMap::Map options = IPU3Options; + data->optionInfo_ = ControlInfoMap(std::move(options), options::options); + + return 0; +} + /** * \brief Update the camera controls * \param[in] data The camera data @@ -1118,6 +1327,10 @@ int PipelineHandlerIPU3::registerCameras() /* Initialize the camera properties. */ data->properties_ = cio2->sensor()->properties(); + ret = initOptions(data.get()); + if (ret) + continue; + ret = initControls(data.get()); if (ret) continue;