From patchwork Wed Aug 18 15:53:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13391 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 28C93BD87C for ; Wed, 18 Aug 2021 15:54:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 89EB768890; Wed, 18 Aug 2021 17:54:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="mslvsvhd"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DAAF7688A5 for ; Wed, 18 Aug 2021 17:54:09 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:946e:2bbb:370e:41e4]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8A1EFEE; Wed, 18 Aug 2021 17:54:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1629302049; bh=y+QeKCpspQTZXZ16dntwKNgRxMQVPK309nlXoVMPIAI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mslvsvhdcD7dMyO+Zo8DOBsGBeOvTYEYSWSOB3JfoIFtmu9WMzvsmb6OvlRxsLEqJ A9C5x6aBtDUNOsUIbefnCZniMYJu2y60gxwm9cpXRzly/oZxcoXNnKS6Zi+DxutZGB Ut4hj2RiQ/pt/JOZrvVtxs2/scD4sJeQszwxw2Eg= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 18 Aug 2021 17:53:58 +0200 Message-Id: <20210818155403.268694-5-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> References: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 4/9] ipa: ipu3: Introduce modular algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a new modular framework for algorithms with a common context structure that is passed to each algorithm through a common API. This patch: - removes all the local references from IPAIPU3 and uses IPAContext - implements the list of pointers and the loop at configure call on each algorithm - loops in fillParams on each prepare() call on the algorithm list - loops in prepareStats on each process() call on the algorithm list Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipa_context.h | 27 +++-------- src/ipa/ipu3/ipu3.cpp | 94 +++++++++++++++++++++++++++++++++----- 2 files changed, 89 insertions(+), 32 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 76534f7f..46291d9e 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -10,37 +10,22 @@ #include +#include + namespace libcamera { namespace ipa::ipu3 { -/** - * Fixed configuration of the IPA - * - * This structure contains data set at the configure state of the IPA. - * It can't be modified while streaming. - */ struct IPAConfiguration { + struct Grid { + ipu3_uapi_grid_config bdsGrid; + Size bdsOutputSize; + } grid; }; -/** - * Context of a frame for each algorithms - * - * This may be stored in a way that is associated with a given request - * lifetime, though for now a single instance is used. - * It contains informations updated while streaming by the algorithms. - */ struct IPAFrameContext { }; -/** - * Context information shared between the algorithms - * - * The global context structure contains: - * - a configuration structure set while the IPA is configured, and not - * modified when streaming - * - a frameContext structure updated by the algorithms while streaming - */ struct IPAContext { IPAConfiguration configuration; IPAFrameContext frameContext; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index c34fa460..dc6f0735 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -29,6 +29,7 @@ #include "libcamera/internal/mapped_framebuffer.h" +#include "algorithms/algorithm.h" #include "ipu3_agc.h" #include "ipu3_awb.h" #include "libipa/camera_sensor_helper.h" @@ -36,6 +37,56 @@ static constexpr uint32_t kMaxCellWidthPerSet = 160; static constexpr uint32_t kMaxCellHeightPerSet = 56; +/** + * \file ipa_context.h + * \brief Context information shared between the algorithms + */ + +/** + * \struct IPAConfiguration + * \brief Fixed configuration of the IPA + * + * This structure contains data set at the configure state of the IPA. + * It can't be modified while streaming. + */ + +/** + * \struct IPAConfiguration::Grid + * \brief Grid configuration of the IPA + * + * This structure contains all the parameters needed to use the statistics. + * + */ + +/** + * \var IPAConfiguration::Grid::bdsGrid + * \brief Bayer Down Scaler grid plane config used by the kernel + */ + +/** + * \var IPAConfiguration::Grid::bdsOutputSize + * \brief BDS output size configured by the pipeline handler + */ + +/** + * \struct IPAFrameContext + * \brief Context of a frame for each algorithms + * + * This may be stored in a way that is associated with a given request + * lifetime, though for now a single instance is used. + * It contains informations updated while streaming by the algorithms. + */ + +/** + * \struct IPAContext + * \brief Context information shared between the algorithms + * + * The global context structure contains: + * - a configuration structure set while the IPA is configured, and not + * modified when streaming + * - a frameContext structure updated by the algorithms while streaming + */ + namespace libcamera { LOG_DEFINE_CATEGORY(IPAIPU3) @@ -91,10 +142,12 @@ private: /* Interface to the Camera Helper */ std::unique_ptr camHelper_; + /* Maintain the algorithms used by the IPA */ + std::list> algorithms_; + /* Local parameter storage */ + struct IPAContext context_; struct ipu3_uapi_params params_; - - struct ipu3_uapi_grid_config bdsGrid_; }; /** @@ -191,7 +244,13 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) uint32_t minError = std::numeric_limits::max(); Size best; Size bestLog2; - bdsGrid_ = {}; + ipu3_uapi_grid_config &bdsGrid = context_.configuration.grid.bdsGrid; + + /* Set the BDS output size in the IPAConfiguration structure */ + context_.configuration.grid.bdsOutputSize = bdsOutputSize; + + bdsGrid.x_start = 0; + bdsGrid.y_start = 0; for (uint32_t widthShift = 3; widthShift <= 7; ++widthShift) { uint32_t width = std::min(kMaxCellWidthPerSet, @@ -215,14 +274,14 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) } } - bdsGrid_.width = best.width >> bestLog2.width; - bdsGrid_.block_width_log2 = bestLog2.width; - bdsGrid_.height = best.height >> bestLog2.height; - bdsGrid_.block_height_log2 = bestLog2.height; + bdsGrid.width = best.width >> bestLog2.width; + bdsGrid.block_width_log2 = bestLog2.width; + bdsGrid.height = best.height >> bestLog2.height; + bdsGrid.block_height_log2 = bestLog2.height; LOG(IPAIPU3, Debug) << "Best grid found is: (" - << (int)bdsGrid_.width << " << " << (int)bdsGrid_.block_width_log2 << ") x (" - << (int)bdsGrid_.height << " << " << (int)bdsGrid_.block_height_log2 << ")"; + << (int)bdsGrid.width << " << " << (int)bdsGrid.block_width_log2 << ") x (" + << (int)bdsGrid.height << " << " << (int)bdsGrid.block_height_log2 << ")"; } int IPAIPU3::configure(const IPAConfigInfo &configInfo) @@ -264,15 +323,22 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) defVBlank_ = itVBlank->second.def().get(); + /* Clean context and IPU3 parameters at configuration */ params_ = {}; + context_ = {}; calculateBdsGrid(configInfo.bdsOutputSize); + for (auto const &algo : algorithms_) { + int ret = algo->configure(context_, configInfo); + if (ret) + return ret; + } awbAlgo_ = std::make_unique(); - awbAlgo_->initialise(params_, configInfo.bdsOutputSize, bdsGrid_); + awbAlgo_->initialise(params_, context_.configuration.grid.bdsOutputSize, context_.configuration.grid.bdsGrid); agcAlgo_ = std::make_unique(); - agcAlgo_->initialise(bdsGrid_, sensorInfo_); + agcAlgo_->initialise(context_.configuration.grid.bdsGrid, sensorInfo_); return 0; } @@ -346,6 +412,9 @@ void IPAIPU3::processControls([[maybe_unused]] unsigned int frame, void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) { + for (auto const &algo : algorithms_) + algo->prepare(context_, params_); + if (agcAlgo_->updateControls()) awbAlgo_->updateWbParameters(params_, agcAlgo_->gamma()); @@ -363,6 +432,9 @@ void IPAIPU3::parseStatistics(unsigned int frame, { ControlList ctrls(controls::controls); + for (auto const &algo : algorithms_) + algo->process(context_, stats); + double gain = camHelper_->gain(gain_); agcAlgo_->process(stats, exposure_, gain); gain_ = camHelper_->gainCode(gain);