From patchwork Wed Aug 18 15:53:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13389 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 76D60C3241 for ; Wed, 18 Aug 2021 15:54:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A6313688A3; Wed, 18 Aug 2021 17:54:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="SBJpma6c"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4AEDD68890 for ; Wed, 18 Aug 2021 17:54:09 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:946e:2bbb:370e:41e4]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B383EEE; Wed, 18 Aug 2021 17:54:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1629302048; bh=Z+UXjf6Idj7j65LOkAIaDNWuJS/dYp2ITP+yDqjiLr4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SBJpma6cf1AFAlxZ72ExAnQ25h67pjEavg9qjdTCaSh1kk/kW6lhlK7wsPAPLZ920 efn5nTkQAFu1IeK3P4xw2zs9hxX8T3RnhATAM++irk/6jgDheas1niD5ks1or2p6/o ZtvlQlYOY+bgwgOqdwv+/1a6n0ZA2fMfHVp2dVAg= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 18 Aug 2021 17:53:55 +0200 Message-Id: <20210818155403.268694-2-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> References: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/9] ipa: move libipa::Algorithm to ipa/ipu3/algorithms X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The abstract Algorithm class was originally placed in libipa as an attempt to define a generic algorithm container. This was a little optimistic and pushed a bit far too early. Move the Algorithm class into the IPU3 which is the only user of the class, as we adapt it to support modular algorithm components for the IPU3. Not documenting the namespace may cause issues with Doxygen in libipa. The file libipa.cpp is thus created as an empty file for now, but we can leverage it in the future to add more global libipa documentation, and possibly code too. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham kieran.bingham@ideasonboard.com Reviewed-by: Laurent Pinchart --- .../{libipa => ipu3/algorithms}/algorithm.cpp | 17 ++++--------- src/ipa/ipu3/algorithms/algorithm.h | 24 +++++++++++++++++++ src/ipa/ipu3/algorithms/meson.build | 5 ++++ src/ipa/ipu3/ipu3_agc.h | 2 +- src/ipa/ipu3/ipu3_awb.h | 2 +- src/ipa/ipu3/meson.build | 4 ++++ src/ipa/libipa/algorithm.h | 24 ------------------- src/ipa/libipa/libipa.cpp | 22 +++++++++++++++++ src/ipa/libipa/meson.build | 5 ++-- 9 files changed, 63 insertions(+), 42 deletions(-) rename src/ipa/{libipa => ipu3/algorithms}/algorithm.cpp (55%) create mode 100644 src/ipa/ipu3/algorithms/algorithm.h create mode 100644 src/ipa/ipu3/algorithms/meson.build delete mode 100644 src/ipa/libipa/algorithm.h create mode 100644 src/ipa/libipa/libipa.cpp diff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/ipu3/algorithms/algorithm.cpp similarity index 55% rename from src/ipa/libipa/algorithm.cpp rename to src/ipa/ipu3/algorithms/algorithm.cpp index 930f9353..325d34f0 100644 --- a/src/ipa/libipa/algorithm.cpp +++ b/src/ipa/ipu3/algorithms/algorithm.cpp @@ -2,7 +2,7 @@ /* * Copyright (C) 2021, Ideas On Board * - * algorithm.cpp - ISP control algorithms + * algorithm.h - IPU3 control algorithm interface */ #include "algorithm.h" @@ -14,26 +14,17 @@ namespace libcamera { -/** - * \brief The IPA namespace - * - * The IPA namespace groups all types specific to IPA modules. It serves as the - * top-level namespace for the IPA library libipa, and also contains - * module-specific namespaces for IPA modules. - */ -namespace ipa { +namespace ipa::ipu3 { /** * \class Algorithm - * \brief The base class for all IPA algorithms + * \brief The base class for all IPU3 algorithms * * The Algorithm class defines a standard interface for IPA algorithms. By * abstracting algorithms, it makes possible the implementation of generic code * to manage algorithms regardless of their specific type. */ -Algorithm::~Algorithm() = default; - -} /* namespace ipa */ +} /* namespace ipa::ipu3 */ } /* namespace libcamera */ diff --git a/src/ipa/ipu3/algorithms/algorithm.h b/src/ipa/ipu3/algorithms/algorithm.h new file mode 100644 index 00000000..072f01c4 --- /dev/null +++ b/src/ipa/ipu3/algorithms/algorithm.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * algorithm.h - IPU3 control algorithm interface + */ +#ifndef __LIBCAMERA_IPA_IPU3_ALGORITHM_H__ +#define __LIBCAMERA_IPA_IPU3_ALGORITHM_H__ + +namespace libcamera { + +namespace ipa::ipu3 { + +class Algorithm +{ +public: + virtual ~Algorithm() {} +}; + +} /* namespace ipa::ipu3 */ + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_IPA_IPU3_ALGORITHM_H__ */ diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build new file mode 100644 index 00000000..dc538b79 --- /dev/null +++ b/src/ipa/ipu3/algorithms/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +ipu3_ipa_algorithms = files([ + 'algorithm.cpp', +]) diff --git a/src/ipa/ipu3/ipu3_agc.h b/src/ipa/ipu3/ipu3_agc.h index 9f3d4257..f00b98d6 100644 --- a/src/ipa/ipu3/ipu3_agc.h +++ b/src/ipa/ipu3/ipu3_agc.h @@ -13,7 +13,7 @@ #include -#include "libipa/algorithm.h" +#include "algorithms/algorithm.h" namespace libcamera { diff --git a/src/ipa/ipu3/ipu3_awb.h b/src/ipa/ipu3/ipu3_awb.h index 122cf68c..ea2d4320 100644 --- a/src/ipa/ipu3/ipu3_awb.h +++ b/src/ipa/ipu3/ipu3_awb.h @@ -13,7 +13,7 @@ #include -#include "libipa/algorithm.h" +#include "algorithms/algorithm.h" namespace libcamera { diff --git a/src/ipa/ipu3/meson.build b/src/ipa/ipu3/meson.build index b6364190..fcb27d68 100644 --- a/src/ipa/ipu3/meson.build +++ b/src/ipa/ipu3/meson.build @@ -1,5 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 +subdir('algorithms') + ipa_name = 'ipa_ipu3' ipu3_ipa_sources = files([ @@ -8,6 +10,8 @@ ipu3_ipa_sources = files([ 'ipu3_awb.cpp', ]) +ipu3_ipa_sources += ipu3_ipa_algorithms + mod = shared_module(ipa_name, [ipu3_ipa_sources, libcamera_generated_ipa_headers], name_prefix : '', diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h deleted file mode 100644 index 89cee4c4..00000000 --- a/src/ipa/libipa/algorithm.h +++ /dev/null @@ -1,24 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2021, Ideas On Board - * - * algorithm.h - ISP control algorithm interface - */ -#ifndef __LIBCAMERA_IPA_LIBIPA_ALGORITHM_H__ -#define __LIBCAMERA_IPA_LIBIPA_ALGORITHM_H__ - -namespace libcamera { - -namespace ipa { - -class Algorithm -{ -public: - virtual ~Algorithm(); -}; - -} /* namespace ipa */ - -} /* namespace libcamera */ - -#endif /* __LIBCAMERA_IPA_LIBIPA_ALGORITHM_H__ */ diff --git a/src/ipa/libipa/libipa.cpp b/src/ipa/libipa/libipa.cpp new file mode 100644 index 00000000..08bc3541 --- /dev/null +++ b/src/ipa/libipa/libipa.cpp @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * libipa.cpp - libipa interface + */ + +namespace libcamera { + +/** + * \brief The IPA namespace + * + * The IPA namespace groups all types specific to IPA modules. It serves as the + * top-level namespace for the IPA library libipa, and also contains + * module-specific namespaces for IPA modules. + */ +namespace ipa { + +} /* namespace ipa */ + +} /* namespace libcamera */ + diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index 3fda7c00..4d073a03 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -1,15 +1,14 @@ # SPDX-License-Identifier: CC0-1.0 libipa_headers = files([ - 'algorithm.h', 'camera_sensor_helper.h', 'histogram.h' ]) libipa_sources = files([ - 'algorithm.cpp', 'camera_sensor_helper.cpp', - 'histogram.cpp' + 'histogram.cpp', + 'libipa.cpp', ]) libipa_includes = include_directories('..') From patchwork Wed Aug 18 15:53:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13390 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C42F9BD87C for ; Wed, 18 Aug 2021 15:54:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 65FD1688A6; Wed, 18 Aug 2021 17:54:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="T7qGSFMf"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8F4CD68890 for ; Wed, 18 Aug 2021 17:54:09 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:946e:2bbb:370e:41e4]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2E360F1; Wed, 18 Aug 2021 17:54:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1629302049; bh=wwrgTHKy2CdyyyI3wyOynvu4iR2y/MegybHoDf442Vc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=T7qGSFMfh++oElU5epqPIBxocEBFys4USFDPcV8bhuQaXYQD7AAVsZJZNChUKUGq1 0qGnIdWphBiN87ZtcDZEIl6dc9HPA4qrCtHwFgA62zF96aiFcTsaxrjIjqxPPqYHjl pQUq/FLeWi+CZ65vFmkmSJpbkPrf27LUALV55Ti4= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 18 Aug 2021 17:53:56 +0200 Message-Id: <20210818155403.268694-3-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> References: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/9] ipa: ipu3: Introduce a Context structure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" An increasing amount of data and information needs to be shared between the components that build up to implement image processing algorithms. Create a context structure which will allow us to work towards calling algorithms in a modular way, and sharing information between the modules. The IPA context is a global context set at configure time (IPAConfiguration) and a per-frame context (IPAFrameContext) used while streaming. Signed-off-by: Kieran Bingham Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipa_context.h | 53 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 src/ipa/ipu3/ipa_context.h diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h new file mode 100644 index 00000000..76534f7f --- /dev/null +++ b/src/ipa/ipu3/ipa_context.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Google Inc. + * + * ipu3_ipa_context.h - IPU3 IPA Context + * + */ +#ifndef __LIBCAMERA_IPU3_IPA_CONTEXT_H__ +#define __LIBCAMERA_IPU3_IPA_CONTEXT_H__ + +#include + +namespace libcamera { + +namespace ipa::ipu3 { + +/** + * Fixed configuration of the IPA + * + * This structure contains data set at the configure state of the IPA. + * It can't be modified while streaming. + */ +struct IPAConfiguration { +}; + +/** + * Context of a frame for each algorithms + * + * This may be stored in a way that is associated with a given request + * lifetime, though for now a single instance is used. + * It contains informations updated while streaming by the algorithms. + */ +struct IPAFrameContext { +}; + +/** + * Context information shared between the algorithms + * + * The global context structure contains: + * - a configuration structure set while the IPA is configured, and not + * modified when streaming + * - a frameContext structure updated by the algorithms while streaming + */ +struct IPAContext { + IPAConfiguration configuration; + IPAFrameContext frameContext; +}; + +} /* namespace ipa::ipu3 */ + +} /* namespace libcamera*/ + +#endif /* __LIBCAMERA_IPU3_IPA_CONTEXT_H__ */ From patchwork Wed Aug 18 15:53:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13391 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 28C93BD87C for ; Wed, 18 Aug 2021 15:54:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 89EB768890; Wed, 18 Aug 2021 17:54:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="mslvsvhd"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DAAF7688A5 for ; Wed, 18 Aug 2021 17:54:09 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:946e:2bbb:370e:41e4]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8A1EFEE; Wed, 18 Aug 2021 17:54:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1629302049; bh=y+QeKCpspQTZXZ16dntwKNgRxMQVPK309nlXoVMPIAI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mslvsvhdcD7dMyO+Zo8DOBsGBeOvTYEYSWSOB3JfoIFtmu9WMzvsmb6OvlRxsLEqJ A9C5x6aBtDUNOsUIbefnCZniMYJu2y60gxwm9cpXRzly/oZxcoXNnKS6Zi+DxutZGB Ut4hj2RiQ/pt/JOZrvVtxs2/scD4sJeQszwxw2Eg= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 18 Aug 2021 17:53:58 +0200 Message-Id: <20210818155403.268694-5-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> References: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 4/9] ipa: ipu3: Introduce modular algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a new modular framework for algorithms with a common context structure that is passed to each algorithm through a common API. This patch: - removes all the local references from IPAIPU3 and uses IPAContext - implements the list of pointers and the loop at configure call on each algorithm - loops in fillParams on each prepare() call on the algorithm list - loops in prepareStats on each process() call on the algorithm list Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipa_context.h | 27 +++-------- src/ipa/ipu3/ipu3.cpp | 94 +++++++++++++++++++++++++++++++++----- 2 files changed, 89 insertions(+), 32 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 76534f7f..46291d9e 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -10,37 +10,22 @@ #include +#include + namespace libcamera { namespace ipa::ipu3 { -/** - * Fixed configuration of the IPA - * - * This structure contains data set at the configure state of the IPA. - * It can't be modified while streaming. - */ struct IPAConfiguration { + struct Grid { + ipu3_uapi_grid_config bdsGrid; + Size bdsOutputSize; + } grid; }; -/** - * Context of a frame for each algorithms - * - * This may be stored in a way that is associated with a given request - * lifetime, though for now a single instance is used. - * It contains informations updated while streaming by the algorithms. - */ struct IPAFrameContext { }; -/** - * Context information shared between the algorithms - * - * The global context structure contains: - * - a configuration structure set while the IPA is configured, and not - * modified when streaming - * - a frameContext structure updated by the algorithms while streaming - */ struct IPAContext { IPAConfiguration configuration; IPAFrameContext frameContext; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index c34fa460..dc6f0735 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -29,6 +29,7 @@ #include "libcamera/internal/mapped_framebuffer.h" +#include "algorithms/algorithm.h" #include "ipu3_agc.h" #include "ipu3_awb.h" #include "libipa/camera_sensor_helper.h" @@ -36,6 +37,56 @@ static constexpr uint32_t kMaxCellWidthPerSet = 160; static constexpr uint32_t kMaxCellHeightPerSet = 56; +/** + * \file ipa_context.h + * \brief Context information shared between the algorithms + */ + +/** + * \struct IPAConfiguration + * \brief Fixed configuration of the IPA + * + * This structure contains data set at the configure state of the IPA. + * It can't be modified while streaming. + */ + +/** + * \struct IPAConfiguration::Grid + * \brief Grid configuration of the IPA + * + * This structure contains all the parameters needed to use the statistics. + * + */ + +/** + * \var IPAConfiguration::Grid::bdsGrid + * \brief Bayer Down Scaler grid plane config used by the kernel + */ + +/** + * \var IPAConfiguration::Grid::bdsOutputSize + * \brief BDS output size configured by the pipeline handler + */ + +/** + * \struct IPAFrameContext + * \brief Context of a frame for each algorithms + * + * This may be stored in a way that is associated with a given request + * lifetime, though for now a single instance is used. + * It contains informations updated while streaming by the algorithms. + */ + +/** + * \struct IPAContext + * \brief Context information shared between the algorithms + * + * The global context structure contains: + * - a configuration structure set while the IPA is configured, and not + * modified when streaming + * - a frameContext structure updated by the algorithms while streaming + */ + namespace libcamera { LOG_DEFINE_CATEGORY(IPAIPU3) @@ -91,10 +142,12 @@ private: /* Interface to the Camera Helper */ std::unique_ptr camHelper_; + /* Maintain the algorithms used by the IPA */ + std::list> algorithms_; + /* Local parameter storage */ + struct IPAContext context_; struct ipu3_uapi_params params_; - - struct ipu3_uapi_grid_config bdsGrid_; }; /** @@ -191,7 +244,13 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) uint32_t minError = std::numeric_limits::max(); Size best; Size bestLog2; - bdsGrid_ = {}; + ipu3_uapi_grid_config &bdsGrid = context_.configuration.grid.bdsGrid; + + /* Set the BDS output size in the IPAConfiguration structure */ + context_.configuration.grid.bdsOutputSize = bdsOutputSize; + + bdsGrid.x_start = 0; + bdsGrid.y_start = 0; for (uint32_t widthShift = 3; widthShift <= 7; ++widthShift) { uint32_t width = std::min(kMaxCellWidthPerSet, @@ -215,14 +274,14 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) } } - bdsGrid_.width = best.width >> bestLog2.width; - bdsGrid_.block_width_log2 = bestLog2.width; - bdsGrid_.height = best.height >> bestLog2.height; - bdsGrid_.block_height_log2 = bestLog2.height; + bdsGrid.width = best.width >> bestLog2.width; + bdsGrid.block_width_log2 = bestLog2.width; + bdsGrid.height = best.height >> bestLog2.height; + bdsGrid.block_height_log2 = bestLog2.height; LOG(IPAIPU3, Debug) << "Best grid found is: (" - << (int)bdsGrid_.width << " << " << (int)bdsGrid_.block_width_log2 << ") x (" - << (int)bdsGrid_.height << " << " << (int)bdsGrid_.block_height_log2 << ")"; + << (int)bdsGrid.width << " << " << (int)bdsGrid.block_width_log2 << ") x (" + << (int)bdsGrid.height << " << " << (int)bdsGrid.block_height_log2 << ")"; } int IPAIPU3::configure(const IPAConfigInfo &configInfo) @@ -264,15 +323,22 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) defVBlank_ = itVBlank->second.def().get(); + /* Clean context and IPU3 parameters at configuration */ params_ = {}; + context_ = {}; calculateBdsGrid(configInfo.bdsOutputSize); + for (auto const &algo : algorithms_) { + int ret = algo->configure(context_, configInfo); + if (ret) + return ret; + } awbAlgo_ = std::make_unique(); - awbAlgo_->initialise(params_, configInfo.bdsOutputSize, bdsGrid_); + awbAlgo_->initialise(params_, context_.configuration.grid.bdsOutputSize, context_.configuration.grid.bdsGrid); agcAlgo_ = std::make_unique(); - agcAlgo_->initialise(bdsGrid_, sensorInfo_); + agcAlgo_->initialise(context_.configuration.grid.bdsGrid, sensorInfo_); return 0; } @@ -346,6 +412,9 @@ void IPAIPU3::processControls([[maybe_unused]] unsigned int frame, void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) { + for (auto const &algo : algorithms_) + algo->prepare(context_, params_); + if (agcAlgo_->updateControls()) awbAlgo_->updateWbParameters(params_, agcAlgo_->gamma()); @@ -363,6 +432,9 @@ void IPAIPU3::parseStatistics(unsigned int frame, { ControlList ctrls(controls::controls); + for (auto const &algo : algorithms_) + algo->process(context_, stats); + double gain = camHelper_->gain(gain_); agcAlgo_->process(stats, exposure_, gain); gain_ = camHelper_->gainCode(gain); From patchwork Wed Aug 18 15:53:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13392 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A165BC3242 for ; Wed, 18 Aug 2021 15:54:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 405A868915; Wed, 18 Aug 2021 17:54:16 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="k3k5mKsd"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1912F688AC for ; Wed, 18 Aug 2021 17:54:10 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:946e:2bbb:370e:41e4]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B63AAF1; Wed, 18 Aug 2021 17:54:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1629302049; bh=JkmKYXviQbxf7ujkqVVVqQwg8IKjW+jcFGowcVDxeG0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=k3k5mKsd8960oZVNTnW7rC9deI76hEO1FJo5bwhLaFeCMhCZvy5tm4mNUQwpUYFmX hijHgvzXb0DgFdgk34BPj4THSAiUjNQxkJ4bdsfH2X5DHkQdswVB0aJlQ+/5sAlm9t vQFeFCuimdyIexOV6XY6OajpQAI34oAGNdWrjswc= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 18 Aug 2021 17:53:59 +0200 Message-Id: <20210818155403.268694-6-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> References: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 5/9] ipa: ipu3: Introduce a modular contrast algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Introduce a new algorithm to manage the contrast handling of the IPU3. The initial algorithm is chosen to configure the gamma contrast curve which moves the implementation out of AWB for simplicity. As it is initialised with a default gamma value of 1.1, there is no need to use the default table at initialisation anymore. This demonstrates the way to use process() call when the EventStatReady comes in. The function calculates the LUT in the context of a frame, and when prepare() is called, the parameters are filled with the updated values. AGC is modified to take the new process interface into account. Signed-off-by: Kieran Bingham Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/contrast.cpp | 53 ++++++++++++++++++++++++++++ src/ipa/ipu3/algorithms/contrast.h | 32 +++++++++++++++++ src/ipa/ipu3/algorithms/meson.build | 1 + src/ipa/ipu3/ipa_context.h | 8 +++++ src/ipa/ipu3/ipu3.cpp | 15 +++++--- src/ipa/ipu3/ipu3_agc.cpp | 9 +++-- src/ipa/ipu3/ipu3_agc.h | 5 +-- src/ipa/ipu3/ipu3_awb.cpp | 41 ++------------------- src/ipa/ipu3/ipu3_awb.h | 2 +- 9 files changed, 113 insertions(+), 53 deletions(-) create mode 100644 src/ipa/ipu3/algorithms/contrast.cpp create mode 100644 src/ipa/ipu3/algorithms/contrast.h diff --git a/src/ipa/ipu3/algorithms/contrast.cpp b/src/ipa/ipu3/algorithms/contrast.cpp new file mode 100644 index 00000000..eb92d3c7 --- /dev/null +++ b/src/ipa/ipu3/algorithms/contrast.cpp @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Google inc. + * + * contrast.cpp - IPU3 Contrast and Gamma control + */ + +#include "contrast.h" + +#include + +namespace libcamera { + +namespace ipa::ipu3::algorithms { + + +Contrast::Contrast() + : gamma_(1.0) +{ +} + +void Contrast::prepare([[maybe_unused]] IPAContext &context, ipu3_uapi_params ¶ms) +{ + /* Copy the calculated LUT into the parameters buffer */ + memcpy(params.acc_param.gamma.gc_lut.lut, + context.frameContext.contrast.gammaCorrection.lut, + IPU3_UAPI_GAMMA_CORR_LUT_ENTRIES * sizeof(params.acc_param.gamma.gc_lut.lut[0])); + + /* Enable the custom gamma table */ + params.use.acc_gamma = 1; + params.acc_param.gamma.gc_ctrl.enable = 1; +} + +void Contrast::process([[maybe_unused]] IPAContext &context, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) +{ + /* Limit the gamma effect for now */ + /* \todo expose gamma control setting through the libcamera control API */ + gamma_ = 1.1; + + /* Plot the gamma curve into the look up table */ + for (uint32_t i = 0; i < IPU3_UAPI_GAMMA_CORR_LUT_ENTRIES; i++) { + double j = static_cast(i) + / (IPU3_UAPI_GAMMA_CORR_LUT_ENTRIES - 1); + double gamma = std::pow(j, 1.0 / gamma_); + + /* The maximum value 255 is represented on 13 bits in the IPU3 */ + context.frameContext.contrast.gammaCorrection.lut[i] = gamma * 8191; + } +} + +} /* namespace ipa::ipu3::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/ipu3/algorithms/contrast.h b/src/ipa/ipu3/algorithms/contrast.h new file mode 100644 index 00000000..6cd6c5db --- /dev/null +++ b/src/ipa/ipu3/algorithms/contrast.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Google inc. + * + * contrast.h - IPU3 Contrast and Gamma control + */ +#ifndef __LIBCAMERA_IPU3_ALGORITHMS_CONTRAST_H__ +#define __LIBCAMERA_IPU3_ALGORITHMS_CONTRAST_H__ + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::ipu3::algorithms { + +class Contrast : public Algorithm +{ +public: + Contrast(); + ~Contrast() = default; + + void prepare(IPAContext &context, ipu3_uapi_params ¶ms) override; + void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; +private: + double gamma_; +}; + +} /* namespace ipa::ipu3::algorithms */ + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_IPU3_ALGORITHMS_CONTRAST_H__ */ diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build index dc538b79..e3ff3b78 100644 --- a/src/ipa/ipu3/algorithms/meson.build +++ b/src/ipa/ipu3/algorithms/meson.build @@ -2,4 +2,5 @@ ipu3_ipa_algorithms = files([ 'algorithm.cpp', + 'contrast.cpp', ]) diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 46291d9e..5964ba6d 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -24,6 +24,14 @@ struct IPAConfiguration { }; struct IPAFrameContext { + struct Agc { + uint32_t exposure; + double gain; + } agc; + + struct Contrast { + struct ipu3_uapi_gamma_corr_lut gammaCorrection; + } contrast; }; struct IPAContext { diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index dc6f0735..ee0dd9fe 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -30,6 +30,7 @@ #include "libcamera/internal/mapped_framebuffer.h" #include "algorithms/algorithm.h" +#include "algorithms/contrast.h" #include "ipu3_agc.h" #include "ipu3_awb.h" #include "libipa/camera_sensor_helper.h" @@ -218,6 +219,9 @@ int IPAIPU3::init(const IPASettings &settings, *ipaControls = ControlInfoMap(std::move(controls), controls::controls); + /* Construct our Algorithms */ + algorithms_.emplace_back(new algorithms::Contrast()); + return 0; } @@ -416,7 +420,7 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) algo->prepare(context_, params_); if (agcAlgo_->updateControls()) - awbAlgo_->updateWbParameters(params_, agcAlgo_->gamma()); + awbAlgo_->updateWbParameters(params_); *params = params_; @@ -431,13 +435,16 @@ void IPAIPU3::parseStatistics(unsigned int frame, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) { ControlList ctrls(controls::controls); + /* \todo These fields should not be written by the IPAIPU3 layer */ + context_.frameContext.agc.gain = camHelper_->gain(gain_); + context_.frameContext.agc.exposure = exposure_; for (auto const &algo : algorithms_) algo->process(context_, stats); - double gain = camHelper_->gain(gain_); - agcAlgo_->process(stats, exposure_, gain); - gain_ = camHelper_->gainCode(gain); + agcAlgo_->process(context_, stats); + exposure_ = context_.frameContext.agc.exposure; + gain_ = camHelper_->gainCode(context_.frameContext.agc.gain); awbAlgo_->calculateWBGains(stats); diff --git a/src/ipa/ipu3/ipu3_agc.cpp b/src/ipa/ipu3/ipu3_agc.cpp index 408eb849..c6782ff4 100644 --- a/src/ipa/ipu3/ipu3_agc.cpp +++ b/src/ipa/ipu3/ipu3_agc.cpp @@ -52,7 +52,7 @@ static constexpr uint8_t kCellSize = 8; IPU3Agc::IPU3Agc() : frameCount_(0), lastFrame_(0), converged_(false), - updateControls_(false), iqMean_(0.0), gamma_(1.0), + updateControls_(false), iqMean_(0.0), lineDuration_(0s), maxExposureTime_(0s), prevExposure_(0s), prevExposureNoDg_(0s), currentExposure_(0s), currentExposureNoDg_(0s) @@ -104,9 +104,6 @@ void IPU3Agc::processBrightness(const ipu3_uapi_stats_3a *stats) } } - /* Limit the gamma effect for now */ - gamma_ = 1.1; - /* Estimate the quantile mean of the top 2% of the histogram */ iqMean_ = Histogram(Span(hist)).interQuantileMean(0.98, 1.0); } @@ -193,8 +190,10 @@ void IPU3Agc::lockExposureGain(uint32_t &exposure, double &gain) lastFrame_ = frameCount_; } -void IPU3Agc::process(const ipu3_uapi_stats_3a *stats, uint32_t &exposure, double &gain) +void IPU3Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { + uint32_t &exposure = context.frameContext.agc.exposure; + double &gain = context.frameContext.agc.gain; processBrightness(stats); lockExposureGain(exposure, gain); frameCount_++; diff --git a/src/ipa/ipu3/ipu3_agc.h b/src/ipa/ipu3/ipu3_agc.h index f00b98d6..f8290abd 100644 --- a/src/ipa/ipu3/ipu3_agc.h +++ b/src/ipa/ipu3/ipu3_agc.h @@ -30,11 +30,9 @@ public: ~IPU3Agc() = default; void initialise(struct ipu3_uapi_grid_config &bdsGrid, const IPACameraSensorInfo &sensorInfo); - void process(const ipu3_uapi_stats_3a *stats, uint32_t &exposure, double &gain); + void process(IPAContext &context, const ipu3_uapi_stats_3a *stats); bool converged() { return converged_; } bool updateControls() { return updateControls_; } - /* \todo Use a metadata exchange between IPAs */ - double gamma() { return gamma_; } private: void processBrightness(const ipu3_uapi_stats_3a *stats); @@ -50,7 +48,6 @@ private: bool updateControls_; double iqMean_; - double gamma_; Duration lineDuration_; Duration maxExposureTime_; diff --git a/src/ipa/ipu3/ipu3_awb.cpp b/src/ipa/ipu3/ipu3_awb.cpp index 4bb321b3..c2d9e0c1 100644 --- a/src/ipa/ipu3/ipu3_awb.cpp +++ b/src/ipa/ipu3/ipu3_awb.cpp @@ -133,31 +133,6 @@ static const struct ipu3_uapi_ccm_mat_config imguCssCcmDefault = { 0, 0, 8191, 0 }; -/* Default settings for Gamma correction */ -const struct ipu3_uapi_gamma_corr_lut imguCssGammaLut = { { - 63, 79, 95, 111, 127, 143, 159, 175, 191, 207, 223, 239, 255, 271, 287, - 303, 319, 335, 351, 367, 383, 399, 415, 431, 447, 463, 479, 495, 511, - 527, 543, 559, 575, 591, 607, 623, 639, 655, 671, 687, 703, 719, 735, - 751, 767, 783, 799, 815, 831, 847, 863, 879, 895, 911, 927, 943, 959, - 975, 991, 1007, 1023, 1039, 1055, 1071, 1087, 1103, 1119, 1135, 1151, - 1167, 1183, 1199, 1215, 1231, 1247, 1263, 1279, 1295, 1311, 1327, 1343, - 1359, 1375, 1391, 1407, 1423, 1439, 1455, 1471, 1487, 1503, 1519, 1535, - 1551, 1567, 1583, 1599, 1615, 1631, 1647, 1663, 1679, 1695, 1711, 1727, - 1743, 1759, 1775, 1791, 1807, 1823, 1839, 1855, 1871, 1887, 1903, 1919, - 1935, 1951, 1967, 1983, 1999, 2015, 2031, 2047, 2063, 2079, 2095, 2111, - 2143, 2175, 2207, 2239, 2271, 2303, 2335, 2367, 2399, 2431, 2463, 2495, - 2527, 2559, 2591, 2623, 2655, 2687, 2719, 2751, 2783, 2815, 2847, 2879, - 2911, 2943, 2975, 3007, 3039, 3071, 3103, 3135, 3167, 3199, 3231, 3263, - 3295, 3327, 3359, 3391, 3423, 3455, 3487, 3519, 3551, 3583, 3615, 3647, - 3679, 3711, 3743, 3775, 3807, 3839, 3871, 3903, 3935, 3967, 3999, 4031, - 4063, 4095, 4127, 4159, 4223, 4287, 4351, 4415, 4479, 4543, 4607, 4671, - 4735, 4799, 4863, 4927, 4991, 5055, 5119, 5183, 5247, 5311, 5375, 5439, - 5503, 5567, 5631, 5695, 5759, 5823, 5887, 5951, 6015, 6079, 6143, 6207, - 6271, 6335, 6399, 6463, 6527, 6591, 6655, 6719, 6783, 6847, 6911, 6975, - 7039, 7103, 7167, 7231, 7295, 7359, 7423, 7487, 7551, 7615, 7679, 7743, - 7807, 7871, 7935, 7999, 8063, 8127, 8191 -} }; - IPU3Awb::IPU3Awb() : Algorithm() { @@ -197,10 +172,6 @@ void IPU3Awb::initialise(ipu3_uapi_params ¶ms, const Size &bdsOutputSize, st params.use.acc_ccm = 1; params.acc_param.ccm = imguCssCcmDefault; - params.use.acc_gamma = 1; - params.acc_param.gamma.gc_lut = imguCssGammaLut; - params.acc_param.gamma.gc_ctrl.enable = 1; - zones_.reserve(kAwbStatsSizeX * kAwbStatsSizeY); } @@ -350,7 +321,7 @@ void IPU3Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) } } -void IPU3Awb::updateWbParameters(ipu3_uapi_params ¶ms, double agcGamma) +void IPU3Awb::updateWbParameters(ipu3_uapi_params ¶ms) { /* * Green gains should not be touched and considered 1. @@ -362,18 +333,10 @@ void IPU3Awb::updateWbParameters(ipu3_uapi_params ¶ms, double agcGamma) params.acc_param.bnr.wb_gains.b = 4096 * asyncResults_.blueGain; params.acc_param.bnr.wb_gains.gb = 16; - LOG(IPU3Awb, Debug) << "Color temperature estimated: " << asyncResults_.temperatureK - << " and gamma calculated: " << agcGamma; + LOG(IPU3Awb, Debug) << "Color temperature estimated: " << asyncResults_.temperatureK; /* The CCM matrix may change when color temperature will be used */ params.acc_param.ccm = imguCssCcmDefault; - - for (uint32_t i = 0; i < 256; i++) { - double j = i / 255.0; - double gamma = std::pow(j, 1.0 / agcGamma); - /* The maximum value 255 is represented on 13 bits in the IPU3 */ - params.acc_param.gamma.gc_lut.lut[i] = gamma * 8191; - } } } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3_awb.h b/src/ipa/ipu3/ipu3_awb.h index ea2d4320..eeb2920b 100644 --- a/src/ipa/ipu3/ipu3_awb.h +++ b/src/ipa/ipu3/ipu3_awb.h @@ -31,7 +31,7 @@ public: void initialise(ipu3_uapi_params ¶ms, const Size &bdsOutputSize, struct ipu3_uapi_grid_config &bdsGrid); void calculateWBGains(const ipu3_uapi_stats_3a *stats); - void updateWbParameters(ipu3_uapi_params ¶ms, double agcGamma); + void updateWbParameters(ipu3_uapi_params ¶ms); struct Ipu3AwbCell { unsigned char greenRedAvg; From patchwork Wed Aug 18 15:54:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13393 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3829DC3243 for ; Wed, 18 Aug 2021 15:54:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C7DE76890C; Wed, 18 Aug 2021 17:54:16 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="sOQWfWGD"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3C6B1688C4 for ; Wed, 18 Aug 2021 17:54:10 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:946e:2bbb:370e:41e4]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E6F3F466; Wed, 18 Aug 2021 17:54:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1629302050; bh=QlqwPGZFRsLJyTE62+Hxl365naFAKq40/nKHsupaPEA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sOQWfWGDOvSUon/K4N8GtDRRiiejrwhlYfkodaypj/YBfAgSmRLJbo0BBI73R4En0 ntSebGjQdG9SVl3NDyazhDd5mvg4QxDmtSYYONv3K6u8GxuIk/7Uc37K5+zpDvYWYe iINs7o0FY1Tu08myNvOEH9fgO66yRwrajrDZg86U= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 18 Aug 2021 17:54:00 +0200 Message-Id: <20210818155403.268694-7-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> References: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 6/9] ipa: ipu3: convert AWB to the new algorithm interface X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the stats are received, pass it with the context to the existing AWB algorithm. IPAFrameContext now has a new structure to store the gains calculated by the AWB algorithm. When an EventFillParams event is received, call prepare() and set the new gains accordingly in the params structure. There is no more a need for the IPU3Awb::initialise() function, as the params are always set in prepare(). Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipa_context.h | 8 +++ src/ipa/ipu3/ipu3.cpp | 5 +- src/ipa/ipu3/ipu3_awb.cpp | 107 ++++++++++++++++++------------------- src/ipa/ipu3/ipu3_awb.h | 6 +-- 4 files changed, 65 insertions(+), 61 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 5964ba6d..beae3ac6 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -29,6 +29,14 @@ struct IPAFrameContext { double gain; } agc; + struct Awb { + struct Gains { + double red; + double green; + double blue; + } gains; + } awb; + struct Contrast { struct ipu3_uapi_gamma_corr_lut gammaCorrection; } contrast; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index ee0dd9fe..2b4a4c8c 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -339,7 +339,6 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) } awbAlgo_ = std::make_unique(); - awbAlgo_->initialise(params_, context_.configuration.grid.bdsOutputSize, context_.configuration.grid.bdsGrid); agcAlgo_ = std::make_unique(); agcAlgo_->initialise(context_.configuration.grid.bdsGrid, sensorInfo_); @@ -420,7 +419,7 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) algo->prepare(context_, params_); if (agcAlgo_->updateControls()) - awbAlgo_->updateWbParameters(params_); + awbAlgo_->prepare(context_, params_); *params = params_; @@ -446,7 +445,7 @@ void IPAIPU3::parseStatistics(unsigned int frame, exposure_ = context_.frameContext.agc.exposure; gain_ = camHelper_->gainCode(context_.frameContext.agc.gain); - awbAlgo_->calculateWBGains(stats); + awbAlgo_->process(context_, stats); if (agcAlgo_->updateControls()) setControls(frame); diff --git a/src/ipa/ipu3/ipu3_awb.cpp b/src/ipa/ipu3/ipu3_awb.cpp index c2d9e0c1..f90aa492 100644 --- a/src/ipa/ipu3/ipu3_awb.cpp +++ b/src/ipa/ipu3/ipu3_awb.cpp @@ -107,25 +107,6 @@ static const struct ipu3_uapi_bnr_static_config imguCssBnrDefaults = { .opt_center_sqr = { 419904, 133956 }, }; -/* Default settings for Auto White Balance replicated from the Kernel*/ -static const struct ipu3_uapi_awb_config_s imguCssAwbDefaults = { - .rgbs_thr_gr = 8191, - .rgbs_thr_r = 8191, - .rgbs_thr_gb = 8191, - .rgbs_thr_b = 8191 | IPU3_UAPI_AWB_RGBS_THR_B_EN | IPU3_UAPI_AWB_RGBS_THR_B_INCL_SAT, - .grid = { - .width = 160, - .height = 36, - .block_width_log2 = 3, - .block_height_log2 = 4, - .height_per_slice = 1, /* Overridden by kernel. */ - .x_start = 0, - .y_start = 0, - .x_end = 0, - .y_end = 0, - }, -}; - /* Default color correction matrix defined as an identity matrix */ static const struct ipu3_uapi_ccm_mat_config imguCssCcmDefault = { 8191, 0, 0, 0, @@ -140,39 +121,12 @@ IPU3Awb::IPU3Awb() asyncResults_.greenGain = 1.0; asyncResults_.redGain = 1.0; asyncResults_.temperatureK = 4500; -} -IPU3Awb::~IPU3Awb() -{ + zones_.reserve(kAwbStatsSizeX * kAwbStatsSizeY); } -void IPU3Awb::initialise(ipu3_uapi_params ¶ms, const Size &bdsOutputSize, struct ipu3_uapi_grid_config &bdsGrid) +IPU3Awb::~IPU3Awb() { - params.use.acc_awb = 1; - params.acc_param.awb.config = imguCssAwbDefaults; - - awbGrid_ = bdsGrid; - params.acc_param.awb.config.grid = awbGrid_; - - params.use.acc_bnr = 1; - params.acc_param.bnr = imguCssBnrDefaults; - /** - * Optical center is column (respectively row) startminus X (respectively Y) center. - * For the moment use BDS as a first approximation, but it should - * be calculated based on Shading (SHD) parameters. - */ - params.acc_param.bnr.column_size = bdsOutputSize.width; - params.acc_param.bnr.opt_center.x_reset = awbGrid_.x_start - (bdsOutputSize.width / 2); - params.acc_param.bnr.opt_center.y_reset = awbGrid_.y_start - (bdsOutputSize.height / 2); - params.acc_param.bnr.opt_center_sqr.x_sqr_reset = params.acc_param.bnr.opt_center.x_reset - * params.acc_param.bnr.opt_center.x_reset; - params.acc_param.bnr.opt_center_sqr.y_sqr_reset = params.acc_param.bnr.opt_center.y_reset - * params.acc_param.bnr.opt_center.y_reset; - - params.use.acc_ccm = 1; - params.acc_param.ccm = imguCssCcmDefault; - - zones_.reserve(kAwbStatsSizeX * kAwbStatsSizeY); } /** @@ -321,22 +275,65 @@ void IPU3Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) } } -void IPU3Awb::updateWbParameters(ipu3_uapi_params ¶ms) +void IPU3Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { + calculateWBGains(stats); + + /* + * Gains are only recalculated if enough zones were detected. + * The results are cached, so if no results were calculated, we set the + * cached values from asyncResults_ here. + */ + context.frameContext.awb.gains.blue = asyncResults_.blueGain; + context.frameContext.awb.gains.green = asyncResults_.greenGain; + context.frameContext.awb.gains.red = asyncResults_.redGain; +} + +void IPU3Awb::prepare(IPAContext &context, ipu3_uapi_params ¶ms) +{ + params.acc_param.awb.config.rgbs_thr_gr = 8191; + params.acc_param.awb.config.rgbs_thr_r = 8191; + params.acc_param.awb.config.rgbs_thr_gb = 8191; + params.acc_param.awb.config.rgbs_thr_b = 8191 | IPU3_UAPI_AWB_RGBS_THR_B_EN | IPU3_UAPI_AWB_RGBS_THR_B_INCL_SAT; + + awbGrid_ = context.configuration.grid.bdsGrid; + + params.acc_param.awb.config.grid = context.configuration.grid.bdsGrid; + params.acc_param.bnr = imguCssBnrDefaults; + + /* + * Optical center is column start (respectively row start) of the + * region of interest minus its X center (respectively Y center). + * + * For the moment use BDS as a first approximation, but it should + * be calculated based on Shading (SHD) parameters. + */ + Size &bdsOutputSize = context.configuration.grid.bdsOutputSize; + params.acc_param.bnr.column_size = bdsOutputSize.width; + params.acc_param.bnr.opt_center.x_reset = awbGrid_.x_start - (bdsOutputSize.width / 2); + params.acc_param.bnr.opt_center.y_reset = awbGrid_.y_start - (bdsOutputSize.height / 2); + params.acc_param.bnr.opt_center_sqr.x_sqr_reset = params.acc_param.bnr.opt_center.x_reset + * params.acc_param.bnr.opt_center.x_reset; + params.acc_param.bnr.opt_center_sqr.y_sqr_reset = params.acc_param.bnr.opt_center.y_reset + * params.acc_param.bnr.opt_center.y_reset; + + params.acc_param.ccm = imguCssCcmDefault; + /* * Green gains should not be touched and considered 1. * Default is 16, so do not change it at all. * 4096 is the value for a gain of 1.0 */ - params.acc_param.bnr.wb_gains.gr = 16; - params.acc_param.bnr.wb_gains.r = 4096 * asyncResults_.redGain; - params.acc_param.bnr.wb_gains.b = 4096 * asyncResults_.blueGain; - params.acc_param.bnr.wb_gains.gb = 16; + params.acc_param.bnr.wb_gains.gr = 16 * context.frameContext.awb.gains.green; + params.acc_param.bnr.wb_gains.r = 4096 * context.frameContext.awb.gains.red; + params.acc_param.bnr.wb_gains.b = 4096 * context.frameContext.awb.gains.blue; + params.acc_param.bnr.wb_gains.gb = 16 * context.frameContext.awb.gains.green; LOG(IPU3Awb, Debug) << "Color temperature estimated: " << asyncResults_.temperatureK; - /* The CCM matrix may change when color temperature will be used */ - params.acc_param.ccm = imguCssCcmDefault; + params.use.acc_awb = 1; + params.use.acc_bnr = 1; + params.use.acc_ccm = 1; } } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3_awb.h b/src/ipa/ipu3/ipu3_awb.h index eeb2920b..4de3fae2 100644 --- a/src/ipa/ipu3/ipu3_awb.h +++ b/src/ipa/ipu3/ipu3_awb.h @@ -29,9 +29,8 @@ public: IPU3Awb(); ~IPU3Awb(); - void initialise(ipu3_uapi_params ¶ms, const Size &bdsOutputSize, struct ipu3_uapi_grid_config &bdsGrid); - void calculateWBGains(const ipu3_uapi_stats_3a *stats); - void updateWbParameters(ipu3_uapi_params ¶ms); + void prepare(IPAContext &context, ipu3_uapi_params ¶ms) override; + void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; struct Ipu3AwbCell { unsigned char greenRedAvg; @@ -72,6 +71,7 @@ public: }; private: + void calculateWBGains(const ipu3_uapi_stats_3a *stats); void generateZones(std::vector &zones); void generateAwbStats(const ipu3_uapi_stats_3a *stats); void clearAwbStats(); From patchwork Wed Aug 18 15:54:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13394 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 299B0C3241 for ; Wed, 18 Aug 2021 15:54:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5DF036892B; Wed, 18 Aug 2021 17:54:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="AS3B3lXr"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B1E83688D2 for ; Wed, 18 Aug 2021 17:54:10 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:946e:2bbb:370e:41e4]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 58BABF1; Wed, 18 Aug 2021 17:54:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1629302050; bh=ulDT1M7nYRYHkE13EUkkjGLbh6IjfrZT36P7GfdwUWo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AS3B3lXrJAGrtWXthLnRRmlz2o1ZgY85BxxOalC7L7DaPuQy8/T+mC9+FeD69LhRF 4vCAeZ2ZKHY83rORhdgVEA7YlHR/wUpV0g77S0T+I4QFJPQcsLsKZUnqmsSl6Qlpod sqWTWTAa/K+7OTJKfK01hfo3NN09VrdZnOFqarlw= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 18 Aug 2021 17:54:02 +0200 Message-Id: <20210818155403.268694-9-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> References: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 8/9] ipa: ipu3: Move IPU3 awb into algorithms X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that the interface is properly used by the AWB class, move it into ipa::ipu3::algorithms and let the loops do the calls. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- .../ipu3/{ipu3_awb.cpp => algorithms/awb.cpp} | 28 +++++++++---------- src/ipa/ipu3/{ipu3_awb.h => algorithms/awb.h} | 20 ++++++------- src/ipa/ipu3/algorithms/meson.build | 1 + src/ipa/ipu3/ipu3.cpp | 11 ++------ src/ipa/ipu3/meson.build | 1 - 5 files changed, 27 insertions(+), 34 deletions(-) rename src/ipa/ipu3/{ipu3_awb.cpp => algorithms/awb.cpp} (94%) rename src/ipa/ipu3/{ipu3_awb.h => algorithms/awb.h} (84%) diff --git a/src/ipa/ipu3/ipu3_awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp similarity index 94% rename from src/ipa/ipu3/ipu3_awb.cpp rename to src/ipa/ipu3/algorithms/awb.cpp index f90aa492..73081808 100644 --- a/src/ipa/ipu3/ipu3_awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -2,9 +2,9 @@ /* * Copyright (C) 2021, Ideas On Board * - * ipu3_awb.cpp - AWB control algorithm + * awb.cpp - AWB control algorithm */ -#include "ipu3_awb.h" +#include "awb.h" #include #include @@ -13,7 +13,7 @@ namespace libcamera { -namespace ipa::ipu3 { +namespace ipa::ipu3::algorithms { LOG_DEFINE_CATEGORY(IPU3Awb) @@ -114,7 +114,7 @@ static const struct ipu3_uapi_ccm_mat_config imguCssCcmDefault = { 0, 0, 8191, 0 }; -IPU3Awb::IPU3Awb() +Awb::Awb() : Algorithm() { asyncResults_.blueGain = 1.0; @@ -125,7 +125,7 @@ IPU3Awb::IPU3Awb() zones_.reserve(kAwbStatsSizeX * kAwbStatsSizeY); } -IPU3Awb::~IPU3Awb() +Awb::~Awb() { } @@ -143,7 +143,7 @@ IPU3Awb::~IPU3Awb() * More detailed information can be found in: * https://en.wikipedia.org/wiki/Color_temperature#Approximation */ -uint32_t IPU3Awb::estimateCCT(double red, double green, double blue) +uint32_t Awb::estimateCCT(double red, double green, double blue) { /* Convert the RGB values to CIE tristimulus values (XYZ) */ double X = (-0.14282) * (red) + (1.54924) * (green) + (-0.95641) * (blue); @@ -160,7 +160,7 @@ uint32_t IPU3Awb::estimateCCT(double red, double green, double blue) } /* Generate an RGB vector with the average values for each region */ -void IPU3Awb::generateZones(std::vector &zones) +void Awb::generateZones(std::vector &zones) { for (unsigned int i = 0; i < kAwbStatsSizeX * kAwbStatsSizeY; i++) { RGB zone; @@ -177,7 +177,7 @@ void IPU3Awb::generateZones(std::vector &zones) } /* Translate the IPU3 statistics into the default statistics region array */ -void IPU3Awb::generateAwbStats(const ipu3_uapi_stats_3a *stats) +void Awb::generateAwbStats(const ipu3_uapi_stats_3a *stats) { uint32_t regionWidth = round(awbGrid_.width / static_cast(kAwbStatsSizeX)); uint32_t regionHeight = round(awbGrid_.height / static_cast(kAwbStatsSizeY)); @@ -209,7 +209,7 @@ void IPU3Awb::generateAwbStats(const ipu3_uapi_stats_3a *stats) } } -void IPU3Awb::clearAwbStats() +void Awb::clearAwbStats() { for (unsigned int i = 0; i < kAwbStatsSizeX * kAwbStatsSizeY; i++) { awbStats_[i].bSum = 0; @@ -220,7 +220,7 @@ void IPU3Awb::clearAwbStats() } } -void IPU3Awb::awbGreyWorld() +void Awb::awbGreyWorld() { LOG(IPU3Awb, Debug) << "Grey world AWB"; /* @@ -260,7 +260,7 @@ void IPU3Awb::awbGreyWorld() asyncResults_.blueGain = blueGain; } -void IPU3Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) +void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) { ASSERT(stats->stats_3a_status.awb_en); zones_.clear(); @@ -275,7 +275,7 @@ void IPU3Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) } } -void IPU3Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) +void Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { calculateWBGains(stats); @@ -289,7 +289,7 @@ void IPU3Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) context.frameContext.awb.gains.red = asyncResults_.redGain; } -void IPU3Awb::prepare(IPAContext &context, ipu3_uapi_params ¶ms) +void Awb::prepare(IPAContext &context, ipu3_uapi_params ¶ms) { params.acc_param.awb.config.rgbs_thr_gr = 8191; params.acc_param.awb.config.rgbs_thr_r = 8191; @@ -336,6 +336,6 @@ void IPU3Awb::prepare(IPAContext &context, ipu3_uapi_params ¶ms) params.use.acc_ccm = 1; } -} /* namespace ipa::ipu3 */ +} /* namespace ipa::ipu3::algorithms */ } /* namespace libcamera */ diff --git a/src/ipa/ipu3/ipu3_awb.h b/src/ipa/ipu3/algorithms/awb.h similarity index 84% rename from src/ipa/ipu3/ipu3_awb.h rename to src/ipa/ipu3/algorithms/awb.h index 4de3fae2..ad640521 100644 --- a/src/ipa/ipu3/ipu3_awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -2,10 +2,10 @@ /* * Copyright (C) 2021, Ideas On Board * - * ipu3_awb.h - IPU3 AWB control algorithm + * awb.h - IPU3 AWB control algorithm */ -#ifndef __LIBCAMERA_IPU3_AWB_H__ -#define __LIBCAMERA_IPU3_AWB_H__ +#ifndef __LIBCAMERA_IPU3_ALGORITHMS_AWB_H__ +#define __LIBCAMERA_IPU3_ALGORITHMS_AWB_H__ #include @@ -13,21 +13,21 @@ #include -#include "algorithms/algorithm.h" +#include "algorithm.h" namespace libcamera { -namespace ipa::ipu3 { +namespace ipa::ipu3::algorithms { /* Region size for the statistics generation algorithm */ static constexpr uint32_t kAwbStatsSizeX = 16; static constexpr uint32_t kAwbStatsSizeY = 12; -class IPU3Awb : public Algorithm +class Awb : public Algorithm { public: - IPU3Awb(); - ~IPU3Awb(); + Awb(); + ~Awb(); void prepare(IPAContext &context, ipu3_uapi_params ¶ms) override; void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; @@ -85,7 +85,7 @@ private: AwbStatus asyncResults_; }; -} /* namespace ipa::ipu3 */ +} /* namespace ipa::ipu3::algorithms */ } /* namespace libcamera*/ -#endif /* __LIBCAMERA_IPU3_AWB_H__ */ +#endif /* __LIBCAMERA_IPU3_ALGORITHMS_AWB_H__ */ diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build index e3ff3b78..87377f4e 100644 --- a/src/ipa/ipu3/algorithms/meson.build +++ b/src/ipa/ipu3/algorithms/meson.build @@ -2,5 +2,6 @@ ipu3_ipa_algorithms = files([ 'algorithm.cpp', + 'awb.cpp', 'contrast.cpp', ]) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 423cc957..f4f49025 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -30,9 +30,9 @@ #include "libcamera/internal/mapped_framebuffer.h" #include "algorithms/algorithm.h" +#include "algorithms/awb.h" #include "algorithms/contrast.h" #include "ipu3_agc.h" -#include "ipu3_awb.h" #include "libipa/camera_sensor_helper.h" static constexpr uint32_t kMaxCellWidthPerSet = 160; @@ -136,8 +136,6 @@ private: uint32_t minGain_; uint32_t maxGain_; - /* Interface to the AWB algorithm */ - std::unique_ptr awbAlgo_; /* Interface to the AEC/AGC algorithm */ std::unique_ptr agcAlgo_; /* Interface to the Camera Helper */ @@ -220,6 +218,7 @@ int IPAIPU3::init(const IPASettings &settings, *ipaControls = ControlInfoMap(std::move(controls), controls::controls); /* Construct our Algorithms */ + algorithms_.emplace_back(new algorithms::Awb()); algorithms_.emplace_back(new algorithms::Contrast()); return 0; @@ -338,7 +337,6 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) return ret; } - awbAlgo_ = std::make_unique(); agcAlgo_ = std::make_unique(); agcAlgo_->configure(context_, configInfo); @@ -418,8 +416,6 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) for (auto const &algo : algorithms_) algo->prepare(context_, params_); - awbAlgo_->prepare(context_, params_); - *params = params_; IPU3Action op; @@ -443,9 +439,6 @@ void IPAIPU3::parseStatistics(unsigned int frame, agcAlgo_->process(context_, stats); exposure_ = context_.frameContext.agc.exposure; gain_ = camHelper_->gainCode(context_.frameContext.agc.gain); - - awbAlgo_->process(context_, stats); - setControls(frame); /* \todo Use VBlank value calculated from each frame exposure. */ diff --git a/src/ipa/ipu3/meson.build b/src/ipa/ipu3/meson.build index fcb27d68..d1126947 100644 --- a/src/ipa/ipu3/meson.build +++ b/src/ipa/ipu3/meson.build @@ -7,7 +7,6 @@ ipa_name = 'ipa_ipu3' ipu3_ipa_sources = files([ 'ipu3.cpp', 'ipu3_agc.cpp', - 'ipu3_awb.cpp', ]) ipu3_ipa_sources += ipu3_ipa_algorithms From patchwork Wed Aug 18 15:54:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13395 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B2DF3BD87C for ; Wed, 18 Aug 2021 15:54:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 660EA6891F; Wed, 18 Aug 2021 17:54:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CRUD1MV7"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DE61768895 for ; Wed, 18 Aug 2021 17:54:10 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:946e:2bbb:370e:41e4]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 902D1EE; Wed, 18 Aug 2021 17:54:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1629302050; bh=XEdBzGBGCWWLbfKjxcTgrWiUb1aS5gSnnYPdSjZ5dj0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CRUD1MV7dmk6h0hEEBNSp9SuQ19RIAofJCL5YcRdj2UIHH/zusEdlpnE5ddQFJ4so YK1842cHAsDpY8bK1a0xeLs/EJ+lgVP9Ngs8Y56rRArY4+DIXn6g1TlMvAKqAPV/Sh oPCrGJjvhu2iahKykCRdPShLNrCyZOI52Wde3KKU= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 18 Aug 2021 17:54:03 +0200 Message-Id: <20210818155403.268694-10-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> References: <20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 9/9] ipa: ipu3: Move IPU3 agc into algorithms X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that the interface is properly used by the AGC class, move it into ipa::ipu3::algorithms and let the loops do the calls. As we need to exchange the exposure_ and gain_ by passing them through the FrameContext, use the calculated values in setControls() function to ease the reading. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- .../ipu3/{ipu3_agc.cpp => algorithms/agc.cpp} | 21 +++++++++---------- src/ipa/ipu3/{ipu3_agc.h => algorithms/agc.h} | 20 +++++++++--------- src/ipa/ipu3/algorithms/meson.build | 1 + src/ipa/ipu3/ipu3.cpp | 15 +++++-------- src/ipa/ipu3/meson.build | 1 - 5 files changed, 26 insertions(+), 32 deletions(-) rename src/ipa/ipu3/{ipu3_agc.cpp => algorithms/agc.cpp} (93%) rename src/ipa/ipu3/{ipu3_agc.h => algorithms/agc.h} (73%) diff --git a/src/ipa/ipu3/ipu3_agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp similarity index 93% rename from src/ipa/ipu3/ipu3_agc.cpp rename to src/ipa/ipu3/algorithms/agc.cpp index 1c1f5fb5..bb119cb4 100644 --- a/src/ipa/ipu3/ipu3_agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -5,7 +5,7 @@ * ipu3_agc.cpp - AGC/AEC control algorithm */ -#include "ipu3_agc.h" +#include "agc.h" #include #include @@ -21,7 +21,7 @@ namespace libcamera { using namespace std::literals::chrono_literals; -namespace ipa::ipu3 { +namespace ipa::ipu3::algorithms { LOG_DEFINE_CATEGORY(IPU3Agc) @@ -50,14 +50,13 @@ static constexpr double kEvGainTarget = 0.5; /* A cell is 8 bytes and contains averages for RGB values and saturation ratio */ static constexpr uint8_t kCellSize = 8; -IPU3Agc::IPU3Agc() +Agc::Agc() : frameCount_(0), lastFrame_(0), iqMean_(0.0), lineDuration_(0s), - maxExposureTime_(0s), prevExposure_(0s), prevExposureNoDg_(0s), - currentExposure_(0s), currentExposureNoDg_(0s) + maxExposureTime_(0s), prevExposure_(0s), prevExposureNoDg_(0s) { } -int IPU3Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) +int Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) { aeGrid_ = context.configuration.grid.bdsGrid; @@ -67,7 +66,7 @@ int IPU3Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) return 0; } -void IPU3Agc::processBrightness(const ipu3_uapi_stats_3a *stats) +void Agc::processBrightness(const ipu3_uapi_stats_3a *stats) { const struct ipu3_uapi_grid_config statsAeGrid = stats->stats_4a_config.awb_config.grid; Rectangle aeRegion = { statsAeGrid.x_start, @@ -108,7 +107,7 @@ void IPU3Agc::processBrightness(const ipu3_uapi_stats_3a *stats) iqMean_ = Histogram(Span(hist)).interQuantileMean(0.98, 1.0); } -void IPU3Agc::filterExposure() +void Agc::filterExposure() { double speed = 0.2; if (prevExposure_ == 0s) { @@ -142,7 +141,7 @@ void IPU3Agc::filterExposure() LOG(IPU3Agc, Debug) << "After filtering, total_exposure " << prevExposure_; } -void IPU3Agc::lockExposureGain(uint32_t &exposure, double &gain) +void Agc::lockExposureGain(uint32_t &exposure, double &gain) { /* Algorithm initialization should wait for first valid frames */ /* \todo - have a number of frames given by DelayedControls ? @@ -185,7 +184,7 @@ void IPU3Agc::lockExposureGain(uint32_t &exposure, double &gain) lastFrame_ = frameCount_; } -void IPU3Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) +void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { uint32_t &exposure = context.frameContext.agc.exposure; double &gain = context.frameContext.agc.gain; @@ -194,6 +193,6 @@ void IPU3Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) frameCount_++; } -} /* namespace ipa::ipu3 */ +} /* namespace ipa::ipu3::algorithms */ } /* namespace libcamera */ diff --git a/src/ipa/ipu3/ipu3_agc.h b/src/ipa/ipu3/algorithms/agc.h similarity index 73% rename from src/ipa/ipu3/ipu3_agc.h rename to src/ipa/ipu3/algorithms/agc.h index 0e922664..1739d2fd 100644 --- a/src/ipa/ipu3/ipu3_agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -2,10 +2,10 @@ /* * Copyright (C) 2021, Ideas On Board * - * ipu3_agc.h - IPU3 AGC/AEC control algorithm + * agc.h - IPU3 AGC/AEC control algorithm */ -#ifndef __LIBCAMERA_IPU3_AGC_H__ -#define __LIBCAMERA_IPU3_AGC_H__ +#ifndef __LIBCAMERA_IPU3_ALGORITHMS_AGC_H__ +#define __LIBCAMERA_IPU3_ALGORITHMS_AGC_H__ #include @@ -13,21 +13,21 @@ #include -#include "algorithms/algorithm.h" +#include "algorithm.h" namespace libcamera { struct IPACameraSensorInfo; -namespace ipa::ipu3 { +namespace ipa::ipu3::algorithms { using utils::Duration; -class IPU3Agc : public Algorithm +class Agc : public Algorithm { public: - IPU3Agc(); - ~IPU3Agc() = default; + Agc(); + ~Agc() = default; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; @@ -53,8 +53,8 @@ private: Duration currentExposureNoDg_; }; -} /* namespace ipa::ipu3 */ +} /* namespace ipa::ipu3::algorithms */ } /* namespace libcamera */ -#endif /* __LIBCAMERA_IPU3_AGC_H__ */ +#endif /* __LIBCAMERA_IPU3_ALGORITHMS_AGC_H__ */ diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build index 87377f4e..acaadd90 100644 --- a/src/ipa/ipu3/algorithms/meson.build +++ b/src/ipa/ipu3/algorithms/meson.build @@ -2,6 +2,7 @@ ipu3_ipa_algorithms = files([ 'algorithm.cpp', + 'agc.cpp', 'awb.cpp', 'contrast.cpp', ]) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index f4f49025..d73ec79d 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -30,9 +30,9 @@ #include "libcamera/internal/mapped_framebuffer.h" #include "algorithms/algorithm.h" +#include "algorithms/agc.h" #include "algorithms/awb.h" #include "algorithms/contrast.h" -#include "ipu3_agc.h" #include "libipa/camera_sensor_helper.h" static constexpr uint32_t kMaxCellWidthPerSet = 160; @@ -136,8 +136,6 @@ private: uint32_t minGain_; uint32_t maxGain_; - /* Interface to the AEC/AGC algorithm */ - std::unique_ptr agcAlgo_; /* Interface to the Camera Helper */ std::unique_ptr camHelper_; @@ -218,6 +216,7 @@ int IPAIPU3::init(const IPASettings &settings, *ipaControls = ControlInfoMap(std::move(controls), controls::controls); /* Construct our Algorithms */ + algorithms_.emplace_back(new algorithms::Agc()); algorithms_.emplace_back(new algorithms::Awb()); algorithms_.emplace_back(new algorithms::Contrast()); @@ -337,10 +336,6 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) return ret; } - - agcAlgo_ = std::make_unique(); - agcAlgo_->configure(context_, configInfo); - return 0; } @@ -436,9 +431,6 @@ void IPAIPU3::parseStatistics(unsigned int frame, for (auto const &algo : algorithms_) algo->process(context_, stats); - agcAlgo_->process(context_, stats); - exposure_ = context_.frameContext.agc.exposure; - gain_ = camHelper_->gainCode(context_.frameContext.agc.gain); setControls(frame); /* \todo Use VBlank value calculated from each frame exposure. */ @@ -458,6 +450,9 @@ void IPAIPU3::setControls(unsigned int frame) IPU3Action op; op.op = ActionSetSensorControls; + exposure_ = context_.frameContext.agc.exposure; + gain_ = camHelper_->gainCode(context_.frameContext.agc.gain); + ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); diff --git a/src/ipa/ipu3/meson.build b/src/ipa/ipu3/meson.build index d1126947..39320980 100644 --- a/src/ipa/ipu3/meson.build +++ b/src/ipa/ipu3/meson.build @@ -6,7 +6,6 @@ ipa_name = 'ipa_ipu3' ipu3_ipa_sources = files([ 'ipu3.cpp', - 'ipu3_agc.cpp', ]) ipu3_ipa_sources += ipu3_ipa_algorithms