From patchwork Fri Aug 20 06:53:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13409 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E9937BD87D for ; Fri, 20 Aug 2021 06:53:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9219E688A6; Fri, 20 Aug 2021 08:53:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rVfgBVuT"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3F7A368890 for ; Fri, 20 Aug 2021 08:53:21 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:e43b:edb2:932:6b0a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CB9E11207; Fri, 20 Aug 2021 08:53:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1629442400; bh=fCwlPqHqmSwtVTrbXxA7IJNmmelo+jPfbfXmZIgSWIk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rVfgBVuTmCkmIXQlSVwMF8GhzArZxdCcLrjkg+ElND1bc8DGWvYJjoP9TT5EmpKin iqy5XkqoYTbB5N2fZsJzrHIhi4AEtXvn9f6aeTvTNbP6eIedbM9wz/mCoc/zlBUzpG yCUuL9WYK4ZMDRt6TrYAARUT3mEwCeDF2rW6MI9Y= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Fri, 20 Aug 2021 08:53:08 +0200 Message-Id: <20210820065316.44343-2-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210820065316.44343-1-jeanmichel.hautbois@ideasonboard.com> References: <20210820065316.44343-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 1/9] ipa: move libipa::Algorithm to ipa/ipu3/algorithms X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The abstract Algorithm class was originally placed in libipa as an attempt define a generic algorithm container. This was a little optimistic and pushed a bit far too early. Move the Algorithm class into the IPU3 which is the only user of the class, as we adapt it to support modular algorithm components for the IPU3. Not documenting the namespace may cause issues with Doxygen in libipa. The file libipa.cpp is thus created as an empty file for now, but we can leverage it in the future to add more global libipa documentation, and possibly code too. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham kieran.bingham@ideasonboard.com Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- .../{libipa => ipu3/algorithms}/algorithm.cpp | 17 ++++--------- src/ipa/ipu3/algorithms/algorithm.h | 24 +++++++++++++++++++ src/ipa/ipu3/algorithms/meson.build | 5 ++++ src/ipa/ipu3/ipu3_agc.h | 2 +- src/ipa/ipu3/ipu3_awb.h | 2 +- src/ipa/ipu3/meson.build | 4 ++++ src/ipa/libipa/algorithm.h | 24 ------------------- src/ipa/libipa/libipa.cpp | 22 +++++++++++++++++ src/ipa/libipa/meson.build | 5 ++-- 9 files changed, 63 insertions(+), 42 deletions(-) rename src/ipa/{libipa => ipu3/algorithms}/algorithm.cpp (55%) create mode 100644 src/ipa/ipu3/algorithms/algorithm.h create mode 100644 src/ipa/ipu3/algorithms/meson.build delete mode 100644 src/ipa/libipa/algorithm.h create mode 100644 src/ipa/libipa/libipa.cpp diff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/ipu3/algorithms/algorithm.cpp similarity index 55% rename from src/ipa/libipa/algorithm.cpp rename to src/ipa/ipu3/algorithms/algorithm.cpp index 930f9353..dd46846a 100644 --- a/src/ipa/libipa/algorithm.cpp +++ b/src/ipa/ipu3/algorithms/algorithm.cpp @@ -2,7 +2,7 @@ /* * Copyright (C) 2021, Ideas On Board * - * algorithm.cpp - ISP control algorithms + * algorithm.cpp - IPU3 control algorithm interface */ #include "algorithm.h" @@ -14,26 +14,17 @@ namespace libcamera { -/** - * \brief The IPA namespace - * - * The IPA namespace groups all types specific to IPA modules. It serves as the - * top-level namespace for the IPA library libipa, and also contains - * module-specific namespaces for IPA modules. - */ -namespace ipa { +namespace ipa::ipu3 { /** * \class Algorithm - * \brief The base class for all IPA algorithms + * \brief The base class for all IPU3 algorithms * * The Algorithm class defines a standard interface for IPA algorithms. By * abstracting algorithms, it makes possible the implementation of generic code * to manage algorithms regardless of their specific type. */ -Algorithm::~Algorithm() = default; - -} /* namespace ipa */ +} /* namespace ipa::ipu3 */ } /* namespace libcamera */ diff --git a/src/ipa/ipu3/algorithms/algorithm.h b/src/ipa/ipu3/algorithms/algorithm.h new file mode 100644 index 00000000..072f01c4 --- /dev/null +++ b/src/ipa/ipu3/algorithms/algorithm.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * algorithm.h - IPU3 control algorithm interface + */ +#ifndef __LIBCAMERA_IPA_IPU3_ALGORITHM_H__ +#define __LIBCAMERA_IPA_IPU3_ALGORITHM_H__ + +namespace libcamera { + +namespace ipa::ipu3 { + +class Algorithm +{ +public: + virtual ~Algorithm() {} +}; + +} /* namespace ipa::ipu3 */ + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_IPA_IPU3_ALGORITHM_H__ */ diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build new file mode 100644 index 00000000..dc538b79 --- /dev/null +++ b/src/ipa/ipu3/algorithms/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +ipu3_ipa_algorithms = files([ + 'algorithm.cpp', +]) diff --git a/src/ipa/ipu3/ipu3_agc.h b/src/ipa/ipu3/ipu3_agc.h index 9f3d4257..f00b98d6 100644 --- a/src/ipa/ipu3/ipu3_agc.h +++ b/src/ipa/ipu3/ipu3_agc.h @@ -13,7 +13,7 @@ #include -#include "libipa/algorithm.h" +#include "algorithms/algorithm.h" namespace libcamera { diff --git a/src/ipa/ipu3/ipu3_awb.h b/src/ipa/ipu3/ipu3_awb.h index 122cf68c..ea2d4320 100644 --- a/src/ipa/ipu3/ipu3_awb.h +++ b/src/ipa/ipu3/ipu3_awb.h @@ -13,7 +13,7 @@ #include -#include "libipa/algorithm.h" +#include "algorithms/algorithm.h" namespace libcamera { diff --git a/src/ipa/ipu3/meson.build b/src/ipa/ipu3/meson.build index b6364190..fcb27d68 100644 --- a/src/ipa/ipu3/meson.build +++ b/src/ipa/ipu3/meson.build @@ -1,5 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 +subdir('algorithms') + ipa_name = 'ipa_ipu3' ipu3_ipa_sources = files([ @@ -8,6 +10,8 @@ ipu3_ipa_sources = files([ 'ipu3_awb.cpp', ]) +ipu3_ipa_sources += ipu3_ipa_algorithms + mod = shared_module(ipa_name, [ipu3_ipa_sources, libcamera_generated_ipa_headers], name_prefix : '', diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h deleted file mode 100644 index 89cee4c4..00000000 --- a/src/ipa/libipa/algorithm.h +++ /dev/null @@ -1,24 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2021, Ideas On Board - * - * algorithm.h - ISP control algorithm interface - */ -#ifndef __LIBCAMERA_IPA_LIBIPA_ALGORITHM_H__ -#define __LIBCAMERA_IPA_LIBIPA_ALGORITHM_H__ - -namespace libcamera { - -namespace ipa { - -class Algorithm -{ -public: - virtual ~Algorithm(); -}; - -} /* namespace ipa */ - -} /* namespace libcamera */ - -#endif /* __LIBCAMERA_IPA_LIBIPA_ALGORITHM_H__ */ diff --git a/src/ipa/libipa/libipa.cpp b/src/ipa/libipa/libipa.cpp new file mode 100644 index 00000000..08bc3541 --- /dev/null +++ b/src/ipa/libipa/libipa.cpp @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * libipa.cpp - libipa interface + */ + +namespace libcamera { + +/** + * \brief The IPA namespace + * + * The IPA namespace groups all types specific to IPA modules. It serves as the + * top-level namespace for the IPA library libipa, and also contains + * module-specific namespaces for IPA modules. + */ +namespace ipa { + +} /* namespace ipa */ + +} /* namespace libcamera */ + diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index 3fda7c00..4d073a03 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -1,15 +1,14 @@ # SPDX-License-Identifier: CC0-1.0 libipa_headers = files([ - 'algorithm.h', 'camera_sensor_helper.h', 'histogram.h' ]) libipa_sources = files([ - 'algorithm.cpp', 'camera_sensor_helper.cpp', - 'histogram.cpp' + 'histogram.cpp', + 'libipa.cpp', ]) libipa_includes = include_directories('..') From patchwork Fri Aug 20 06:53:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13410 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0F2EBC3241 for ; Fri, 20 Aug 2021 06:53:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 85E4C688AC; Fri, 20 Aug 2021 08:53:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="olji0kwz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6334668892 for ; Fri, 20 Aug 2021 08:53:21 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:e43b:edb2:932:6b0a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 07431150A; Fri, 20 Aug 2021 08:53:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1629442401; bh=1LNpQqPrCfIRYk2uPdSvjPph7LnQvqQ2BwsO6OeT/L4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=olji0kwzVg3yG6wdvA+Gk9MaYiJeBn5vflrzeuiucTZZVHixGvEEwgiNXb7u4CjcB aH91c/P8aCr3tQqF0iMO05r6eyPXeletvPkkbH/KB+gR6RGgfsZ8VTSuGXHxcjsOQr snp4CkSVxyf7TdsUzinvGdB4KXR+/BVEYzh9z1Eo= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Fri, 20 Aug 2021 08:53:09 +0200 Message-Id: <20210820065316.44343-3-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210820065316.44343-1-jeanmichel.hautbois@ideasonboard.com> References: <20210820065316.44343-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 2/9] ipa: ipu3: Introduce a Context structure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" An increasing amount of data and information needs to be shared between the components that build up to implement image processing algorithms. Create a context structure which will allow us to work towards calling algorithms in a modular way, and sharing information between the modules. The IPA context is a global context set at configure time (IPASessionConfiguration) and a per-frame context (IPAFrameContext) used while streaming. Signed-off-by: Kieran Bingham Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/ipu3/ipa_context.h | 32 ++++++++++++++++++++++++++ src/ipa/ipu3/ipu3.cpp | 46 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 78 insertions(+) create mode 100644 src/ipa/ipu3/ipa_context.h diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h new file mode 100644 index 00000000..2706d3ca --- /dev/null +++ b/src/ipa/ipu3/ipa_context.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Google Inc. + * + * ipu3_ipa_context.h - IPU3 IPA Context + * + */ +#ifndef __LIBCAMERA_IPU3_IPA_CONTEXT_H__ +#define __LIBCAMERA_IPU3_IPA_CONTEXT_H__ + +#include + +namespace libcamera { + +namespace ipa::ipu3 { + +struct IPASessionConfiguration { +}; + +struct IPAFrameContext { +}; + +struct IPAContext { + IPASessionConfiguration configuration; + IPAFrameContext frameContext; +}; + +} /* namespace ipa::ipu3 */ + +} /* namespace libcamera*/ + +#endif /* __LIBCAMERA_IPU3_IPA_CONTEXT_H__ */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index c34fa460..d9be37e9 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -33,6 +33,52 @@ #include "ipu3_awb.h" #include "libipa/camera_sensor_helper.h" +/** + * \file ipa_context.h + * \brief Context and state information shared between the algorithms + */ + +/** + * \struct IPASessionConfiguration + * \brief Session configuration for the IPA module + * + * The session configuration contains all IPA configuration parameters that + * remain constant during the capture session, from IPA module start to stop. + * It is typically set during the configure() operation of the IPA module, but + * may also be updated in the start() operation. + */ + +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + * + * The frame context stores data specific to a single frame processed by the + * IPA. Each frame processed by the IPA has a context associated with it, + * accessible through the IPAContext structure. + * + * \todo Detail how to access contexts for a particular frame + * + * Each of the fields in the frame context belongs to either a specific + * algorithm, or to the top-level IPA module. A field may be read by any + * algorithm, but should only be written by its owner. + */ + +/** + * \struct IPAContext + * \brief Global IPA context data shared between all algorithms + * + * \var IPAContext::configuration + * \brief The IPA session configuration, immutable during the session + * + * \var IPAContext::frameContext + * \brief The frame context for the frame being processed + * + * \todo While the frame context is supposed to be per-frame, this + * single frame context stores data related to both the current frame + * and the previous frames, with fields being updated as the algorithms + * are run. This needs to be turned into real per-frame data storage. + */ + static constexpr uint32_t kMaxCellWidthPerSet = 160; static constexpr uint32_t kMaxCellHeightPerSet = 56; From patchwork Fri Aug 20 06:53:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13411 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EA2F5C3242 for ; Fri, 20 Aug 2021 06:53:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 487E868921; Fri, 20 Aug 2021 08:53:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="sP9zdJU4"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B5459605A9 for ; Fri, 20 Aug 2021 08:53:21 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:e43b:edb2:932:6b0a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 71C9B18EE; Fri, 20 Aug 2021 08:53:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1629442401; bh=55VDxdprmNyI+WSM6bQpcxugiCnPww7K5wbzTVZ7Pyg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sP9zdJU4QrtRUCdcOeaU67nDCSj2gfwjZTVT/LLyH6rGM2SnMGbr06t9hyRHXdHEp m2YQigimJAh4Ri+ULIUwz1V75UOBaMHOxCbI7HZ7NmCtRrAUXkWNiGFVSBUz3dHaqf UYY0GRmThR5xTfl+NyGz0VwOcLMtQ+EwvHEdE6/M= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Fri, 20 Aug 2021 08:53:11 +0200 Message-Id: <20210820065316.44343-5-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210820065316.44343-1-jeanmichel.hautbois@ideasonboard.com> References: <20210820065316.44343-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 4/9] ipa: ipu3: Introduce modular algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a new modular framework for algorithms with a common context structure that is passed to each algorithm through a common API. This patch: - removes all the local references from IPAIPU3 and uses IPAContext - implements the list of pointers and the loop at configure call on each algorithm - loops in fillParams on each prepare() call on the algorithm list - loops in prepareStats on each process() call on the algorithm list Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/ipu3/ipa_context.h | 6 ++++ src/ipa/ipu3/ipu3.cpp | 57 ++++++++++++++++++++++++++++++-------- 2 files changed, 51 insertions(+), 12 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 2706d3ca..a031ab83 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -10,11 +10,17 @@ #include +#include + namespace libcamera { namespace ipa::ipu3 { struct IPASessionConfiguration { + struct { + ipu3_uapi_grid_config bdsGrid; + Size bdsOutputSize; + } grid; }; struct IPAFrameContext { diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index d9be37e9..4d5b0af1 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -29,6 +29,7 @@ #include "libcamera/internal/mapped_framebuffer.h" +#include "algorithms/algorithm.h" #include "ipu3_agc.h" #include "ipu3_awb.h" #include "libipa/camera_sensor_helper.h" @@ -79,6 +80,17 @@ * are run. This needs to be turned into real per-frame data storage. */ +/** + * \struct IPASessionConfiguration::grid + * \brief Grid configuration of the IPA + * + * \var IPASessionConfiguration::grid::bdsGrid + * \brief Bayer Down Scaler grid plane config used by the kernel + * + * \var IPASessionConfiguration::grid::bdsOutputSize + * \brief BDS output size configured by the pipeline handler + */ + static constexpr uint32_t kMaxCellWidthPerSet = 160; static constexpr uint32_t kMaxCellHeightPerSet = 56; @@ -137,10 +149,12 @@ private: /* Interface to the Camera Helper */ std::unique_ptr camHelper_; + /* Maintain the algorithms used by the IPA */ + std::list> algorithms_; + /* Local parameter storage */ + struct IPAContext context_; struct ipu3_uapi_params params_; - - struct ipu3_uapi_grid_config bdsGrid_; }; /** @@ -237,7 +251,9 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) uint32_t minError = std::numeric_limits::max(); Size best; Size bestLog2; - bdsGrid_ = {}; + + /* Set the BDS output size in the IPAConfiguration structure */ + context_.configuration.grid.bdsOutputSize = bdsOutputSize; for (uint32_t widthShift = 3; widthShift <= 7; ++widthShift) { uint32_t width = std::min(kMaxCellWidthPerSet, @@ -261,14 +277,17 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) } } - bdsGrid_.width = best.width >> bestLog2.width; - bdsGrid_.block_width_log2 = bestLog2.width; - bdsGrid_.height = best.height >> bestLog2.height; - bdsGrid_.block_height_log2 = bestLog2.height; + struct ipu3_uapi_grid_config &bdsGrid = context_.configuration.grid.bdsGrid; + bdsGrid.x_start = 0; + bdsGrid.y_start = 0; + bdsGrid.width = best.width >> bestLog2.width; + bdsGrid.block_width_log2 = bestLog2.width; + bdsGrid.height = best.height >> bestLog2.height; + bdsGrid.block_height_log2 = bestLog2.height; LOG(IPAIPU3, Debug) << "Best grid found is: (" - << (int)bdsGrid_.width << " << " << (int)bdsGrid_.block_width_log2 << ") x (" - << (int)bdsGrid_.height << " << " << (int)bdsGrid_.block_height_log2 << ")"; + << (int)bdsGrid.width << " << " << (int)bdsGrid.block_width_log2 << ") x (" + << (int)bdsGrid.height << " << " << (int)bdsGrid.block_height_log2 << ")"; } int IPAIPU3::configure(const IPAConfigInfo &configInfo) @@ -310,15 +329,23 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) defVBlank_ = itVBlank->second.def().get(); + /* Clean context and IPU3 parameters at configuration */ params_ = {}; + context_ = {}; calculateBdsGrid(configInfo.bdsOutputSize); - awbAlgo_ = std::make_unique(); - awbAlgo_->initialise(params_, configInfo.bdsOutputSize, bdsGrid_); + for (auto const &algo : algorithms_) { + int ret = algo->configure(context_, configInfo); + if (ret) + return ret; + } + awbAlgo_ = std::make_unique(); + awbAlgo_->initialise(params_, context_.configuration.grid.bdsOutputSize, + context_.configuration.grid.bdsGrid); agcAlgo_ = std::make_unique(); - agcAlgo_->initialise(bdsGrid_, sensorInfo_); + agcAlgo_->initialise(context_.configuration.grid.bdsGrid, sensorInfo_); return 0; } @@ -392,6 +419,9 @@ void IPAIPU3::processControls([[maybe_unused]] unsigned int frame, void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) { + for (auto const &algo : algorithms_) + algo->prepare(context_, ¶ms_); + if (agcAlgo_->updateControls()) awbAlgo_->updateWbParameters(params_, agcAlgo_->gamma()); @@ -409,6 +439,9 @@ void IPAIPU3::parseStatistics(unsigned int frame, { ControlList ctrls(controls::controls); + for (auto const &algo : algorithms_) + algo->process(context_, stats); + double gain = camHelper_->gain(gain_); agcAlgo_->process(stats, exposure_, gain); gain_ = camHelper_->gainCode(gain); From patchwork Fri Aug 20 06:53:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13414 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 56E6AC3246 for ; Fri, 20 Aug 2021 06:53:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6D2EE68918; Fri, 20 Aug 2021 08:53:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dKWYKVZR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 389586891A for ; Fri, 20 Aug 2021 08:53:23 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:e43b:edb2:932:6b0a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DD7368C8; Fri, 20 Aug 2021 08:53:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1629442402; bh=Kf8GtVvGycpEgCk/ah5RWS6GYVMnp4Lgo+I526NA+7M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dKWYKVZRTXyx1BgBJTUpyp3Wu9GhYeq9H6GDCApiy+rvjWKa+/CYS9Bk5h4h+8Nyk yNhwcr9lJ/r3C6BRqmpBoVb/kno6GNt2kJ8Tj1ynddSRMYkWHeuAHT5Nm45FnE29/Q Rrk9nqRnRzDe1II9vbvjTdMs7lSnGtKK0nMGGPoI= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Fri, 20 Aug 2021 08:53:13 +0200 Message-Id: <20210820065316.44343-7-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210820065316.44343-1-jeanmichel.hautbois@ideasonboard.com> References: <20210820065316.44343-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 6/9] ipa: ipu3: convert AWB to the new algorithm interface X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the stats are received, pass them with the context to the existing AWB algorithm. IPAFrameContext now has a new structure to store the gains calculated by the AWB algorithm. When an EventFillParams event is received, call prepare() and set the new gains accordingly in the params structure. There is no more a need for the IPU3Awb::initialise() function, as the params are always set in prepare(). Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/ipu3/ipa_context.h | 8 +++ src/ipa/ipu3/ipu3.cpp | 23 ++++++-- src/ipa/ipu3/ipu3_awb.cpp | 109 ++++++++++++++++++------------------- src/ipa/ipu3/ipu3_awb.h | 6 +- 4 files changed, 83 insertions(+), 63 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 4b12f129..24dd8bf7 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -24,6 +24,14 @@ struct IPASessionConfiguration { }; struct IPAFrameContext { + struct { + struct { + double red; + double green; + double blue; + } gains; + } awb; + struct { struct ipu3_uapi_gamma_corr_lut gammaCorrection; } toneMapping; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 3713b07b..823df34b 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -92,6 +92,23 @@ * \brief BDS output size configured by the pipeline handler */ +/** + * \struct IPAFrameContext::awb + * \brief Context for the Automatic White Balance algorithm + * + * \struct IPAFrameContext::awb::gains + * \brief White balance gains + * + * \var IPAFrameContext::awb::gains::red + * \brief White balance gain for R channel + * + * \var IPAFrameContext::awb::gains::green + * \brief White balance gain for G channel + * + * \var IPAFrameContext::awb::gains::blue + * \brief White balance gain for B channel + */ + /** * \struct IPAFrameContext::toneMapping * \brief Context for ToneMapping and Gamma control @@ -357,8 +374,6 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) } awbAlgo_ = std::make_unique(); - awbAlgo_->initialise(params_, context_.configuration.grid.bdsOutputSize, - context_.configuration.grid.bdsGrid); agcAlgo_ = std::make_unique(); agcAlgo_->initialise(context_.configuration.grid.bdsGrid, sensorInfo_); @@ -438,7 +453,7 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) algo->prepare(context_, ¶ms_); if (agcAlgo_->updateControls()) - awbAlgo_->updateWbParameters(params_); + awbAlgo_->prepare(context_, ¶ms_); *params = params_; @@ -461,7 +476,7 @@ void IPAIPU3::parseStatistics(unsigned int frame, agcAlgo_->process(stats, exposure_, gain); gain_ = camHelper_->gainCode(gain); - awbAlgo_->calculateWBGains(stats); + awbAlgo_->process(context_, stats); if (agcAlgo_->updateControls()) setControls(frame); diff --git a/src/ipa/ipu3/ipu3_awb.cpp b/src/ipa/ipu3/ipu3_awb.cpp index c2d9e0c1..728162fe 100644 --- a/src/ipa/ipu3/ipu3_awb.cpp +++ b/src/ipa/ipu3/ipu3_awb.cpp @@ -107,25 +107,6 @@ static const struct ipu3_uapi_bnr_static_config imguCssBnrDefaults = { .opt_center_sqr = { 419904, 133956 }, }; -/* Default settings for Auto White Balance replicated from the Kernel*/ -static const struct ipu3_uapi_awb_config_s imguCssAwbDefaults = { - .rgbs_thr_gr = 8191, - .rgbs_thr_r = 8191, - .rgbs_thr_gb = 8191, - .rgbs_thr_b = 8191 | IPU3_UAPI_AWB_RGBS_THR_B_EN | IPU3_UAPI_AWB_RGBS_THR_B_INCL_SAT, - .grid = { - .width = 160, - .height = 36, - .block_width_log2 = 3, - .block_height_log2 = 4, - .height_per_slice = 1, /* Overridden by kernel. */ - .x_start = 0, - .y_start = 0, - .x_end = 0, - .y_end = 0, - }, -}; - /* Default color correction matrix defined as an identity matrix */ static const struct ipu3_uapi_ccm_mat_config imguCssCcmDefault = { 8191, 0, 0, 0, @@ -140,41 +121,12 @@ IPU3Awb::IPU3Awb() asyncResults_.greenGain = 1.0; asyncResults_.redGain = 1.0; asyncResults_.temperatureK = 4500; -} - -IPU3Awb::~IPU3Awb() -{ -} - -void IPU3Awb::initialise(ipu3_uapi_params ¶ms, const Size &bdsOutputSize, struct ipu3_uapi_grid_config &bdsGrid) -{ - params.use.acc_awb = 1; - params.acc_param.awb.config = imguCssAwbDefaults; - - awbGrid_ = bdsGrid; - params.acc_param.awb.config.grid = awbGrid_; - - params.use.acc_bnr = 1; - params.acc_param.bnr = imguCssBnrDefaults; - /** - * Optical center is column (respectively row) startminus X (respectively Y) center. - * For the moment use BDS as a first approximation, but it should - * be calculated based on Shading (SHD) parameters. - */ - params.acc_param.bnr.column_size = bdsOutputSize.width; - params.acc_param.bnr.opt_center.x_reset = awbGrid_.x_start - (bdsOutputSize.width / 2); - params.acc_param.bnr.opt_center.y_reset = awbGrid_.y_start - (bdsOutputSize.height / 2); - params.acc_param.bnr.opt_center_sqr.x_sqr_reset = params.acc_param.bnr.opt_center.x_reset - * params.acc_param.bnr.opt_center.x_reset; - params.acc_param.bnr.opt_center_sqr.y_sqr_reset = params.acc_param.bnr.opt_center.y_reset - * params.acc_param.bnr.opt_center.y_reset; - - params.use.acc_ccm = 1; - params.acc_param.ccm = imguCssCcmDefault; zones_.reserve(kAwbStatsSizeX * kAwbStatsSizeY); } +IPU3Awb::~IPU3Awb() = default; + /** * The function estimates the correlated color temperature using * from RGB color space input. @@ -321,22 +273,67 @@ void IPU3Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) } } -void IPU3Awb::updateWbParameters(ipu3_uapi_params ¶ms) +void IPU3Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { + calculateWBGains(stats); + + /* + * Gains are only recalculated if enough zones were detected. + * The results are cached, so if no results were calculated, we set the + * cached values from asyncResults_ here. + */ + context.frameContext.awb.gains.blue = asyncResults_.blueGain; + context.frameContext.awb.gains.green = asyncResults_.greenGain; + context.frameContext.awb.gains.red = asyncResults_.redGain; +} + +void IPU3Awb::prepare(IPAContext &context, ipu3_uapi_params *params) +{ + params->acc_param.awb.config.rgbs_thr_gr = 8191; + params->acc_param.awb.config.rgbs_thr_r = 8191; + params->acc_param.awb.config.rgbs_thr_gb = 8191; + params->acc_param.awb.config.rgbs_thr_b = IPU3_UAPI_AWB_RGBS_THR_B_INCL_SAT + | IPU3_UAPI_AWB_RGBS_THR_B_EN + | 8191; + + awbGrid_ = context.configuration.grid.bdsGrid; + + params->acc_param.awb.config.grid = context.configuration.grid.bdsGrid; + + /* + * Optical center is column start (respectively row start) of the + * region of interest minus its X center (respectively Y center). + * + * For the moment use BDS as a first approximation, but it should + * be calculated based on Shading (SHD) parameters. + */ + params->acc_param.bnr = imguCssBnrDefaults; + Size &bdsOutputSize = context.configuration.grid.bdsOutputSize; + params->acc_param.bnr.column_size = bdsOutputSize.width; + params->acc_param.bnr.opt_center.x_reset = awbGrid_.x_start - (bdsOutputSize.width / 2); + params->acc_param.bnr.opt_center.y_reset = awbGrid_.y_start - (bdsOutputSize.height / 2); + params->acc_param.bnr.opt_center_sqr.x_sqr_reset = params->acc_param.bnr.opt_center.x_reset + * params->acc_param.bnr.opt_center.x_reset; + params->acc_param.bnr.opt_center_sqr.y_sqr_reset = params->acc_param.bnr.opt_center.y_reset + * params->acc_param.bnr.opt_center.y_reset; /* * Green gains should not be touched and considered 1. * Default is 16, so do not change it at all. * 4096 is the value for a gain of 1.0 */ - params.acc_param.bnr.wb_gains.gr = 16; - params.acc_param.bnr.wb_gains.r = 4096 * asyncResults_.redGain; - params.acc_param.bnr.wb_gains.b = 4096 * asyncResults_.blueGain; - params.acc_param.bnr.wb_gains.gb = 16; + params->acc_param.bnr.wb_gains.gr = 16 * context.frameContext.awb.gains.green; + params->acc_param.bnr.wb_gains.r = 4096 * context.frameContext.awb.gains.red; + params->acc_param.bnr.wb_gains.b = 4096 * context.frameContext.awb.gains.blue; + params->acc_param.bnr.wb_gains.gb = 16 * context.frameContext.awb.gains.green; LOG(IPU3Awb, Debug) << "Color temperature estimated: " << asyncResults_.temperatureK; /* The CCM matrix may change when color temperature will be used */ - params.acc_param.ccm = imguCssCcmDefault; + params->acc_param.ccm = imguCssCcmDefault; + + params->use.acc_awb = 1; + params->use.acc_bnr = 1; + params->use.acc_ccm = 1; } } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3_awb.h b/src/ipa/ipu3/ipu3_awb.h index eeb2920b..12703489 100644 --- a/src/ipa/ipu3/ipu3_awb.h +++ b/src/ipa/ipu3/ipu3_awb.h @@ -29,9 +29,8 @@ public: IPU3Awb(); ~IPU3Awb(); - void initialise(ipu3_uapi_params ¶ms, const Size &bdsOutputSize, struct ipu3_uapi_grid_config &bdsGrid); - void calculateWBGains(const ipu3_uapi_stats_3a *stats); - void updateWbParameters(ipu3_uapi_params ¶ms); + void prepare(IPAContext &context, ipu3_uapi_params *params) override; + void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; struct Ipu3AwbCell { unsigned char greenRedAvg; @@ -72,6 +71,7 @@ public: }; private: + void calculateWBGains(const ipu3_uapi_stats_3a *stats); void generateZones(std::vector &zones); void generateAwbStats(const ipu3_uapi_stats_3a *stats); void clearAwbStats(); From patchwork Fri Aug 20 06:53:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13412 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3AB3FC3244 for ; Fri, 20 Aug 2021 06:53:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7B13468930; Fri, 20 Aug 2021 08:53:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PvmylGMO"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A5FDC68911 for ; Fri, 20 Aug 2021 08:53:22 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:e43b:edb2:932:6b0a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 54BB3150A; Fri, 20 Aug 2021 08:53:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1629442402; bh=BElcLdYoTow6ihnYzZIMZNuVR1sgLJP9zFru0V/qV7w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PvmylGMO8056LFYHj/vBJZSNZdxWuZ0hDUTxdZ7DXZOa63joaFM20GQt36qloAEG/ pa0v39nBCn7ReICLVGMlrzSoU1/BnpHp37m8a778CDThpCSF16IhtOdNeWlMhiKA/u 15lHjWkftR3fDVf94PdnBSThe4hVY6IAPtctYHDc= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Fri, 20 Aug 2021 08:53:15 +0200 Message-Id: <20210820065316.44343-9-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210820065316.44343-1-jeanmichel.hautbois@ideasonboard.com> References: <20210820065316.44343-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 8/9] ipa: ipu3: Move IPU3 awb into algorithms X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that the interface is properly used by the AWB class, move it into ipa::ipu3::algorithms and let the loops do the calls. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- .../ipu3/{ipu3_awb.cpp => algorithms/awb.cpp} | 28 +++++++++---------- src/ipa/ipu3/{ipu3_awb.h => algorithms/awb.h} | 20 ++++++------- src/ipa/ipu3/algorithms/meson.build | 1 + src/ipa/ipu3/ipu3.cpp | 10 ++----- src/ipa/ipu3/meson.build | 1 - 5 files changed, 27 insertions(+), 33 deletions(-) rename src/ipa/ipu3/{ipu3_awb.cpp => algorithms/awb.cpp} (94%) rename src/ipa/ipu3/{ipu3_awb.h => algorithms/awb.h} (84%) diff --git a/src/ipa/ipu3/ipu3_awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp similarity index 94% rename from src/ipa/ipu3/ipu3_awb.cpp rename to src/ipa/ipu3/algorithms/awb.cpp index 728162fe..e15f7406 100644 --- a/src/ipa/ipu3/ipu3_awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -2,9 +2,9 @@ /* * Copyright (C) 2021, Ideas On Board * - * ipu3_awb.cpp - AWB control algorithm + * awb.cpp - AWB control algorithm */ -#include "ipu3_awb.h" +#include "awb.h" #include #include @@ -13,7 +13,7 @@ namespace libcamera { -namespace ipa::ipu3 { +namespace ipa::ipu3::algorithms { LOG_DEFINE_CATEGORY(IPU3Awb) @@ -114,7 +114,7 @@ static const struct ipu3_uapi_ccm_mat_config imguCssCcmDefault = { 0, 0, 8191, 0 }; -IPU3Awb::IPU3Awb() +Awb::Awb() : Algorithm() { asyncResults_.blueGain = 1.0; @@ -125,7 +125,7 @@ IPU3Awb::IPU3Awb() zones_.reserve(kAwbStatsSizeX * kAwbStatsSizeY); } -IPU3Awb::~IPU3Awb() = default; +Awb::~Awb() = default; /** * The function estimates the correlated color temperature using @@ -141,7 +141,7 @@ IPU3Awb::~IPU3Awb() = default; * More detailed information can be found in: * https://en.wikipedia.org/wiki/Color_temperature#Approximation */ -uint32_t IPU3Awb::estimateCCT(double red, double green, double blue) +uint32_t Awb::estimateCCT(double red, double green, double blue) { /* Convert the RGB values to CIE tristimulus values (XYZ) */ double X = (-0.14282) * (red) + (1.54924) * (green) + (-0.95641) * (blue); @@ -158,7 +158,7 @@ uint32_t IPU3Awb::estimateCCT(double red, double green, double blue) } /* Generate an RGB vector with the average values for each region */ -void IPU3Awb::generateZones(std::vector &zones) +void Awb::generateZones(std::vector &zones) { for (unsigned int i = 0; i < kAwbStatsSizeX * kAwbStatsSizeY; i++) { RGB zone; @@ -175,7 +175,7 @@ void IPU3Awb::generateZones(std::vector &zones) } /* Translate the IPU3 statistics into the default statistics region array */ -void IPU3Awb::generateAwbStats(const ipu3_uapi_stats_3a *stats) +void Awb::generateAwbStats(const ipu3_uapi_stats_3a *stats) { uint32_t regionWidth = round(awbGrid_.width / static_cast(kAwbStatsSizeX)); uint32_t regionHeight = round(awbGrid_.height / static_cast(kAwbStatsSizeY)); @@ -207,7 +207,7 @@ void IPU3Awb::generateAwbStats(const ipu3_uapi_stats_3a *stats) } } -void IPU3Awb::clearAwbStats() +void Awb::clearAwbStats() { for (unsigned int i = 0; i < kAwbStatsSizeX * kAwbStatsSizeY; i++) { awbStats_[i].bSum = 0; @@ -218,7 +218,7 @@ void IPU3Awb::clearAwbStats() } } -void IPU3Awb::awbGreyWorld() +void Awb::awbGreyWorld() { LOG(IPU3Awb, Debug) << "Grey world AWB"; /* @@ -258,7 +258,7 @@ void IPU3Awb::awbGreyWorld() asyncResults_.blueGain = blueGain; } -void IPU3Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) +void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) { ASSERT(stats->stats_3a_status.awb_en); zones_.clear(); @@ -273,7 +273,7 @@ void IPU3Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) } } -void IPU3Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) +void Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { calculateWBGains(stats); @@ -287,7 +287,7 @@ void IPU3Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) context.frameContext.awb.gains.red = asyncResults_.redGain; } -void IPU3Awb::prepare(IPAContext &context, ipu3_uapi_params *params) +void Awb::prepare(IPAContext &context, ipu3_uapi_params *params) { params->acc_param.awb.config.rgbs_thr_gr = 8191; params->acc_param.awb.config.rgbs_thr_r = 8191; @@ -336,6 +336,6 @@ void IPU3Awb::prepare(IPAContext &context, ipu3_uapi_params *params) params->use.acc_ccm = 1; } -} /* namespace ipa::ipu3 */ +} /* namespace ipa::ipu3::algorithms */ } /* namespace libcamera */ diff --git a/src/ipa/ipu3/ipu3_awb.h b/src/ipa/ipu3/algorithms/awb.h similarity index 84% rename from src/ipa/ipu3/ipu3_awb.h rename to src/ipa/ipu3/algorithms/awb.h index 12703489..fac54e45 100644 --- a/src/ipa/ipu3/ipu3_awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -2,10 +2,10 @@ /* * Copyright (C) 2021, Ideas On Board * - * ipu3_awb.h - IPU3 AWB control algorithm + * awb.h - IPU3 AWB control algorithm */ -#ifndef __LIBCAMERA_IPU3_AWB_H__ -#define __LIBCAMERA_IPU3_AWB_H__ +#ifndef __LIBCAMERA_IPU3_ALGORITHMS_AWB_H__ +#define __LIBCAMERA_IPU3_ALGORITHMS_AWB_H__ #include @@ -13,21 +13,21 @@ #include -#include "algorithms/algorithm.h" +#include "algorithm.h" namespace libcamera { -namespace ipa::ipu3 { +namespace ipa::ipu3::algorithms { /* Region size for the statistics generation algorithm */ static constexpr uint32_t kAwbStatsSizeX = 16; static constexpr uint32_t kAwbStatsSizeY = 12; -class IPU3Awb : public Algorithm +class Awb : public Algorithm { public: - IPU3Awb(); - ~IPU3Awb(); + Awb(); + ~Awb(); void prepare(IPAContext &context, ipu3_uapi_params *params) override; void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; @@ -85,7 +85,7 @@ private: AwbStatus asyncResults_; }; -} /* namespace ipa::ipu3 */ +} /* namespace ipa::ipu3::algorithms */ } /* namespace libcamera*/ -#endif /* __LIBCAMERA_IPU3_AWB_H__ */ +#endif /* __LIBCAMERA_IPU3_ALGORITHMS_AWB_H__ */ diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build index 71eedfa4..fa004df4 100644 --- a/src/ipa/ipu3/algorithms/meson.build +++ b/src/ipa/ipu3/algorithms/meson.build @@ -2,5 +2,6 @@ ipu3_ipa_algorithms = files([ 'algorithm.cpp', + 'awb.cpp', 'tone_mapping.cpp', ]) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 2468e94a..5fc358fa 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -30,9 +30,9 @@ #include "libcamera/internal/mapped_framebuffer.h" #include "algorithms/algorithm.h" +#include "algorithms/awb.h" #include "algorithms/tone_mapping.h" #include "ipu3_agc.h" -#include "ipu3_awb.h" #include "libipa/camera_sensor_helper.h" /** @@ -187,8 +187,6 @@ private: uint32_t minGain_; uint32_t maxGain_; - /* Interface to the AWB algorithm */ - std::unique_ptr awbAlgo_; /* Interface to the AEC/AGC algorithm */ std::unique_ptr agcAlgo_; /* Interface to the Camera Helper */ @@ -270,6 +268,7 @@ int IPAIPU3::init(const IPASettings &settings, *ipaControls = ControlInfoMap(std::move(controls), controls::controls); /* Construct our Algorithms */ + algorithms_.push_back(std::make_unique()); algorithms_.push_back(std::make_unique()); return 0; @@ -387,7 +386,6 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) return ret; } - awbAlgo_ = std::make_unique(); agcAlgo_ = std::make_unique(); agcAlgo_->configure(context_, configInfo); @@ -466,8 +464,6 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) for (auto const &algo : algorithms_) algo->prepare(context_, params); - awbAlgo_->prepare(context_, params); - IPU3Action op; op.op = ActionParamFilled; @@ -490,8 +486,6 @@ void IPAIPU3::parseStatistics(unsigned int frame, exposure_ = context_.frameContext.agc.exposure; gain_ = camHelper_->gainCode(context_.frameContext.agc.gain); - awbAlgo_->process(context_, stats); - setControls(frame); /* \todo Use VBlank value calculated from each frame exposure. */ diff --git a/src/ipa/ipu3/meson.build b/src/ipa/ipu3/meson.build index fcb27d68..d1126947 100644 --- a/src/ipa/ipu3/meson.build +++ b/src/ipa/ipu3/meson.build @@ -7,7 +7,6 @@ ipa_name = 'ipa_ipu3' ipu3_ipa_sources = files([ 'ipu3.cpp', 'ipu3_agc.cpp', - 'ipu3_awb.cpp', ]) ipu3_ipa_sources += ipu3_ipa_algorithms From patchwork Fri Aug 20 06:53:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13413 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B972DC3245 for ; Fri, 20 Aug 2021 06:53:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D560F688A2; Fri, 20 Aug 2021 08:53:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="I4YOhEl2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DFDEC68919 for ; Fri, 20 Aug 2021 08:53:22 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:e43b:edb2:932:6b0a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8B69118EE; Fri, 20 Aug 2021 08:53:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1629442402; bh=9YS54Lj9B9mRrvwh0wl/yeWopRzugic0IomrM2dj9gA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=I4YOhEl2fiM5lACCx0Hr1LwRd6ILiPO1p9Uz2C6AF7lrwZvSkP2nEjgoX1LSqCLDe 9MHCl+UTiVIjT98t5RvVn5mx6RPsdmQgtDe7hTboCtw11kmP2OpaNAcA0g6w3Pyb4I ieDzwpBvLweU2l1NcujHzJLq8gOY+fwrBGeEfm3Q= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Fri, 20 Aug 2021 08:53:16 +0200 Message-Id: <20210820065316.44343-10-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210820065316.44343-1-jeanmichel.hautbois@ideasonboard.com> References: <20210820065316.44343-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 9/9] ipa: ipu3: Move IPU3 agc into algorithms X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that the interface is properly used by the AGC class, move it into ipa::ipu3::algorithms and let the loops do the calls. As we need to exchange the exposure_ and gain_ by passing them through the FrameContext, use the calculated values in setControls() function to ease the reading. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- .../ipu3/{ipu3_agc.cpp => algorithms/agc.cpp} | 18 ++++++++--------- src/ipa/ipu3/{ipu3_agc.h => algorithms/agc.h} | 20 +++++++++---------- src/ipa/ipu3/algorithms/meson.build | 1 + src/ipa/ipu3/ipu3.cpp | 20 ++++++++----------- src/ipa/ipu3/meson.build | 1 - 5 files changed, 28 insertions(+), 32 deletions(-) rename src/ipa/ipu3/{ipu3_agc.cpp => algorithms/agc.cpp} (94%) rename src/ipa/ipu3/{ipu3_agc.h => algorithms/agc.h} (73%) diff --git a/src/ipa/ipu3/ipu3_agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp similarity index 94% rename from src/ipa/ipu3/ipu3_agc.cpp rename to src/ipa/ipu3/algorithms/agc.cpp index 1c156b89..e4598b2e 100644 --- a/src/ipa/ipu3/ipu3_agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -5,7 +5,7 @@ * ipu3_agc.cpp - AGC/AEC control algorithm */ -#include "ipu3_agc.h" +#include "agc.h" #include #include @@ -21,7 +21,7 @@ namespace libcamera { using namespace std::literals::chrono_literals; -namespace ipa::ipu3 { +namespace ipa::ipu3::algorithms { LOG_DEFINE_CATEGORY(IPU3Agc) @@ -50,14 +50,14 @@ static constexpr double kEvGainTarget = 0.5; /* A cell is 8 bytes and contains averages for RGB values and saturation ratio */ static constexpr uint8_t kCellSize = 8; -IPU3Agc::IPU3Agc() +Agc::Agc() : frameCount_(0), lastFrame_(0), iqMean_(0.0), lineDuration_(0s), maxExposureTime_(0s), prevExposure_(0s), prevExposureNoDg_(0s), currentExposure_(0s), currentExposureNoDg_(0s) { } -int IPU3Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) +int Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) { aeGrid_ = context.configuration.grid.bdsGrid; @@ -68,7 +68,7 @@ int IPU3Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) return 0; } -void IPU3Agc::processBrightness(const ipu3_uapi_stats_3a *stats) +void Agc::processBrightness(const ipu3_uapi_stats_3a *stats) { const struct ipu3_uapi_grid_config statsAeGrid = stats->stats_4a_config.awb_config.grid; Rectangle aeRegion = { statsAeGrid.x_start, @@ -109,7 +109,7 @@ void IPU3Agc::processBrightness(const ipu3_uapi_stats_3a *stats) iqMean_ = Histogram(Span(hist)).interQuantileMean(0.98, 1.0); } -void IPU3Agc::filterExposure() +void Agc::filterExposure() { double speed = 0.2; if (prevExposure_ == 0s) { @@ -143,7 +143,7 @@ void IPU3Agc::filterExposure() LOG(IPU3Agc, Debug) << "After filtering, total_exposure " << prevExposure_; } -void IPU3Agc::lockExposureGain(uint32_t &exposure, double &gain) +void Agc::lockExposureGain(uint32_t &exposure, double &gain) { /* Algorithm initialization should wait for first valid frames */ /* \todo - have a number of frames given by DelayedControls ? @@ -186,7 +186,7 @@ void IPU3Agc::lockExposureGain(uint32_t &exposure, double &gain) lastFrame_ = frameCount_; } -void IPU3Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) +void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { uint32_t &exposure = context.frameContext.agc.exposure; double &gain = context.frameContext.agc.gain; @@ -195,6 +195,6 @@ void IPU3Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) frameCount_++; } -} /* namespace ipa::ipu3 */ +} /* namespace ipa::ipu3::algorithms */ } /* namespace libcamera */ diff --git a/src/ipa/ipu3/ipu3_agc.h b/src/ipa/ipu3/algorithms/agc.h similarity index 73% rename from src/ipa/ipu3/ipu3_agc.h rename to src/ipa/ipu3/algorithms/agc.h index 0e922664..1739d2fd 100644 --- a/src/ipa/ipu3/ipu3_agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -2,10 +2,10 @@ /* * Copyright (C) 2021, Ideas On Board * - * ipu3_agc.h - IPU3 AGC/AEC control algorithm + * agc.h - IPU3 AGC/AEC control algorithm */ -#ifndef __LIBCAMERA_IPU3_AGC_H__ -#define __LIBCAMERA_IPU3_AGC_H__ +#ifndef __LIBCAMERA_IPU3_ALGORITHMS_AGC_H__ +#define __LIBCAMERA_IPU3_ALGORITHMS_AGC_H__ #include @@ -13,21 +13,21 @@ #include -#include "algorithms/algorithm.h" +#include "algorithm.h" namespace libcamera { struct IPACameraSensorInfo; -namespace ipa::ipu3 { +namespace ipa::ipu3::algorithms { using utils::Duration; -class IPU3Agc : public Algorithm +class Agc : public Algorithm { public: - IPU3Agc(); - ~IPU3Agc() = default; + Agc(); + ~Agc() = default; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; @@ -53,8 +53,8 @@ private: Duration currentExposureNoDg_; }; -} /* namespace ipa::ipu3 */ +} /* namespace ipa::ipu3::algorithms */ } /* namespace libcamera */ -#endif /* __LIBCAMERA_IPU3_AGC_H__ */ +#endif /* __LIBCAMERA_IPU3_ALGORITHMS_AGC_H__ */ diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build index fa004df4..deae225b 100644 --- a/src/ipa/ipu3/algorithms/meson.build +++ b/src/ipa/ipu3/algorithms/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 ipu3_ipa_algorithms = files([ + 'agc.cpp', 'algorithm.cpp', 'awb.cpp', 'tone_mapping.cpp', diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 5fc358fa..0ed0a6f1 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -29,10 +29,10 @@ #include "libcamera/internal/mapped_framebuffer.h" +#include "algorithms/agc.h" #include "algorithms/algorithm.h" #include "algorithms/awb.h" #include "algorithms/tone_mapping.h" -#include "ipu3_agc.h" #include "libipa/camera_sensor_helper.h" /** @@ -187,8 +187,6 @@ private: uint32_t minGain_; uint32_t maxGain_; - /* Interface to the AEC/AGC algorithm */ - std::unique_ptr agcAlgo_; /* Interface to the Camera Helper */ std::unique_ptr camHelper_; @@ -268,6 +266,7 @@ int IPAIPU3::init(const IPASettings &settings, *ipaControls = ControlInfoMap(std::move(controls), controls::controls); /* Construct our Algorithms */ + algorithms_.push_back(std::make_unique()); algorithms_.push_back(std::make_unique()); algorithms_.push_back(std::make_unique()); @@ -386,9 +385,6 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) return ret; } - agcAlgo_ = std::make_unique(); - agcAlgo_->configure(context_, configInfo); - return 0; } @@ -476,15 +472,12 @@ void IPAIPU3::parseStatistics(unsigned int frame, { ControlList ctrls(controls::controls); - for (auto const &algo : algorithms_) - algo->process(context_, stats); - /* \todo These fields should not be written by the IPAIPU3 layer */ context_.frameContext.agc.gain = camHelper_->gain(gain_); context_.frameContext.agc.exposure = exposure_; - agcAlgo_->process(context_, stats); - exposure_ = context_.frameContext.agc.exposure; - gain_ = camHelper_->gainCode(context_.frameContext.agc.gain); + + for (auto const &algo : algorithms_) + algo->process(context_, stats); setControls(frame); @@ -505,6 +498,9 @@ void IPAIPU3::setControls(unsigned int frame) IPU3Action op; op.op = ActionSetSensorControls; + exposure_ = context_.frameContext.agc.exposure; + gain_ = camHelper_->gainCode(context_.frameContext.agc.gain); + ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); diff --git a/src/ipa/ipu3/meson.build b/src/ipa/ipu3/meson.build index d1126947..39320980 100644 --- a/src/ipa/ipu3/meson.build +++ b/src/ipa/ipu3/meson.build @@ -6,7 +6,6 @@ ipa_name = 'ipa_ipu3' ipu3_ipa_sources = files([ 'ipu3.cpp', - 'ipu3_agc.cpp', ]) ipu3_ipa_sources += ipu3_ipa_algorithms