From patchwork Tue Oct 26 11:23:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14334 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AE604BF415 for ; Tue, 26 Oct 2021 11:23:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 693146488B; Tue, 26 Oct 2021 13:23:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="aHL/b+QB"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BADF960124 for ; Tue, 26 Oct 2021 13:23:44 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4EE74DBF; Tue, 26 Oct 2021 13:23:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247424; bh=eJefIjiY20JPAde93DTlPSiZilgWKsnTbQvVNOYo32I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aHL/b+QBlJt/bbuWsJkeNt+5158DWlRD53VhpkBnEVg8+1qF09XKtVHPVY535zq9I MTPTxvXZ9lI9ooN9bPaSZT9LHMMQrz8iWEqCthzAxdlEuEqAyLU7kwcK+rKZn8JH5+ qiJaSjCjhVFF0Wymz0XnXD2ObHYBYrurBX5vl7hI= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:22 +0200 Message-Id: <20211026112340.110169-2-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 01/19] ipa: ipu3: Document IPAIPU3 class interface X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The IPU3 IPA is maturing to a modular and extensible system capable of handling specific algorithms for the processing blocks on the ImgU. Provide a top-level class documentation to provide an overview of the IPA, detailing what events are used and what algorithms are currently supported, as well as the limitations currently imposed. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Umang Jain --- src/ipa/ipu3/ipu3.cpp | 68 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 65d3fd2c..9396926c 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -179,6 +179,74 @@ using namespace std::literals::chrono_literals; namespace ipa::ipu3 { +/** + * \brief The IPU3 IPA implementation + * + * The IPU3 Pipeline defines an IPU3-specific interface for communication + * between the PipelineHandler and the IPA module. + * + * We extend the IPAIPU3Interface to implement our algorithms and handle events + * from the IPU3 PipelineHandler to satisfy requests from the application. + * + * At initialisation time, a CameraSensorHelper is instantiated to support + * camera-specific calculations, while the default controls are computed, and + * the algorithms are constructed and placed in an ordered list. + * + * The IPU3 ImgU operates with a grid layout to divide the overall frame into + * rectangular cells of pixels. When the IPA is configured, we determine the + * best grid for the statistics based on the pipeline handler Bayer Down Scaler + * output size. + * + * Two main events are then handled to operate the IPU3 ImgU by populating its + * parameter buffer, and adapting the settings of the sensor attached to the + * IPU3 CIO2 through sensor-specific V4L2 controls. + * + * When the event \a EventFillParams occurs we populate the ImgU parameter + * buffer with settings to configure the device in preparation for handling the + * frame queued in the Request. + * + * When the frame has completed processing, the ImgU will generate a statistics + * buffer which is given to the IPA as part of the \a EventStatReady event. At + * this event we run the algorithms to parse the statistics and cache any + * results for the next \a EventFillParams event. + * + * The individual algorithms are split into modular components that are called + * iteratively to allow them to process statistics from the ImgU in a defined + * order. + * + * The current implementation supports three core algorithms: + * - Automatic white balance (AWB) + * - Automatic gain and exposure control (AGC) + * - Black level correction (BLC) + * - Tone mapping (Gamma) + * + * AWB is implemented using a Greyworld algorithm, and calculates the red and + * blue gains to apply to generate a neutral grey frame overall. + * + * AGC is handled by calculating a histogram of the green channel to estimate an + * analogue gain and shutter time which will provide a well exposed frame. A + * low-pass IIR filter is used to smooth the changes to the sensor to reduce + * perceivable steps. + * + * The tone mapping algorithm provides a gamma correction table to improve the + * contrast of the scene. + * + * The black level compensation algorithm subtracts a hardcoded black level from + * all pixels. + * + * The IPU3 ImgU has further processing blocks to support image quality + * improvements through bayer and temporal noise reductions, however those are + * not supported in the current implementation, and will use default settings as + * provided by the kernel driver. + * + * Demosaicing is operating with the default parameters and could be further + * optimised to provide improved sharpening coefficients, checker artifact + * removal, and false color correction. + * + * Additional image enhancements can be made by providing lens and + * sensor-specific tuning to adapt for Black Level compensation (BLC), Lens + * shading correction (SHD) and Color correction (CCM). + */ class IPAIPU3 : public IPAIPU3Interface { public: From patchwork Tue Oct 26 11:23:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14335 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C99F9BF415 for ; Tue, 26 Oct 2021 11:23:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 73F3C64893; Tue, 26 Oct 2021 13:23:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="D91eJ6b4"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EDC1760128 for ; Tue, 26 Oct 2021 13:23:44 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 86C26E79; Tue, 26 Oct 2021 13:23:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247424; bh=9IlK6IfsT9mGDzONJKZdJo9o+UAvYTzEOBJO5Hamang=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=D91eJ6b4iqvfmHul03tL+M//lEfOPE8frtV5iyPb0cyy9O32logJW6a7I1GNTHZ1I aW6T6sRYHm3OSBam/9or8mM+UaDdyU8FMpX6Hfgah4mcwt6IitHwTpDbZ7ctHoE9NJ HFSB7ni6nqI96uiOp/2x51WviFj9wBT1EpUzX4xk= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:23 +0200 Message-Id: <20211026112340.110169-3-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 02/19] ipa: ipu3: Document IPAIPU3::configure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Further extend the documentation for the IPAIPU3::configure operation. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain --- src/ipa/ipu3/ipu3.cpp | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 9396926c..1f501821 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -506,6 +506,21 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) << (int)bdsGrid.height << " << " << (int)bdsGrid.block_height_log2 << ")"; } +/** + * \brief Configure the IPU3 IPA + * \param[in] configInfo The IPA configuration data, received from the pipeline + * handler + * \param[in] ipaControls The IPA controls to update + * + * Calculate the best grid for the statistics based on the Pipeline Handler BDS + * output, and parse the minimum and maximum exposure and analogue gain control + * values. + * + * \todo Document what the BDS is, ideally in a block diagram of the ImgU. + * + * All algorithm modules are called to allow them to prepare the + * \a IPASessionConfiguration structure for the \a IPAContext. + */ int IPAIPU3::configure(const IPAConfigInfo &configInfo, ControlInfoMap *ipaControls) { From patchwork Tue Oct 26 11:23:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14336 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 39CDFBF415 for ; Tue, 26 Oct 2021 11:23:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7114B64880; Tue, 26 Oct 2021 13:23:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="O+bCl7NJ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2777760123 for ; Tue, 26 Oct 2021 13:23:45 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BD8543F0; Tue, 26 Oct 2021 13:23:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247424; bh=P093pxkw4YOZQ8H/jSNiEbY6yjgf0lm7dU3cjuD+wao=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=O+bCl7NJL1qEas7Rrj1fbdvj3wAzUEZIMjpgKkOtbGtSNIlZZX2zb3b0bslP8aSmS KgeV+07SYKplotnaV5+iyImib619jectEdIGSRbRO4oqaF9ADhQGESkww/GzHS7eGO 5S9rLX7A8klfYzaqc6LkA7CRiCWB0xOJJgYSIfhY= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:24 +0200 Message-Id: <20211026112340.110169-4-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 03/19] ipa: ipu3: Document the IPAIPU3 class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Clarify the roles and interactions between the pipeline handler events and the algorithm calls by documenting all the remaining functions of the class. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain --- src/ipa/ipu3/ipu3.cpp | 87 +++++++++++++++++++++++++++++++++++++------ 1 file changed, 76 insertions(+), 11 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 1f501821..177b67b0 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -305,9 +305,9 @@ private: struct IPAContext context_; }; -/* - * Compute IPASessionConfiguration using the sensor information and the sensor - * v4l2 controls. +/** + * \brief Compute IPASessionConfiguration using the sensor information and the + * sensor V4L2 controls */ void IPAIPU3::updateSessionConfiguration(const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls) @@ -336,9 +336,9 @@ void IPAIPU3::updateSessionConfiguration(const IPACameraSensorInfo &sensorInfo, context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain); } -/* - * Compute camera controls using the sensor information and the sensor - * v4l2 controls. +/** + * \brief Compute camera controls using the sensor information and the sensor + * V4L2 controls * * Some of the camera controls are computed by the pipeline handler, some others * by the IPA module which is in charge of handling, for example, the exposure @@ -399,7 +399,7 @@ void IPAIPU3::updateControls(const IPACameraSensorInfo &sensorInfo, } /** - * Initialize the IPA module and its controls. + * \brief Initialize the IPA module and its controls * * This function receives the camera sensor information from the pipeline * handler, computes the limits of the controls it handles and returns @@ -430,8 +430,15 @@ int IPAIPU3::init(const IPASettings &settings, return 0; } +/** + * \brief Perform any processing required before the first frame + */ int IPAIPU3::start() { + /* + * Set the sensors V4L2 controls before the first frame to ensure that + * we have an expected and known configuration from the start. + */ setControls(0); return 0; @@ -512,7 +519,7 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) * handler * \param[in] ipaControls The IPA controls to update * - * Calculate the best grid for the statistics based on the Pipeline Handler BDS + * Calculate the best grid for the statistics based on the pipeline handler BDS * output, and parse the minimum and maximum exposure and analogue gain control * values. * @@ -585,6 +592,10 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo, return 0; } +/** + * \brief Map the parameters and stats buffers allocated in the pipeline handler + * \param[in] buffers The buffers to map + */ void IPAIPU3::mapBuffers(const std::vector &buffers) { for (const IPABuffer &buffer : buffers) { @@ -594,6 +605,10 @@ void IPAIPU3::mapBuffers(const std::vector &buffers) } } +/** + * \brief Unmap the parameters and stats buffers + * \param[in] ids The IDs of the buffers to unmap + */ void IPAIPU3::unmapBuffers(const std::vector &ids) { for (unsigned int id : ids) { @@ -605,6 +620,10 @@ void IPAIPU3::unmapBuffers(const std::vector &ids) } } +/** + * \brief Process an event generated by the pipeline handler + * \param[in] event The event sent from pipeline handler + */ void IPAIPU3::processEvent(const IPU3Event &event) { switch (event.op) { @@ -646,12 +665,28 @@ void IPAIPU3::processEvent(const IPU3Event &event) } } +/** + * \brief Process a control list for a request from the application + * \param[in] frame The number of the frame which will be processed next + * \param[in] controls The controls for the \a frame + * + * Parse the request to handle any IPA-managed controls that were set from the + * application such as manual sensor settings. + */ void IPAIPU3::processControls([[maybe_unused]] unsigned int frame, [[maybe_unused]] const ControlList &controls) { /* \todo Start processing for 'frame' based on 'controls'. */ } +/** + * \brief Fill the ImgU parameter buffer for the next frame + * \param[in] frame The number of the latest frame processed + * \param[out] params The parameter buffer to fill + * + * Algorithms are expected to fill the IPU3 parameter buffer for the next + * frame given their most recent processing of the ImgU statistics. + */ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) { /* @@ -674,6 +709,16 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) queueFrameAction.emit(frame, op); } +/** + * \brief Process the statistics generated by the ImgU + * \param[in] frame The number of the latest frame processed + * \param[in] frameTimestamp The current frame timestamp + * \param[in] stats The IPU3 statistics and ISP results + * + * Parse the most recently processed image statistics from the ImgU. The + * statistics are passed to each algorithm module to run their calculations and + * update their state accordingly. + */ void IPAIPU3::parseStatistics(unsigned int frame, [[maybe_unused]] int64_t frameTimestamp, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) @@ -697,6 +742,13 @@ void IPAIPU3::parseStatistics(unsigned int frame, queueFrameAction.emit(frame, op); } +/** + * \brief Handle sensor controls for a given \a frame number + * \param[in] frame The frame on which the sensor controls should be set + * + * Send the desired sensor control values to the pipeline handler to request + * that they are applied on the camera sensor. + */ void IPAIPU3::setControls(unsigned int frame) { IPU3Action op; @@ -715,10 +767,15 @@ void IPAIPU3::setControls(unsigned int frame) } /* namespace ipa::ipu3 */ -/* - * External IPA module interface +/** + * \brief External IPA module interface + * + * The IPAModuleInfo is required to match an IPA module construction against the + * intented pipeline handler with the module. The API and pipeline handler + * versions must match the corresponding IPA interface and pipeline handler. + * + * \sa struct IPAModuleInfo */ - extern "C" { const struct IPAModuleInfo ipaModuleInfo = { IPA_MODULE_API_VERSION, @@ -727,6 +784,14 @@ const struct IPAModuleInfo ipaModuleInfo = { "ipu3", }; +/** + * \brief Create an instance of the IPA interface + * + * This function is the entry point of the IPA module. It is called by the IPA + * manager to create an instance of the IPA interface for each camera. When + * matched against with a pipeline handler, the IPAManager will construct an IPA + * instance for each associated Camera. + */ IPAInterface *ipaCreate() { return new ipa::ipu3::IPAIPU3(); From patchwork Tue Oct 26 11:23:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14337 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D8ADDBF415 for ; Tue, 26 Oct 2021 11:23:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BA0B66489B; Tue, 26 Oct 2021 13:23:54 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="IKR+IYW9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 55A0C64877 for ; Tue, 26 Oct 2021 13:23:45 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 048CFDBF; Tue, 26 Oct 2021 13:23:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247425; bh=EnGGCYCxMtryh7kpm+1yH7bkYgiZ+JtzUkgl1uMuzF0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IKR+IYW9gjf5oioSBvZCcVUAPkBAXkWHmlVURXLGmWkXSZvDXye3nC4rplTMaT7b0 NVV9NVPpgzotuxNwb2yMXrd6VbBamvU/j2+jvkmJFVjU58H01gcCOBkthjYMWAWiDj 3JyCfzsJgF2ykeW25941jvKopH6JYNTVudywYm6g= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:25 +0200 Message-Id: <20211026112340.110169-5-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 04/19] ipa: ipu3: Explicitly use the statistics parameter X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The stats pointer is marked as [[maybe_unused]]. This is a leftover from a previous commit which was here to keep the compatibility while transitioning to the new iterative algorithms. Remove this attribute to make it explicit that stats are really used to feed the algorithms. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain --- src/ipa/ipu3/ipu3.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 177b67b0..bcaa34da 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -721,7 +721,7 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) */ void IPAIPU3::parseStatistics(unsigned int frame, [[maybe_unused]] int64_t frameTimestamp, - [[maybe_unused]] const ipu3_uapi_stats_3a *stats) + const ipu3_uapi_stats_3a *stats) { ControlList ctrls(controls::controls); From patchwork Tue Oct 26 11:23:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14338 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EEA36BF415 for ; Tue, 26 Oct 2021 11:23:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A307564887; Tue, 26 Oct 2021 13:23:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UDc+vgwv"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8C0BD6487A for ; Tue, 26 Oct 2021 13:23:45 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3B5E43F0; Tue, 26 Oct 2021 13:23:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247425; bh=aJzcaOw0CLDeD3nP0A2GpJAEE9VikFQMqZLFac3ZMAw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UDc+vgwvOIzCbQwG3P0W0YOxkdMbseHPOq9/VqQWnmZZqgsjDQdMZv9KSwF6UDs4G xATHEd2ZWziNjNDyWevBowfEwyMjiIQkJHMf3qJcDdqiMyYpJoJLwqxfAdwpTR8pQ5 X5rAi1DTPTMkML/kBQplv15+BCI4Hwg+2zmXtwdI= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:26 +0200 Message-Id: <20211026112340.110169-6-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 05/19] ipa: ipu3: awb: Add AWB class documentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AWB algorithm is based on the Grey world algorithm and uses the statistics generated by the ImgU for that. Explain how it uses those, and reference the original algorithm at the same time. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- v3: - move the diagram from Accumulator to AWB - add a word to say green gains are always set to 1 --- src/ipa/ipu3/algorithms/awb.cpp | 110 ++++++++++++++++++++------------ 1 file changed, 69 insertions(+), 41 deletions(-) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 91364a04..8d22aceb 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -11,6 +11,10 @@ #include +/** + * \file awb.h + */ + namespace libcamera { namespace ipa::ipu3::algorithms { @@ -42,47 +46,6 @@ static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 90 / 100; * \struct Accumulator * \brief RGB statistics for a given zone * - * - Cells are defined in Pixels - * - Zones are defined in Cells - * - * 80 cells - * /───────────── 1280 pixels ───────────\ - * 16 zones - * 16 - * ┌────┬────┬────┬────┬────┬─ ──────┬────┐ \ - * │Cell│ │ │ │ │ | │ │ │ - * 16 │ px │ │ │ │ │ | │ │ │ - * ├────┼────┼────┼────┼────┼─ ──────┼────┤ │ - * │ │ │ │ │ │ | │ │ - * │ │ │ │ │ │ | │ │ 7 - * │ ── │ ── │ ── │ ── │ ── │ ── ── ─┤ ── │ 1 2 4 - * │ │ │ │ │ │ | │ │ 2 0 5 - * - * │ │ │ │ │ │ | │ │ z p c - * ├────┼────┼────┼────┼────┼─ ──────┼────┤ o i e - * │ │ │ │ │ │ | │ │ n x l - * │ │ | │ │ e e l - * ├─── ───┼─ ──────┼────┤ s l s - * │ │ | │ │ s - * │ │ | │ │ - * ├─── Zone of Cells ───┼─ ──────┼────┤ │ - * │ (5 x 4) │ | │ │ │ - * │ │ | │ │ │ - * ├── ───┼─ ──────┼────┤ │ - * │ │ │ | │ │ │ - * │ │ │ │ │ │ | │ │ │ - * └────┴────┴────┴────┴────┴─ ──────┴────┘ / - * - * - * The algorithm works with a fixed number of zones \a kAwbStatsSizeX x - * \a kAwbStatsSizeY. For example, a frame of 1280x720 is divided into 80x45 - * cells of [16x16] pixels. In the case of \a kAwbStatsSizeX=16 and - * \a kAwbStatsSizeY=12 the zones are made of [5x4] cells. The cells are - * left-aligned and calculated by IPAIPU3::calculateBdsGrid(). - * - * Each statistics cell represents the average value of the pixels in that cell - * split by colour components. - * * The Accumulator structure stores the sum of the average of each cell in a * zone of the image, as well as the number of cells which were unsaturated and * therefore included in the average. @@ -152,6 +115,71 @@ static const struct ipu3_uapi_ccm_mat_config imguCssCcmDefault = { 0, 0, 8191, 0 }; +/** + * \class Awb + * \brief A Grey world white balance correction algorithm + * + * The Grey World algorithm assumes that the scene, in average, is neutral grey. + * Reference: Lam, Edmund & Fung, George. (2008). Automatic White Balancing in + * Digital Photography. 10.1201/9781420054538.ch10. + * + * The IPU3 generates statistics from the Bayer Down Scaler output into a grid + * defined in the ipu3_uapi_awb_config_s structure. + * + * - Cells are defined in Pixels + * - Zones are defined in Cells + * + * 80 cells + * /───────────── 1280 pixels ───────────\ + * 16 zones + * 16 + * ┌────┬────┬────┬────┬────┬─ ──────┬────┐ \ + * │Cell│ │ │ │ │ | │ │ │ + * 16 │ px │ │ │ │ │ | │ │ │ + * ├────┼────┼────┼────┼────┼─ ──────┼────┤ │ + * │ │ │ │ │ │ | │ │ + * │ │ │ │ │ │ | │ │ 7 + * │ ── │ ── │ ── │ ── │ ── │ ── ── ─┤ ── │ 1 2 4 + * │ │ │ │ │ │ | │ │ 2 0 5 + * + * │ │ │ │ │ │ | │ │ z p c + * ├────┼────┼────┼────┼────┼─ ──────┼────┤ o i e + * │ │ │ │ │ │ | │ │ n x l + * │ │ | │ │ e e l + * ├─── ───┼─ ──────┼────┤ s l s + * │ │ | │ │ s + * │ │ | │ │ + * ├─── Zone of Cells ───┼─ ──────┼────┤ │ + * │ (5 x 4) │ | │ │ │ + * │ │ | │ │ │ + * ├── ───┼─ ──────┼────┤ │ + * │ │ │ | │ │ │ + * │ │ │ │ │ │ | │ │ │ + * └────┴────┴────┴────┴────┴─ ──────┴────┘ / + * + * + * In each cell, the ImgU computes for each colour component the average of all + * unsaturated pixels (below a programmable threshold). It also provides the + * ratio of saturated pixels in the cell. + * + * The AWB algorithm operates on a coarser grid, made by grouping cells from the + * hardware grid into zones. The number of zones is fixed to \a kAwbStatsSizeX x + * \a kAwbStatsSizeY. For example, a frame of 1280x720 is divided into 80x45 + * cells of [16x16] pixels and 16x12 zones of [5x4] cells each + * (\a kAwbStatsSizeX=16 and \a kAwbStatsSizeY=12). If the number of cells isn't + * an exact multiple of the number of zones, the right-most and bottom-most + * cells are ignored. The grid configuration is computed by + * IPAIPU3::calculateBdsGrid(). + * + * Before calculating the gains, the algorithm aggregates the cell averages for + * each zone in generateAwbStats(). Cells that have a too high ratio of + * saturated pixels are ignored, and only zones that contain enough + * non-saturated cells are then used by the algorithm. + * + * The Grey World algorithm will then estimate the red and blue gains to apply, and + * store the results in the metadata. The green gain is always set to 1. + */ + Awb::Awb() : Algorithm() { From patchwork Tue Oct 26 11:23:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14339 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E29CEBF415 for ; Tue, 26 Oct 2021 11:23:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 02C8C64882; Tue, 26 Oct 2021 13:23:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pItCy+hC"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BA19460124 for ; Tue, 26 Oct 2021 13:23:45 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6CEA6E79; Tue, 26 Oct 2021 13:23:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247425; bh=/InymkwLqn0LdiSbSz7zL78uflekTRN5T3OwQu1tV1M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pItCy+hC2xB7JzNhBGFDxaQOXJ1gCCv5rrgtBV0+wWcRex1a/2jJpbqOL16RVHctV 6g6EWFxmjjh/Bi9tSOXYQ3m1acExxJaaQlHF1h8CBl5WsmI/enWi6Cf1F+9OGlKZK/ ocQCDeE6w6LNWO8kONAwstqosb3yktVAbDLUHjoQ= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:27 +0200 Message-Id: <20211026112340.110169-7-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 06/19] ipa: ipu3: awb: Reword accumulator documentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that we moved the diagram into the AWB class documentation, reword the accumulator documentation to make it clear it is not meant to be used only in AWB. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/awb.cpp | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 8d22aceb..41fd5fc4 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -46,12 +46,12 @@ static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 90 / 100; * \struct Accumulator * \brief RGB statistics for a given zone * - * The Accumulator structure stores the sum of the average of each cell in a - * zone of the image, as well as the number of cells which were unsaturated and - * therefore included in the average. + * Accumulate red, green and blue values for each non-saturated item over a + * zone. Items can for instance be pixels, but also the average of groups of + * pixels, depending on who uses the accumulator. * \todo move this description and structure into a common header * - * Cells which are saturated beyond the threshold defined in + * Zones which are saturated beyond the threshold defined in * ipu3_uapi_awb_config_s are not included in the average. * * \var Accumulator::counted From patchwork Tue Oct 26 11:23:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14340 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0B63EBF415 for ; Tue, 26 Oct 2021 11:23:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EAEF364883; Tue, 26 Oct 2021 13:23:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="htUHnLe4"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EBD0564879 for ; Tue, 26 Oct 2021 13:23:45 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A05BCDBF; Tue, 26 Oct 2021 13:23:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247425; bh=hoDs3INkuwpY3zdfR2YhQq2BGfKqfqp4QiDoEmPIVmY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=htUHnLe47T/UQEH8P1i+q0X5TyY2lUyZJrDXAvJ4va7YXoB9p1yJ2ULFtr03EqGlE C8FPjhxuEXpAheK6vl/hUIEul3zdUG8FmuSoySzp9ew+0ggj88wnKuVQgO6B1v8PzM CO2TzO3bNuR01+mYIlUduBlqUuFCXb0ojdToMCZM= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:28 +0200 Message-Id: <20211026112340.110169-8-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 07/19] ipa: ipu3: agc: Document AGC mean-based algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AGC class was not documented while developing. Extend that to reference the origins of the implementation, and improve the descriptions on how the algorithm operates internally. While at it, rename the functions which have bad names. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- v3: - rename processBrightness to measureBrightness - rename lockExposureGain to computeExposure - reword some comments --- src/ipa/ipu3/algorithms/agc.cpp | 80 ++++++++++++++++++++++++++++++--- src/ipa/ipu3/algorithms/agc.h | 6 +-- 2 files changed, 76 insertions(+), 10 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 6c151232..b89d1559 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -2,7 +2,7 @@ /* * Copyright (C) 2021, Ideas On Board * - * ipu3_agc.cpp - AGC/AEC control algorithm + * ipu3_agc.cpp - AGC/AEC mean-based control algorithm */ #include "agc.h" @@ -17,17 +17,37 @@ #include "libipa/histogram.h" +/** + * \file agc.h + */ + namespace libcamera { using namespace std::literals::chrono_literals; namespace ipa::ipu3::algorithms { +/** + * \class Agc + * \brief A mean-based auto-exposure algorithm + * + * This algorithm calculates a shutter time and an analogue gain so that the + * average value of the green channel of the brightest 2% of pixels approaches + * 0.5. The AWB gains are not used here, and all cells in the grid have the same + * weight, like an average-metering case. In this metering mode, the camera uses + * light information from the entire scene and creates an average for the final + * exposure setting, giving no weighting to any particular portion of the + * metered area. + * + * Reference: Battiato, Messina & Castorina. (2008). Exposure + * Correction for Imaging Devices: An Overview. 10.1201/9781420054538.ch12. + */ + LOG_DEFINE_CATEGORY(IPU3Agc) /* Number of frames to wait before calculating stats on minimum exposure */ static constexpr uint32_t kInitialFrameMinAECount = 4; -/* Number of frames to wait between new gain/exposure estimations */ +/* Number of frames to wait between new gain/shutter time estimations */ static constexpr uint32_t kFrameSkipCount = 6; /* Limits for analogue gain values */ @@ -36,6 +56,8 @@ static constexpr double kMaxAnalogueGain = 8.0; /* Histogram constants */ static constexpr uint32_t knumHistogramBins = 256; + +/* Target value to reach for the top 2% of the histogram */ static constexpr double kEvGainTarget = 0.5; Agc::Agc() @@ -45,10 +67,18 @@ Agc::Agc() { } +/** + * \brief Configure the AGC given a configInfo + * \param[in] context The shared IPA context + * \param[in] configInfo The IPA configuration data + * + * \return 0 + */ int Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) { stride_ = context.configuration.grid.stride; + /* \todo use the IPAContext to provide the limits */ lineDuration_ = configInfo.sensorInfo.lineLength * 1.0s / configInfo.sensorInfo.pixelRate; @@ -70,9 +100,15 @@ int Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) return 0; } -void Agc::processBrightness(const ipu3_uapi_stats_3a *stats, +/** + * \brief Estimate the mean value of the top 2% of the histogram + * \param[in] stats The statistics computed by the ImgU + * \param[in] grid The grid used to store the statistics in the IPU3 + */ +void Agc::measureBrightness(const ipu3_uapi_stats_3a *stats, const ipu3_uapi_grid_config &grid) { + /* Initialise the histogram array */ uint32_t hist[knumHistogramBins] = { 0 }; for (unsigned int cellY = 0; cellY < grid.height; cellY++) { @@ -87,6 +123,11 @@ void Agc::processBrightness(const ipu3_uapi_stats_3a *stats, if (cell->sat_ratio == 0) { uint8_t gr = cell->Gr_avg; uint8_t gb = cell->Gb_avg; + /* + * Store the average green value to estimate the + * brightness. Even the overexposed pixels are + * taken into account. + */ hist[(gr + gb) / 2]++; } } @@ -96,6 +137,9 @@ void Agc::processBrightness(const ipu3_uapi_stats_3a *stats, iqMean_ = Histogram(Span(hist)).interQuantileMean(0.98, 1.0); } +/** + * \brief Apply a filter on the exposure value to limit the speed of changes + */ void Agc::filterExposure() { double speed = 0.2; @@ -106,7 +150,7 @@ void Agc::filterExposure() /* * If we are close to the desired result, go faster to avoid making * multiple micro-adjustments. - * \ todo: Make this customisable? + * \todo Make this customisable? */ if (filteredExposure_ < 1.2 * currentExposure_ && filteredExposure_ > 0.8 * currentExposure_) @@ -119,7 +163,12 @@ void Agc::filterExposure() LOG(IPU3Agc, Debug) << "After filtering, total_exposure " << filteredExposure_; } -void Agc::lockExposureGain(uint32_t &exposure, double &analogueGain) +/** + * \brief Estimate the new exposure and gain values + * \param[inout] exposure The exposure value reference as a number of lines + * \param[inout] gain The gain reference to be updated + */ +void Agc::computeExposure(uint32_t &exposure, double &analogueGain) { /* Algorithm initialization should wait for first valid frames */ /* \todo - have a number of frames given by DelayedControls ? @@ -136,19 +185,26 @@ void Agc::lockExposureGain(uint32_t &exposure, double &analogueGain) return; } + /* Estimate the gain needed to have the proportion wanted */ double evGain = kEvGainTarget * knumHistogramBins / iqMean_; /* extracted from Rpi::Agc::computeTargetExposure */ + /* Calculate the shutter time in seconds */ utils::Duration currentShutter = exposure * lineDuration_; LOG(IPU3Agc, Debug) << "Actual total exposure " << currentShutter * analogueGain << " Shutter speed " << currentShutter << " Gain " << analogueGain << " Needed ev gain " << evGain; + /* + * Calculate the current exposure value for the scene as the latest + * exposure value applied multiplied by the new estimated gain. + */ currentExposure_ = prevExposureValue_ * evGain; utils::Duration minShutterSpeed = minExposureLines_ * lineDuration_; utils::Duration maxShutterSpeed = maxExposureLines_ * lineDuration_; + /* Clamp the exposure value to the min and max authorized */ utils::Duration maxTotalExposure = maxShutterSpeed * maxAnalogueGain_; currentExposure_ = std::min(currentExposure_, maxTotalExposure); LOG(IPU3Agc, Debug) << "Target total exposure " << currentExposure_ @@ -157,6 +213,7 @@ void Agc::lockExposureGain(uint32_t &exposure, double &analogueGain) /* \todo: estimate if we need to desaturate */ filterExposure(); + /* Divide the exposure value as new exposure and gain values */ utils::Duration exposureValue = filteredExposure_; utils::Duration shutterTime = minShutterSpeed; @@ -186,12 +243,21 @@ void Agc::lockExposureGain(uint32_t &exposure, double &analogueGain) prevExposureValue_ = shutterTime * analogueGain; } +/** + * \brief Process IPU3 statistics, and run AGC operations + * \param[in] context The shared IPA context + * \param[in] stats The IPU3 statistics and ISP results + * + * Identify the current image brightness, and use that to estimate the optimal + * new exposure and gain for the scene. + */ void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { + /* Get the latest exposure and gain applied */ uint32_t &exposure = context.frameContext.agc.exposure; double &analogueGain = context.frameContext.agc.gain; - processBrightness(stats, context.configuration.grid.bdsGrid); - lockExposureGain(exposure, analogueGain); + measureBrightness(stats, context.configuration.grid.bdsGrid); + computeExposure(exposure, analogueGain); frameCount_++; } diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 1840205b..69e0b831 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -2,7 +2,7 @@ /* * Copyright (C) 2021, Ideas On Board * - * agc.h - IPU3 AGC/AEC control algorithm + * agc.h - IPU3 AGC/AEC mean-based control algorithm */ #ifndef __LIBCAMERA_IPU3_ALGORITHMS_AGC_H__ #define __LIBCAMERA_IPU3_ALGORITHMS_AGC_H__ @@ -31,10 +31,10 @@ public: void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; private: - void processBrightness(const ipu3_uapi_stats_3a *stats, + void measureBrightness(const ipu3_uapi_stats_3a *stats, const ipu3_uapi_grid_config &grid); void filterExposure(); - void lockExposureGain(uint32_t &exposure, double &gain); + void computeExposure(uint32_t &exposure, double &gain); uint64_t frameCount_; uint64_t lastFrame_; From patchwork Tue Oct 26 11:23:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14341 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E82ADBF415 for ; Tue, 26 Oct 2021 11:23:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4B3A264895; Tue, 26 Oct 2021 13:23:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="P/LuKH3D"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3A72764884 for ; Tue, 26 Oct 2021 13:23:46 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D611C3F0; Tue, 26 Oct 2021 13:23:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247425; bh=HPMVdf68yWVnoSXsmho/S+LzQSwSso72VFKAkZXohZM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P/LuKH3DuDLM6wPyi7e6/eQRxy4tbe0DReCP2VjDdLelwZo1/G6WAZLB8OW9/TNZw 5AzZWtIBp0exAG+1ffb+tY7+q7FhXabssHNE35UiInLX+ms4vaRMFFgxXo9pfM30oJ x67TN2dj7zKNe/QS9KZ0LbkCaArkuGS+Lea6fHrI= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:29 +0200 Message-Id: <20211026112340.110169-9-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 08/19] ipa: ipu3: tonemapping: Generate the LUT only on gamma change X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The tone mapping algorithm calculates the gamma curve for every frame, regardless of whether the gamma value has changed or not. This issue is exasperated as we currently hardcode the gamma to a single value. Optimise the implementation to only recalculate the look up table when the gamma setting is changed, and store the gamma setting of the LUT curve as part of the IPA context. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/tone_mapping.cpp | 5 +++++ src/ipa/ipu3/ipa_context.h | 1 + src/ipa/ipu3/ipu3.cpp | 3 +++ 3 files changed, 9 insertions(+) diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 3af96261..40337f9d 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -43,6 +43,9 @@ void ToneMapping::process([[maybe_unused]] IPAContext &context, */ gamma_ = 1.1; + if (context.frameContext.toneMapping.gamma == gamma_) + return; + struct ipu3_uapi_gamma_corr_lut &lut = context.frameContext.toneMapping.gammaCorrection; @@ -53,6 +56,8 @@ void ToneMapping::process([[maybe_unused]] IPAContext &context, /* The output value is expressed on 13 bits. */ lut.lut[i] = gamma * 8191; } + + context.frameContext.toneMapping.gamma = gamma_; } } /* namespace ipa::ipu3::algorithms */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 847c03fe..e14bb561 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -48,6 +48,7 @@ struct IPAFrameContext { } awb; struct { + double gamma; struct ipu3_uapi_gamma_corr_lut gammaCorrection; } toneMapping; }; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index bcaa34da..4c0fff00 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -151,6 +151,9 @@ * \struct IPAFrameContext::toneMapping * \brief Context for ToneMapping and Gamma control * + * \var IPAFrameContext::toneMapping::gamma + * \brief Gamma value for the LUT + * * \var IPAFrameContext::toneMapping::gammaCorrection * \brief Per-pixel tone mapping implemented as a LUT * From patchwork Tue Oct 26 11:23:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14342 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1C712BF415 for ; Tue, 26 Oct 2021 11:24:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CB38C6487D; Tue, 26 Oct 2021 13:24:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Mgq4zJMg"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 69DE464887 for ; Tue, 26 Oct 2021 13:23:46 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 16056DBF; Tue, 26 Oct 2021 13:23:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247426; bh=+hSQOQoqtykLWcHx2GBqz1QrBOoOZUzi8gu547oRCY0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Mgq4zJMg8EPfQwycCtyvk7ITRyWYHGm2jSPh//0DvCg/MxRuUf/y5dwlJb7SOnwOn POlkHZfMYZjL2i5u6J9pwxlqglmG95L32yJ1TfYp/NLS4Yypbq5J9ynpfO7F4b2qQd UUr5Ol91AUBK6scKEG2ZYWWuEm2d/5pxd1n47izE= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:30 +0200 Message-Id: <20211026112340.110169-10-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 09/19] ipa: ipu3: tonemapping: Implement configure call X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Let the algorithm perform its initial configuration. Implement configure() to set a default gamma value and let process do the updates needed. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/tone_mapping.cpp | 16 ++++++++++++++++ src/ipa/ipu3/algorithms/tone_mapping.h | 1 + 2 files changed, 17 insertions(+) diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 40337f9d..fcd22d0b 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -19,6 +19,22 @@ ToneMapping::ToneMapping() { } +/** + * \brief Configure the tone mapping given a configInfo + * \param[in] context The shared IPA context + * \param[in] configInfo The IPA configuration data + * + * \return 0 + */ +int ToneMapping::configure(IPAContext &context, + [[maybe_unused]] const IPAConfigInfo &configInfo) +{ + /* Initialise tone mapping gamma value. */ + context.frameContext.toneMapping.gamma = 0.0; + + return 0; +} + void ToneMapping::prepare([[maybe_unused]] IPAContext &context, ipu3_uapi_params *params) { diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index 1dae4f9a..46dd6171 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -18,6 +18,7 @@ class ToneMapping : public Algorithm public: ToneMapping(); + int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, ipu3_uapi_params *params) override; void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; From patchwork Tue Oct 26 11:23:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14343 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8D4FDC324E for ; Tue, 26 Oct 2021 11:24:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4CCF264888; Tue, 26 Oct 2021 13:24:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="O7AjUqEH"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9417E60128 for ; Tue, 26 Oct 2021 13:23:46 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 488F6E79; Tue, 26 Oct 2021 13:23:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247426; bh=7BGr7hiTSDbpG7tzJHV7R/26QoDeqAxF3NbLgzjonZY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=O7AjUqEHY947udl17UlyexPab+v2Le+hgIuSxe8hdKPkBUWD4BJ+F5kYmBr1zf0by xVSllxItPkbV3YJze5Yyk9fXhNxCbNZD6c65fZYx9MwUxrkx9+aefcm3FCDfp+SPeW VoZKXuqzVm86c/pJkF3yZigoHG1NiHkzOvW1HlYA= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:31 +0200 Message-Id: <20211026112340.110169-11-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 10/19] ipa: ipu3: tonemapping: Add the documentation for ToneMapping X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The tone mapping algorithm is currently undocumented. Provide an introduction and overview to the implementation as the class definition and document how the algorithm operates in the process and prepare methods. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- v3: - rename gamma control as tone mapping --- src/ipa/ipu3/algorithms/tone_mapping.cpp | 30 +++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index fcd22d0b..2040eda5 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -10,10 +10,22 @@ #include #include +/** + * \file tone_mapping.h + */ + namespace libcamera { namespace ipa::ipu3::algorithms { +/** + * \class ToneMapping + * \brief A class to handle tone mapping based on gamma + * + * This algorithm improves the image dynamic using a look-up table which is + * generated based on a gamma parameter. + */ + ToneMapping::ToneMapping() : gamma_(1.0) { @@ -35,6 +47,14 @@ int ToneMapping::configure(IPAContext &context, return 0; } +/** + * \brief Fill in the parameter structure, and enable gamma control + * \param context The shared IPA context + * \param params The IPU3 parameters + * + * Populate the IPU3 parameter structure with our tone mapping look up table and + * enable the gamma control module in the processing blocks. + */ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, ipu3_uapi_params *params) { @@ -49,7 +69,15 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, params->acc_param.gamma.gc_ctrl.enable = 1; } -void ToneMapping::process([[maybe_unused]] IPAContext &context, +/** + * \brief Calculate the tone mapping look up table + * \param context The shared IPA context + * \param stats The IPU3 statistics and ISP results + * + * The tone mapping look up table is generated as an inverse power curve from + * our gamma setting. + */ +void ToneMapping::process(IPAContext &context, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) { /* From patchwork Tue Oct 26 11:23:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14344 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 450BDBF415 for ; Tue, 26 Oct 2021 11:24:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DAAB964881; Tue, 26 Oct 2021 13:24:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="p8uFEhVl"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CBFA864888 for ; Tue, 26 Oct 2021 13:23:46 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7ACE43F0; Tue, 26 Oct 2021 13:23:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247426; bh=SSi1w6AEKxHmvUi5enuxvrAbDk64vPlPqXrbgRiUJsc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=p8uFEhVlA8dp+xCY64BaZHxnFrCaePt3y5Oas9AAncP5ZNQGtvhM1ZA76rcHhy51u k6NJaXGzsmgWcwdXWZrPm5z+iKXiA2N+RPh3GwyWODS6B4smkmIAxgFyrNsPGsoFCN tq/CvWyFkmKIa/1okHc+EZSibHR1xyb7ORjadXGA= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:32 +0200 Message-Id: <20211026112340.110169-12-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 11/19] Documentation: IPU3 IPA Design guide X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The IPU3 IPA implements the basic 3A using the ImgU ISP. Provide an overview document to describe its operations, and provide a block diagram to help visualise how the components are put together to assist any new developers exploring the code. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Jean-Michel Hautbois --- v2: - /accelerator cluster/processing block/ (and refactor) - /Pipeline/pipeline/ - /Camera Sensor/camera sensor/ - /CPU accessible/CPU-accessible/ - Remove updated control parameters from IPASessionConfiguration - Expand pre-frame preparation to match post-frame with the event descriptions. - Add Sensor Controls brief - Move to src/ipa/ipu3/ - Lower indentation of the block diagrams (keep under 80chars) - reference mapBuffers() call for passing buffers in - reference unmapBuffers() after stop() --- src/ipa/ipu3/ipu3-ipa-design-guide.rst | 155 +++++++++++++++++++++++++ 1 file changed, 155 insertions(+) create mode 100644 src/ipa/ipu3/ipu3-ipa-design-guide.rst diff --git a/src/ipa/ipu3/ipu3-ipa-design-guide.rst b/src/ipa/ipu3/ipu3-ipa-design-guide.rst new file mode 100644 index 00000000..89c71108 --- /dev/null +++ b/src/ipa/ipu3/ipu3-ipa-design-guide.rst @@ -0,0 +1,155 @@ +IPU3 IPA Architecture Design and Overview +========================================= + +The IPU3 IPA is built as a modular and extensible framework with an +upper layer to manage the interactions with the pipeline handler, and +the image processing algorithms split to compartmentalise the processing +required for each processing block, making use of the fixed-function +accelerators provided by the ImgU ISP. + +The core IPU3 class is responsible for initialisation and construction +of the algorithm components, processing controls set by the requests +from applications, and managing events from the pipeline handler. + +:: + + ┌───────────────────────────────────────────┐ + │ IPU3 Pipeline Handler │ + │ ┌────────┐ ┌────────┐ ┌────────┐ │ + │ │ │ │ │ │ │ │ + │ │ Sensor ├───►│ CIO2 ├───►│ ImgU ├──► + │ │ │ │ │ │ │ │ + │ └────────┘ └────────┘ └─▲────┬─┘ │ P: Parameter Buffer + │ │P │ │ S: Statistics Buffer + │ │ │S │ + └─┬───┬───┬──────┬────┬────┬────┬─┴────▼─┬──┘ 1: init() + │ │ │ │ ▲ │ ▲ │ ▲ │ ▲ │ 2: configure() + │1 │2 │3 │4│ │4│ │4│ │4│ │5 3: mapBuffers(), start() + ▼ ▼ ▼ ▼ │ ▼ │ ▼ │ ▼ │ ▼ 4: processEvent() + ┌──────────────────┴────┴────┴────┴─────────┐ 5: stop(), unmapBuffers() + │ IPU3 IPA │ + │ ┌───────────────────────┐ │ + │ ┌───────────┐ │ Algorithms │ │ + │ │IPAContext │ │ ┌─────────┐ │ │ + │ │ ┌───────┐ │ │ │ ... │ │ │ + │ │ │ │ │ │ ┌─┴───────┐ │ │ │ + │ │ │ SC │ │ │ │ Tonemap ├─┘ │ │ + │ │ │ │ ◄───► ┌─┴───────┐ │ │ │ + │ │ ├───────┤ │ │ │ AWB ├─┘ │ │ + │ │ │ │ │ │ ┌─┴───────┐ │ │ │ + │ │ │ FC │ │ │ │ AGC ├─┘ │ │ + │ │ │ │ │ │ │ │ │ │ + │ │ └───────┘ │ │ └─────────┘ │ │ + │ └───────────┘ └───────────────────────┘ │ + └───────────────────────────────────────────┘ + SC: IPASessionConfiguration + FC: IPAFrameContext(s) + +The IPA instance is constructed and initialised at the point a Camera is +created by the IPU3 pipeline handler. The initialisation call provides +details about which camera sensor is being used, and the controls that +it has available, along with their default values and ranges. + +Buffers +~~~~~~~ + +The IPA will have Parameter and Statistics buffers shared with it from +the IPU3 Pipeline handler. These buffers will be passed to the IPA using +the ``mapBuffers()`` call before the ``start()`` operation occurs. + +The IPA will map the buffers into CPU-accessible memory, associated with +a buffer ID, and further events for sending or receiving parameter and +statistics buffers will reference the ID to avoid expensive memory +mapping operations, or the passing of file handles during streaming. + +After the ``stop()`` operation occurs, these buffers will be unmapped +when requested by the pipeline handler using the ``unmapBuffers()`` call +and no further access to the buffers is permitted. + +Context +~~~~~~~ + +Algorithm calls will always have the ``IPAContext`` available to them. +This context comprises of two parts: + +- IPA Session Configuration +- IPA Frame Context + +The session configuration structure ``IPASessionConfiguration`` +represents constant parameters determined before streaming commenced +during ``configure()``. + +The IPA Frame Context provides the storage for algorithms for a single +frame operation. + +The ``IPAFrameContext`` structure may be extended to an array, list, or +queue to store historical state for each frame, allowing algorithms to +obtain and reference results of calculations which are deeply pipelined. +This may only be done if an algorithm needs to know the context that was +applied at the frame the statistics were produced for, rather than the +previous or current frame. + +Presently there is a single ``IPAFrameContext`` without historical data, +and the context is maintained and updated through successive processing +operations. + +Operating +~~~~~~~~~ + +There are three main interactions with the algorithms for the IPU3 IPA +to operate when running: + +- configure() +- processEvent(``EventFillParams``) +- processEvent(``EventStatReady``) + +The configuration phase allows the pipeline-handler to inform the IPA of +the current stream configurations, which is then passed into each +algorithm to provide an opportunity to identify and track state of the +hardware, such as image size or ImgU pipeline configurations. + +Pre-frame preparation +~~~~~~~~~~~~~~~~~~~~~ + +When configured, the IPA is notified by the pipeline handler of the +Camera ``start()`` event, after which incoming requests will be queued +for processing, requiring a parameter buffer (``ipu3_uapi_params``) to +be populated for the ImgU. This is given to the IPA through the +``EventFillParams`` event, and then passed directly to each algorithm +through the ``prepare()`` call allowing the ISP configuration to be +updated for the needs of each component that the algorithm is +responsible for. + +The algorithm should set the use flag (``ipu3_uapi_flags``) for any +structure that it modifies, and it should take care to ensure that any +structure set by a use flag is fully initialised to suitable values. + +The parameter buffer is returned to the pipeline handler through the +``ActionParamFilled`` event, and from there queued to the ImgU along +with a raw frame captured with the CIO2. + +Post-frame completion +~~~~~~~~~~~~~~~~~~~~~ + +When the capture of an image is completed, and successfully processed +through the ImgU, the generated statistics buffer +(``ipu3_uapi_stats_3a``) is given to the IPA through the +``EventStatReady`` event. This provides the IPA with an opportunity to +examine the results of the ISP and run the calculations required by each +algorithm on the new data. The algorithms may require context from the +operations of other algorithms, for example, the AWB might choose to use +a scene brightness determined by the AGC. It is important that the +algorithms are ordered to ensure that required results are determined +before they are needed. + +The ordering of the algorithm processing is determined by their +placement in the ``IPU3::algorithms_`` ordered list. + +Sensor Controls +~~~~~~~~~~~~~~~ + +The AutoExposure and AutoGain (AGC) algorithm differs slightly from the +others as it requires operating directly on the sensor, as opposed to +through the ImgU ISP. To support this, there is a dedicated action +`ActionSetSensorControls` to allow the IPA to request controls to be set +on the camera sensor through the pipeline handler. From patchwork Tue Oct 26 11:23:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14345 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DFCFBBF415 for ; Tue, 26 Oct 2021 11:24:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8A70A6488F; Tue, 26 Oct 2021 13:24:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="hia2JUPe"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1C9AE64889 for ; Tue, 26 Oct 2021 13:23:47 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B2607DBF; Tue, 26 Oct 2021 13:23:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247426; bh=xwRU0h5/NStPknJnHGdHrQAdpvpu99PZFX6QcYJ1Og4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hia2JUPe1+tMXa0twjOn8D0TkA/PIZskzZBeIL90/cwFHBOVJPIqRk8dgFs0jDDf9 GQmG2+ic4HK9Cmo/HW8YJf+KTiuit77VJxzgkPXImWQ9qnl3RfaT9jvaejulwhe+6+ oGnXws8qSor1UDNlXwcGwOqn0kWTn6GKx+trPmGk= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:33 +0200 Message-Id: <20211026112340.110169-13-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 12/19] ipa: ipu3: ipa_context: Fix file reference X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The ipa_context.h entry incorrectly referenced its file name. Fix it. Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipa_context.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index e14bb561..1e46c61a 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -2,7 +2,7 @@ /* * Copyright (C) 2021, Google Inc. * - * ipu3_ipa_context.h - IPU3 IPA Context + * ipa_context.h - IPU3 IPA Context * */ #ifndef __LIBCAMERA_IPU3_IPA_CONTEXT_H__ From patchwork Tue Oct 26 11:23:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14346 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9590FBF415 for ; Tue, 26 Oct 2021 11:24:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 385326487D; Tue, 26 Oct 2021 13:24:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="qA5uBeik"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4032F6488A for ; Tue, 26 Oct 2021 13:23:47 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E986BE79; Tue, 26 Oct 2021 13:23:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247427; bh=IQKsI33iYTHwdFhO8mOJt/5WGBgIcWqXqH60HMbZak0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qA5uBeikGtRkoTwdDwRiH4iplyyKKbWVM3zt3Ey+bfoEUg7FvtaCT6m/y5bolf6l5 cmrHshPa9Yn9vua2jSDIECtf0SNKQMecxY1dUvnEgV6FWMrVBR8vtpot3hj59g39JK Zpboox9As0cx4wGcYY/e5JatpJGFrrHh19TpkXG0= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:34 +0200 Message-Id: <20211026112340.110169-14-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 13/19] ipa: ipu3: Fix the IPU3 AWB doxygen references X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The AWB AwbStatus structure is contained within the Awb class. Fix the Doxygen reference so that it can be found. Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/awb.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 41fd5fc4..a4114659 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -71,7 +71,7 @@ static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 90 / 100; */ /** - * \struct AwbStatus + * \struct Awb::AwbStatus * \brief AWB parameters calculated * * The AwbStatus structure is intended to store the AWB From patchwork Tue Oct 26 11:23:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14347 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 22460BF415 for ; Tue, 26 Oct 2021 11:24:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D3B2464888; Tue, 26 Oct 2021 13:24:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ll7O91IA"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A5A86488C for ; Tue, 26 Oct 2021 13:23:47 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 293EC3F0; Tue, 26 Oct 2021 13:23:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247427; bh=/r1saF9a2jSCqdIjQ+uHgXanZ5OgtGl4EHjeSuJTvWk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ll7O91IAfbPs16wEe0e1oipRUAhXao4uBvnRvB6P7u3RdeSHspV9KZyAAo5ssWN/d vakN8c8EWRePbfZKlhUye0AdvuHTH0elbyf+knBq90zr6OrpejFxZyPTYooqI+Wdxe RsvLdDx6LSjBlgj2DwBPxz7NUTT1i5qaQ1XhPOrM= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:35 +0200 Message-Id: <20211026112340.110169-15-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 14/19] ipa: ipu3: algorithms: awb: Privatize internal structures X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The struct RGB and struct AwbStatus are used only by the internal implementation of the AWB algorithm module. Move them into the private class declaration. Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/awb.h | 1 + 1 file changed, 1 insertion(+) diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index 0c81e39e..b90782c1 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -42,6 +42,7 @@ public: void prepare(IPAContext &context, ipu3_uapi_params *params) override; void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; +private: /* \todo Make these structs available to all the ISPs ? */ struct RGB { RGB(double _R = 0, double _G = 0, double _B = 0) From patchwork Tue Oct 26 11:23:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14348 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C3DB1BF415 for ; Tue, 26 Oct 2021 11:24:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7080D64891; Tue, 26 Oct 2021 13:24:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="hkyJvQL9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AD7E16488E for ; Tue, 26 Oct 2021 13:23:47 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5BE99DBF; Tue, 26 Oct 2021 13:23:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247427; bh=pst3JPVUNmp/asmcNXBtC/aHyzbUwCtJNPA5V/ec35w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hkyJvQL9HPM79+KaMR+jz9g95hMQ80HFk3mQmd5fainkWq1YBUdYh6csAQCQ+SEpR gB0HsXXJob+dQtpW27BgvZn1vnN2hvzIX6iU/qj4b7VUEhe8wthb6Im+D8GdYXvoBs VIU9t792yEZo1wgpJDiAOItdi13jher5NqtJFrL8= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:36 +0200 Message-Id: <20211026112340.110169-16-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 15/19] ipa: ipu3: Isolate ipa_context documentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The IPU3 IPA core is growing with additional documentation. The ipa_context documentation is stored here, but it pushes the IPU3 documentation and implementation further from the head of the file. Furthermore, the ipa_context documentation is outside of the ipa::ipu3 namespace and isn't identified correctly by Doxygen. Move the ipa_context to its own compilation object even though there isn't any code, but to maintain consistency with our documentation model. Correctly re-introduce the documentation into the libcamera::ipa::ipu3 namespace during the move. Signed-off-by: Kieran Bingham Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jean-Michel Hautbois --- v4: - put the \file block before the namespace --- src/ipa/ipu3/ipa_context.cpp | 136 +++++++++++++++++++++++++++++++++++ src/ipa/ipu3/ipu3.cpp | 124 -------------------------------- src/ipa/ipu3/meson.build | 1 + 3 files changed, 137 insertions(+), 124 deletions(-) create mode 100644 src/ipa/ipu3/ipa_context.cpp diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp new file mode 100644 index 00000000..94a5f7f6 --- /dev/null +++ b/src/ipa/ipu3/ipa_context.cpp @@ -0,0 +1,136 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Google Inc. + * + * ipa_context.cpp - IPU3 IPA Context + */ + +#include "ipa_context.h" + +/** + * \file ipa_context.h + * \brief Context and state information shared between the algorithms + */ + +namespace libcamera::ipa::ipu3 { + +/** + * \struct IPASessionConfiguration + * \brief Session configuration for the IPA module + * + * The session configuration contains all IPA configuration parameters that + * remain constant during the capture session, from IPA module start to stop. + * It is typically set during the configure() operation of the IPA module, but + * may also be updated in the start() operation. + */ + +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + * + * The frame context stores data specific to a single frame processed by the + * IPA. Each frame processed by the IPA has a context associated with it, + * accessible through the IPAContext structure. + * + * \todo Detail how to access contexts for a particular frame + * + * Each of the fields in the frame context belongs to either a specific + * algorithm, or to the top-level IPA module. A field may be read by any + * algorithm, but should only be written by its owner. + */ + +/** + * \struct IPAContext + * \brief Global IPA context data shared between all algorithms + * + * \var IPAContext::configuration + * \brief The IPA session configuration, immutable during the session + * + * \var IPAContext::frameContext + * \brief The frame context for the frame being processed + * + * \todo While the frame context is supposed to be per-frame, this + * single frame context stores data related to both the current frame + * and the previous frames, with fields being updated as the algorithms + * are run. This needs to be turned into real per-frame data storage. + */ + +/** + * \struct IPASessionConfiguration::grid + * \brief Grid configuration of the IPA + * + * \var IPASessionConfiguration::grid::bdsGrid + * \brief Bayer Down Scaler grid plane config used by the kernel + * + * \var IPASessionConfiguration::grid::bdsOutputSize + * \brief BDS output size configured by the pipeline handler + * + * \var IPASessionConfiguration::grid::stride + * \brief Number of cells on one line including the ImgU padding + */ + +/** + * \struct IPASessionConfiguration::agc + * \brief AGC parameters configuration of the IPA + * + * \var IPASessionConfiguration::agc::minShutterSpeed + * \brief Minimum shutter speed supported with the configured sensor + * + * \var IPASessionConfiguration::grid::maxShutterSpeed + * \brief Maximum shutter speed supported with the configured sensor + * + * \var IPASessionConfiguration::grid::minAnalogueGain + * \brief Minimum analogue gain supported with the configured sensor + * + * \var IPASessionConfiguration::grid::maxAnalogueGain + * \brief Maximum analogue gain supported with the configured sensor + */ + +/** + * \struct IPAFrameContext::agc + * \brief Context for the Automatic Gain Control algorithm + * + * The exposure and gain determined are expected to be applied to the sensor + * at the earliest opportunity. + * + * \var IPAFrameContext::agc::exposure + * \brief Exposure time expressed as a number of lines + * + * \var IPAFrameContext::agc::gain + * \brief Analogue gain multiplier + * + * The gain should be adapted to the sensor specific gain code before applying. + */ + +/** + * \struct IPAFrameContext::awb + * \brief Context for the Automatic White Balance algorithm + * + * \struct IPAFrameContext::awb::gains + * \brief White balance gains + * + * \var IPAFrameContext::awb::gains::red + * \brief White balance gain for R channel + * + * \var IPAFrameContext::awb::gains::green + * \brief White balance gain for G channel + * + * \var IPAFrameContext::awb::gains::blue + * \brief White balance gain for B channel + */ + +/** + * \struct IPAFrameContext::toneMapping + * \brief Context for ToneMapping and Gamma control + * + * \var IPAFrameContext::toneMapping::gamma + * \brief Gamma value for the LUT + * + * \var IPAFrameContext::toneMapping::gammaCorrection + * \brief Per-pixel tone mapping implemented as a LUT + * + * The LUT structure is defined by the IPU3 kernel interface. See + * struct ipu3_uapi_gamma_corr_lut for further details. + */ + +} /* namespace libcamera::ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 4c0fff00..a10fdd4a 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -37,130 +37,6 @@ #include "algorithms/tone_mapping.h" #include "libipa/camera_sensor_helper.h" -/** - * \file ipa_context.h - * \brief Context and state information shared between the algorithms - */ - -/** - * \struct IPASessionConfiguration - * \brief Session configuration for the IPA module - * - * The session configuration contains all IPA configuration parameters that - * remain constant during the capture session, from IPA module start to stop. - * It is typically set during the configure() operation of the IPA module, but - * may also be updated in the start() operation. - */ - -/** - * \struct IPAFrameContext - * \brief Per-frame context for algorithms - * - * The frame context stores data specific to a single frame processed by the - * IPA. Each frame processed by the IPA has a context associated with it, - * accessible through the IPAContext structure. - * - * \todo Detail how to access contexts for a particular frame - * - * Each of the fields in the frame context belongs to either a specific - * algorithm, or to the top-level IPA module. A field may be read by any - * algorithm, but should only be written by its owner. - */ - -/** - * \struct IPAContext - * \brief Global IPA context data shared between all algorithms - * - * \var IPAContext::configuration - * \brief The IPA session configuration, immutable during the session - * - * \var IPAContext::frameContext - * \brief The frame context for the frame being processed - * - * \todo While the frame context is supposed to be per-frame, this - * single frame context stores data related to both the current frame - * and the previous frames, with fields being updated as the algorithms - * are run. This needs to be turned into real per-frame data storage. - */ - -/** - * \struct IPASessionConfiguration::grid - * \brief Grid configuration of the IPA - * - * \var IPASessionConfiguration::grid::bdsGrid - * \brief Bayer Down Scaler grid plane config used by the kernel - * - * \var IPASessionConfiguration::grid::bdsOutputSize - * \brief BDS output size configured by the pipeline handler - * - * \var IPASessionConfiguration::grid::stride - * \brief Number of cells on one line including the ImgU padding - */ - -/** - * \struct IPASessionConfiguration::agc - * \brief AGC parameters configuration of the IPA - * - * \var IPASessionConfiguration::agc::minShutterSpeed - * \brief Minimum shutter speed supported with the configured sensor - * - * \var IPASessionConfiguration::grid::maxShutterSpeed - * \brief Maximum shutter speed supported with the configured sensor - * - * \var IPASessionConfiguration::grid::minAnalogueGain - * \brief Minimum analogue gain supported with the configured sensor - * - * \var IPASessionConfiguration::grid::maxAnalogueGain - * \brief Maximum analogue gain supported with the configured sensor - */ - -/** - * \struct IPAFrameContext::agc - * \brief Context for the Automatic Gain Control algorithm - * - * The exposure and gain determined are expected to be applied to the sensor - * at the earliest opportunity. - * - * \var IPAFrameContext::agc::exposure - * \brief Exposure time expressed as a number of lines - * - * \var IPAFrameContext::agc::gain - * \brief Analogue gain multiplier - * - * The gain should be adapted to the sensor specific gain code before applying. - */ - -/** - * \struct IPAFrameContext::awb - * \brief Context for the Automatic White Balance algorithm - * - * \struct IPAFrameContext::awb::gains - * \brief White balance gains - * - * \var IPAFrameContext::awb::gains::red - * \brief White balance gain for R channel - * - * \var IPAFrameContext::awb::gains::green - * \brief White balance gain for G channel - * - * \var IPAFrameContext::awb::gains::blue - * \brief White balance gain for B channel - */ - -/** - * \struct IPAFrameContext::toneMapping - * \brief Context for ToneMapping and Gamma control - * - * \var IPAFrameContext::toneMapping::gamma - * \brief Gamma value for the LUT - * - * \var IPAFrameContext::toneMapping::gammaCorrection - * \brief Per-pixel tone mapping implemented as a LUT - * - * The LUT structure is defined by the IPU3 kernel interface. See - * struct ipu3_uapi_gamma_corr_lut for further details. - */ - /* Minimum grid width, expressed as a number of cells */ static constexpr uint32_t kMinGridWidth = 16; /* Maximum grid width, expressed as a number of cells */ diff --git a/src/ipa/ipu3/meson.build b/src/ipa/ipu3/meson.build index 39320980..3194111a 100644 --- a/src/ipa/ipu3/meson.build +++ b/src/ipa/ipu3/meson.build @@ -5,6 +5,7 @@ subdir('algorithms') ipa_name = 'ipa_ipu3' ipu3_ipa_sources = files([ + 'ipa_context.cpp', 'ipu3.cpp', ]) From patchwork Tue Oct 26 11:23:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14349 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 56D49BF415 for ; Tue, 26 Oct 2021 11:24:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0196E64893; Tue, 26 Oct 2021 13:24:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="g/AoHgD8"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DF2C86487D for ; Tue, 26 Oct 2021 13:23:47 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8F578F7B; Tue, 26 Oct 2021 13:23:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247427; bh=abOFK9TYbtC/GuVD0qHYqrfD54ddE0NvQw4SazfiJ1c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g/AoHgD89rOE02C7tUHe2BteDTFs+9XFxV4aDyfCa7hmSiaoU9a3e2yb/CQbzCxPV 7P+LcwXClIwZ1d1v7M+/vg9OKFCI5yd9Miec8bmzqiXJyioDCK2DoRYHsDbWtetQTv eLHr+ARQO5J6nwYmiP624HiOdtM26KVUDU7Fr0zU= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:37 +0200 Message-Id: <20211026112340.110169-17-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 16/19] ipa: ipu3: Fix badly documented context variables X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Doxygen warns us because the structures are referenced as \struct while they should be \var. Fix it. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipa_context.cpp | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 94a5f7f6..e73ec248 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -56,7 +56,7 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPASessionConfiguration::grid + * \var IPASessionConfiguration::grid * \brief Grid configuration of the IPA * * \var IPASessionConfiguration::grid::bdsGrid @@ -70,7 +70,7 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPASessionConfiguration::agc + * \var IPASessionConfiguration::agc * \brief AGC parameters configuration of the IPA * * \var IPASessionConfiguration::agc::minShutterSpeed @@ -87,7 +87,7 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPAFrameContext::agc + * \var IPAFrameContext::agc * \brief Context for the Automatic Gain Control algorithm * * The exposure and gain determined are expected to be applied to the sensor @@ -103,7 +103,7 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPAFrameContext::awb + * \var IPAFrameContext::awb * \brief Context for the Automatic White Balance algorithm * * \struct IPAFrameContext::awb::gains @@ -120,7 +120,7 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPAFrameContext::toneMapping + * \var IPAFrameContext::toneMapping * \brief Context for ToneMapping and Gamma control * * \var IPAFrameContext::toneMapping::gamma From patchwork Tue Oct 26 11:23:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14350 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EAB62BF415 for ; Tue, 26 Oct 2021 11:24:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A0CE164885; Tue, 26 Oct 2021 13:24:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="aIykpiR9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 237416488F for ; Tue, 26 Oct 2021 13:23:48 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C30823F0; Tue, 26 Oct 2021 13:23:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247427; bh=onf/vbo0mNGwxIM/y0dDxOjuPvjc9jWislN3kdAmCq0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aIykpiR9nOck/Or62x2dSYmln4f9gfnf516GZypX3d8qOjghYOIoaXxMFfYYSPQ6l jSNPTygaktQsAzrzRu93cb4aWZ+WNnTn8JUknl4ZzI3w/4ZJA/kG+c8d81GNmfqg0f tp0ciCI1R3XEIbQN8PeFmMUJCip71hZPEVrzwSbg= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:38 +0200 Message-Id: <20211026112340.110169-18-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 17/19] ipa: ipu3: Implement an empty stop() function X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" While the stop() function does not currently perform any action, it forms part of the IPA interface and is a public function in the class. Promote it to a full (but basic) function implementation and begin the documentation accordingly so that there is an appropriate stub to perform stop operations if they come up. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Umang Jain --- src/ipa/ipu3/ipu3.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index a10fdd4a..5c51607d 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -135,7 +135,7 @@ public: ControlInfoMap *ipaControls) override; int start() override; - void stop() override {} + void stop() override; int configure(const IPAConfigInfo &configInfo, ControlInfoMap *ipaControls) override; @@ -323,6 +323,13 @@ int IPAIPU3::start() return 0; } +/** + * \brief Ensure that all processing has completed + */ +void IPAIPU3::stop() +{ +} + /** * \brief Calculate a grid for the AWB statistics * From patchwork Tue Oct 26 11:23:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14351 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 89257BF415 for ; Tue, 26 Oct 2021 11:24:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3BFE264895; Tue, 26 Oct 2021 13:24:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="eYbXWme/"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 56E7B64881 for ; Tue, 26 Oct 2021 13:23:48 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 06CE4DBF; Tue, 26 Oct 2021 13:23:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247428; bh=13MtrzHbEjSHQOoACCv7B8pGEKDpZqX4vSCK6vKtW9k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eYbXWme/Lmv+Y+ORb6zeSY/4zG6cH/WrdSzwhG3DdXiTJYDak/OjRphGBuMQbVHRi 8ufS811uY1uyNSNfbPZN0t3Vf/1TkmyJj+ybhrY/dbKvPR6kiDAM5cUC1JM6s0PMg4 I8D7lcg5XfEN95yIHF22KNdtWbAyMxeed8F5+pX4= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:39 +0200 Message-Id: <20211026112340.110169-19-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 18/19] Documentation: Include IPU3 in Doxygen build X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The IPU3 is documented using Doxygen within the code. Include the IPU3 IPA as part of the doxygen build sources. This will ideally be split to its own 'section' of the doxygen output. Signed-off-by: Kieran Bingham Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart --- Documentation/Doxyfile.in | 1 + Documentation/meson.build | 1 + 2 files changed, 2 insertions(+) diff --git a/Documentation/Doxyfile.in b/Documentation/Doxyfile.in index 6e627192..3d20e3ca 100644 --- a/Documentation/Doxyfile.in +++ b/Documentation/Doxyfile.in @@ -789,6 +789,7 @@ WARN_LOGFILE = # Note: If this tag is empty the current directory is searched. INPUT = "@TOP_SRCDIR@/include/libcamera" \ + "@TOP_SRCDIR@/src/ipa/ipu3" \ "@TOP_SRCDIR@/src/ipa/libipa" \ "@TOP_SRCDIR@/src/libcamera" \ "@TOP_BUILDDIR@/include/libcamera" \ diff --git a/Documentation/meson.build b/Documentation/meson.build index c4bd3c7f..df36a808 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -22,6 +22,7 @@ if doxygen.found() and dot.found() custom_target('doxygen', input : [ doxyfile, + ipu3_ipa_sources, libcamera_base_headers, libcamera_base_sources, libcamera_internal_headers, From patchwork Tue Oct 26 11:23:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14352 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1C600BF415 for ; Tue, 26 Oct 2021 11:24:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D125764890; Tue, 26 Oct 2021 13:24:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="TSsrYG0B"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8B33164891 for ; Tue, 26 Oct 2021 13:23:48 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3A4E3E79; Tue, 26 Oct 2021 13:23:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635247428; bh=ycdYbtWvxoVukojwfHHcE1kt6ctnxNoh1EvsFmhSnjc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TSsrYG0BMvCadwH6xDrLhi+1KFUqDd/2zpATJplkhpsDKqVOMp2fbk51PzH3qZ97v Mrc80p2sK/f/XrYYXKhCYX0FzgV3KFBF/jUziLVV36Qxfi7FmzFt5WxTlDHS55RqM0 iUv7kpMzv1z9IKFDGRtySt5gamn9y/DKVHkWI0UY= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 13:23:40 +0200 Message-Id: <20211026112340.110169-20-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026112340.110169-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 19/19] ipa: ipu3: ipa_context: Fix doxygen warnings X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The IPAFrameContext uses unnamed structures to group items. Doxygen doesn't seem to support this properly, documentation isn't properly generated and warnings are output during compilation. Suppress the warning with a workaround that still results in incorrect generated documentation until Doxygen gets fixed. Signed-off-by: Kieran Bingham [JMH: Fix doxygen variable usage] Signed-off-by: Jean-Michel Hautbois Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipa_context.cpp | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index e73ec248..2355a9c7 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -59,13 +59,13 @@ namespace libcamera::ipa::ipu3 { * \var IPASessionConfiguration::grid * \brief Grid configuration of the IPA * - * \var IPASessionConfiguration::grid::bdsGrid + * \var IPASessionConfiguration::grid.bdsGrid * \brief Bayer Down Scaler grid plane config used by the kernel * - * \var IPASessionConfiguration::grid::bdsOutputSize + * \var IPASessionConfiguration::grid.bdsOutputSize * \brief BDS output size configured by the pipeline handler * - * \var IPASessionConfiguration::grid::stride + * \var IPASessionConfiguration::grid.stride * \brief Number of cells on one line including the ImgU padding */ @@ -73,16 +73,16 @@ namespace libcamera::ipa::ipu3 { * \var IPASessionConfiguration::agc * \brief AGC parameters configuration of the IPA * - * \var IPASessionConfiguration::agc::minShutterSpeed + * \var IPASessionConfiguration::agc.minShutterSpeed * \brief Minimum shutter speed supported with the configured sensor * - * \var IPASessionConfiguration::grid::maxShutterSpeed + * \var IPASessionConfiguration::grid.maxShutterSpeed * \brief Maximum shutter speed supported with the configured sensor * - * \var IPASessionConfiguration::grid::minAnalogueGain + * \var IPASessionConfiguration::grid.minAnalogueGain * \brief Minimum analogue gain supported with the configured sensor * - * \var IPASessionConfiguration::grid::maxAnalogueGain + * \var IPASessionConfiguration::grid.maxAnalogueGain * \brief Maximum analogue gain supported with the configured sensor */ @@ -93,10 +93,10 @@ namespace libcamera::ipa::ipu3 { * The exposure and gain determined are expected to be applied to the sensor * at the earliest opportunity. * - * \var IPAFrameContext::agc::exposure + * \var IPAFrameContext::agc.exposure * \brief Exposure time expressed as a number of lines * - * \var IPAFrameContext::agc::gain + * \var IPAFrameContext::agc.gain * \brief Analogue gain multiplier * * The gain should be adapted to the sensor specific gain code before applying. @@ -106,16 +106,16 @@ namespace libcamera::ipa::ipu3 { * \var IPAFrameContext::awb * \brief Context for the Automatic White Balance algorithm * - * \struct IPAFrameContext::awb::gains + * \struct IPAFrameContext::awb.gains * \brief White balance gains * - * \var IPAFrameContext::awb::gains::red + * \var IPAFrameContext::awb.gains.red * \brief White balance gain for R channel * - * \var IPAFrameContext::awb::gains::green + * \var IPAFrameContext::awb.gains.green * \brief White balance gain for G channel * - * \var IPAFrameContext::awb::gains::blue + * \var IPAFrameContext::awb.gains.blue * \brief White balance gain for B channel */ @@ -123,10 +123,10 @@ namespace libcamera::ipa::ipu3 { * \var IPAFrameContext::toneMapping * \brief Context for ToneMapping and Gamma control * - * \var IPAFrameContext::toneMapping::gamma + * \var IPAFrameContext::toneMapping.gamma * \brief Gamma value for the LUT * - * \var IPAFrameContext::toneMapping::gammaCorrection + * \var IPAFrameContext::toneMapping.gammaCorrection * \brief Per-pixel tone mapping implemented as a LUT * * The LUT structure is defined by the IPU3 kernel interface. See