From patchwork Tue Oct 26 09:55:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14312 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3C181BDB1C for ; Tue, 26 Oct 2021 09:55:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8FFD064887; Tue, 26 Oct 2021 11:55:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="N5lBsImn"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BF27464874 for ; Tue, 26 Oct 2021 11:55:44 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4D1EFDBF; Tue, 26 Oct 2021 11:55:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242144; bh=eJefIjiY20JPAde93DTlPSiZilgWKsnTbQvVNOYo32I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=N5lBsImnkApn4sFwvf5iWV5g98A0OEqO8KrGjfHrdzKioLODvPvlgUn0vSeiZ8syu alZdmeFuPZ5KhHTTFQd7pkCdiVueiXUYl8qmNFomotKRHu9CXwTBp8/AgpwZ737fiO A2PlfHEui0RfAYQf9APMizLjBoxXqw5xyJN398KA= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:16 +0200 Message-Id: <20211026095534.90348-2-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 01/19] ipa: ipu3: Document IPAIPU3 class interface X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The IPU3 IPA is maturing to a modular and extensible system capable of handling specific algorithms for the processing blocks on the ImgU. Provide a top-level class documentation to provide an overview of the IPA, detailing what events are used and what algorithms are currently supported, as well as the limitations currently imposed. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Umang Jain --- src/ipa/ipu3/ipu3.cpp | 68 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 65d3fd2c..9396926c 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -179,6 +179,74 @@ using namespace std::literals::chrono_literals; namespace ipa::ipu3 { +/** + * \brief The IPU3 IPA implementation + * + * The IPU3 Pipeline defines an IPU3-specific interface for communication + * between the PipelineHandler and the IPA module. + * + * We extend the IPAIPU3Interface to implement our algorithms and handle events + * from the IPU3 PipelineHandler to satisfy requests from the application. + * + * At initialisation time, a CameraSensorHelper is instantiated to support + * camera-specific calculations, while the default controls are computed, and + * the algorithms are constructed and placed in an ordered list. + * + * The IPU3 ImgU operates with a grid layout to divide the overall frame into + * rectangular cells of pixels. When the IPA is configured, we determine the + * best grid for the statistics based on the pipeline handler Bayer Down Scaler + * output size. + * + * Two main events are then handled to operate the IPU3 ImgU by populating its + * parameter buffer, and adapting the settings of the sensor attached to the + * IPU3 CIO2 through sensor-specific V4L2 controls. + * + * When the event \a EventFillParams occurs we populate the ImgU parameter + * buffer with settings to configure the device in preparation for handling the + * frame queued in the Request. + * + * When the frame has completed processing, the ImgU will generate a statistics + * buffer which is given to the IPA as part of the \a EventStatReady event. At + * this event we run the algorithms to parse the statistics and cache any + * results for the next \a EventFillParams event. + * + * The individual algorithms are split into modular components that are called + * iteratively to allow them to process statistics from the ImgU in a defined + * order. + * + * The current implementation supports three core algorithms: + * - Automatic white balance (AWB) + * - Automatic gain and exposure control (AGC) + * - Black level correction (BLC) + * - Tone mapping (Gamma) + * + * AWB is implemented using a Greyworld algorithm, and calculates the red and + * blue gains to apply to generate a neutral grey frame overall. + * + * AGC is handled by calculating a histogram of the green channel to estimate an + * analogue gain and shutter time which will provide a well exposed frame. A + * low-pass IIR filter is used to smooth the changes to the sensor to reduce + * perceivable steps. + * + * The tone mapping algorithm provides a gamma correction table to improve the + * contrast of the scene. + * + * The black level compensation algorithm subtracts a hardcoded black level from + * all pixels. + * + * The IPU3 ImgU has further processing blocks to support image quality + * improvements through bayer and temporal noise reductions, however those are + * not supported in the current implementation, and will use default settings as + * provided by the kernel driver. + * + * Demosaicing is operating with the default parameters and could be further + * optimised to provide improved sharpening coefficients, checker artifact + * removal, and false color correction. + * + * Additional image enhancements can be made by providing lens and + * sensor-specific tuning to adapt for Black Level compensation (BLC), Lens + * shading correction (SHD) and Color correction (CCM). + */ class IPAIPU3 : public IPAIPU3Interface { public: From patchwork Tue Oct 26 09:55:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14313 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 16318BDB1C for ; Tue, 26 Oct 2021 09:55:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 47F3564892; Tue, 26 Oct 2021 11:55:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PjxCabu7"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DDD1064875 for ; Tue, 26 Oct 2021 11:55:44 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 84A81E79; Tue, 26 Oct 2021 11:55:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242144; bh=9IlK6IfsT9mGDzONJKZdJo9o+UAvYTzEOBJO5Hamang=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PjxCabu7MRE4GrCN/Q0g0BNl+nIREMHU2RmLYkJKVzoXeq5MjHHLJUrvYljBMe5ho 2pE9bd+FpFoa5z2/HJPHhj+Wk+jVHEaKTKOATZuMcehy8zZlzX/6UlCfe2IWBig6Dn PIDR252h7IG7bZA74VXBbUTE3Chx43ELqNQseflk= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:17 +0200 Message-Id: <20211026095534.90348-3-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 02/19] ipa: ipu3: Document IPAIPU3::configure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Further extend the documentation for the IPAIPU3::configure operation. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain --- src/ipa/ipu3/ipu3.cpp | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 9396926c..1f501821 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -506,6 +506,21 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) << (int)bdsGrid.height << " << " << (int)bdsGrid.block_height_log2 << ")"; } +/** + * \brief Configure the IPU3 IPA + * \param[in] configInfo The IPA configuration data, received from the pipeline + * handler + * \param[in] ipaControls The IPA controls to update + * + * Calculate the best grid for the statistics based on the Pipeline Handler BDS + * output, and parse the minimum and maximum exposure and analogue gain control + * values. + * + * \todo Document what the BDS is, ideally in a block diagram of the ImgU. + * + * All algorithm modules are called to allow them to prepare the + * \a IPASessionConfiguration structure for the \a IPAContext. + */ int IPAIPU3::configure(const IPAConfigInfo &configInfo, ControlInfoMap *ipaControls) { From patchwork Tue Oct 26 09:55:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14314 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 95F14BDB1C for ; Tue, 26 Oct 2021 09:55:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 53BAC6487C; Tue, 26 Oct 2021 11:55:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="EiuzhUyq"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0C67764878 for ; Tue, 26 Oct 2021 11:55:45 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BB35D3F0; Tue, 26 Oct 2021 11:55:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242144; bh=DxJ5Yo4Uw6l/zUlCftN9xwt2kYJ68GXJScJGBUT4QEY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EiuzhUyqyluBpyaLO4vTLLxQ1HEXLUawJEcKjiVdmXm+xW6OBlxHMZZ3KhIgPSfr5 3ejo1RCz4ZobgEFh7grTsl2xdLhql5tAGNNgQB507icMSk7lz2BNW/Fxx1LCGo9JMf DHeOKnABi7q21M2upjcyYzNBESaB2xzxObDxJGh4= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:18 +0200 Message-Id: <20211026095534.90348-4-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 03/19] ipa: ipu3: Document the IPAIPU3 class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Clarify the roles and interactions between the pipeline handler events and the algorithm calls by documenting all the remaining functions of the class. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain --- src/ipa/ipu3/ipu3.cpp | 87 +++++++++++++++++++++++++++++++++++++------ 1 file changed, 76 insertions(+), 11 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 1f501821..177b67b0 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -305,9 +305,9 @@ private: struct IPAContext context_; }; -/* - * Compute IPASessionConfiguration using the sensor information and the sensor - * v4l2 controls. +/** + * \brief Compute IPASessionConfiguration using the sensor information and the + * sensor V4L2 controls */ void IPAIPU3::updateSessionConfiguration(const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls) @@ -336,9 +336,9 @@ void IPAIPU3::updateSessionConfiguration(const IPACameraSensorInfo &sensorInfo, context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain); } -/* - * Compute camera controls using the sensor information and the sensor - * v4l2 controls. +/** + * \brief Compute camera controls using the sensor information and the sensor + * V4L2 controls * * Some of the camera controls are computed by the pipeline handler, some others * by the IPA module which is in charge of handling, for example, the exposure @@ -399,7 +399,7 @@ void IPAIPU3::updateControls(const IPACameraSensorInfo &sensorInfo, } /** - * Initialize the IPA module and its controls. + * \brief Initialize the IPA module and its controls * * This function receives the camera sensor information from the pipeline * handler, computes the limits of the controls it handles and returns @@ -430,8 +430,15 @@ int IPAIPU3::init(const IPASettings &settings, return 0; } +/** + * \brief Perform any processing required before the first frame + */ int IPAIPU3::start() { + /* + * Set the sensors V4L2 controls before the first frame to ensure that + * we have an expected and known configuration from the start. + */ setControls(0); return 0; @@ -512,7 +519,7 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) * handler * \param[in] ipaControls The IPA controls to update * - * Calculate the best grid for the statistics based on the Pipeline Handler BDS + * Calculate the best grid for the statistics based on the pipeline handler BDS * output, and parse the minimum and maximum exposure and analogue gain control * values. * @@ -585,6 +592,10 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo, return 0; } +/** + * \brief Map the parameters and stats buffers allocated in the pipeline handler + * \param[in] buffers The buffers to map + */ void IPAIPU3::mapBuffers(const std::vector &buffers) { for (const IPABuffer &buffer : buffers) { @@ -594,6 +605,10 @@ void IPAIPU3::mapBuffers(const std::vector &buffers) } } +/** + * \brief Unmap the parameters and stats buffers + * \param[in] ids The IDs of the buffers to unmap + */ void IPAIPU3::unmapBuffers(const std::vector &ids) { for (unsigned int id : ids) { @@ -605,6 +620,10 @@ void IPAIPU3::unmapBuffers(const std::vector &ids) } } +/** + * \brief Process an event generated by the pipeline handler + * \param[in] event The event sent from pipeline handler + */ void IPAIPU3::processEvent(const IPU3Event &event) { switch (event.op) { @@ -646,12 +665,28 @@ void IPAIPU3::processEvent(const IPU3Event &event) } } +/** + * \brief Process a control list for a request from the application + * \param[in] frame The number of the frame which will be processed next + * \param[in] controls The controls for the \a frame + * + * Parse the request to handle any IPA-managed controls that were set from the + * application such as manual sensor settings. + */ void IPAIPU3::processControls([[maybe_unused]] unsigned int frame, [[maybe_unused]] const ControlList &controls) { /* \todo Start processing for 'frame' based on 'controls'. */ } +/** + * \brief Fill the ImgU parameter buffer for the next frame + * \param[in] frame The number of the latest frame processed + * \param[out] params The parameter buffer to fill + * + * Algorithms are expected to fill the IPU3 parameter buffer for the next + * frame given their most recent processing of the ImgU statistics. + */ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) { /* @@ -674,6 +709,16 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) queueFrameAction.emit(frame, op); } +/** + * \brief Process the statistics generated by the ImgU + * \param[in] frame The number of the latest frame processed + * \param[in] frameTimestamp The current frame timestamp + * \param[in] stats The IPU3 statistics and ISP results + * + * Parse the most recently processed image statistics from the ImgU. The + * statistics are passed to each algorithm module to run their calculations and + * update their state accordingly. + */ void IPAIPU3::parseStatistics(unsigned int frame, [[maybe_unused]] int64_t frameTimestamp, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) @@ -697,6 +742,13 @@ void IPAIPU3::parseStatistics(unsigned int frame, queueFrameAction.emit(frame, op); } +/** + * \brief Handle sensor controls for a given \a frame number + * \param[in] frame The frame on which the sensor controls should be set + * + * Send the desired sensor control values to the pipeline handler to request + * that they are applied on the camera sensor. + */ void IPAIPU3::setControls(unsigned int frame) { IPU3Action op; @@ -715,10 +767,15 @@ void IPAIPU3::setControls(unsigned int frame) } /* namespace ipa::ipu3 */ -/* - * External IPA module interface +/** + * \brief External IPA module interface + * + * The IPAModuleInfo is required to match an IPA module construction against the + * intented pipeline handler with the module. The API and pipeline handler + * versions must match the corresponding IPA interface and pipeline handler. + * + * \sa struct IPAModuleInfo */ - extern "C" { const struct IPAModuleInfo ipaModuleInfo = { IPA_MODULE_API_VERSION, @@ -727,6 +784,14 @@ const struct IPAModuleInfo ipaModuleInfo = { "ipu3", }; +/** + * \brief Create an instance of the IPA interface + * + * This function is the entry point of the IPA module. It is called by the IPA + * manager to create an instance of the IPA interface for each camera. When + * matched against with a pipeline handler, the IPAManager will construct an IPA + * instance for each associated Camera. + */ IPAInterface *ipaCreate() { return new ipa::ipu3::IPAIPU3(); From patchwork Tue Oct 26 09:55:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14315 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4AF53BDB1C for ; Tue, 26 Oct 2021 09:55:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4408364883; Tue, 26 Oct 2021 11:55:52 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="MfRmX7SH"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 44B596487A for ; Tue, 26 Oct 2021 11:55:45 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E862EF7B; Tue, 26 Oct 2021 11:55:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242145; bh=EnGGCYCxMtryh7kpm+1yH7bkYgiZ+JtzUkgl1uMuzF0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MfRmX7SHHsRcZL0etEztF4edLf0kjfvGWcEkWs4AFL9+z3HM/WGwGknjMMtRcjULY cP2Lw2hEHWyvqTl0gn2ki/nOtAu4cKksPD+mjCbWM9r0Tz7+ZXU017cSVftMnW5EZb 6Y992FX0mFlaYFlj3syZlpbvxeLbz7Ot6xFzooMY= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:19 +0200 Message-Id: <20211026095534.90348-5-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 04/19] ipa: ipu3: Explicitly use the statistics parameter X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The stats pointer is marked as [[maybe_unused]]. This is a leftover from a previous commit which was here to keep the compatibility while transitioning to the new iterative algorithms. Remove this attribute to make it explicit that stats are really used to feed the algorithms. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain --- src/ipa/ipu3/ipu3.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 177b67b0..bcaa34da 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -721,7 +721,7 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) */ void IPAIPU3::parseStatistics(unsigned int frame, [[maybe_unused]] int64_t frameTimestamp, - [[maybe_unused]] const ipu3_uapi_stats_3a *stats) + const ipu3_uapi_stats_3a *stats) { ControlList ctrls(controls::controls); From patchwork Tue Oct 26 09:55:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14316 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0DA01BDB1C for ; Tue, 26 Oct 2021 09:55:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 27BF364899; Tue, 26 Oct 2021 11:55:54 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="F2bdonMw"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 77AED60123 for ; Tue, 26 Oct 2021 11:55:45 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2C9EF3F0; Tue, 26 Oct 2021 11:55:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242145; bh=aJzcaOw0CLDeD3nP0A2GpJAEE9VikFQMqZLFac3ZMAw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F2bdonMwxRrRIb1PlDbkJPeIMlxTwbdRsEzHYBx+F3tttCGwBg0KUmGFiImo82xIS qNcgTU7olNrZoNrB4X3CT/osOI9LV2k5GCqoXjg0uPajgospGZYTMi4mQ9+upWkWvj rlQETR8YKD1gJirtuEfsbtWFmHvLV88M3S4x+tRM= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:20 +0200 Message-Id: <20211026095534.90348-6-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 05/19] ipa: ipu3: awb: Add AWB class documentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AWB algorithm is based on the Grey world algorithm and uses the statistics generated by the ImgU for that. Explain how it uses those, and reference the original algorithm at the same time. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- v3: - move the diagram from Accumulator to AWB - add a word to say green gains are always set to 1 --- src/ipa/ipu3/algorithms/awb.cpp | 110 ++++++++++++++++++++------------ 1 file changed, 69 insertions(+), 41 deletions(-) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 91364a04..8d22aceb 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -11,6 +11,10 @@ #include +/** + * \file awb.h + */ + namespace libcamera { namespace ipa::ipu3::algorithms { @@ -42,47 +46,6 @@ static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 90 / 100; * \struct Accumulator * \brief RGB statistics for a given zone * - * - Cells are defined in Pixels - * - Zones are defined in Cells - * - * 80 cells - * /───────────── 1280 pixels ───────────\ - * 16 zones - * 16 - * ┌────┬────┬────┬────┬────┬─ ──────┬────┐ \ - * │Cell│ │ │ │ │ | │ │ │ - * 16 │ px │ │ │ │ │ | │ │ │ - * ├────┼────┼────┼────┼────┼─ ──────┼────┤ │ - * │ │ │ │ │ │ | │ │ - * │ │ │ │ │ │ | │ │ 7 - * │ ── │ ── │ ── │ ── │ ── │ ── ── ─┤ ── │ 1 2 4 - * │ │ │ │ │ │ | │ │ 2 0 5 - * - * │ │ │ │ │ │ | │ │ z p c - * ├────┼────┼────┼────┼────┼─ ──────┼────┤ o i e - * │ │ │ │ │ │ | │ │ n x l - * │ │ | │ │ e e l - * ├─── ───┼─ ──────┼────┤ s l s - * │ │ | │ │ s - * │ │ | │ │ - * ├─── Zone of Cells ───┼─ ──────┼────┤ │ - * │ (5 x 4) │ | │ │ │ - * │ │ | │ │ │ - * ├── ───┼─ ──────┼────┤ │ - * │ │ │ | │ │ │ - * │ │ │ │ │ │ | │ │ │ - * └────┴────┴────┴────┴────┴─ ──────┴────┘ / - * - * - * The algorithm works with a fixed number of zones \a kAwbStatsSizeX x - * \a kAwbStatsSizeY. For example, a frame of 1280x720 is divided into 80x45 - * cells of [16x16] pixels. In the case of \a kAwbStatsSizeX=16 and - * \a kAwbStatsSizeY=12 the zones are made of [5x4] cells. The cells are - * left-aligned and calculated by IPAIPU3::calculateBdsGrid(). - * - * Each statistics cell represents the average value of the pixels in that cell - * split by colour components. - * * The Accumulator structure stores the sum of the average of each cell in a * zone of the image, as well as the number of cells which were unsaturated and * therefore included in the average. @@ -152,6 +115,71 @@ static const struct ipu3_uapi_ccm_mat_config imguCssCcmDefault = { 0, 0, 8191, 0 }; +/** + * \class Awb + * \brief A Grey world white balance correction algorithm + * + * The Grey World algorithm assumes that the scene, in average, is neutral grey. + * Reference: Lam, Edmund & Fung, George. (2008). Automatic White Balancing in + * Digital Photography. 10.1201/9781420054538.ch10. + * + * The IPU3 generates statistics from the Bayer Down Scaler output into a grid + * defined in the ipu3_uapi_awb_config_s structure. + * + * - Cells are defined in Pixels + * - Zones are defined in Cells + * + * 80 cells + * /───────────── 1280 pixels ───────────\ + * 16 zones + * 16 + * ┌────┬────┬────┬────┬────┬─ ──────┬────┐ \ + * │Cell│ │ │ │ │ | │ │ │ + * 16 │ px │ │ │ │ │ | │ │ │ + * ├────┼────┼────┼────┼────┼─ ──────┼────┤ │ + * │ │ │ │ │ │ | │ │ + * │ │ │ │ │ │ | │ │ 7 + * │ ── │ ── │ ── │ ── │ ── │ ── ── ─┤ ── │ 1 2 4 + * │ │ │ │ │ │ | │ │ 2 0 5 + * + * │ │ │ │ │ │ | │ │ z p c + * ├────┼────┼────┼────┼────┼─ ──────┼────┤ o i e + * │ │ │ │ │ │ | │ │ n x l + * │ │ | │ │ e e l + * ├─── ───┼─ ──────┼────┤ s l s + * │ │ | │ │ s + * │ │ | │ │ + * ├─── Zone of Cells ───┼─ ──────┼────┤ │ + * │ (5 x 4) │ | │ │ │ + * │ │ | │ │ │ + * ├── ───┼─ ──────┼────┤ │ + * │ │ │ | │ │ │ + * │ │ │ │ │ │ | │ │ │ + * └────┴────┴────┴────┴────┴─ ──────┴────┘ / + * + * + * In each cell, the ImgU computes for each colour component the average of all + * unsaturated pixels (below a programmable threshold). It also provides the + * ratio of saturated pixels in the cell. + * + * The AWB algorithm operates on a coarser grid, made by grouping cells from the + * hardware grid into zones. The number of zones is fixed to \a kAwbStatsSizeX x + * \a kAwbStatsSizeY. For example, a frame of 1280x720 is divided into 80x45 + * cells of [16x16] pixels and 16x12 zones of [5x4] cells each + * (\a kAwbStatsSizeX=16 and \a kAwbStatsSizeY=12). If the number of cells isn't + * an exact multiple of the number of zones, the right-most and bottom-most + * cells are ignored. The grid configuration is computed by + * IPAIPU3::calculateBdsGrid(). + * + * Before calculating the gains, the algorithm aggregates the cell averages for + * each zone in generateAwbStats(). Cells that have a too high ratio of + * saturated pixels are ignored, and only zones that contain enough + * non-saturated cells are then used by the algorithm. + * + * The Grey World algorithm will then estimate the red and blue gains to apply, and + * store the results in the metadata. The green gain is always set to 1. + */ + Awb::Awb() : Algorithm() { From patchwork Tue Oct 26 09:55:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14317 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E1E45BDB1C for ; Tue, 26 Oct 2021 09:55:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1911C64889; Tue, 26 Oct 2021 11:55:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="neTvf+Ac"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A675C6487B for ; Tue, 26 Oct 2021 11:55:45 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5E638DBF; Tue, 26 Oct 2021 11:55:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242145; bh=Pcp9Mc6UW2yRJj7DFkomENi5B1HeAE+xxza0qDQCuxE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=neTvf+AcCcQ7sPMVZ6Z6b+h9f6jyzyvhhmX5Fr1VihwikjsaPoJZtzPvYAebH2iCf 1nFqD0nL7wzv/MpG10iHPAdYMijdgk8wr3Z0yO3j5/eGJih2WpOy846W1Pw6P9k2ng nagQaZTw+a/HItsxmGIUC0mDBXThBDJQGtUUz57Q= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:21 +0200 Message-Id: <20211026095534.90348-7-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 06/19] ipa: ipu3: awb: Reword accumulator documentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that we moved the diagram into the AWB class documentation, reword the accumulator documentation to make it clear it is not meant to be used only in AWB. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/awb.cpp | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 8d22aceb..41fd5fc4 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -46,12 +46,12 @@ static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 90 / 100; * \struct Accumulator * \brief RGB statistics for a given zone * - * The Accumulator structure stores the sum of the average of each cell in a - * zone of the image, as well as the number of cells which were unsaturated and - * therefore included in the average. + * Accumulate red, green and blue values for each non-saturated item over a + * zone. Items can for instance be pixels, but also the average of groups of + * pixels, depending on who uses the accumulator. * \todo move this description and structure into a common header * - * Cells which are saturated beyond the threshold defined in + * Zones which are saturated beyond the threshold defined in * ipu3_uapi_awb_config_s are not included in the average. * * \var Accumulator::counted From patchwork Tue Oct 26 09:55:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14318 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 93573BDB1C for ; Tue, 26 Oct 2021 09:55:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ECC786488C; Tue, 26 Oct 2021 11:55:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="tNVYecM/"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D91A864879 for ; Tue, 26 Oct 2021 11:55:45 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8BBDAE79; Tue, 26 Oct 2021 11:55:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242145; bh=hoDs3INkuwpY3zdfR2YhQq2BGfKqfqp4QiDoEmPIVmY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tNVYecM/CvWsxWtUgPUOQpfLcHsmaOg62OAAx89PPSDa/3YApD/Gipt+0un6gI8HK h07QSL2HLoGLC65zriTS6qOmZkg7VCzWs3FukhZ+8pMnAK1XoZ4W61NWGNQ3gCKgI7 stjg+DdcjcV893QzSqQ5KH6mf719PyM1FlA6IGo0= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:22 +0200 Message-Id: <20211026095534.90348-8-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 07/19] ipa: ipu3: agc: Document AGC mean-based algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AGC class was not documented while developing. Extend that to reference the origins of the implementation, and improve the descriptions on how the algorithm operates internally. While at it, rename the functions which have bad names. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- v3: - rename processBrightness to measureBrightness - rename lockExposureGain to computeExposure - reword some comments --- src/ipa/ipu3/algorithms/agc.cpp | 80 ++++++++++++++++++++++++++++++--- src/ipa/ipu3/algorithms/agc.h | 6 +-- 2 files changed, 76 insertions(+), 10 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 6c151232..b89d1559 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -2,7 +2,7 @@ /* * Copyright (C) 2021, Ideas On Board * - * ipu3_agc.cpp - AGC/AEC control algorithm + * ipu3_agc.cpp - AGC/AEC mean-based control algorithm */ #include "agc.h" @@ -17,17 +17,37 @@ #include "libipa/histogram.h" +/** + * \file agc.h + */ + namespace libcamera { using namespace std::literals::chrono_literals; namespace ipa::ipu3::algorithms { +/** + * \class Agc + * \brief A mean-based auto-exposure algorithm + * + * This algorithm calculates a shutter time and an analogue gain so that the + * average value of the green channel of the brightest 2% of pixels approaches + * 0.5. The AWB gains are not used here, and all cells in the grid have the same + * weight, like an average-metering case. In this metering mode, the camera uses + * light information from the entire scene and creates an average for the final + * exposure setting, giving no weighting to any particular portion of the + * metered area. + * + * Reference: Battiato, Messina & Castorina. (2008). Exposure + * Correction for Imaging Devices: An Overview. 10.1201/9781420054538.ch12. + */ + LOG_DEFINE_CATEGORY(IPU3Agc) /* Number of frames to wait before calculating stats on minimum exposure */ static constexpr uint32_t kInitialFrameMinAECount = 4; -/* Number of frames to wait between new gain/exposure estimations */ +/* Number of frames to wait between new gain/shutter time estimations */ static constexpr uint32_t kFrameSkipCount = 6; /* Limits for analogue gain values */ @@ -36,6 +56,8 @@ static constexpr double kMaxAnalogueGain = 8.0; /* Histogram constants */ static constexpr uint32_t knumHistogramBins = 256; + +/* Target value to reach for the top 2% of the histogram */ static constexpr double kEvGainTarget = 0.5; Agc::Agc() @@ -45,10 +67,18 @@ Agc::Agc() { } +/** + * \brief Configure the AGC given a configInfo + * \param[in] context The shared IPA context + * \param[in] configInfo The IPA configuration data + * + * \return 0 + */ int Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) { stride_ = context.configuration.grid.stride; + /* \todo use the IPAContext to provide the limits */ lineDuration_ = configInfo.sensorInfo.lineLength * 1.0s / configInfo.sensorInfo.pixelRate; @@ -70,9 +100,15 @@ int Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) return 0; } -void Agc::processBrightness(const ipu3_uapi_stats_3a *stats, +/** + * \brief Estimate the mean value of the top 2% of the histogram + * \param[in] stats The statistics computed by the ImgU + * \param[in] grid The grid used to store the statistics in the IPU3 + */ +void Agc::measureBrightness(const ipu3_uapi_stats_3a *stats, const ipu3_uapi_grid_config &grid) { + /* Initialise the histogram array */ uint32_t hist[knumHistogramBins] = { 0 }; for (unsigned int cellY = 0; cellY < grid.height; cellY++) { @@ -87,6 +123,11 @@ void Agc::processBrightness(const ipu3_uapi_stats_3a *stats, if (cell->sat_ratio == 0) { uint8_t gr = cell->Gr_avg; uint8_t gb = cell->Gb_avg; + /* + * Store the average green value to estimate the + * brightness. Even the overexposed pixels are + * taken into account. + */ hist[(gr + gb) / 2]++; } } @@ -96,6 +137,9 @@ void Agc::processBrightness(const ipu3_uapi_stats_3a *stats, iqMean_ = Histogram(Span(hist)).interQuantileMean(0.98, 1.0); } +/** + * \brief Apply a filter on the exposure value to limit the speed of changes + */ void Agc::filterExposure() { double speed = 0.2; @@ -106,7 +150,7 @@ void Agc::filterExposure() /* * If we are close to the desired result, go faster to avoid making * multiple micro-adjustments. - * \ todo: Make this customisable? + * \todo Make this customisable? */ if (filteredExposure_ < 1.2 * currentExposure_ && filteredExposure_ > 0.8 * currentExposure_) @@ -119,7 +163,12 @@ void Agc::filterExposure() LOG(IPU3Agc, Debug) << "After filtering, total_exposure " << filteredExposure_; } -void Agc::lockExposureGain(uint32_t &exposure, double &analogueGain) +/** + * \brief Estimate the new exposure and gain values + * \param[inout] exposure The exposure value reference as a number of lines + * \param[inout] gain The gain reference to be updated + */ +void Agc::computeExposure(uint32_t &exposure, double &analogueGain) { /* Algorithm initialization should wait for first valid frames */ /* \todo - have a number of frames given by DelayedControls ? @@ -136,19 +185,26 @@ void Agc::lockExposureGain(uint32_t &exposure, double &analogueGain) return; } + /* Estimate the gain needed to have the proportion wanted */ double evGain = kEvGainTarget * knumHistogramBins / iqMean_; /* extracted from Rpi::Agc::computeTargetExposure */ + /* Calculate the shutter time in seconds */ utils::Duration currentShutter = exposure * lineDuration_; LOG(IPU3Agc, Debug) << "Actual total exposure " << currentShutter * analogueGain << " Shutter speed " << currentShutter << " Gain " << analogueGain << " Needed ev gain " << evGain; + /* + * Calculate the current exposure value for the scene as the latest + * exposure value applied multiplied by the new estimated gain. + */ currentExposure_ = prevExposureValue_ * evGain; utils::Duration minShutterSpeed = minExposureLines_ * lineDuration_; utils::Duration maxShutterSpeed = maxExposureLines_ * lineDuration_; + /* Clamp the exposure value to the min and max authorized */ utils::Duration maxTotalExposure = maxShutterSpeed * maxAnalogueGain_; currentExposure_ = std::min(currentExposure_, maxTotalExposure); LOG(IPU3Agc, Debug) << "Target total exposure " << currentExposure_ @@ -157,6 +213,7 @@ void Agc::lockExposureGain(uint32_t &exposure, double &analogueGain) /* \todo: estimate if we need to desaturate */ filterExposure(); + /* Divide the exposure value as new exposure and gain values */ utils::Duration exposureValue = filteredExposure_; utils::Duration shutterTime = minShutterSpeed; @@ -186,12 +243,21 @@ void Agc::lockExposureGain(uint32_t &exposure, double &analogueGain) prevExposureValue_ = shutterTime * analogueGain; } +/** + * \brief Process IPU3 statistics, and run AGC operations + * \param[in] context The shared IPA context + * \param[in] stats The IPU3 statistics and ISP results + * + * Identify the current image brightness, and use that to estimate the optimal + * new exposure and gain for the scene. + */ void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { + /* Get the latest exposure and gain applied */ uint32_t &exposure = context.frameContext.agc.exposure; double &analogueGain = context.frameContext.agc.gain; - processBrightness(stats, context.configuration.grid.bdsGrid); - lockExposureGain(exposure, analogueGain); + measureBrightness(stats, context.configuration.grid.bdsGrid); + computeExposure(exposure, analogueGain); frameCount_++; } diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 1840205b..69e0b831 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -2,7 +2,7 @@ /* * Copyright (C) 2021, Ideas On Board * - * agc.h - IPU3 AGC/AEC control algorithm + * agc.h - IPU3 AGC/AEC mean-based control algorithm */ #ifndef __LIBCAMERA_IPU3_ALGORITHMS_AGC_H__ #define __LIBCAMERA_IPU3_ALGORITHMS_AGC_H__ @@ -31,10 +31,10 @@ public: void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; private: - void processBrightness(const ipu3_uapi_stats_3a *stats, + void measureBrightness(const ipu3_uapi_stats_3a *stats, const ipu3_uapi_grid_config &grid); void filterExposure(); - void lockExposureGain(uint32_t &exposure, double &gain); + void computeExposure(uint32_t &exposure, double &gain); uint64_t frameCount_; uint64_t lastFrame_; From patchwork Tue Oct 26 09:55:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14319 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3A9F9BDB1C for ; Tue, 26 Oct 2021 09:55:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C777864890; Tue, 26 Oct 2021 11:55:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="wamfLqMY"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 285ED64881 for ; Tue, 26 Oct 2021 11:55:46 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BF6B43F0; Tue, 26 Oct 2021 11:55:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242145; bh=HPMVdf68yWVnoSXsmho/S+LzQSwSso72VFKAkZXohZM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wamfLqMYEKVu1IkV3ZhJlUh7HGwK/rPoipvJ0lBAG889UCKdMkzC5EKV4TLLoB1bu 6VWWQyMdHqHTqmDWtJRRmOvNDc08Kx2bfaAAEAzecfOT51FZshxTXobwu3+pEctaTz xQYdMi+FiG6CzXaIcbhYEIPsu+zvvBR2Xs5w/njw= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:23 +0200 Message-Id: <20211026095534.90348-9-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 08/19] ipa: ipu3: tonemapping: Generate the LUT only on gamma change X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The tone mapping algorithm calculates the gamma curve for every frame, regardless of whether the gamma value has changed or not. This issue is exasperated as we currently hardcode the gamma to a single value. Optimise the implementation to only recalculate the look up table when the gamma setting is changed, and store the gamma setting of the LUT curve as part of the IPA context. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/tone_mapping.cpp | 5 +++++ src/ipa/ipu3/ipa_context.h | 1 + src/ipa/ipu3/ipu3.cpp | 3 +++ 3 files changed, 9 insertions(+) diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 3af96261..40337f9d 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -43,6 +43,9 @@ void ToneMapping::process([[maybe_unused]] IPAContext &context, */ gamma_ = 1.1; + if (context.frameContext.toneMapping.gamma == gamma_) + return; + struct ipu3_uapi_gamma_corr_lut &lut = context.frameContext.toneMapping.gammaCorrection; @@ -53,6 +56,8 @@ void ToneMapping::process([[maybe_unused]] IPAContext &context, /* The output value is expressed on 13 bits. */ lut.lut[i] = gamma * 8191; } + + context.frameContext.toneMapping.gamma = gamma_; } } /* namespace ipa::ipu3::algorithms */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 847c03fe..e14bb561 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -48,6 +48,7 @@ struct IPAFrameContext { } awb; struct { + double gamma; struct ipu3_uapi_gamma_corr_lut gammaCorrection; } toneMapping; }; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index bcaa34da..4c0fff00 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -151,6 +151,9 @@ * \struct IPAFrameContext::toneMapping * \brief Context for ToneMapping and Gamma control * + * \var IPAFrameContext::toneMapping::gamma + * \brief Gamma value for the LUT + * * \var IPAFrameContext::toneMapping::gammaCorrection * \brief Per-pixel tone mapping implemented as a LUT * From patchwork Tue Oct 26 09:55:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14320 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 06620BDB1C for ; Tue, 26 Oct 2021 09:55:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A12316487C; Tue, 26 Oct 2021 11:55:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="TVvSZhOZ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5412064884 for ; Tue, 26 Oct 2021 11:55:46 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F29BBDBF; Tue, 26 Oct 2021 11:55:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242146; bh=+hSQOQoqtykLWcHx2GBqz1QrBOoOZUzi8gu547oRCY0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TVvSZhOZvJJqOUx8z8hdFXS1XE03D/8JjxoOaY6nA3FaxYRtx3VkRLkkMdLaAM6Ii +S1Ph6Qbr5H49p8t+pVeQloYXchAT3vM578g4NmrB7n8FPJkztAws9bdjq9NS28xHC hgXFoP6QLQMOQzTKf9IisCaL8+kWhJlYmGMdBC4U= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:24 +0200 Message-Id: <20211026095534.90348-10-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 09/19] ipa: ipu3: tonemapping: Implement configure call X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Let the algorithm perform its initial configuration. Implement configure() to set a default gamma value and let process do the updates needed. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/tone_mapping.cpp | 16 ++++++++++++++++ src/ipa/ipu3/algorithms/tone_mapping.h | 1 + 2 files changed, 17 insertions(+) diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 40337f9d..fcd22d0b 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -19,6 +19,22 @@ ToneMapping::ToneMapping() { } +/** + * \brief Configure the tone mapping given a configInfo + * \param[in] context The shared IPA context + * \param[in] configInfo The IPA configuration data + * + * \return 0 + */ +int ToneMapping::configure(IPAContext &context, + [[maybe_unused]] const IPAConfigInfo &configInfo) +{ + /* Initialise tone mapping gamma value. */ + context.frameContext.toneMapping.gamma = 0.0; + + return 0; +} + void ToneMapping::prepare([[maybe_unused]] IPAContext &context, ipu3_uapi_params *params) { diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index 1dae4f9a..46dd6171 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -18,6 +18,7 @@ class ToneMapping : public Algorithm public: ToneMapping(); + int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, ipu3_uapi_params *params) override; void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; From patchwork Tue Oct 26 09:55:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14321 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1141CBDB1C for ; Tue, 26 Oct 2021 09:55:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9EB7B64895; Tue, 26 Oct 2021 11:55:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="MpKP3Pli"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8197F6487E for ; Tue, 26 Oct 2021 11:55:46 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3374AE79; Tue, 26 Oct 2021 11:55:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242146; bh=7BGr7hiTSDbpG7tzJHV7R/26QoDeqAxF3NbLgzjonZY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MpKP3PliVsikYi47Uuwbwb6pYjvKk2hPDyau/FRXUvy5WMkwgVOBi/zzupUSuyeRJ UzQ/nqFlBtR8SgpV1LjaLTBe4mM5/HqbLSi5JRQtajRuzsq7E+4jCHfS9EhJYMwTwN u+VsMqUQ5ur4dwBxwWrrsQCmRvAnwcw8R/RMoYUc= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:25 +0200 Message-Id: <20211026095534.90348-11-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 10/19] ipa: ipu3: tonemapping: Add the documentation for ToneMapping X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The tone mapping algorithm is currently undocumented. Provide an introduction and overview to the implementation as the class definition and document how the algorithm operates in the process and prepare methods. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- v3: - rename gamma control as tone mapping --- src/ipa/ipu3/algorithms/tone_mapping.cpp | 30 +++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index fcd22d0b..2040eda5 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -10,10 +10,22 @@ #include #include +/** + * \file tone_mapping.h + */ + namespace libcamera { namespace ipa::ipu3::algorithms { +/** + * \class ToneMapping + * \brief A class to handle tone mapping based on gamma + * + * This algorithm improves the image dynamic using a look-up table which is + * generated based on a gamma parameter. + */ + ToneMapping::ToneMapping() : gamma_(1.0) { @@ -35,6 +47,14 @@ int ToneMapping::configure(IPAContext &context, return 0; } +/** + * \brief Fill in the parameter structure, and enable gamma control + * \param context The shared IPA context + * \param params The IPU3 parameters + * + * Populate the IPU3 parameter structure with our tone mapping look up table and + * enable the gamma control module in the processing blocks. + */ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, ipu3_uapi_params *params) { @@ -49,7 +69,15 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, params->acc_param.gamma.gc_ctrl.enable = 1; } -void ToneMapping::process([[maybe_unused]] IPAContext &context, +/** + * \brief Calculate the tone mapping look up table + * \param context The shared IPA context + * \param stats The IPU3 statistics and ISP results + * + * The tone mapping look up table is generated as an inverse power curve from + * our gamma setting. + */ +void ToneMapping::process(IPAContext &context, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) { /* From patchwork Tue Oct 26 09:55:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14322 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C20ADBDB1C for ; Tue, 26 Oct 2021 09:56:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8023D648A0; Tue, 26 Oct 2021 11:56:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fsspV2wc"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B70F264888 for ; Tue, 26 Oct 2021 11:55:46 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 666333F0; Tue, 26 Oct 2021 11:55:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242146; bh=SSi1w6AEKxHmvUi5enuxvrAbDk64vPlPqXrbgRiUJsc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fsspV2wcaeByRc8gw8JR+0VIL9GSIrKflgVjL9hpUiYaHs4eqqSyU6zMtHzIRb4x6 VGLruz47wjhtm5m21jCDnUUQ4Ks26NCO0uHXr2DpljpOO8blQC1JeYfu6GBu+USd5y pLl8UdyeBkAKfCbgaudAYWPloxTA4LvZ0DSDnk6M= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:26 +0200 Message-Id: <20211026095534.90348-12-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 11/19] Documentation: IPU3 IPA Design guide X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The IPU3 IPA implements the basic 3A using the ImgU ISP. Provide an overview document to describe its operations, and provide a block diagram to help visualise how the components are put together to assist any new developers exploring the code. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Jean-Michel Hautbois --- v2: - /accelerator cluster/processing block/ (and refactor) - /Pipeline/pipeline/ - /Camera Sensor/camera sensor/ - /CPU accessible/CPU-accessible/ - Remove updated control parameters from IPASessionConfiguration - Expand pre-frame preparation to match post-frame with the event descriptions. - Add Sensor Controls brief - Move to src/ipa/ipu3/ - Lower indentation of the block diagrams (keep under 80chars) - reference mapBuffers() call for passing buffers in - reference unmapBuffers() after stop() --- src/ipa/ipu3/ipu3-ipa-design-guide.rst | 155 +++++++++++++++++++++++++ 1 file changed, 155 insertions(+) create mode 100644 src/ipa/ipu3/ipu3-ipa-design-guide.rst diff --git a/src/ipa/ipu3/ipu3-ipa-design-guide.rst b/src/ipa/ipu3/ipu3-ipa-design-guide.rst new file mode 100644 index 00000000..89c71108 --- /dev/null +++ b/src/ipa/ipu3/ipu3-ipa-design-guide.rst @@ -0,0 +1,155 @@ +IPU3 IPA Architecture Design and Overview +========================================= + +The IPU3 IPA is built as a modular and extensible framework with an +upper layer to manage the interactions with the pipeline handler, and +the image processing algorithms split to compartmentalise the processing +required for each processing block, making use of the fixed-function +accelerators provided by the ImgU ISP. + +The core IPU3 class is responsible for initialisation and construction +of the algorithm components, processing controls set by the requests +from applications, and managing events from the pipeline handler. + +:: + + ┌───────────────────────────────────────────┐ + │ IPU3 Pipeline Handler │ + │ ┌────────┐ ┌────────┐ ┌────────┐ │ + │ │ │ │ │ │ │ │ + │ │ Sensor ├───►│ CIO2 ├───►│ ImgU ├──► + │ │ │ │ │ │ │ │ + │ └────────┘ └────────┘ └─▲────┬─┘ │ P: Parameter Buffer + │ │P │ │ S: Statistics Buffer + │ │ │S │ + └─┬───┬───┬──────┬────┬────┬────┬─┴────▼─┬──┘ 1: init() + │ │ │ │ ▲ │ ▲ │ ▲ │ ▲ │ 2: configure() + │1 │2 │3 │4│ │4│ │4│ │4│ │5 3: mapBuffers(), start() + ▼ ▼ ▼ ▼ │ ▼ │ ▼ │ ▼ │ ▼ 4: processEvent() + ┌──────────────────┴────┴────┴────┴─────────┐ 5: stop(), unmapBuffers() + │ IPU3 IPA │ + │ ┌───────────────────────┐ │ + │ ┌───────────┐ │ Algorithms │ │ + │ │IPAContext │ │ ┌─────────┐ │ │ + │ │ ┌───────┐ │ │ │ ... │ │ │ + │ │ │ │ │ │ ┌─┴───────┐ │ │ │ + │ │ │ SC │ │ │ │ Tonemap ├─┘ │ │ + │ │ │ │ ◄───► ┌─┴───────┐ │ │ │ + │ │ ├───────┤ │ │ │ AWB ├─┘ │ │ + │ │ │ │ │ │ ┌─┴───────┐ │ │ │ + │ │ │ FC │ │ │ │ AGC ├─┘ │ │ + │ │ │ │ │ │ │ │ │ │ + │ │ └───────┘ │ │ └─────────┘ │ │ + │ └───────────┘ └───────────────────────┘ │ + └───────────────────────────────────────────┘ + SC: IPASessionConfiguration + FC: IPAFrameContext(s) + +The IPA instance is constructed and initialised at the point a Camera is +created by the IPU3 pipeline handler. The initialisation call provides +details about which camera sensor is being used, and the controls that +it has available, along with their default values and ranges. + +Buffers +~~~~~~~ + +The IPA will have Parameter and Statistics buffers shared with it from +the IPU3 Pipeline handler. These buffers will be passed to the IPA using +the ``mapBuffers()`` call before the ``start()`` operation occurs. + +The IPA will map the buffers into CPU-accessible memory, associated with +a buffer ID, and further events for sending or receiving parameter and +statistics buffers will reference the ID to avoid expensive memory +mapping operations, or the passing of file handles during streaming. + +After the ``stop()`` operation occurs, these buffers will be unmapped +when requested by the pipeline handler using the ``unmapBuffers()`` call +and no further access to the buffers is permitted. + +Context +~~~~~~~ + +Algorithm calls will always have the ``IPAContext`` available to them. +This context comprises of two parts: + +- IPA Session Configuration +- IPA Frame Context + +The session configuration structure ``IPASessionConfiguration`` +represents constant parameters determined before streaming commenced +during ``configure()``. + +The IPA Frame Context provides the storage for algorithms for a single +frame operation. + +The ``IPAFrameContext`` structure may be extended to an array, list, or +queue to store historical state for each frame, allowing algorithms to +obtain and reference results of calculations which are deeply pipelined. +This may only be done if an algorithm needs to know the context that was +applied at the frame the statistics were produced for, rather than the +previous or current frame. + +Presently there is a single ``IPAFrameContext`` without historical data, +and the context is maintained and updated through successive processing +operations. + +Operating +~~~~~~~~~ + +There are three main interactions with the algorithms for the IPU3 IPA +to operate when running: + +- configure() +- processEvent(``EventFillParams``) +- processEvent(``EventStatReady``) + +The configuration phase allows the pipeline-handler to inform the IPA of +the current stream configurations, which is then passed into each +algorithm to provide an opportunity to identify and track state of the +hardware, such as image size or ImgU pipeline configurations. + +Pre-frame preparation +~~~~~~~~~~~~~~~~~~~~~ + +When configured, the IPA is notified by the pipeline handler of the +Camera ``start()`` event, after which incoming requests will be queued +for processing, requiring a parameter buffer (``ipu3_uapi_params``) to +be populated for the ImgU. This is given to the IPA through the +``EventFillParams`` event, and then passed directly to each algorithm +through the ``prepare()`` call allowing the ISP configuration to be +updated for the needs of each component that the algorithm is +responsible for. + +The algorithm should set the use flag (``ipu3_uapi_flags``) for any +structure that it modifies, and it should take care to ensure that any +structure set by a use flag is fully initialised to suitable values. + +The parameter buffer is returned to the pipeline handler through the +``ActionParamFilled`` event, and from there queued to the ImgU along +with a raw frame captured with the CIO2. + +Post-frame completion +~~~~~~~~~~~~~~~~~~~~~ + +When the capture of an image is completed, and successfully processed +through the ImgU, the generated statistics buffer +(``ipu3_uapi_stats_3a``) is given to the IPA through the +``EventStatReady`` event. This provides the IPA with an opportunity to +examine the results of the ISP and run the calculations required by each +algorithm on the new data. The algorithms may require context from the +operations of other algorithms, for example, the AWB might choose to use +a scene brightness determined by the AGC. It is important that the +algorithms are ordered to ensure that required results are determined +before they are needed. + +The ordering of the algorithm processing is determined by their +placement in the ``IPU3::algorithms_`` ordered list. + +Sensor Controls +~~~~~~~~~~~~~~~ + +The AutoExposure and AutoGain (AGC) algorithm differs slightly from the +others as it requires operating directly on the sensor, as opposed to +through the ImgU ISP. To support this, there is a dedicated action +`ActionSetSensorControls` to allow the IPA to request controls to be set +on the camera sensor through the pipeline handler. From patchwork Tue Oct 26 09:55:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14323 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3245FC324E for ; Tue, 26 Oct 2021 09:56:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D44EC64885; Tue, 26 Oct 2021 11:56:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WeMJOcTw"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ECD1264874 for ; Tue, 26 Oct 2021 11:55:46 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9DCACDBF; Tue, 26 Oct 2021 11:55:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242146; bh=xwRU0h5/NStPknJnHGdHrQAdpvpu99PZFX6QcYJ1Og4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WeMJOcTwe5uutcpSsGjNmUld8AnRx6zszOrxJE99ycbuIKWautBqDV6M+5rHFekF0 JqNUrr7rMG8QYEbfF0wtt2Q0zOeGSqtbusbBnl20mV+YiteVrtBQcLEmizlDR2FcxC hAQO+8X7GcJt4wZ7guu3Hym7oAWwGNAMIBVDj5dM= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:27 +0200 Message-Id: <20211026095534.90348-13-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 12/19] ipa: ipu3: ipa_context: Fix file reference X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The ipa_context.h entry incorrectly referenced its file name. Fix it. Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipa_context.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index e14bb561..1e46c61a 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -2,7 +2,7 @@ /* * Copyright (C) 2021, Google Inc. * - * ipu3_ipa_context.h - IPU3 IPA Context + * ipa_context.h - IPU3 IPA Context * */ #ifndef __LIBCAMERA_IPU3_IPA_CONTEXT_H__ From patchwork Tue Oct 26 09:55:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14324 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CBC98BDB1C for ; Tue, 26 Oct 2021 09:56:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6EFBB6488C; Tue, 26 Oct 2021 11:56:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GMKCZlAf"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 28AF56488A for ; Tue, 26 Oct 2021 11:55:47 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D0CA0E79; Tue, 26 Oct 2021 11:55:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242146; bh=IQKsI33iYTHwdFhO8mOJt/5WGBgIcWqXqH60HMbZak0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GMKCZlAfInL68B7+e1E05aFxDVJuyHhXaN1J7xy2tjKv6sVp1WHELsJx8u+wvyYsf hiOF1Ihiqcwg4R35RNoDYlxVocQ1pDK2UxJLDyrDr+Xfnp3yAwvhhtZqUCnv5EVo79 xc78GARXvUda5gpmoFeickasjNdWKf+U4rg5pbhs= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:28 +0200 Message-Id: <20211026095534.90348-14-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 13/19] ipa: ipu3: Fix the IPU3 AWB doxygen references X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The AWB AwbStatus structure is contained within the Awb class. Fix the Doxygen reference so that it can be found. Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/awb.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 41fd5fc4..a4114659 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -71,7 +71,7 @@ static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 90 / 100; */ /** - * \struct AwbStatus + * \struct Awb::AwbStatus * \brief AWB parameters calculated * * The AwbStatus structure is intended to store the AWB From patchwork Tue Oct 26 09:55:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14325 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 560ECC324E for ; Tue, 26 Oct 2021 09:56:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 101E86487D; Tue, 26 Oct 2021 11:56:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pjYKEVVy"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5E5C664880 for ; Tue, 26 Oct 2021 11:55:47 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 103693F0; Tue, 26 Oct 2021 11:55:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242147; bh=/r1saF9a2jSCqdIjQ+uHgXanZ5OgtGl4EHjeSuJTvWk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pjYKEVVyKWIKeAkvL3soFGE5Ebt1jMI1+UvdWTk/11Th50PLG7GUd3lJecqlAFr6N k/jJZdvIPhCD81JTUi/5+NvE//poutaxFprNLOh3rn2oy2ToHPTKDlWZvJKiA1fY2A fcfmSwi2I8Q9pNCdEHhfvHWSLsOdtzNGw+UZ2QqE= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:29 +0200 Message-Id: <20211026095534.90348-15-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 14/19] ipa: ipu3: algorithms: awb: Privatize internal structures X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The struct RGB and struct AwbStatus are used only by the internal implementation of the AWB algorithm module. Move them into the private class declaration. Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/awb.h | 1 + 1 file changed, 1 insertion(+) diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index 0c81e39e..b90782c1 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -42,6 +42,7 @@ public: void prepare(IPAContext &context, ipu3_uapi_params *params) override; void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; +private: /* \todo Make these structs available to all the ISPs ? */ struct RGB { RGB(double _R = 0, double _G = 0, double _B = 0) From patchwork Tue Oct 26 09:55:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14326 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E1A83BDB1C for ; Tue, 26 Oct 2021 09:56:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8A46D64896; Tue, 26 Oct 2021 11:56:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="iqwsJL+k"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9167A6488B for ; Tue, 26 Oct 2021 11:55:47 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 427EFDBF; Tue, 26 Oct 2021 11:55:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242147; bh=U/ju/H+2xzAq3FCepb7Wh2VYf+q8plgp1Rcz8vzkctM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iqwsJL+kPQbfo8b9mHBUPIPPsMuuBNyCT4DTKKIx87rcKPfxG0hwEp/lzHYGoyyav 8J/5MhyMVAgvVBtkUSiJUCJeft4I00z17Bi0ihpDHrR8yhmkz8r6aQ3IqcA2hobNSs la7klLEP9mEDghEvhZE0XHHn1q/Xl6maLSl2TqAE= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:30 +0200 Message-Id: <20211026095534.90348-16-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 15/19] ipa: ipu3: Isolate ipa_context documentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The IPU3 IPA core is growing with additional documentation. The ipa_context documentation is stored here, but it pushes the IPU3 documentation and implementation further from the head of the file. Furthermore, the ipa_context documentation is outside of the ipa::ipu3 namespace and isn't identified correctly by Doxygen. Move the ipa_context to its own compilation object even though there isn't any code, but to maintain consistency with our documentation model. Correctly re-introduce the documentation into the libcamera::ipa::ipu3 namespace during the move. Signed-off-by: Kieran Bingham Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipa_context.cpp | 136 +++++++++++++++++++++++++++++++++++ src/ipa/ipu3/ipu3.cpp | 124 -------------------------------- src/ipa/ipu3/meson.build | 1 + 3 files changed, 137 insertions(+), 124 deletions(-) create mode 100644 src/ipa/ipu3/ipa_context.cpp diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp new file mode 100644 index 00000000..3e154e60 --- /dev/null +++ b/src/ipa/ipu3/ipa_context.cpp @@ -0,0 +1,136 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Google Inc. + * + * ipa_context.cpp - IPU3 IPA Context + */ + +#include "ipa_context.h" + +namespace libcamera::ipa::ipu3 { + +/** + * \file ipa_context.h + * \brief Context and state information shared between the algorithms + */ + +/** + * \struct IPASessionConfiguration + * \brief Session configuration for the IPA module + * + * The session configuration contains all IPA configuration parameters that + * remain constant during the capture session, from IPA module start to stop. + * It is typically set during the configure() operation of the IPA module, but + * may also be updated in the start() operation. + */ + +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + * + * The frame context stores data specific to a single frame processed by the + * IPA. Each frame processed by the IPA has a context associated with it, + * accessible through the IPAContext structure. + * + * \todo Detail how to access contexts for a particular frame + * + * Each of the fields in the frame context belongs to either a specific + * algorithm, or to the top-level IPA module. A field may be read by any + * algorithm, but should only be written by its owner. + */ + +/** + * \struct IPAContext + * \brief Global IPA context data shared between all algorithms + * + * \var IPAContext::configuration + * \brief The IPA session configuration, immutable during the session + * + * \var IPAContext::frameContext + * \brief The frame context for the frame being processed + * + * \todo While the frame context is supposed to be per-frame, this + * single frame context stores data related to both the current frame + * and the previous frames, with fields being updated as the algorithms + * are run. This needs to be turned into real per-frame data storage. + */ + +/** + * \struct IPASessionConfiguration::grid + * \brief Grid configuration of the IPA + * + * \var IPASessionConfiguration::grid::bdsGrid + * \brief Bayer Down Scaler grid plane config used by the kernel + * + * \var IPASessionConfiguration::grid::bdsOutputSize + * \brief BDS output size configured by the pipeline handler + * + * \var IPASessionConfiguration::grid::stride + * \brief Number of cells on one line including the ImgU padding + */ + +/** + * \struct IPASessionConfiguration::agc + * \brief AGC parameters configuration of the IPA + * + * \var IPASessionConfiguration::agc::minShutterSpeed + * \brief Minimum shutter speed supported with the configured sensor + * + * \var IPASessionConfiguration::grid::maxShutterSpeed + * \brief Maximum shutter speed supported with the configured sensor + * + * \var IPASessionConfiguration::grid::minAnalogueGain + * \brief Minimum analogue gain supported with the configured sensor + * + * \var IPASessionConfiguration::grid::maxAnalogueGain + * \brief Maximum analogue gain supported with the configured sensor + */ + +/** + * \struct IPAFrameContext::agc + * \brief Context for the Automatic Gain Control algorithm + * + * The exposure and gain determined are expected to be applied to the sensor + * at the earliest opportunity. + * + * \var IPAFrameContext::agc::exposure + * \brief Exposure time expressed as a number of lines + * + * \var IPAFrameContext::agc::gain + * \brief Analogue gain multiplier + * + * The gain should be adapted to the sensor specific gain code before applying. + */ + +/** + * \struct IPAFrameContext::awb + * \brief Context for the Automatic White Balance algorithm + * + * \struct IPAFrameContext::awb::gains + * \brief White balance gains + * + * \var IPAFrameContext::awb::gains::red + * \brief White balance gain for R channel + * + * \var IPAFrameContext::awb::gains::green + * \brief White balance gain for G channel + * + * \var IPAFrameContext::awb::gains::blue + * \brief White balance gain for B channel + */ + +/** + * \struct IPAFrameContext::toneMapping + * \brief Context for ToneMapping and Gamma control + * + * \var IPAFrameContext::toneMapping::gamma + * \brief Gamma value for the LUT + * + * \var IPAFrameContext::toneMapping::gammaCorrection + * \brief Per-pixel tone mapping implemented as a LUT + * + * The LUT structure is defined by the IPU3 kernel interface. See + * struct ipu3_uapi_gamma_corr_lut for further details. + */ + +} /* namespace libcamera::ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 4c0fff00..a10fdd4a 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -37,130 +37,6 @@ #include "algorithms/tone_mapping.h" #include "libipa/camera_sensor_helper.h" -/** - * \file ipa_context.h - * \brief Context and state information shared between the algorithms - */ - -/** - * \struct IPASessionConfiguration - * \brief Session configuration for the IPA module - * - * The session configuration contains all IPA configuration parameters that - * remain constant during the capture session, from IPA module start to stop. - * It is typically set during the configure() operation of the IPA module, but - * may also be updated in the start() operation. - */ - -/** - * \struct IPAFrameContext - * \brief Per-frame context for algorithms - * - * The frame context stores data specific to a single frame processed by the - * IPA. Each frame processed by the IPA has a context associated with it, - * accessible through the IPAContext structure. - * - * \todo Detail how to access contexts for a particular frame - * - * Each of the fields in the frame context belongs to either a specific - * algorithm, or to the top-level IPA module. A field may be read by any - * algorithm, but should only be written by its owner. - */ - -/** - * \struct IPAContext - * \brief Global IPA context data shared between all algorithms - * - * \var IPAContext::configuration - * \brief The IPA session configuration, immutable during the session - * - * \var IPAContext::frameContext - * \brief The frame context for the frame being processed - * - * \todo While the frame context is supposed to be per-frame, this - * single frame context stores data related to both the current frame - * and the previous frames, with fields being updated as the algorithms - * are run. This needs to be turned into real per-frame data storage. - */ - -/** - * \struct IPASessionConfiguration::grid - * \brief Grid configuration of the IPA - * - * \var IPASessionConfiguration::grid::bdsGrid - * \brief Bayer Down Scaler grid plane config used by the kernel - * - * \var IPASessionConfiguration::grid::bdsOutputSize - * \brief BDS output size configured by the pipeline handler - * - * \var IPASessionConfiguration::grid::stride - * \brief Number of cells on one line including the ImgU padding - */ - -/** - * \struct IPASessionConfiguration::agc - * \brief AGC parameters configuration of the IPA - * - * \var IPASessionConfiguration::agc::minShutterSpeed - * \brief Minimum shutter speed supported with the configured sensor - * - * \var IPASessionConfiguration::grid::maxShutterSpeed - * \brief Maximum shutter speed supported with the configured sensor - * - * \var IPASessionConfiguration::grid::minAnalogueGain - * \brief Minimum analogue gain supported with the configured sensor - * - * \var IPASessionConfiguration::grid::maxAnalogueGain - * \brief Maximum analogue gain supported with the configured sensor - */ - -/** - * \struct IPAFrameContext::agc - * \brief Context for the Automatic Gain Control algorithm - * - * The exposure and gain determined are expected to be applied to the sensor - * at the earliest opportunity. - * - * \var IPAFrameContext::agc::exposure - * \brief Exposure time expressed as a number of lines - * - * \var IPAFrameContext::agc::gain - * \brief Analogue gain multiplier - * - * The gain should be adapted to the sensor specific gain code before applying. - */ - -/** - * \struct IPAFrameContext::awb - * \brief Context for the Automatic White Balance algorithm - * - * \struct IPAFrameContext::awb::gains - * \brief White balance gains - * - * \var IPAFrameContext::awb::gains::red - * \brief White balance gain for R channel - * - * \var IPAFrameContext::awb::gains::green - * \brief White balance gain for G channel - * - * \var IPAFrameContext::awb::gains::blue - * \brief White balance gain for B channel - */ - -/** - * \struct IPAFrameContext::toneMapping - * \brief Context for ToneMapping and Gamma control - * - * \var IPAFrameContext::toneMapping::gamma - * \brief Gamma value for the LUT - * - * \var IPAFrameContext::toneMapping::gammaCorrection - * \brief Per-pixel tone mapping implemented as a LUT - * - * The LUT structure is defined by the IPU3 kernel interface. See - * struct ipu3_uapi_gamma_corr_lut for further details. - */ - /* Minimum grid width, expressed as a number of cells */ static constexpr uint32_t kMinGridWidth = 16; /* Maximum grid width, expressed as a number of cells */ diff --git a/src/ipa/ipu3/meson.build b/src/ipa/ipu3/meson.build index 39320980..3194111a 100644 --- a/src/ipa/ipu3/meson.build +++ b/src/ipa/ipu3/meson.build @@ -5,6 +5,7 @@ subdir('algorithms') ipa_name = 'ipa_ipu3' ipu3_ipa_sources = files([ + 'ipa_context.cpp', 'ipu3.cpp', ]) From patchwork Tue Oct 26 09:55:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14327 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7B1F9BDB1C for ; Tue, 26 Oct 2021 09:56:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 313FE6487C; Tue, 26 Oct 2021 11:56:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="iHlZr0ki"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C26FE6488D for ; Tue, 26 Oct 2021 11:55:47 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7614DE79; Tue, 26 Oct 2021 11:55:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242147; bh=u21P/Y1juNxsxjeUGsEQxPyWIzNNgBOReAkBAh6+UBA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iHlZr0kiBaUKbhW1sM6u9iJwOVtNdeCBMthnMiD+0RfYxnECjV1tHmaaImABkt0wQ A6VSr0PYfyloiH4okkZlmcQDtKJM0jjFXVLO9Ty355CYxSErWvIuj6hbDnK2Mzhe6t vM4ccdx5B9JaB1jkFXHznhMCpcY1A0C/UUbp2CEU= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:31 +0200 Message-Id: <20211026095534.90348-17-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 16/19] ipa: ipu3: Fix badly documented context variables X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Doxygen warns us because the structures are referenced as \struct while they should be \var. Fix it. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipa_context.cpp | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 3e154e60..8917ba54 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -56,7 +56,7 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPASessionConfiguration::grid + * \var IPASessionConfiguration::grid * \brief Grid configuration of the IPA * * \var IPASessionConfiguration::grid::bdsGrid @@ -70,7 +70,7 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPASessionConfiguration::agc + * \var IPASessionConfiguration::agc * \brief AGC parameters configuration of the IPA * * \var IPASessionConfiguration::agc::minShutterSpeed @@ -87,7 +87,7 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPAFrameContext::agc + * \var IPAFrameContext::agc * \brief Context for the Automatic Gain Control algorithm * * The exposure and gain determined are expected to be applied to the sensor @@ -103,7 +103,7 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPAFrameContext::awb + * \var IPAFrameContext::awb * \brief Context for the Automatic White Balance algorithm * * \struct IPAFrameContext::awb::gains @@ -120,7 +120,7 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPAFrameContext::toneMapping + * \var IPAFrameContext::toneMapping * \brief Context for ToneMapping and Gamma control * * \var IPAFrameContext::toneMapping::gamma From patchwork Tue Oct 26 09:55:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14328 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 14425BDB1C for ; Tue, 26 Oct 2021 09:56:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B9D7164895; Tue, 26 Oct 2021 11:56:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="lSbx0pz1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EED2B6488E for ; Tue, 26 Oct 2021 11:55:47 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A88FE3F0; Tue, 26 Oct 2021 11:55:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242147; bh=emAFdu0cmjby8wwzoMgtn6UQmp+hbOx+RZtA7MKwzQA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lSbx0pz1JHRDWzQsMRznwkvyyrQvxPrt+xIXTXFTHGfI8oH/tMP7jaybhqyt2+o67 LWznD47+ZaFglczGnddNaVFehPMbrw/DIW4dCRqx3QTZ0/2s9Nc6hsRFB6XIKZUGpi xssFLuKjBD+4tk6FIr6YfnmTjLNmqLx2OwOanPFw= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:32 +0200 Message-Id: <20211026095534.90348-18-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 17/19] ipa: ipu3: Implement an empty stop() function X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" While the stop() function does not currently perform any action, it forms part of the IPA interface and is a public function in the class. Promote it to a full (but basic) function implementation and begin the documentation accordingly so that there is an appropriate stub to perform stop operations if they come up. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Umang Jain --- src/ipa/ipu3/ipu3.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index a10fdd4a..5c51607d 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -135,7 +135,7 @@ public: ControlInfoMap *ipaControls) override; int start() override; - void stop() override {} + void stop() override; int configure(const IPAConfigInfo &configInfo, ControlInfoMap *ipaControls) override; @@ -323,6 +323,13 @@ int IPAIPU3::start() return 0; } +/** + * \brief Ensure that all processing has completed + */ +void IPAIPU3::stop() +{ +} + /** * \brief Calculate a grid for the AWB statistics * From patchwork Tue Oct 26 09:55:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14329 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BDF31BDB1C for ; Tue, 26 Oct 2021 09:56:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 617876488E; Tue, 26 Oct 2021 11:56:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BzKYTFcf"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3431A6487D for ; Tue, 26 Oct 2021 11:55:48 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D6880DBF; Tue, 26 Oct 2021 11:55:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242147; bh=By4NbGw62IO9TKAyl1skF93xNPfuQr76B937kmz6LfU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BzKYTFcffvfLtQGLDuL9NeLkqa5BXjYuiPvaWA0hoW9WEq8cpr3GeCXejnvwGErpV zt0hvgd3jV7n/tyn2nxLblsz8+XQ+vZJRotDf8bVbqo5B8R/gMWiXqabKaimoiPIUk MZiOxSQ+g3xAxL/MoLdXUXtcSaFhHWg1pFM1krlI= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:33 +0200 Message-Id: <20211026095534.90348-19-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 18/19] Documentation: Include IPU3 in Doxygen build X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The IPU3 is documented using Doxygen within the code. Include the IPU3 IPA as part of the doxygen build sources. This will ideally be split to its own 'section' of the doxygen output. Signed-off-by: Kieran Bingham Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart --- Documentation/Doxyfile.in | 3 ++- Documentation/meson.build | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/Documentation/Doxyfile.in b/Documentation/Doxyfile.in index 6e627192..28605b8f 100644 --- a/Documentation/Doxyfile.in +++ b/Documentation/Doxyfile.in @@ -788,7 +788,8 @@ WARN_LOGFILE = # spaces. See also FILE_PATTERNS and EXTENSION_MAPPING # Note: If this tag is empty the current directory is searched. -INPUT = "@TOP_SRCDIR@/include/libcamera" \ +INPUT = "@TOP_SRCDIR@/src/ipa/ipu3" \ + "@TOP_SRCDIR@/include/libcamera" \ "@TOP_SRCDIR@/src/ipa/libipa" \ "@TOP_SRCDIR@/src/libcamera" \ "@TOP_BUILDDIR@/include/libcamera" \ diff --git a/Documentation/meson.build b/Documentation/meson.build index c4bd3c7f..df36a808 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -22,6 +22,7 @@ if doxygen.found() and dot.found() custom_target('doxygen', input : [ doxyfile, + ipu3_ipa_sources, libcamera_base_headers, libcamera_base_sources, libcamera_internal_headers, From patchwork Tue Oct 26 09:55:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14330 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 72A6CBDB1C for ; Tue, 26 Oct 2021 09:56:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1FC156487F; Tue, 26 Oct 2021 11:56:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="OyK7F4MR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6261064878 for ; Tue, 26 Oct 2021 11:55:48 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:dce3:eb54:18d7:6f3d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1438C3F0; Tue, 26 Oct 2021 11:55:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1635242148; bh=y29dMe+Ji0gHcapQLIvTgtYyTE2Wwu/NRepCtLq7ANM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OyK7F4MRl+jtTPLTm5KkGXnMTz8C/tCqQbjO+jc9A1Kl9SqSUg0VYkD42LBmo/M+h G5TfFTHCfy7QcPEB2sGQ8EsE6zhMevSP0aRppfl8W7nYY9OIBB/nQx1vuL2oZYMbmJ 5gwFj49uHZH8TWi7RXab0sZafHF2xCe+3gqDBfh8= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Oct 2021 11:55:34 +0200 Message-Id: <20211026095534.90348-20-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> References: <20211026095534.90348-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 19/19] ipa: ipu3: ipa_context: Fix doxygen references X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The IPAFrameContext uses unamed structures to group items. Doxygen can not reference the member variables of the container structures through the variable names, and expects the structure type name. As this is not given, the structure variables are referenced from the parent structure. Signed-off-by: Kieran Bingham [JMH: Fix doxygen variable usage] Signed-off-by: Jean-Michel Hautbois Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipa_context.cpp | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 8917ba54..68e2f4aa 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -59,13 +59,13 @@ namespace libcamera::ipa::ipu3 { * \var IPASessionConfiguration::grid * \brief Grid configuration of the IPA * - * \var IPASessionConfiguration::grid::bdsGrid + * \var IPASessionConfiguration::grid.bdsGrid * \brief Bayer Down Scaler grid plane config used by the kernel * - * \var IPASessionConfiguration::grid::bdsOutputSize + * \var IPASessionConfiguration::grid.bdsOutputSize * \brief BDS output size configured by the pipeline handler * - * \var IPASessionConfiguration::grid::stride + * \var IPASessionConfiguration::grid.stride * \brief Number of cells on one line including the ImgU padding */ @@ -73,16 +73,16 @@ namespace libcamera::ipa::ipu3 { * \var IPASessionConfiguration::agc * \brief AGC parameters configuration of the IPA * - * \var IPASessionConfiguration::agc::minShutterSpeed + * \var IPASessionConfiguration::agc.minShutterSpeed * \brief Minimum shutter speed supported with the configured sensor * - * \var IPASessionConfiguration::grid::maxShutterSpeed + * \var IPASessionConfiguration::grid.maxShutterSpeed * \brief Maximum shutter speed supported with the configured sensor * - * \var IPASessionConfiguration::grid::minAnalogueGain + * \var IPASessionConfiguration::grid.minAnalogueGain * \brief Minimum analogue gain supported with the configured sensor * - * \var IPASessionConfiguration::grid::maxAnalogueGain + * \var IPASessionConfiguration::grid.maxAnalogueGain * \brief Maximum analogue gain supported with the configured sensor */ @@ -93,10 +93,10 @@ namespace libcamera::ipa::ipu3 { * The exposure and gain determined are expected to be applied to the sensor * at the earliest opportunity. * - * \var IPAFrameContext::agc::exposure + * \var IPAFrameContext::agc.exposure * \brief Exposure time expressed as a number of lines * - * \var IPAFrameContext::agc::gain + * \var IPAFrameContext::agc.gain * \brief Analogue gain multiplier * * The gain should be adapted to the sensor specific gain code before applying. @@ -106,16 +106,16 @@ namespace libcamera::ipa::ipu3 { * \var IPAFrameContext::awb * \brief Context for the Automatic White Balance algorithm * - * \struct IPAFrameContext::awb::gains + * \struct IPAFrameContext::awb.gains * \brief White balance gains * - * \var IPAFrameContext::awb::gains::red + * \var IPAFrameContext::awb.gains.red * \brief White balance gain for R channel * - * \var IPAFrameContext::awb::gains::green + * \var IPAFrameContext::awb.gains.green * \brief White balance gain for G channel * - * \var IPAFrameContext::awb::gains::blue + * \var IPAFrameContext::awb.gains.blue * \brief White balance gain for B channel */ @@ -123,10 +123,10 @@ namespace libcamera::ipa::ipu3 { * \var IPAFrameContext::toneMapping * \brief Context for ToneMapping and Gamma control * - * \var IPAFrameContext::toneMapping::gamma + * \var IPAFrameContext::toneMapping.gamma * \brief Gamma value for the LUT * - * \var IPAFrameContext::toneMapping::gammaCorrection + * \var IPAFrameContext::toneMapping.gammaCorrection * \brief Per-pixel tone mapping implemented as a LUT * * The LUT structure is defined by the IPU3 kernel interface. See