From patchwork Thu Sep 30 09:54:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13997 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8FD7DC3243 for ; Thu, 30 Sep 2021 09:55:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C3126691B0; Thu, 30 Sep 2021 11:55:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="af9Tq6aM"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 99387691A3 for ; Thu, 30 Sep 2021 11:55:17 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 32C65B2B; Thu, 30 Sep 2021 11:55:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995717; bh=E2mESPxTMSuWK8k3JEjStbjEpEKFAesMcH1KthhAQQ8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=af9Tq6aMBBPkPybJdDoOQU+igkp1hL0p39xw744Fv9ZYmcHwKeHeTLbmGw7grjUv1 KmSOxoLYXdUlDTA+JUu9Pzo4oBvkzx5LyKBIT5r6GcHgOVGW2BN8pxX5Ishf+2OAaH UkwHyeOJxHwQr4q9Bk4NHO6ja5r6dwIDsCyncj0o= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:54:56 +0200 Message-Id: <20210930095513.76213-2-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 01/18] ipa: ipu3: Document IPAIPU3 class interface X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The IPU3 IPA is maturing to a modular and extensible system capable of handling specific algorithms for the accelleration clusters on the ImgU. Provide a top-level class documentation to provide an overview of the IPA, detailing what events are used and what algorithms are currently supported, as well as the limitations currently imposed. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham --- src/ipa/ipu3/ipu3.cpp | 64 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 7cc0fb89..bf8fc1e8 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -157,6 +157,70 @@ LOG_DEFINE_CATEGORY(IPAIPU3) namespace ipa::ipu3 { +/** + * \brief The IPU3 IPA implementation + * + * The IPU3 Pipeline defines an IPU3 specific interface for communication + * between the PipelineHandler, and the IPA module. + * + * We extend the IPAIPU3Interface to implement our algorithms and handle events + * from the IPU3 PipelineHandler to satisfy requests from the application. + * + * At initialisation time, a CameraSensorHelper is instantiated to support + * camera specific calculations, while the default controls are computed, and + * the algorithms are constructed and placed in an ordered list. + * + * The IPU3 ImgU operates with a grid layout to divide the overall frame into + * rectangular cells of pixels. When the IPA is configured, we determine the + * best grid for the statistics based on the pipeline handler Bayer Down Scaler + * output size. + * + * Two main events are then handled to facilitate the operation of the IPU3 ImgU + * by populating its parameter buffer, and adapting the settings of the sensor + * attached to the IPU3 CIO2 through sensor specific V4L2 controls. + * + * When the event \a EventFillParams occurs we populate the ImgU parameter + * buffer with settings to configure the device in preparation for handling the + * frame queued in the Request. + * + * When the frame has completed processing, the ImgU will generate a statistics + * buffer which is given to the IPA as part of the \a EventStatReady event. At + * this event we run the algorithms to parse the statistics and cache any + * results for the next \a EventFillParams event. + * + * The individual algorithms are split into modular components that are called + * iteratively to allow them to process statistics from the ImgU in a defined + * order. + * + * The current implementation supports three core algorithms: + * - Automatic white balance (AWB) + * - Automatic gain and exposure control (AGC) + * - Tonemapping (Gamma) + * + * AWB is implemented using a Greyworld algorithm, and calculates the red and + * blue gains to apply to generate a neutral grey frame overall. + * + * AGC is handled by calculating a histogram of the green channel to estimate an + * analogue gain and shutter time which will provide a well exposed frame. An + * IIR filter is used to smooth the changes to the sensor to reduce perceivable + * steps. + * + * The Tonemapping algorithm provides a gamma correction table to improve the + * contrast of the scene. + * + * The IPU3 ImgU has further accellerator clusters to support image quality + * improvements through bayer and temporal noise reductions, however those are + * not supported in the current implementation, and will use default settings as + * provided by the kernel driver. + * + * Demosaicing is operating on the default values and could be further optimised + * to provide improved sharpening coefficients, checker artifact removal, and + * false color correction. + * + * Additional image enhancements can be made by providing lens and sensor + * specific tuning to adapt for Black Level compensation (BLC), Lens shading + * correction (SHD) and Color correction (CCM) + */ class IPAIPU3 : public IPAIPU3Interface { public: From patchwork Thu Sep 30 09:54:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13998 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5A62DC3243 for ; Thu, 30 Sep 2021 09:55:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B44CB691C6; Thu, 30 Sep 2021 11:55:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Nl3cTzaG"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C6C47691AA for ; Thu, 30 Sep 2021 11:55:17 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 67FD518BF; Thu, 30 Sep 2021 11:55:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995717; bh=G9QEBDTYt6izYMiDhdHV4jT7heExdaHb0xfMyCTfUPQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Nl3cTzaGLtgPAJY/AIHLKY0wrJmU+Zm71tgVH2Zko5NTj9zsPJ8D/cz1dLfotNwr5 CwpZHwKdQICaNcNGhKG49uO3BiUnPj5pAn7I74m79fKneHq87b50puxMH3zM77YUbZ mDg7/uoHAOrHk5h7Q+DItTY2Zr1iJbV4jdeoGh4E= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:54:57 +0200 Message-Id: <20210930095513.76213-3-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 02/18] ipa: ipu3: Document IPAIPU3::configure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Further extend the documentation for the IPAIPU3::configure operation. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham --- src/ipa/ipu3/ipu3.cpp | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index bf8fc1e8..b4c889f1 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -426,6 +426,18 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) << (int)bdsGrid.height << " << " << (int)bdsGrid.block_height_log2 << ")"; } +/** + * \brief Configure the IPU3 IPA. + * \param[in] configInfo The IPA configuration data, received from the pipeline + * handler + * + * Calculate the best grid for the statistics based on the Pipeline Handler BDS + * output, and parse the minimum and maximum exposure and analogue gain control + * values. + * + * All algorithm modules are called to allow them to prepare the + * \a IPASessionConfiguration structure for the \a IPAContext. + */ int IPAIPU3::configure(const IPAConfigInfo &configInfo) { if (configInfo.sensorControls.empty()) { From patchwork Thu Sep 30 09:54:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13999 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9EFB1C3243 for ; Thu, 30 Sep 2021 09:55:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1A261691AC; Thu, 30 Sep 2021 11:55:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="SehyQ1Nw"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EE5DD69189 for ; Thu, 30 Sep 2021 11:55:17 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9B9E92A8; Thu, 30 Sep 2021 11:55:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995717; bh=yOsB8XT62JX49W8dhh3QMzEYUCAFAnIt6NiUF1eqDns=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SehyQ1Nwc31LLrYFBAlhr6tPdfrAPoL5P52BMmX7ABtUnNmjIHehnO00lEpfyeBgI VohZIfDIGRcxpv+6dJPJn2JrtnCxWVP9PGMcsq8ra6TW+h93KgoUYersJz4KGlBGJu 8K7WEoHIO5ZYcTpmWw3NUNT/JQk78ybZVuJVPYwU= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:54:58 +0200 Message-Id: <20210930095513.76213-4-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 03/18] ipa: ipu3: Document the IPAIPU3 class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Clarify the roles and interactions between the pipeline handler events and the algorithm calls by documenting all the remaining functions of the class. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham --- src/ipa/ipu3/ipu3.cpp | 65 +++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 62 insertions(+), 3 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index b4c889f1..147b40ab 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -274,7 +274,7 @@ private: }; /** - * Initialize the IPA module and its controls. + * \brief Initialize the IPA module and its controls. * * This function receives the camera sensor information from the pipeline * handler, computes the limits of the controls it handles and returns @@ -350,8 +350,15 @@ int IPAIPU3::init(const IPASettings &settings, return 0; } +/** + * \brief Perform any processing required before the first frame. + */ int IPAIPU3::start() { + /* + * Set the sensors V4L2 controls before the first frame to ensure that + * we have an expected and known configuration from the start. + */ setControls(0); return 0; @@ -491,6 +498,10 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) return 0; } +/** + * \brief Map the parameters and stats buffers allocated in the Pipeline Handler + * \param[out] buffers A vector with param and stat buffers + */ void IPAIPU3::mapBuffers(const std::vector &buffers) { for (const IPABuffer &buffer : buffers) { @@ -500,6 +511,10 @@ void IPAIPU3::mapBuffers(const std::vector &buffers) } } +/** + * \brief Unmap the parameters and stats buffers + * \param[in] ids A list of buffer ids + */ void IPAIPU3::unmapBuffers(const std::vector &ids) { for (unsigned int id : ids) { @@ -511,6 +526,10 @@ void IPAIPU3::unmapBuffers(const std::vector &ids) } } +/** + * \brief Process events generated by the Pipeline Handler + * \param[in] event An event sent from Pipeline Handler + */ void IPAIPU3::processEvent(const IPU3Event &event) { switch (event.op) { @@ -552,12 +571,26 @@ void IPAIPU3::processEvent(const IPU3Event &event) } } +/** + * \brief Process a control list for a request from the application + * + * Parse the request to handle any IPA managed controls that were set from the + * application such as manual sensor settings. + */ void IPAIPU3::processControls([[maybe_unused]] unsigned int frame, [[maybe_unused]] const ControlList &controls) { /* \todo Start processing for 'frame' based on 'controls'. */ } +/** + * \brief Fill the ImgU parameter buffer for the next frame + * \param[in] frame The current frame number + * \param[inout] params the parameter buffer to update + * + * Algorithms are expected to fill the IPU3 parameter buffer for the next + * frame given their most recent processing of the ImgU statistics. + */ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) { /* @@ -580,6 +613,16 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) queueFrameAction.emit(frame, op); } +/** + * \brief Process the statistics generated by the ImgU + * \param[in] frame The current frame number + * \param[in] frameTimestamp The current frame timestamp + * \param[in] stats The IPU3 statistics and ISP results + * + * Parse the most recently processed image statistics from the ImgU. The + * statistics are passed to each algorithm module to run their calculations and + * update their state accordingly. + */ void IPAIPU3::parseStatistics(unsigned int frame, [[maybe_unused]] int64_t frameTimestamp, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) @@ -607,6 +650,13 @@ void IPAIPU3::parseStatistics(unsigned int frame, queueFrameAction.emit(frame, op); } +/** + * \brief Handle V4L2 controls for a given \a frame number + * \param[in] frame The frame on which the V4L2 controls should be set + * + * Send the desired sensor control values to the Pipeline handler to request + * that they are applied on the Camera sensor. + */ void IPAIPU3::setControls(unsigned int frame) { IPU3Action op; @@ -625,10 +675,15 @@ void IPAIPU3::setControls(unsigned int frame) } /* namespace ipa::ipu3 */ -/* +/** * External IPA module interface + * + * The IPAModuleInfo is required to match an IPA module construction against the + * intented pipeline handler with the module. The API and Pipeline handler + * versions must match the corresponding IPA interface and pipeline handler. + * + * \sa struct IPAModuleInfo */ - extern "C" { const struct IPAModuleInfo ipaModuleInfo = { IPA_MODULE_API_VERSION, @@ -637,6 +692,10 @@ const struct IPAModuleInfo ipaModuleInfo = { "ipu3", }; +/** + * When matched against with a pipeline handler, the IPAManager will construct + * an IPA instance for each associated Camera. + */ IPAInterface *ipaCreate() { return new ipa::ipu3::IPAIPU3(); From patchwork Thu Sep 30 09:54:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14000 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E858BC3243 for ; Thu, 30 Sep 2021 09:55:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C8DB7691B3; Thu, 30 Sep 2021 11:55:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="EuSP3/MV"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2BD9B691AC for ; Thu, 30 Sep 2021 11:55:18 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CEFEBB2B; Thu, 30 Sep 2021 11:55:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995717; bh=t80ah5Wsd63136u+dGEl+x6CmOw5/4+94EtMJhac4UQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EuSP3/MVgE4hWH4h5gsAGLoo8gc/j/kV3yZEp0rwhBrn235M9DgjKIFtDrTHEc/jS glVzZrfqJZuy+XxRmnKyAgZUYICLD3QUS7bmIBBRdq+cwkK8+IRSdibCjUR1mKRfcC ho5OSJZHOBlRdhQpgQzLUBZMzHCl5GXVAAgK6CfE= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:54:59 +0200 Message-Id: <20210930095513.76213-5-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 04/18] ipa: ipu3: Explicitly use the statistics parameter X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The stats pointer is marked as [[maybe_unused]]. This is a leftover from a previous commit which was here to keep the compatibility while transitioning to the new iterative algorithms. Remove this attribute to make it explicit that stats are really used to feed the algorithms. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham --- src/ipa/ipu3/ipu3.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 147b40ab..a27f5e98 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -625,7 +625,7 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) */ void IPAIPU3::parseStatistics(unsigned int frame, [[maybe_unused]] int64_t frameTimestamp, - [[maybe_unused]] const ipu3_uapi_stats_3a *stats) + const ipu3_uapi_stats_3a *stats) { ControlList ctrls(controls::controls); From patchwork Thu Sep 30 09:55:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14001 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D495FC3243 for ; Thu, 30 Sep 2021 09:55:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AA969691CE; Thu, 30 Sep 2021 11:55:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="qcyVB/1l"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5AE92691AE for ; Thu, 30 Sep 2021 11:55:18 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0EAFC18BF; Thu, 30 Sep 2021 11:55:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995718; bh=dIv8+hbMTv6KVHFopv7lWNYMQQfmHJhPArfKDw8tIBo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qcyVB/1lmbq/a9kG36ba7v3r312rDvufUIWDYtwEuKFE11xHJSSX402D9tPQ42DjS Ah6wbKJqUUWMFPlviQ+/ufY/Cd4uiViO4Sg3Ry3s+IW3+EMGwIhLDju7jfq14pZAKK gYXnf7QEw0v1tNF1tFMz19GEosF1OjzcJwKhQzC8= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:55:00 +0200 Message-Id: <20210930095513.76213-6-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 05/18] ipa: ipu3: awb: Add AWB class documentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AWB algorithm is based on the Grey world algorithm and uses the statistics generated by the ImgU for that. Explain how it uses those, and reference the original algorithm at the same time. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/awb.cpp | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 4558f85a..f9e8cffc 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -11,6 +11,10 @@ #include +/** + * \file awb.h + */ + namespace libcamera { namespace ipa::ipu3::algorithms { @@ -129,6 +133,35 @@ static const struct ipu3_uapi_ccm_mat_config imguCssCcmDefault = { 0, 0, 8191, 0 }; +/** + * \class Awb + * \brief A Grey world white balance correction algorithm + * + * The Grey World algorithm assumes that the scene, in average, is neutral grey. + * Reference: Lam, Edmund & Fung, George. (2008). Automatic White Balancing in + * Digital Photography. 10.1201/9781420054538.ch10. + * + * The IPU3 generates statistics from the Bayer Down Scaler output into a grid + * defined in the ipu3_uapi_awb_config_s structure. + * + * For example, when the BDS outputs a frame of 2592x1944, the grid may be + * configured to 80x30 cells each with a size of 32x64 pixels. + * We then have an average of 2048 R, G and B pixels per cell. + * + * The AWB algorithm uses a fixed grid size of kAwbStatsSizeX x kAwbStatsSizeY. + * Each of this new grid cell will be called a zone. + * + * Before calculating the gains, we will convert the statistics from the BDS + * grid to an internal grid configuration in generateAwbStats. + * As part of converting the statistics to an internal grid, the saturation + * flag from the originating grid cell is used to decide if the zone contains + * saturated pixels or not, making the zone relevant or not. + * A saturated zone will be excluded from the calculation. + * + * The Grey World algorithm will then estimate the red and blue gains to apply, and + * store the results in the metadata. + */ + Awb::Awb() : Algorithm() { From patchwork Thu Sep 30 09:55:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14002 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9E2C9C3243 for ; Thu, 30 Sep 2021 09:55:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8828E691C8; Thu, 30 Sep 2021 11:55:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="YUJblwbb"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8212A691A3 for ; Thu, 30 Sep 2021 11:55:18 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3CF162A8; Thu, 30 Sep 2021 11:55:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995718; bh=pzB+SmLd/iPDdUY9sZq09VPRGurVnZ0DZvxabPCWeEc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YUJblwbb0SEOeCtqnOPg+jZGD8mvVdKV3By88VFuvMr998Quu0VUXsmJY3DgVoRIG ejso4LvopnHQ9VxXAP8Ejg4/WfJ8UzqQK8YIl9NpQQ9spOQypkLH+VGwIVCL0Wqimx V5/PqGqvXPb0c1RZ6BcIP6SLIUL90ukmZ64PDcDw= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:55:01 +0200 Message-Id: <20210930095513.76213-7-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 06/18] ipa: ipu3: agc: Document AGC mean-based algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AGC class was not documented while developing. Extend that to reference the origins of the implementation, and improve the descriptions on how the algorithm operates internally. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham --- src/ipa/ipu3/algorithms/agc.cpp | 102 ++++++++++++++++++++++++++++---- src/ipa/ipu3/algorithms/agc.h | 2 +- 2 files changed, 90 insertions(+), 14 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 44008632..2ddbf438 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -2,7 +2,7 @@ /* * Copyright (C) 2021, Ideas On Board * - * ipu3_agc.cpp - AGC/AEC control algorithm + * ipu3_agc.cpp - AGC/AEC mean-based control algorithm */ #include "agc.h" @@ -17,27 +17,49 @@ #include "libipa/histogram.h" +/** + * \file agc.h + */ + namespace libcamera { using namespace std::literals::chrono_literals; namespace ipa::ipu3::algorithms { +/** + * \class Agc + * \brief A mean-based auto-exposure algorithm + * + * This algorithm calculates a shutter time and a gain so that the average value + * of the green channel of the brightest 2% of pixels approaches 0.5. The AWB + * gains are not used here, and all cells in the grid have the same weight, like + * an average-metering case. In this metering mode, the camera uses light + * information from the entire scene and creates an average for the final + * exposure setting, giving no weighting to any particular portion of the + * metered area. + * + * Reference: Battiato, Messina & Castorina. (2008). Exposure + * Correction for Imaging Devices: An Overview. 10.1201/9781420054538.ch12. + */ + LOG_DEFINE_CATEGORY(IPU3Agc) /* Number of frames to wait before calculating stats on minimum exposure */ static constexpr uint32_t kInitialFrameMinAECount = 4; -/* Number of frames to wait between new gain/exposure estimations */ +/* Number of frames to wait between new gain/shutter time estimations */ static constexpr uint32_t kFrameSkipCount = 6; -/* Maximum ISO value for analogue gain */ -static constexpr uint32_t kMinISO = 100; -static constexpr uint32_t kMaxISO = 1500; - -/* Maximum analogue gain value - * \todo grab it from a camera helper */ -static constexpr uint32_t kMinGain = kMinISO / 100; -static constexpr uint32_t kMaxGain = kMaxISO / 100; +/* + * Minimum analogue gain value + * \todo grab it from a camera helper + */ +static constexpr uint32_t kMinGain = 1; +/* + * Maximum analogue gain value + * \todo grab it from a camera helper + */ +static constexpr uint32_t kMaxGain = 15; /* \todo use calculated value based on sensor */ static constexpr uint32_t kMinExposure = 1; @@ -45,6 +67,7 @@ static constexpr uint32_t kMaxExposure = 1976; /* Histogram constants */ static constexpr uint32_t knumHistogramBins = 256; +/* Target value to reach for the top 2% of the histogram */ static constexpr double kEvGainTarget = 0.5; Agc::Agc() @@ -54,10 +77,18 @@ Agc::Agc() { } +/** + * \brief Configure the AGC given a configInfo + * \param[in] context The shared IPA context + * \param[in] configInfo The IPA configuration data + * + * \return 0 + */ int Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) { stride_ = context.configuration.grid.stride; + /* \todo use the configInfo fields and IPAContext to store the limits */ lineDuration_ = configInfo.sensorInfo.lineLength * 1.0s / configInfo.sensorInfo.pixelRate; maxExposureTime_ = kMaxExposure * lineDuration_; @@ -65,9 +96,15 @@ int Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) return 0; } +/** + * \brief Estimate the mean quantile of the top 2% of the histogram + * \param[in] stats The statistics computed by the ImgU + * \param[in] grid The grid used to store the statistics in the IPU3 + */ void Agc::processBrightness(const ipu3_uapi_stats_3a *stats, const ipu3_uapi_grid_config &grid) { + /* Initialise the histogram array */ uint32_t hist[knumHistogramBins] = { 0 }; for (unsigned int cellY = 0; cellY < grid.height; cellY++) { @@ -83,6 +120,11 @@ void Agc::processBrightness(const ipu3_uapi_stats_3a *stats, if (cell->sat_ratio == 0) { uint8_t gr = cell->Gr_avg; uint8_t gb = cell->Gb_avg; + /* + * Store the average green value to estimate the + * brightness. Even the over exposed pixels are + * taken into account. + */ hist[(gr + gb) / 2]++; } } @@ -92,18 +134,24 @@ void Agc::processBrightness(const ipu3_uapi_stats_3a *stats, iqMean_ = Histogram(Span(hist)).interQuantileMean(0.98, 1.0); } +/** + * \brief Apply a filter on the exposure value to limit the speed of changes + */ void Agc::filterExposure() { double speed = 0.2; if (prevExposure_ == 0s) { - /* DG stands for digital gain.*/ + /* + * DG stands for digital gain, which is always 1.0 for now as it + * is not implemented right now. + */ prevExposure_ = currentExposure_; prevExposureNoDg_ = currentExposureNoDg_; } else { /* * If we are close to the desired result, go faster to avoid making * multiple micro-adjustments. - * \ todo: Make this customisable? + * \todo: Make this customisable? */ if (prevExposure_ < 1.2 * currentExposure_ && prevExposure_ > 0.8 * currentExposure_) @@ -114,10 +162,12 @@ void Agc::filterExposure() prevExposureNoDg_ = speed * currentExposureNoDg_ + prevExposureNoDg_ * (1.0 - speed); } + /* * We can't let the no_dg exposure deviate too far below the * total exposure, as there might not be enough digital gain available * in the ISP to hide it (which will cause nasty oscillation). + * \todo implement digital gain setting */ double fastReduceThreshold = 0.4; if (prevExposureNoDg_ < @@ -126,6 +176,11 @@ void Agc::filterExposure() LOG(IPU3Agc, Debug) << "After filtering, total_exposure " << prevExposure_; } +/** + * \brief Estimate the new exposure and gain values + * \param[inout] exposure The exposure value reference as a number of lines + * \param[inout] gain The gain reference to be updated + */ void Agc::lockExposureGain(uint32_t &exposure, double &gain) { /* Algorithm initialization should wait for first valid frames */ @@ -138,22 +193,33 @@ void Agc::lockExposureGain(uint32_t &exposure, double &gain) if (std::abs(iqMean_ - kEvGainTarget * knumHistogramBins) <= 1) { LOG(IPU3Agc, Debug) << "!!! Good exposure with iqMean = " << iqMean_; } else { + /* Estimate the gain needed to have the proportion wanted */ double newGain = kEvGainTarget * knumHistogramBins / iqMean_; /* extracted from Rpi::Agc::computeTargetExposure */ + /* Calculate the shutter time in seconds */ libcamera::utils::Duration currentShutter = exposure * lineDuration_; + + /* + * Estimate the current exposure value for the scene as shutter + * time multiplicated by the analogue gain. + */ currentExposureNoDg_ = currentShutter * gain; LOG(IPU3Agc, Debug) << "Actual total exposure " << currentExposureNoDg_ << " Shutter speed " << currentShutter << " Gain " << gain; + + /* Apply the gain calculated to the current exposure value */ currentExposure_ = currentExposureNoDg_ * newGain; + + /* Clamp the exposure value to the min and max authorized */ libcamera::utils::Duration maxTotalExposure = maxExposureTime_ * kMaxGain; currentExposure_ = std::min(currentExposure_, maxTotalExposure); LOG(IPU3Agc, Debug) << "Target total exposure " << currentExposure_; - /* \todo: estimate if we need to desaturate */ filterExposure(); + /* Divide the exposure value as new exposure and gain values */ libcamera::utils::Duration newExposure = 0.0s; if (currentShutter < maxExposureTime_) { exposure = std::clamp(static_cast(exposure * currentExposure_ / currentExposureNoDg_), kMinExposure, kMaxExposure); @@ -169,10 +235,20 @@ void Agc::lockExposureGain(uint32_t &exposure, double &gain) lastFrame_ = frameCount_; } +/** + * \brief Process IPU3 statistics, and run AGC operations + * \param[in] context The shared IPA context + * \param[in] stats The IPU3 statistics and ISP results + * + * Identify the current image brightness, and use that to estimate the optimal + * new exposure and gain for the scene. + */ void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { + /* Get the latest exposure and gain applied */ uint32_t &exposure = context.frameContext.agc.exposure; double &gain = context.frameContext.agc.gain; + processBrightness(stats, context.configuration.grid.bdsGrid); lockExposureGain(exposure, gain); frameCount_++; diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 64b71c65..c81ed6e7 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -2,7 +2,7 @@ /* * Copyright (C) 2021, Ideas On Board * - * agc.h - IPU3 AGC/AEC control algorithm + * agc.h - IPU3 AGC/AEC mean-based control algorithm */ #ifndef __LIBCAMERA_IPU3_ALGORITHMS_AGC_H__ #define __LIBCAMERA_IPU3_ALGORITHMS_AGC_H__ From patchwork Thu Sep 30 09:55:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14003 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4E3B9C3243 for ; Thu, 30 Sep 2021 09:55:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 69366691C2; Thu, 30 Sep 2021 11:55:27 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FLLzi0Dk"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BB8DC691AF for ; Thu, 30 Sep 2021 11:55:18 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 71924B2B; Thu, 30 Sep 2021 11:55:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995718; bh=J78eq31SfqKEyjJaVAd2zPmJfXOgqBM/lhSqzO2a36E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FLLzi0Dk1+TbW+c81PClC1jOlFkXcUNeUEGnMlzwRLeG6bZfvPSipGGclh6lIzs5g vC2/juncGkWE4OVy5OI323V7QzTjrRYzwOGRz/DlElkMFY09IphtuUMgBKE61ejBLu J3/DJHQh1BRxMcbF0IcsJWlqI7DjeqWSgjj5taLU= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:55:02 +0200 Message-Id: <20210930095513.76213-8-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 07/18] ipa: ipu3: tonemapping: Generate the LUT only on gamma change X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Tonemapping algorithm calculates the gamma curve for every frame, regardless of whether the gamma value has changed or not. This issue is exasperated as we currently hardcode the gamma to a single value. Optimise the implementation to only recalculate the look up table when the gamma setting is changed, and store the gamma setting of the LUT curve as part of the IPA context. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham --- src/ipa/ipu3/algorithms/tone_mapping.cpp | 5 +++++ src/ipa/ipu3/ipa_context.h | 1 + src/ipa/ipu3/ipu3.cpp | 3 +++ 3 files changed, 9 insertions(+) diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 3af96261..40337f9d 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -43,6 +43,9 @@ void ToneMapping::process([[maybe_unused]] IPAContext &context, */ gamma_ = 1.1; + if (context.frameContext.toneMapping.gamma == gamma_) + return; + struct ipu3_uapi_gamma_corr_lut &lut = context.frameContext.toneMapping.gammaCorrection; @@ -53,6 +56,8 @@ void ToneMapping::process([[maybe_unused]] IPAContext &context, /* The output value is expressed on 13 bits. */ lut.lut[i] = gamma * 8191; } + + context.frameContext.toneMapping.gamma = gamma_; } } /* namespace ipa::ipu3::algorithms */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 5bab684c..e50de726 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -39,6 +39,7 @@ struct IPAFrameContext { } awb; struct { + double gamma; struct ipu3_uapi_gamma_corr_lut gammaCorrection; } toneMapping; }; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index a27f5e98..9867c993 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -131,6 +131,9 @@ * \struct IPAFrameContext::toneMapping * \brief Context for ToneMapping and Gamma control * + * \var IPAFrameContext::toneMapping::gamma + * \brief Gamma value for the LUT + * * \var IPAFrameContext::toneMapping::gammaCorrection * \brief Per-pixel tone mapping implemented as a LUT * From patchwork Thu Sep 30 09:55:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14004 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 82BB8C3243 for ; Thu, 30 Sep 2021 09:55:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 54A47691AF; Thu, 30 Sep 2021 11:55:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nbZK1Mq5"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 02E6A691AA for ; Thu, 30 Sep 2021 11:55:19 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A552A2A8; Thu, 30 Sep 2021 11:55:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995718; bh=LFh8BNmwrDEmjBPspaawQKvWN2mk1Rd89f6qorKcmug=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nbZK1Mq5EJmOQNmEv6qALE5V6lthaEsOp8tnCa6r/rCZ8U9wNwtDhcnOGYYA0E54i 4BDPsGmpSS3+r6XX5Oo73ArwQ0AVu70w658bAnizYtAsgbBkZ+hE5GhC6pJMxHvVjp KK/YfZ77rTJ4yC2Oj+cbQ6lmUsYjPUXyLXCzqIss= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:55:03 +0200 Message-Id: <20210930095513.76213-9-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 08/18] ipa: ipu3: tonemapping: Add the documentation for ToneMapping X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Tonemapping algorithm is currently undocumented. Provide an introduction and overview to the implementation as the class definition and document how the algorithm operates in the process and prepare methods. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham --- src/ipa/ipu3/algorithms/tone_mapping.cpp | 35 +++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 40337f9d..15e9ab3f 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -10,15 +10,40 @@ #include #include +/** + * \file tone_mapping.h + */ + namespace libcamera { namespace ipa::ipu3::algorithms { +/** + * \class ToneMapping + * \brief A class to handle tone mapping based on gamma + * + * This algorithm improves the image dynamic using a look-up table which is + * generated based on a gamma parameter. + * + * Gamma values less than one have the effect of compressing the image histogram + * while values over 1 will expand it. + * + * Expanding the histogram has the effect of providing better overall contrast. + */ + ToneMapping::ToneMapping() : gamma_(1.0) { } +/** + * \brief Fill in the parameter structure, and enable gamma control + * \param context The shared IPA context + * \param params The IPU3 parameters + * + * Populate the IPU3 parameter structure with our gamma correction table, and + * enable the gamma control module in the accelerator cluster. + */ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, ipu3_uapi_params *params) { @@ -33,7 +58,15 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, params->acc_param.gamma.gc_ctrl.enable = 1; } -void ToneMapping::process([[maybe_unused]] IPAContext &context, +/** + * \brief Calculate the Gamma curve + * \param context The shared IPA context + * \param stats The IPU3 statistics and ISP results + * + * The gamma correction look up table is generated as an inverse power curve + * from our gamma setting. + */ +void ToneMapping::process(IPAContext &context, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) { /* From patchwork Thu Sep 30 09:55:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14006 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B8BE7C3243 for ; Thu, 30 Sep 2021 09:55:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5E955691BB; Thu, 30 Sep 2021 11:55:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rHju91ut"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 43189691B7 for ; Thu, 30 Sep 2021 11:55:19 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D9A22B2B; Thu, 30 Sep 2021 11:55:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995719; bh=10pMTuaNamcZCYhMPUduPcS3cQZQN+nqiaCIc96BWqM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rHju91ut7ieWoib/BdXCttMKiF/zm0ohDUsi/jruN9sui+mKslL4y47F9CmK+4Fjk AhfK7f8E23UQJMgooMd00xIZlWs1svE1zu7Br6lqAhNFPQ6MEY9hRKSHWlxiXoO/7D 8KcDTAn5eEm/uKO53zgPOAQlrW6HXfM1MN6wRcSU= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:55:04 +0200 Message-Id: <20210930095513.76213-10-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 09/18] Documentation: IPU3 IPA Design guide X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The IPU3 IPA implements the basic 3A using the ImgU ISP. Provide an overview document to describe its operations, and provide a block diagram to help visualise how the components are put together to assist any new developers exploring the code. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain --- v2: - /accelerator cluster/processing block/ (and refactor) - /Pipeline/pipeline/ - /Camera Sensor/camera sensor/ - /CPU accessible/CPU-accessible/ - Remove updated control parameters from IPASessionConfiguration - Expand pre-frame preparation to match post-frame with the event descriptions. - Add Sensor Controls brief - Move to src/ipa/ipu3/ - Lower indentation of the block diagrams (keep under 80chars) - reference mapBuffers() call for passing buffers in - reference unmapBuffers() after stop() --- src/ipa/ipu3/ipu3-ipa-design-guide.rst | 155 +++++++++++++++++++++++++ 1 file changed, 155 insertions(+) create mode 100644 src/ipa/ipu3/ipu3-ipa-design-guide.rst diff --git a/src/ipa/ipu3/ipu3-ipa-design-guide.rst b/src/ipa/ipu3/ipu3-ipa-design-guide.rst new file mode 100644 index 00000000..e5ee508d --- /dev/null +++ b/src/ipa/ipu3/ipu3-ipa-design-guide.rst @@ -0,0 +1,155 @@ +IPU3 IPA Architecture Design and Overview +========================================= + +The IPU3 IPA is built as a modular and extensible framework with an +upper layer to manage the interactions with the pipeline handler, and +the image processing algorithms split to compartmentalise the processing +required for each processing block, making use of the fixed-function +accelerators provided by the ImgU ISP. + +The core IPU3 class is responsible for initialisation and construction +of the algorithm components, processing controls set by the requests +from applications, and managing events from the pipeline handler. + +:: + + ┌───────────────────────────────────────────┐ + │ IPU3 Pipeline Handler │ + │ ┌────────┐ ┌────────┐ ┌────────┐ │ + │ │ │ │ │ │ │ │ + │ │ Sensor ├───►│ CIO2 ├───►│ ImgU ├──► + │ │ │ │ │ │ │ │ + │ └────────┘ └────────┘ └─▲────┬─┘ │ P: Parameter Buffer + │ │P │ │ S: Statistics Buffer + │ │ │S │ + └─┬───┬───┬──────┬────┬────┬────┬─┴────▼─┬──┘ 1: init() + │ │ │ │ ▲ │ ▲ │ ▲ │ ▲ │ 2: configure() + │1 │2 │3 │4│ │4│ │4│ │4│ │5 3: mapBuffers(), start() + ▼ ▼ ▼ ▼ │ ▼ │ ▼ │ ▼ │ ▼ 4: processEvent() + ┌──────────────────┴────┴────┴────┴─────────┐ 5: stop(), unmapBuffers() + │ IPU3 IPA │ + │ ┌───────────────────────┐ │ + │ ┌───────────┐ │ Algorithms │ │ + │ │IPAContext │ │ ┌─────────┐ │ │ + │ │ ┌───────┐ │ │ │ ... │ │ │ + │ │ │ │ │ │ ┌─┴───────┐ │ │ │ + │ │ │ SC │ │ │ │ Tonemap ├─┘ │ │ + │ │ │ │ ◄───► ┌─┴───────┐ │ │ │ + │ │ ├───────┤ │ │ │ AWB ├─┘ │ │ + │ │ │ │ │ │ ┌─┴───────┐ │ │ │ + │ │ │ FC │ │ │ │ AGC ├─┘ │ │ + │ │ │ │ │ │ │ │ │ │ + │ │ └───────┘ │ │ └─────────┘ │ │ + │ └───────────┘ └───────────────────────┘ │ + └───────────────────────────────────────────┘ + SC: IPASessionConfiguration + FC: IPAFrameContext(s) + +The IPA instance is constructed and initialised at the point a Camera is +created by the IPU3 pipeline handler. The initialisation call provides +details about which camera sensor is being used, and the controls that +it has available, along with their default values and ranges. + +Buffers +~~~~~~~ + +The IPA will have Parameter and Statistics buffers shared with it from +the IPU3 Pipeline handler. These buffers will be passed to the IPA using +the ``mapBuffers()`` call before the ``start()`` operation occurs. + +The IPA will map the buffers into CPU-accessible memory, associated with +a buffer ID, and further events for sending or receiving parameter and +statistics buffers will reference the ID to avoid expensive memory +mapping operations, or the passing of file handles during streaming. + +After the ``stop()`` operation occurs, these buffers will be unmapped +when requested by the pipeline handler using the ``unmapBuffers()`` call +and no further access to the buffers is permitted. + +Context +~~~~~~~ + +Algorithm calls will always have the ``IPAContext`` available to them. +This context comprises of two parts: + +- IPA Session Configuration +- IPA Frame Context + +The session configuration structure ``IPASessionConfiguration`` +represents constant parameters determined before streaming commenced +during ``configure()``. + +The IPA Frame Context provides the storage for algorithms for a single +frame operation. + +The ``IPAFrameContext`` structure may be extended to an array, list, or +queue to store historical state for each frame, allowing algorithms to +obtain and reference results of calculations which are deeply pipelined. +This may only be done if an algorithm needs to know the context that was +applied at the frame the statistics were produced for, rather than the +previous or current frame. + +Presently there is a single ``IPAFrameContext`` without historical data, +and the context is maintained and updated through successive processing +operations. + +Operating +~~~~~~~~~ + +There are three main interactions with the algorithms for the IPU3 IPA +to operate when running: + +- configure() +- processEvent(``EventFillParams``) +- processEvent(``EventStatReady``) + +The configuration phase allows the pipeline-handler to inform the IPA of +the current stream configurations, which is then passed into each +algorithm to provide an opportunity to identify and track state of the +hardware, such as image size or ImgU pipeline configurations. + +Pre-frame preparation +~~~~~~~~~~~~~~~~~~~~~ + +When configured, the IPA is notified by the pipeline handler of the +Camera ``start()`` event, after which incoming requests will be queued +for processing, requiring a parameter buffer (``ipu3_uapi_params``) to +be populated for the ImgU. This is given to the IPA through the +``EventFillParams`` event, and then passed directly to each algorithm +through the ``prepare()`` call allowing the ISP configuration to be +updated for the needs of each component that the algorithm is +responsible for. + +The algorithm should set the use flag (``ipu3_uapi_flags``) for any +structure that it modifies, and it should take care to ensure that any +structure set by a use flag is fully initialised to suitable values. + +The parameter buffer is returned to the pipeline handler through the +``ActionParamFilled`` event, and from there queued to the ImgU along +with a raw frame captured with the CIO2. + +Post-frame completion +~~~~~~~~~~~~~~~~~~~~~ + +When the capture of an image is completed, and successfully processed +through the ImgU, the generated statistics buffer +(``ipu3_uapi_stats_3a``) is given to the IPA through the +``EventStatReady`` event. This provides the IPA with an opportunity to +examine the results of the ISP and run the calculations required by each +algorithm on the new data. The algorithms may require context from the +operations of other algorithms, for example, the AWB might choose to use +a scene brightness determined by the AGC. It is important that the +algorithms are ordered to ensure that required results are determined +before they are needed. + +The ordering of the algorithm processing is determined by their +placement in the ``IPU3::algorithms_`` ordered list. + +Sensor Controls +~~~~~~~~~~~~~~~ + +The AutoExposure and AutoGain (AGC) algorithm differs slightly from the +others as it requires operating directly on the sensor, as opposed to +through the ImgU ISP. To support this, there is a dedicated action +`ActionSetSensorControls` to allow the IPA to request controls to be set +on the camera sensor through the pipeline handler. \ No newline at end of file From patchwork Thu Sep 30 09:55:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14005 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F0E76C3243 for ; Thu, 30 Sep 2021 09:55:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C0657691D3; Thu, 30 Sep 2021 11:55:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CRzmyD/B"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A2E2691B9 for ; Thu, 30 Sep 2021 11:55:19 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1992518BF; Thu, 30 Sep 2021 11:55:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995719; bh=m8tpIzKV7MpAL7uaLgKeSkVY5qYp2AxxOVa+20j9Otk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CRzmyD/BlkyhYR9DLA6wf3mt9m7ofwWajnBVqiK28j7PqV+AXoIef59vVKTmnnOq1 NNSZDCiBKQc3x1ASybv8SR+5G5aKsC5HmdI/4k073pJAgBIOsPlA52T7Skiaq7kVKE 5MuHcj8/cwsG20Mhvn2nwrjme2R2xEXOu6V+BITk= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:55:05 +0200 Message-Id: <20210930095513.76213-11-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 10/18] ipa: ipu3: ipa_context: Fix file reference. X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The ipa_context.h entry incorrectly referenced its file name. Fix it. Signed-off-by: Kieran Bingham --- src/ipa/ipu3/ipa_context.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index e50de726..84b3ffae 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -2,7 +2,7 @@ /* * Copyright (C) 2021, Google Inc. * - * ipu3_ipa_context.h - IPU3 IPA Context + * ipa_context.h - IPU3 IPA Context * */ #ifndef __LIBCAMERA_IPU3_IPA_CONTEXT_H__ From patchwork Thu Sep 30 09:55:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14007 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 53BE4C3243 for ; Thu, 30 Sep 2021 09:55:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0B7F8691BE; Thu, 30 Sep 2021 11:55:35 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jBfjIzsu"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B3A1769189 for ; Thu, 30 Sep 2021 11:55:19 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 520A52A8; Thu, 30 Sep 2021 11:55:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995719; bh=bflav3BoMtMRml3kTokt0YFbozvO8ZSVq6DYSjRytmw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jBfjIzsuFFdo+n8HUqlmEixD2SmvTXHjYyTm3FnfrwiryeYEqhQU1ay+cOa9TvUBN 94UjRtSSAXtAT0aute6ppRtz5XXS+b06qQRw3EyZkJCbJto8BnsCiOZjsfZoWIvmsO eDRU426a0D5Z8OieFzpkF68hWhjWBZpiLMcBAikM= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:55:06 +0200 Message-Id: <20210930095513.76213-12-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 11/18] ipa: ipu3: Fix the IPU3 AWB doxygen references X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The AWB AwbStatus structure is contained within the Awb class. Fix the Doxygen reference so that it can be found. Signed-off-by: Kieran Bingham --- src/ipa/ipu3/algorithms/awb.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index f9e8cffc..59bc6900 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -89,7 +89,7 @@ static constexpr uint32_t kMinGreenLevelInZone = 32; */ /** - * \struct AwbStatus + * \struct Awb::AwbStatus * \brief AWB parameters calculated * * The AwbStatus structure is intended to store the AWB From patchwork Thu Sep 30 09:55:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14008 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E19D4C3243 for ; Thu, 30 Sep 2021 09:55:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A410A691D7; Thu, 30 Sep 2021 11:55:35 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="YxAg0xXV"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E4185691BB for ; Thu, 30 Sep 2021 11:55:19 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8B259B2B; Thu, 30 Sep 2021 11:55:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995719; bh=b1L0NSvEGUD2zE1QToOQEk16rOfGtL8CZiSqo+WYPUY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YxAg0xXVnY8ZDhwsJWNa8QGqRiXNBG+w7/r63JZc7FWNRKe/sw38NukRF11rq/tiv n176UUuiDl2FiC4/+oh4d5AR4O8+RGlomxYkd6IleTu2th1YNGLSD3auK8ZmrTnj0n Q1Rc9yoHCq6Ffmit0etDuRliZ/jQOYagDHayIs8Q= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:55:07 +0200 Message-Id: <20210930095513.76213-13-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 12/18] ipa: ipu3: algorithms: awb: Privatize internal structures X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The struct RGB and struct AwbStatus are used only by the internal implementation of the AWB algorithm module. Move them into the private class declaration. Signed-off-by: Kieran Bingham --- src/ipa/ipu3/algorithms/awb.h | 1 + 1 file changed, 1 insertion(+) diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index 677384ed..a56362ff 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -42,6 +42,7 @@ public: void prepare(IPAContext &context, ipu3_uapi_params *params) override; void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; +private: /* \todo Make these structs available to all the ISPs ? */ struct RGB { RGB(double _R = 0, double _G = 0, double _B = 0) From patchwork Thu Sep 30 09:55:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14010 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 38C67C3243 for ; Thu, 30 Sep 2021 09:55:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D82CB691D8; Thu, 30 Sep 2021 11:55:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jAqhDvva"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0B6BE691BC for ; Thu, 30 Sep 2021 11:55:20 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BBDA418BF; Thu, 30 Sep 2021 11:55:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995719; bh=iMfGRg22aly9CzR3uara502xhyiRU1XI3l8BXJ+9HCs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jAqhDvvallpo05oGq6SBO5AwmZpzemv3GRXUyLRqg8nJ0d3fEpyirmODFmEdhrIMt mLpMjF5d1KcH924mN+T9Ks+dKrM4W5j0T801tj3Ph42YQ9YPshIx3aGD2zFxrEj+2w u7MiDv2q5FzfulZrL9dhiewLHme21qsKNnkpIylg= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:55:08 +0200 Message-Id: <20210930095513.76213-14-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 13/18] ipa: ipu3: Isolate ipa_context documentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The IPU3 IPA core is growing with additional documentation. The ipa_context documentation is stored here, but it pushes the IPU3 documentation and implementation further from the head of the file. Furthermore, the ipa_context documentation is outside of the ipa::ipu3 namespace and isn't identified correctly by Doxygen. Move the ipa_context to its own compilation object even though there isn't any code, but to maintain consistency with our documentation model. Correctly re-introduce the documentation into the libcamera::ipa::ipu3 namespace during the move. Signed-off-by: Kieran Bingham --- src/ipa/ipu3/ipa_context.cpp | 113 +++++++++++++++++++++++++++++++++++ src/ipa/ipu3/ipu3.cpp | 104 -------------------------------- src/ipa/ipu3/meson.build | 1 + 3 files changed, 114 insertions(+), 104 deletions(-) create mode 100644 src/ipa/ipu3/ipa_context.cpp diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp new file mode 100644 index 00000000..40d79772 --- /dev/null +++ b/src/ipa/ipu3/ipa_context.cpp @@ -0,0 +1,113 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Google Inc. + * + * ipa_context.cpp - IPU3 IPA Context + */ + +#include "ipa_context.h" + +namespace libcamera::ipa::ipu3 { + +/** + * \file ipa_context.h + * \brief Context and state information shared between the algorithms + */ + +/** + * \struct IPASessionConfiguration + * \brief Session configuration for the IPA module + * + * The session configuration contains all IPA configuration parameters that + * remain constant during the capture session, from IPA module start to stop. + * It is typically set during the configure() operation of the IPA module, but + * may also be updated in the start() operation. + */ + +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + * + * The frame context stores data specific to a single frame processed by the + * IPA. Each frame processed by the IPA has a context associated with it, + * accessible through the IPAContext structure. + * + * \todo Detail how to access contexts for a particular frame + * + * Each of the fields in the frame context belongs to either a specific + * algorithm, or to the top-level IPA module. A field may be read by any + * algorithm, but should only be written by its owner. + */ + +/** + * \struct IPAContext + * \brief Global IPA context data shared between all algorithms + * + * \var IPAContext::configuration + * \brief The IPA session configuration, immutable during the session + * + * \var IPAContext::frameContext + * \brief The frame context for the frame being processed + * + * \todo While the frame context is supposed to be per-frame, this + * single frame context stores data related to both the current frame + * and the previous frames, with fields being updated as the algorithms + * are run. This needs to be turned into real per-frame data storage. + */ + +/** + * \struct IPASessionConfiguration::grid + * \brief Grid configuration of the IPA + * + * \var IPASessionConfiguration::grid::bdsGrid + * \brief Bayer Down Scaler grid plane config used by the kernel + * + * \var IPASessionConfiguration::grid::bdsOutputSize + * \brief BDS output size configured by the pipeline handler + */ + +/** + * \struct IPAFrameContext::agc + * \brief Context for the Automatic Gain Control algorithm + * + * The exposure and gain determined are expected to be applied to the sensor + * at the earliest opportunity. + * + * \var IPAFrameContext::agc::exposure + * \brief Exposure time expressed as a number of lines + * + * \var IPAFrameContext::agc::gain + * \brief Analogue gain multiplier + * + * The gain should be adapted to the sensor specific gain code before applying. + */ + +/** + * \struct IPAFrameContext::awb + * \brief Context for the Automatic White Balance algorithm + * + * \struct IPAFrameContext::awb::gains + * \brief White balance gains + * + * \var IPAFrameContext::awb::gains::red + * \brief White balance gain for R channel + * + * \var IPAFrameContext::awb::gains::green + * \brief White balance gain for G channel + * + * \var IPAFrameContext::awb::gains::blue + * \brief White balance gain for B channel + */ + +/** + * \struct IPAFrameContext::toneMapping + * \brief Context for ToneMapping and Gamma control + * + * \var IPAFrameContext::toneMapping::gammaCorrection + * \brief Per-pixel tone mapping implemented as a LUT + * + * The LUT structure is defined by the IPU3 kernel interface. See + * struct ipu3_uapi_gamma_corr_lut for further details. + */ + +} /* namespace libcamera::ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 9867c993..1b1fc052 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -37,110 +37,6 @@ #include "algorithms/tone_mapping.h" #include "libipa/camera_sensor_helper.h" -/** - * \file ipa_context.h - * \brief Context and state information shared between the algorithms - */ - -/** - * \struct IPASessionConfiguration - * \brief Session configuration for the IPA module - * - * The session configuration contains all IPA configuration parameters that - * remain constant during the capture session, from IPA module start to stop. - * It is typically set during the configure() operation of the IPA module, but - * may also be updated in the start() operation. - */ - -/** - * \struct IPAFrameContext - * \brief Per-frame context for algorithms - * - * The frame context stores data specific to a single frame processed by the - * IPA. Each frame processed by the IPA has a context associated with it, - * accessible through the IPAContext structure. - * - * \todo Detail how to access contexts for a particular frame - * - * Each of the fields in the frame context belongs to either a specific - * algorithm, or to the top-level IPA module. A field may be read by any - * algorithm, but should only be written by its owner. - */ - -/** - * \struct IPAContext - * \brief Global IPA context data shared between all algorithms - * - * \var IPAContext::configuration - * \brief The IPA session configuration, immutable during the session - * - * \var IPAContext::frameContext - * \brief The frame context for the frame being processed - * - * \todo While the frame context is supposed to be per-frame, this - * single frame context stores data related to both the current frame - * and the previous frames, with fields being updated as the algorithms - * are run. This needs to be turned into real per-frame data storage. - */ - -/** - * \struct IPASessionConfiguration::grid - * \brief Grid configuration of the IPA - * - * \var IPASessionConfiguration::grid::bdsGrid - * \brief Bayer Down Scaler grid plane config used by the kernel - * - * \var IPASessionConfiguration::grid::bdsOutputSize - * \brief BDS output size configured by the pipeline handler - */ - -/** - * \struct IPAFrameContext::agc - * \brief Context for the Automatic Gain Control algorithm - * - * The exposure and gain determined are expected to be applied to the sensor - * at the earliest opportunity. - * - * \var IPAFrameContext::agc::exposure - * \brief Exposure time expressed as a number of lines - * - * \var IPAFrameContext::agc::gain - * \brief Analogue gain multiplier - * - * The gain should be adapted to the sensor specific gain code before applying. - */ - -/** - * \struct IPAFrameContext::awb - * \brief Context for the Automatic White Balance algorithm - * - * \struct IPAFrameContext::awb::gains - * \brief White balance gains - * - * \var IPAFrameContext::awb::gains::red - * \brief White balance gain for R channel - * - * \var IPAFrameContext::awb::gains::green - * \brief White balance gain for G channel - * - * \var IPAFrameContext::awb::gains::blue - * \brief White balance gain for B channel - */ - -/** - * \struct IPAFrameContext::toneMapping - * \brief Context for ToneMapping and Gamma control - * - * \var IPAFrameContext::toneMapping::gamma - * \brief Gamma value for the LUT - * - * \var IPAFrameContext::toneMapping::gammaCorrection - * \brief Per-pixel tone mapping implemented as a LUT - * - * The LUT structure is defined by the IPU3 kernel interface. See - * struct ipu3_uapi_gamma_corr_lut for further details. - */ - /* Minimum grid width, expressed as a number of cells */ static constexpr uint32_t kMinGridWidth = 16; /* Maximum grid width, expressed as a number of cells */ diff --git a/src/ipa/ipu3/meson.build b/src/ipa/ipu3/meson.build index 39320980..3194111a 100644 --- a/src/ipa/ipu3/meson.build +++ b/src/ipa/ipu3/meson.build @@ -5,6 +5,7 @@ subdir('algorithms') ipa_name = 'ipa_ipu3' ipu3_ipa_sources = files([ + 'ipa_context.cpp', 'ipu3.cpp', ]) From patchwork Thu Sep 30 09:55:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14011 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C143BC3243 for ; Thu, 30 Sep 2021 09:55:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 821E4691C8; Thu, 30 Sep 2021 11:55:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="E6+eBIxC"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 156A3691BE for ; Thu, 30 Sep 2021 11:55:21 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EA7A52A8; Thu, 30 Sep 2021 11:55:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995720; bh=ZDsC5U9elk9Kl89mk/8RV4aoN2ujV09pb+sjRK5GPHo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=E6+eBIxCi6Wuq4VtWesda1impfZD9VL24Q47LESzit+okRh4jORfmuso1Ch8CT4EB n6eN/hO8qkPvj2W0Z2dfe+XvDtzIpqtU159+TrhL+iWLfDAql6tlRARg+FkfSlPZDa jrY8MUNbLnXu3XiFpphEaQZuC4Ro6YSVRe2Ig4A0= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:55:09 +0200 Message-Id: <20210930095513.76213-15-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 14/18] ipa: ipu3: ipa_context: Fix doxygen references X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The IPAFrameContext use unamed structures to group items. Doxygen can not reference the member variables of the container structures through the variable names, and expects the structure type name. As this is not given, the structure variables are referenced from the parent structure. Remove the member name from the annonymous type to allow Doxygen to match the implementation from the documentation. Signed-off-by: Kieran Bingham --- src/ipa/ipu3/ipa_context.cpp | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 40d79772..a38e6053 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -59,10 +59,10 @@ namespace libcamera::ipa::ipu3 { * \struct IPASessionConfiguration::grid * \brief Grid configuration of the IPA * - * \var IPASessionConfiguration::grid::bdsGrid + * \var IPASessionConfiguration::bdsGrid * \brief Bayer Down Scaler grid plane config used by the kernel * - * \var IPASessionConfiguration::grid::bdsOutputSize + * \var IPASessionConfiguration::bdsOutputSize * \brief BDS output size configured by the pipeline handler */ @@ -73,10 +73,10 @@ namespace libcamera::ipa::ipu3 { * The exposure and gain determined are expected to be applied to the sensor * at the earliest opportunity. * - * \var IPAFrameContext::agc::exposure + * \var IPAFrameContext::exposure * \brief Exposure time expressed as a number of lines * - * \var IPAFrameContext::agc::gain + * \var IPAFrameContext::gain * \brief Analogue gain multiplier * * The gain should be adapted to the sensor specific gain code before applying. @@ -89,13 +89,13 @@ namespace libcamera::ipa::ipu3 { * \struct IPAFrameContext::awb::gains * \brief White balance gains * - * \var IPAFrameContext::awb::gains::red + * \var IPAFrameContext::red * \brief White balance gain for R channel * - * \var IPAFrameContext::awb::gains::green + * \var IPAFrameContext::green * \brief White balance gain for G channel * - * \var IPAFrameContext::awb::gains::blue + * \var IPAFrameContext::blue * \brief White balance gain for B channel */ @@ -103,7 +103,10 @@ namespace libcamera::ipa::ipu3 { * \struct IPAFrameContext::toneMapping * \brief Context for ToneMapping and Gamma control * - * \var IPAFrameContext::toneMapping::gammaCorrection + * \var IPAFrameContext::gamma + * \brief Gamma value for the LUT + * + * \var IPAFrameContext::gammaCorrection * \brief Per-pixel tone mapping implemented as a LUT * * The LUT structure is defined by the IPU3 kernel interface. See From patchwork Thu Sep 30 09:55:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14009 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 87C43C3243 for ; Thu, 30 Sep 2021 09:55:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 408B6691C3; Thu, 30 Sep 2021 11:55:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Gf8v3eSR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D2C72691B4 for ; Thu, 30 Sep 2021 11:55:20 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2552FB2B; Thu, 30 Sep 2021 11:55:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995720; bh=4zm0+6D4Z79mH0HKtoHlRe16zrM2CXyCnMByCJcsfAU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Gf8v3eSRPSOz4o5Fy/8jY2dFRnrIq6sb64mPh//0ejOF3jUT++mtfhDI2bUFK87r8 O5d6gWBOZCfhYYafdiAuHLatKRuWw/wC8ymKmsAzmOvdDcG6W6pXtfSioQhmda+MWS 825b9LG3zT6UJq1svKVfqeZPIgpC3mZseLPs3QLM= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:55:10 +0200 Message-Id: <20210930095513.76213-16-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 15/18] Documentation: Include IPU3 in Doxygen build X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The IPU3 is documented using Doxygen within the code. Include the IPU3 IPA as part of the doxygen build sources. This will ideally be split to it's own 'section' of the doxygen output. Signed-off-by: Kieran Bingham --- Documentation/Doxyfile.in | 1 + Documentation/meson.build | 1 + 2 files changed, 2 insertions(+) diff --git a/Documentation/Doxyfile.in b/Documentation/Doxyfile.in index 6e627192..37b231bc 100644 --- a/Documentation/Doxyfile.in +++ b/Documentation/Doxyfile.in @@ -790,6 +790,7 @@ WARN_LOGFILE = INPUT = "@TOP_SRCDIR@/include/libcamera" \ "@TOP_SRCDIR@/src/ipa/libipa" \ + "@TOP_SRCDIR@/src/ipa/ipu3" \ "@TOP_SRCDIR@/src/libcamera" \ "@TOP_BUILDDIR@/include/libcamera" \ "@TOP_BUILDDIR@/src/libcamera" diff --git a/Documentation/meson.build b/Documentation/meson.build index c4bd3c7f..c174c945 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -31,6 +31,7 @@ if doxygen.found() and dot.found() libcamera_sources, libipa_headers, libipa_sources, + ipu3_ipa_sources, ], output : 'api-html', command : [doxygen, doxyfile], From patchwork Thu Sep 30 09:55:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14012 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 57B12C3243 for ; Thu, 30 Sep 2021 09:55:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 12464691B1; Thu, 30 Sep 2021 11:55:38 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="o8HjtiCH"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 21C96691BF for ; Thu, 30 Sep 2021 11:55:21 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5368E18BF; Thu, 30 Sep 2021 11:55:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995720; bh=A/hOwYdwCOMxXcL/EkM0IZNHF0/YRKXrGjdUpGmcp50=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=o8HjtiCH6B2GStnegtqIrLFyKIlU7ezYeUEd6IRFOGACHK6qYjGzyzA8kngy7uHpW uQr5/xNivLBMMW55L2OjZDfIFEb7ayTg1Mh8ge1i/EqMTW/PVTI18CkLjGX8hacuFg 8PdUsyy2D8GpbGQNrO9n7SphOAP7ZU+PBx1VaYtA= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:55:11 +0200 Message-Id: <20210930095513.76213-17-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 16/18] ipa: ipu3: awb: Document the Accumulator::sum variable X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Doxygen warns about the undocumented variable sum. Fix it by adding a small brief. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/awb.cpp | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 59bc6900..af1edd11 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -86,6 +86,9 @@ static constexpr uint32_t kMinGreenLevelInZone = 32; * * \var Accumulator::blue * \brief Sum of the average blue values of each unsaturated cell in the zone + * + * \var Accumulator::sum + * \brief A structure containing the average red, green and blue sums */ /** From patchwork Thu Sep 30 09:55:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14014 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8F00CC3243 for ; Thu, 30 Sep 2021 09:55:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4B00D691CE; Thu, 30 Sep 2021 11:55:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dxoOpLXp"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 442D3691C0 for ; Thu, 30 Sep 2021 11:55:21 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8CDB41AFA; Thu, 30 Sep 2021 11:55:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995720; bh=CG01d/s9BCS/Njg9FnQjNARQBt5oCZ0zZEiOIWJjcGc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dxoOpLXpcU+hTzgnPwtsG5GPvsl/jTAILVWipWyZf58HGo57lxNMU0Z1zfmjCciqX hGjUHt/IJPKQ/SkLd/2qQadQFpA3ks9oPQ4E80IeHnFS+ze9tK6f6iEk2KtPX8I1XT hYqw7tVZhWEMJpNRFWyN1ameLtkLAeYGJ3dAqj8c= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:55:12 +0200 Message-Id: <20210930095513.76213-18-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 17/18] ipa: ipu3: Fix badly documented context variables X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Doxygen warns us because the structures are referenced as \struct while they should be \var. Fix it. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipa_context.cpp | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index a38e6053..5a4c48f1 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -56,7 +56,7 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPASessionConfiguration::grid + * \var IPASessionConfiguration::grid * \brief Grid configuration of the IPA * * \var IPASessionConfiguration::bdsGrid @@ -67,7 +67,7 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPAFrameContext::agc + * \var IPAFrameContext::agc * \brief Context for the Automatic Gain Control algorithm * * The exposure and gain determined are expected to be applied to the sensor @@ -83,7 +83,7 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPAFrameContext::awb + * \var IPAFrameContext::awb * \brief Context for the Automatic White Balance algorithm * * \struct IPAFrameContext::awb::gains @@ -100,7 +100,7 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPAFrameContext::toneMapping + * \var IPAFrameContext::toneMapping * \brief Context for ToneMapping and Gamma control * * \var IPAFrameContext::gamma From patchwork Thu Sep 30 09:55:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14013 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F03BFC3243 for ; Thu, 30 Sep 2021 09:55:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A8C44691B0; Thu, 30 Sep 2021 11:55:38 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ZvxOxBKK"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 68B3C691C1 for ; Thu, 30 Sep 2021 11:55:21 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BC2491B48; Thu, 30 Sep 2021 11:55:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995720; bh=DDSdwo8bZwyghQYxYyzFaX9+s+vyW6k7QMqpYPha5CM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZvxOxBKKbNf++FzPMxWPPG6E2oVXM7xzVWWHqoHLtaX/gZtycU9rM81/nt4dz+sRH 9f8Sc1s9s71KUAjifVvfkqHiZUdmRUVNO+U2x6b5XPgjznAW4QJzKhOR6tzBZMsWqJ 92Wm3kj59x0+CSfalFZGwolb8kGfztZijLu54YPI= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:55:13 +0200 Message-Id: <20210930095513.76213-19-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 18/18] ipa: ipu3: Implement an empty stop() function X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" While the stop() method does not currently perform any action, it forms part of the IPA interface and is a public function in the class. Promote it to a full (but basic) function implementation and begin the documentation accordingly so that there is an appropriate stub to perform stop operations if they come up. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipu3.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 1b1fc052..4f2fbf34 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -129,7 +129,7 @@ public: ControlInfoMap *ipaControls) override; int start() override; - void stop() override {} + void stop() override; int configure(const IPAConfigInfo &configInfo) override; @@ -263,6 +263,13 @@ int IPAIPU3::start() return 0; } +/** + * \brief Ensure that all processing has completed. + */ +void IPAIPU3::stop() +{ +} + /** * \brief Calculate a grid for the AWB statistics *