[{"id":20306,"web_url":"https://patchwork.libcamera.org/comment/20306/","msgid":"","date":"2021-10-19T12:11:47","subject":"Re: [libcamera-devel] [PATCH 01/18] ipa: ipu3: Document IPAIPU3\n\tclass interface","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Jean-Michel,\n\nThank you for the patch.\n\nOn Thu, Sep 30, 2021 at 11:54:56AM +0200, Jean-Michel Hautbois wrote:\n> The IPU3 IPA is maturing to a modular and extensible system capable of\n> handling specific algorithms for the accelleration clusters on the ImgU.\n\ns/accelleration/acceleration/\n\nActually my comments on v1\n(20210913145810.66515-2-jeanmichel.hautbois@ideasonboard.com) have all\nbeen ignored, so I'll review v3.\n\n> Provide a top-level class documentation to provide an overview of the\n> IPA, detailing what events are used and what algorithms are currently\n> supported, as well as the limitations currently imposed.\n> \n> Signed-off-by: Jean-Michel Hautbois \n> Signed-off-by: Kieran Bingham \n> ---\n> src/ipa/ipu3/ipu3.cpp | 64 +++++++++++++++++++++++++++++++++++++++++++\n> 1 file changed, 64 insertions(+)\n> \n> diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp\n> index 7cc0fb89..bf8fc1e8 100644\n> --- a/src/ipa/ipu3/ipu3.cpp\n> +++ b/src/ipa/ipu3/ipu3.cpp\n> @@ -157,6 +157,70 @@ LOG_DEFINE_CATEGORY(IPAIPU3)\n> \n> namespace ipa::ipu3 {\n> \n> +/**\n> + * \\brief The IPU3 IPA implementation\n> + *\n> + * The IPU3 Pipeline defines an IPU3 specific interface for communication\n> + * between the PipelineHandler, and the IPA module.\n> + *\n> + * We extend the IPAIPU3Interface to implement our algorithms and handle events\n> + * from the IPU3 PipelineHandler to satisfy requests from the application.\n> + *\n> + * At initialisation time, a CameraSensorHelper is instantiated to support\n> + * camera specific calculations, while the default controls are computed, and\n> + * the algorithms are constructed and placed in an ordered list.\n> + *\n> + * The IPU3 ImgU operates with a grid layout to divide the overall frame into\n> + * rectangular cells of pixels. When the IPA is configured, we determine the\n> + * best grid for the statistics based on the pipeline handler Bayer Down Scaler\n> + * output size.\n> + *\n> + * Two main events are then handled to facilitate the operation of the IPU3 ImgU\n> + * by populating its parameter buffer, and adapting the settings of the sensor\n> + * attached to the IPU3 CIO2 through sensor specific V4L2 controls.\n> + *\n> + * When the event \\a EventFillParams occurs we populate the ImgU parameter\n> + * buffer with settings to configure the device in preparation for handling the\n> + * frame queued in the Request.\n> + *\n> + * When the frame has completed processing, the ImgU will generate a statistics\n> + * buffer which is given to the IPA as part of the \\a EventStatReady event. At\n> + * this event we run the algorithms to parse the statistics and cache any\n> + * results for the next \\a EventFillParams event.\n> + *\n> + * The individual algorithms are split into modular components that are called\n> + * iteratively to allow them to process statistics from the ImgU in a defined\n> + * order.\n> + *\n> + * The current implementation supports three core algorithms:\n> + * - Automatic white balance (AWB)\n> + * - Automatic gain and exposure control (AGC)\n> + * - Tonemapping (Gamma)\n> + *\n> + * AWB is implemented using a Greyworld algorithm, and calculates the red and\n> + * blue gains to apply to generate a neutral grey frame overall.\n> + *\n> + * AGC is handled by calculating a histogram of the green channel to estimate an\n> + * analogue gain and shutter time which will provide a well exposed frame. An\n> + * IIR filter is used to smooth the changes to the sensor to reduce perceivable\n> + * steps.\n> + *\n> + * The Tonemapping algorithm provides a gamma correction table to improve the\n> + * contrast of the scene.\n> + *\n> + * The IPU3 ImgU has further accellerator clusters to support image quality\n> + * improvements through bayer and temporal noise reductions, however those are\n> + * not supported in the current implementation, and will use default settings as\n> + * provided by the kernel driver.\n> + *\n> + * Demosaicing is operating on the default values and could be further optimised\n> + * to provide improved sharpening coefficients, checker artifact removal, and\n> + * false color correction.\n> + *\n> + * Additional image enhancements can be made by providing lens and sensor\n> + * specific tuning to adapt for Black Level compensation (BLC), Lens shading\n> + * correction (SHD) and Color correction (CCM)\n> + */\n> class IPAIPU3 : public IPAIPU3Interface\n> {\n> public:","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A3134C323E\n\tfor ;\n\tTue, 19 Oct 2021 12:12:07 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 5E6DD68F59;\n\tTue, 19 Oct 2021 14:12:07 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 56AD1604FE\n\tfor ;\n\tTue, 19 Oct 2021 14:12:05 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id ECD618AE;\n\tTue, 19 Oct 2021 14:12:04 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"OKHnFBSC\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1634645525;\n\tbh=Vv6b0N5Qu7Lm/IoQQVoKyAZGt9WYLVBtbj/gHGFJvbs=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=OKHnFBSC0GgPNdZyyqYMud05bmHJqsbco0HzWBWSTVCcdlKkYif0MzQQmswJ/wC99\n\ttULCCI8FIaCGqAxtfAPkw6SYri52ImnbBZ+Er50PGJ7t2Mw+atXjJcXJ9v/5dbWG8S\n\tcSc0gRz/K/LmG5/wwD+fcr/7P4a3kgpiUvEdq0BI=","Date":"Tue, 19 Oct 2021 15:11:47 +0300","From":"Laurent Pinchart ","To":"Jean-Michel Hautbois ","Message-ID":"","References":"<20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com>\n\t<20210930095513.76213-2-jeanmichel.hautbois@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20210930095513.76213-2-jeanmichel.hautbois@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH 01/18] ipa: ipu3: Document IPAIPU3\n\tclass interface","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]