From patchwork Thu Sep 30 09:54:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 13997 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8FD7DC3243 for ; Thu, 30 Sep 2021 09:55:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C3126691B0; Thu, 30 Sep 2021 11:55:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="af9Tq6aM"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 99387691A3 for ; Thu, 30 Sep 2021 11:55:17 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:bab4:22c5:662d:e478]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 32C65B2B; Thu, 30 Sep 2021 11:55:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1632995717; bh=E2mESPxTMSuWK8k3JEjStbjEpEKFAesMcH1KthhAQQ8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=af9Tq6aMBBPkPybJdDoOQU+igkp1hL0p39xw744Fv9ZYmcHwKeHeTLbmGw7grjUv1 KmSOxoLYXdUlDTA+JUu9Pzo4oBvkzx5LyKBIT5r6GcHgOVGW2BN8pxX5Ishf+2OAaH UkwHyeOJxHwQr4q9Bk4NHO6ja5r6dwIDsCyncj0o= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Sep 2021 11:54:56 +0200 Message-Id: <20210930095513.76213-2-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> References: <20210930095513.76213-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 01/18] ipa: ipu3: Document IPAIPU3 class interface X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The IPU3 IPA is maturing to a modular and extensible system capable of handling specific algorithms for the accelleration clusters on the ImgU. Provide a top-level class documentation to provide an overview of the IPA, detailing what events are used and what algorithms are currently supported, as well as the limitations currently imposed. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham --- src/ipa/ipu3/ipu3.cpp | 64 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 7cc0fb89..bf8fc1e8 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -157,6 +157,70 @@ LOG_DEFINE_CATEGORY(IPAIPU3) namespace ipa::ipu3 { +/** + * \brief The IPU3 IPA implementation + * + * The IPU3 Pipeline defines an IPU3 specific interface for communication + * between the PipelineHandler, and the IPA module. + * + * We extend the IPAIPU3Interface to implement our algorithms and handle events + * from the IPU3 PipelineHandler to satisfy requests from the application. + * + * At initialisation time, a CameraSensorHelper is instantiated to support + * camera specific calculations, while the default controls are computed, and + * the algorithms are constructed and placed in an ordered list. + * + * The IPU3 ImgU operates with a grid layout to divide the overall frame into + * rectangular cells of pixels. When the IPA is configured, we determine the + * best grid for the statistics based on the pipeline handler Bayer Down Scaler + * output size. + * + * Two main events are then handled to facilitate the operation of the IPU3 ImgU + * by populating its parameter buffer, and adapting the settings of the sensor + * attached to the IPU3 CIO2 through sensor specific V4L2 controls. + * + * When the event \a EventFillParams occurs we populate the ImgU parameter + * buffer with settings to configure the device in preparation for handling the + * frame queued in the Request. + * + * When the frame has completed processing, the ImgU will generate a statistics + * buffer which is given to the IPA as part of the \a EventStatReady event. At + * this event we run the algorithms to parse the statistics and cache any + * results for the next \a EventFillParams event. + * + * The individual algorithms are split into modular components that are called + * iteratively to allow them to process statistics from the ImgU in a defined + * order. + * + * The current implementation supports three core algorithms: + * - Automatic white balance (AWB) + * - Automatic gain and exposure control (AGC) + * - Tonemapping (Gamma) + * + * AWB is implemented using a Greyworld algorithm, and calculates the red and + * blue gains to apply to generate a neutral grey frame overall. + * + * AGC is handled by calculating a histogram of the green channel to estimate an + * analogue gain and shutter time which will provide a well exposed frame. An + * IIR filter is used to smooth the changes to the sensor to reduce perceivable + * steps. + * + * The Tonemapping algorithm provides a gamma correction table to improve the + * contrast of the scene. + * + * The IPU3 ImgU has further accellerator clusters to support image quality + * improvements through bayer and temporal noise reductions, however those are + * not supported in the current implementation, and will use default settings as + * provided by the kernel driver. + * + * Demosaicing is operating on the default values and could be further optimised + * to provide improved sharpening coefficients, checker artifact removal, and + * false color correction. + * + * Additional image enhancements can be made by providing lens and sensor + * specific tuning to adapt for Black Level compensation (BLC), Lens shading + * correction (SHD) and Color correction (CCM) + */ class IPAIPU3 : public IPAIPU3Interface { public: