From patchwork Fri Oct 22 15:12:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14267 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DAEECBF415 for ; Fri, 22 Oct 2021 15:12:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BDA6668F6E; Fri, 22 Oct 2021 17:12:27 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="q62koB17"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 14B0C6023B for ; Fri, 22 Oct 2021 17:12:23 +0200 (CEST) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:22cc:3af6:5ccb:8367]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A159089A; Fri, 22 Oct 2021 17:12:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1634915542; bh=99QgrP6v8KkJTGcDtcYdmC1irxDEMwtMvMOLOkoGjRg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=q62koB17elRYhLGbgdfwSOFVvcoTUqDr+3QUXmkNHvQY6hwe8aX3ESll2TTbwutQy sY70t+6gE7rSJCNWQj9/jNPUDC6+X788ujboQMf8LdIVFDdw2dhCKR+XVppuQa3new uqX2va/tsrtrCGXhZADwBoXgOE1K3oHUBJlHEWQ0= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Fri, 22 Oct 2021 17:12:00 +0200 Message-Id: <20211022151218.111966-2-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211022151218.111966-1-jeanmichel.hautbois@ideasonboard.com> References: <20211022151218.111966-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 01/19] ipa: ipu3: Document IPAIPU3 class interface X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The IPU3 IPA is maturing to a modular and extensible system capable of handling specific algorithms for the processing blocks on the ImgU. Provide a top-level class documentation to provide an overview of the IPA, detailing what events are used and what algorithms are currently supported, as well as the limitations currently imposed. Signed-off-by: Jean-Michel Hautbois Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain --- src/ipa/ipu3/ipu3.cpp | 64 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 65d3fd2c..b5438ed1 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -179,6 +179,70 @@ using namespace std::literals::chrono_literals; namespace ipa::ipu3 { +/** + * \brief The IPU3 IPA implementation + * + * The IPU3 Pipeline defines an IPU3-specific interface for communication + * between the PipelineHandler and the IPA module. + * + * We extend the IPAIPU3Interface to implement our algorithms and handle events + * from the IPU3 PipelineHandler to satisfy requests from the application. + * + * At initialisation time, a CameraSensorHelper is instantiated to support + * camera-specific calculations, while the default controls are computed, and + * the algorithms are constructed and placed in an ordered list. + * + * The IPU3 ImgU operates with a grid layout to divide the overall frame into + * rectangular cells of pixels. When the IPA is configured, we determine the + * best grid for the statistics based on the pipeline handler Bayer Down Scaler + * output size. + * + * Two main events are then handled to operate the IPU3 ImgU by populating its + * parameter buffer, and adapting the settings of the sensor attached to the + * IPU3 CIO2 through sensor-specific V4L2 controls. + * + * When the event \a EventFillParams occurs we populate the ImgU parameter + * buffer with settings to configure the device in preparation for handling the + * frame queued in the Request. + * + * When the frame has completed processing, the ImgU will generate a statistics + * buffer which is given to the IPA as part of the \a EventStatReady event. At + * this event we run the algorithms to parse the statistics and cache any + * results for the next \a EventFillParams event. + * + * The individual algorithms are split into modular components that are called + * iteratively to allow them to process statistics from the ImgU in a defined + * order. + * + * The current implementation supports three core algorithms: + * - Automatic white balance (AWB) + * - Automatic gain and exposure control (AGC) + * - Tone mapping (Gamma) + * + * AWB is implemented using a Greyworld algorithm, and calculates the red and + * blue gains to apply to generate a neutral grey frame overall. + * + * AGC is handled by calculating a histogram of the green channel to estimate an + * analogue gain and shutter time which will provide a well exposed frame. A + * low-pass IIR filter is used to smooth the changes to the sensor to reduce + * perceivable steps. + * + * The Tone mapping algorithm provides a gamma correction table to improve the + * contrast of the scene. + * + * The IPU3 ImgU has further processing blocks to support image quality + * improvements through bayer and temporal noise reductions, however those are + * not supported in the current implementation, and will use default settings as + * provided by the kernel driver. + * + * Demosaicing is operating with the default parameters and could be further + * optimised to provide improved sharpening coefficients, checker artifact + * removal, and false color correction. + * + * Additional image enhancements can be made by providing lens and + * sensor-specific tuning to adapt for Black Level compensation (BLC), Lens + * shading correction (SHD) and Color correction (CCM). + */ class IPAIPU3 : public IPAIPU3Interface { public: