Message ID | 20210930095513.76213-2-jeanmichel.hautbois@ideasonboard.com |
---|---|
State | Changes Requested |
Headers | show |
Series |
|
Related | show |
Hi Jean-Michel, Thank you for the patch. On Thu, Sep 30, 2021 at 11:54:56AM +0200, Jean-Michel Hautbois wrote: > The IPU3 IPA is maturing to a modular and extensible system capable of > handling specific algorithms for the accelleration clusters on the ImgU. s/accelleration/acceleration/ Actually my comments on v1 (20210913145810.66515-2-jeanmichel.hautbois@ideasonboard.com) have all been ignored, so I'll review v3. > Provide a top-level class documentation to provide an overview of the > IPA, detailing what events are used and what algorithms are currently > supported, as well as the limitations currently imposed. > > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> > Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> > --- > src/ipa/ipu3/ipu3.cpp | 64 +++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 64 insertions(+) > > diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp > index 7cc0fb89..bf8fc1e8 100644 > --- a/src/ipa/ipu3/ipu3.cpp > +++ b/src/ipa/ipu3/ipu3.cpp > @@ -157,6 +157,70 @@ LOG_DEFINE_CATEGORY(IPAIPU3) > > namespace ipa::ipu3 { > > +/** > + * \brief The IPU3 IPA implementation > + * > + * The IPU3 Pipeline defines an IPU3 specific interface for communication > + * between the PipelineHandler, and the IPA module. > + * > + * We extend the IPAIPU3Interface to implement our algorithms and handle events > + * from the IPU3 PipelineHandler to satisfy requests from the application. > + * > + * At initialisation time, a CameraSensorHelper is instantiated to support > + * camera specific calculations, while the default controls are computed, and > + * the algorithms are constructed and placed in an ordered list. > + * > + * The IPU3 ImgU operates with a grid layout to divide the overall frame into > + * rectangular cells of pixels. When the IPA is configured, we determine the > + * best grid for the statistics based on the pipeline handler Bayer Down Scaler > + * output size. > + * > + * Two main events are then handled to facilitate the operation of the IPU3 ImgU > + * by populating its parameter buffer, and adapting the settings of the sensor > + * attached to the IPU3 CIO2 through sensor specific V4L2 controls. > + * > + * When the event \a EventFillParams occurs we populate the ImgU parameter > + * buffer with settings to configure the device in preparation for handling the > + * frame queued in the Request. > + * > + * When the frame has completed processing, the ImgU will generate a statistics > + * buffer which is given to the IPA as part of the \a EventStatReady event. At > + * this event we run the algorithms to parse the statistics and cache any > + * results for the next \a EventFillParams event. > + * > + * The individual algorithms are split into modular components that are called > + * iteratively to allow them to process statistics from the ImgU in a defined > + * order. > + * > + * The current implementation supports three core algorithms: > + * - Automatic white balance (AWB) > + * - Automatic gain and exposure control (AGC) > + * - Tonemapping (Gamma) > + * > + * AWB is implemented using a Greyworld algorithm, and calculates the red and > + * blue gains to apply to generate a neutral grey frame overall. > + * > + * AGC is handled by calculating a histogram of the green channel to estimate an > + * analogue gain and shutter time which will provide a well exposed frame. An > + * IIR filter is used to smooth the changes to the sensor to reduce perceivable > + * steps. > + * > + * The Tonemapping algorithm provides a gamma correction table to improve the > + * contrast of the scene. > + * > + * The IPU3 ImgU has further accellerator clusters to support image quality > + * improvements through bayer and temporal noise reductions, however those are > + * not supported in the current implementation, and will use default settings as > + * provided by the kernel driver. > + * > + * Demosaicing is operating on the default values and could be further optimised > + * to provide improved sharpening coefficients, checker artifact removal, and > + * false color correction. > + * > + * Additional image enhancements can be made by providing lens and sensor > + * specific tuning to adapt for Black Level compensation (BLC), Lens shading > + * correction (SHD) and Color correction (CCM) > + */ > class IPAIPU3 : public IPAIPU3Interface > { > public:
diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 7cc0fb89..bf8fc1e8 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -157,6 +157,70 @@ LOG_DEFINE_CATEGORY(IPAIPU3) namespace ipa::ipu3 { +/** + * \brief The IPU3 IPA implementation + * + * The IPU3 Pipeline defines an IPU3 specific interface for communication + * between the PipelineHandler, and the IPA module. + * + * We extend the IPAIPU3Interface to implement our algorithms and handle events + * from the IPU3 PipelineHandler to satisfy requests from the application. + * + * At initialisation time, a CameraSensorHelper is instantiated to support + * camera specific calculations, while the default controls are computed, and + * the algorithms are constructed and placed in an ordered list. + * + * The IPU3 ImgU operates with a grid layout to divide the overall frame into + * rectangular cells of pixels. When the IPA is configured, we determine the + * best grid for the statistics based on the pipeline handler Bayer Down Scaler + * output size. + * + * Two main events are then handled to facilitate the operation of the IPU3 ImgU + * by populating its parameter buffer, and adapting the settings of the sensor + * attached to the IPU3 CIO2 through sensor specific V4L2 controls. + * + * When the event \a EventFillParams occurs we populate the ImgU parameter + * buffer with settings to configure the device in preparation for handling the + * frame queued in the Request. + * + * When the frame has completed processing, the ImgU will generate a statistics + * buffer which is given to the IPA as part of the \a EventStatReady event. At + * this event we run the algorithms to parse the statistics and cache any + * results for the next \a EventFillParams event. + * + * The individual algorithms are split into modular components that are called + * iteratively to allow them to process statistics from the ImgU in a defined + * order. + * + * The current implementation supports three core algorithms: + * - Automatic white balance (AWB) + * - Automatic gain and exposure control (AGC) + * - Tonemapping (Gamma) + * + * AWB is implemented using a Greyworld algorithm, and calculates the red and + * blue gains to apply to generate a neutral grey frame overall. + * + * AGC is handled by calculating a histogram of the green channel to estimate an + * analogue gain and shutter time which will provide a well exposed frame. An + * IIR filter is used to smooth the changes to the sensor to reduce perceivable + * steps. + * + * The Tonemapping algorithm provides a gamma correction table to improve the + * contrast of the scene. + * + * The IPU3 ImgU has further accellerator clusters to support image quality + * improvements through bayer and temporal noise reductions, however those are + * not supported in the current implementation, and will use default settings as + * provided by the kernel driver. + * + * Demosaicing is operating on the default values and could be further optimised + * to provide improved sharpening coefficients, checker artifact removal, and + * false color correction. + * + * Additional image enhancements can be made by providing lens and sensor + * specific tuning to adapt for Black Level compensation (BLC), Lens shading + * correction (SHD) and Color correction (CCM) + */ class IPAIPU3 : public IPAIPU3Interface { public: