[{"id":16458,"web_url":"https://patchwork.libcamera.org/comment/16458/","msgid":"<20210421141531.wyia7nw6chvr44rn@uno.localdomain>","date":"2021-04-21T14:15:31","subject":"Re: [libcamera-devel] [PATCH v5 0/4] Implement IPA algorithms and\n\tdemo with IPU3","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hello,\n\nOn Fri, Apr 16, 2021 at 09:49:05AM +0200, Jean-Michel Hautbois wrote:\n> Here is v5 of the IPA demonstration on IPU3.\n> It demonstrates a conjunction of automatic exposure (AGC)\n> and automatic white balance (AWB).\n>\n> The skeleton are extracted from RPi controller and probably subject to\n> modifications and discussions :\n> - are the mandatory methods really those ?\n> - how to exchange data between algorithms ?\n> - metadata like RPi?\n> - IPC ?\n> - public methods?\n> - controller API ?\n> - is the controller needed at all ?\n>\n> The implementation in IPU3 is tested on SGo2.\n> It is not perfect but shows a really better behaviour than default.\n> Right now, the AGC algorithm is part of the RPi one, adapted to the\n> stats and parameters we have in IPU3 on SGo2.\n\nI've run CTS on master with this series applied on top (testing\nLIMITED mode then).\n\nI have an additional test that fails\n- android.hardware.camera2.cts.AllocationTest#testAllocationFromCameraFlexibleYuv\n\nIt occasionally fails on master too.\n\nThe test tries to capture in all supported YUV formats, and check if\nthe \"input buffer\" (in my understanding what would be sent to display\nif any surface was allocated) is not all 0s\n\nIt fails when testing the result from 320x240 and I'm tempted to postpone\nthis to when 320x240 is produced by post-processing as we know the\nImgU has not been validated for resolutions < VGA.\n\nI would say\nTested-by: Jacopo Mondi \n\nThanks\n j\n\n> Changes in v5:\n> - adress multiple issues found at review time\n> - use RPi grey-world AWB algorithm and correct gains\n> - prepare for a more generic ISP stat usage\n>\n> Changes in v4:\n> - removed unneeded patch setting exposure and gains\n> - improve documentation of Histogram class\n> - take review comments\n>\n> Changes in v3:\n> - introduce histogram class (for AGC)\n> - add a patch to introduce parameter context saving\n> - changes in AGC to simplify it using part of RPi work\n> - changes in the AWB loop\n> - calculate the grid for the parameters using the BDS size from pipeline\n> handler\n>\n> In v2:\n> - splits AWB and AEC/AGC algorithms.\n> - remove unused Controller\n> - remove unused variables/methods\n> - rename most of the class members\n>\n> Jean-Michel Hautbois (4):\n> ipa: Add a common interface for algorithm objects\n> ipa: ipu3: Add a histogram class\n> ipa: ipu3: Add support for IPU3 AWB algorithm\n> ipa: ipu3: Add support for IPU3 AEC/AGC algorithm\n>\n> src/ipa/ipu3/ipu3.cpp | 96 +++++++++-\n> src/ipa/ipu3/ipu3_agc.cpp | 206 ++++++++++++++++++++\n> src/ipa/ipu3/ipu3_agc.h | 62 +++++++\n> src/ipa/ipu3/ipu3_awb.cpp | 351 +++++++++++++++++++++++++++++++++++\n> src/ipa/ipu3/ipu3_awb.h | 91 +++++++++\n> src/ipa/ipu3/meson.build | 8 +-\n> src/ipa/libipa/algorithm.cpp | 39 ++++\n> src/ipa/libipa/algorithm.h | 24 +++\n> src/ipa/libipa/histogram.cpp | 150 +++++++++++++++\n> src/ipa/libipa/histogram.h | 40 ++++\n> src/ipa/libipa/meson.build | 6 +-\n> 11 files changed, 1063 insertions(+), 10 deletions(-)\n> create mode 100644 src/ipa/ipu3/ipu3_agc.cpp\n> create mode 100644 src/ipa/ipu3/ipu3_agc.h\n> create mode 100644 src/ipa/ipu3/ipu3_awb.cpp\n> create mode 100644 src/ipa/ipu3/ipu3_awb.h\n> create mode 100644 src/ipa/libipa/algorithm.cpp\n> create mode 100644 src/ipa/libipa/algorithm.h\n> create mode 100644 src/ipa/libipa/histogram.cpp\n> create mode 100644 src/ipa/libipa/histogram.h\n>\n> --\n> 2.27.0\n>\n> _______________________________________________\n> libcamera-devel mailing list\n> libcamera-devel@lists.libcamera.org\n> https://lists.libcamera.org/listinfo/libcamera-devel","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 7FDFDBDB16\n\tfor ;\n\tWed, 21 Apr 2021 14:14:52 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 0FC2868849;\n\tWed, 21 Apr 2021 16:14:52 +0200 (CEST)","from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net\n\t[217.70.183.197])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 21DAF68835\n\tfor ;\n\tWed, 21 Apr 2021 16:14:51 +0200 (CEST)","from uno.localdomain (93-34-118-233.ip49.fastwebnet.it\n\t[93.34.118.233]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay5-d.mail.gandi.net (Postfix) with ESMTPSA id 998F41C0004;\n\tWed, 21 Apr 2021 14:14:50 +0000 (UTC)"],"X-Originating-IP":"93.34.118.233","Date":"Wed, 21 Apr 2021 16:15:31 +0200","From":"Jacopo Mondi ","To":"Jean-Michel Hautbois ","Message-ID":"<20210421141531.wyia7nw6chvr44rn@uno.localdomain>","References":"<20210416074909.24218-1-jeanmichel.hautbois@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20210416074909.24218-1-jeanmichel.hautbois@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v5 0/4] Implement IPA algorithms and\n\tdemo with IPU3","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":16461,"web_url":"https://patchwork.libcamera.org/comment/16461/","msgid":"","date":"2021-04-21T14:49:27","subject":"Re: [libcamera-devel] [PATCH v5 0/4] Implement IPA algorithms and\n\tdemo with IPU3","submitter":{"id":75,"url":"https://patchwork.libcamera.org/api/people/75/","name":"Jean-Michel Hautbois","email":"jeanmichel.hautbois@ideasonboard.com"},"content":"Hi Jacopo,\n\nOn 21/04/2021 16:15, Jacopo Mondi wrote:\n> Hello,\n> \n> On Fri, Apr 16, 2021 at 09:49:05AM +0200, Jean-Michel Hautbois wrote:\n>> Here is v5 of the IPA demonstration on IPU3.\n>> It demonstrates a conjunction of automatic exposure (AGC)\n>> and automatic white balance (AWB).\n>>\n>> The skeleton are extracted from RPi controller and probably subject to\n>> modifications and discussions :\n>> - are the mandatory methods really those ?\n>> - how to exchange data between algorithms ?\n>> - metadata like RPi?\n>> - IPC ?\n>> - public methods?\n>> - controller API ?\n>> - is the controller needed at all ?\n>>\n>> The implementation in IPU3 is tested on SGo2.\n>> It is not perfect but shows a really better behaviour than default.\n>> Right now, the AGC algorithm is part of the RPi one, adapted to the\n>> stats and parameters we have in IPU3 on SGo2.\n> \n> I've run CTS on master with this series applied on top (testing\n> LIMITED mode then).\n> \n> I have an additional test that fails\n> - android.hardware.camera2.cts.AllocationTest#testAllocationFromCameraFlexibleYuv\n> \n> It occasionally fails on master too.\n> \n> The test tries to capture in all supported YUV formats, and check if\n> the \"input buffer\" (in my understanding what would be sent to display\n> if any surface was allocated) is not all 0s\n\nOne thing we are not doing right now, but would make sense, is dropping\nframes while AGC has not converged. It would make the first frame ok for\nthe test, if everything is fine :-).\n\n> It fails when testing the result from 320x240 and I'm tempted to postpone\n> this to when 320x240 is produced by post-processing as we know the\n> ImgU has not been validated for resolutions < VGA.\n> \n> I would say\n> Tested-by: Jacopo Mondi \n> \n> Thanks\n> j\n> \n>> Changes in v5:\n>> - adress multiple issues found at review time\n>> - use RPi grey-world AWB algorithm and correct gains\n>> - prepare for a more generic ISP stat usage\n>>\n>> Changes in v4:\n>> - removed unneeded patch setting exposure and gains\n>> - improve documentation of Histogram class\n>> - take review comments\n>>\n>> Changes in v3:\n>> - introduce histogram class (for AGC)\n>> - add a patch to introduce parameter context saving\n>> - changes in AGC to simplify it using part of RPi work\n>> - changes in the AWB loop\n>> - calculate the grid for the parameters using the BDS size from pipeline\n>> handler\n>>\n>> In v2:\n>> - splits AWB and AEC/AGC algorithms.\n>> - remove unused Controller\n>> - remove unused variables/methods\n>> - rename most of the class members\n>>\n>> Jean-Michel Hautbois (4):\n>> ipa: Add a common interface for algorithm objects\n>> ipa: ipu3: Add a histogram class\n>> ipa: ipu3: Add support for IPU3 AWB algorithm\n>> ipa: ipu3: Add support for IPU3 AEC/AGC algorithm\n>>\n>> src/ipa/ipu3/ipu3.cpp | 96 +++++++++-\n>> src/ipa/ipu3/ipu3_agc.cpp | 206 ++++++++++++++++++++\n>> src/ipa/ipu3/ipu3_agc.h | 62 +++++++\n>> src/ipa/ipu3/ipu3_awb.cpp | 351 +++++++++++++++++++++++++++++++++++\n>> src/ipa/ipu3/ipu3_awb.h | 91 +++++++++\n>> src/ipa/ipu3/meson.build | 8 +-\n>> src/ipa/libipa/algorithm.cpp | 39 ++++\n>> src/ipa/libipa/algorithm.h | 24 +++\n>> src/ipa/libipa/histogram.cpp | 150 +++++++++++++++\n>> src/ipa/libipa/histogram.h | 40 ++++\n>> src/ipa/libipa/meson.build | 6 +-\n>> 11 files changed, 1063 insertions(+), 10 deletions(-)\n>> create mode 100644 src/ipa/ipu3/ipu3_agc.cpp\n>> create mode 100644 src/ipa/ipu3/ipu3_agc.h\n>> create mode 100644 src/ipa/ipu3/ipu3_awb.cpp\n>> create mode 100644 src/ipa/ipu3/ipu3_awb.h\n>> create mode 100644 src/ipa/libipa/algorithm.cpp\n>> create mode 100644 src/ipa/libipa/algorithm.h\n>> create mode 100644 src/ipa/libipa/histogram.cpp\n>> create mode 100644 src/ipa/libipa/histogram.h\n>>\n>> --\n>> 2.27.0\n>>\n>> _______________________________________________\n>> libcamera-devel mailing list\n>> libcamera-devel@lists.libcamera.org\n>> https://lists.libcamera.org/listinfo/libcamera-devel\n> _______________________________________________\n> libcamera-devel mailing list\n> libcamera-devel@lists.libcamera.org\n> https://lists.libcamera.org/listinfo/libcamera-devel\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8ABDBBDB15\n\tfor ;\n\tWed, 21 Apr 2021 14:49:30 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 0A3CC6884B;\n\tWed, 21 Apr 2021 16:49:30 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 96D4D68835\n\tfor ;\n\tWed, 21 Apr 2021 16:49:28 +0200 (CEST)","from [IPv6:2a01:e0a:169:7140:d802:535e:7770:5328] (unknown\n\t[IPv6:2a01:e0a:169:7140:d802:535e:7770:5328])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 1845D4AE;\n\tWed, 21 Apr 2021 16:49:28 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"qM22lnoc\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1619016568;\n\tbh=/mm/Jl6urJijOx5mLTR+pIAjNnw7FngjcV9/flzvnwc=;\n\th=Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=qM22lnoc4508iaILr8wF8ztILPUx2h5ssJHWNgc76I3EDPw6aWMqUeqBMOw41Bcd2\n\tSNnAOBfTNrMg3Bk8TdYjPL0zQuaYweqWrKwkMoySy5fveLU6xM9WGGWPr/bPv2FVLl\n\t1aft/hgQVo0jxt+SYTvFDdXT8AoriArEF2ZbIYOc=","To":"Jacopo Mondi ,\n\tJean-Michel Hautbois ","References":"<20210416074909.24218-1-jeanmichel.hautbois@ideasonboard.com>\n\t<20210421141531.wyia7nw6chvr44rn@uno.localdomain>","From":"Jean-Michel Hautbois ","Message-ID":"","Date":"Wed, 21 Apr 2021 16:49:27 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.7.1","MIME-Version":"1.0","In-Reply-To":"<20210421141531.wyia7nw6chvr44rn@uno.localdomain>","Content-Language":"en-US","Subject":"Re: [libcamera-devel] [PATCH v5 0/4] Implement IPA algorithms and\n\tdemo with IPU3","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]