[{"id":15171,"web_url":"https://patchwork.libcamera.org/comment/15171/","msgid":"","date":"2021-02-15T12:25:20","subject":"Re: [libcamera-devel] [PATCH 0/5] DelayedControls updates and fixes","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"content":"Hi all,\n\nWe think there may be still further underlying issues causing some agc\noscillations. As such please do not review this patch set until I send an\nupdate.\n\nRegards,\nNaush\n\nOn Fri, 12 Feb 2021, 11:37 am Naushir Patuck, wrote:\n\n> Hi all,\n>\n> This patchset mainly addresses some minor issues we have encountered with\n> DelayedControls\n> when running on the Raspberry Pi platform. Apologies for the slightly\n> long cover letter,\n> but I wanted to explain the problems we are seeing in a bit of detail :-)\n>\n> Patch 1/5\n> This adds the notion of priority write to fixup the known issue of\n> settings VBLANK and\n> EXPOSURE in a single batch. It is simply a port of the fix applied to\n> StaggeredCtrl\n> that had been reviewed and subsequently deprecated, so hopefully nothing\n> too controversial.\n>\n> Patch 2/5\n> This is simply a python script that I am using to debug the small problems\n> (more details\n> below) that we have encountered. It parses the DelayedControls logs and\n> nicely tabulates\n> the results to show what controls and values have been set/queued/get on\n> each frame.\n> This has helped me tremendously in identifying problems and fixing them.\n> However, this\n> may not be useful to others, so I am happy to not have this merged if\n> folks do not think\n> it is the right place to put it.\n>\n> Patch 3/5\n> Fixes a spurious write to the device on startup. The following is an\n> extract from using\n> the script to parse the logs:\n>\n> Frame Action Gain Exposure Vblank\n> 0 Write 0 [0] 52 [0] 531 [0] <<<<<\n> 0 Get 0 [0] 52 [0] 531 [0]\n> 0 Queue --- [-] --- [-] --- [-]\n> 1 Write 0 [0] --- [-] --- [-]\n> 1 Get 0 [0] 52 [0] 531 [0]\n> 1 Queue --- [-] --- [-] --- [-]\n> 2 Write --- [-] --- [-] --- [-]\n> 2 Get 0 [1] 52 [1] 531 [1]\n> 2 Queue 192 [4] 1664 [4] 531 [4]\n>\n> You can see above, on frame 0 we are writing controls to the sensor, but\n> this is unneeded.\n> This spurious write should really not happen as there is no controls\n> queued by the\n> pipeline_handler at this point. It is, however, mostly inconsequential at\n> runtime.\n>\n> Patch 4/5\n> This fixes an issue where controls queued by the pipeline handler are\n> delayed by and\n> additional frame when writing. You can see better in the parsed log:\n>\n> Frame Action Gain Exposure Vblank\n> 2 Write --- [-] --- [-] --- [-]\n> 2 Get 0 [1] 52 [1] 531 [1]\n> 2 Queue 192 [4] 1664 [4] 531 [4] <<<<<\n> 3 Write --- [-] --- [-] --- [-] <<<<<\n> 3 Get 0 [2] 52 [2] 531 [2]\n> 3 Queue 192 [5] 1664 [5] 531 [5]\n> 4 Write --- [-] 1664 [4] 531 [4] <<<<<\n> 4 Get 0 [3] 52 [3] 531 [3]\n> 4 Queue 192 [6] 1664 [6] 531 [6]\n>\n> On frame 2, we queue controls from the pipeline handler. Exposure and\n> Vblank must be\n> written one frame before gain, so you would expect them to be written on\n> frame 3 as\n> nothing else is in the queue. However, they only get written on frame 4,\n> one frame\n> later than expected.\n>\n> This is because of how DelayedControls handles \"no-op\" queue items, i.e.\n> frames where\n> we do not provide the helper with controls to use. It was adding one more\n> no-op than\n> needed, and causing an extra frame delay when setting the control on the\n> device.\n>\n> Patch 5/5\n> We had an off-by-one error when reading back values from the queues. See\n> the parsed\n> logs below:\n>\n> Frame Action Gain Exposure Vblank\n> 7 Write 192 [6] 1664 [7] 531 [7] <<<<<\n> 7 Get 192 [6] 1664 [6] 531 [6]\n> 7 Queue 210 [9] 3174 [9] 1946 [9]\n> 8 Write 192 [7] 3526 [8] 2298 [8]\n> 8 Get 192 [7] 1664 [7] 531 [7] <<<<<\n> 8 Queue 210 [10] 3174 [10] 1946 [10]\n> 9 Write 213 [8] 3174 [9] 1946 [9]\n> 9 Get 213 [8] 3526 [8] 2298 [8] <<<<<\n> 9 Queue 213 [12] 3526 [12] 2298 [12]\n>\n> On frame 7, we write exposure and vblank with values 1664 and 531\n> respectively. These\n> values take 2 frames to consume, so should be retuned to the pipeline\n> handler by the\n> DelayedControls::get() on frame 9. However, it returns on frame 8 instead.\n>\n> This only causes slight (but visible) oscillations in brightness in the\n> AGC loop as\n> the values used by the sensor are not in lock-step to what is reported by\n> DelayedControls::get().\n>\n> Hope that is all reasonably well explained :-) Any questions, please do\n> shout.\n>\n> Regards,\n> Naush\n>\n> Naushir Patuck (5):\n> libcamera: delayed_controls: Add notion of priority write\n> utils: raspberrypi: Add a DelayedControls log parser\n> libcamera: delayed_controls: Remove unneeded write when starting up\n> libcamera: delayed_controls: Remove spurious no-op queued controls\n> libcamera: delayed_controls: Fix off-by-one error in get()\n>\n> include/libcamera/internal/delayed_controls.h | 13 ++-\n> src/libcamera/delayed_controls.cpp | 72 ++++++++++------\n> src/libcamera/pipeline/ipu3/ipu3.cpp | 8 +-\n> .../pipeline/raspberrypi/raspberrypi.cpp | 13 ++-\n> src/libcamera/pipeline/rkisp1/rkisp1.cpp | 8 +-\n> test/delayed_contols.cpp | 20 ++---\n> utils/raspberrypi/delayedctrls_parse.py | 82 +++++++++++++++++++\n> 7 files changed, 162 insertions(+), 54 deletions(-)\n> create mode 100644 utils/raspberrypi/delayedctrls_parse.py\n>\n> --\n> 2.25.1\n>\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 42216BD162\n\tfor ;\n\tMon, 15 Feb 2021 12:25:36 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 80637637CA;\n\tMon, 15 Feb 2021 13:25:35 +0100 (CET)","from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com\n\t[IPv6:2a00:1450:4864:20::12b])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 563E36165D\n\tfor ;\n\tMon, 15 Feb 2021 13:25:33 +0100 (CET)","by mail-lf1-x12b.google.com with SMTP id d24so9798379lfs.8\n\tfor ;\n\tMon, 15 Feb 2021 04:25:33 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"Sc8983ea\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to;\n\tbh=GH/mWbht0eAWpnw2e+tx6Cru4X3BpeSKrO/9TIT0S3Y=;\n\tb=Sc8983eaF1zih7XjFAHZNQubfTOJ+JAXwunPvZ5YxqQkLCXItiT8rZrils9KO34b6S\n\t52nqKTCF8L71MAtWkumTpJ6V7mh28nbCEDoplSvCtDDNxa77DpIpMHmm5ZKpvOqab1G+\n\t7HMKwRSUsxr0Ftq17R1pdYEec3J5aWNJq8ElaPBIeBIkJd6t86lGXR1doRlF75+SQOmz\n\tCJ+RmG7y1QSag06G9PI6XbaQ4KQy6vxrc5Nxbf/qnZhp0CDk3UJ0MKLl9yyOwwoYMjji\n\tKwd+G/mNW5fuBTJO40TAYcHms9oAbph+JaD17HZRNoXNiBtxgG1P8iMPXB2WA19Ei4TG\n\tZU3g==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to;\n\tbh=GH/mWbht0eAWpnw2e+tx6Cru4X3BpeSKrO/9TIT0S3Y=;\n\tb=cBXBaVxMnEAMgxXeuemEK1Twfew3KFQkTIW74ycx4CnX/BKVsEr/a1wYWVbQEpJzSR\n\tRGK/Bs6SVUAeHBTDvekBNrIRF8r8i+cBYUs4UMcthOt38TldvK2QagdWkk1T8xeo2c4Y\n\tm7lMGFAEd5Nx61Q4vV5wJX+CgEwBTkF7ZwawhdGxytnkHi+dF6eXaLCUgn2l7VdYxYpj\n\tJ2FNx6oObCBMJPh3MPFwZXplWkFoICNmKnCSxgGpy1oO/ZrsN7ngshISMC6OKIkLA3zb\n\tlfPugurNF0JS05R0abU8T8oN0fYeNldzeHTmXb21jD9RjpmOP/zYxFg2VihlC4AfIW8g\n\tLG2A==","X-Gm-Message-State":"AOAM531NsmNhQiFRaKP1z7hOPUf0wUuCyH38DJDGbPza4PxGsM+z+rBw\n\tLi/QjxQQ/bLqTncAXrjU8EsoutPEhlTLJAfzMYw9Dcm2arf8mA==","X-Google-Smtp-Source":"ABdhPJy01KIXMyU9ZzLto59pQWJtlPJWu+27QvB/cvoJ8lmd58vLAbYs3QK06i2HYw7H7PGmfIOPO7BPjXHU9+2HZXU=","X-Received":"by 2002:a05:6512:114e:: with SMTP id\n\tm14mr8508288lfg.617.1613391932299; \n\tMon, 15 Feb 2021 04:25:32 -0800 (PST)","MIME-Version":"1.0","References":"<20210212113312.239076-1-naush@raspberrypi.com>","In-Reply-To":"<20210212113312.239076-1-naush@raspberrypi.com>","From":"Naushir Patuck ","Date":"Mon, 15 Feb 2021 12:25:20 +0000","Message-ID":"","To":"libcamera devel ","Subject":"Re: [libcamera-devel] [PATCH 0/5] DelayedControls updates and fixes","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"multipart/mixed;\n\tboundary=\"===============6929038259040504725==\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]