[{"id":20675,"web_url":"https://patchwork.libcamera.org/comment/20675/","msgid":"","date":"2021-11-04T11:45:12","subject":"Re: [libcamera-devel] [PATCH v2 2/4] ipu3: ipa: Report effective\n\tsensor controls with stastistics to IPA","submitter":{"id":75,"url":"https://patchwork.libcamera.org/api/people/75/","name":"Jean-Michel Hautbois","email":"jeanmichel.hautbois@ideasonboard.com"},"content":"Hi Han-Lin,\n\nOn 29/10/2021 13:59, Han-Lin Chen wrote:\n> The Intel close sourced IPA requires the effective controls applied to\n> the sensor when the stastistics are generated. Report effective sensor controls\n> with the stastistics to IPA.\n> \n> Signed-off-by: Han-Lin Chen \n> ---\n> src/libcamera/pipeline/ipu3/frames.h | 3 +++\n> src/libcamera/pipeline/ipu3/ipu3.cpp | 5 +++++\n> 2 files changed, 8 insertions(+)\n> \n> diff --git a/src/libcamera/pipeline/ipu3/frames.h b/src/libcamera/pipeline/ipu3/frames.h\n> index 3ef7e445..a897e307 100644\n> --- a/src/libcamera/pipeline/ipu3/frames.h\n> +++ b/src/libcamera/pipeline/ipu3/frames.h\n> @@ -12,6 +12,7 @@\n> #include \n> #include \n> \n> +#include \n> #include \n> \n> namespace libcamera {\n> @@ -34,6 +35,8 @@ public:\n> \t\tFrameBuffer *paramBuffer;\n> \t\tFrameBuffer *statBuffer;\n> \n> +\t\tControlList effectiveSensorControls;\n> +\n> \t\tbool paramDequeued;\n> \t\tbool metadataProcessed;\n> \t};\n> diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> index 8816efc5..6a7f5b9a 100644\n> --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> @@ -667,6 +667,8 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c)\n> \t\treturn ret;\n> \t}\n> \n> +\tdata->delayedCtrls_->reset();\n> +\n\nIs it a separate bug maybe ? One patch for this single line ;-) ?\n\n> \treturn updateControls(data);\n> }\n> \n> @@ -1363,6 +1365,8 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer)\n> \trequest->metadata().set(controls::SensorTimestamp,\n> \t\t\t\tbuffer->metadata().timestamp);\n> \n> +\tinfo->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence);\n> +\n> \tif (request->findBuffer(&rawStream_))\n> \t\tpipe()->completeBuffer(request, buffer);\n> \n> @@ -1419,6 +1423,7 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer)\n> \tev.frame = info->id;\n> \tev.bufferId = info->statBuffer->cookie();\n> \tev.frameTimestamp = request->metadata().get(controls::SensorTimestamp);\n> +\tev.sensorControls = info->effectiveSensorControls;\n\nIt means the IPA needs to parse the controls in its \"EventStatReady\" \nevent parsing. Fine for me, even if we expected in open IPA to use the \n\"EventProcessControls\" event for that, using a queue to store the \ncontrols per frame.\n\n> \tipa_->processEvent(ev);\n> }\n> \n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B6BBEBF415\n\tfor ;\n\tThu, 4 Nov 2021 11:45:16 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 240F76032F;\n\tThu, 4 Nov 2021 12:45:16 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id D0D206032C\n\tfor ;\n\tThu, 4 Nov 2021 12:45:14 +0100 (CET)","from [IPV6:2a01:e0a:169:7140:3875:cc64:ce12:aabc] (unknown\n\t[IPv6:2a01:e0a:169:7140:3875:cc64:ce12:aabc])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 7248F4A1;\n\tThu, 4 Nov 2021 12:45:14 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"X+Klfd/D\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1636026314;\n\tbh=ZW1Y7XGTQ0oI9AfLaSX4Q0rofZR/ymYEG6EiFDN/IjM=;\n\th=Date:Subject:To:References:From:In-Reply-To:From;\n\tb=X+Klfd/DuUuTOmSz3IL7VgkTt09LbOKNj8H18riBbO5k+e2iMRdFhHwuTlTjWwpm2\n\tRpGrbfjzBd8QXB/39vTqhq/9u31rAed/++spd0zrPK0OW5F8wPvHrTbW6fZFcq+g8S\n\tarJu6rF/W7mNsOlGclWFb2s0YaycPr9nDXqFaNPM=","Message-ID":"","Date":"Thu, 4 Nov 2021 12:45:12 +0100","MIME-Version":"1.0","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101\n\tThunderbird/91.1.2","Content-Language":"en-US","To":"Han-Lin Chen ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20211029115917.2467936-1-hanlinchen@chromium.org>\n\t<20211029115917.2467936-2-hanlinchen@chromium.org>","From":"Jean-Michel Hautbois ","In-Reply-To":"<20211029115917.2467936-2-hanlinchen@chromium.org>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [PATCH v2 2/4] ipu3: ipa: Report effective\n\tsensor controls with stastistics to IPA","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":20798,"web_url":"https://patchwork.libcamera.org/comment/20798/","msgid":"","date":"2021-11-10T12:44:17","subject":"Re: [libcamera-devel] [PATCH v2 2/4] ipu3: ipa: Report effective\n\tsensor controls with stastistics to IPA","submitter":{"id":98,"url":"https://patchwork.libcamera.org/api/people/98/","name":"Hanlin Chen","email":"hanlinchen@chromium.org"},"content":"Hi Jean-Michel,\n\nOn Thu, Nov 4, 2021 at 7:45 PM Jean-Michel Hautbois\n wrote:\n>\n> Hi Han-Lin,\n>\n> On 29/10/2021 13:59, Han-Lin Chen wrote:\n> > The Intel close sourced IPA requires the effective controls applied to\n> > the sensor when the stastistics are generated. Report effective sensor controls\n> > with the stastistics to IPA.\n> >\n> > Signed-off-by: Han-Lin Chen \n> > ---\n> > src/libcamera/pipeline/ipu3/frames.h | 3 +++\n> > src/libcamera/pipeline/ipu3/ipu3.cpp | 5 +++++\n> > 2 files changed, 8 insertions(+)\n> >\n> > diff --git a/src/libcamera/pipeline/ipu3/frames.h b/src/libcamera/pipeline/ipu3/frames.h\n> > index 3ef7e445..a897e307 100644\n> > --- a/src/libcamera/pipeline/ipu3/frames.h\n> > +++ b/src/libcamera/pipeline/ipu3/frames.h\n> > @@ -12,6 +12,7 @@\n> > #include \n> > #include \n> >\n> > +#include \n> > #include \n> >\n> > namespace libcamera {\n> > @@ -34,6 +35,8 @@ public:\n> > FrameBuffer *paramBuffer;\n> > FrameBuffer *statBuffer;\n> >\n> > + ControlList effectiveSensorControls;\n> > +\n> > bool paramDequeued;\n> > bool metadataProcessed;\n> > };\n> > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > index 8816efc5..6a7f5b9a 100644\n> > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > @@ -667,6 +667,8 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c)\n> > return ret;\n> > }\n> >\n> > + data->delayedCtrls_->reset();\n> > +\n>\n> Is it a separate bug maybe ? One patch for this single line ;-) ?\n>\n> > return updateControls(data);\n> > }\n> >\n> > @@ -1363,6 +1365,8 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer)\n> > request->metadata().set(controls::SensorTimestamp,\n> > buffer->metadata().timestamp);\n> >\n> > + info->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence);\n> > +\n> > if (request->findBuffer(&rawStream_))\n> > pipe()->completeBuffer(request, buffer);\n> >\n> > @@ -1419,6 +1423,7 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer)\n> > ev.frame = info->id;\n> > ev.bufferId = info->statBuffer->cookie();\n> > ev.frameTimestamp = request->metadata().get(controls::SensorTimestamp);\n> > + ev.sensorControls = info->effectiveSensorControls;\n>\n> It means the IPA needs to parse the controls in its \"EventStatReady\"\n> event parsing. Fine for me, even if we expected in open IPA to use the\n> \"EventProcessControls\" event for that, using a queue to store the\n> controls per frame.\n\nIt seems that they have different meanings, but only differentiated by\nthe context.\n\n1. I suppose the controls in \"EventProcessControls\" are the Controls\nfor per-frame control from application, for example, AF pre-catpure or\nmanual NR controls?\n2. On the other hand, the effective sensor controls for\n\"EventStatReady\" means the exact controls which are effective on the\nsensor\nwhen the raw frame, and hence the statistics, is generated. The series\nneeds that because the close IPA is sensitive to it.\nNot sure if it's worth changing the interface to make it clearer.\n\n>\n> > ipa_->processEvent(ev);\n> > }\n> >\n> >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 7C4ABBDB1C\n\tfor ;\n\tWed, 10 Nov 2021 12:44:31 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9CECA6035D;\n\tWed, 10 Nov 2021 13:44:30 +0100 (CET)","from mail-oi1-x229.google.com (mail-oi1-x229.google.com\n\t[IPv6:2607:f8b0:4864:20::229])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B5FC86033C\n\tfor ;\n\tWed, 10 Nov 2021 13:44:29 +0100 (CET)","by mail-oi1-x229.google.com with SMTP id o83so4986588oif.4\n\tfor ;\n\tWed, 10 Nov 2021 04:44:29 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"Dbt/IWwg\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=Clc8AkHjZjmQNwjZTW1WPuzJlRK6R9XbbD78NPMHflI=;\n\tb=Dbt/IWwgrTuVxTUqhNcoN7//dc6upJ4in2W5I31+1uYqGLCJNlqnAUu5HF8qUP15T7\n\t0CCPm/c+10QyXQlMYYNxuPv5oDYs2r8TdVYGyFHLw5MOKGWvZJDwdYRfA3KXxy+05BMr\n\t70WHobh/8yKW4zT2E01i/78tDcxQwGvxyEZH0=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=Clc8AkHjZjmQNwjZTW1WPuzJlRK6R9XbbD78NPMHflI=;\n\tb=CEs36QT7cNuu+p1hbBEdn90IAGpAncoR8PLx7gBffI+yG2UE3vvTPbOawMSxjaD7in\n\tCKTfPqNEJIeIdc2KRJ7fuRjJFkUzTRQEhHPY0JJOtcGWLjO/nhgjnOZJ8kSQ6hAJly9K\n\twJwZrqFuOydIU4m/R/Qi49O8yiEjYs3bt5egSiF9TmSWzeQDTRWv8jA3qyUafFFgPk98\n\t6YUKrIbav0OG/uhU9t0a13cj0MiwR82O25JQfhDb3p8LYEBFy/AQtpbcSCpvEQXUiGTf\n\tQdaXtGqj/0mJW9H60OLNAsHOQxuog3UpSIjMoluMekvbAfU4LBVxTDxPKsukv+7sdBi8\n\t5DJw==","X-Gm-Message-State":"AOAM532+eCihyKGkrxkA6dp/ohV4FQqW7bhC5rHcj8pvP9VDw0tcxOVA\n\tSqij1KgtBboiEVDZX3T6Uc4lJz3SAIeBvD9d9htJ6g==","X-Google-Smtp-Source":"ABdhPJwHIS4wUfasxmYW3zTWnqeXQUXe2mHxEpY3F4FmASA78dQNAmkFmLN7lYz/B3CLmIDyiqTxKhL6OC1+VlnkYbM=","X-Received":"by 2002:a05:6808:1802:: with SMTP id\n\tbh2mr1085044oib.142.1636548268255; \n\tWed, 10 Nov 2021 04:44:28 -0800 (PST)","MIME-Version":"1.0","References":"<20211029115917.2467936-1-hanlinchen@chromium.org>\n\t<20211029115917.2467936-2-hanlinchen@chromium.org>\n\t","In-Reply-To":"","From":"Hanlin Chen ","Date":"Wed, 10 Nov 2021 20:44:17 +0800","Message-ID":"","To":"Jean-Michel Hautbois ","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [PATCH v2 2/4] ipu3: ipa: Report effective\n\tsensor controls with stastistics to IPA","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]