[{"id":16949,"web_url":"https://patchwork.libcamera.org/comment/16949/","msgid":"<3446978c-91f7-9375-4aca-05e20eda0301@ideasonboard.com>","date":"2021-05-14T10:38:45","subject":"Re: [libcamera-devel] [PATCH v1 6/6] libcamera: pipeline: ipu3:\n\tPass request metadata to IPA","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Me,\n\nOn 14/05/2021 08:58, Umang Jain wrote:\n> From: Kieran Bingham \n> \n> Pass the request metadata control list to the IPA when setting\n> passing the statistics buffers, or after the raw buffer completion.\n> \n> This allows us to pass in the timestamp and other data from the request\n> which will be used by the IPA.\n> \n> Signed-off-by: Kieran Bingham \n> ---\n> src/libcamera/pipeline/ipu3/ipu3.cpp | 5 ++++-\n> 1 file changed, 4 insertions(+), 1 deletion(-)\n> \n> diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> index 5b15ca90..38f66919 100644\n> --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> @@ -1285,6 +1285,7 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer)\n> \tev.op = ipa::ipu3::EventFillParams;\n> \tev.frame = info->id;\n> \tev.bufferId = info->paramBuffer->cookie();\n> +\tev.controls = request->metadata();\n> \tipa_->processEvent(ev);\n> }\n> \n> @@ -1314,6 +1315,8 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer)\n> \tif (!info)\n> \t\treturn;\n> \n> +\tRequest *request = info->request;\n> +\n> \tif (buffer->metadata().status == FrameMetadata::FrameCancelled) {\n> \t\tinfo->metadataProcessed = true;\n> \n> @@ -1321,7 +1324,6 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer)\n> \t\t* tryComplete() will delete info if it completes the IPU3Frame.\n> \t\t* In that event, we must have obtained the Request before hand.\n> \t\t*/\n> -\t\tRequest *request = info->request;\n> \n> \t\tif (frameInfos_.tryComplete(info))\n> \t\t\tpipe_->completeRequest(request);\n> @@ -1333,6 +1335,7 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer)\n> \tev.op = ipa::ipu3::EventStatReady;\n> \tev.frame = info->id;\n> \tev.bufferId = info->statBuffer->cookie();\n> +\tev.controls = request->metadata();\n\nI only wonder if we'll ever need to send both controls and metadata as\nseparate lists. And if so, I suspect we should have a distinct separate\nmetadata or properties entry in the IPA/IPC events.\n\nBut I quite dislike that we have 'controls' 'metadata' and 'properties'\nwhich are all quite ambiguous as to which is which.\n\nAre properties always metadata? Is metadata always properties? or\nsometimes the value of a control as it was set?\n\nAre the controls libcamera controls? or V4L2 controls?\n\n\n\n> \tipa_->processEvent(ev);\n> }\n> \n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id F1531C31F6\n\tfor ;\n\tFri, 14 May 2021 10:38:51 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 525EF68919;\n\tFri, 14 May 2021 12:38:51 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E3487602B5\n\tfor ;\n\tFri, 14 May 2021 12:38:49 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 2105E3E5;\n\tFri, 14 May 2021 12:38:49 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"O2x33LCJ\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1620988729;\n\tbh=d3dsNZKKoeM77X98LT40gqq6OzDl28g5XBBDakyrUj0=;\n\th=Reply-To:Subject:To:References:From:Date:In-Reply-To:From;\n\tb=O2x33LCJpK0JjusKaQNgnBFenV559j9EYRbDCkIdwmfqsDEwtXXEuvSoKrDozBoVK\n\tkTRMRZJawDrVSk+/FJQ//Eu6td0diEMtlbrkHs2G1msF5rp+BRYTqHYasLCl0XhReX\n\tIi8m5Xz86bDcNCkfsRFKvd3yyXSanpGHIfI/Z6ws=","To":"Umang Jain ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20210514075808.282479-1-umang.jain@ideasonboard.com>\n\t<20210514075808.282479-7-umang.jain@ideasonboard.com>","From":"Kieran Bingham ","Organization":"Ideas on Board","Message-ID":"<3446978c-91f7-9375-4aca-05e20eda0301@ideasonboard.com>","Date":"Fri, 14 May 2021 11:38:45 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.7.1","MIME-Version":"1.0","In-Reply-To":"<20210514075808.282479-7-umang.jain@ideasonboard.com>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH v1 6/6] libcamera: pipeline: ipu3:\n\tPass request metadata to IPA","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":16959,"web_url":"https://patchwork.libcamera.org/comment/16959/","msgid":"","date":"2021-05-15T16:55:45","subject":"Re: [libcamera-devel] [PATCH v1 6/6] libcamera: pipeline: ipu3:\n\tPass request metadata to IPA","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Kieran,\n\nOn Fri, May 14, 2021 at 11:38:45AM +0100, Kieran Bingham wrote:\n> On 14/05/2021 08:58, Umang Jain wrote:\n> > From: Kieran Bingham \n> > \n> > Pass the request metadata control list to the IPA when setting\n> > passing the statistics buffers, or after the raw buffer completion.\n> > \n> > This allows us to pass in the timestamp and other data from the request\n> > which will be used by the IPA.\n\nMetadata are reported by the camera to applications through the Request\nclass. While the pipeline handler may store information in\nRequest::metadata() that would be of interest to the IPA, I'd rather\npass those pieces of information explicitly than passing\nRequest::metadata(), as conceptually speaking, Request::metadata() is\nfilled by the IPA, not passed to the IPA.\n\n> > Signed-off-by: Kieran Bingham \n> > ---\n> > src/libcamera/pipeline/ipu3/ipu3.cpp | 5 ++++-\n> > 1 file changed, 4 insertions(+), 1 deletion(-)\n> > \n> > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > index 5b15ca90..38f66919 100644\n> > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > @@ -1285,6 +1285,7 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer)\n> > \tev.op = ipa::ipu3::EventFillParams;\n> > \tev.frame = info->id;\n> > \tev.bufferId = info->paramBuffer->cookie();\n> > +\tev.controls = request->metadata();\n> > \tipa_->processEvent(ev);\n> > }\n> > \n> > @@ -1314,6 +1315,8 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer)\n> > \tif (!info)\n> > \t\treturn;\n> > \n> > +\tRequest *request = info->request;\n> > +\n> > \tif (buffer->metadata().status == FrameMetadata::FrameCancelled) {\n> > \t\tinfo->metadataProcessed = true;\n> > \n> > @@ -1321,7 +1324,6 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer)\n> > \t\t* tryComplete() will delete info if it completes the IPU3Frame.\n> > \t\t* In that event, we must have obtained the Request before hand.\n> > \t\t*/\n> > -\t\tRequest *request = info->request;\n> > \n> > \t\tif (frameInfos_.tryComplete(info))\n> > \t\t\tpipe_->completeRequest(request);\n> > @@ -1333,6 +1335,7 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer)\n> > \tev.op = ipa::ipu3::EventStatReady;\n> > \tev.frame = info->id;\n> > \tev.bufferId = info->statBuffer->cookie();\n> > +\tev.controls = request->metadata();\n> \n> I only wonder if we'll ever need to send both controls and metadata as\n> separate lists. And if so, I suspect we should have a distinct separate\n> metadata or properties entry in the IPA/IPC events.\n> \n> But I quite dislike that we have 'controls' 'metadata' and 'properties'\n> which are all quite ambiguous as to which is which.\n> \n> Are properties always metadata? Is metadata always properties? or\n> sometimes the value of a control as it was set?\n\nIf you're talking about the Camera properties and the Request controls\nand metadata, I think it's fairly well-defined (but if it's not\nwell-documented, patches are welcome :-)). Properties are \"semi-static\"\ninformation about a camera (they used to be static, but we've allowed\nthem to change in response to Camera::configure(), which I don't like\nmuch and would like to revisit at some point). Controls are passed by\nthe application to the camera, and metadata are reported by the camera\nto the application. property_ids.yaml lists the possibly properties,\nwhile control_ids.yaml lists the possible controls and metadata. I\ndon't like the split that much, and I've considered merging the files\ntogether, with an additional YAML property for each entry to tell\nwhether a given entry can be used as a property, a control and/or\nmetadata.\n\n> Are the controls libcamera controls? or V4L2 controls?\n\nWhich controls ?\n\n> > \tipa_->processEvent(ev);\n> > }","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id AFA4EC31FB\n\tfor ;\n\tSat, 15 May 2021 16:55:56 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 298926891B;\n\tSat, 15 May 2021 18:55:56 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E501F6891A\n\tfor ;\n\tSat, 15 May 2021 18:55:54 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 55C8D436;\n\tSat, 15 May 2021 18:55:54 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"XemwMp4b\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1621097754;\n\tbh=Ac0QNUwZ+S9OCE1wSWLev9oB72wW9eAYlVtQym9AJQc=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=XemwMp4bYRlYAh0bwBaQPLBEVWunNwqABbBZzCjfSXrftbkQdfibByFCpaAp36N5X\n\tH3yYmihk/o7tqLr317XDVEWkR6jJFHZbX9XfjtD32B6XCxUsrd5qi0tG+5OhxwVz45\n\tEetfodJDc5D6tHeJ3cud9/uQ7hu2SVIsfxWbnIYw=","Date":"Sat, 15 May 2021 19:55:45 +0300","From":"Laurent Pinchart ","To":"Kieran Bingham ","Message-ID":"","References":"<20210514075808.282479-1-umang.jain@ideasonboard.com>\n\t<20210514075808.282479-7-umang.jain@ideasonboard.com>\n\t<3446978c-91f7-9375-4aca-05e20eda0301@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<3446978c-91f7-9375-4aca-05e20eda0301@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v1 6/6] libcamera: pipeline: ipu3:\n\tPass request metadata to IPA","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]