[{"id":5375,"web_url":"https://patchwork.libcamera.org/comment/5375/","msgid":"<20200624093942.aetg63xln4audpdw@uno.localdomain>","date":"2020-06-24T09:39:42","subject":"Re: [libcamera-devel] [PATCH v3 03/10] libcamera: ipu3: Breakout\n\tstream assignment to new function","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Niklas,\n\nOn Tue, Jun 23, 2020 at 04:09:23AM +0200, Niklas Söderlund wrote:\n> Selecting which stream is the most suitable for the requested\n> configuration is mixed with adjusting the requested format when\n> validating configurations. This is hard to read and got worse when\n> support for Bayer formats was added. Break it out to a separate\n> function.\n>\n> Signed-off-by: Niklas Söderlund \n> Reviewed-by: Laurent Pinchart \n> ---\n> * Changes since v2\n> - Document the fact that the number of requested streams in config_\n> should be validated before calling assignStreams(). Add an ASSERT()\n> and comment to catch the issue early if this should ever change.\n>\n> * Changes since v1\n> - Update commit message.\n> - Rename updateStreams() to assignStreams().\n> - Revert and keep old comment on how streams are picked.\n> - Do not modify behavior on how streams are picked which means\n> assignStreams() now can't fail so no need for it to return an int,\n> switch to void.\n> ---\n> src/libcamera/pipeline/ipu3/ipu3.cpp | 88 ++++++++++++++++------------\n> 1 file changed, 52 insertions(+), 36 deletions(-)\n>\n> diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> index c525e30a5ad35011..3b2ec684244881e5 100644\n> --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> @@ -191,6 +191,7 @@ private:\n> \tstatic constexpr unsigned int IPU3_BUFFER_COUNT = 4;\n> \tstatic constexpr unsigned int IPU3_MAX_STREAMS = 3;\n>\n> +\tvoid assignStreams();\n> \tvoid adjustStream(StreamConfiguration &cfg, bool scale);\n>\n> \t/*\n> @@ -257,6 +258,50 @@ IPU3CameraConfiguration::IPU3CameraConfiguration(Camera *camera,\n> \tdata_ = data;\n> }\n>\n> +void IPU3CameraConfiguration::assignStreams()\n> +{\n> +\t/*\n> +\t * Verify and update all configuration entries, and assign a stream to\n> +\t * each of them. The viewfinder stream can scale, while the output\n> +\t * stream can crop only, so select the output stream when the requested\n> +\t * resolution is equal to the sensor resolution, and the viewfinder\n> +\t * stream otherwise.\n> +\t */\n> +\tstd::set availableStreams = {\n> +\t\t&data_->outStream_,\n> +\t\t&data_->vfStream_,\n> +\t\t&data_->rawStream_,\n> +\t};\n> +\n> +\t/*\n> +\t * The caller is responsible to limit the number of requested streams\n> +\t * to a number supported by the pipeline before calling this function.\n> +\t */\n> +\tASSERT(availableStreams.size() >= config_.size());\n> +\n> +\tstreams_.clear();\n> +\tstreams_.reserve(config_.size());\n> +\n> +\tfor (const StreamConfiguration &cfg : config_) {\n> +\t\tconst PixelFormatInfo &info =\n> +\t\t\tPixelFormatInfo::info(cfg.pixelFormat);\n> +\t\tconst IPU3Stream *stream;\n> +\n> +\t\tif (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW)\n> +\t\t\tstream = &data_->rawStream_;\n> +\t\telse if (cfg.size == sensorFormat_.size)\n> +\t\t\tstream = &data_->outStream_;\n> +\t\telse\n> +\t\t\tstream = &data_->vfStream_;\n> +\n> +\t\tif (availableStreams.find(stream) == availableStreams.end())\n> +\t\t\tstream = *availableStreams.begin();\n\n\nI understand this was here already, but it's worth pointing it out\nanyway: doesn't this allow an application to require 3 YUV streams ?\nOnce we have assigned vfStream and outStream to the first two, the\nlast one will be assigned to rawStream, which should not happen.\n\n> +\n> +\t\tstreams_.push_back(stream);\n> +\t\tavailableStreams.erase(stream);\n> +\t}\n> +}\n> +\n> void IPU3CameraConfiguration::adjustStream(StreamConfiguration &cfg, bool scale)\n> {\n> \t/* The only pixel format the driver supports is NV12. */\n> @@ -343,41 +388,14 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate()\n> \tif (!sensorFormat_.size.width || !sensorFormat_.size.height)\n> \t\tsensorFormat_.size = sensor->resolution();\n>\n> -\t/*\n> -\t * Verify and update all configuration entries, and assign a stream to\n> -\t * each of them. The viewfinder stream can scale, while the output\n> -\t * stream can crop only, so select the output stream when the requested\n> -\t * resolution is equal to the sensor resolution, and the viewfinder\n> -\t * stream otherwise.\n> -\t */\n> -\tstd::set availableStreams = {\n> -\t\t&data_->outStream_,\n> -\t\t&data_->vfStream_,\n> -\t\t&data_->rawStream_,\n> -\t};\n> -\n> -\tstreams_.clear();\n> -\tstreams_.reserve(config_.size());\n> +\t/* Assign streams to each configuration entry. */\n> +\tassignStreams();\n>\n> +\t/* Verify and adjust configuration if needed. */\n> \tfor (unsigned int i = 0; i < config_.size(); ++i) {\n> \t\tStreamConfiguration &cfg = config_[i];\n> -\t\tconst PixelFormat pixelFormat = cfg.pixelFormat;\n> -\t\tconst PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat);\n> -\t\tconst Size size = cfg.size;\n> -\t\tconst IPU3Stream *stream;\n> -\n> -\t\tif (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW)\n> -\t\t\tstream = &data_->rawStream_;\n> -\t\telse if (cfg.size == sensorFormat_.size)\n> -\t\t\tstream = &data_->outStream_;\n> -\t\telse\n> -\t\t\tstream = &data_->vfStream_;\n> -\n> -\t\tif (availableStreams.find(stream) == availableStreams.end())\n> -\t\t\tstream = *availableStreams.begin();\n> -\n> -\t\tLOG(IPU3, Debug)\n> -\t\t\t<< \"Assigned '\" << stream->name_ << \"' to stream \" << i;\n> +\t\tconst StreamConfiguration oldCfg = cfg;\n> +\t\tconst IPU3Stream *stream = streams_[i];\n>\n> \t\tif (stream->raw_) {\n> \t\t\tconst auto &itFormat =\n> @@ -394,15 +412,13 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate()\n>\n> \t\tcfg.bufferCount = IPU3_BUFFER_COUNT;\n>\n> -\t\tif (cfg.pixelFormat != pixelFormat || cfg.size != size) {\n> +\t\tif (cfg.pixelFormat != oldCfg.pixelFormat ||\n> +\t\t cfg.size != oldCfg.size) {\n> \t\t\tLOG(IPU3, Debug)\n> \t\t\t\t<< \"Stream \" << i << \" configuration adjusted to \"\n> \t\t\t\t<< cfg.toString();\n> \t\t\tstatus = Adjusted;\n> \t\t}\n> -\n> -\t\tstreams_.push_back(stream);\n> -\t\tavailableStreams.erase(stream);\n\nThe rest looks good.\nReviewed-by: Jacopo Mondi \n\nThanks\n j\n> \t}\n>\n> \treturn status;\n> --\n> 2.27.0\n>\n> _______________________________________________\n> libcamera-devel mailing list\n> libcamera-devel@lists.libcamera.org\n> https://lists.libcamera.org/listinfo/libcamera-devel","headers":{"Return-Path":"","Received":["from relay12.mail.gandi.net (relay12.mail.gandi.net\n\t[217.70.178.232])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7371B609A7\n\tfor ;\n\tWed, 24 Jun 2020 11:36:15 +0200 (CEST)","from uno.localdomain (93-34-118-233.ip49.fastwebnet.it\n\t[93.34.118.233]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay12.mail.gandi.net (Postfix) with ESMTPSA id 0DC0920000B;\n\tWed, 24 Jun 2020 09:36:14 +0000 (UTC)"],"Date":"Wed, 24 Jun 2020 11:39:42 +0200","From":"Jacopo Mondi ","To":"Niklas =?utf-8?q?S=C3=B6derlund?= ","Cc":"libcamera-devel@lists.libcamera.org","Message-ID":"<20200624093942.aetg63xln4audpdw@uno.localdomain>","References":"<20200623020930.1781469-1-niklas.soderlund@ragnatech.se>\n\t<20200623020930.1781469-4-niklas.soderlund@ragnatech.se>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20200623020930.1781469-4-niklas.soderlund@ragnatech.se>","Subject":"Re: [libcamera-devel] [PATCH v3 03/10] libcamera: ipu3: Breakout\n\tstream assignment to new function","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Wed, 24 Jun 2020 09:36:15 -0000"}},{"id":5380,"web_url":"https://patchwork.libcamera.org/comment/5380/","msgid":"<20200624234830.GN5980@pendragon.ideasonboard.com>","date":"2020-06-24T23:48:30","subject":"Re: [libcamera-devel] [PATCH v3 03/10] libcamera: ipu3: Breakout\n\tstream assignment to new function","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Jacopo,\n\nOn Wed, Jun 24, 2020 at 11:39:42AM +0200, Jacopo Mondi wrote:\n> On Tue, Jun 23, 2020 at 04:09:23AM +0200, Niklas Söderlund wrote:\n> > Selecting which stream is the most suitable for the requested\n> > configuration is mixed with adjusting the requested format when\n> > validating configurations. This is hard to read and got worse when\n> > support for Bayer formats was added. Break it out to a separate\n> > function.\n> >\n> > Signed-off-by: Niklas Söderlund \n> > Reviewed-by: Laurent Pinchart \n> > ---\n> > * Changes since v2\n> > - Document the fact that the number of requested streams in config_\n> > should be validated before calling assignStreams(). Add an ASSERT()\n> > and comment to catch the issue early if this should ever change.\n> >\n> > * Changes since v1\n> > - Update commit message.\n> > - Rename updateStreams() to assignStreams().\n> > - Revert and keep old comment on how streams are picked.\n> > - Do not modify behavior on how streams are picked which means\n> > assignStreams() now can't fail so no need for it to return an int,\n> > switch to void.\n> > ---\n> > src/libcamera/pipeline/ipu3/ipu3.cpp | 88 ++++++++++++++++------------\n> > 1 file changed, 52 insertions(+), 36 deletions(-)\n> >\n> > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > index c525e30a5ad35011..3b2ec684244881e5 100644\n> > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > @@ -191,6 +191,7 @@ private:\n> > \tstatic constexpr unsigned int IPU3_BUFFER_COUNT = 4;\n> > \tstatic constexpr unsigned int IPU3_MAX_STREAMS = 3;\n> >\n> > +\tvoid assignStreams();\n> > \tvoid adjustStream(StreamConfiguration &cfg, bool scale);\n> >\n> > \t/*\n> > @@ -257,6 +258,50 @@ IPU3CameraConfiguration::IPU3CameraConfiguration(Camera *camera,\n> > \tdata_ = data;\n> > }\n> >\n> > +void IPU3CameraConfiguration::assignStreams()\n> > +{\n> > +\t/*\n> > +\t * Verify and update all configuration entries, and assign a stream to\n> > +\t * each of them. The viewfinder stream can scale, while the output\n> > +\t * stream can crop only, so select the output stream when the requested\n> > +\t * resolution is equal to the sensor resolution, and the viewfinder\n> > +\t * stream otherwise.\n> > +\t */\n> > +\tstd::set availableStreams = {\n> > +\t\t&data_->outStream_,\n> > +\t\t&data_->vfStream_,\n> > +\t\t&data_->rawStream_,\n> > +\t};\n> > +\n> > +\t/*\n> > +\t * The caller is responsible to limit the number of requested streams\n> > +\t * to a number supported by the pipeline before calling this function.\n> > +\t */\n> > +\tASSERT(availableStreams.size() >= config_.size());\n> > +\n> > +\tstreams_.clear();\n> > +\tstreams_.reserve(config_.size());\n> > +\n> > +\tfor (const StreamConfiguration &cfg : config_) {\n> > +\t\tconst PixelFormatInfo &info =\n> > +\t\t\tPixelFormatInfo::info(cfg.pixelFormat);\n> > +\t\tconst IPU3Stream *stream;\n> > +\n> > +\t\tif (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW)\n> > +\t\t\tstream = &data_->rawStream_;\n> > +\t\telse if (cfg.size == sensorFormat_.size)\n> > +\t\t\tstream = &data_->outStream_;\n> > +\t\telse\n> > +\t\t\tstream = &data_->vfStream_;\n> > +\n> > +\t\tif (availableStreams.find(stream) == availableStreams.end())\n> > +\t\t\tstream = *availableStreams.begin();\n> \n> I understand this was here already, but it's worth pointing it out\n> anyway: doesn't this allow an application to require 3 YUV streams ?\n> Once we have assigned vfStream and outStream to the first two, the\n> last one will be assigned to rawStream, which should not happen.\n\nAgreed. It should be fixed on top though, as it's a change in behaviour.\n\n> > +\n> > +\t\tstreams_.push_back(stream);\n> > +\t\tavailableStreams.erase(stream);\n> > +\t}\n> > +}\n> > +\n> > void IPU3CameraConfiguration::adjustStream(StreamConfiguration &cfg, bool scale)\n> > {\n> > \t/* The only pixel format the driver supports is NV12. */\n> > @@ -343,41 +388,14 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate()\n> > \tif (!sensorFormat_.size.width || !sensorFormat_.size.height)\n> > \t\tsensorFormat_.size = sensor->resolution();\n> >\n> > -\t/*\n> > -\t * Verify and update all configuration entries, and assign a stream to\n> > -\t * each of them. The viewfinder stream can scale, while the output\n> > -\t * stream can crop only, so select the output stream when the requested\n> > -\t * resolution is equal to the sensor resolution, and the viewfinder\n> > -\t * stream otherwise.\n> > -\t */\n> > -\tstd::set availableStreams = {\n> > -\t\t&data_->outStream_,\n> > -\t\t&data_->vfStream_,\n> > -\t\t&data_->rawStream_,\n> > -\t};\n> > -\n> > -\tstreams_.clear();\n> > -\tstreams_.reserve(config_.size());\n> > +\t/* Assign streams to each configuration entry. */\n> > +\tassignStreams();\n> >\n> > +\t/* Verify and adjust configuration if needed. */\n> > \tfor (unsigned int i = 0; i < config_.size(); ++i) {\n> > \t\tStreamConfiguration &cfg = config_[i];\n> > -\t\tconst PixelFormat pixelFormat = cfg.pixelFormat;\n> > -\t\tconst PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat);\n> > -\t\tconst Size size = cfg.size;\n> > -\t\tconst IPU3Stream *stream;\n> > -\n> > -\t\tif (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW)\n> > -\t\t\tstream = &data_->rawStream_;\n> > -\t\telse if (cfg.size == sensorFormat_.size)\n> > -\t\t\tstream = &data_->outStream_;\n> > -\t\telse\n> > -\t\t\tstream = &data_->vfStream_;\n> > -\n> > -\t\tif (availableStreams.find(stream) == availableStreams.end())\n> > -\t\t\tstream = *availableStreams.begin();\n> > -\n> > -\t\tLOG(IPU3, Debug)\n> > -\t\t\t<< \"Assigned '\" << stream->name_ << \"' to stream \" << i;\n> > +\t\tconst StreamConfiguration oldCfg = cfg;\n> > +\t\tconst IPU3Stream *stream = streams_[i];\n> >\n> > \t\tif (stream->raw_) {\n> > \t\t\tconst auto &itFormat =\n> > @@ -394,15 +412,13 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate()\n> >\n> > \t\tcfg.bufferCount = IPU3_BUFFER_COUNT;\n> >\n> > -\t\tif (cfg.pixelFormat != pixelFormat || cfg.size != size) {\n> > +\t\tif (cfg.pixelFormat != oldCfg.pixelFormat ||\n> > +\t\t cfg.size != oldCfg.size) {\n> > \t\t\tLOG(IPU3, Debug)\n> > \t\t\t\t<< \"Stream \" << i << \" configuration adjusted to \"\n> > \t\t\t\t<< cfg.toString();\n> > \t\t\tstatus = Adjusted;\n> > \t\t}\n> > -\n> > -\t\tstreams_.push_back(stream);\n> > -\t\tavailableStreams.erase(stream);\n> \n> The rest looks good.\n> Reviewed-by: Jacopo Mondi \n> \n> > \t}\n> >\n> > \treturn status;","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 1C86AC0100\n\tfor ;\n\tWed, 24 Jun 2020 23:48:34 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id DFE3C609C3;\n\tThu, 25 Jun 2020 01:48:33 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3EB37603BE\n\tfor ;\n\tThu, 25 Jun 2020 01:48:32 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C5C412A8;\n\tThu, 25 Jun 2020 01:48:31 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"KAwn+igY\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1593042512;\n\tbh=V10IWiDoaEL2kcVS9Vd5OEUKp3CJM/VtrRKWqSeZxiA=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=KAwn+igYrHhOaNA2VDA0+wyYjrv+54eYK6/Lt8z1RtN03A/Ay1t5vrSexcCN4Xfrf\n\tmJau+c5+08m1u+NGlJY7Hj1j7HmvO/1vFI4sTlRFopITH5ifmFVq6klekHACbpaqkY\n\t5X3HCaJEEmyAL/h5lkfjYLzdEGJ9zFmSimcTaSwI=","Date":"Thu, 25 Jun 2020 02:48:30 +0300","From":"Laurent Pinchart ","To":"Jacopo Mondi ","Message-ID":"<20200624234830.GN5980@pendragon.ideasonboard.com>","References":"<20200623020930.1781469-1-niklas.soderlund@ragnatech.se>\n\t<20200623020930.1781469-4-niklas.soderlund@ragnatech.se>\n\t<20200624093942.aetg63xln4audpdw@uno.localdomain>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200624093942.aetg63xln4audpdw@uno.localdomain>","Subject":"Re: [libcamera-devel] [PATCH v3 03/10] libcamera: ipu3: Breakout\n\tstream assignment to new function","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]