[{"id":4959,"web_url":"https://patchwork.libcamera.org/comment/4959/","msgid":"<20200602095358.pk6jnzgocxxonsna@uno.localdomain>","date":"2020-06-02T09:53:58","subject":"Re: [libcamera-devel] [PATCH 04/10] libcamera: ipu3: Breakout\n\tstream assignment to new function","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Niklas,\n\nOn Tue, Jun 02, 2020 at 03:39:03AM +0200, Niklas Söderlund wrote:\n> Picking which stream that is most suitable for the requested\n\ns/Picking/Selecting\ns/that is/is the/\n\n> configuration is mixed with adjusting the requested format when\n> validating configurations. This is hard to read and got worse when\n> support for Bayer formats where added, break it out into a separate\ns/where/were or better, was\n\n> function.\n>\n> Signed-off-by: Niklas Söderlund \n> ---\n> src/libcamera/pipeline/ipu3/ipu3.cpp | 87 +++++++++++++++++-----------\n> 1 file changed, 53 insertions(+), 34 deletions(-)\n>\n> diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> index f7363244e1d2d0ff..0e7555c716b36749 100644\n> --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> @@ -190,6 +190,7 @@ private:\n> \tstatic constexpr unsigned int IPU3_BUFFER_COUNT = 4;\n> \tstatic constexpr unsigned int IPU3_MAX_STREAMS = 3;\n>\n> +\tint updateStreams();\n> \tvoid adjustStream(StreamConfiguration &cfg, bool scale);\n>\n> \t/*\n> @@ -256,6 +257,51 @@ IPU3CameraConfiguration::IPU3CameraConfiguration(Camera *camera,\n> \tdata_ = data;\n> }\n>\n> +int IPU3CameraConfiguration::updateStreams()\n> +{\n> +\tstd::set availableStreams = {\n> +\t\t&data_->outStream_,\n> +\t\t&data_->vfStream_,\n> +\t\t&data_->rawStream_,\n> +\t};\n> +\n> +\t/* Pick the stream most suitable for the requested configuration. */\n> +\tstd::vector streams;\n> +\tfor (unsigned int i = 0; i < config_.size(); ++i) {\n\nCan't you iterate on config_ ?\n\n> +\t\tconst StreamConfiguration &cfg = config_[i];\n> +\t\tconst IPU3Stream *stream;\n> +\n> +\t\t/*\n> +\t\t * Only the raw stream can support Bayer formats.\n\nFits in one line\n\n> +\t\t */\n> +\t\tif (cfg.pixelFormat.modifier() == IPU3_FORMAT_MOD_PACKED)\n> +\t\t\tstream = &data_->rawStream_;\nEmply line ?\n\n> +\t\t/*\n> +\t\t * Output stream can't scale so can only be used if the size\n> +\t\t * matches the size of the sensor.\n> +\t\t *\n> +\t\t * NOTE: It can crop but is not supported.\n\n\\todo ?\n\nAnd I would keep the original comment. We can crop, the issue is that\nthe FOV will be reduce when asking for a lower resolution than the\none of the frame input the IMGU if I'm not mistaken.\n\n> +\t\t */\n> +\t\telse if (cfg.size == sensorFormat_.size)\n> +\t\t\tstream = &data_->outStream_;\n> +\t\t/*\n> +\t\t * Pick the view finder stream last as it may scale.\n\nfits in one line ?\n\n> +\t\t */\n> +\t\telse\n> +\t\t\tstream = &data_->vfStream_;\n> +\n> +\t\tif (availableStreams.find(stream) == availableStreams.end())\n> +\t\t\treturn -EINVAL;\n\nIs this right ? Shouldn't we fall back to use the first available\nstream if any ?\n\nI would re-structure the code as\n\nfor (config : config_) {\n\n if (raw) {\n it = availableStreams.find(rawStream);\n if (it == end()) {\n // Fail as we can't provide two raw streams.\n }\n\n availagleStreams.erase(rawStream);\n continue;\n }\n\n /* Handle non-raw streams. */\n if (size == sensorSize)\n stream = output;\n else\n stream = viewfinder;\n\n if (availableStream.find(stream) == end())\n stream = begin()\n\n availableStreams.erase(stream);\n}\n\n\n}\n> +\n> +\t\tstreams.push_back(stream);\n> +\t\tavailableStreams.erase(stream);\n> +\t}\n> +\n> +\tstreams_ = streams;\n\nWhy don't you fill streams_ directly ?\n\n> +\n> +\treturn 0;\n> +}\n> +\n> void IPU3CameraConfiguration::adjustStream(StreamConfiguration &cfg, bool scale)\n> {\n> \t/* The only pixel format the driver supports is NV12. */\n> @@ -342,40 +388,16 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate()\n> \tif (!sensorFormat_.size.width || !sensorFormat_.size.height)\n> \t\tsensorFormat_.size = sensor->resolution();\n>\n> -\t/*\n> -\t * Verify and update all configuration entries, and assign a stream to\n> -\t * each of them. The viewfinder stream can scale, while the output\n> -\t * stream can crop only, so select the output stream when the requested\n> -\t * resolution is equal to the sensor resolution, and the viewfinder\n> -\t * stream otherwise.\n> -\t */\n> -\tstd::set availableStreams = {\n> -\t\t&data_->outStream_,\n> -\t\t&data_->vfStream_,\n> -\t\t&data_->rawStream_,\n> -\t};\n>\n\nThis results in two empty lines. Doesn't checkstyle report that ?\n\n> -\tstreams_.clear();\n> -\tstreams_.reserve(config_.size());\n> +\t/* Assign streams to each configuration entry. */\n> +\tif (updateStreams())\n> +\t\treturn Invalid;\n>\n> +\t/* Verify and adjust configuration if needed. */\n> \tfor (unsigned int i = 0; i < config_.size(); ++i) {\n\nCan you iterate on config_ ?\n\n> \t\tStreamConfiguration &cfg = config_[i];\n> -\t\tconst PixelFormat pixelFormat = cfg.pixelFormat;\n> -\t\tconst Size size = cfg.size;\n> -\t\tconst IPU3Stream *stream;\n> -\n> -\t\tif (cfg.pixelFormat.modifier() == IPU3_FORMAT_MOD_PACKED)\n> -\t\t\tstream = &data_->rawStream_;\n> -\t\telse if (cfg.size == sensorFormat_.size)\n> -\t\t\tstream = &data_->outStream_;\n> -\t\telse\n> -\t\t\tstream = &data_->vfStream_;\n> -\n> -\t\tif (availableStreams.find(stream) == availableStreams.end())\n> -\t\t\tstream = *availableStreams.begin();\n> -\n> -\t\tLOG(IPU3, Debug)\n> -\t\t\t<< \"Assigned '\" << stream->name_ << \"' to stream \" << i;\n> +\t\tconst StreamConfiguration org = cfg;\n> +\t\tconst IPU3Stream *stream = streams_[i];\n\nAh no, you need i here.\nHowever, associating on index seems a bit weak, and as we can't assign\nstreams to stream configurations here, have you considered having\nupdateStreams() return a map of configs to streams ?\n\nThanks\n j\n\n>\n> \t\tif (stream->raw_) {\n> \t\t\tconst auto &itFormat =\n> @@ -392,15 +414,12 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate()\n>\n> \t\tcfg.bufferCount = IPU3_BUFFER_COUNT;\n>\n> -\t\tif (cfg.pixelFormat != pixelFormat || cfg.size != size) {\n> +\t\tif (cfg.pixelFormat != org.pixelFormat || cfg.size != org.size) {\n> \t\t\tLOG(IPU3, Debug)\n> \t\t\t\t<< \"Stream \" << i << \" configuration adjusted to \"\n> \t\t\t\t<< cfg.toString();\n> \t\t\tstatus = Adjusted;\n> \t\t}\n> -\n> -\t\tstreams_.push_back(stream);\n> -\t\tavailableStreams.erase(stream);\n> \t}\n>\n> \treturn status;\n> --\n> 2.26.2\n>\n> _______________________________________________\n> libcamera-devel mailing list\n> libcamera-devel@lists.libcamera.org\n> https://lists.libcamera.org/listinfo/libcamera-devel","headers":{"Return-Path":"","Received":["from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net\n\t[217.70.183.199])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 81B27603CA\n\tfor ;\n\tTue, 2 Jun 2020 11:50:37 +0200 (CEST)","from uno.localdomain (2-224-242-101.ip172.fastwebnet.it\n\t[2.224.242.101]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 0BEFEFF814;\n\tTue, 2 Jun 2020 09:50:36 +0000 (UTC)"],"X-Originating-IP":"2.224.242.101","Date":"Tue, 2 Jun 2020 11:53:58 +0200","From":"Jacopo Mondi ","To":"Niklas =?utf-8?q?S=C3=B6derlund?= ","Cc":"libcamera-devel@lists.libcamera.org","Message-ID":"<20200602095358.pk6jnzgocxxonsna@uno.localdomain>","References":"<20200602013909.3170593-1-niklas.soderlund@ragnatech.se>\n\t<20200602013909.3170593-5-niklas.soderlund@ragnatech.se>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20200602013909.3170593-5-niklas.soderlund@ragnatech.se>","Subject":"Re: [libcamera-devel] [PATCH 04/10] libcamera: ipu3: Breakout\n\tstream assignment to new function","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Tue, 02 Jun 2020 09:50:37 -0000"}},{"id":5071,"web_url":"https://patchwork.libcamera.org/comment/5071/","msgid":"<20200605215802.GJ26752@pendragon.ideasonboard.com>","date":"2020-06-05T21:58:02","subject":"Re: [libcamera-devel] [PATCH 04/10] libcamera: ipu3: Breakout\n\tstream assignment to new function","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hello,\n\nOn Tue, Jun 02, 2020 at 11:53:58AM +0200, Jacopo Mondi wrote:\n> On Tue, Jun 02, 2020 at 03:39:03AM +0200, Niklas Söderlund wrote:\n> > Picking which stream that is most suitable for the requested\n> \n> s/Picking/Selecting\n> s/that is/is the/\n> \n> > configuration is mixed with adjusting the requested format when\n> > validating configurations. This is hard to read and got worse when\n> > support for Bayer formats where added, break it out into a separate\n>\n> s/where/were or better, was\n\nAnd s/into/to/\n\n> > function.\n> >\n> > Signed-off-by: Niklas Söderlund \n> > ---\n> > src/libcamera/pipeline/ipu3/ipu3.cpp | 87 +++++++++++++++++-----------\n> > 1 file changed, 53 insertions(+), 34 deletions(-)\n> >\n> > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > index f7363244e1d2d0ff..0e7555c716b36749 100644\n> > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > @@ -190,6 +190,7 @@ private:\n> > \tstatic constexpr unsigned int IPU3_BUFFER_COUNT = 4;\n> > \tstatic constexpr unsigned int IPU3_MAX_STREAMS = 3;\n> >\n> > +\tint updateStreams();\n> > \tvoid adjustStream(StreamConfiguration &cfg, bool scale);\n> >\n> > \t/*\n> > @@ -256,6 +257,51 @@ IPU3CameraConfiguration::IPU3CameraConfiguration(Camera *camera,\n> > \tdata_ = data;\n> > }\n> >\n> > +int IPU3CameraConfiguration::updateStreams()\n\nShould this be called assignStreams() ?\n\n> > +{\n> > +\tstd::set availableStreams = {\n> > +\t\t&data_->outStream_,\n> > +\t\t&data_->vfStream_,\n> > +\t\t&data_->rawStream_,\n> > +\t};\n> > +\n> > +\t/* Pick the stream most suitable for the requested configuration. */\n> > +\tstd::vector streams;\n> > +\tfor (unsigned int i = 0; i < config_.size(); ++i) {\n> \n> Can't you iterate on config_ ?\n> \n> > +\t\tconst StreamConfiguration &cfg = config_[i];\n> > +\t\tconst IPU3Stream *stream;\n> > +\n> > +\t\t/*\n> > +\t\t * Only the raw stream can support Bayer formats.\n> \n> Fits in one line\n> \n> > +\t\t */\n> > +\t\tif (cfg.pixelFormat.modifier() == IPU3_FORMAT_MOD_PACKED)\n> > +\t\t\tstream = &data_->rawStream_;\n> Emply line ?\n> \n> > +\t\t/*\n> > +\t\t * Output stream can't scale so can only be used if the size\n> > +\t\t * matches the size of the sensor.\n> > +\t\t *\n> > +\t\t * NOTE: It can crop but is not supported.\n> \n> \\todo ?\n> \n> And I would keep the original comment. We can crop, the issue is that\n> the FOV will be reduce when asking for a lower resolution than the\n> one of the frame input the IMGU if I'm not mistaken.\n> \n> > +\t\t */\n> > +\t\telse if (cfg.size == sensorFormat_.size)\n> > +\t\t\tstream = &data_->outStream_;\n> > +\t\t/*\n> > +\t\t * Pick the view finder stream last as it may scale.\n> \n> fits in one line ?\n> \n> > +\t\t */\n> > +\t\telse\n> > +\t\t\tstream = &data_->vfStream_;\n> > +\n> > +\t\tif (availableStreams.find(stream) == availableStreams.end())\n> > +\t\t\treturn -EINVAL;\n> \n> Is this right ? Shouldn't we fall back to use the first available\n> stream if any ?\n\nAt least that's what the code used to do, and I think it's best to keep\nthat behaviour in this patch. If a change of behaviour is desired, it\nshould be implemented in a separate patch.\n\nI would make availableStreams a vector to support this, as otherwise\navailableStreams.begin() will return a random stream (not truly random,\nbut std::set is sorted using value comparison, and values are pointers\nhere).\n\n> I would re-structure the code as\n> \n> for (config : config_) {\n> \n> if (raw) {\n> it = availableStreams.find(rawStream);\n> if (it == end()) {\n> // Fail as we can't provide two raw streams.\n> }\n> \n> availagleStreams.erase(rawStream);\n> continue;\n> }\n\nThis is also a change of behaviour that should be discussed in a\nseparate patch.\n\n> \n> /* Handle non-raw streams. */\n> if (size == sensorSize)\n> stream = output;\n> else\n> stream = viewfinder;\n> \n> if (availableStream.find(stream) == end())\n> stream = begin()\n> \n> availableStreams.erase(stream);\n> }\n> \n> \n> }\n> > +\n> > +\t\tstreams.push_back(stream);\n> > +\t\tavailableStreams.erase(stream);\n> > +\t}\n> > +\n> > +\tstreams_ = streams;\n> \n> Why don't you fill streams_ directly ?\n> \n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > void IPU3CameraConfiguration::adjustStream(StreamConfiguration &cfg, bool scale)\n> > {\n> > \t/* The only pixel format the driver supports is NV12. */\n> > @@ -342,40 +388,16 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate()\n> > \tif (!sensorFormat_.size.width || !sensorFormat_.size.height)\n> > \t\tsensorFormat_.size = sensor->resolution();\n> >\n> > -\t/*\n> > -\t * Verify and update all configuration entries, and assign a stream to\n> > -\t * each of them. The viewfinder stream can scale, while the output\n> > -\t * stream can crop only, so select the output stream when the requested\n> > -\t * resolution is equal to the sensor resolution, and the viewfinder\n> > -\t * stream otherwise.\n> > -\t */\n> > -\tstd::set availableStreams = {\n> > -\t\t&data_->outStream_,\n> > -\t\t&data_->vfStream_,\n> > -\t\t&data_->rawStream_,\n> > -\t};\n> >\n> \n> This results in two empty lines. Doesn't checkstyle report that ?\n> \n> > -\tstreams_.clear();\n> > -\tstreams_.reserve(config_.size());\n> > +\t/* Assign streams to each configuration entry. */\n> > +\tif (updateStreams())\n> > +\t\treturn Invalid;\n> >\n> > +\t/* Verify and adjust configuration if needed. */\n> > \tfor (unsigned int i = 0; i < config_.size(); ++i) {\n> \n> Can you iterate on config_ ?\n> \n> > \t\tStreamConfiguration &cfg = config_[i];\n> > -\t\tconst PixelFormat pixelFormat = cfg.pixelFormat;\n> > -\t\tconst Size size = cfg.size;\n> > -\t\tconst IPU3Stream *stream;\n> > -\n> > -\t\tif (cfg.pixelFormat.modifier() == IPU3_FORMAT_MOD_PACKED)\n> > -\t\t\tstream = &data_->rawStream_;\n> > -\t\telse if (cfg.size == sensorFormat_.size)\n> > -\t\t\tstream = &data_->outStream_;\n> > -\t\telse\n> > -\t\t\tstream = &data_->vfStream_;\n> > -\n> > -\t\tif (availableStreams.find(stream) == availableStreams.end())\n> > -\t\t\tstream = *availableStreams.begin();\n> > -\n> > -\t\tLOG(IPU3, Debug)\n> > -\t\t\t<< \"Assigned '\" << stream->name_ << \"' to stream \" << i;\n> > +\t\tconst StreamConfiguration org = cfg;\n\norg isn't a very explicit name. origCfg or oldCfg would be better.\n\n> > +\t\tconst IPU3Stream *stream = streams_[i];\n> \n> Ah no, you need i here.\n> However, associating on index seems a bit weak, and as we can't assign\n> streams to stream configurations here, have you considered having\n> updateStreams() return a map of configs to streams ?\n> \n> >\n> > \t\tif (stream->raw_) {\n> > \t\t\tconst auto &itFormat =\n> > @@ -392,15 +414,12 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate()\n> >\n> > \t\tcfg.bufferCount = IPU3_BUFFER_COUNT;\n> >\n> > -\t\tif (cfg.pixelFormat != pixelFormat || cfg.size != size) {\n> > +\t\tif (cfg.pixelFormat != org.pixelFormat || cfg.size != org.size) {\n> > \t\t\tLOG(IPU3, Debug)\n> > \t\t\t\t<< \"Stream \" << i << \" configuration adjusted to \"\n> > \t\t\t\t<< cfg.toString();\n> > \t\t\tstatus = Adjusted;\n> > \t\t}\n> > -\n> > -\t\tstreams_.push_back(stream);\n> > -\t\tavailableStreams.erase(stream);\n> > \t}\n> >\n> > \treturn status;","headers":{"Return-Path":"","Received":["from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1CBF9603CA\n\tfor ;\n\tFri, 5 Jun 2020 23:58:23 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 2713427C;\n\tFri, 5 Jun 2020 23:58:21 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"PKZscWpB\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1591394302;\n\tbh=uDnkyT0aBpWiiwiZC7m4mw+tWDEKRCVSGfU/KgLX/aw=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=PKZscWpBobRxbPwH27ylI+1cpwLFzviO1SAlZXJFbY2lzzTt5nEUXADcoOecbbyfE\n\t5UFxG0V7ARTSR9aNC2tcdkp6P94rWVUlFv3zqPIXjr/4DZwMnAtCfi5h+JBJYmGq00\n\tNgXZcPCe7LuOWpGMa+Mp83YNfwLkKScjDKOeW1dI=","Date":"Sat, 6 Jun 2020 00:58:02 +0300","From":"Laurent Pinchart ","To":"Jacopo Mondi ","Cc":"Niklas =?utf-8?q?S=C3=B6derlund?= ,\n\tlibcamera-devel@lists.libcamera.org","Message-ID":"<20200605215802.GJ26752@pendragon.ideasonboard.com>","References":"<20200602013909.3170593-1-niklas.soderlund@ragnatech.se>\n\t<20200602013909.3170593-5-niklas.soderlund@ragnatech.se>\n\t<20200602095358.pk6jnzgocxxonsna@uno.localdomain>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20200602095358.pk6jnzgocxxonsna@uno.localdomain>","Subject":"Re: [libcamera-devel] [PATCH 04/10] libcamera: ipu3: Breakout\n\tstream assignment to new function","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Fri, 05 Jun 2020 21:58:23 -0000"}}]