[{"id":25607,"web_url":"https://patchwork.libcamera.org/comment/25607/","msgid":"<20221026153241.bm4l7j762z3tnnaw@uno.localdomain>","date":"2022-10-26T15:32:41","subject":"Re: [libcamera-devel] [PATCH v3 10/13] pipeline: rkisp1: Query the\n\tdriver for formats","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Laurent\n\nOn Mon, Oct 24, 2022 at 03:03:53AM +0300, Laurent Pinchart via libcamera-devel wrote:\n> From: Paul Elder \n>\n> Query the driver for the output formats and sizes that it supports,\n> instead of hardcoding them. This allows future-proofing for formats that\n> are supported by some but not all versions of the driver.\n>\n> As the rkisp1 driver currently does not support VIDIOC_ENUM_FRAMESIZES,\n> fallback to the hardcoded list of supported formats and framesizes. This\n> feature will be added to the driver in parallel, though we cannot\n> guarantee that users will have a new enough kernel for it.\n>\n> Signed-off-by: Paul Elder \n> Signed-off-by: Laurent Pinchart \n> ---\n> Changes since v2:\n>\n> - Don't pass V4L2VideoDevice to RkISP1Path::populateFormats()\n> - Use structured binding in for range loop\n> - Use std::set::count() to replace std::find_if()\n> - Don't cache sizes, use std::set instead of std::map\n> - Update min and max resolutions based on enumerated sizes\n> - Drop comment about aspect ratio\n>\n> Changes since v1:\n>\n> - Enumerate and cache framesizes as well\n> - Massage generateConfiguration accordingly\n> - This lets us skip modifying V4L2VideoDevice::formats() to support lack\n> of ENUM_FRAMESIZES\n> - Also requires us to keep the list of hardcoded formats for backward\n> compatibility\n> ---\n> src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 51 +++++++++++++++++--\n> src/libcamera/pipeline/rkisp1/rkisp1_path.h | 8 ++-\n> 2 files changed, 54 insertions(+), 5 deletions(-)\n>\n> diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\n> index 2d38f0fb37ab..8077a54494a5 100644\n> --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\n> +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\n> @@ -41,6 +41,8 @@ bool RkISP1Path::init(MediaDevice *media)\n> \tif (video_->open() < 0)\n> \t\treturn false;\n>\n> +\tpopulateFormats();\n> +\n> \tlink_ = media->link(\"rkisp1_isp\", 2, resizer, 0);\n> \tif (!link_)\n> \t\treturn false;\n> @@ -48,6 +50,41 @@ bool RkISP1Path::init(MediaDevice *media)\n> \treturn true;\n> }\n>\n> +void RkISP1Path::populateFormats()\n> +{\n> +\tV4L2VideoDevice::Formats v4l2Formats = video_->formats();\n> +\tif (v4l2Formats.empty()) {\n> +\t\tLOG(RkISP1, Warning)\n\nThis might warn users that something went wrong, while as the driver\nhas been upstream without this feature for some time and there's\nnothing wrong. I would demote the error\n\nThis apart\nReviewed-by: Jacopo Mondi \n\n\n> +\t\t\t<< \"Failed to enumerate framesizes. Loading default framesizes\";\n> +\n> +\t\tfor (const PixelFormat &format : formats_)\n> +\t\t\tstreamFormats_.insert(format);\n> +\t\treturn;\n> +\t}\n> +\n> +\tminResolution_ = { 65535, 65535 };\n> +\tmaxResolution_ = { 0, 0 };\n> +\n> +\tstd::vector formats;\n> +\tfor (const auto &[format, sizes] : v4l2Formats) {\n> +\t\tconst PixelFormat pixelFormat = format.toPixelFormat();\n> +\t\tconst PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat);\n> +\n> +\t\t/* \\todo Add support for RAW formats. */\n> +\t\tif (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW)\n> +\t\t\tcontinue;\n> +\n> +\t\tstreamFormats_.insert(pixelFormat);\n> +\n> +\t\tfor (const auto &size : sizes) {\n> +\t\t\tif (minResolution_ > size.min)\n> +\t\t\t\tminResolution_ = size.min;\n> +\t\t\tif (maxResolution_ < size.max)\n> +\t\t\t\tmaxResolution_ = size.max;\n> +\t\t}\n> +\t}\n> +}\n> +\n> StreamConfiguration RkISP1Path::generateConfiguration(const Size &resolution)\n> {\n> \tSize maxResolution = maxResolution_.boundedToAspectRatio(resolution)\n> @@ -55,7 +92,7 @@ StreamConfiguration RkISP1Path::generateConfiguration(const Size &resolution)\n> \tSize minResolution = minResolution_.expandedToAspectRatio(resolution);\n>\n> \tstd::map> streamFormats;\n> -\tfor (const PixelFormat &format : formats_)\n> +\tfor (const auto &format : streamFormats_)\n> \t\tstreamFormats[format] = { { minResolution, maxResolution } };\n>\n> \tStreamFormats formats(streamFormats);\n> @@ -72,8 +109,12 @@ CameraConfiguration::Status RkISP1Path::validate(StreamConfiguration *cfg)\n> \tconst StreamConfiguration reqCfg = *cfg;\n> \tCameraConfiguration::Status status = CameraConfiguration::Valid;\n>\n> -\tif (std::find(formats_.begin(), formats_.end(), cfg->pixelFormat) ==\n> -\t formats_.end())\n> +\t/*\n> +\t * Default to NV12 if the requested format is not supported. All\n> +\t * versions of the ISP are guaranteed to support NV12 on both the main\n> +\t * and self paths.\n> +\t */\n> +\tif (!streamFormats_.count(cfg->pixelFormat))\n> \t\tcfg->pixelFormat = formats::NV12;\n>\n> \tcfg->size.boundTo(maxResolution_);\n> @@ -204,6 +245,10 @@ void RkISP1Path::stop()\n> \trunning_ = false;\n> }\n>\n> +/*\n> + * \\todo Remove the hardcoded resolutions and formats once all users will have\n> + * migrated to a recent enough kernel.\n> + */\n> namespace {\n> constexpr Size RKISP1_RSZ_MP_SRC_MIN{ 32, 16 };\n> constexpr Size RKISP1_RSZ_MP_SRC_MAX{ 4416, 3312 };\n> diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.h b/src/libcamera/pipeline/rkisp1/rkisp1_path.h\n> index f3f1ae391d65..d88effbb6f56 100644\n> --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.h\n> +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.h\n> @@ -8,6 +8,7 @@\n> #pragma once\n>\n> #include \n> +#include \n> #include \n>\n> #include \n> @@ -57,14 +58,17 @@ public:\n> \tSignal &bufferReady() { return video_->bufferReady; }\n>\n> private:\n> +\tvoid populateFormats();\n> +\n> \tstatic constexpr unsigned int RKISP1_BUFFER_COUNT = 4;\n>\n> \tconst char *name_;\n> \tbool running_;\n>\n> \tconst Span formats_;\n> -\tconst Size minResolution_;\n> -\tconst Size maxResolution_;\n> +\tstd::set streamFormats_;\n> +\tSize minResolution_;\n> +\tSize maxResolution_;\n>\n> \tstd::unique_ptr resizer_;\n> \tstd::unique_ptr video_;\n> --\n> Regards,\n>\n> Laurent Pinchart\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 89B88BDB16\n\tfor ;\n\tWed, 26 Oct 2022 15:32:46 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D95CD62F5C;\n\tWed, 26 Oct 2022 17:32:45 +0200 (CEST)","from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net\n\t[217.70.183.196])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 014F461F4B\n\tfor ;\n\tWed, 26 Oct 2022 17:32:43 +0200 (CEST)","(Authenticated sender: jacopo@jmondi.org)\n\tby mail.gandi.net (Postfix) with ESMTPSA id 821D5E0004;\n\tWed, 26 Oct 2022 15:32:43 +0000 (UTC)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1666798365;\n\tbh=ct6FhqIdiUlWZqX+lQX5mzKC6A0v9VBknOIiMaFMssw=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=Nvpmy0eD4QOvC0bwpNKYNn6Epi0rF+OzN6FPApptdZC2/S9Nku21pr5XpL5uDHSCo\n\tbIr/hRHz9B1ZBvEXUDf+UcsJakqgXnx0o4DwddAQK4J3MYd+A55S+oPU/x6vnAy6N6\n\t94V1HV6qQmjCgn7yF90cTv5MnRxQDvvQLtekrykG8YS41gtRr3VtZuwiwt4kRrXYO/\n\t5ASw/7+UUbWkrLqyISih3v9OqjqrVszfF6RIbiS5OuF4gvU5bPECN98YMdCa/XhMAU\n\tX6y94sgvWCLMmeQbEWOVAAnDjarJ211Ixmze6ThcqAmbGl/DoaLNxclnipmcXooQyv\n\tYBFi0Yx/XXhJg==","Date":"Wed, 26 Oct 2022 17:32:41 +0200","To":"Laurent Pinchart ","Message-ID":"<20221026153241.bm4l7j762z3tnnaw@uno.localdomain>","References":"<20221024000356.29521-1-laurent.pinchart@ideasonboard.com>\n\t<20221024000356.29521-11-laurent.pinchart@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20221024000356.29521-11-laurent.pinchart@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v3 10/13] pipeline: rkisp1: Query the\n\tdriver for formats","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Jacopo Mondi via libcamera-devel ","Reply-To":"Jacopo Mondi ","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":25678,"web_url":"https://patchwork.libcamera.org/comment/25678/","msgid":"<166699533865.404886.12660284224397664072@Monstersaurus>","date":"2022-10-28T22:15:38","subject":"Re: [libcamera-devel] [PATCH v3 10/13] pipeline: rkisp1: Query the\n\tdriver for formats","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Jacopo Mondi via libcamera-devel (2022-10-26 16:32:41)\n> Hi Laurent\n> \n> On Mon, Oct 24, 2022 at 03:03:53AM +0300, Laurent Pinchart via libcamera-devel wrote:\n> > From: Paul Elder \n> >\n> > Query the driver for the output formats and sizes that it supports,\n> > instead of hardcoding them. This allows future-proofing for formats that\n> > are supported by some but not all versions of the driver.\n> >\n> > As the rkisp1 driver currently does not support VIDIOC_ENUM_FRAMESIZES,\n> > fallback to the hardcoded list of supported formats and framesizes. This\n> > feature will be added to the driver in parallel, though we cannot\n> > guarantee that users will have a new enough kernel for it.\n> >\n> > Signed-off-by: Paul Elder \n> > Signed-off-by: Laurent Pinchart \n> > ---\n> > Changes since v2:\n> >\n> > - Don't pass V4L2VideoDevice to RkISP1Path::populateFormats()\n> > - Use structured binding in for range loop\n> > - Use std::set::count() to replace std::find_if()\n> > - Don't cache sizes, use std::set instead of std::map\n> > - Update min and max resolutions based on enumerated sizes\n> > - Drop comment about aspect ratio\n> >\n> > Changes since v1:\n> >\n> > - Enumerate and cache framesizes as well\n> > - Massage generateConfiguration accordingly\n> > - This lets us skip modifying V4L2VideoDevice::formats() to support lack\n> > of ENUM_FRAMESIZES\n> > - Also requires us to keep the list of hardcoded formats for backward\n> > compatibility\n> > ---\n> > src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 51 +++++++++++++++++--\n> > src/libcamera/pipeline/rkisp1/rkisp1_path.h | 8 ++-\n> > 2 files changed, 54 insertions(+), 5 deletions(-)\n> >\n> > diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\n> > index 2d38f0fb37ab..8077a54494a5 100644\n> > --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\n> > +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\n> > @@ -41,6 +41,8 @@ bool RkISP1Path::init(MediaDevice *media)\n> > if (video_->open() < 0)\n> > return false;\n> >\n> > + populateFormats();\n> > +\n> > link_ = media->link(\"rkisp1_isp\", 2, resizer, 0);\n> > if (!link_)\n> > return false;\n> > @@ -48,6 +50,41 @@ bool RkISP1Path::init(MediaDevice *media)\n> > return true;\n> > }\n> >\n> > +void RkISP1Path::populateFormats()\n> > +{\n> > + V4L2VideoDevice::Formats v4l2Formats = video_->formats();\n> > + if (v4l2Formats.empty()) {\n> > + LOG(RkISP1, Warning)\n> \n> This might warn users that something went wrong, while as the driver\n> has been upstream without this feature for some time and there's\n> nothing wrong. I would demote the error\n> \n> This apart\n> Reviewed-by: Jacopo Mondi \n\nIt may be helpful to detect any input size limitations too ... but I\nthink they also need hardcoding in as defaults too.\n--\nKieran\n\n\n> \n> \n> > + << \"Failed to enumerate framesizes. Loading default framesizes\";\n> > +\n> > + for (const PixelFormat &format : formats_)\n> > + streamFormats_.insert(format);\n> > + return;\n> > + }\n> > +\n> > + minResolution_ = { 65535, 65535 };\n> > + maxResolution_ = { 0, 0 };\n> > +\n> > + std::vector formats;\n> > + for (const auto &[format, sizes] : v4l2Formats) {\n> > + const PixelFormat pixelFormat = format.toPixelFormat();\n> > + const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat);\n> > +\n> > + /* \\todo Add support for RAW formats. */\n> > + if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW)\n> > + continue;\n> > +\n> > + streamFormats_.insert(pixelFormat);\n> > +\n> > + for (const auto &size : sizes) {\n> > + if (minResolution_ > size.min)\n> > + minResolution_ = size.min;\n> > + if (maxResolution_ < size.max)\n> > + maxResolution_ = size.max;\n> > + }\n> > + }\n> > +}\n> > +\n> > StreamConfiguration RkISP1Path::generateConfiguration(const Size &resolution)\n> > {\n> > Size maxResolution = maxResolution_.boundedToAspectRatio(resolution)\n> > @@ -55,7 +92,7 @@ StreamConfiguration RkISP1Path::generateConfiguration(const Size &resolution)\n> > Size minResolution = minResolution_.expandedToAspectRatio(resolution);\n> >\n> > std::map> streamFormats;\n> > - for (const PixelFormat &format : formats_)\n> > + for (const auto &format : streamFormats_)\n> > streamFormats[format] = { { minResolution, maxResolution } };\n> >\n> > StreamFormats formats(streamFormats);\n> > @@ -72,8 +109,12 @@ CameraConfiguration::Status RkISP1Path::validate(StreamConfiguration *cfg)\n> > const StreamConfiguration reqCfg = *cfg;\n> > CameraConfiguration::Status status = CameraConfiguration::Valid;\n> >\n> > - if (std::find(formats_.begin(), formats_.end(), cfg->pixelFormat) ==\n> > - formats_.end())\n> > + /*\n> > + * Default to NV12 if the requested format is not supported. All\n> > + * versions of the ISP are guaranteed to support NV12 on both the main\n> > + * and self paths.\n> > + */\n> > + if (!streamFormats_.count(cfg->pixelFormat))\n> > cfg->pixelFormat = formats::NV12;\n> >\n> > cfg->size.boundTo(maxResolution_);\n> > @@ -204,6 +245,10 @@ void RkISP1Path::stop()\n> > running_ = false;\n> > }\n> >\n> > +/*\n> > + * \\todo Remove the hardcoded resolutions and formats once all users will have\n> > + * migrated to a recent enough kernel.\n> > + */\n> > namespace {\n> > constexpr Size RKISP1_RSZ_MP_SRC_MIN{ 32, 16 };\n> > constexpr Size RKISP1_RSZ_MP_SRC_MAX{ 4416, 3312 };\n> > diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.h b/src/libcamera/pipeline/rkisp1/rkisp1_path.h\n> > index f3f1ae391d65..d88effbb6f56 100644\n> > --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.h\n> > +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.h\n> > @@ -8,6 +8,7 @@\n> > #pragma once\n> >\n> > #include \n> > +#include \n> > #include \n> >\n> > #include \n> > @@ -57,14 +58,17 @@ public:\n> > Signal &bufferReady() { return video_->bufferReady; }\n> >\n> > private:\n> > + void populateFormats();\n> > +\n> > static constexpr unsigned int RKISP1_BUFFER_COUNT = 4;\n> >\n> > const char *name_;\n> > bool running_;\n> >\n> > const Span formats_;\n> > - const Size minResolution_;\n> > - const Size maxResolution_;\n> > + std::set streamFormats_;\n> > + Size minResolution_;\n> > + Size maxResolution_;\n> >\n> > std::unique_ptr resizer_;\n> > std::unique_ptr video_;\n> > --\n> > Regards,\n> >\n> > Laurent Pinchart\n> >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 82C1ABDB16\n\tfor ;\n\tFri, 28 Oct 2022 22:15:43 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id E057F62FF1;\n\tSat, 29 Oct 2022 00:15:42 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A011262F89\n\tfor ;\n\tSat, 29 Oct 2022 00:15:41 +0200 (CEST)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 1BE56440;\n\tSat, 29 Oct 2022 00:15:41 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1666995342;\n\tbh=krtLNXdL1CS5hr0EGYr1s17913MZeCum6uYoEUl5wus=;\n\th=In-Reply-To:References:To:Date:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=AXx+AlT23ciS9in+uhAUL/qL9iy3famEib/5gb2ci5H9kCjh2CKRuBM8ExN/Yly6g\n\tlUbqXPjVJjrYfRe9jewCpNMyzHd40Wb2/CyyF730KiV5plMsDf+UpLUSGYRnL1Gkzl\n\t4HXJAnSpic0Z72wT1GNWbRBy7L9aysEZ7mttMyfqbYVSVcacZ9uq078AICxu8OrCme\n\tqGc/16o0sn3CB/g1N3R07qi9kLLNwGd6ED/7HYadd34lnIBBMlPDqT3Lvxb0qes1Ko\n\tzyIxBItOW5rIK03atYKADEVdlm4qsadjIPEaLYjecBi9e60IecpuYSOhkqEU8Eo4aU\n\tCaumx4+rtPYrg==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1666995341;\n\tbh=krtLNXdL1CS5hr0EGYr1s17913MZeCum6uYoEUl5wus=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=TCWO6XqUM4z0K8sfsfvL+W7et3xRLvxeAlUPMvq7fDP0kqWeYrWR/vRPTw2tqZpzz\n\tILeF2mJDvc1a5hiVFaNkE0MX9lag58TWO+O9HZOCSIMkUc6xqnzgTPfh510ZY0WmOA\n\tlJzHyY+Sdub2tWGJ8KpPQMGPT7QnRFQSmA4F8b4M="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"TCWO6XqU\"; dkim-atps=neutral","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20221026153241.bm4l7j762z3tnnaw@uno.localdomain>","References":"<20221024000356.29521-1-laurent.pinchart@ideasonboard.com>\n\t<20221024000356.29521-11-laurent.pinchart@ideasonboard.com>\n\t<20221026153241.bm4l7j762z3tnnaw@uno.localdomain>","To":"Jacopo Mondi , Jacopo Mondi via libcamera-devel\n\t, \n\tLaurent Pinchart ","Date":"Fri, 28 Oct 2022 23:15:38 +0100","Message-ID":"<166699533865.404886.12660284224397664072@Monstersaurus>","User-Agent":"alot/0.10","Subject":"Re: [libcamera-devel] [PATCH v3 10/13] pipeline: rkisp1: Query the\n\tdriver for formats","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Kieran Bingham via libcamera-devel\n\t","Reply-To":"Kieran Bingham ","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":25681,"web_url":"https://patchwork.libcamera.org/comment/25681/","msgid":"","date":"2022-10-30T16:46:27","subject":"Re: [libcamera-devel] [PATCH v3 10/13] pipeline: rkisp1: Query the\n\tdriver for formats","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hello,\n\nOn Fri, Oct 28, 2022 at 11:15:38PM +0100, Kieran Bingham wrote:\n> Quoting Jacopo Mondi via libcamera-devel (2022-10-26 16:32:41)\n> > On Mon, Oct 24, 2022 at 03:03:53AM +0300, Laurent Pinchart via libcamera-devel wrote:\n> > > From: Paul Elder \n> > >\n> > > Query the driver for the output formats and sizes that it supports,\n> > > instead of hardcoding them. This allows future-proofing for formats that\n> > > are supported by some but not all versions of the driver.\n> > >\n> > > As the rkisp1 driver currently does not support VIDIOC_ENUM_FRAMESIZES,\n> > > fallback to the hardcoded list of supported formats and framesizes. This\n> > > feature will be added to the driver in parallel, though we cannot\n> > > guarantee that users will have a new enough kernel for it.\n> > >\n> > > Signed-off-by: Paul Elder \n> > > Signed-off-by: Laurent Pinchart \n> > > ---\n> > > Changes since v2:\n> > >\n> > > - Don't pass V4L2VideoDevice to RkISP1Path::populateFormats()\n> > > - Use structured binding in for range loop\n> > > - Use std::set::count() to replace std::find_if()\n> > > - Don't cache sizes, use std::set instead of std::map\n> > > - Update min and max resolutions based on enumerated sizes\n> > > - Drop comment about aspect ratio\n> > >\n> > > Changes since v1:\n> > >\n> > > - Enumerate and cache framesizes as well\n> > > - Massage generateConfiguration accordingly\n> > > - This lets us skip modifying V4L2VideoDevice::formats() to support lack\n> > > of ENUM_FRAMESIZES\n> > > - Also requires us to keep the list of hardcoded formats for backward\n> > > compatibility\n> > > ---\n> > > src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 51 +++++++++++++++++--\n> > > src/libcamera/pipeline/rkisp1/rkisp1_path.h | 8 ++-\n> > > 2 files changed, 54 insertions(+), 5 deletions(-)\n> > >\n> > > diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\n> > > index 2d38f0fb37ab..8077a54494a5 100644\n> > > --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\n> > > +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\n> > > @@ -41,6 +41,8 @@ bool RkISP1Path::init(MediaDevice *media)\n> > > if (video_->open() < 0)\n> > > return false;\n> > >\n> > > + populateFormats();\n> > > +\n> > > link_ = media->link(\"rkisp1_isp\", 2, resizer, 0);\n> > > if (!link_)\n> > > return false;\n> > > @@ -48,6 +50,41 @@ bool RkISP1Path::init(MediaDevice *media)\n> > > return true;\n> > > }\n> > >\n> > > +void RkISP1Path::populateFormats()\n> > > +{\n> > > + V4L2VideoDevice::Formats v4l2Formats = video_->formats();\n> > > + if (v4l2Formats.empty()) {\n> > > + LOG(RkISP1, Warning)\n> > \n> > This might warn users that something went wrong, while as the driver\n> > has been upstream without this feature for some time and there's\n> > nothing wrong. I would demote the error\n\nSounds good, I'll go for Info.\n\n> > This apart\n> > Reviewed-by: Jacopo Mondi \n> \n> It may be helpful to detect any input size limitations too ... but I\n> think they also need hardcoding in as defaults too.\n\nPatches are welcome :-)\n\n> > > + << \"Failed to enumerate framesizes. Loading default framesizes\";\n> > > +\n> > > + for (const PixelFormat &format : formats_)\n> > > + streamFormats_.insert(format);\n> > > + return;\n> > > + }\n> > > +\n> > > + minResolution_ = { 65535, 65535 };\n> > > + maxResolution_ = { 0, 0 };\n> > > +\n> > > + std::vector formats;\n> > > + for (const auto &[format, sizes] : v4l2Formats) {\n> > > + const PixelFormat pixelFormat = format.toPixelFormat();\n> > > + const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat);\n> > > +\n> > > + /* \\todo Add support for RAW formats. */\n> > > + if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW)\n> > > + continue;\n> > > +\n> > > + streamFormats_.insert(pixelFormat);\n> > > +\n> > > + for (const auto &size : sizes) {\n> > > + if (minResolution_ > size.min)\n> > > + minResolution_ = size.min;\n> > > + if (maxResolution_ < size.max)\n> > > + maxResolution_ = size.max;\n> > > + }\n> > > + }\n> > > +}\n> > > +\n> > > StreamConfiguration RkISP1Path::generateConfiguration(const Size &resolution)\n> > > {\n> > > Size maxResolution = maxResolution_.boundedToAspectRatio(resolution)\n> > > @@ -55,7 +92,7 @@ StreamConfiguration RkISP1Path::generateConfiguration(const Size &resolution)\n> > > Size minResolution = minResolution_.expandedToAspectRatio(resolution);\n> > >\n> > > std::map> streamFormats;\n> > > - for (const PixelFormat &format : formats_)\n> > > + for (const auto &format : streamFormats_)\n> > > streamFormats[format] = { { minResolution, maxResolution } };\n> > >\n> > > StreamFormats formats(streamFormats);\n> > > @@ -72,8 +109,12 @@ CameraConfiguration::Status RkISP1Path::validate(StreamConfiguration *cfg)\n> > > const StreamConfiguration reqCfg = *cfg;\n> > > CameraConfiguration::Status status = CameraConfiguration::Valid;\n> > >\n> > > - if (std::find(formats_.begin(), formats_.end(), cfg->pixelFormat) ==\n> > > - formats_.end())\n> > > + /*\n> > > + * Default to NV12 if the requested format is not supported. All\n> > > + * versions of the ISP are guaranteed to support NV12 on both the main\n> > > + * and self paths.\n> > > + */\n> > > + if (!streamFormats_.count(cfg->pixelFormat))\n> > > cfg->pixelFormat = formats::NV12;\n> > >\n> > > cfg->size.boundTo(maxResolution_);\n> > > @@ -204,6 +245,10 @@ void RkISP1Path::stop()\n> > > running_ = false;\n> > > }\n> > >\n> > > +/*\n> > > + * \\todo Remove the hardcoded resolutions and formats once all users will have\n> > > + * migrated to a recent enough kernel.\n> > > + */\n> > > namespace {\n> > > constexpr Size RKISP1_RSZ_MP_SRC_MIN{ 32, 16 };\n> > > constexpr Size RKISP1_RSZ_MP_SRC_MAX{ 4416, 3312 };\n> > > diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.h b/src/libcamera/pipeline/rkisp1/rkisp1_path.h\n> > > index f3f1ae391d65..d88effbb6f56 100644\n> > > --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.h\n> > > +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.h\n> > > @@ -8,6 +8,7 @@\n> > > #pragma once\n> > >\n> > > #include \n> > > +#include \n> > > #include \n> > >\n> > > #include \n> > > @@ -57,14 +58,17 @@ public:\n> > > Signal &bufferReady() { return video_->bufferReady; }\n> > >\n> > > private:\n> > > + void populateFormats();\n> > > +\n> > > static constexpr unsigned int RKISP1_BUFFER_COUNT = 4;\n> > >\n> > > const char *name_;\n> > > bool running_;\n> > >\n> > > const Span formats_;\n> > > - const Size minResolution_;\n> > > - const Size maxResolution_;\n> > > + std::set streamFormats_;\n> > > + Size minResolution_;\n> > > + Size maxResolution_;\n> > >\n> > > std::unique_ptr resizer_;\n> > > std::unique_ptr video_;","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B4896BD16B\n\tfor ;\n\tSun, 30 Oct 2022 16:46:53 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1E80A6301F;\n\tSun, 30 Oct 2022 17:46:53 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id F289A61F47\n\tfor ;\n\tSun, 30 Oct 2022 17:46:51 +0100 (CET)","from pendragon.ideasonboard.com (85-76-1-64-nat.elisa-mobile.fi\n\t[85.76.1.64])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C7D847C5;\n\tSun, 30 Oct 2022 17:46:50 +0100 (CET)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1667148413;\n\tbh=tPFbqqsIbhFqxYH13De6v/YvEWmPsERz9H8bvu5TNCw=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=0teCs+afaxiRXv0jcEJoJSJ8xeOg/R4iIfsNd8ARmgOReJBPy0MItcmdnMKeH3/NM\n\tvcnbe3oc/uKaBuBI0hlNYtZwKxPJoLLGYFJP+akjKX2mqEnCIfrTusjPRLJBruREiW\n\tl3airO4MDG5ZNlYFiicxqGqg8UopcGUT7Z14CUj/hLAP++OWdytUHqKSayEdiEbG9R\n\tnM9h3p0VvzUZndHd23L5TKzejJuTMtZkaTlpOeHlkCn6AHbmPqM8dcbm6z/MsPTq00\n\tMkQaBnErEhxal5I3G8OTSi5mxdgiCQ2+57ulaVwnKlfeWYNOUKwesWRQQgNDPpuBUV\n\tnt+kXj0U8Jwvg==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1667148411;\n\tbh=tPFbqqsIbhFqxYH13De6v/YvEWmPsERz9H8bvu5TNCw=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=HlfZlpWAsJiXKf7+Wm1nFP5lmxlTuLkaihWJZ1NYpMy1cNznywFmiS8SQfZIpgaRW\n\t/xy7aqxqed3Wl1Tl9PZi9tybP1Q96MuDpmZoxAONQxt6WwzZD7FrNJINvVi2VQ2dNL\n\tAI3JUPjgyOQFbj69dQl9fUBCpHbu0QWULbCt9mxs="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"HlfZlpWA\"; dkim-atps=neutral","Date":"Sun, 30 Oct 2022 18:46:27 +0200","To":"Kieran Bingham ","Message-ID":"","References":"<20221024000356.29521-1-laurent.pinchart@ideasonboard.com>\n\t<20221024000356.29521-11-laurent.pinchart@ideasonboard.com>\n\t<20221026153241.bm4l7j762z3tnnaw@uno.localdomain>\n\t<166699533865.404886.12660284224397664072@Monstersaurus>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<166699533865.404886.12660284224397664072@Monstersaurus>","Subject":"Re: [libcamera-devel] [PATCH v3 10/13] pipeline: rkisp1: Query the\n\tdriver for formats","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Laurent Pinchart via libcamera-devel\n\t","Reply-To":"Laurent Pinchart ","Cc":"Jacopo Mondi via libcamera-devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]