[{"id":22911,"web_url":"https://patchwork.libcamera.org/comment/22911/","msgid":"<YnlLBLXr9mKJRCkE@pendragon.ideasonboard.com>","date":"2022-05-09T17:10:28","subject":"Re: [libcamera-devel] [PATCH v6 1/3] libcamera: Introduce\n\tYamlParser as a helper to parse yaml files","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Han-Lin,\n\nThank you for the patch.\n\nOn Wed, Apr 27, 2022 at 10:09:27PM +0800, Han-Lin Chen via libcamera-devel wrote:\n> Introduce YamlParser as a helper to convert contents of a yaml file to\n> a tree based structure for easier reading, and to avoid writing parser\n> with raw yaml tokens. The class is based on libyaml, and only support\n> reading but not writing a yaml file.\n> \n> The interface is inspired by Json::Value class from jsoncpp:\n> http://jsoncpp.sourceforge.net/class_json_1_1_value.html\n> \n> Signed-off-by: Han-Lin Chen <hanlinchen@chromium.org>\n> ---\n>  README.rst                               |   4 +-\n>  include/libcamera/internal/meson.build   |   1 +\n>  include/libcamera/internal/yaml_parser.h |  87 +++\n>  src/libcamera/meson.build                |   3 +\n>  src/libcamera/yaml_parser.cpp            | 679 +++++++++++++++++++++++\n>  5 files changed, 772 insertions(+), 2 deletions(-)\n>  create mode 100644 include/libcamera/internal/yaml_parser.h\n>  create mode 100644 src/libcamera/yaml_parser.cpp\n\n[snip]\n\n> diff --git a/src/libcamera/yaml_parser.cpp b/src/libcamera/yaml_parser.cpp\n> new file mode 100644\n> index 00000000..4a047494\n> --- /dev/null\n> +++ b/src/libcamera/yaml_parser.cpp\n> @@ -0,0 +1,679 @@\n\n[snip]\n\n> +/**\n> + * \\fn YamlParserContext::parseDictionaryOrList()\n> + * \\brief A helper function to abstract the common part of parsing dictionary or list\n> + *\n> + * \\param[in] isDictionary True for parsing a dictionary, and false for a list\n> + * \\param[in] parseItem The callback to handle an item\n> + *\n> + * A helper function to abstract parsing an item from a dictionary or a list.\n> + * The differences of them in a YAML event stream are:\n> + *\n> + * 1. The start and end event types are different\n> + * 2. There is a leading scalar string as key in the items of a dictionary\n> + *\n> + * The caller should handle the leading key string in its callback parseItem\n> + * when it's a dictionary.\n> + *\n> + * \\return 0 on success or a negative error code otherwise\n> + * \\retval -EINVAL The parser is failed to initialize\n> + */\n> +int YamlParserContext::parseDictionaryOrList(YamlObject::Type type,\n> +\t\t\t\t\t     const std::function<int(EventPtr event)> &parseItem)\n> +{\n> +\tyaml_event_type_t endEventType = YAML_SEQUENCE_END_EVENT;\n> +\tif (type == YamlObject::Dictionary)\n> +\t\tendEventType = YAML_MAPPING_END_EVENT;\n> +\n> +\t/*\n> +\t * Add a safety counter to make sure we don't loop indefinitely in case\n> +\t * the YAML file is malformed.\n> +\t */\n> +\tfor (unsigned int sentinel = 1000; sentinel; sentinel--) {\n> +\t\tauto evt = nextEvent();\n> +\t\tif (!evt)\n> +\t\t\treturn -EINVAL;\n> +\n> +\t\tif (evt->type == endEventType)\n> +\t\t\treturn 0;\n> +\n> +\t\tint ret = parseItem(std::move(evt));\n> +\t\tif (ret)\n> +\t\t\treturn ret;\n> +\t}\n> +\n> +\tLOG(YamlParser, Error) << \"The YAML file contains a List or Dictionary\"\n> +\t\t\t\t  \" whose size exceeding the parser's limit(1000)\";\n\ns/exceeding/exceeds/\n\n> +\n> +\treturn -EINVAL;\n> +}\n> +\n> +/**\n> + * \\fn YamlParserContext::parseNextYamlObject()\n> + * \\brief Parse next YAML event and read it as a YamlObject\n> + * \\param[in] yamlObject The result of YamlObject\n> + * \\param[in] event The leading event of the object\n> + *\n> + * Parse next YAML object separately as a value, list or dictionary.\n> + *\n> + * \\return 0 on success or a negative error code otherwise\n> + * \\retval -EINVAL Fail to parse the YAML file.\n> + */\n> +int YamlParserContext::parseNextYamlObject(YamlObject &yamlObject, EventPtr event)\n> +{\n> +\tif (!event)\n> +\t\treturn -EINVAL;\n> +\n> +\tswitch (event->type) {\n> +\tcase YAML_SCALAR_EVENT:\n> +\t\tyamlObject.type_ = YamlObject::Value;\n> +\t\treadValue(yamlObject.value_, std::move(event));\n> +\t\treturn 0;\n> +\n> +\tcase YAML_SEQUENCE_START_EVENT: {\n> +\t\tyamlObject.type_ = YamlObject::List;\n> +\t\tauto &list = yamlObject.list_;\n> +\t\tauto handler = [this, &list](EventPtr evt) {\n> +\t\t\tlist.emplace_back(new YamlObject());\n> +\t\t\treturn parseNextYamlObject(*list.back(), std::move(evt));\n> +\t\t};\n> +\t\treturn parseDictionaryOrList(YamlObject::List, handler);\n> +\t}\n> +\n> +\tcase YAML_MAPPING_START_EVENT: {\n> +\t\tyamlObject.type_ = YamlObject::Dictionary;\n> +\t\tauto &dictionary = yamlObject.dictionary_;\n> +\t\tauto handler = [this, &dictionary](EventPtr evtKey) {\n> +\t\t\t/* Parse key */\n> +\t\t\tif (evtKey->type != YAML_SCALAR_EVENT) {\n> +\t\t\t\tLOG(YamlParser, Error) << \"Expect key at line: \"\n> +\t\t\t\t\t\t       << evtKey->start_mark.line\n> +\t\t\t\t\t\t       << \" column: \"\n> +\t\t\t\t\t\t       << evtKey->start_mark.column;\n> +\t\t\t\treturn -EINVAL;\n> +\t\t\t}\n> +\n> +\t\t\tstd::string key;\n> +\t\t\treadValue(key, std::move(evtKey));\n> +\n> +\t\t\t/* Parse value */\n> +\t\t\tEventPtr evtValue = nextEvent();\n> +\t\t\tif (!evtValue)\n> +\t\t\t\treturn -EINVAL;\n> +\n> +\t\t\tauto elem = dictionary.emplace(key, std::make_unique<YamlObject>());\n> +\t\t\treturn parseNextYamlObject(*elem.first->second.get(), std::move(evtValue));\n> +\t\t};\n> +\t\treturn parseDictionaryOrList(YamlObject::Dictionary, handler);\n> +\t}\n\nMissing blank line.\n\nI'll fix these when applying.\n\n> +\tdefault:\n> +\t\tLOG(YamlParser, Error) << \"Invalid YAML file\";\n> +\t\treturn -EINVAL;\n> +\t}\n> +}\n\n[snip]","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id AB9E7C0F2A\n\tfor <parsemail@patchwork.libcamera.org>;\n\tMon,  9 May 2022 17:10:35 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C229D65646;\n\tMon,  9 May 2022 19:10:34 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2916E604A2\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon,  9 May 2022 19:10:33 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 86A06E0C;\n\tMon,  9 May 2022 19:10:32 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1652116234;\n\tbh=bjrDyb8Jwv6i3/EK8Gvxda6B5JVI/FzwEZcvHX/kciU=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=f6mEoR/hYvtk5QGSuv0nhGyiooW/tjDAe5Z+QttrhKkOU/nqk36n3//AWmxVteKlv\n\t4VG0hCjS8uDIKHgcyD7DTZSTggIMfiyBmfDzpoNAtucZqw7+teueE0pPuT8SUHJ7aT\n\tok/bqnkcs7tsUY+LVPq6ejn3i4/nmgXPdUcqeg8K44UgXNqaXv80nIRl1vU0H1EeJv\n\t4551Pvj38tgEWAwh/DPi/GIRj6vbWxz2cM7zzkw09jMVF8Osgvhs7/aVJ/4be3dmac\n\tCW0woyFVNnC1Aev1cOp85ySQj6UXo6rsaFEUovRho0pVOVGoYvtVkGH3z9Je+hrPcQ\n\tRtU+Kocr0mXQQ==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1652116232;\n\tbh=bjrDyb8Jwv6i3/EK8Gvxda6B5JVI/FzwEZcvHX/kciU=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=azZd2XYJJSufp8Be2tDdBKn3VZxphjquDyumugFiULvGhLtzi7aTEt9wF+YwnHDds\n\t7yuUD7ExiHJCOyoXa9p4UQeC+khbbE4RSxfh56u9BQopJVd/eXBhxaKZguBAodaNje\n\tDjX+H++WoBM3UgN4le3KzSHI1QNY29Hrje/e9HoA="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"azZd2XYJ\"; dkim-atps=neutral","Date":"Mon, 9 May 2022 20:10:28 +0300","To":"Han-Lin Chen <hanlinchen@chromium.org>","Message-ID":"<YnlLBLXr9mKJRCkE@pendragon.ideasonboard.com>","References":"<20220427140929.429977-1-hanlinchen@chromium.org>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20220427140929.429977-1-hanlinchen@chromium.org>","Subject":"Re: [libcamera-devel] [PATCH v6 1/3] libcamera: Introduce\n\tYamlParser as a helper to parse yaml files","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","From":"Laurent Pinchart via libcamera-devel\n\t<libcamera-devel@lists.libcamera.org>","Reply-To":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":22913,"web_url":"https://patchwork.libcamera.org/comment/22913/","msgid":"<YnlL0birKl+NRVIr@pendragon.ideasonboard.com>","date":"2022-05-09T17:13:53","subject":"Re: [libcamera-devel] [PATCH v6 1/3] libcamera: Introduce\n\tYamlParser as a helper to parse yaml files","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Mon, May 09, 2022 at 08:10:28PM +0300, Laurent Pinchart via libcamera-devel wrote:\n> Hi Han-Lin,\n> \n> Thank you for the patch.\n> \n> On Wed, Apr 27, 2022 at 10:09:27PM +0800, Han-Lin Chen via libcamera-devel wrote:\n> > Introduce YamlParser as a helper to convert contents of a yaml file to\n> > a tree based structure for easier reading, and to avoid writing parser\n> > with raw yaml tokens. The class is based on libyaml, and only support\n> > reading but not writing a yaml file.\n> > \n> > The interface is inspired by Json::Value class from jsoncpp:\n> > http://jsoncpp.sourceforge.net/class_json_1_1_value.html\n> > \n> > Signed-off-by: Han-Lin Chen <hanlinchen@chromium.org>\n> > ---\n> >  README.rst                               |   4 +-\n> >  include/libcamera/internal/meson.build   |   1 +\n> >  include/libcamera/internal/yaml_parser.h |  87 +++\n> >  src/libcamera/meson.build                |   3 +\n> >  src/libcamera/yaml_parser.cpp            | 679 +++++++++++++++++++++++\n> >  5 files changed, 772 insertions(+), 2 deletions(-)\n> >  create mode 100644 include/libcamera/internal/yaml_parser.h\n> >  create mode 100644 src/libcamera/yaml_parser.cpp\n> \n> [snip]\n> \n> > diff --git a/src/libcamera/yaml_parser.cpp b/src/libcamera/yaml_parser.cpp\n> > new file mode 100644\n> > index 00000000..4a047494\n> > --- /dev/null\n> > +++ b/src/libcamera/yaml_parser.cpp\n> > @@ -0,0 +1,679 @@\n> \n> [snip]\n> \n> > +/**\n> > + * \\fn YamlParserContext::parseDictionaryOrList()\n> > + * \\brief A helper function to abstract the common part of parsing dictionary or list\n> > + *\n> > + * \\param[in] isDictionary True for parsing a dictionary, and false for a list\n> > + * \\param[in] parseItem The callback to handle an item\n> > + *\n> > + * A helper function to abstract parsing an item from a dictionary or a list.\n> > + * The differences of them in a YAML event stream are:\n> > + *\n> > + * 1. The start and end event types are different\n> > + * 2. There is a leading scalar string as key in the items of a dictionary\n> > + *\n> > + * The caller should handle the leading key string in its callback parseItem\n> > + * when it's a dictionary.\n> > + *\n> > + * \\return 0 on success or a negative error code otherwise\n> > + * \\retval -EINVAL The parser is failed to initialize\n> > + */\n> > +int YamlParserContext::parseDictionaryOrList(YamlObject::Type type,\n> > +\t\t\t\t\t     const std::function<int(EventPtr event)> &parseItem)\n> > +{\n> > +\tyaml_event_type_t endEventType = YAML_SEQUENCE_END_EVENT;\n> > +\tif (type == YamlObject::Dictionary)\n> > +\t\tendEventType = YAML_MAPPING_END_EVENT;\n> > +\n> > +\t/*\n> > +\t * Add a safety counter to make sure we don't loop indefinitely in case\n> > +\t * the YAML file is malformed.\n> > +\t */\n> > +\tfor (unsigned int sentinel = 1000; sentinel; sentinel--) {\n> > +\t\tauto evt = nextEvent();\n> > +\t\tif (!evt)\n> > +\t\t\treturn -EINVAL;\n> > +\n> > +\t\tif (evt->type == endEventType)\n> > +\t\t\treturn 0;\n> > +\n> > +\t\tint ret = parseItem(std::move(evt));\n> > +\t\tif (ret)\n> > +\t\t\treturn ret;\n> > +\t}\n> > +\n> > +\tLOG(YamlParser, Error) << \"The YAML file contains a List or Dictionary\"\n> > +\t\t\t\t  \" whose size exceeding the parser's limit(1000)\";\n> \n> s/exceeding/exceeds/\n> \n> > +\n> > +\treturn -EINVAL;\n> > +}\n> > +\n> > +/**\n> > + * \\fn YamlParserContext::parseNextYamlObject()\n> > + * \\brief Parse next YAML event and read it as a YamlObject\n> > + * \\param[in] yamlObject The result of YamlObject\n> > + * \\param[in] event The leading event of the object\n> > + *\n> > + * Parse next YAML object separately as a value, list or dictionary.\n> > + *\n> > + * \\return 0 on success or a negative error code otherwise\n> > + * \\retval -EINVAL Fail to parse the YAML file.\n> > + */\n> > +int YamlParserContext::parseNextYamlObject(YamlObject &yamlObject, EventPtr event)\n> > +{\n> > +\tif (!event)\n> > +\t\treturn -EINVAL;\n> > +\n> > +\tswitch (event->type) {\n> > +\tcase YAML_SCALAR_EVENT:\n> > +\t\tyamlObject.type_ = YamlObject::Value;\n> > +\t\treadValue(yamlObject.value_, std::move(event));\n> > +\t\treturn 0;\n> > +\n> > +\tcase YAML_SEQUENCE_START_EVENT: {\n> > +\t\tyamlObject.type_ = YamlObject::List;\n> > +\t\tauto &list = yamlObject.list_;\n> > +\t\tauto handler = [this, &list](EventPtr evt) {\n> > +\t\t\tlist.emplace_back(new YamlObject());\n> > +\t\t\treturn parseNextYamlObject(*list.back(), std::move(evt));\n> > +\t\t};\n> > +\t\treturn parseDictionaryOrList(YamlObject::List, handler);\n> > +\t}\n> > +\n> > +\tcase YAML_MAPPING_START_EVENT: {\n> > +\t\tyamlObject.type_ = YamlObject::Dictionary;\n> > +\t\tauto &dictionary = yamlObject.dictionary_;\n> > +\t\tauto handler = [this, &dictionary](EventPtr evtKey) {\n> > +\t\t\t/* Parse key */\n> > +\t\t\tif (evtKey->type != YAML_SCALAR_EVENT) {\n> > +\t\t\t\tLOG(YamlParser, Error) << \"Expect key at line: \"\n> > +\t\t\t\t\t\t       << evtKey->start_mark.line\n> > +\t\t\t\t\t\t       << \" column: \"\n> > +\t\t\t\t\t\t       << evtKey->start_mark.column;\n> > +\t\t\t\treturn -EINVAL;\n> > +\t\t\t}\n> > +\n> > +\t\t\tstd::string key;\n> > +\t\t\treadValue(key, std::move(evtKey));\n> > +\n> > +\t\t\t/* Parse value */\n> > +\t\t\tEventPtr evtValue = nextEvent();\n> > +\t\t\tif (!evtValue)\n> > +\t\t\t\treturn -EINVAL;\n> > +\n> > +\t\t\tauto elem = dictionary.emplace(key, std::make_unique<YamlObject>());\n> > +\t\t\treturn parseNextYamlObject(*elem.first->second.get(), std::move(evtValue));\n> > +\t\t};\n> > +\t\treturn parseDictionaryOrList(YamlObject::Dictionary, handler);\n> > +\t}\n> \n> Missing blank line.\n> \n> I'll fix these when applying.\n\nAnd\n\nReviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>\n\n:-)\n\n> > +\tdefault:\n> > +\t\tLOG(YamlParser, Error) << \"Invalid YAML file\";\n> > +\t\treturn -EINVAL;\n> > +\t}\n> > +}\n> \n> [snip]","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A564DC3256\n\tfor <parsemail@patchwork.libcamera.org>;\n\tMon,  9 May 2022 17:13:59 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 5E30A65641;\n\tMon,  9 May 2022 19:13:59 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 0381F604A2\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon,  9 May 2022 19:13:58 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 7C13BE0C;\n\tMon,  9 May 2022 19:13:57 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1652116439;\n\tbh=V5GeGq82XIXOcyFH4k5hhP5404HTfuy0Q7Cd+ASPZCE=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=rch8+A3gKzxlImVUCWJlcudkJG+cRbWtvFQPA1GL7554CslXobyhHK1ywDRvuYywv\n\tZys+YH9HCXupyMHlAahHeQuKdOw63L0r/JKTPpcWNLO0MLIlb73anEHT5Y4Nh+Rlfv\n\tvTelF+HRka6SAeo4Vf/AA/qN2Zd3FavN5n4hP1ryQ4I0Ktm6HR27kjHbkIC6N6bm/g\n\t3bGF9/OLFvxu3InS0AVdKIl0/+gqKJv7ARdLv5ZSL8kR6q8XJN0Ti1sCTvGMTnPgBW\n\t7btME5Xxvi67Ex9XA2wOQ26gobuG2nDR7ARxDY84msz5GBnNIkKuH3Ry/0J72jrXR2\n\tsU757nR/OxbjA==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1652116437;\n\tbh=V5GeGq82XIXOcyFH4k5hhP5404HTfuy0Q7Cd+ASPZCE=;\n\th=Date:From:To:Subject:References:In-Reply-To:From;\n\tb=iua/7PpQD+/NfqP4VpHKWJdzVecWTMnLDYrnDDTVmWSUeKC5UPVDnNeQn1N2EMhL1\n\tblCTT3fADOPci8D5z4RJPMopDwTetAkA8AXRI/g3Fa2/1fq2uUJO+cgGjvJZWm6W4K\n\tpsU9VscFmMUd39Jx/FdvPLBBL2flKhG0b+mCUfcE="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"iua/7PpQ\"; dkim-atps=neutral","Date":"Mon, 9 May 2022 20:13:53 +0300","To":"Han-Lin Chen <hanlinchen@chromium.org>,\n\tlibcamera-devel@lists.libcamera.org","Message-ID":"<YnlL0birKl+NRVIr@pendragon.ideasonboard.com>","References":"<20220427140929.429977-1-hanlinchen@chromium.org>\n\t<YnlLBLXr9mKJRCkE@pendragon.ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<YnlLBLXr9mKJRCkE@pendragon.ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v6 1/3] libcamera: Introduce\n\tYamlParser as a helper to parse yaml files","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","From":"Laurent Pinchart via libcamera-devel\n\t<libcamera-devel@lists.libcamera.org>","Reply-To":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":22915,"web_url":"https://patchwork.libcamera.org/comment/22915/","msgid":"<165212950688.2416244.17654378141772397615@Monstersaurus>","date":"2022-05-09T20:51:46","subject":"Re: [libcamera-devel] [PATCH v6 1/3] libcamera: Introduce\n\tYamlParser as a helper to parse yaml files","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Han-Lin Chen via libcamera-devel (2022-04-27 15:09:27)\n> Introduce YamlParser as a helper to convert contents of a yaml file to\n> a tree based structure for easier reading, and to avoid writing parser\n> with raw yaml tokens. The class is based on libyaml, and only support\n> reading but not writing a yaml file.\n> \n> The interface is inspired by Json::Value class from jsoncpp:\n> http://jsoncpp.sourceforge.net/class_json_1_1_value.html\n> \n> Signed-off-by: Han-Lin Chen <hanlinchen@chromium.org>\n\nThis causes compile failures on GCC-8. (I haven't yet tested further\nthan that, as it's the first failure that hits).\n\nFAILED: src/libcamera/libcamera.so.0.0.0.p/yaml_parser.cpp.o \ng++-8 -Isrc/libcamera/libcamera.so.0.0.0.p -Isrc/libcamera -I../../../src/libcamera/src/libcamera -Iinclude -I../../../src/libcamera/include -Iinclude/libcamera -Iinclude/libcamera/ipa -Iinclude/libcamera/internal -Isrc/libcamera/proxy -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wnon-virtual-dtor -Wextra -Werror -std=c++17 -g -Wl,--start-group -lstdc++fs -Wl,--end-group -Wshadow -include config.h -fPIC -DLIBCAMERA_BASE_PRIVATE -MD -MQ src/libcamera/libcamera.so.0.0.0.p/yaml_parser.cpp.o -MF src/libcamera/libcamera.so.0.0.0.p/yaml_parser.cpp.o.d -o src/libcamera/libcamera.so.0.0.0.p/yaml_parser.cpp.o -c ../../../src/libcamera/src/libcamera/yaml_parser.cpp\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:360:18: error: ‘function’ in namespace ‘std’ does not name a template type\n       const std::function<int(EventPtr event)> &parseItem);\n                  ^~~~~~~~\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:360:13: note: ‘std::function’ is defined in header ‘<functional>’; did you forget to ‘#include <functional>’?\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:16:1:\n+#include <functional>\n \n../../../src/libcamera/src/libcamera/yaml_parser.cpp:360:13:\n       const std::function<int(EventPtr event)> &parseItem);\n             ^~~\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:360:26: error: expected ‘,’ or ‘...’ before ‘<’ token\n       const std::function<int(EventPtr event)> &parseItem);\n                          ^\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:509:22: error: ‘function’ in namespace ‘std’ does not name a template type\n           const std::function<int(EventPtr event)> &parseItem)\n                      ^~~~~~~~\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:509:17: note: ‘std::function’ is defined in header ‘<functional>’; did you forget to ‘#include <functional>’?\n           const std::function<int(EventPtr event)> &parseItem)\n                 ^~~\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:509:30: error: expected ‘,’ or ‘...’ before ‘<’ token\n           const std::function<int(EventPtr event)> &parseItem)\n                              ^\n../../../src/libcamera/src/libcamera/yaml_parser.cpp: In member function ‘int libcamera::YamlParserContext::parseDictionaryOrList(libcamera::YamlObject::Type, int)’:\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:527:13: error: ‘parseItem’ was not declared in this scope\n   int ret = parseItem(std::move(evt));\n             ^~~~~~~~~\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:527:13: note: suggested alternative: ‘parser_’\n   int ret = parseItem(std::move(evt));\n             ^~~~~~~~~\n             parser_\n../../../src/libcamera/src/libcamera/yaml_parser.cpp: In member function ‘int libcamera::YamlParserContext::parseNextYamlObject(libcamera::YamlObject&, libcamera::YamlParserContext::EventPtr)’:\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:567:57: error: no matching function for call to ‘libcamera::YamlParserContext::parseDictionaryOrList(libcamera::YamlObject::Type, libcamera::YamlParserContext::parseNextYamlObject(libcamera::YamlObject&, libcamera::YamlParserContext::EventPtr)::<lambda(libcamera::YamlParserContext::EventPtr)>&)’\n   return parseDictionaryOrList(YamlObject::List, handler);\n                                                         ^\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:508:5: note: candidate: ‘int libcamera::YamlParserContext::parseDictionaryOrList(libcamera::YamlObject::Type, int)’\n int YamlParserContext::parseDictionaryOrList(YamlObject::Type type,\n     ^~~~~~~~~~~~~~~~~\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:508:5: note:   no known conversion for argument 2 from ‘libcamera::YamlParserContext::parseNextYamlObject(libcamera::YamlObject&, libcamera::YamlParserContext::EventPtr)::<lambda(libcamera::YamlParserContext::EventPtr)>’ to ‘int’\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:594:63: error: no matching function for call to ‘libcamera::YamlParserContext::parseDictionaryOrList(libcamera::YamlObject::Type, libcamera::YamlParserContext::parseNextYamlObject(libcamera::YamlObject&, libcamera::YamlParserContext::EventPtr)::<lambda(libcamera::YamlParserContext::EventPtr)>&)’\n   return parseDictionaryOrList(YamlObject::Dictionary, handler);\n                                                               ^\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:508:5: note: candidate: ‘int libcamera::YamlParserContext::parseDictionaryOrList(libcamera::YamlObject::Type, int)’\n int YamlParserContext::parseDictionaryOrList(YamlObject::Type type,\n     ^~~~~~~~~~~~~~~~~\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:508:5: note:   no known conversion for argument 2 from ‘libcamera::YamlParserContext::parseNextYamlObject(libcamera::YamlObject&, libcamera::YamlParserContext::EventPtr)::<lambda(libcamera::YamlParserContext::EventPtr)>’ to ‘int’\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:567:58: error: this statement may fall through [-Werror=implicit-fallthrough=]\n   return parseDictionaryOrList(YamlObject::List, handler);\n                                                          ^\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:570:2: note: here\n  case YAML_MAPPING_START_EVENT: {\n  ^~~~\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:594:64: error: this statement may fall through [-Werror=implicit-fallthrough=]\n   return parseDictionaryOrList(YamlObject::Dictionary, handler);\n                                                                ^\n../../../src/libcamera/src/libcamera/yaml_parser.cpp:597:2: note: here\n  default:\n  ^~~~~~~\ncc1plus: all warnings being treated as errors\n\nAre we still supporting gcc-8 ?\n\n--\nKieran\n\n\n> ---\n>  README.rst                               |   4 +-\n>  include/libcamera/internal/meson.build   |   1 +\n>  include/libcamera/internal/yaml_parser.h |  87 +++\n>  src/libcamera/meson.build                |   3 +\n>  src/libcamera/yaml_parser.cpp            | 679 +++++++++++++++++++++++\n>  5 files changed, 772 insertions(+), 2 deletions(-)\n>  create mode 100644 include/libcamera/internal/yaml_parser.h\n>  create mode 100644 src/libcamera/yaml_parser.cpp\n> \n> diff --git a/README.rst b/README.rst\n> index aae6b79f..b5a2d448 100644\n> --- a/README.rst\n> +++ b/README.rst\n> @@ -60,7 +60,7 @@ Meson Build system: [required]\n>              pip3 install --user --upgrade meson\n>  \n>  for the libcamera core: [required]\n> -        python3-yaml python3-ply python3-jinja2\n> +        libyaml-dev python3-yaml python3-ply python3-jinja2\n>  \n>  for IPA module signing: [required]\n>          libgnutls28-dev openssl\n> @@ -98,7 +98,7 @@ for tracing with lttng: [optional]\n>          liblttng-ust-dev python3-jinja2 lttng-tools\n>  \n>  for android: [optional]\n> -        libexif-dev libjpeg-dev libyaml-dev\n> +        libexif-dev libjpeg-dev\n>  \n>  for lc-compliance: [optional]\n>          libevent-dev\n> diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build\n> index c9e055d4..7a780d48 100644\n> --- a/include/libcamera/internal/meson.build\n> +++ b/include/libcamera/internal/meson.build\n> @@ -42,4 +42,5 @@ libcamera_internal_headers = files([\n>      'v4l2_pixelformat.h',\n>      'v4l2_subdevice.h',\n>      'v4l2_videodevice.h',\n> +    'yaml_parser.h',\n>  ])\n> diff --git a/include/libcamera/internal/yaml_parser.h b/include/libcamera/internal/yaml_parser.h\n> new file mode 100644\n> index 00000000..3a4f3052\n> --- /dev/null\n> +++ b/include/libcamera/internal/yaml_parser.h\n> @@ -0,0 +1,87 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2022, Google Inc.\n> + *\n> + * yaml_parser.h - libcamera YAML parsing helper\n> + */\n> +\n> +#pragma once\n> +\n> +#include <cstdio>\n> +#include <map>\n> +#include <string>\n> +#include <vector>\n> +\n> +#include <libcamera/base/class.h>\n> +\n> +#include <libcamera/geometry.h>\n> +\n> +namespace libcamera {\n> +\n> +class YamlParserContext;\n> +\n> +class YamlObject\n> +{\n> +public:\n> +       YamlObject();\n> +       ~YamlObject();\n> +\n> +       bool isValue() const\n> +       {\n> +               return type_ == Value;\n> +       }\n> +       bool isList() const\n> +       {\n> +               return type_ == List;\n> +       }\n> +       bool isDictionary() const\n> +       {\n> +               return type_ == Dictionary;\n> +       }\n> +\n> +#ifndef __DOXYGEN__\n> +       template<typename T,\n> +                typename std::enable_if_t<\n> +                        std::is_same<bool, T>::value ||\n> +                        std::is_same<double, T>::value ||\n> +                        std::is_same<int32_t, T>::value ||\n> +                        std::is_same<uint32_t, T>::value ||\n> +                        std::is_same<std::string, T>::value ||\n> +                        std::is_same<Size, T>::value> * = nullptr>\n> +#else\n> +       template<typename T>\n> +#endif\n> +       T get(const T &defaultValue, bool *ok = nullptr) const;\n> +\n> +       std::size_t size() const;\n> +       const YamlObject &operator[](std::size_t index) const;\n> +\n> +       bool contains(const std::string &key) const;\n> +       const YamlObject &operator[](const std::string &key) const;\n> +       std::vector<std::string> memberNames() const;\n> +\n> +private:\n> +       LIBCAMERA_DISABLE_COPY_AND_MOVE(YamlObject)\n> +\n> +       friend class YamlParserContext;\n> +\n> +       enum Type {\n> +               Dictionary,\n> +               List,\n> +               Value,\n> +       };\n> +\n> +       Type type_;\n> +\n> +       std::string value_;\n> +       std::vector<std::unique_ptr<YamlObject>> list_;\n> +       std::map<const std::string, std::unique_ptr<YamlObject>> dictionary_;\n> +};\n> +\n> +class YamlParser final\n> +{\n> +public:\n> +       static std::unique_ptr<YamlObject> parse(std::FILE *fh);\n> +};\n> +\n> +} /* namespace libcamera */\n> diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build\n> index 26912ca1..f8e18e03 100644\n> --- a/src/libcamera/meson.build\n> +++ b/src/libcamera/meson.build\n> @@ -46,6 +46,7 @@ libcamera_sources = files([\n>      'v4l2_pixelformat.cpp',\n>      'v4l2_subdevice.cpp',\n>      'v4l2_videodevice.cpp',\n> +    'yaml_parser.cpp',\n>  ])\n>  \n>  libcamera_sources += libcamera_public_headers\n> @@ -66,6 +67,7 @@ subdir('proxy')\n>  libdl = cc.find_library('dl')\n>  libgnutls = cc.find_library('gnutls', required : true)\n>  libudev = dependency('libudev', required : false)\n> +libyaml = dependency('yaml-0.1', required : true)\n>  \n>  if libgnutls.found()\n>      config_h.set('HAVE_GNUTLS', 1)\n> @@ -126,6 +128,7 @@ libcamera_deps = [\n>      libgnutls,\n>      liblttng,\n>      libudev,\n> +    libyaml,\n>  ]\n>  \n>  # We add '/' to the build_rpath as a 'safe' path to act as a boolean flag.\n> diff --git a/src/libcamera/yaml_parser.cpp b/src/libcamera/yaml_parser.cpp\n> new file mode 100644\n> index 00000000..4a047494\n> --- /dev/null\n> +++ b/src/libcamera/yaml_parser.cpp\n> @@ -0,0 +1,679 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2022, Google Inc.\n> + *\n> + * yaml_parser.cpp - libcamera YAML parsing helper\n> + */\n> +\n> +#include \"libcamera/internal/yaml_parser.h\"\n> +\n> +#include <cerrno>\n> +#include <cstdlib>\n> +\n> +#include <libcamera/base/log.h>\n> +\n> +#include <yaml.h>\n> +\n> +/**\n> + * \\file libcamera/internal/yaml_parser.h\n> + * \\brief A YAML parser helper\n> + */\n> +\n> +namespace libcamera {\n> +\n> +LOG_DEFINE_CATEGORY(YamlParser)\n> +\n> +namespace {\n> +\n> +/* Empty static YamlObject as a safe result for invalid operations */\n> +static const YamlObject empty;\n> +\n> +void setOk(bool *ok, bool result)\n> +{\n> +       if (ok)\n> +               *ok = result;\n> +}\n> +\n> +} /* namespace */\n> +\n> +/**\n> + * \\class YamlObject\n> + * \\brief A class representing the tree structure of the YAML content\n> + *\n> + * The YamlObject class represents the tree structure of YAML content. A\n> + * YamlObject can be a dictionary or list of YamlObjects or a value if a tree\n> + * leaf.\n> + */\n> +\n> +YamlObject::YamlObject()\n> +       : type_(Value)\n> +{\n> +}\n> +\n> +YamlObject::~YamlObject() = default;\n> +\n> +/**\n> + * \\fn YamlObject::isValue()\n> + * \\brief Return whether the YamlObject is a value\n> + *\n> + * \\return True if the YamlObject is a value, false otherwise\n> + */\n> +\n> +/**\n> + * \\fn YamlObject::isList()\n> + * \\brief Return whether the YamlObject is a list\n> + *\n> + * \\return True if the YamlObject is a list, false otherwise\n> + */\n> +\n> +/**\n> + * \\fn YamlObject::isDictionary()\n> + * \\brief Return whether the YamlObject is a dictionary\n> + *\n> + * \\return True if the YamlObject is a dictionary, false otherwise\n> + */\n> +\n> +/**\n> + * \\fn template<typename T> YamlObject::get<T>(\n> + *     const T &defaultValue, bool *ok) const\n> + * \\brief Parse the YamlObject as a \\a T value\n> + * \\param[in] defaultValue The default value when failing to parse\n> + * \\param[out] ok The result of whether the parse succeeded\n> + *\n> + * This function parses the value of the YamlObject as a \\a T object, and\n> + * returns the value. If parsing fails (usually because the YamlObject doesn't\n> + * store a \\a T value), the \\a defaultValue is returned, and \\a ok is set to\n> + * false. Otherwise, the YamlObject value is returned, and \\a ok is set to true.\n> + *\n> + * The \\a ok pointer is optional and can be a nullptr if the caller doesn't\n> + * need to know if parsing succeeded.\n> + *\n> + * \\return Value as a bool type\n> + */\n> +\n> +#ifndef __DOXYGEN__\n> +\n> +template<>\n> +bool YamlObject::get(const bool &defaultValue, bool *ok) const\n> +{\n> +       setOk(ok, false);\n> +\n> +       if (type_ != Value)\n> +               return defaultValue;\n> +\n> +       if (value_ == \"true\") {\n> +               setOk(ok, true);\n> +               return true;\n> +       } else if (value_ == \"false\") {\n> +               setOk(ok, true);\n> +               return false;\n> +       }\n> +\n> +       return defaultValue;\n> +}\n> +\n> +template<>\n> +int32_t YamlObject::get(const int32_t &defaultValue, bool *ok) const\n> +{\n> +       setOk(ok, false);\n> +\n> +       if (type_ != Value)\n> +               return defaultValue;\n> +\n> +       if (value_ == \"\")\n> +               return defaultValue;\n> +\n> +       char *end;\n> +\n> +       errno = 0;\n> +       int32_t value = std::strtol(value_.c_str(), &end, 10);\n> +\n> +       if ('\\0' != *end || errno == ERANGE)\n> +               return defaultValue;\n> +\n> +       setOk(ok, true);\n> +       return value;\n> +}\n> +\n> +template<>\n> +uint32_t YamlObject::get(const uint32_t &defaultValue, bool *ok) const\n> +{\n> +       setOk(ok, false);\n> +\n> +       if (type_ != Value)\n> +               return defaultValue;\n> +\n> +       if (value_ == \"\")\n> +               return defaultValue;\n> +\n> +       /*\n> +        * libyaml parses all scalar values as strings. When a string has\n> +        * leading spaces before a minus sign, for example \"  -10\", strtoul\n> +        * skips leading spaces, accepts the leading minus sign, and the\n> +        * calculated digits are negated as if by unary minus. Rule it out in\n> +        * case the user gets a large number when the value is negative.\n> +        */\n> +       std::size_t found = value_.find_first_not_of(\" \\t\");\n> +       if (found != std::string::npos && value_[found] == '-')\n> +               return defaultValue;\n> +\n> +       char *end;\n> +\n> +       errno = 0;\n> +       uint32_t value = std::strtoul(value_.c_str(), &end, 10);\n> +\n> +       if ('\\0' != *end || errno == ERANGE)\n> +               return defaultValue;\n> +\n> +       setOk(ok, true);\n> +       return value;\n> +}\n> +\n> +template<>\n> +double YamlObject::get(const double &defaultValue, bool *ok) const\n> +{\n> +       setOk(ok, false);\n> +\n> +       if (type_ != Value)\n> +               return defaultValue;\n> +\n> +       if (value_ == \"\")\n> +               return defaultValue;\n> +\n> +       char *end;\n> +\n> +       errno = 0;\n> +       double value = std::strtod(value_.c_str(), &end);\n> +\n> +       if ('\\0' != *end || errno == ERANGE)\n> +               return defaultValue;\n> +\n> +       setOk(ok, true);\n> +       return value;\n> +}\n> +\n> +template<>\n> +std::string YamlObject::get(const std::string &defaultValue, bool *ok) const\n> +{\n> +       setOk(ok, false);\n> +\n> +       if (type_ != Value)\n> +               return defaultValue;\n> +\n> +       setOk(ok, true);\n> +       return value_;\n> +}\n> +\n> +template<>\n> +Size YamlObject::get(const Size &defaultValue, bool *ok) const\n> +{\n> +       setOk(ok, false);\n> +\n> +       if (type_ != List)\n> +               return defaultValue;\n> +\n> +       if (list_.size() != 2)\n> +               return defaultValue;\n> +\n> +       /*\n> +        * Add a local variable to validate each dimension in case\n> +        * that ok == nullptr.\n> +        */\n> +       bool valid;\n> +       uint32_t width = list_[0]->get<uint32_t>(0, &valid);\n> +       if (!valid)\n> +               return defaultValue;\n> +\n> +       uint32_t height = list_[1]->get<uint32_t>(0, &valid);\n> +       if (!valid)\n> +               return defaultValue;\n> +\n> +       setOk(ok, true);\n> +       return Size(width, height);\n> +}\n> +\n> +#endif /* __DOXYGEN__ */\n> +\n> +/**\n> + * \\fn YamlObject::size()\n> + * \\brief Retrieve the number of elements in a list YamlObject\n> + *\n> + * This function retrieves the size of the YamlObject, defined as the number of\n> + * child elements it contains. Only YamlObject instances of List type have a\n> + * size, calling this function on other types of instances is invalid and\n> + * results in undefined behaviour.\n> + *\n> + * \\return The size of the YamlObject\n> + */\n> +std::size_t YamlObject::size() const\n> +{\n> +       if (type_ != List)\n> +               return 0;\n> +\n> +       return list_.size();\n> +}\n> +\n> +/**\n> + * \\fn YamlObject::operator[](std::size_t index) const\n> + * \\brief Retrieve the element from list YamlObject by index\n> + *\n> + * This function retrieves an element of the YamlObject. Only YamlObject\n> + * instances of List type associate elements with index, calling this function\n> + * on other types of instances is invalid and results in undefined behaviour.\n> + *\n> + * \\return The YamlObject as an element of the list\n> + */\n> +const YamlObject &YamlObject::operator[](std::size_t index) const\n> +{\n> +       if (type_ != List || index >= size())\n> +               return empty;\n> +\n> +       return *list_[index];\n> +}\n> +\n> +/**\n> + * \\fn YamlObject::contains()\n> + * \\brief Check if an element of a dictionary exists\n> + *\n> + * This function check if the YamlObject contains an element. Only YamlObject\n> + * instances of Dictionary type associate elements with names, calling this\n> + * function on other types of instances is invalid and results in undefined\n> + * behaviour.\n> + *\n> + * \\return True if an element exists, false otherwise\n> + */\n> +bool YamlObject::contains(const std::string &key) const\n> +{\n> +       if (dictionary_.find(key) == dictionary_.end())\n> +               return false;\n> +\n> +       return true;\n> +}\n> +\n> +/**\n> + * \\fn YamlObject::memberNames()\n> + * \\brief Retrieve all member names of the dictionary\n> + *\n> + * This function retrieve member names of a YamlObject. Only YamlObject\n> + * instances of Dictionary type associate elements with names, calling this\n> + * function on other types of instances is invalid and results in undefined\n> + * behaviour.\n> + *\n> + * \\todo Replace this function with an iterator-based API\n> + *\n> + * \\return A vector of string as the member names\n> + */\n> +std::vector<std::string> YamlObject::memberNames() const\n> +{\n> +       std::vector<std::string> memberNames;\n> +       for (auto &[key, _] : dictionary_)\n> +               memberNames.push_back(key);\n> +\n> +       return memberNames;\n> +}\n> +\n> +/**\n> + * \\fn YamlObject::operator[](const std::string &key) const\n> + * \\brief Retrieve a member by name from the dictionary\n> + *\n> + * This function retrieve a member of a YamlObject by name. Only YamlObject\n> + * instances of Dictionary type associate elements with names, calling this\n> + * function on other types of instances is invalid and results in undefined\n> + * behaviour.\n> + *\n> + * \\return The YamlObject corresponding to the \\a key member\n> + */\n> +const YamlObject &YamlObject::operator[](const std::string &key) const\n> +{\n> +       if (type_ != Dictionary || !contains(key))\n> +               return empty;\n> +\n> +       auto iter = dictionary_.find(key);\n> +       return *iter->second;\n> +}\n> +\n> +#ifndef __DOXYGEN__\n> +\n> +class YamlParserContext\n> +{\n> +public:\n> +       YamlParserContext();\n> +       ~YamlParserContext();\n> +\n> +       int init(std::FILE *fh);\n> +       int parseContent(YamlObject &yamlObject);\n> +\n> +private:\n> +       struct EventDeleter {\n> +               void operator()(yaml_event_t *event) const\n> +               {\n> +                       yaml_event_delete(event);\n> +                       delete event;\n> +               }\n> +       };\n> +       using EventPtr = std::unique_ptr<yaml_event_t, EventDeleter>;\n> +\n> +       EventPtr nextEvent();\n> +\n> +       void readValue(std::string &value, EventPtr event);\n> +       int parseDictionaryOrList(YamlObject::Type type,\n> +                                 const std::function<int(EventPtr event)> &parseItem);\n> +       int parseNextYamlObject(YamlObject &yamlObject, EventPtr event);\n> +\n> +       bool parserValid_;\n> +       yaml_parser_t parser_;\n> +};\n> +\n> +/**\n> + * \\class YamlParserContext\n> + * \\brief Class for YamlParser parsing and context data\n> + *\n> + * The YamlParserContext class stores the internal yaml_parser_t and provides\n> + * helper functions to do event-based parsing for YAML files.\n> + */\n> +YamlParserContext::YamlParserContext()\n> +       : parserValid_(false)\n> +{\n> +}\n> +\n> +/**\n> + * \\class YamlParserContext\n> + * \\brief Destructor of YamlParserContext\n> + */\n> +YamlParserContext::~YamlParserContext()\n> +{\n> +       if (parserValid_) {\n> +               yaml_parser_delete(&parser_);\n> +               parserValid_ = false;\n> +       }\n> +}\n> +\n> +/**\n> + * \\fn YamlParserContext::init()\n> + * \\brief Initialize a parser with an opened file for parsing\n> + * \\param[in] fh The YAML file to parse\n> + *\n> + * Prior to parsing the YAML content, the YamlParserContext must be initialized\n> + * with an opened FILE to create an internal parser. The FILE needs to stay\n> + * valid during the process.\n> + *\n> + * \\return 0 on success or a negative error code otherwise\n> + * \\retval -EINVAL The parser has failed to initialize\n> + */\n> +int YamlParserContext::init(std::FILE *fh)\n> +{\n> +       /* yaml_parser_initialize returns 1 when it succeededs */\n> +       if (!yaml_parser_initialize(&parser_)) {\n> +               LOG(YamlParser, Error) << \"Failed to initialize YAML parser\";\n> +               return -EINVAL;\n> +       }\n> +       parserValid_ = true;\n> +       yaml_parser_set_input_file(&parser_, fh);\n> +\n> +       return 0;\n> +}\n> +\n> +/**\n> + * \\fn YamlParserContext::nextEvent()\n> + * \\brief Get the next event\n> + *\n> + * Get the next event in the current YAML event stream, and return nullptr when\n> + * there is no more event.\n> + *\n> + * \\return The next event on success or nullptr otherwise\n> + */\n> +YamlParserContext::EventPtr YamlParserContext::nextEvent()\n> +{\n> +       EventPtr event(new yaml_event_t);\n> +\n> +       /* yaml_parser_parse returns 1 when it succeeds */\n> +       if (!yaml_parser_parse(&parser_, event.get()))\n> +               return nullptr;\n> +\n> +       return event;\n> +}\n> +\n> +/**\n> + * \\fn YamlParserContext::parseContent()\n> + * \\brief Parse the content of a YAML document\n> + * \\param[in] yamlObject The result of YamlObject\n> + *\n> + * Check YAML start and end events of a YAML document, and parse the root object\n> + * of the YAML document into a YamlObject.\n> + *\n> + * \\return 0 on success or a negative error code otherwise\n> + * \\retval -EINVAL The parser has failed to validate end of a YAML file\n> + */\n> +int YamlParserContext::parseContent(YamlObject &yamlObject)\n> +{\n> +       /* Check start of the YAML file. */\n> +       EventPtr event = nextEvent();\n> +       if (!event || event->type != YAML_STREAM_START_EVENT)\n> +               return -EINVAL;\n> +\n> +       event = nextEvent();\n> +       if (!event || event->type != YAML_DOCUMENT_START_EVENT)\n> +               return -EINVAL;\n> +\n> +       /* Parse the root object. */\n> +       event = nextEvent();\n> +       if (parseNextYamlObject(yamlObject, std::move(event)))\n> +               return -EINVAL;\n> +\n> +       /* Check end of the YAML file. */\n> +       event = nextEvent();\n> +       if (!event || event->type != YAML_DOCUMENT_END_EVENT)\n> +               return -EINVAL;\n> +\n> +       event = nextEvent();\n> +       if (!event || event->type != YAML_STREAM_END_EVENT)\n> +               return -EINVAL;\n> +\n> +       return 0;\n> +}\n> +\n> +/**\n> + * \\fn YamlParserContext::readValue()\n> + * \\brief Parse event scalar and fill its content into a string\n> + * \\param[in] value The string reference to fill value\n> + *\n> + * A helper function to parse a scalar event as string. The caller needs to\n> + * guarantee the event is of scaler type.\n> + */\n> +void YamlParserContext::readValue(std::string &value, EventPtr event)\n> +{\n> +       value.assign(reinterpret_cast<char *>(event->data.scalar.value),\n> +                    event->data.scalar.length);\n> +}\n> +\n> +/**\n> + * \\fn YamlParserContext::parseDictionaryOrList()\n> + * \\brief A helper function to abstract the common part of parsing dictionary or list\n> + *\n> + * \\param[in] isDictionary True for parsing a dictionary, and false for a list\n> + * \\param[in] parseItem The callback to handle an item\n> + *\n> + * A helper function to abstract parsing an item from a dictionary or a list.\n> + * The differences of them in a YAML event stream are:\n> + *\n> + * 1. The start and end event types are different\n> + * 2. There is a leading scalar string as key in the items of a dictionary\n> + *\n> + * The caller should handle the leading key string in its callback parseItem\n> + * when it's a dictionary.\n> + *\n> + * \\return 0 on success or a negative error code otherwise\n> + * \\retval -EINVAL The parser is failed to initialize\n> + */\n> +int YamlParserContext::parseDictionaryOrList(YamlObject::Type type,\n> +                                            const std::function<int(EventPtr event)> &parseItem)\n> +{\n> +       yaml_event_type_t endEventType = YAML_SEQUENCE_END_EVENT;\n> +       if (type == YamlObject::Dictionary)\n> +               endEventType = YAML_MAPPING_END_EVENT;\n> +\n> +       /*\n> +        * Add a safety counter to make sure we don't loop indefinitely in case\n> +        * the YAML file is malformed.\n> +        */\n> +       for (unsigned int sentinel = 1000; sentinel; sentinel--) {\n> +               auto evt = nextEvent();\n> +               if (!evt)\n> +                       return -EINVAL;\n> +\n> +               if (evt->type == endEventType)\n> +                       return 0;\n> +\n> +               int ret = parseItem(std::move(evt));\n> +               if (ret)\n> +                       return ret;\n> +       }\n> +\n> +       LOG(YamlParser, Error) << \"The YAML file contains a List or Dictionary\"\n> +                                 \" whose size exceeding the parser's limit(1000)\";\n> +\n> +       return -EINVAL;\n> +}\n> +\n> +/**\n> + * \\fn YamlParserContext::parseNextYamlObject()\n> + * \\brief Parse next YAML event and read it as a YamlObject\n> + * \\param[in] yamlObject The result of YamlObject\n> + * \\param[in] event The leading event of the object\n> + *\n> + * Parse next YAML object separately as a value, list or dictionary.\n> + *\n> + * \\return 0 on success or a negative error code otherwise\n> + * \\retval -EINVAL Fail to parse the YAML file.\n> + */\n> +int YamlParserContext::parseNextYamlObject(YamlObject &yamlObject, EventPtr event)\n> +{\n> +       if (!event)\n> +               return -EINVAL;\n> +\n> +       switch (event->type) {\n> +       case YAML_SCALAR_EVENT:\n> +               yamlObject.type_ = YamlObject::Value;\n> +               readValue(yamlObject.value_, std::move(event));\n> +               return 0;\n> +\n> +       case YAML_SEQUENCE_START_EVENT: {\n> +               yamlObject.type_ = YamlObject::List;\n> +               auto &list = yamlObject.list_;\n> +               auto handler = [this, &list](EventPtr evt) {\n> +                       list.emplace_back(new YamlObject());\n> +                       return parseNextYamlObject(*list.back(), std::move(evt));\n> +               };\n> +               return parseDictionaryOrList(YamlObject::List, handler);\n> +       }\n> +\n> +       case YAML_MAPPING_START_EVENT: {\n> +               yamlObject.type_ = YamlObject::Dictionary;\n> +               auto &dictionary = yamlObject.dictionary_;\n> +               auto handler = [this, &dictionary](EventPtr evtKey) {\n> +                       /* Parse key */\n> +                       if (evtKey->type != YAML_SCALAR_EVENT) {\n> +                               LOG(YamlParser, Error) << \"Expect key at line: \"\n> +                                                      << evtKey->start_mark.line\n> +                                                      << \" column: \"\n> +                                                      << evtKey->start_mark.column;\n> +                               return -EINVAL;\n> +                       }\n> +\n> +                       std::string key;\n> +                       readValue(key, std::move(evtKey));\n> +\n> +                       /* Parse value */\n> +                       EventPtr evtValue = nextEvent();\n> +                       if (!evtValue)\n> +                               return -EINVAL;\n> +\n> +                       auto elem = dictionary.emplace(key, std::make_unique<YamlObject>());\n> +                       return parseNextYamlObject(*elem.first->second.get(), std::move(evtValue));\n> +               };\n> +               return parseDictionaryOrList(YamlObject::Dictionary, handler);\n> +       }\n> +       default:\n> +               LOG(YamlParser, Error) << \"Invalid YAML file\";\n> +               return -EINVAL;\n> +       }\n> +}\n> +\n> +#endif /* __DOXYGEN__ */\n> +\n> +/**\n> + * \\class YamlParser\n> + * \\brief A helper class for parsing a YAML file\n> + *\n> + * The YamlParser class provides an easy interface to parse the contents of a\n> + * YAML file into a tree of YamlObject instances.\n> + *\n> + * Example usage:\n> + *\n> + * \\code{.unparsed}\n> + *\n> + * name:\n> + *     \"John\"\n> + * numbers:\n> + *     - 1\n> + *     - 2\n> + *\n> + * \\endcode\n> + *\n> + * The following code illustrates how to parse the above YAML file:\n> + *\n> + * \\code{.cpp}\n> + *\n> + * std::unique_ptr<YamlObject> root = YamlParser::parse(fh);\n> + * if (!root)\n> + *   return;\n> + *\n> + * if (!root->isDictionary())\n> + *   return;\n> + *\n> + * const YamlObject &name = (*root)[\"name\"];\n> + * std::cout << name.get<std::string>(\"\") << std::endl;\n> + *\n> + * const YamlObject &numbers = (*root)[\"numbers\"];\n> + * if (!numbers.isList())\n> + *   return;\n> + *\n> + * for (std::size_t i = 0; i < numbers.size(); i++)\n> + *   std::cout << numbers[i].get<int32_t>(0) << std::endl;\n> + *\n> + * \\endcode\n> + *\n> + * The YamlParser::parse() function takes an open FILE, parses its contents, and\n> + * returns a pointer to a YamlObject corresponding to the root node of the YAML\n> + * document.\n> + */\n> +\n> +/**\n> + * \\fn YamlParser::parse()\n> + * \\brief Parse a YAML file as a YamlObject\n> + * \\param[in] fh The YAML file to parse\n> + *\n> + * The YamlParser::parse() function takes an open FILE, parses its contents, and\n> + * returns a pointer to a YamlObject corresponding to the root node of the YAML\n> + * document. The caller is responsible for closing the file.\n> + *\n> + * \\return Pointer to result YamlObject on success or nullptr otherwise\n> + */\n> +std::unique_ptr<YamlObject> YamlParser::parse(std::FILE *fh)\n> +{\n> +       YamlParserContext context;\n> +\n> +       if (context.init(fh))\n> +               return nullptr;\n> +\n> +       std::unique_ptr<YamlObject> root(new YamlObject());\n> +\n> +       if (context.parseContent(*root)) {\n> +               LOG(YamlParser, Error) << \"Failed to parse YAML content\";\n> +               return nullptr;\n> +       }\n> +\n> +       return root;\n> +}\n> +\n> +} /* namespace libcamera */\n> -- \n> 2.36.0.rc2.479.g8af0fa9b8e-goog\n>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id DE9BFC3256\n\tfor <parsemail@patchwork.libcamera.org>;\n\tMon,  9 May 2022 20:51:51 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C30DE65648;\n\tMon,  9 May 2022 22:51:50 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id CCBE4604A2\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon,  9 May 2022 22:51:49 +0200 (CEST)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 51F7FA1C;\n\tMon,  9 May 2022 22:51:49 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1652129510;\n\tbh=G5TRF66fR3TyxwgYs7BYrHl4+ZCU+2ijt4g8Bj0VEV0=;\n\th=In-Reply-To:References:To:Date:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=DZHtZ2cmf6mFbrbCZUKjIV2zCislZAd7vIoqoj3gVbbzfzvpxOgbgs7I7s86nSbD5\n\tDK2qVRlZl3mpY+rG6O5lb5rE+8P9prOKaisK7Qg3fM3nkaZ41WoXpVsfdqj6Awe9jO\n\tsNWCrpiHcim8/6xs1LHGqIWJvlk90ZoT7B7noWQEa1Tq6MSsR2dtGq9NBEE6vBxKAW\n\t6nJqsG1RIHihH6N/QjBdoBXZqC7dBtj3mBIIsoF3ALYJw4We7l5Xz/7wvT3zN2T/Ee\n\t7iTl5WQmDJa3slN0UtSMyWMKYbIQo31rjv1o+DtSZhtYlEEdhIZqhoz1cWVAR/iat/\n\tM49HYungkIyMA==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1652129509;\n\tbh=G5TRF66fR3TyxwgYs7BYrHl4+ZCU+2ijt4g8Bj0VEV0=;\n\th=In-Reply-To:References:Subject:From:To:Date:From;\n\tb=F/+J+7W9iewjQ1wzmgB1TbR2bDQfjdcVfsUIrvL0Hls7L4ivk/Hrn8gZSbkgGdttp\n\tgh8UBjmzyIKm+NrkP83If8s5+2MyK8Bx242vaZjIUZQ9z6hsHJ8QPFAzUi+0agnhHj\n\tCeD671BIQtHLvlZjWDAvNCvnd+nSOo538c6+FFKc="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"F/+J+7W9\"; dkim-atps=neutral","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20220427140929.429977-1-hanlinchen@chromium.org>","References":"<20220427140929.429977-1-hanlinchen@chromium.org>","To":"Han-Lin Chen <hanlinchen@chromium.org>,\n\tlibcamera-devel@lists.libcamera.org","Date":"Mon, 09 May 2022 21:51:46 +0100","Message-ID":"<165212950688.2416244.17654378141772397615@Monstersaurus>","User-Agent":"alot/0.10","Subject":"Re: [libcamera-devel] [PATCH v6 1/3] libcamera: Introduce\n\tYamlParser as a helper to parse yaml files","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","From":"Kieran Bingham via libcamera-devel\n\t<libcamera-devel@lists.libcamera.org>","Reply-To":"Kieran Bingham <kieran.bingham@ideasonboard.com>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":22919,"web_url":"https://patchwork.libcamera.org/comment/22919/","msgid":"<YnmFFMDJR6AWjN49@pendragon.ideasonboard.com>","date":"2022-05-09T21:18:12","subject":"Re: [libcamera-devel] [PATCH v6 1/3] libcamera: Introduce\n\tYamlParser as a helper to parse yaml files","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Mon, May 09, 2022 at 09:51:46PM +0100, Kieran Bingham via libcamera-devel wrote:\n> Quoting Han-Lin Chen via libcamera-devel (2022-04-27 15:09:27)\n> > Introduce YamlParser as a helper to convert contents of a yaml file to\n> > a tree based structure for easier reading, and to avoid writing parser\n> > with raw yaml tokens. The class is based on libyaml, and only support\n> > reading but not writing a yaml file.\n> > \n> > The interface is inspired by Json::Value class from jsoncpp:\n> > http://jsoncpp.sourceforge.net/class_json_1_1_value.html\n> > \n> > Signed-off-by: Han-Lin Chen <hanlinchen@chromium.org>\n> \n> This causes compile failures on GCC-8. (I haven't yet tested further\n> than that, as it's the first failure that hits).\n> \n> FAILED: src/libcamera/libcamera.so.0.0.0.p/yaml_parser.cpp.o \n> g++-8 -Isrc/libcamera/libcamera.so.0.0.0.p -Isrc/libcamera -I../../../src/libcamera/src/libcamera -Iinclude -I../../../src/libcamera/include -Iinclude/libcamera -Iinclude/libcamera/ipa -Iinclude/libcamera/internal -Isrc/libcamera/proxy -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wnon-virtual-dtor -Wextra -Werror -std=c++17 -g -Wl,--start-group -lstdc++fs -Wl,--end-group -Wshadow -include config.h -fPIC -DLIBCAMERA_BASE_PRIVATE -MD -MQ src/libcamera/libcamera.so.0.0.0.p/yaml_parser.cpp.o -MF src/libcamera/libcamera.so.0.0.0.p/yaml_parser.cpp.o.d -o src/libcamera/libcamera.so.0.0.0.p/yaml_parser.cpp.o -c ../../../src/libcamera/src/libcamera/yaml_parser.cpp\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:360:18: error: ‘function’ in namespace ‘std’ does not name a template type\n>        const std::function<int(EventPtr event)> &parseItem);\n>                   ^~~~~~~~\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:360:13: note: ‘std::function’ is defined in header ‘<functional>’; did you forget to ‘#include <functional>’?\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:16:1:\n> +#include <functional>\n\nThis indeed fixes the failure for me.\n\n>  \n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:360:13:\n>        const std::function<int(EventPtr event)> &parseItem);\n>              ^~~\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:360:26: error: expected ‘,’ or ‘...’ before ‘<’ token\n>        const std::function<int(EventPtr event)> &parseItem);\n>                           ^\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:509:22: error: ‘function’ in namespace ‘std’ does not name a template type\n>            const std::function<int(EventPtr event)> &parseItem)\n>                       ^~~~~~~~\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:509:17: note: ‘std::function’ is defined in header ‘<functional>’; did you forget to ‘#include <functional>’?\n>            const std::function<int(EventPtr event)> &parseItem)\n>                  ^~~\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:509:30: error: expected ‘,’ or ‘...’ before ‘<’ token\n>            const std::function<int(EventPtr event)> &parseItem)\n>                               ^\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp: In member function ‘int libcamera::YamlParserContext::parseDictionaryOrList(libcamera::YamlObject::Type, int)’:\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:527:13: error: ‘parseItem’ was not declared in this scope\n>    int ret = parseItem(std::move(evt));\n>              ^~~~~~~~~\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:527:13: note: suggested alternative: ‘parser_’\n>    int ret = parseItem(std::move(evt));\n>              ^~~~~~~~~\n>              parser_\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp: In member function ‘int libcamera::YamlParserContext::parseNextYamlObject(libcamera::YamlObject&, libcamera::YamlParserContext::EventPtr)’:\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:567:57: error: no matching function for call to ‘libcamera::YamlParserContext::parseDictionaryOrList(libcamera::YamlObject::Type, libcamera::YamlParserContext::parseNextYamlObject(libcamera::YamlObject&, libcamera::YamlParserContext::EventPtr)::<lambda(libcamera::YamlParserContext::EventPtr)>&)’\n>    return parseDictionaryOrList(YamlObject::List, handler);\n>                                                          ^\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:508:5: note: candidate: ‘int libcamera::YamlParserContext::parseDictionaryOrList(libcamera::YamlObject::Type, int)’\n>  int YamlParserContext::parseDictionaryOrList(YamlObject::Type type,\n>      ^~~~~~~~~~~~~~~~~\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:508:5: note:   no known conversion for argument 2 from ‘libcamera::YamlParserContext::parseNextYamlObject(libcamera::YamlObject&, libcamera::YamlParserContext::EventPtr)::<lambda(libcamera::YamlParserContext::EventPtr)>’ to ‘int’\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:594:63: error: no matching function for call to ‘libcamera::YamlParserContext::parseDictionaryOrList(libcamera::YamlObject::Type, libcamera::YamlParserContext::parseNextYamlObject(libcamera::YamlObject&, libcamera::YamlParserContext::EventPtr)::<lambda(libcamera::YamlParserContext::EventPtr)>&)’\n>    return parseDictionaryOrList(YamlObject::Dictionary, handler);\n>                                                                ^\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:508:5: note: candidate: ‘int libcamera::YamlParserContext::parseDictionaryOrList(libcamera::YamlObject::Type, int)’\n>  int YamlParserContext::parseDictionaryOrList(YamlObject::Type type,\n>      ^~~~~~~~~~~~~~~~~\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:508:5: note:   no known conversion for argument 2 from ‘libcamera::YamlParserContext::parseNextYamlObject(libcamera::YamlObject&, libcamera::YamlParserContext::EventPtr)::<lambda(libcamera::YamlParserContext::EventPtr)>’ to ‘int’\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:567:58: error: this statement may fall through [-Werror=implicit-fallthrough=]\n>    return parseDictionaryOrList(YamlObject::List, handler);\n>                                                           ^\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:570:2: note: here\n>   case YAML_MAPPING_START_EVENT: {\n>   ^~~~\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:594:64: error: this statement may fall through [-Werror=implicit-fallthrough=]\n>    return parseDictionaryOrList(YamlObject::Dictionary, handler);\n>                                                                 ^\n> ../../../src/libcamera/src/libcamera/yaml_parser.cpp:597:2: note: here\n>   default:\n>   ^~~~~~~\n> cc1plus: all warnings being treated as errors\n> \n> Are we still supporting gcc-8 ?\n\nYes we are. I'll fix locally in my branch, no need to resubmit.\n\n> > ---\n> >  README.rst                               |   4 +-\n> >  include/libcamera/internal/meson.build   |   1 +\n> >  include/libcamera/internal/yaml_parser.h |  87 +++\n> >  src/libcamera/meson.build                |   3 +\n> >  src/libcamera/yaml_parser.cpp            | 679 +++++++++++++++++++++++\n> >  5 files changed, 772 insertions(+), 2 deletions(-)\n> >  create mode 100644 include/libcamera/internal/yaml_parser.h\n> >  create mode 100644 src/libcamera/yaml_parser.cpp\n> > \n> > diff --git a/README.rst b/README.rst\n> > index aae6b79f..b5a2d448 100644\n> > --- a/README.rst\n> > +++ b/README.rst\n> > @@ -60,7 +60,7 @@ Meson Build system: [required]\n> >              pip3 install --user --upgrade meson\n> >  \n> >  for the libcamera core: [required]\n> > -        python3-yaml python3-ply python3-jinja2\n> > +        libyaml-dev python3-yaml python3-ply python3-jinja2\n> >  \n> >  for IPA module signing: [required]\n> >          libgnutls28-dev openssl\n> > @@ -98,7 +98,7 @@ for tracing with lttng: [optional]\n> >          liblttng-ust-dev python3-jinja2 lttng-tools\n> >  \n> >  for android: [optional]\n> > -        libexif-dev libjpeg-dev libyaml-dev\n> > +        libexif-dev libjpeg-dev\n> >  \n> >  for lc-compliance: [optional]\n> >          libevent-dev\n> > diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build\n> > index c9e055d4..7a780d48 100644\n> > --- a/include/libcamera/internal/meson.build\n> > +++ b/include/libcamera/internal/meson.build\n> > @@ -42,4 +42,5 @@ libcamera_internal_headers = files([\n> >      'v4l2_pixelformat.h',\n> >      'v4l2_subdevice.h',\n> >      'v4l2_videodevice.h',\n> > +    'yaml_parser.h',\n> >  ])\n> > diff --git a/include/libcamera/internal/yaml_parser.h b/include/libcamera/internal/yaml_parser.h\n> > new file mode 100644\n> > index 00000000..3a4f3052\n> > --- /dev/null\n> > +++ b/include/libcamera/internal/yaml_parser.h\n> > @@ -0,0 +1,87 @@\n> > +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> > +/*\n> > + * Copyright (C) 2022, Google Inc.\n> > + *\n> > + * yaml_parser.h - libcamera YAML parsing helper\n> > + */\n> > +\n> > +#pragma once\n> > +\n> > +#include <cstdio>\n> > +#include <map>\n> > +#include <string>\n> > +#include <vector>\n> > +\n> > +#include <libcamera/base/class.h>\n> > +\n> > +#include <libcamera/geometry.h>\n> > +\n> > +namespace libcamera {\n> > +\n> > +class YamlParserContext;\n> > +\n> > +class YamlObject\n> > +{\n> > +public:\n> > +       YamlObject();\n> > +       ~YamlObject();\n> > +\n> > +       bool isValue() const\n> > +       {\n> > +               return type_ == Value;\n> > +       }\n> > +       bool isList() const\n> > +       {\n> > +               return type_ == List;\n> > +       }\n> > +       bool isDictionary() const\n> > +       {\n> > +               return type_ == Dictionary;\n> > +       }\n> > +\n> > +#ifndef __DOXYGEN__\n> > +       template<typename T,\n> > +                typename std::enable_if_t<\n> > +                        std::is_same<bool, T>::value ||\n> > +                        std::is_same<double, T>::value ||\n> > +                        std::is_same<int32_t, T>::value ||\n> > +                        std::is_same<uint32_t, T>::value ||\n> > +                        std::is_same<std::string, T>::value ||\n> > +                        std::is_same<Size, T>::value> * = nullptr>\n> > +#else\n> > +       template<typename T>\n> > +#endif\n> > +       T get(const T &defaultValue, bool *ok = nullptr) const;\n> > +\n> > +       std::size_t size() const;\n> > +       const YamlObject &operator[](std::size_t index) const;\n> > +\n> > +       bool contains(const std::string &key) const;\n> > +       const YamlObject &operator[](const std::string &key) const;\n> > +       std::vector<std::string> memberNames() const;\n> > +\n> > +private:\n> > +       LIBCAMERA_DISABLE_COPY_AND_MOVE(YamlObject)\n> > +\n> > +       friend class YamlParserContext;\n> > +\n> > +       enum Type {\n> > +               Dictionary,\n> > +               List,\n> > +               Value,\n> > +       };\n> > +\n> > +       Type type_;\n> > +\n> > +       std::string value_;\n> > +       std::vector<std::unique_ptr<YamlObject>> list_;\n> > +       std::map<const std::string, std::unique_ptr<YamlObject>> dictionary_;\n> > +};\n> > +\n> > +class YamlParser final\n> > +{\n> > +public:\n> > +       static std::unique_ptr<YamlObject> parse(std::FILE *fh);\n> > +};\n> > +\n> > +} /* namespace libcamera */\n> > diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build\n> > index 26912ca1..f8e18e03 100644\n> > --- a/src/libcamera/meson.build\n> > +++ b/src/libcamera/meson.build\n> > @@ -46,6 +46,7 @@ libcamera_sources = files([\n> >      'v4l2_pixelformat.cpp',\n> >      'v4l2_subdevice.cpp',\n> >      'v4l2_videodevice.cpp',\n> > +    'yaml_parser.cpp',\n> >  ])\n> >  \n> >  libcamera_sources += libcamera_public_headers\n> > @@ -66,6 +67,7 @@ subdir('proxy')\n> >  libdl = cc.find_library('dl')\n> >  libgnutls = cc.find_library('gnutls', required : true)\n> >  libudev = dependency('libudev', required : false)\n> > +libyaml = dependency('yaml-0.1', required : true)\n> >  \n> >  if libgnutls.found()\n> >      config_h.set('HAVE_GNUTLS', 1)\n> > @@ -126,6 +128,7 @@ libcamera_deps = [\n> >      libgnutls,\n> >      liblttng,\n> >      libudev,\n> > +    libyaml,\n> >  ]\n> >  \n> >  # We add '/' to the build_rpath as a 'safe' path to act as a boolean flag.\n> > diff --git a/src/libcamera/yaml_parser.cpp b/src/libcamera/yaml_parser.cpp\n> > new file mode 100644\n> > index 00000000..4a047494\n> > --- /dev/null\n> > +++ b/src/libcamera/yaml_parser.cpp\n> > @@ -0,0 +1,679 @@\n> > +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> > +/*\n> > + * Copyright (C) 2022, Google Inc.\n> > + *\n> > + * yaml_parser.cpp - libcamera YAML parsing helper\n> > + */\n> > +\n> > +#include \"libcamera/internal/yaml_parser.h\"\n> > +\n> > +#include <cerrno>\n> > +#include <cstdlib>\n> > +\n> > +#include <libcamera/base/log.h>\n> > +\n> > +#include <yaml.h>\n> > +\n> > +/**\n> > + * \\file libcamera/internal/yaml_parser.h\n> > + * \\brief A YAML parser helper\n> > + */\n> > +\n> > +namespace libcamera {\n> > +\n> > +LOG_DEFINE_CATEGORY(YamlParser)\n> > +\n> > +namespace {\n> > +\n> > +/* Empty static YamlObject as a safe result for invalid operations */\n> > +static const YamlObject empty;\n> > +\n> > +void setOk(bool *ok, bool result)\n> > +{\n> > +       if (ok)\n> > +               *ok = result;\n> > +}\n> > +\n> > +} /* namespace */\n> > +\n> > +/**\n> > + * \\class YamlObject\n> > + * \\brief A class representing the tree structure of the YAML content\n> > + *\n> > + * The YamlObject class represents the tree structure of YAML content. A\n> > + * YamlObject can be a dictionary or list of YamlObjects or a value if a tree\n> > + * leaf.\n> > + */\n> > +\n> > +YamlObject::YamlObject()\n> > +       : type_(Value)\n> > +{\n> > +}\n> > +\n> > +YamlObject::~YamlObject() = default;\n> > +\n> > +/**\n> > + * \\fn YamlObject::isValue()\n> > + * \\brief Return whether the YamlObject is a value\n> > + *\n> > + * \\return True if the YamlObject is a value, false otherwise\n> > + */\n> > +\n> > +/**\n> > + * \\fn YamlObject::isList()\n> > + * \\brief Return whether the YamlObject is a list\n> > + *\n> > + * \\return True if the YamlObject is a list, false otherwise\n> > + */\n> > +\n> > +/**\n> > + * \\fn YamlObject::isDictionary()\n> > + * \\brief Return whether the YamlObject is a dictionary\n> > + *\n> > + * \\return True if the YamlObject is a dictionary, false otherwise\n> > + */\n> > +\n> > +/**\n> > + * \\fn template<typename T> YamlObject::get<T>(\n> > + *     const T &defaultValue, bool *ok) const\n> > + * \\brief Parse the YamlObject as a \\a T value\n> > + * \\param[in] defaultValue The default value when failing to parse\n> > + * \\param[out] ok The result of whether the parse succeeded\n> > + *\n> > + * This function parses the value of the YamlObject as a \\a T object, and\n> > + * returns the value. If parsing fails (usually because the YamlObject doesn't\n> > + * store a \\a T value), the \\a defaultValue is returned, and \\a ok is set to\n> > + * false. Otherwise, the YamlObject value is returned, and \\a ok is set to true.\n> > + *\n> > + * The \\a ok pointer is optional and can be a nullptr if the caller doesn't\n> > + * need to know if parsing succeeded.\n> > + *\n> > + * \\return Value as a bool type\n> > + */\n> > +\n> > +#ifndef __DOXYGEN__\n> > +\n> > +template<>\n> > +bool YamlObject::get(const bool &defaultValue, bool *ok) const\n> > +{\n> > +       setOk(ok, false);\n> > +\n> > +       if (type_ != Value)\n> > +               return defaultValue;\n> > +\n> > +       if (value_ == \"true\") {\n> > +               setOk(ok, true);\n> > +               return true;\n> > +       } else if (value_ == \"false\") {\n> > +               setOk(ok, true);\n> > +               return false;\n> > +       }\n> > +\n> > +       return defaultValue;\n> > +}\n> > +\n> > +template<>\n> > +int32_t YamlObject::get(const int32_t &defaultValue, bool *ok) const\n> > +{\n> > +       setOk(ok, false);\n> > +\n> > +       if (type_ != Value)\n> > +               return defaultValue;\n> > +\n> > +       if (value_ == \"\")\n> > +               return defaultValue;\n> > +\n> > +       char *end;\n> > +\n> > +       errno = 0;\n> > +       int32_t value = std::strtol(value_.c_str(), &end, 10);\n> > +\n> > +       if ('\\0' != *end || errno == ERANGE)\n> > +               return defaultValue;\n> > +\n> > +       setOk(ok, true);\n> > +       return value;\n> > +}\n> > +\n> > +template<>\n> > +uint32_t YamlObject::get(const uint32_t &defaultValue, bool *ok) const\n> > +{\n> > +       setOk(ok, false);\n> > +\n> > +       if (type_ != Value)\n> > +               return defaultValue;\n> > +\n> > +       if (value_ == \"\")\n> > +               return defaultValue;\n> > +\n> > +       /*\n> > +        * libyaml parses all scalar values as strings. When a string has\n> > +        * leading spaces before a minus sign, for example \"  -10\", strtoul\n> > +        * skips leading spaces, accepts the leading minus sign, and the\n> > +        * calculated digits are negated as if by unary minus. Rule it out in\n> > +        * case the user gets a large number when the value is negative.\n> > +        */\n> > +       std::size_t found = value_.find_first_not_of(\" \\t\");\n> > +       if (found != std::string::npos && value_[found] == '-')\n> > +               return defaultValue;\n> > +\n> > +       char *end;\n> > +\n> > +       errno = 0;\n> > +       uint32_t value = std::strtoul(value_.c_str(), &end, 10);\n> > +\n> > +       if ('\\0' != *end || errno == ERANGE)\n> > +               return defaultValue;\n> > +\n> > +       setOk(ok, true);\n> > +       return value;\n> > +}\n> > +\n> > +template<>\n> > +double YamlObject::get(const double &defaultValue, bool *ok) const\n> > +{\n> > +       setOk(ok, false);\n> > +\n> > +       if (type_ != Value)\n> > +               return defaultValue;\n> > +\n> > +       if (value_ == \"\")\n> > +               return defaultValue;\n> > +\n> > +       char *end;\n> > +\n> > +       errno = 0;\n> > +       double value = std::strtod(value_.c_str(), &end);\n> > +\n> > +       if ('\\0' != *end || errno == ERANGE)\n> > +               return defaultValue;\n> > +\n> > +       setOk(ok, true);\n> > +       return value;\n> > +}\n> > +\n> > +template<>\n> > +std::string YamlObject::get(const std::string &defaultValue, bool *ok) const\n> > +{\n> > +       setOk(ok, false);\n> > +\n> > +       if (type_ != Value)\n> > +               return defaultValue;\n> > +\n> > +       setOk(ok, true);\n> > +       return value_;\n> > +}\n> > +\n> > +template<>\n> > +Size YamlObject::get(const Size &defaultValue, bool *ok) const\n> > +{\n> > +       setOk(ok, false);\n> > +\n> > +       if (type_ != List)\n> > +               return defaultValue;\n> > +\n> > +       if (list_.size() != 2)\n> > +               return defaultValue;\n> > +\n> > +       /*\n> > +        * Add a local variable to validate each dimension in case\n> > +        * that ok == nullptr.\n> > +        */\n> > +       bool valid;\n> > +       uint32_t width = list_[0]->get<uint32_t>(0, &valid);\n> > +       if (!valid)\n> > +               return defaultValue;\n> > +\n> > +       uint32_t height = list_[1]->get<uint32_t>(0, &valid);\n> > +       if (!valid)\n> > +               return defaultValue;\n> > +\n> > +       setOk(ok, true);\n> > +       return Size(width, height);\n> > +}\n> > +\n> > +#endif /* __DOXYGEN__ */\n> > +\n> > +/**\n> > + * \\fn YamlObject::size()\n> > + * \\brief Retrieve the number of elements in a list YamlObject\n> > + *\n> > + * This function retrieves the size of the YamlObject, defined as the number of\n> > + * child elements it contains. Only YamlObject instances of List type have a\n> > + * size, calling this function on other types of instances is invalid and\n> > + * results in undefined behaviour.\n> > + *\n> > + * \\return The size of the YamlObject\n> > + */\n> > +std::size_t YamlObject::size() const\n> > +{\n> > +       if (type_ != List)\n> > +               return 0;\n> > +\n> > +       return list_.size();\n> > +}\n> > +\n> > +/**\n> > + * \\fn YamlObject::operator[](std::size_t index) const\n> > + * \\brief Retrieve the element from list YamlObject by index\n> > + *\n> > + * This function retrieves an element of the YamlObject. Only YamlObject\n> > + * instances of List type associate elements with index, calling this function\n> > + * on other types of instances is invalid and results in undefined behaviour.\n> > + *\n> > + * \\return The YamlObject as an element of the list\n> > + */\n> > +const YamlObject &YamlObject::operator[](std::size_t index) const\n> > +{\n> > +       if (type_ != List || index >= size())\n> > +               return empty;\n> > +\n> > +       return *list_[index];\n> > +}\n> > +\n> > +/**\n> > + * \\fn YamlObject::contains()\n> > + * \\brief Check if an element of a dictionary exists\n> > + *\n> > + * This function check if the YamlObject contains an element. Only YamlObject\n> > + * instances of Dictionary type associate elements with names, calling this\n> > + * function on other types of instances is invalid and results in undefined\n> > + * behaviour.\n> > + *\n> > + * \\return True if an element exists, false otherwise\n> > + */\n> > +bool YamlObject::contains(const std::string &key) const\n> > +{\n> > +       if (dictionary_.find(key) == dictionary_.end())\n> > +               return false;\n> > +\n> > +       return true;\n> > +}\n> > +\n> > +/**\n> > + * \\fn YamlObject::memberNames()\n> > + * \\brief Retrieve all member names of the dictionary\n> > + *\n> > + * This function retrieve member names of a YamlObject. Only YamlObject\n> > + * instances of Dictionary type associate elements with names, calling this\n> > + * function on other types of instances is invalid and results in undefined\n> > + * behaviour.\n> > + *\n> > + * \\todo Replace this function with an iterator-based API\n> > + *\n> > + * \\return A vector of string as the member names\n> > + */\n> > +std::vector<std::string> YamlObject::memberNames() const\n> > +{\n> > +       std::vector<std::string> memberNames;\n> > +       for (auto &[key, _] : dictionary_)\n> > +               memberNames.push_back(key);\n> > +\n> > +       return memberNames;\n> > +}\n> > +\n> > +/**\n> > + * \\fn YamlObject::operator[](const std::string &key) const\n> > + * \\brief Retrieve a member by name from the dictionary\n> > + *\n> > + * This function retrieve a member of a YamlObject by name. Only YamlObject\n> > + * instances of Dictionary type associate elements with names, calling this\n> > + * function on other types of instances is invalid and results in undefined\n> > + * behaviour.\n> > + *\n> > + * \\return The YamlObject corresponding to the \\a key member\n> > + */\n> > +const YamlObject &YamlObject::operator[](const std::string &key) const\n> > +{\n> > +       if (type_ != Dictionary || !contains(key))\n> > +               return empty;\n> > +\n> > +       auto iter = dictionary_.find(key);\n> > +       return *iter->second;\n> > +}\n> > +\n> > +#ifndef __DOXYGEN__\n> > +\n> > +class YamlParserContext\n> > +{\n> > +public:\n> > +       YamlParserContext();\n> > +       ~YamlParserContext();\n> > +\n> > +       int init(std::FILE *fh);\n> > +       int parseContent(YamlObject &yamlObject);\n> > +\n> > +private:\n> > +       struct EventDeleter {\n> > +               void operator()(yaml_event_t *event) const\n> > +               {\n> > +                       yaml_event_delete(event);\n> > +                       delete event;\n> > +               }\n> > +       };\n> > +       using EventPtr = std::unique_ptr<yaml_event_t, EventDeleter>;\n> > +\n> > +       EventPtr nextEvent();\n> > +\n> > +       void readValue(std::string &value, EventPtr event);\n> > +       int parseDictionaryOrList(YamlObject::Type type,\n> > +                                 const std::function<int(EventPtr event)> &parseItem);\n> > +       int parseNextYamlObject(YamlObject &yamlObject, EventPtr event);\n> > +\n> > +       bool parserValid_;\n> > +       yaml_parser_t parser_;\n> > +};\n> > +\n> > +/**\n> > + * \\class YamlParserContext\n> > + * \\brief Class for YamlParser parsing and context data\n> > + *\n> > + * The YamlParserContext class stores the internal yaml_parser_t and provides\n> > + * helper functions to do event-based parsing for YAML files.\n> > + */\n> > +YamlParserContext::YamlParserContext()\n> > +       : parserValid_(false)\n> > +{\n> > +}\n> > +\n> > +/**\n> > + * \\class YamlParserContext\n> > + * \\brief Destructor of YamlParserContext\n> > + */\n> > +YamlParserContext::~YamlParserContext()\n> > +{\n> > +       if (parserValid_) {\n> > +               yaml_parser_delete(&parser_);\n> > +               parserValid_ = false;\n> > +       }\n> > +}\n> > +\n> > +/**\n> > + * \\fn YamlParserContext::init()\n> > + * \\brief Initialize a parser with an opened file for parsing\n> > + * \\param[in] fh The YAML file to parse\n> > + *\n> > + * Prior to parsing the YAML content, the YamlParserContext must be initialized\n> > + * with an opened FILE to create an internal parser. The FILE needs to stay\n> > + * valid during the process.\n> > + *\n> > + * \\return 0 on success or a negative error code otherwise\n> > + * \\retval -EINVAL The parser has failed to initialize\n> > + */\n> > +int YamlParserContext::init(std::FILE *fh)\n> > +{\n> > +       /* yaml_parser_initialize returns 1 when it succeededs */\n> > +       if (!yaml_parser_initialize(&parser_)) {\n> > +               LOG(YamlParser, Error) << \"Failed to initialize YAML parser\";\n> > +               return -EINVAL;\n> > +       }\n> > +       parserValid_ = true;\n> > +       yaml_parser_set_input_file(&parser_, fh);\n> > +\n> > +       return 0;\n> > +}\n> > +\n> > +/**\n> > + * \\fn YamlParserContext::nextEvent()\n> > + * \\brief Get the next event\n> > + *\n> > + * Get the next event in the current YAML event stream, and return nullptr when\n> > + * there is no more event.\n> > + *\n> > + * \\return The next event on success or nullptr otherwise\n> > + */\n> > +YamlParserContext::EventPtr YamlParserContext::nextEvent()\n> > +{\n> > +       EventPtr event(new yaml_event_t);\n> > +\n> > +       /* yaml_parser_parse returns 1 when it succeeds */\n> > +       if (!yaml_parser_parse(&parser_, event.get()))\n> > +               return nullptr;\n> > +\n> > +       return event;\n> > +}\n> > +\n> > +/**\n> > + * \\fn YamlParserContext::parseContent()\n> > + * \\brief Parse the content of a YAML document\n> > + * \\param[in] yamlObject The result of YamlObject\n> > + *\n> > + * Check YAML start and end events of a YAML document, and parse the root object\n> > + * of the YAML document into a YamlObject.\n> > + *\n> > + * \\return 0 on success or a negative error code otherwise\n> > + * \\retval -EINVAL The parser has failed to validate end of a YAML file\n> > + */\n> > +int YamlParserContext::parseContent(YamlObject &yamlObject)\n> > +{\n> > +       /* Check start of the YAML file. */\n> > +       EventPtr event = nextEvent();\n> > +       if (!event || event->type != YAML_STREAM_START_EVENT)\n> > +               return -EINVAL;\n> > +\n> > +       event = nextEvent();\n> > +       if (!event || event->type != YAML_DOCUMENT_START_EVENT)\n> > +               return -EINVAL;\n> > +\n> > +       /* Parse the root object. */\n> > +       event = nextEvent();\n> > +       if (parseNextYamlObject(yamlObject, std::move(event)))\n> > +               return -EINVAL;\n> > +\n> > +       /* Check end of the YAML file. */\n> > +       event = nextEvent();\n> > +       if (!event || event->type != YAML_DOCUMENT_END_EVENT)\n> > +               return -EINVAL;\n> > +\n> > +       event = nextEvent();\n> > +       if (!event || event->type != YAML_STREAM_END_EVENT)\n> > +               return -EINVAL;\n> > +\n> > +       return 0;\n> > +}\n> > +\n> > +/**\n> > + * \\fn YamlParserContext::readValue()\n> > + * \\brief Parse event scalar and fill its content into a string\n> > + * \\param[in] value The string reference to fill value\n> > + *\n> > + * A helper function to parse a scalar event as string. The caller needs to\n> > + * guarantee the event is of scaler type.\n> > + */\n> > +void YamlParserContext::readValue(std::string &value, EventPtr event)\n> > +{\n> > +       value.assign(reinterpret_cast<char *>(event->data.scalar.value),\n> > +                    event->data.scalar.length);\n> > +}\n> > +\n> > +/**\n> > + * \\fn YamlParserContext::parseDictionaryOrList()\n> > + * \\brief A helper function to abstract the common part of parsing dictionary or list\n> > + *\n> > + * \\param[in] isDictionary True for parsing a dictionary, and false for a list\n> > + * \\param[in] parseItem The callback to handle an item\n> > + *\n> > + * A helper function to abstract parsing an item from a dictionary or a list.\n> > + * The differences of them in a YAML event stream are:\n> > + *\n> > + * 1. The start and end event types are different\n> > + * 2. There is a leading scalar string as key in the items of a dictionary\n> > + *\n> > + * The caller should handle the leading key string in its callback parseItem\n> > + * when it's a dictionary.\n> > + *\n> > + * \\return 0 on success or a negative error code otherwise\n> > + * \\retval -EINVAL The parser is failed to initialize\n> > + */\n> > +int YamlParserContext::parseDictionaryOrList(YamlObject::Type type,\n> > +                                            const std::function<int(EventPtr event)> &parseItem)\n> > +{\n> > +       yaml_event_type_t endEventType = YAML_SEQUENCE_END_EVENT;\n> > +       if (type == YamlObject::Dictionary)\n> > +               endEventType = YAML_MAPPING_END_EVENT;\n> > +\n> > +       /*\n> > +        * Add a safety counter to make sure we don't loop indefinitely in case\n> > +        * the YAML file is malformed.\n> > +        */\n> > +       for (unsigned int sentinel = 1000; sentinel; sentinel--) {\n> > +               auto evt = nextEvent();\n> > +               if (!evt)\n> > +                       return -EINVAL;\n> > +\n> > +               if (evt->type == endEventType)\n> > +                       return 0;\n> > +\n> > +               int ret = parseItem(std::move(evt));\n> > +               if (ret)\n> > +                       return ret;\n> > +       }\n> > +\n> > +       LOG(YamlParser, Error) << \"The YAML file contains a List or Dictionary\"\n> > +                                 \" whose size exceeding the parser's limit(1000)\";\n> > +\n> > +       return -EINVAL;\n> > +}\n> > +\n> > +/**\n> > + * \\fn YamlParserContext::parseNextYamlObject()\n> > + * \\brief Parse next YAML event and read it as a YamlObject\n> > + * \\param[in] yamlObject The result of YamlObject\n> > + * \\param[in] event The leading event of the object\n> > + *\n> > + * Parse next YAML object separately as a value, list or dictionary.\n> > + *\n> > + * \\return 0 on success or a negative error code otherwise\n> > + * \\retval -EINVAL Fail to parse the YAML file.\n> > + */\n> > +int YamlParserContext::parseNextYamlObject(YamlObject &yamlObject, EventPtr event)\n> > +{\n> > +       if (!event)\n> > +               return -EINVAL;\n> > +\n> > +       switch (event->type) {\n> > +       case YAML_SCALAR_EVENT:\n> > +               yamlObject.type_ = YamlObject::Value;\n> > +               readValue(yamlObject.value_, std::move(event));\n> > +               return 0;\n> > +\n> > +       case YAML_SEQUENCE_START_EVENT: {\n> > +               yamlObject.type_ = YamlObject::List;\n> > +               auto &list = yamlObject.list_;\n> > +               auto handler = [this, &list](EventPtr evt) {\n> > +                       list.emplace_back(new YamlObject());\n> > +                       return parseNextYamlObject(*list.back(), std::move(evt));\n> > +               };\n> > +               return parseDictionaryOrList(YamlObject::List, handler);\n> > +       }\n> > +\n> > +       case YAML_MAPPING_START_EVENT: {\n> > +               yamlObject.type_ = YamlObject::Dictionary;\n> > +               auto &dictionary = yamlObject.dictionary_;\n> > +               auto handler = [this, &dictionary](EventPtr evtKey) {\n> > +                       /* Parse key */\n> > +                       if (evtKey->type != YAML_SCALAR_EVENT) {\n> > +                               LOG(YamlParser, Error) << \"Expect key at line: \"\n> > +                                                      << evtKey->start_mark.line\n> > +                                                      << \" column: \"\n> > +                                                      << evtKey->start_mark.column;\n> > +                               return -EINVAL;\n> > +                       }\n> > +\n> > +                       std::string key;\n> > +                       readValue(key, std::move(evtKey));\n> > +\n> > +                       /* Parse value */\n> > +                       EventPtr evtValue = nextEvent();\n> > +                       if (!evtValue)\n> > +                               return -EINVAL;\n> > +\n> > +                       auto elem = dictionary.emplace(key, std::make_unique<YamlObject>());\n> > +                       return parseNextYamlObject(*elem.first->second.get(), std::move(evtValue));\n> > +               };\n> > +               return parseDictionaryOrList(YamlObject::Dictionary, handler);\n> > +       }\n> > +       default:\n> > +               LOG(YamlParser, Error) << \"Invalid YAML file\";\n> > +               return -EINVAL;\n> > +       }\n> > +}\n> > +\n> > +#endif /* __DOXYGEN__ */\n> > +\n> > +/**\n> > + * \\class YamlParser\n> > + * \\brief A helper class for parsing a YAML file\n> > + *\n> > + * The YamlParser class provides an easy interface to parse the contents of a\n> > + * YAML file into a tree of YamlObject instances.\n> > + *\n> > + * Example usage:\n> > + *\n> > + * \\code{.unparsed}\n> > + *\n> > + * name:\n> > + *     \"John\"\n> > + * numbers:\n> > + *     - 1\n> > + *     - 2\n> > + *\n> > + * \\endcode\n> > + *\n> > + * The following code illustrates how to parse the above YAML file:\n> > + *\n> > + * \\code{.cpp}\n> > + *\n> > + * std::unique_ptr<YamlObject> root = YamlParser::parse(fh);\n> > + * if (!root)\n> > + *   return;\n> > + *\n> > + * if (!root->isDictionary())\n> > + *   return;\n> > + *\n> > + * const YamlObject &name = (*root)[\"name\"];\n> > + * std::cout << name.get<std::string>(\"\") << std::endl;\n> > + *\n> > + * const YamlObject &numbers = (*root)[\"numbers\"];\n> > + * if (!numbers.isList())\n> > + *   return;\n> > + *\n> > + * for (std::size_t i = 0; i < numbers.size(); i++)\n> > + *   std::cout << numbers[i].get<int32_t>(0) << std::endl;\n> > + *\n> > + * \\endcode\n> > + *\n> > + * The YamlParser::parse() function takes an open FILE, parses its contents, and\n> > + * returns a pointer to a YamlObject corresponding to the root node of the YAML\n> > + * document.\n> > + */\n> > +\n> > +/**\n> > + * \\fn YamlParser::parse()\n> > + * \\brief Parse a YAML file as a YamlObject\n> > + * \\param[in] fh The YAML file to parse\n> > + *\n> > + * The YamlParser::parse() function takes an open FILE, parses its contents, and\n> > + * returns a pointer to a YamlObject corresponding to the root node of the YAML\n> > + * document. The caller is responsible for closing the file.\n> > + *\n> > + * \\return Pointer to result YamlObject on success or nullptr otherwise\n> > + */\n> > +std::unique_ptr<YamlObject> YamlParser::parse(std::FILE *fh)\n> > +{\n> > +       YamlParserContext context;\n> > +\n> > +       if (context.init(fh))\n> > +               return nullptr;\n> > +\n> > +       std::unique_ptr<YamlObject> root(new YamlObject());\n> > +\n> > +       if (context.parseContent(*root)) {\n> > +               LOG(YamlParser, Error) << \"Failed to parse YAML content\";\n> > +               return nullptr;\n> > +       }\n> > +\n> > +       return root;\n> > +}\n> > +\n> > +} /* namespace libcamera */","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B32FBC3256\n\tfor <parsemail@patchwork.libcamera.org>;\n\tMon,  9 May 2022 21:18:23 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id DE54165648;\n\tMon,  9 May 2022 23:18:22 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 0AC17604A2\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon,  9 May 2022 23:18:19 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 50A35A1C;\n\tMon,  9 May 2022 23:18:18 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1652131102;\n\tbh=Tb+MAoVMgSLj2CqFdFUc/3wCWmIuABG999nS3+n77OQ=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=DpKge1w2mTorPSppiWX5Ot08kNdMghFGeaW/OKNkipP0K0BXyl8j9ppi9vbkgmn94\n\tVqTKGjtkwbVQz3FgiqgI8iEI+va6kbV7gFPDsHZC7xSjVmabT/QY6KVd0HqcO/I1rF\n\tBTQImijr+K+kyId+tCogfehfUYMccyah1gcNkgTJDrbKtoVwXItVvR6XQIPmP1o0PY\n\tdSOQGRpNwhBRwDyU0ZPNNn59nSx/AJ5GA1tEzw8sXcIEkUrLXZc6P7SQxKde7h9u+n\n\thjG4thGX8TiOKPBGfh2Ai9PcewY366UQvXwkbh6lAncJTjcLM36LM257Cjh4ZxsFij\n\tug6VfSdEqjP9g==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1652131098;\n\tbh=Tb+MAoVMgSLj2CqFdFUc/3wCWmIuABG999nS3+n77OQ=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=XztEi0+vNyKYTVeGxU1K80OMKAG9YSDyePH5EHebgMovyWYCEaOrb0AvUzyQrRLRt\n\tXsURJpN98xZnqrEjQ9hDFMjwqq5G/yiSBPC1dMTZher2zSxsgL1d8Y5eiDtZjJu7q9\n\t7My4vP7LS+bH1lWPkz8Ig0B5GblY10OwwIHIWZ6Y="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"XztEi0+v\"; dkim-atps=neutral","Date":"Tue, 10 May 2022 00:18:12 +0300","To":"Kieran Bingham <kieran.bingham@ideasonboard.com>","Message-ID":"<YnmFFMDJR6AWjN49@pendragon.ideasonboard.com>","References":"<20220427140929.429977-1-hanlinchen@chromium.org>\n\t<165212950688.2416244.17654378141772397615@Monstersaurus>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<165212950688.2416244.17654378141772397615@Monstersaurus>","Subject":"Re: [libcamera-devel] [PATCH v6 1/3] libcamera: Introduce\n\tYamlParser as a helper to parse yaml files","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","From":"Laurent Pinchart via libcamera-devel\n\t<libcamera-devel@lists.libcamera.org>","Reply-To":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}}]