[{"id":31289,"web_url":"https://patchwork.libcamera.org/comment/31289/","msgid":"<172692257740.2374548.8278981520464589401@ping.linuxembedded.co.uk>","date":"2024-09-21T12:42:57","subject":"Re: [PATCH v3 3/3] libcamera: yaml-parser: Differentiate between\n\tempty and empty string","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Stefan Klug (2024-09-20 14:28:10)\n> When accessing a nonexistent key on a dict the YamlObject returns an\n> empty element. This element can happily be cast to a string which is\n> unexpected. For example the following statement:\n> \n> yamlDict[\"nonexistent\"].get(\"default\")\n> \n> is expected to return \"default\" but actually returns \"\". Fix this by\n> introducing a empty type to distinguish between an empty YamlObject and\n\ns/introducing a empty/introducing an empty/\n\n> a YamlObject of type value containing an empty string. For completeness\n> add an isEmpty() function and an explicit cast to bool to be able to\n> test for that type.\n> \n> Extend the tests accordingly.\n> \n> Signed-off-by: Stefan Klug \n> \n> ---\n> Changes in v3:\n> - Move tests for the existing functionality into own patches\n> - Fixed some typos\n> \n> Changes in v2:\n> - Added explicit to bool conversion operator\n> - Fixed typos from review\n> - I didn't add the rb tags, because the v2 adds quite some logic\n> (especially tests) and this is a central piece of code\n> ---\n> include/libcamera/internal/yaml_parser.h | 9 ++++++++\n> src/libcamera/yaml_parser.cpp | 27 ++++++++++++++++++------\n> test/yaml-parser.cpp | 18 ++++++++++++++++\n> 3 files changed, 48 insertions(+), 6 deletions(-)\n> \n> diff --git a/include/libcamera/internal/yaml_parser.h b/include/libcamera/internal/yaml_parser.h\n> index 16708e488d88..6211ff4ae563 100644\n> --- a/include/libcamera/internal/yaml_parser.h\n> +++ b/include/libcamera/internal/yaml_parser.h\n> @@ -159,6 +159,14 @@ public:\n> {\n> return type_ == Type::Dictionary;\n> }\n> + bool isEmpty() const\n> + {\n> + return type_ == Type::Empty;\n> + }\n> + explicit operator bool() const\n> + {\n> + return type_ != Type::Empty;\n\nI presume this could also be :\n\t\treturn !isEmpty();\n\nBut there's no harm in open coding here too.\n\n> + }\n> \n> std::size_t size() const;\n> \n> @@ -212,6 +220,7 @@ private:\n> Dictionary,\n> List,\n> Value,\n> + Empty,\n> };\n> \n> template\n> diff --git a/src/libcamera/yaml_parser.cpp b/src/libcamera/yaml_parser.cpp\n> index 8b6a403898be..4784f2dc3d62 100644\n> --- a/src/libcamera/yaml_parser.cpp\n> +++ b/src/libcamera/yaml_parser.cpp\n> @@ -38,12 +38,12 @@ static const YamlObject empty;\n> * \\brief A class representing the tree structure of the YAML content\n> *\n> * The YamlObject class represents the tree structure of YAML content. A\n> - * YamlObject can be a dictionary or list of YamlObjects or a value if a tree\n> - * leaf.\n> + * YamlObject can be empty, a dictionary or list of YamlObjects, or a value if a\n> + * tree leaf.\n> */\n> \n> YamlObject::YamlObject()\n> - : type_(Type::Value)\n> + : type_(Type::Empty)\n> {\n> }\n> \n> @@ -70,6 +70,20 @@ YamlObject::~YamlObject() = default;\n> * \\return True if the YamlObject is a dictionary, false otherwise\n> */\n> \n> +/**\n> + * \\fn YamlObject::isEmpty()\n> + * \\brief Return whether the YamlObject is an empty\n> + *\n\nI'm stumped on 'is an empty' here ... but I actually think it could be\ncorrect as it is an 'empty' object. Not that the object is empty ... ;-)\nAnd that sounds like it means all the same but somehow there seems to be\na distinction in my head.\n\nAnyway, I think it's fine.\n\n> + * \\return True if the YamlObject is empty, false otherwise\n> + */\n> +\n> +/**\n> + * \\fn YamlObject::operator bool()\n> + * \\brief Return whether the YamlObject is a non-empty\n> + *\n> + * \\return False if the YamlObject is empty, true otherwise\n> + */\n> +\n> /**\n> * \\fn YamlObject::size()\n> * \\brief Retrieve the number of elements in a dictionary or list YamlObject\n> @@ -443,7 +457,8 @@ template std::optional> YamlObject::getList() const;\n> *\n> * This function retrieves an element of the YamlObject. Only YamlObject\n> * instances of List type associate elements with index, calling this function\n> - * on other types of instances is invalid and results in undefined behaviour.\n> + * on other types of instances or with an invalid index results in an empty\n> + * object.\n> *\n> * \\return The YamlObject as an element of the list\n> */\n> @@ -480,8 +495,8 @@ bool YamlObject::contains(const std::string &key) const\n> *\n> * This function retrieve a member of a YamlObject by name. Only YamlObject\n> * instances of Dictionary type associate elements with names, calling this\n> - * function on other types of instances is invalid and results in undefined\n> - * behaviour.\n> + * function on other types of instances or with a nonexistent key results in an\n> + * empty object.\n> *\n> * \\return The YamlObject corresponding to the \\a key member\n> */\n> diff --git a/test/yaml-parser.cpp b/test/yaml-parser.cpp\n> index 4cc77e26ae39..1b22c87b72f1 100644\n> --- a/test/yaml-parser.cpp\n> +++ b/test/yaml-parser.cpp\n> @@ -542,6 +542,24 @@ protected:\n> return TestFail;\n> }\n> \n> + /* Test nonexistent object has value type empty. */\n> + if (!dictObj[\"nonexistent\"].isEmpty()) {\n> + cerr << \"Accessing nonexistent object returns non-empty object\" << std::endl;\n> + return TestFail;\n> + }\n> +\n> + /* Test explicit cast to bool on an empty object returns true. */\n> + if (!!dictObj[\"empty\"] != true) {\n> + cerr << \"Casting empty entry to bool returns false\" << std::endl;\n> + return TestFail;\n> + }\n> +\n> + /* Test explicit cast to bool on nonexistent object returns false. */\n> + if (!!dictObj[\"nonexistent\"] != false) {\n> + cerr << \"Casting nonexistent dict entry to bool returns true\" << std::endl;\n> + return TestFail;\n> + }\n> +\n> /* Make sure utils::map_keys() works on the adapter. */\n> (void)utils::map_keys(dictObj.asDict());\n> \n\nAnd again, now that the tests pass here - adding the following hunk to\n'undo' the addition proposed in 2/3:\n\ndiff --git a/test/meson.build b/test/meson.build\nindex dcd169a8793e..5ed052ed62c8 100644\n--- a/test/meson.build\n+++ b/test/meson.build\n@@ -73,7 +73,7 @@ internal_tests = [\n {'name': 'timer-thread', 'sources': ['timer-thread.cpp']},\n {'name': 'unique-fd', 'sources': ['unique-fd.cpp']},\n {'name': 'utils', 'sources': ['utils.cpp']},\n- {'name': 'yaml-parser', 'sources': ['yaml-parser.cpp'], 'should_fail': true},\n+ {'name': 'yaml-parser', 'sources': ['yaml-parser.cpp']},\n ]\n\n internal_non_parallel_tests = [\n\nAnd then:\n\n\nReviewed-by: Kieran Bingham \n\n> -- \n> 2.43.0\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 7CC8EC324C\n\tfor ;\n\tSat, 21 Sep 2024 12:43:03 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 53D86634FC;\n\tSat, 21 Sep 2024 14:43:02 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 9794C618E4\n\tfor ;\n\tSat, 21 Sep 2024 14:43:00 +0200 (CEST)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C084E890;\n\tSat, 21 Sep 2024 14:41:35 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"LWWJ1sY3\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1726922495;\n\tbh=ToBcGQ4yz7fYaYsHHFY+KAtpYf1RuI83VXrVjXkiS9Y=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=LWWJ1sY3yBHgwRZv/eplv/dYbzMw7N3bN1sH+gGYNomfD5WY6uPD62jVCXK+ETsrF\n\tCKJeOSu5YYE1Q+8ne+wll0kNA+T6baP9TKOiOWpBZD9Fcbt8FJDSLpBcI97egUaj9Y\n\tSRWypCQNcpHO2Cb6S7GfeVR8glkvkBeiOVJhWCwA=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20240920132823.88433-4-stefan.klug@ideasonboard.com>","References":"<20240920132823.88433-1-stefan.klug@ideasonboard.com>\n\t<20240920132823.88433-4-stefan.klug@ideasonboard.com>","Subject":"Re: [PATCH v3 3/3] libcamera: yaml-parser: Differentiate between\n\tempty and empty string","From":"Kieran Bingham ","Cc":"Stefan Klug ","To":"Stefan Klug ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Sat, 21 Sep 2024 13:42:57 +0100","Message-ID":"<172692257740.2374548.8278981520464589401@ping.linuxembedded.co.uk>","User-Agent":"alot/0.10","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":31303,"web_url":"https://patchwork.libcamera.org/comment/31303/","msgid":"","date":"2024-09-23T10:09:26","subject":"Re: [PATCH v3 3/3] libcamera: yaml-parser: Differentiate between\n\tempty and empty string","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"content":"On Fri, Sep 20, 2024 at 03:28:10PM +0200, Stefan Klug wrote:\n> When accessing a nonexistent key on a dict the YamlObject returns an\n> empty element. This element can happily be cast to a string which is\n> unexpected. For example the following statement:\n> \n> yamlDict[\"nonexistent\"].get(\"default\")\n> \n> is expected to return \"default\" but actually returns \"\". Fix this by\n> introducing a empty type to distinguish between an empty YamlObject and\n> a YamlObject of type value containing an empty string. For completeness\n> add an isEmpty() function and an explicit cast to bool to be able to\n> test for that type.\n> \n> Extend the tests accordingly.\n> \n> Signed-off-by: Stefan Klug \n\nYou should either squash this with 2/3 or add the should_fail like Kieran\nsuggested to avoid bisection failures.\n\nBut otherwise,\n\nReviewed-by: Paul Elder \n\n> \n> ---\n> Changes in v3:\n> - Move tests for the existing functionality into own patches\n> - Fixed some typos\n> \n> Changes in v2:\n> - Added explicit to bool conversion operator\n> - Fixed typos from review\n> - I didn't add the rb tags, because the v2 adds quite some logic\n> (especially tests) and this is a central piece of code\n> ---\n> include/libcamera/internal/yaml_parser.h | 9 ++++++++\n> src/libcamera/yaml_parser.cpp | 27 ++++++++++++++++++------\n> test/yaml-parser.cpp | 18 ++++++++++++++++\n> 3 files changed, 48 insertions(+), 6 deletions(-)\n> \n> diff --git a/include/libcamera/internal/yaml_parser.h b/include/libcamera/internal/yaml_parser.h\n> index 16708e488d88..6211ff4ae563 100644\n> --- a/include/libcamera/internal/yaml_parser.h\n> +++ b/include/libcamera/internal/yaml_parser.h\n> @@ -159,6 +159,14 @@ public:\n> \t{\n> \t\treturn type_ == Type::Dictionary;\n> \t}\n> +\tbool isEmpty() const\n> +\t{\n> +\t\treturn type_ == Type::Empty;\n> +\t}\n> +\texplicit operator bool() const\n> +\t{\n> +\t\treturn type_ != Type::Empty;\n> +\t}\n> \n> \tstd::size_t size() const;\n> \n> @@ -212,6 +220,7 @@ private:\n> \t\tDictionary,\n> \t\tList,\n> \t\tValue,\n> +\t\tEmpty,\n> \t};\n> \n> \ttemplate\n> diff --git a/src/libcamera/yaml_parser.cpp b/src/libcamera/yaml_parser.cpp\n> index 8b6a403898be..4784f2dc3d62 100644\n> --- a/src/libcamera/yaml_parser.cpp\n> +++ b/src/libcamera/yaml_parser.cpp\n> @@ -38,12 +38,12 @@ static const YamlObject empty;\n> * \\brief A class representing the tree structure of the YAML content\n> *\n> * The YamlObject class represents the tree structure of YAML content. A\n> - * YamlObject can be a dictionary or list of YamlObjects or a value if a tree\n> - * leaf.\n> + * YamlObject can be empty, a dictionary or list of YamlObjects, or a value if a\n> + * tree leaf.\n> */\n> \n> YamlObject::YamlObject()\n> -\t: type_(Type::Value)\n> +\t: type_(Type::Empty)\n> {\n> }\n> \n> @@ -70,6 +70,20 @@ YamlObject::~YamlObject() = default;\n> * \\return True if the YamlObject is a dictionary, false otherwise\n> */\n> \n> +/**\n> + * \\fn YamlObject::isEmpty()\n> + * \\brief Return whether the YamlObject is an empty\n> + *\n> + * \\return True if the YamlObject is empty, false otherwise\n> + */\n> +\n> +/**\n> + * \\fn YamlObject::operator bool()\n> + * \\brief Return whether the YamlObject is a non-empty\n> + *\n> + * \\return False if the YamlObject is empty, true otherwise\n> + */\n> +\n> /**\n> * \\fn YamlObject::size()\n> * \\brief Retrieve the number of elements in a dictionary or list YamlObject\n> @@ -443,7 +457,8 @@ template std::optional> YamlObject::getList() const;\n> *\n> * This function retrieves an element of the YamlObject. Only YamlObject\n> * instances of List type associate elements with index, calling this function\n> - * on other types of instances is invalid and results in undefined behaviour.\n> + * on other types of instances or with an invalid index results in an empty\n> + * object.\n> *\n> * \\return The YamlObject as an element of the list\n> */\n> @@ -480,8 +495,8 @@ bool YamlObject::contains(const std::string &key) const\n> *\n> * This function retrieve a member of a YamlObject by name. Only YamlObject\n> * instances of Dictionary type associate elements with names, calling this\n> - * function on other types of instances is invalid and results in undefined\n> - * behaviour.\n> + * function on other types of instances or with a nonexistent key results in an\n> + * empty object.\n> *\n> * \\return The YamlObject corresponding to the \\a key member\n> */\n> diff --git a/test/yaml-parser.cpp b/test/yaml-parser.cpp\n> index 4cc77e26ae39..1b22c87b72f1 100644\n> --- a/test/yaml-parser.cpp\n> +++ b/test/yaml-parser.cpp\n> @@ -542,6 +542,24 @@ protected:\n> \t\t\treturn TestFail;\n> \t\t}\n> \n> +\t\t/* Test nonexistent object has value type empty. */\n> +\t\tif (!dictObj[\"nonexistent\"].isEmpty()) {\n> +\t\t\tcerr << \"Accessing nonexistent object returns non-empty object\" << std::endl;\n> +\t\t\treturn TestFail;\n> +\t\t}\n> +\n> +\t\t/* Test explicit cast to bool on an empty object returns true. */\n> +\t\tif (!!dictObj[\"empty\"] != true) {\n> +\t\t\tcerr << \"Casting empty entry to bool returns false\" << std::endl;\n> +\t\t\treturn TestFail;\n> +\t\t}\n> +\n> +\t\t/* Test explicit cast to bool on nonexistent object returns false. */\n> +\t\tif (!!dictObj[\"nonexistent\"] != false) {\n> +\t\t\tcerr << \"Casting nonexistent dict entry to bool returns true\" << std::endl;\n> +\t\t\treturn TestFail;\n> +\t\t}\n> +\n> \t\t/* Make sure utils::map_keys() works on the adapter. */\n> \t\t(void)utils::map_keys(dictObj.asDict());\n> \n> -- \n> 2.43.0\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id E1CC9C0F1B\n\tfor ;\n\tMon, 23 Sep 2024 10:09:36 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8B67663510;\n\tMon, 23 Sep 2024 12:09:36 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 40CA06037E\n\tfor ;\n\tMon, 23 Sep 2024 12:09:35 +0200 (CEST)","from pyrite.rasen.tech (unknown\n\t[IPv6:2404:7a81:160:2100:2fac:fd68:a1d3:63f3])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 2FF182E0;\n\tMon, 23 Sep 2024 12:08:07 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"CempSjul\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1727086089;\n\tbh=+4HSkQ7JVhPobn5MFU08ert+lxzFRFqSM6tjXt6qZOw=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=CempSjulICY4cwFzv6rfMJp3uidwVAWfkjlUb9SniOYfEmrKQEOF7ao2Yxg7FIv5H\n\t9tk7Du6kaJ9aqMt3bT29eZIQdO/qYd+Sg7kzIxNo5nZf9PAA4EW1hstK1CuRYNDMOP\n\tcOPOfosonSsEuN8/kOHngvNtgZg/cpvHENpNigsg=","Date":"Mon, 23 Sep 2024 19:09:26 +0900","From":"Paul Elder ","To":"Stefan Klug ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [PATCH v3 3/3] libcamera: yaml-parser: Differentiate between\n\tempty and empty string","Message-ID":"","References":"<20240920132823.88433-1-stefan.klug@ideasonboard.com>\n\t<20240920132823.88433-4-stefan.klug@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20240920132823.88433-4-stefan.klug@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]