From patchwork Thu Jul 14 15:24:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 16629 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0FDBBBD1F1 for ; Thu, 14 Jul 2022 15:24:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C039463319; Thu, 14 Jul 2022 17:24:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1657812259; bh=XeV57vJfqqCjsrL8TJwPSIRYNK8578AqVNFk5CrKRKM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=KKIGgH/XYFwgw8Sv9sSd3fQi26Xzxjmjoicdx5MjD9jHQhbaxLTdVlxSa8hYmaOPT dIPiNCK6t332WNx3nxbaSwxU6waov5FM+py+oZLpaDIxKEC4ftVzK8zp0Eiy0OF0Kc fBqO71pIwGRc4UBItTHKvm2CnSvhrShaR0i70w4YUBItdMskIGAcGBhLUYEdf5C/jW 9ocQI8c9NHgGSzdFZBswdvwR1eSu5nnHBUdvkgN045rVpHubbvesGvdbUH94H09Kzn ww87Mq0Kd/BU0UhoBNX3LpKA3esc/Svr0GY3KnedTB84wFNUvGg8BpDaH9wRQ1Hid3 q+gWnrhkteD5w== Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B703963311 for ; Thu, 14 Jul 2022 17:24:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="FzbKAoY9"; dkim-atps=neutral Received: by mail-wm1-x32f.google.com with SMTP id n185so1250899wmn.4 for ; Thu, 14 Jul 2022 08:24:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0i+rzFI/7f7fcqw8XWNmTV4K6QvSwvGv9hb4ewIRxws=; b=FzbKAoY9rpwKQyOoPOMWAxbOGHgcMtJoEIsik40xXkpm/6yhZoFyOtFl7PcCTst//D lXeyruSs7+LuGc8VntBJ4pEv7cu1btyqOBMXvxVD8MTt1mgzHcG/4CDZCBX5BJfM3eoh erEJ+62Wve3tdGb+7h/cKdrPiprkK7fyor6vHCnNyjIxnd8wK+yW1ImoYviaN88s31X7 +CGLr61TiwQZjOeGZPf4nXDndyYjG7YhmpwojMZjlNhO7a0akB5WQlB7MDua/n0aXV1l ym8Ysr1zitQRYfaEYCpONgGp7tp2jLoNRcLzSo8B+/LCq8tRQPmqhTur8l4U80BAxQdC 0wmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0i+rzFI/7f7fcqw8XWNmTV4K6QvSwvGv9hb4ewIRxws=; b=TmCBjkmJJQ6cZj1bqRi3jX/CnLLMXiO2hzslI1gREmQDUKdOmL5mg01BbEumzWOW9X n93nAJEJAed6SqAcoK5PTqmrtsFCHAh2Qg/eg7nCExdkAIf/R5rtpR6RzMXl+NuMv1c0 qZsEWW0IBLVKgjpmjc9xowK++Qlr1RH9Ap0BWczOdOuMON36eYN3gQcXT05Eu6BbKpSV 3OSscyr0qo1WhLdFvhZ6ZdyPdgOr02WE+tIe1H7P2f0nPpAB+YEtPvBt0w4xpY3n1BwE AARZXBZosVMGjS2JN0yllb181YAZC0CMMBZiD7Mvps7xpiLXXkV45CMBSIrWa3PQIao4 hkvA== X-Gm-Message-State: AJIora+zrCKuwn3o411gE/tYFhu1kH+WhaXmRUz8lDTIjWsnZT7cBlnQ TP7T5Ez+ry0zMGksAeNcGMopTb/ea1x9rA== X-Google-Smtp-Source: AGRyM1tco70pViMGng0+9TQDhZlTPXth36l/JcpBRdE/ska+rex6U0ekfLLolExaVEVkYcbzrNgHQg== X-Received: by 2002:a05:600c:2411:b0:3a2:e09a:6cb8 with SMTP id 17-20020a05600c241100b003a2e09a6cb8mr9463192wmp.44.1657812257142; Thu, 14 Jul 2022 08:24:17 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h14-20020a1ccc0e000000b003a2e7c13a3asm2322476wmb.42.2022.07.14.08.24.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Jul 2022 08:24:14 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Jul 2022 16:24:03 +0100 Message-Id: <20220714152409.9780-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220714152409.9780-1-naush@raspberrypi.com> References: <20220714152409.9780-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 2/8] libcamera: yaml_parser: Preserve order of items in dictionary X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart The std::map container used to store dictionary items in YamlObject doesn't preserve the YAML data order, as maps are ordered by key, not by insertion order. While this is compliant with the YAML specification which doesn't guarantee ordering of mappings, the Raspberry Pi IPA relies on elements being ordered as in the YAML data. To replace the dependency on boost with the YamlParser class, we thus need to guarantee that the order is preserved. Preserve the order by storing items in list_ unconditionally. Turn the list_ vector from storing YamlObject unique pointers to storing key-value pairs, with the key being absent when the object is a list, not a dictionary. The YamlObject implementation is updated to preserve the existing API, with the only difference being that YamlObject::memberNames() now returns member names in the same order as in the YAML file. The ordering is an implementation detail, so changing it doesn't violate the YAML specification. The documentation is not updated to reflect this, as we don't want any new user to rely on a particular ordering. This commit could be reverted if desired when the Raspberry Pi IPA updates to a new tuning data format and drops support for the old format. Signed-off-by: Laurent Pinchart Reviewed-by: Naushir Patuck Tested-by: Naushir Patuck --- include/libcamera/internal/yaml_parser.h | 37 +++++++++++++++--------- src/libcamera/yaml_parser.cpp | 35 ++++++++++++++-------- 2 files changed, 46 insertions(+), 26 deletions(-) diff --git a/include/libcamera/internal/yaml_parser.h b/include/libcamera/internal/yaml_parser.h index 064cf44381d7..eb39ea8c79e4 100644 --- a/include/libcamera/internal/yaml_parser.h +++ b/include/libcamera/internal/yaml_parser.h @@ -24,12 +24,21 @@ class YamlParserContext; class YamlObject { private: - using DictContainer = std::map>; + struct Value { + Value(std::string &&k, std::unique_ptr &&v) + : key(std::move(k)), value(std::move(v)) + { + } + std::string key; + std::unique_ptr value; + }; + + using Container = std::vector; using ListContainer = std::vector>; public: #ifndef __DOXYGEN__ - template + template class Iterator { public: @@ -65,10 +74,10 @@ public: } protected: - typename Container::const_iterator it_; + Container::const_iterator it_; }; - template + template class Adapter { public: @@ -91,7 +100,7 @@ public: const Container &container_; }; - class ListIterator : public Iterator + class ListIterator : public Iterator { public: using value_type = const YamlObject &; @@ -100,16 +109,16 @@ public: value_type operator*() const { - return *it_->get(); + return *it_->value.get(); } pointer operator->() const { - return it_->get(); + return it_->value.get(); } }; - class DictIterator : public Iterator + class DictIterator : public Iterator { public: using value_type = std::pair; @@ -118,17 +127,17 @@ public: value_type operator*() const { - return { it_->first, *it_->second.get() }; + return { it_->key, *it_->value.get() }; } }; - class DictAdapter : public Adapter + class DictAdapter : public Adapter { public: using key_type = std::string; }; - class ListAdapter : public Adapter + class ListAdapter : public Adapter { }; #endif /* __DOXYGEN__ */ @@ -167,7 +176,7 @@ public: #endif T get(const T &defaultValue, bool *ok = nullptr) const; - DictAdapter asDict() const { return DictAdapter{ dictionary_ }; } + DictAdapter asDict() const { return DictAdapter{ list_ }; } ListAdapter asList() const { return ListAdapter{ list_ }; } const YamlObject &operator[](std::size_t index) const; @@ -189,8 +198,8 @@ private: Type type_; std::string value_; - ListContainer list_; - DictContainer dictionary_; + Container list_; + std::map dictionary_; }; class YamlParser final diff --git a/src/libcamera/yaml_parser.cpp b/src/libcamera/yaml_parser.cpp index 5c45e44e49c3..e30de050a0ea 100644 --- a/src/libcamera/yaml_parser.cpp +++ b/src/libcamera/yaml_parser.cpp @@ -91,7 +91,6 @@ std::size_t YamlObject::size() const { switch (type_) { case Type::Dictionary: - return dictionary_.size(); case Type::List: return list_.size(); default: @@ -311,11 +310,11 @@ Size YamlObject::get(const Size &defaultValue, bool *ok) const * that ok == nullptr. */ bool valid; - uint32_t width = list_[0]->get(0, &valid); + uint32_t width = list_[0].value->get(0, &valid); if (!valid) return defaultValue; - uint32_t height = list_[1]->get(0, &valid); + uint32_t height = list_[1].value->get(0, &valid); if (!valid) return defaultValue; @@ -379,7 +378,7 @@ const YamlObject &YamlObject::operator[](std::size_t index) const if (type_ != Type::List || index >= size()) return empty; - return *list_[index]; + return *list_[index].value; } /** @@ -395,7 +394,7 @@ const YamlObject &YamlObject::operator[](std::size_t index) const */ bool YamlObject::contains(const std::string &key) const { - if (dictionary_.find(key) == dictionary_.end()) + if (dictionary_.find(std::ref(key)) == dictionary_.end()) return false; return true; @@ -667,16 +666,16 @@ int YamlParserContext::parseNextYamlObject(YamlObject &yamlObject, EventPtr even yamlObject.type_ = YamlObject::Type::List; auto &list = yamlObject.list_; auto handler = [this, &list](EventPtr evt) { - list.emplace_back(new YamlObject()); - return parseNextYamlObject(*list.back(), std::move(evt)); + list.emplace_back(std::string{}, std::make_unique()); + return parseNextYamlObject(*list.back().value, std::move(evt)); }; return parseDictionaryOrList(YamlObject::Type::List, handler); } case YAML_MAPPING_START_EVENT: { yamlObject.type_ = YamlObject::Type::Dictionary; - auto &dictionary = yamlObject.dictionary_; - auto handler = [this, &dictionary](EventPtr evtKey) { + auto &list = yamlObject.list_; + auto handler = [this, &list](EventPtr evtKey) { /* Parse key */ if (evtKey->type != YAML_SCALAR_EVENT) { LOG(YamlParser, Error) << "Expect key at line: " @@ -694,10 +693,19 @@ int YamlParserContext::parseNextYamlObject(YamlObject &yamlObject, EventPtr even if (!evtValue) return -EINVAL; - auto elem = dictionary.emplace(key, std::make_unique()); - return parseNextYamlObject(*elem.first->second.get(), std::move(evtValue)); + auto &elem = list.emplace_back(std::move(key), + std::make_unique()); + return parseNextYamlObject(*elem.value, std::move(evtValue)); }; - return parseDictionaryOrList(YamlObject::Type::Dictionary, handler); + int ret = parseDictionaryOrList(YamlObject::Type::Dictionary, handler); + if (ret) + return ret; + + auto &dictionary = yamlObject.dictionary_; + for (const auto &elem : list) + dictionary.emplace(elem.key, elem.value.get()); + + return 0; } default: @@ -753,6 +761,9 @@ int YamlParserContext::parseNextYamlObject(YamlObject &yamlObject, EventPtr even * The YamlParser::parse() function takes an open FILE, parses its contents, and * returns a pointer to a YamlObject corresponding to the root node of the YAML * document. + * + * The parser preserves the order of items in the YAML file, for both lists and + * dictionaries. */ /**