[{"id":34264,"web_url":"https://patchwork.libcamera.org/comment/34264/","msgid":"<174741664081.476729.1844022348649238166@calcite>","date":"2025-05-16T17:30:40","subject":"Re: [RFC PATCH v1 8/8] utils: codegen: ipc: Simplify deserialization","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"content":"Quoting Barnabás Pőcze (2025-05-15 14:00:12)\n> First, introduce the `SeriReader` type, which is a collection\n> of bytes and file descriptors, with the appropriate methods to\n> consume the first couples bytes / file descriptors.\n> \n> Then a new method is added to `IPCMessage` that returns an appropriately\n> constructed `SeriReader` for parsing the message contents.\n> \n> Then three of the four `deserialize()` overloads are removed, the\n> remaining one is converted to have a single `SeriReader` and an\n> optional `ControlSerializer` as arguments.\n> \n> The remaining `deserialize()` function is also changed to return an\n> `std::optional` to be able to report deserialization failure.\n> \n> There is also a more fundamental change in the serialization: previously,\n> the number of bytes taken up by an item has been written before the serialized\n> bytes (and conditionally the number of file descriptors) when the item is\n> serialized as part of a struct, array, map, function parameter list. This\n> is changed: the number of bytes and file descriptors are *not* serialized\n> into the final buffer. This affords some simplification of the serialization\n> related code paths, but most importantly, it greatly simplifies and unifies\n> how an object is (de)serialized because the deserialization of every object\n> becomes completely self-contained.\n> \n> As a consequence of that, strings now include their lengths as part of the\n> string serialization, and it is not left to an \"upper\" layer.\n> \n> Another consequence is that an \"out parameter\" of a remote function call\n> must be deserialized if a later out parameter is needed, even if itself\n> is not. This does not appear to be a great limitation since in all\n> situations presently none of the out parameters are ignored.\n> \n> Finally, the code generation templates are adapted to the above changes.\n> This allows the simplification of the deserialization templates as now\n> calling `IPADataSerializer::deserialize(reader, &controlSerializer_)`\n> is appropriate for any type.\n> \n> Bug: https://bugs.libcamera.org/show_bug.cgi?id=269\n> Signed-off-by: Barnabás Pőcze \n> ---\n> .../libcamera/internal/ipa_data_serializer.h | 228 +++--------\n> include/libcamera/internal/ipc_pipe.h | 3 +\n> include/libcamera/internal/serialization.h | 91 +++++\n> src/libcamera/ipa_data_serializer.cpp | 356 ++++--------------\n> src/libcamera/ipc_pipe.cpp | 5 +\n> test/ipc/unixsocket_ipc.cpp | 18 +-\n> .../generated_serializer_test.cpp | 17 +-\n> .../ipa_data_serializer_test.cpp | 32 +-\n> .../module_ipa_proxy.cpp.tmpl | 30 +-\n> .../module_ipa_proxy.h.tmpl | 5 +-\n> .../module_ipa_proxy_worker.cpp.tmpl | 5 +-\n> .../libcamera_templates/proxy_functions.tmpl | 113 +-----\n> .../libcamera_templates/serializer.tmpl | 238 +-----------\n> 13 files changed, 316 insertions(+), 825 deletions(-)\n> create mode 100644 include/libcamera/internal/serialization.h\n> \n> diff --git a/include/libcamera/internal/ipa_data_serializer.h b/include/libcamera/internal/ipa_data_serializer.h\n> index 564f59e25..0d03729a1 100644\n> --- a/include/libcamera/internal/ipa_data_serializer.h\n> +++ b/include/libcamera/internal/ipa_data_serializer.h\n> @@ -7,6 +7,7 @@\n> \n> #pragma once\n> \n> +#include \n> #include \n> #include \n> #include \n> @@ -23,6 +24,7 @@\n> #include \n> \n> #include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/serialization.h\"\n> \n> namespace libcamera {\n> \n> @@ -39,26 +41,6 @@ void appendPOD(std::vector &vec, T val)\n> memcpy(&*(vec.end() - byteWidth), &val, byteWidth);\n> }\n> \n> -template - std::enable_if_t> * = nullptr>\n> -T readPOD(std::vector::const_iterator it, size_t pos,\n> - std::vector::const_iterator end)\n> -{\n> - ASSERT(pos + it < end);\n> -\n> - T ret = 0;\n> - memcpy(&ret, &(*(it + pos)), sizeof(ret));\n> -\n> - return ret;\n> -}\n> -\n> -template - std::enable_if_t> * = nullptr>\n> -T readPOD(std::vector &vec, size_t pos)\n> -{\n> - return readPOD(vec.cbegin(), pos, vec.end());\n> -}\n> -\n> } /* namespace */\n> \n> template\n> @@ -68,20 +50,8 @@ public:\n> static std::tuple, std::vector>\n> serialize(const T &data, ControlSerializer *cs = nullptr);\n> \n> - static T deserialize(const std::vector &data,\n> - ControlSerializer *cs = nullptr);\n> - static T deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - ControlSerializer *cs = nullptr);\n> -\n> - static T deserialize(const std::vector &data,\n> - const std::vector &fds,\n> - ControlSerializer *cs = nullptr);\n> - static T deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - std::vector::const_iterator fdsBegin,\n> - std::vector::const_iterator fdsEnd,\n> - ControlSerializer *cs = nullptr);\n> + [[nodiscard]] static std::optional\n> + deserialize(SeriReader &reader, ControlSerializer *cs = nullptr);\n> };\n> \n> #ifndef __DOXYGEN__\n> @@ -121,9 +91,6 @@ public:\n> std::tie(dvec, fvec) =\n> IPADataSerializer::serialize(it, cs);\n> \n> - appendPOD(dataVec, dvec.size());\n> - appendPOD(dataVec, fvec.size());\n> -\n> dataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n> fdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n> }\n> @@ -131,52 +98,25 @@ public:\n> return { dataVec, fdsVec };\n> }\n> \n> - static std::vector deserialize(std::vector &data, ControlSerializer *cs = nullptr)\n> + [[nodiscard]] static std::optional>\n> + deserialize(SeriReader &reader, ControlSerializer *cs = nullptr)\n> {\n> - return deserialize(data.cbegin(), data.cend(), cs);\n> - }\n> + uint32_t vecLen;\n> + if (!reader.read(vecLen))\n> + return {};\n> \n> - static std::vector deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - ControlSerializer *cs = nullptr)\n> - {\n> - std::vector fds;\n> - return deserialize(dataBegin, dataEnd, fds.cbegin(), fds.cend(), cs);\n> - }\n> + std::vector ret;\n> + ret.reserve(vecLen);\n> \n> - static std::vector deserialize(std::vector &data, std::vector &fds,\n> - ControlSerializer *cs = nullptr)\n> - {\n> - return deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n> - }\n> -\n> - static std::vector deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - std::vector::const_iterator fdsBegin,\n> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> - ControlSerializer *cs = nullptr)\n> - {\n> - uint32_t vecLen = readPOD(dataBegin, 0, dataEnd);\n> - std::vector ret(vecLen);\n> -\n> - std::vector::const_iterator dataIter = dataBegin + 4;\n> - std::vector::const_iterator fdIter = fdsBegin;\n> for (uint32_t i = 0; i < vecLen; i++) {\n> - uint32_t sizeofData = readPOD(dataIter, 0, dataEnd);\n> - uint32_t sizeofFds = readPOD(dataIter, 4, dataEnd);\n> - dataIter += 8;\n> -\n> - ret[i] = IPADataSerializer::deserialize(dataIter,\n> - dataIter + sizeofData,\n> - fdIter,\n> - fdIter + sizeofFds,\n> - cs);\n> -\n> - dataIter += sizeofData;\n> - fdIter += sizeofFds;\n> + auto item = IPADataSerializer::deserialize(reader, cs);\n> + if (!item)\n> + return {};\n> +\n> + ret.emplace_back(std::move(*item));\n> }\n> \n> - return ret;\n> + return std::move(ret);\n> }\n> };\n> \n> @@ -218,18 +158,12 @@ public:\n> std::tie(dvec, fvec) =\n> IPADataSerializer::serialize(it.first, cs);\n> \n> - appendPOD(dataVec, dvec.size());\n> - appendPOD(dataVec, fvec.size());\n> -\n> dataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n> fdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n> \n> std::tie(dvec, fvec) =\n> IPADataSerializer::serialize(it.second, cs);\n> \n> - appendPOD(dataVec, dvec.size());\n> - appendPOD(dataVec, fvec.size());\n> -\n> dataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n> fdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n> }\n> @@ -237,63 +171,25 @@ public:\n> return { dataVec, fdsVec };\n> }\n> \n> - static std::map deserialize(std::vector &data, ControlSerializer *cs = nullptr)\n> - {\n> - return deserialize(data.cbegin(), data.cend(), cs);\n> - }\n> -\n> - static std::map deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - ControlSerializer *cs = nullptr)\n> + [[nodiscard]] static std::optional>\n> + deserialize(SeriReader &reader, ControlSerializer *cs = nullptr)\n> {\n> - std::vector fds;\n> - return deserialize(dataBegin, dataEnd, fds.cbegin(), fds.cend(), cs);\n> - }\n> + uint32_t mapLen;\n> + if (!reader.read(mapLen))\n> + return {};\n> \n> - static std::map deserialize(std::vector &data, std::vector &fds,\n> - ControlSerializer *cs = nullptr)\n> - {\n> - return deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n> - }\n> -\n> - static std::map deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - std::vector::const_iterator fdsBegin,\n> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> - ControlSerializer *cs = nullptr)\n> - {\n> std::map ret;\n> \n> - uint32_t mapLen = readPOD(dataBegin, 0, dataEnd);\n> -\n> - std::vector::const_iterator dataIter = dataBegin + 4;\n> - std::vector::const_iterator fdIter = fdsBegin;\n> for (uint32_t i = 0; i < mapLen; i++) {\n> - uint32_t sizeofData = readPOD(dataIter, 0, dataEnd);\n> - uint32_t sizeofFds = readPOD(dataIter, 4, dataEnd);\n> - dataIter += 8;\n> -\n> - K key = IPADataSerializer::deserialize(dataIter,\n> - dataIter + sizeofData,\n> - fdIter,\n> - fdIter + sizeofFds,\n> - cs);\n> -\n> - dataIter += sizeofData;\n> - fdIter += sizeofFds;\n> - sizeofData = readPOD(dataIter, 0, dataEnd);\n> - sizeofFds = readPOD(dataIter, 4, dataEnd);\n> - dataIter += 8;\n> -\n> - const V value = IPADataSerializer::deserialize(dataIter,\n> - dataIter + sizeofData,\n> - fdIter,\n> - fdIter + sizeofFds,\n> - cs);\n> - ret.insert({ key, value });\n> -\n> - dataIter += sizeofData;\n> - fdIter += sizeofFds;\n> + auto key = IPADataSerializer::deserialize(reader, cs);\n> + if (!key)\n> + return {};\n> +\n> + auto value = IPADataSerializer::deserialize(reader, cs);\n> + if (!value)\n> + return {};\n> +\n> + ret.try_emplace(std::move(*key), std::move(*value));\n\nI'm curious, what's the reason for using try_emplace() here?\n\n> }\n> \n> return ret;\n> @@ -314,33 +210,14 @@ public:\n> return { dataVec, {} };\n> }\n> \n> - static Flags deserialize(std::vector &data,\n> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n> - {\n> - return deserialize(data.cbegin(), data.cend());\n> - }\n> -\n> - static Flags deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n> + [[nodiscard]] static std::optional>\n> + deserialize(SeriReader &reader, [[maybe_unused]] ControlSerializer *cs = nullptr)\n> {\n> - return Flags{ static_cast(readPOD(dataBegin, 0, dataEnd)) };\n> - }\n> + uint32_t value;\n> + if (!reader.read(value))\n> + return {};\n> \n> - static Flags deserialize(std::vector &data,\n> - [[maybe_unused]] std::vector &fds,\n> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n> - {\n> - return deserialize(data.cbegin(), data.cend());\n> - }\n> -\n> - static Flags deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n> - {\n> - return deserialize(dataBegin, dataEnd);\n> + return Flags{ static_cast(value) };\n> }\n> };\n> \n> @@ -360,33 +237,14 @@ public:\n> return { dataVec, {} };\n> }\n> \n> - static E deserialize(std::vector &data,\n> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n> + [[nodiscard]] static std::optional\n> + deserialize(SeriReader &reader, [[maybe_unused]] ControlSerializer *cs = nullptr)\n> {\n> - return deserialize(data.cbegin(), data.cend());\n> - }\n> + U value;\n> + if (!reader.read(value))\n> + return {};\n> \n> - static E deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n> - {\n> - return static_cast(readPOD(dataBegin, 0, dataEnd));\n> - }\n> -\n> - static E deserialize(std::vector &data,\n> - [[maybe_unused]] std::vector &fds,\n> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n> - {\n> - return deserialize(data.cbegin(), data.cend());\n> - }\n> -\n> - static E deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n> - {\n> - return deserialize(dataBegin, dataEnd);\n> + return static_cast(value);\n> }\n> };\n> \n> diff --git a/include/libcamera/internal/ipc_pipe.h b/include/libcamera/internal/ipc_pipe.h\n> index 418c4622f..2b6cde042 100644\n> --- a/include/libcamera/internal/ipc_pipe.h\n> +++ b/include/libcamera/internal/ipc_pipe.h\n> @@ -14,6 +14,7 @@\n> #include \n> \n> #include \"libcamera/internal/ipc_unixsocket.h\"\n> +#include \"libcamera/internal/serialization.h\"\n> \n> namespace libcamera {\n> \n> @@ -40,6 +41,8 @@ public:\n> const std::vector &data() const { return data_; }\n> const std::vector &fds() const { return fds_; }\n> \n> + SeriReader reader() const { return { data(), fds() }; }\n> +\n> private:\n> Header header_;\n> \n> diff --git a/include/libcamera/internal/serialization.h b/include/libcamera/internal/serialization.h\n> new file mode 100644\n> index 000000000..daa7e5438\n> --- /dev/null\n> +++ b/include/libcamera/internal/serialization.h\n> @@ -0,0 +1,91 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2025, Ideas On Board Oy\n> + *\n> + * Data (de)serialization helper structures\n> + */\n> +#pragma once\n> +\n> +#include \n> +#include \n> +#include \n> +\n> +#include \n> +#include \n> +\n> +namespace libcamera {\n> +\n> +#ifndef __DOXYGEN__\n> +class SeriReader\n> +{\n> +public:\n> + SeriReader(Span data, Span fds = {})\n> + : data_(data),\n> + fds_(fds)\n> + {\n> + }\n> +\n> + SeriReader(Span data, Span fds = {})\n> + : data_(reinterpret_cast(data.data()),\n> + reinterpret_cast(data.data() + data.size())),\n> + fds_(fds)\n> + {\n> + }\n> +\n> + [[nodiscard]] const std::byte *consume(std::size_t s)\n> + {\n> + if (data_.size() < s)\n> + return nullptr;\n> +\n> + const auto *p = data_.data();\n> + data_ = data_.subspan(s);\n> +\n> + return p;\n> + }\n> +\n> + template\n> + [[nodiscard]] bool read(Ts &...xs)\n> + {\n> + static_assert((std::is_trivially_copyable_v && ...));\n> +\n> + const auto *p = consume((sizeof(xs) + ...));\n> + if (p)\n> + ((memcpy(&xs, p, sizeof(xs)), p += sizeof(xs)), ...);\n> +\n> + return p;\n> + }\n\nThat is so cool.\n\n> +\n> + template\n> + [[nodiscard]] auto read()\n> + {\n> + std::tuple xs;\n> + bool ok = std::apply([&](auto &...vs) {\n> + return read(vs...);\n> + }, xs);\n> +\n> + if constexpr (sizeof...(Ts) == 1)\n> + return ok ? std::optional(std::get<0>(xs)) : std::nullopt;\n> + else\n> + return ok ? std::optional(xs) : std::nullopt;\n\nThis means that each std::nullopt is of different types no? afaict you only use\nthis function when sizeof...(Ts) == 1.\n\n> + }\n> +\n> + [[nodiscard]] const SharedFD *nextFd()\n> + {\n> + if (fds_.empty())\n> + return nullptr;\n> +\n> + const auto *p = fds_.data();\n> + fds_ = fds_.subspan(1);\n> +\n> + return p;\n> + }\n> +\n> + [[nodiscard]] bool empty() const { return data_.empty() && fds_.empty(); }\n> +\n> +private:\n> + Span data_;\n> + Span fds_;\n> +};\n> +#endif\n> +\n> +} /* namespace libcamera */\n> diff --git a/src/libcamera/ipa_data_serializer.cpp b/src/libcamera/ipa_data_serializer.cpp\n> index 0537f785b..979a1e0db 100644\n> --- a/src/libcamera/ipa_data_serializer.cpp\n> +++ b/src/libcamera/ipa_data_serializer.cpp\n> @@ -50,42 +50,6 @@ namespace {\n> * generated IPA proxies.\n> */\n> \n> -/**\n> - * \\fn template T readPOD(std::vector::iterator it, size_t pos,\n> - * std::vector::iterator end)\n> - * \\brief Read POD from byte vector, in little-endian order\n> - * \\tparam T Type of POD to read\n> - * \\param[in] it Iterator of byte vector to read from\n> - * \\param[in] pos Index in byte vector to read from\n> - * \\param[in] end Iterator marking end of byte vector\n> - *\n> - * This function is meant to be used by the IPA data serializer, and the\n> - * generated IPA proxies.\n> - *\n> - * If the \\a pos plus the byte-width of the desired POD is past \\a end, it is\n> - * a fata error will occur, as it means there is insufficient data for\n> - * deserialization, which should never happen.\n> - *\n> - * \\return The POD read from \\a it at index \\a pos\n> - */\n> -\n> -/**\n> - * \\fn template T readPOD(std::vector &vec, size_t pos)\n> - * \\brief Read POD from byte vector, in little-endian order\n> - * \\tparam T Type of POD to read\n> - * \\param[in] vec Byte vector to read from\n> - * \\param[in] pos Index in vec to start reading from\n> - *\n> - * This function is meant to be used by the IPA data serializer, and the\n> - * generated IPA proxies.\n> - *\n> - * If the \\a pos plus the byte-width of the desired POD is past the end of\n> - * \\a vec, a fatal error will occur, as it means there is insufficient data\n> - * for deserialization, which should never happen.\n> - *\n> - * \\return The POD read from \\a vec at index \\a pos\n> - */\n> -\n> } /* namespace */\n> \n> /**\n> @@ -106,80 +70,13 @@ namespace {\n> \n> /**\n> * \\fn template IPADataSerializer::deserialize(\n> - * const std::vector &data,\n> + * SeriReader &reader,\n> * ControlSerializer *cs = nullptr)\n> - * \\brief Deserialize byte vector into an object\n> + * \\brief Deserialize bytes and file descriptors vector into an object\n> * \\tparam T Type of object to deserialize to\n> - * \\param[in] data Byte vector to deserialize from\n> + * \\param[in] reader Source of bytes and file descriptors\n> * \\param[in] cs ControlSerializer\n> *\n> - * This version of deserialize() can be used if the object type \\a T and its\n> - * members don't have any SharedFD.\n> - *\n> - * \\a cs is only necessary if the object type \\a T or its members contain\n> - * ControlList or ControlInfoMap.\n> - *\n> - * \\return The deserialized object\n> - */\n> -\n> -/**\n> - * \\fn template IPADataSerializer::deserialize(\n> - * std::vector::const_iterator dataBegin,\n> - * std::vector::const_iterator dataEnd,\n> - * ControlSerializer *cs = nullptr)\n> - * \\brief Deserialize byte vector into an object\n> - * \\tparam T Type of object to deserialize to\n> - * \\param[in] dataBegin Begin iterator of byte vector to deserialize from\n> - * \\param[in] dataEnd End iterator of byte vector to deserialize from\n> - * \\param[in] cs ControlSerializer\n> - *\n> - * This version of deserialize() can be used if the object type \\a T and its\n> - * members don't have any SharedFD.\n> - *\n> - * \\a cs is only necessary if the object type \\a T or its members contain\n> - * ControlList or ControlInfoMap.\n> - *\n> - * \\return The deserialized object\n> - */\n> -\n> -/**\n> - * \\fn template IPADataSerializer::deserialize(\n> - * const std::vector &data,\n> - * const std::vector &fds,\n> - * ControlSerializer *cs = nullptr)\n> - * \\brief Deserialize byte vector and fd vector into an object\n> - * \\tparam T Type of object to deserialize to\n> - * \\param[in] data Byte vector to deserialize from\n> - * \\param[in] fds Fd vector to deserialize from\n> - * \\param[in] cs ControlSerializer\n> - *\n> - * This version of deserialize() (or the iterator version) must be used if\n> - * the object type \\a T or its members contain SharedFD.\n> - *\n> - * \\a cs is only necessary if the object type \\a T or its members contain\n> - * ControlList or ControlInfoMap.\n> - *\n> - * \\return The deserialized object\n> - */\n> -\n> -/**\n> - * \\fn template IPADataSerializer::deserialize(\n> - * std::vector::const_iterator dataBegin,\n> - * std::vector::const_iterator dataEnd,\n> - * std::vector::const_iterator fdsBegin,\n> - * std::vector::const_iterator fdsEnd,\n> - * ControlSerializer *cs = nullptr)\n> - * \\brief Deserialize byte vector and fd vector into an object\n> - * \\tparam T Type of object to deserialize to\n> - * \\param[in] dataBegin Begin iterator of byte vector to deserialize from\n> - * \\param[in] dataEnd End iterator of byte vector to deserialize from\n> - * \\param[in] fdsBegin Begin iterator of fd vector to deserialize from\n> - * \\param[in] fdsEnd End iterator of fd vector to deserialize from\n> - * \\param[in] cs ControlSerializer\n> - *\n> - * This version of deserialize() (or the vector version) must be used if\n> - * the object type \\a T or its members contain SharedFD.\n> - *\n> * \\a cs is only necessary if the object type \\a T or its members contain\n> * ControlList or ControlInfoMap.\n> *\n> @@ -202,37 +99,11 @@ IPADataSerializer::serialize(const type &data, \\\n> } \\\n> \\\n> template<> \\\n> -type IPADataSerializer::deserialize(std::vector::const_iterator dataBegin, \\\n> - std::vector::const_iterator dataEnd, \\\n> - [[maybe_unused]] ControlSerializer *cs) \\\n> +std::optional IPADataSerializer::deserialize(SeriReader &reader, \\\n> + [[maybe_unused]] ControlSerializer *cs) \\\n> { \\\n> - return readPOD(dataBegin, 0, dataEnd); \\\n> + return reader.read(); \\\n> } \\\n> - \\\n> -template<> \\\n> -type IPADataSerializer::deserialize(const std::vector &data, \\\n> - ControlSerializer *cs) \\\n> -{ \\\n> - return deserialize(data.cbegin(), data.end(), cs); \\\n> -} \\\n> - \\\n> -template<> \\\n> -type IPADataSerializer::deserialize(const std::vector &data, \\\n> - [[maybe_unused]] const std::vector &fds, \\\n> - ControlSerializer *cs) \\\n> -{ \\\n> - return deserialize(data.cbegin(), data.end(), cs); \\\n> -} \\\n> - \\\n> -template<> \\\n> -type IPADataSerializer::deserialize(std::vector::const_iterator dataBegin, \\\n> - std::vector::const_iterator dataEnd, \\\n> - [[maybe_unused]] std::vector::const_iterator fdsBegin, \\\n> - [[maybe_unused]] std::vector::const_iterator fdsEnd, \\\n> - ControlSerializer *cs) \\\n> -{ \\\n> - return deserialize(dataBegin, dataEnd, cs); \\\n> -}\n> \n> DEFINE_POD_SERIALIZER(bool)\n> DEFINE_POD_SERIALIZER(uint8_t)\n> @@ -256,44 +127,29 @@ std::tuple, std::vector>\n> IPADataSerializer::serialize(const std::string &data,\n> [[maybe_unused]] ControlSerializer *cs)\n> {\n> - return { { data.cbegin(), data.end() }, {} };\n> -}\n> + std::vector dataVec;\n> \n> -template<>\n> -std::string\n> -IPADataSerializer::deserialize(const std::vector &data,\n> - [[maybe_unused]] ControlSerializer *cs)\n> -{\n> - return { data.cbegin(), data.cend() };\n> -}\n> + ASSERT(data.size() <= std::numeric_limits::max());\n> + appendPOD(dataVec, data.size());\n> + dataVec.insert(dataVec.end(), data.c_str(), data.c_str() + data.size());\n> \n> -template<>\n> -std::string\n> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - [[maybe_unused]] ControlSerializer *cs)\n> -{\n> - return { dataBegin, dataEnd };\n> + return { dataVec, {} };\n> }\n> \n> template<>\n> -std::string\n> -IPADataSerializer::deserialize(const std::vector &data,\n> - [[maybe_unused]] const std::vector &fds,\n> +std::optional\n> +IPADataSerializer::deserialize(SeriReader &reader,\n> [[maybe_unused]] ControlSerializer *cs)\n> {\n> - return { data.cbegin(), data.cend() };\n> -}\n> + uint32_t length;\n> + if (!reader.read(length))\n> + return {};\n> \n> -template<>\n> -std::string\n> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> - [[maybe_unused]] ControlSerializer *cs)\n> -{\n> - return { dataBegin, dataEnd };\n> + const auto *p = reader.consume(length);\n> + if (!p)\n> + return {};\n> +\n> + return { { reinterpret_cast(p), std::size_t(length) } };\n> }\n> \n> /*\n> @@ -356,73 +212,43 @@ IPADataSerializer::serialize(const ControlList &data, ControlSerial\n> }\n> \n> template<>\n> -ControlList\n> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> +std::optional\n> +IPADataSerializer::deserialize(SeriReader &reader,\n> ControlSerializer *cs)\n> {\n> if (!cs)\n> LOG(IPADataSerializer, Fatal)\n> << \"ControlSerializer not provided for deserialization of ControlList\";\n> \n> - if (std::distance(dataBegin, dataEnd) < 8)\n> - return {};\n> -\n> - uint32_t infoDataSize = readPOD(dataBegin, 0, dataEnd);\n> - uint32_t listDataSize = readPOD(dataBegin, 4, dataEnd);\n> -\n> - std::vector::const_iterator it = dataBegin + 8;\n> -\n> - if (infoDataSize + listDataSize < infoDataSize ||\n> - static_cast(std::distance(it, dataEnd)) < infoDataSize + listDataSize)\n> + uint32_t infoDataSize, listDataSize;\n> + if (!reader.read(infoDataSize, listDataSize))\n> return {};\n> \n> if (infoDataSize > 0) {\n> - ByteStreamBuffer buffer(&*it, infoDataSize);\n> + const auto *p = reader.consume(infoDataSize);\n> + if (!p)\n> + return {};\n> +\n> + ByteStreamBuffer buffer(reinterpret_cast(p), infoDataSize);\n\nThat is cool.\n\n> ControlInfoMap map = cs->deserialize(buffer);\n> /* It's fine if map is empty. */\n> if (buffer.overflow()) {\n> LOG(IPADataSerializer, Error)\n> << \"Failed to deserialize ControlLists's ControlInfoMap: buffer overflow\";\n> - return ControlList();\n> + return {};\n> }\n> }\n> \n> - it += infoDataSize;\n> - ByteStreamBuffer buffer(&*it, listDataSize);\n> + const auto *p = reader.consume(listDataSize);\n> +\n> + ByteStreamBuffer buffer(reinterpret_cast(p), listDataSize);\n> ControlList list = cs->deserialize(buffer);\n> - if (buffer.overflow())\n> + if (buffer.overflow()) {\n> LOG(IPADataSerializer, Error) << \"Failed to deserialize ControlList: buffer overflow\";\n> + return {};\n> + }\n> \n> - return list;\n> -}\n> -\n> -template<>\n> -ControlList\n> -IPADataSerializer::deserialize(const std::vector &data,\n> - ControlSerializer *cs)\n> -{\n> - return deserialize(data.cbegin(), data.end(), cs);\n> -}\n> -\n> -template<>\n> -ControlList\n> -IPADataSerializer::deserialize(const std::vector &data,\n> - [[maybe_unused]] const std::vector &fds,\n> - ControlSerializer *cs)\n> -{\n> - return deserialize(data.cbegin(), data.end(), cs);\n> -}\n> -\n> -template<>\n> -ControlList\n> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> - ControlSerializer *cs)\n> -{\n> - return deserialize(dataBegin, dataEnd, cs);\n> + return std::move(list);\n> }\n> \n> /*\n> @@ -458,57 +284,28 @@ IPADataSerializer::serialize(const ControlInfoMap &map,\n> }\n> \n> template<>\n> -ControlInfoMap\n> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> +std::optional\n> +IPADataSerializer::deserialize(SeriReader &reader,\n> ControlSerializer *cs)\n> {\n> if (!cs)\n> LOG(IPADataSerializer, Fatal)\n> << \"ControlSerializer not provided for deserialization of ControlInfoMap\";\n> \n> - if (std::distance(dataBegin, dataEnd) < 4)\n> + uint32_t infoDataSize;\n> + if (!reader.read(infoDataSize))\n> return {};\n> \n> - uint32_t infoDataSize = readPOD(dataBegin, 0, dataEnd);\n> -\n> - std::vector::const_iterator it = dataBegin + 4;\n> -\n> - if (static_cast(std::distance(it, dataEnd)) < infoDataSize)\n> + const auto *p = reader.consume(infoDataSize);\n> + if (!p)\n> return {};\n> \n> - ByteStreamBuffer buffer(&*it, infoDataSize);\n> + ByteStreamBuffer buffer(reinterpret_cast(p), infoDataSize);\n> ControlInfoMap map = cs->deserialize(buffer);\n> + if (buffer.overflow())\n> + return {};\n> \n> - return map;\n> -}\n> -\n> -template<>\n> -ControlInfoMap\n> -IPADataSerializer::deserialize(const std::vector &data,\n> - ControlSerializer *cs)\n> -{\n> - return deserialize(data.cbegin(), data.end(), cs);\n> -}\n> -\n> -template<>\n> -ControlInfoMap\n> -IPADataSerializer::deserialize(const std::vector &data,\n> - [[maybe_unused]] const std::vector &fds,\n> - ControlSerializer *cs)\n> -{\n> - return deserialize(data.cbegin(), data.end(), cs);\n> -}\n> -\n> -template<>\n> -ControlInfoMap\n> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> - ControlSerializer *cs)\n> -{\n> - return deserialize(dataBegin, dataEnd, cs);\n> + return std::move(map);\n> }\n> \n> /*\n> @@ -542,27 +339,23 @@ IPADataSerializer::serialize(const SharedFD &data,\n> }\n> \n> template<>\n> -SharedFD IPADataSerializer::deserialize([[maybe_unused]] std::vector::const_iterator dataBegin,\n> - [[maybe_unused]] std::vector::const_iterator dataEnd,\n> - std::vector::const_iterator fdsBegin,\n> - std::vector::const_iterator fdsEnd,\n> - [[maybe_unused]] ControlSerializer *cs)\n> +std::optional\n> +IPADataSerializer::deserialize(SeriReader &reader,\n> + [[maybe_unused]] ControlSerializer *cs)\n> {\n> - ASSERT(std::distance(dataBegin, dataEnd) >= 4);\n> + uint32_t valid;\n> \n> - uint32_t valid = readPOD(dataBegin, 0, dataEnd);\n> + if (!reader.read(valid))\n> + return {};\n> \n> - ASSERT(!(valid && std::distance(fdsBegin, fdsEnd) < 1));\n> + if (!valid)\n> + return SharedFD{};\n> \n> - return valid ? *fdsBegin : SharedFD();\n> -}\n> + const auto *fd = reader.nextFd();\n> + if (!fd)\n> + return {};\n> \n> -template<>\n> -SharedFD IPADataSerializer::deserialize(const std::vector &data,\n> - const std::vector &fds,\n> - [[maybe_unused]] ControlSerializer *cs)\n> -{\n> - return deserialize(data.cbegin(), data.end(), fds.cbegin(), fds.end());\n> + return *fd;\n> }\n> \n> /*\n> @@ -594,30 +387,19 @@ IPADataSerializer::serialize(const FrameBuffer::Plane &data,\n> }\n> \n> template<>\n> -FrameBuffer::Plane\n> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - std::vector::const_iterator fdsBegin,\n> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> +std::optional\n> +IPADataSerializer::deserialize(SeriReader &reader,\n> [[maybe_unused]] ControlSerializer *cs)\n> {\n> - FrameBuffer::Plane ret;\n> -\n> - ret.fd = IPADataSerializer::deserialize(dataBegin, dataBegin + 4,\n> - fdsBegin, fdsBegin + 1);\n> - ret.offset = readPOD(dataBegin, 4, dataEnd);\n> - ret.length = readPOD(dataBegin, 8, dataEnd);\n> + auto fd = IPADataSerializer::deserialize(reader);\n> + if (!fd)\n> + return {};\n> \n> - return ret;\n> -}\n> + uint32_t offset, length;\n> + if (!reader.read(offset, length))\n> + return {};\n> \n> -template<>\n> -FrameBuffer::Plane\n> -IPADataSerializer::deserialize(const std::vector &data,\n> - const std::vector &fds,\n> - ControlSerializer *cs)\n> -{\n> - return deserialize(data.cbegin(), data.end(), fds.cbegin(), fds.end(), cs);\n> + return { { std::move(*fd), offset, length } };\n> }\n> \n> #endif /* __DOXYGEN__ */\n> diff --git a/src/libcamera/ipc_pipe.cpp b/src/libcamera/ipc_pipe.cpp\n> index 548299d05..71d0c2cb4 100644\n> --- a/src/libcamera/ipc_pipe.cpp\n> +++ b/src/libcamera/ipc_pipe.cpp\n> @@ -148,6 +148,11 @@ IPCUnixSocket::Payload IPCMessage::payload() const\n> * \\brief Returns a const reference to the vector containing file descriptors\n> */\n> \n> +/**\n> + * \\fn IPCMessage::reader() const\n> + * \\brief Returns a `SeriReader` instance for parsing the message\n> + */\n> +\n> /**\n> * \\class IPCPipe\n> * \\brief IPC message pipe for IPA isolation\n> diff --git a/test/ipc/unixsocket_ipc.cpp b/test/ipc/unixsocket_ipc.cpp\n> index df7d9c2b4..c0e174ba9 100644\n> --- a/test/ipc/unixsocket_ipc.cpp\n> +++ b/test/ipc/unixsocket_ipc.cpp\n> @@ -102,7 +102,14 @@ private:\n> }\n> \n> case CmdSetAsync: {\n> - value_ = IPADataSerializer::deserialize(ipcMessage.data());\n> + SeriReader reader = ipcMessage.reader();\n> + auto value = IPADataSerializer::deserialize(reader);\n> + if (!value) {\n> + cerr << \"Failed to deserialize value\" << endl;\n> + stop(-ENODATA);\n> + } else {\n> + value_ = *value;\n> + }\n> break;\n> }\n> }\n> @@ -155,7 +162,14 @@ protected:\n> return ret;\n> }\n> \n> - return IPADataSerializer::deserialize(buf.data());\n> + SeriReader reader = buf.reader();\n> + auto value = IPADataSerializer::deserialize(reader);\n> + if (!value) {\n> + cerr << \"Failed to deserialize value\" << endl;\n> + return -ENODATA;\n> + }\n> +\n> + return *value;\n> }\n> \n> int exit()\n> diff --git a/test/serialization/generated_serializer/generated_serializer_test.cpp b/test/serialization/generated_serializer/generated_serializer_test.cpp\n> index dd6968850..0640e741d 100644\n> --- a/test/serialization/generated_serializer/generated_serializer_test.cpp\n> +++ b/test/serialization/generated_serializer/generated_serializer_test.cpp\n> @@ -43,7 +43,7 @@ if (struct1.field != struct2.field) { \\\n> }\n> \n> \n> - ipa::test::TestStruct t, u;\n> + ipa::test::TestStruct t;\n> \n> t.m = {\n> { \"a\", \"z\" },\n> @@ -71,8 +71,13 @@ if (struct1.field != struct2.field) { \\\n> \n> std::tie(serialized, ignore) =\n> IPADataSerializer::serialize(t);\n> + SeriReader reader(serialized);\n> \n> - u = IPADataSerializer::deserialize(serialized);\n> + auto optu = IPADataSerializer::deserialize(reader);\n> + if (!optu)\n> + return TestFail;\n> +\n> + auto &u = *optu;\n> \n> if (!equals(t.m, u.m))\n> return TestFail;\n> @@ -91,12 +96,16 @@ if (struct1.field != struct2.field) { \\\n> \n> /* Test vector of generated structs */\n> std::vector v = { t, u };\n> - std::vector w;\n> \n> std::tie(serialized, ignore) =\n> IPADataSerializer>::serialize(v);\n> + reader = SeriReader(serialized);\n> +\n> + auto optw = IPADataSerializer>::deserialize(reader);\n> + if (!optw)\n> + return TestFail;\n> \n> - w = IPADataSerializer>::deserialize(serialized);\n> + auto &w = *optw;\n> \n> if (!equals(v[0].m, w[0].m) ||\n> !equals(v[1].m, w[1].m))\n> diff --git a/test/serialization/ipa_data_serializer_test.cpp b/test/serialization/ipa_data_serializer_test.cpp\n> index afea93a6c..3873808a6 100644\n> --- a/test/serialization/ipa_data_serializer_test.cpp\n> +++ b/test/serialization/ipa_data_serializer_test.cpp\n> @@ -52,7 +52,9 @@ int testPodSerdes(T in)\n> std::vector fds;\n> \n> std::tie(buf, fds) = IPADataSerializer::serialize(in);\n> - T out = IPADataSerializer::deserialize(buf, fds);\n> + SeriReader reader(buf, fds);\n> +\n> + auto out = IPADataSerializer::deserialize(reader);\n> if (in == out)\n> return TestPass;\n> \n> @@ -71,7 +73,9 @@ int testVectorSerdes(const std::vector &in,\n> std::vector fds;\n> \n> std::tie(buf, fds) = IPADataSerializer>::serialize(in, cs);\n> - std::vector out = IPADataSerializer>::deserialize(buf, fds, cs);\n> + SeriReader reader(buf, fds);\n> +\n> + auto out = IPADataSerializer>::deserialize(reader, cs);\n> if (in == out)\n> return TestPass;\n> \n> @@ -91,7 +95,9 @@ int testMapSerdes(const std::map &in,\n> std::vector fds;\n> \n> std::tie(buf, fds) = IPADataSerializer>::serialize(in, cs);\n> - std::map out = IPADataSerializer>::deserialize(buf, fds, cs);\n> + SeriReader reader(buf, fds);\n> +\n> + auto out = IPADataSerializer>::deserialize(reader, cs);\n> if (in == out)\n> return TestPass;\n> \n> @@ -171,17 +177,27 @@ private:\n> std::tie(listBuf, std::ignore) =\n> IPADataSerializer::serialize(list, &cs);\n> \n> - const ControlInfoMap infoMapOut =\n> - IPADataSerializer::deserialize(infoMapBuf, &cs);\n> + SeriReader listReader(listBuf);\n> + SeriReader infoMapReader(infoMapBuf);\n> +\n> + auto infoMapOut = IPADataSerializer::deserialize(infoMapReader, &cs);\n> + if (!infoMapOut) {\n> + cerr << \"`ControlInfoMap` cannot be deserialized\" << endl;\n> + return TestFail;\n> + }\n> \n> - ControlList listOut = IPADataSerializer::deserialize(listBuf, &cs);\n> + auto listOut = IPADataSerializer::deserialize(listReader, &cs);\n> + if (!listOut) {\n> + cerr << \"`ControlList` cannot be deserialized\" << endl;\n> + return TestFail;\n> + }\n> \n> - if (!SerializationTest::equals(infoMap, infoMapOut)) {\n> + if (!SerializationTest::equals(infoMap, *infoMapOut)) {\n> cerr << \"Deserialized map doesn't match original\" << endl;\n> return TestFail;\n> }\n> \n> - if (!SerializationTest::equals(list, listOut)) {\n> + if (!SerializationTest::equals(list, *listOut)) {\n> cerr << \"Deserialized list doesn't match original\" << endl;\n> return TestFail;\n> }\n> diff --git a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> index 9a3aadbd2..843260b4b 100644\n> --- a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> +++ b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> @@ -107,13 +107,13 @@ namespace {{ns}} {\n> {% if interface_event.methods|length > 0 %}\n> void {{proxy_name}}::recvMessage(const IPCMessage &data)\n> {\n> - size_t dataSize = data.data().size();\n> + SeriReader reader = data.reader();\n> {{cmd_event_enum_name}} _cmd = static_cast<{{cmd_event_enum_name}}>(data.header().cmd);\n> \n> switch (_cmd) {\n> {%- for method in interface_event.methods %}\n> case {{cmd_event_enum_name}}::{{method.mojom_name|cap}}: {\n> - {{method.mojom_name}}IPC(data.data().cbegin(), dataSize, data.fds());\n> + {{method.mojom_name}}IPC(reader);\n> break;\n> }\n> {%- endfor %}\n> @@ -211,18 +211,23 @@ void {{proxy_name}}::recvMessage(const IPCMessage &data)\n> return;\n> {%- endif %}\n> }\n> +{% if has_output %}\n> + SeriReader _outputReader = _ipcOutputBuf.reader();\n> +{% endif -%}\n> {% if method|method_return_value != \"void\" %}\n> - {{method|method_return_value}} _retValue = IPADataSerializer<{{method|method_return_value}}>::deserialize(_ipcOutputBuf.data(), 0);\n> -\n> -{{proxy_funcs.deserialize_call(method|method_param_outputs, '_ipcOutputBuf.data()', '_ipcOutputBuf.fds()', init_offset = method|method_return_value|byte_width|int)}}\n> + auto _retValue = IPADataSerializer<{{method|method_return_value}}>::deserialize(_outputReader);\n> + ASSERT(_retValue);\n> +{% endif -%}\n> \n> - return _retValue;\n> +{% if has_output %}\n\nI think this should be {% if method|method_param_outputs|length > 0 %} because\nhas_output also covers `method|method_return_value != \"void\"`, so you'd be\ndeserializing the single _retValue twice.\n\nOr maybe an ASSERT(_outputReader.empty()) might be valuable after\nASSERT(_retValue) above.\n\n> + {{proxy_funcs.deserialize_call(method|method_param_outputs, '_outputReader')}}\n> + ASSERT(_outputReader.empty());\n> +{% endif -%}\n> \n> -{% elif method|method_param_outputs|length > 0 %}\n> -{{proxy_funcs.deserialize_call(method|method_param_outputs, '_ipcOutputBuf.data()', '_ipcOutputBuf.fds()')}}\n> +{% if method|method_return_value != \"void\" %}\n> + return std::move(*_retValue);\n> {% endif -%}\n> }\n> -\n> {% endfor %}\n> \n> {% for method in interface_event.methods %}\n> @@ -232,12 +237,9 @@ void {{proxy_name}}::recvMessage(const IPCMessage &data)\n> {{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n> }\n> \n> -void {{proxy_name}}::{{method.mojom_name}}IPC(\n> - [[maybe_unused]] std::vector::const_iterator data,\n> - [[maybe_unused]] size_t dataSize,\n> - [[maybe_unused]] const std::vector &fds)\n> +void {{proxy_name}}::{{method.mojom_name}}IPC([[maybe_unused]] SeriReader &reader)\n> {\n> -{{proxy_funcs.deserialize_call(method.parameters, 'data', 'fds', false, true, true, 'dataSize')}}\n> + {{proxy_funcs.deserialize_call(method.parameters, 'reader', false, true)}}\n\nI think this is one level too much indentation? Same for the other places you\nadd this call.\n\n> {{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n> }\n> {% endfor %}\n> diff --git a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> index a0312a7c1..945a4ded9 100644\n> --- a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> +++ b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> @@ -53,10 +53,7 @@ private:\n> {% endfor %}\n> {% for method in interface_event.methods %}\n> {{proxy_funcs.func_sig(proxy_name, method, \"Thread\", false)|indent(8, true)}};\n> - void {{method.mojom_name}}IPC(\n> - std::vector::const_iterator data,\n> - size_t dataSize,\n> - const std::vector &fds);\n> + void {{method.mojom_name}}IPC(SeriReader &reader);\n> {% endfor %}\n> \n> /* Helper class to invoke async functions in another thread. */\n> diff --git a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> index 1f990d3f9..de6378be0 100644\n> --- a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> +++ b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> @@ -71,6 +71,7 @@ public:\n> }\n> \n> IPCMessage _ipcMessage(_message);\n> + SeriReader _reader = _ipcMessage.reader();\n> \n> {{cmd_enum_name}} _cmd = static_cast<{{cmd_enum_name}}>(_ipcMessage.header().cmd);\n> \n> @@ -85,7 +86,9 @@ public:\n> {%- if method.mojom_name == \"configure\" %}\n> controlSerializer_.reset();\n> {%- endif %}\n> - {{proxy_funcs.deserialize_call(method|method_param_inputs, '_ipcMessage.data()', '_ipcMessage.fds()', false, true)|indent(16, true)}}\n> + {{proxy_funcs.deserialize_call(method|method_param_inputs, '_reader', false, true)|indent(16, true)}}\n> + ASSERT(_reader.empty());\n> +\n> {% for param in method|method_param_outputs %}\n> {{param|name}} {{param.mojom_name}};\n> {% endfor %}\n> diff --git a/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl b/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> index 01e2567ca..b6835ca35 100644\n> --- a/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> +++ b/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> @@ -64,15 +64,6 @@\n> );\n> {%- endfor %}\n> \n> -{%- if params|length > 1 %}\n> -{%- for param in params %}\n> - appendPOD({{buf}}, {{param.mojom_name}}Buf.size());\n> -{%- if param|has_fd %}\n> - appendPOD({{buf}}, {{param.mojom_name}}Fds.size());\n> -{%- endif %}\n> -{%- endfor %}\n> -{%- endif %}\n> -\n> {%- for param in params %}\n> {{buf}}.insert({{buf}}.end(), {{param.mojom_name}}Buf.begin(), {{param.mojom_name}}Buf.end());\n> {%- endfor %}\n> @@ -84,104 +75,28 @@\n> {%- endfor %}\n> {%- endmacro -%}\n> \n> -\n> -{#\n> - # \\brief Deserialize a single object from data buffer and fd vector\n> - #\n> - # \\param pointer If true, deserialize the object into a dereferenced pointer\n> - # \\param iter If true, treat \\a buf as an iterator instead of a vector\n> - # \\param data_size Variable that holds the size of the vector referenced by \\a buf\n> - #\n> - # Generate code to deserialize a single object, as specified in \\a param,\n> - # from \\a buf data buffer and \\a fds fd vector.\n> - # This code is meant to be used by macro deserialize_call.\n> - #}\n> -{%- macro deserialize_param(param, pointer, loop, buf, fds, iter, data_size) -%}\n> -{{\"*\" if pointer}}{{param.mojom_name}} =\n> -IPADataSerializer<{{param|name_full}}>::deserialize(\n> - {{buf}}{{- \".cbegin()\" if not iter}} + {{param.mojom_name}}Start,\n> -{%- if loop.last and not iter %}\n> - {{buf}}.cend()\n> -{%- elif not iter %}\n> - {{buf}}.cbegin() + {{param.mojom_name}}Start + {{param.mojom_name}}BufSize\n> -{%- elif iter and loop.length == 1 %}\n> - {{buf}} + {{data_size}}\n> -{%- else %}\n> - {{buf}} + {{param.mojom_name}}Start + {{param.mojom_name}}BufSize\n> -{%- endif -%}\n> -{{- \",\" if param|has_fd}}\n> -{%- if param|has_fd %}\n> - {{fds}}.cbegin() + {{param.mojom_name}}FdStart,\n> -{%- if loop.last %}\n> - {{fds}}.cend()\n> -{%- else %}\n> - {{fds}}.cbegin() + {{param.mojom_name}}FdStart + {{param.mojom_name}}FdsSize\n> -{%- endif -%}\n> -{%- endif -%}\n> -{{- \",\" if param|needs_control_serializer}}\n> -{%- if param|needs_control_serializer %}\n> - &controlSerializer_\n> -{%- endif -%}\n> -);\n> -{%- endmacro -%}\n> -\n> -\n> {#\n> - # \\brief Deserialize multiple objects from data buffer and fd vector\n> + # \\brief Deserialize multiple objects\n> #\n> # \\param pointer If true, deserialize objects into pointers, and adds a null check.\n> # \\param declare If true, declare the objects in addition to deserialization.\n> - # \\param iter if true, treat \\a buf as an iterator instead of a vector\n> - # \\param data_size Variable that holds the size of the vector referenced by \\a buf\n> - #\n> - # Generate code to deserialize multiple objects, as specified in \\a params\n> - # (which are the parameters to some function), from \\a buf data buffer and\n> - # \\a fds fd vector.\n> - # This code is meant to be used by the proxy, for deserializing after IPC calls.\n> #\n> - # \\todo Avoid intermediate vectors\n> + # Generate code to deserialize multiple objects, as specified in \\a params,\n> + # from \\a reader. This code is meant to be used by the proxy, for deserializing\n> + # after IPC calls.\n> #}\n> -{%- macro deserialize_call(params, buf, fds, pointer = true, declare = false, iter = false, data_size = '', init_offset = 0) -%}\n> -{% set ns = namespace(size_offset = init_offset) %}\n> -{%- if params|length > 1 %}\n> +{%- macro deserialize_call(params, reader, pointer = true, declare = false) -%}\n> {%- for param in params %}\n> - [[maybe_unused]] const size_t {{param.mojom_name}}BufSize = readPOD({{buf}}, {{ns.size_offset}}\n> -{%- if iter -%}\n> -, {{buf}} + {{data_size}}\n> -{%- endif -%}\n> -);\n> - {%- set ns.size_offset = ns.size_offset + 4 %}\n> -{%- if param|has_fd %}\n> - [[maybe_unused]] const size_t {{param.mojom_name}}FdsSize = readPOD({{buf}}, {{ns.size_offset}}\n> -{%- if iter -%}\n> -, {{buf}} + {{data_size}}\n> -{%- endif -%}\n> -);\n> - {%- set ns.size_offset = ns.size_offset + 4 %}\n> -{%- endif %}\n> -{%- endfor %}\n> -{%- endif %}\n> -{% for param in params %}\n> -{%- if loop.first %}\n> - const size_t {{param.mojom_name}}Start = {{ns.size_offset}};\n> -{%- else %}\n> - const size_t {{param.mojom_name}}Start = {{loop.previtem.mojom_name}}Start + {{loop.previtem.mojom_name}}BufSize;\n> -{%- endif %}\n> -{%- endfor %}\n> -{% for param in params|with_fds %}\n> -{%- if loop.first %}\n> - const size_t {{param.mojom_name}}FdStart = 0;\n> + auto param_{{param.mojom_name}} = IPADataSerializer<{{param|name_full}}>::deserialize({{reader}}, &controlSerializer_);\n> + ASSERT(param_{{param.mojom_name}});\n> +\n> +{%- if pointer %}\n> + if ({{param.mojom_name}})\n> + *{{param.mojom_name}} = std::move(*param_{{param.mojom_name}});\n> +{%- elif declare %}\n> + {{param|name}} &{{param.mojom_name}} = *param_{{param.mojom_name}};\n> {%- else %}\n> - const size_t {{param.mojom_name}}FdStart = {{loop.previtem.mojom_name}}FdStart + {{loop.previtem.mojom_name}}FdsSize;\n> + {{param.mojom_name}} = std::move(*param_{{param.mojom_name}});\n> {%- endif %}\n> -{%- endfor %}\n> -{% for param in params %}\n> - {%- if pointer %}\n> - if ({{param.mojom_name}}) {\n> -{{deserialize_param(param, pointer, loop, buf, fds, iter, data_size)|indent(16, True)}}\n> - }\n> - {%- else %}\n> - {{param|name + \" \" if declare}}{{deserialize_param(param, pointer, loop, buf, fds, iter, data_size)|indent(8)}}\n> - {%- endif %}\n\nThis is beautiful.\n\n\nWhat an amazing patch. I'm excited for v2.\n\nThanks,\n\nPaul\n\n\n> {% endfor %}\n> {%- endmacro -%}\n> diff --git a/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl b/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl\n> index e316dd88a..9e9dd0ca6 100644\n> --- a/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl\n> +++ b/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl\n> @@ -2,22 +2,6 @@\n> # SPDX-License-Identifier: LGPL-2.1-or-later\n> # Copyright (C) 2020, Google Inc.\n> -#}\n> -{#\n> - # \\brief Verify that there is enough bytes to deserialize\n> - #\n> - # Generate code that verifies that \\a size is not greater than \\a dataSize.\n> - # Otherwise log an error with \\a name and \\a typename.\n> - #}\n> -{%- macro check_data_size(size, dataSize, name, typename) %}\n> - if ({{dataSize}} < {{size}}) {\n> - LOG(IPADataSerializer, Error)\n> - << \"Failed to deserialize \" << \"{{name}}\"\n> - << \": not enough {{typename}}, expected \"\n> - << ({{size}}) << \", got \" << ({{dataSize}});\n> - return ret;\n> - }\n> -{%- endmacro %}\n> -\n> \n> {#\n> # \\brief Serialize a field into return vector\n> @@ -42,15 +26,10 @@\n> retData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> retFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> {%- elif field|is_controls %}\n> - if (data.{{field.mojom_name}}.size() > 0) {\n> - std::vector {{field.mojom_name}};\n> - std::tie({{field.mojom_name}}, std::ignore) =\n> - IPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> - appendPOD(retData, {{field.mojom_name}}.size());\n> - retData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> - } else {\n> - appendPOD(retData, 0);\n> - }\n> + std::vector {{field.mojom_name}};\n> + std::tie({{field.mojom_name}}, std::ignore) =\n> + IPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> + retData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> {%- elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> std::vector {{field.mojom_name}};\n> {%- if field|has_fd %}\n> @@ -65,10 +44,6 @@\n> IPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> {%- else %}\n> IPADataSerializer<{{field|name_full}}>::serialize(data.{{field.mojom_name}}, cs);\n> - {%- endif %}\n> - appendPOD(retData, {{field.mojom_name}}.size());\n> - {%- if field|has_fd %}\n> - appendPOD(retData, {{field.mojom_name}}Fds.size());\n> {%- endif %}\n> retData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> {%- if field|has_fd %}\n> @@ -79,89 +54,6 @@\n> {%- endif %}\n> {%- endmacro %}\n> \n> -\n> -{#\n> - # \\brief Deserialize a field into return struct\n> - #\n> - # Generate code to deserialize \\a field into object ret.\n> - # This code is meant to be used by the IPADataSerializer specialization.\n> - #}\n> -{%- macro deserializer_field(field, loop) %}\n> -{% if field|is_pod or field|is_enum %}\n> - {%- set field_size = (field|bit_width|int / 8)|int %}\n> - {{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> - ret.{{field.mojom_name}} = IPADataSerializer<{{field|name_full}}>::deserialize(m, m + {{field_size}});\n> - {%- if not loop.last %}\n> - m += {{field_size}};\n> - dataSize -= {{field_size}};\n> - {%- endif %}\n> -{% elif field|is_fd %}\n> - {%- set field_size = 4 %}\n> - {{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> - ret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field_size}}, n, n + 1, cs);\n> - {%- if not loop.last %}\n> - m += {{field_size}};\n> - dataSize -= {{field_size}};\n> - n += ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> - fdsSize -= ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> - {%- endif %}\n> -{% elif field|is_controls %}\n> - {%- set field_size = 4 %}\n> - {{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> - const size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> - m += {{field_size}};\n> - dataSize -= {{field_size}};\n> - {%- set field_size = field.mojom_name + 'Size' -%}\n> - {{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> - if ({{field.mojom_name}}Size > 0)\n> - ret.{{field.mojom_name}} =\n> - IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> - {%- if not loop.last %}\n> - m += {{field_size}};\n> - dataSize -= {{field_size}};\n> - {%- endif %}\n> -{% elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> - {%- set field_size = 4 %}\n> - {{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> - const size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> - m += {{field_size}};\n> - dataSize -= {{field_size}};\n> - {%- if field|has_fd %}\n> - {%- set field_size = 4 %}\n> - {{- check_data_size(field_size, 'dataSize', field.mojom_name + 'FdsSize', 'data')}}\n> - const size_t {{field.mojom_name}}FdsSize = readPOD(m, 0, dataEnd);\n> - m += {{field_size}};\n> - dataSize -= {{field_size}};\n> - {{- check_data_size(field.mojom_name + 'FdsSize', 'fdsSize', field.mojom_name, 'fds')}}\n> - {%- endif %}\n> - {%- set field_size = field.mojom_name + 'Size' -%}\n> - {{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> - ret.{{field.mojom_name}} =\n> - {%- if field|is_str %}\n> - IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size);\n> - {%- elif field|has_fd and (field|is_array or field|is_map) %}\n> - IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> - {%- elif field|has_fd and (not (field|is_array or field|is_map)) %}\n> - IPADataSerializer<{{field|name_full}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> - {%- elif (not field|has_fd) and (field|is_array or field|is_map) %}\n> - IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> - {%- else %}\n> - IPADataSerializer<{{field|name_full}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> - {%- endif %}\n> - {%- if not loop.last %}\n> - m += {{field_size}};\n> - dataSize -= {{field_size}};\n> - {%- if field|has_fd %}\n> - n += {{field.mojom_name}}FdsSize;\n> - fdsSize -= {{field.mojom_name}}FdsSize;\n> - {%- endif %}\n> - {%- endif %}\n> -{% else %}\n> - /* Unknown deserialization for {{field.mojom_name}}. */\n> -{%- endif %}\n> -{%- endmacro %}\n> -\n> -\n> {#\n> # \\brief Serialize a struct\n> #\n> @@ -194,126 +86,30 @@\n> \n> \n> {#\n> - # \\brief Deserialize a struct that has fds\n> + # \\brief Deserialize a struct\n> #\n> - # Generate code for IPADataSerializer specialization, for deserializing\n> - # \\a struct, in the case that \\a struct has file descriptors.\n> + # Generate code for IPADataSerializer specialization, for deserializing \\a struct.\n> #}\n> -{%- macro deserializer_fd(struct) %}\n> - static {{struct|name_full}}\n> - deserialize(std::vector &data,\n> - std::vector &fds,\n> -{%- if struct|needs_control_serializer %}\n> - ControlSerializer *cs)\n> -{%- else %}\n> - ControlSerializer *cs = nullptr)\n> -{%- endif %}\n> - {\n> - return IPADataSerializer<{{struct|name_full}}>::deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n> - }\n> -\n> +{%- macro deserializer(struct) %}\n> {# \\todo Don't inline this function #}\n> - static {{struct|name_full}}\n> - deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - std::vector::const_iterator fdsBegin,\n> - std::vector::const_iterator fdsEnd,\n> + [[nodiscard]] static std::optional<{{struct|name_full}}>\n> + deserialize(SeriReader &reader,\n> {%- if struct|needs_control_serializer %}\n> ControlSerializer *cs)\n> {%- else %}\n> [[maybe_unused]] ControlSerializer *cs = nullptr)\n> {%- endif %}\n> {\n> - {{struct|name_full}} ret;\n> - std::vector::const_iterator m = dataBegin;\n> - std::vector::const_iterator n = fdsBegin;\n> -\n> - size_t dataSize = std::distance(dataBegin, dataEnd);\n> - [[maybe_unused]] size_t fdsSize = std::distance(fdsBegin, fdsEnd);\n> -{%- for field in struct.fields -%}\n> -{{deserializer_field(field, loop)}}\n> +{%- for field in struct.fields %}\n> + auto {{field.mojom_name}} = IPADataSerializer<{{field|name_full}}>::deserialize(reader, cs);\n> + if (!{{field.mojom_name}})\n> + return {};\n> {%- endfor %}\n> - return ret;\n> - }\n> -{%- endmacro %}\n> -\n> -{#\n> - # \\brief Deserialize a struct that has fds, using non-fd\n> - #\n> - # Generate code for IPADataSerializer specialization, for deserializing\n> - # \\a struct, in the case that \\a struct has no file descriptors but requires\n> - # deserializers with file descriptors.\n> - #}\n> -{%- macro deserializer_fd_simple(struct) %}\n> - static {{struct|name_full}}\n> - deserialize(std::vector &data,\n> - [[maybe_unused]] std::vector &fds,\n> - ControlSerializer *cs = nullptr)\n> - {\n> - return IPADataSerializer<{{struct|name_full}}>::deserialize(data.cbegin(), data.cend(), cs);\n> - }\n> -\n> - static {{struct|name_full}}\n> - deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> - [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> - ControlSerializer *cs = nullptr)\n> - {\n> - return IPADataSerializer<{{struct|name_full}}>::deserialize(dataBegin, dataEnd, cs);\n> - }\n> -{%- endmacro %}\n> -\n> -\n> -{#\n> - # \\brief Deserialize a struct that has no fds\n> - #\n> - # Generate code for IPADataSerializer specialization, for deserializing\n> - # \\a struct, in the case that \\a struct does not have file descriptors.\n> - #}\n> -{%- macro deserializer_no_fd(struct) %}\n> - static {{struct|name_full}}\n> - deserialize(std::vector &data,\n> -{%- if struct|needs_control_serializer %}\n> - ControlSerializer *cs)\n> -{%- else %}\n> - ControlSerializer *cs = nullptr)\n> -{%- endif %}\n> - {\n> - return IPADataSerializer<{{struct|name_full}}>::deserialize(data.cbegin(), data.cend(), cs);\n> - }\n> \n> -{# \\todo Don't inline this function #}\n> - static {{struct|name_full}}\n> - deserialize(std::vector::const_iterator dataBegin,\n> - std::vector::const_iterator dataEnd,\n> -{%- if struct|needs_control_serializer %}\n> - ControlSerializer *cs)\n> -{%- else %}\n> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n> -{%- endif %}\n> - {\n> - {{struct|name_full}} ret;\n> - std::vector::const_iterator m = dataBegin;\n> -\n> - size_t dataSize = std::distance(dataBegin, dataEnd);\n> -{%- for field in struct.fields -%}\n> -{{deserializer_field(field, loop)}}\n> + return { {\n> +{%- for field in struct.fields %}\n> + std::move(*{{field.mojom_name}}),\n> {%- endfor %}\n> - return ret;\n> + } };\n> }\n> {%- endmacro %}\n> -\n> -{#\n> - # \\brief Deserialize a struct\n> - #\n> - # Generate code for IPADataSerializer specialization, for deserializing \\a struct.\n> - #}\n> -{%- macro deserializer(struct) %}\n> -{%- if struct|has_fd %}\n> -{{deserializer_fd(struct)}}\n> -{%- else %}\n> -{{deserializer_no_fd(struct)}}\n> -{{deserializer_fd_simple(struct)}}\n> -{%- endif %}\n> -{%- endmacro %}\n> -- \n> 2.49.0\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id EA861BD78E\n\tfor ;\n\tFri, 16 May 2025 17:30:49 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id CD5BB68C92;\n\tFri, 16 May 2025 19:30:48 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B76A068B66\n\tfor ;\n\tFri, 16 May 2025 19:30:46 +0200 (CEST)","from pyrite.rasen.tech (unknown\n\t[IPv6:2a01:cb16:2023:569c:38dc:e618:519:2cda])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id E7EBF1D5;\n\tFri, 16 May 2025 19:30:27 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"fYJRTYa1\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1747416628;\n\tbh=HJc4idSZDmJVgJB4KTRb+8R3B7LPMcO/zB11iK+TK4U=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=fYJRTYa1jVVct8ZZzWT8IyNO/U5zKJv1fLHgzMAaoETd2W/FsvyW6cIpAeyx8wXej\n\t3BePRihCNQ7fiY55cjfBwFIpRpaVvm+lfvd34qeAKiZcCWFQyeEslAs1/gJyhn1uBg\n\tAfbwvlcbjx6KdY7EnTznyzGYYPPq4vhrphknOeGs=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20250515120012.3127231-9-barnabas.pocze@ideasonboard.com>","References":"<20250515120012.3127231-1-barnabas.pocze@ideasonboard.com>\n\t<20250515120012.3127231-9-barnabas.pocze@ideasonboard.com>","Subject":"Re: [RFC PATCH v1 8/8] utils: codegen: ipc: Simplify deserialization","From":"Paul Elder ","Cc":"","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Fri, 16 May 2025 19:30:40 +0200","Message-ID":"<174741664081.476729.1844022348649238166@calcite>","User-Agent":"alot/0.10","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":34269,"web_url":"https://patchwork.libcamera.org/comment/34269/","msgid":"<9b8e5e89-fcf6-4661-a428-3d22c1fa0a37@ideasonboard.com>","date":"2025-05-19T08:51:11","subject":"Re: [RFC PATCH v1 8/8] utils: codegen: ipc: Simplify deserialization","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"Hi\n\n\n2025. 05. 16. 19:30 keltezéssel, Paul Elder írta:\n> Quoting Barnabás Pőcze (2025-05-15 14:00:12)\n>> First, introduce the `SeriReader` type, which is a collection\n>> of bytes and file descriptors, with the appropriate methods to\n>> consume the first couples bytes / file descriptors.\n>>\n>> Then a new method is added to `IPCMessage` that returns an appropriately\n>> constructed `SeriReader` for parsing the message contents.\n>>\n>> Then three of the four `deserialize()` overloads are removed, the\n>> remaining one is converted to have a single `SeriReader` and an\n>> optional `ControlSerializer` as arguments.\n>>\n>> The remaining `deserialize()` function is also changed to return an\n>> `std::optional` to be able to report deserialization failure.\n>>\n>> There is also a more fundamental change in the serialization: previously,\n>> the number of bytes taken up by an item has been written before the serialized\n>> bytes (and conditionally the number of file descriptors) when the item is\n>> serialized as part of a struct, array, map, function parameter list. This\n>> is changed: the number of bytes and file descriptors are *not* serialized\n>> into the final buffer. This affords some simplification of the serialization\n>> related code paths, but most importantly, it greatly simplifies and unifies\n>> how an object is (de)serialized because the deserialization of every object\n>> becomes completely self-contained.\n>>\n>> As a consequence of that, strings now include their lengths as part of the\n>> string serialization, and it is not left to an \"upper\" layer.\n>>\n>> Another consequence is that an \"out parameter\" of a remote function call\n>> must be deserialized if a later out parameter is needed, even if itself\n>> is not. This does not appear to be a great limitation since in all\n>> situations presently none of the out parameters are ignored.\n>>\n>> Finally, the code generation templates are adapted to the above changes.\n>> This allows the simplification of the deserialization templates as now\n>> calling `IPADataSerializer::deserialize(reader, &controlSerializer_)`\n>> is appropriate for any type.\n>>\n>> Bug: https://bugs.libcamera.org/show_bug.cgi?id=269\n>> Signed-off-by: Barnabás Pőcze \n>> ---\n>> .../libcamera/internal/ipa_data_serializer.h | 228 +++--------\n>> include/libcamera/internal/ipc_pipe.h | 3 +\n>> include/libcamera/internal/serialization.h | 91 +++++\n>> src/libcamera/ipa_data_serializer.cpp | 356 ++++--------------\n>> src/libcamera/ipc_pipe.cpp | 5 +\n>> test/ipc/unixsocket_ipc.cpp | 18 +-\n>> .../generated_serializer_test.cpp | 17 +-\n>> .../ipa_data_serializer_test.cpp | 32 +-\n>> .../module_ipa_proxy.cpp.tmpl | 30 +-\n>> .../module_ipa_proxy.h.tmpl | 5 +-\n>> .../module_ipa_proxy_worker.cpp.tmpl | 5 +-\n>> .../libcamera_templates/proxy_functions.tmpl | 113 +-----\n>> .../libcamera_templates/serializer.tmpl | 238 +-----------\n>> 13 files changed, 316 insertions(+), 825 deletions(-)\n>> create mode 100644 include/libcamera/internal/serialization.h\n>>\n>> diff --git a/include/libcamera/internal/ipa_data_serializer.h b/include/libcamera/internal/ipa_data_serializer.h\n>> index 564f59e25..0d03729a1 100644\n>> --- a/include/libcamera/internal/ipa_data_serializer.h\n>> +++ b/include/libcamera/internal/ipa_data_serializer.h\n>> @@ -7,6 +7,7 @@\n>> \n>> #pragma once\n>> \n>> +#include \n>> #include \n>> #include \n>> #include \n>> @@ -23,6 +24,7 @@\n>> #include \n>> \n>> #include \"libcamera/internal/control_serializer.h\"\n>> +#include \"libcamera/internal/serialization.h\"\n>> \n>> namespace libcamera {\n>> \n>> @@ -39,26 +41,6 @@ void appendPOD(std::vector &vec, T val)\n>> memcpy(&*(vec.end() - byteWidth), &val, byteWidth);\n>> }\n>> \n>> -template> - std::enable_if_t> * = nullptr>\n>> -T readPOD(std::vector::const_iterator it, size_t pos,\n>> - std::vector::const_iterator end)\n>> -{\n>> - ASSERT(pos + it < end);\n>> -\n>> - T ret = 0;\n>> - memcpy(&ret, &(*(it + pos)), sizeof(ret));\n>> -\n>> - return ret;\n>> -}\n>> -\n>> -template> - std::enable_if_t> * = nullptr>\n>> -T readPOD(std::vector &vec, size_t pos)\n>> -{\n>> - return readPOD(vec.cbegin(), pos, vec.end());\n>> -}\n>> -\n>> } /* namespace */\n>> \n>> template\n>> @@ -68,20 +50,8 @@ public:\n>> static std::tuple, std::vector>\n>> serialize(const T &data, ControlSerializer *cs = nullptr);\n>> \n>> - static T deserialize(const std::vector &data,\n>> - ControlSerializer *cs = nullptr);\n>> - static T deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - ControlSerializer *cs = nullptr);\n>> -\n>> - static T deserialize(const std::vector &data,\n>> - const std::vector &fds,\n>> - ControlSerializer *cs = nullptr);\n>> - static T deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - std::vector::const_iterator fdsBegin,\n>> - std::vector::const_iterator fdsEnd,\n>> - ControlSerializer *cs = nullptr);\n>> + [[nodiscard]] static std::optional\n>> + deserialize(SeriReader &reader, ControlSerializer *cs = nullptr);\n>> };\n>> \n>> #ifndef __DOXYGEN__\n>> @@ -121,9 +91,6 @@ public:\n>> std::tie(dvec, fvec) =\n>> IPADataSerializer::serialize(it, cs);\n>> \n>> - appendPOD(dataVec, dvec.size());\n>> - appendPOD(dataVec, fvec.size());\n>> -\n>> dataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n>> fdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n>> }\n>> @@ -131,52 +98,25 @@ public:\n>> return { dataVec, fdsVec };\n>> }\n>> \n>> - static std::vector deserialize(std::vector &data, ControlSerializer *cs = nullptr)\n>> + [[nodiscard]] static std::optional>\n>> + deserialize(SeriReader &reader, ControlSerializer *cs = nullptr)\n>> {\n>> - return deserialize(data.cbegin(), data.cend(), cs);\n>> - }\n>> + uint32_t vecLen;\n>> + if (!reader.read(vecLen))\n>> + return {};\n>> \n>> - static std::vector deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - ControlSerializer *cs = nullptr)\n>> - {\n>> - std::vector fds;\n>> - return deserialize(dataBegin, dataEnd, fds.cbegin(), fds.cend(), cs);\n>> - }\n>> + std::vector ret;\n>> + ret.reserve(vecLen);\n>> \n>> - static std::vector deserialize(std::vector &data, std::vector &fds,\n>> - ControlSerializer *cs = nullptr)\n>> - {\n>> - return deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n>> - }\n>> -\n>> - static std::vector deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - std::vector::const_iterator fdsBegin,\n>> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> - ControlSerializer *cs = nullptr)\n>> - {\n>> - uint32_t vecLen = readPOD(dataBegin, 0, dataEnd);\n>> - std::vector ret(vecLen);\n>> -\n>> - std::vector::const_iterator dataIter = dataBegin + 4;\n>> - std::vector::const_iterator fdIter = fdsBegin;\n>> for (uint32_t i = 0; i < vecLen; i++) {\n>> - uint32_t sizeofData = readPOD(dataIter, 0, dataEnd);\n>> - uint32_t sizeofFds = readPOD(dataIter, 4, dataEnd);\n>> - dataIter += 8;\n>> -\n>> - ret[i] = IPADataSerializer::deserialize(dataIter,\n>> - dataIter + sizeofData,\n>> - fdIter,\n>> - fdIter + sizeofFds,\n>> - cs);\n>> -\n>> - dataIter += sizeofData;\n>> - fdIter += sizeofFds;\n>> + auto item = IPADataSerializer::deserialize(reader, cs);\n>> + if (!item)\n>> + return {};\n>> +\n>> + ret.emplace_back(std::move(*item));\n>> }\n>> \n>> - return ret;\n>> + return std::move(ret);\n>> }\n>> };\n>> \n>> @@ -218,18 +158,12 @@ public:\n>> std::tie(dvec, fvec) =\n>> IPADataSerializer::serialize(it.first, cs);\n>> \n>> - appendPOD(dataVec, dvec.size());\n>> - appendPOD(dataVec, fvec.size());\n>> -\n>> dataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n>> fdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n>> \n>> std::tie(dvec, fvec) =\n>> IPADataSerializer::serialize(it.second, cs);\n>> \n>> - appendPOD(dataVec, dvec.size());\n>> - appendPOD(dataVec, fvec.size());\n>> -\n>> dataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n>> fdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n>> }\n>> @@ -237,63 +171,25 @@ public:\n>> return { dataVec, fdsVec };\n>> }\n>> \n>> - static std::map deserialize(std::vector &data, ControlSerializer *cs = nullptr)\n>> - {\n>> - return deserialize(data.cbegin(), data.cend(), cs);\n>> - }\n>> -\n>> - static std::map deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - ControlSerializer *cs = nullptr)\n>> + [[nodiscard]] static std::optional>\n>> + deserialize(SeriReader &reader, ControlSerializer *cs = nullptr)\n>> {\n>> - std::vector fds;\n>> - return deserialize(dataBegin, dataEnd, fds.cbegin(), fds.cend(), cs);\n>> - }\n>> + uint32_t mapLen;\n>> + if (!reader.read(mapLen))\n>> + return {};\n>> \n>> - static std::map deserialize(std::vector &data, std::vector &fds,\n>> - ControlSerializer *cs = nullptr)\n>> - {\n>> - return deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n>> - }\n>> -\n>> - static std::map deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - std::vector::const_iterator fdsBegin,\n>> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> - ControlSerializer *cs = nullptr)\n>> - {\n>> std::map ret;\n>> \n>> - uint32_t mapLen = readPOD(dataBegin, 0, dataEnd);\n>> -\n>> - std::vector::const_iterator dataIter = dataBegin + 4;\n>> - std::vector::const_iterator fdIter = fdsBegin;\n>> for (uint32_t i = 0; i < mapLen; i++) {\n>> - uint32_t sizeofData = readPOD(dataIter, 0, dataEnd);\n>> - uint32_t sizeofFds = readPOD(dataIter, 4, dataEnd);\n>> - dataIter += 8;\n>> -\n>> - K key = IPADataSerializer::deserialize(dataIter,\n>> - dataIter + sizeofData,\n>> - fdIter,\n>> - fdIter + sizeofFds,\n>> - cs);\n>> -\n>> - dataIter += sizeofData;\n>> - fdIter += sizeofFds;\n>> - sizeofData = readPOD(dataIter, 0, dataEnd);\n>> - sizeofFds = readPOD(dataIter, 4, dataEnd);\n>> - dataIter += 8;\n>> -\n>> - const V value = IPADataSerializer::deserialize(dataIter,\n>> - dataIter + sizeofData,\n>> - fdIter,\n>> - fdIter + sizeofFds,\n>> - cs);\n>> - ret.insert({ key, value });\n>> -\n>> - dataIter += sizeofData;\n>> - fdIter += sizeofFds;\n>> + auto key = IPADataSerializer::deserialize(reader, cs);\n>> + if (!key)\n>> + return {};\n>> +\n>> + auto value = IPADataSerializer::deserialize(reader, cs);\n>> + if (!value)\n>> + return {};\n>> +\n>> + ret.try_emplace(std::move(*key), std::move(*value));\n> \n> I'm curious, what's the reason for using try_emplace() here?\n\nAlternatives:\n * operator[]\n * requires default constructibility\n * must default construct and then (move) assign\n * emplace()\n * may need to allocate and construct a node (even if the key exists)\n * insert_or_assign()\n * could be used if the latest value is desired\n * must generate conditional code for (move) assignment\n\nSo my rule of thumb is to basically always use `try_emplace()` or `insert_or_assign()`,\nbut I think I will modify the map deserialization code to reject duplicate keys,\nso that leaves `try_emplace()` as the best option in my opinion.\n\n\n> \n>> }\n>> \n>> return ret;\n>> @@ -314,33 +210,14 @@ public:\n>> return { dataVec, {} };\n>> }\n>> \n>> - static Flags deserialize(std::vector &data,\n>> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> - {\n>> - return deserialize(data.cbegin(), data.cend());\n>> - }\n>> -\n>> - static Flags deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> + [[nodiscard]] static std::optional>\n>> + deserialize(SeriReader &reader, [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> {\n>> - return Flags{ static_cast(readPOD(dataBegin, 0, dataEnd)) };\n>> - }\n>> + uint32_t value;\n>> + if (!reader.read(value))\n>> + return {};\n>> \n>> - static Flags deserialize(std::vector &data,\n>> - [[maybe_unused]] std::vector &fds,\n>> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> - {\n>> - return deserialize(data.cbegin(), data.cend());\n>> - }\n>> -\n>> - static Flags deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - [[maybe_unused]] std::vector::const_iterator fdsBegin,\n>> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> - {\n>> - return deserialize(dataBegin, dataEnd);\n>> + return Flags{ static_cast(value) };\n>> }\n>> };\n>> \n>> @@ -360,33 +237,14 @@ public:\n>> return { dataVec, {} };\n>> }\n>> \n>> - static E deserialize(std::vector &data,\n>> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> + [[nodiscard]] static std::optional\n>> + deserialize(SeriReader &reader, [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> {\n>> - return deserialize(data.cbegin(), data.cend());\n>> - }\n>> + U value;\n>> + if (!reader.read(value))\n>> + return {};\n>> \n>> - static E deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> - {\n>> - return static_cast(readPOD(dataBegin, 0, dataEnd));\n>> - }\n>> -\n>> - static E deserialize(std::vector &data,\n>> - [[maybe_unused]] std::vector &fds,\n>> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> - {\n>> - return deserialize(data.cbegin(), data.cend());\n>> - }\n>> -\n>> - static E deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - [[maybe_unused]] std::vector::const_iterator fdsBegin,\n>> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> - {\n>> - return deserialize(dataBegin, dataEnd);\n>> + return static_cast(value);\n>> }\n>> };\n>> \n>> diff --git a/include/libcamera/internal/ipc_pipe.h b/include/libcamera/internal/ipc_pipe.h\n>> index 418c4622f..2b6cde042 100644\n>> --- a/include/libcamera/internal/ipc_pipe.h\n>> +++ b/include/libcamera/internal/ipc_pipe.h\n>> @@ -14,6 +14,7 @@\n>> #include \n>> \n>> #include \"libcamera/internal/ipc_unixsocket.h\"\n>> +#include \"libcamera/internal/serialization.h\"\n>> \n>> namespace libcamera {\n>> \n>> @@ -40,6 +41,8 @@ public:\n>> const std::vector &data() const { return data_; }\n>> const std::vector &fds() const { return fds_; }\n>> \n>> + SeriReader reader() const { return { data(), fds() }; }\n>> +\n>> private:\n>> Header header_;\n>> \n>> diff --git a/include/libcamera/internal/serialization.h b/include/libcamera/internal/serialization.h\n>> new file mode 100644\n>> index 000000000..daa7e5438\n>> --- /dev/null\n>> +++ b/include/libcamera/internal/serialization.h\n>> @@ -0,0 +1,91 @@\n>> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n>> +/*\n>> + * Copyright (C) 2025, Ideas On Board Oy\n>> + *\n>> + * Data (de)serialization helper structures\n>> + */\n>> +#pragma once\n>> +\n>> +#include \n>> +#include \n>> +#include \n>> +\n>> +#include \n>> +#include \n>> +\n>> +namespace libcamera {\n>> +\n>> +#ifndef __DOXYGEN__\n>> +class SeriReader\n>> +{\n>> +public:\n>> + SeriReader(Span data, Span fds = {})\n>> + : data_(data),\n>> + fds_(fds)\n>> + {\n>> + }\n>> +\n>> + SeriReader(Span data, Span fds = {})\n>> + : data_(reinterpret_cast(data.data()),\n>> + reinterpret_cast(data.data() + data.size())),\n>> + fds_(fds)\n>> + {\n>> + }\n>> +\n>> + [[nodiscard]] const std::byte *consume(std::size_t s)\n>> + {\n>> + if (data_.size() < s)\n>> + return nullptr;\n>> +\n>> + const auto *p = data_.data();\n>> + data_ = data_.subspan(s);\n>> +\n>> + return p;\n>> + }\n>> +\n>> + template\n>> + [[nodiscard]] bool read(Ts &...xs)\n>> + {\n>> + static_assert((std::is_trivially_copyable_v && ...));\n>> +\n>> + const auto *p = consume((sizeof(xs) + ...));\n>> + if (p)\n>> + ((memcpy(&xs, p, sizeof(xs)), p += sizeof(xs)), ...);\n>> +\n>> + return p;\n>> + }\n> \n> That is so cool.\n> \n>> +\n>> + template\n>> + [[nodiscard]] auto read()\n>> + {\n>> + std::tuple xs;\n>> + bool ok = std::apply([&](auto &...vs) {\n>> + return read(vs...);\n>> + }, xs);\n>> +\n>> + if constexpr (sizeof...(Ts) == 1)\n>> + return ok ? std::optional(std::get<0>(xs)) : std::nullopt;\n>> + else\n>> + return ok ? std::optional(xs) : std::nullopt;\n> \n> This means that each std::nullopt is of different types no? afaict you only use\n> this function when sizeof...(Ts) == 1.\n\nWell, `std::nullopt` is always `std::nullopt`, but you're right that the conditional\noperator will resolve to different types. I thought it is best for usability to have\n`read()` return `std::optional` and `read()` return `std::optional>`.\nBut yes, this is overload not really used, so it could be be removed.\n\n\n> \n>> + }\n>> +\n>> + [[nodiscard]] const SharedFD *nextFd()\n>> + {\n>> + if (fds_.empty())\n>> + return nullptr;\n>> +\n>> + const auto *p = fds_.data();\n>> + fds_ = fds_.subspan(1);\n>> +\n>> + return p;\n>> + }\n>> +\n>> + [[nodiscard]] bool empty() const { return data_.empty() && fds_.empty(); }\n>> +\n>> +private:\n>> + Span data_;\n>> + Span fds_;\n>> +};\n>> +#endif\n>> +\n>> +} /* namespace libcamera */\n>> diff --git a/src/libcamera/ipa_data_serializer.cpp b/src/libcamera/ipa_data_serializer.cpp\n>> index 0537f785b..979a1e0db 100644\n>> --- a/src/libcamera/ipa_data_serializer.cpp\n>> +++ b/src/libcamera/ipa_data_serializer.cpp\n>> @@ -50,42 +50,6 @@ namespace {\n>> * generated IPA proxies.\n>> */\n>> \n>> -/**\n>> - * \\fn template T readPOD(std::vector::iterator it, size_t pos,\n>> - * std::vector::iterator end)\n>> - * \\brief Read POD from byte vector, in little-endian order\n>> - * \\tparam T Type of POD to read\n>> - * \\param[in] it Iterator of byte vector to read from\n>> - * \\param[in] pos Index in byte vector to read from\n>> - * \\param[in] end Iterator marking end of byte vector\n>> - *\n>> - * This function is meant to be used by the IPA data serializer, and the\n>> - * generated IPA proxies.\n>> - *\n>> - * If the \\a pos plus the byte-width of the desired POD is past \\a end, it is\n>> - * a fata error will occur, as it means there is insufficient data for\n>> - * deserialization, which should never happen.\n>> - *\n>> - * \\return The POD read from \\a it at index \\a pos\n>> - */\n>> -\n>> -/**\n>> - * \\fn template T readPOD(std::vector &vec, size_t pos)\n>> - * \\brief Read POD from byte vector, in little-endian order\n>> - * \\tparam T Type of POD to read\n>> - * \\param[in] vec Byte vector to read from\n>> - * \\param[in] pos Index in vec to start reading from\n>> - *\n>> - * This function is meant to be used by the IPA data serializer, and the\n>> - * generated IPA proxies.\n>> - *\n>> - * If the \\a pos plus the byte-width of the desired POD is past the end of\n>> - * \\a vec, a fatal error will occur, as it means there is insufficient data\n>> - * for deserialization, which should never happen.\n>> - *\n>> - * \\return The POD read from \\a vec at index \\a pos\n>> - */\n>> -\n>> } /* namespace */\n>> \n>> /**\n>> @@ -106,80 +70,13 @@ namespace {\n>> \n>> /**\n>> * \\fn template IPADataSerializer::deserialize(\n>> - * const std::vector &data,\n>> + * SeriReader &reader,\n>> * ControlSerializer *cs = nullptr)\n>> - * \\brief Deserialize byte vector into an object\n>> + * \\brief Deserialize bytes and file descriptors vector into an object\n>> * \\tparam T Type of object to deserialize to\n>> - * \\param[in] data Byte vector to deserialize from\n>> + * \\param[in] reader Source of bytes and file descriptors\n>> * \\param[in] cs ControlSerializer\n>> *\n>> - * This version of deserialize() can be used if the object type \\a T and its\n>> - * members don't have any SharedFD.\n>> - *\n>> - * \\a cs is only necessary if the object type \\a T or its members contain\n>> - * ControlList or ControlInfoMap.\n>> - *\n>> - * \\return The deserialized object\n>> - */\n>> -\n>> -/**\n>> - * \\fn template IPADataSerializer::deserialize(\n>> - * std::vector::const_iterator dataBegin,\n>> - * std::vector::const_iterator dataEnd,\n>> - * ControlSerializer *cs = nullptr)\n>> - * \\brief Deserialize byte vector into an object\n>> - * \\tparam T Type of object to deserialize to\n>> - * \\param[in] dataBegin Begin iterator of byte vector to deserialize from\n>> - * \\param[in] dataEnd End iterator of byte vector to deserialize from\n>> - * \\param[in] cs ControlSerializer\n>> - *\n>> - * This version of deserialize() can be used if the object type \\a T and its\n>> - * members don't have any SharedFD.\n>> - *\n>> - * \\a cs is only necessary if the object type \\a T or its members contain\n>> - * ControlList or ControlInfoMap.\n>> - *\n>> - * \\return The deserialized object\n>> - */\n>> -\n>> -/**\n>> - * \\fn template IPADataSerializer::deserialize(\n>> - * const std::vector &data,\n>> - * const std::vector &fds,\n>> - * ControlSerializer *cs = nullptr)\n>> - * \\brief Deserialize byte vector and fd vector into an object\n>> - * \\tparam T Type of object to deserialize to\n>> - * \\param[in] data Byte vector to deserialize from\n>> - * \\param[in] fds Fd vector to deserialize from\n>> - * \\param[in] cs ControlSerializer\n>> - *\n>> - * This version of deserialize() (or the iterator version) must be used if\n>> - * the object type \\a T or its members contain SharedFD.\n>> - *\n>> - * \\a cs is only necessary if the object type \\a T or its members contain\n>> - * ControlList or ControlInfoMap.\n>> - *\n>> - * \\return The deserialized object\n>> - */\n>> -\n>> -/**\n>> - * \\fn template IPADataSerializer::deserialize(\n>> - * std::vector::const_iterator dataBegin,\n>> - * std::vector::const_iterator dataEnd,\n>> - * std::vector::const_iterator fdsBegin,\n>> - * std::vector::const_iterator fdsEnd,\n>> - * ControlSerializer *cs = nullptr)\n>> - * \\brief Deserialize byte vector and fd vector into an object\n>> - * \\tparam T Type of object to deserialize to\n>> - * \\param[in] dataBegin Begin iterator of byte vector to deserialize from\n>> - * \\param[in] dataEnd End iterator of byte vector to deserialize from\n>> - * \\param[in] fdsBegin Begin iterator of fd vector to deserialize from\n>> - * \\param[in] fdsEnd End iterator of fd vector to deserialize from\n>> - * \\param[in] cs ControlSerializer\n>> - *\n>> - * This version of deserialize() (or the vector version) must be used if\n>> - * the object type \\a T or its members contain SharedFD.\n>> - *\n>> * \\a cs is only necessary if the object type \\a T or its members contain\n>> * ControlList or ControlInfoMap.\n>> *\n>> @@ -202,37 +99,11 @@ IPADataSerializer::serialize(const type &data, \\\n>> } \\\n>> \\\n>> template<> \\\n>> -type IPADataSerializer::deserialize(std::vector::const_iterator dataBegin, \\\n>> - std::vector::const_iterator dataEnd, \\\n>> - [[maybe_unused]] ControlSerializer *cs) \\\n>> +std::optional IPADataSerializer::deserialize(SeriReader &reader, \\\n>> + [[maybe_unused]] ControlSerializer *cs) \\\n>> { \\\n>> - return readPOD(dataBegin, 0, dataEnd); \\\n>> + return reader.read(); \\\n>> } \\\n>> - \\\n>> -template<> \\\n>> -type IPADataSerializer::deserialize(const std::vector &data, \\\n>> - ControlSerializer *cs) \\\n>> -{ \\\n>> - return deserialize(data.cbegin(), data.end(), cs); \\\n>> -} \\\n>> - \\\n>> -template<> \\\n>> -type IPADataSerializer::deserialize(const std::vector &data, \\\n>> - [[maybe_unused]] const std::vector &fds, \\\n>> - ControlSerializer *cs) \\\n>> -{ \\\n>> - return deserialize(data.cbegin(), data.end(), cs); \\\n>> -} \\\n>> - \\\n>> -template<> \\\n>> -type IPADataSerializer::deserialize(std::vector::const_iterator dataBegin, \\\n>> - std::vector::const_iterator dataEnd, \\\n>> - [[maybe_unused]] std::vector::const_iterator fdsBegin, \\\n>> - [[maybe_unused]] std::vector::const_iterator fdsEnd, \\\n>> - ControlSerializer *cs) \\\n>> -{ \\\n>> - return deserialize(dataBegin, dataEnd, cs); \\\n>> -}\n>> \n>> DEFINE_POD_SERIALIZER(bool)\n>> DEFINE_POD_SERIALIZER(uint8_t)\n>> @@ -256,44 +127,29 @@ std::tuple, std::vector>\n>> IPADataSerializer::serialize(const std::string &data,\n>> [[maybe_unused]] ControlSerializer *cs)\n>> {\n>> - return { { data.cbegin(), data.end() }, {} };\n>> -}\n>> + std::vector dataVec;\n>> \n>> -template<>\n>> -std::string\n>> -IPADataSerializer::deserialize(const std::vector &data,\n>> - [[maybe_unused]] ControlSerializer *cs)\n>> -{\n>> - return { data.cbegin(), data.cend() };\n>> -}\n>> + ASSERT(data.size() <= std::numeric_limits::max());\n>> + appendPOD(dataVec, data.size());\n>> + dataVec.insert(dataVec.end(), data.c_str(), data.c_str() + data.size());\n>> \n>> -template<>\n>> -std::string\n>> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - [[maybe_unused]] ControlSerializer *cs)\n>> -{\n>> - return { dataBegin, dataEnd };\n>> + return { dataVec, {} };\n>> }\n>> \n>> template<>\n>> -std::string\n>> -IPADataSerializer::deserialize(const std::vector &data,\n>> - [[maybe_unused]] const std::vector &fds,\n>> +std::optional\n>> +IPADataSerializer::deserialize(SeriReader &reader,\n>> [[maybe_unused]] ControlSerializer *cs)\n>> {\n>> - return { data.cbegin(), data.cend() };\n>> -}\n>> + uint32_t length;\n>> + if (!reader.read(length))\n>> + return {};\n>> \n>> -template<>\n>> -std::string\n>> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - [[maybe_unused]] std::vector::const_iterator fdsBegin,\n>> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> - [[maybe_unused]] ControlSerializer *cs)\n>> -{\n>> - return { dataBegin, dataEnd };\n>> + const auto *p = reader.consume(length);\n>> + if (!p)\n>> + return {};\n>> +\n>> + return { { reinterpret_cast(p), std::size_t(length) } };\n>> }\n>> \n>> /*\n>> @@ -356,73 +212,43 @@ IPADataSerializer::serialize(const ControlList &data, ControlSerial\n>> }\n>> \n>> template<>\n>> -ControlList\n>> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> +std::optional\n>> +IPADataSerializer::deserialize(SeriReader &reader,\n>> ControlSerializer *cs)\n>> {\n>> if (!cs)\n>> LOG(IPADataSerializer, Fatal)\n>> << \"ControlSerializer not provided for deserialization of ControlList\";\n>> \n>> - if (std::distance(dataBegin, dataEnd) < 8)\n>> - return {};\n>> -\n>> - uint32_t infoDataSize = readPOD(dataBegin, 0, dataEnd);\n>> - uint32_t listDataSize = readPOD(dataBegin, 4, dataEnd);\n>> -\n>> - std::vector::const_iterator it = dataBegin + 8;\n>> -\n>> - if (infoDataSize + listDataSize < infoDataSize ||\n>> - static_cast(std::distance(it, dataEnd)) < infoDataSize + listDataSize)\n>> + uint32_t infoDataSize, listDataSize;\n>> + if (!reader.read(infoDataSize, listDataSize))\n>> return {};\n>> \n>> if (infoDataSize > 0) {\n>> - ByteStreamBuffer buffer(&*it, infoDataSize);\n>> + const auto *p = reader.consume(infoDataSize);\n>> + if (!p)\n>> + return {};\n>> +\n>> + ByteStreamBuffer buffer(reinterpret_cast(p), infoDataSize);\n> \n> That is cool.\n> \n>> ControlInfoMap map = cs->deserialize(buffer);\n>> /* It's fine if map is empty. */\n>> if (buffer.overflow()) {\n>> LOG(IPADataSerializer, Error)\n>> << \"Failed to deserialize ControlLists's ControlInfoMap: buffer overflow\";\n>> - return ControlList();\n>> + return {};\n>> }\n>> }\n>> \n>> - it += infoDataSize;\n>> - ByteStreamBuffer buffer(&*it, listDataSize);\n>> + const auto *p = reader.consume(listDataSize);\n>> +\n>> + ByteStreamBuffer buffer(reinterpret_cast(p), listDataSize);\n>> ControlList list = cs->deserialize(buffer);\n>> - if (buffer.overflow())\n>> + if (buffer.overflow()) {\n>> LOG(IPADataSerializer, Error) << \"Failed to deserialize ControlList: buffer overflow\";\n>> + return {};\n>> + }\n>> \n>> - return list;\n>> -}\n>> -\n>> -template<>\n>> -ControlList\n>> -IPADataSerializer::deserialize(const std::vector &data,\n>> - ControlSerializer *cs)\n>> -{\n>> - return deserialize(data.cbegin(), data.end(), cs);\n>> -}\n>> -\n>> -template<>\n>> -ControlList\n>> -IPADataSerializer::deserialize(const std::vector &data,\n>> - [[maybe_unused]] const std::vector &fds,\n>> - ControlSerializer *cs)\n>> -{\n>> - return deserialize(data.cbegin(), data.end(), cs);\n>> -}\n>> -\n>> -template<>\n>> -ControlList\n>> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - [[maybe_unused]] std::vector::const_iterator fdsBegin,\n>> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> - ControlSerializer *cs)\n>> -{\n>> - return deserialize(dataBegin, dataEnd, cs);\n>> + return std::move(list);\n>> }\n>> \n>> /*\n>> @@ -458,57 +284,28 @@ IPADataSerializer::serialize(const ControlInfoMap &map,\n>> }\n>> \n>> template<>\n>> -ControlInfoMap\n>> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> +std::optional\n>> +IPADataSerializer::deserialize(SeriReader &reader,\n>> ControlSerializer *cs)\n>> {\n>> if (!cs)\n>> LOG(IPADataSerializer, Fatal)\n>> << \"ControlSerializer not provided for deserialization of ControlInfoMap\";\n>> \n>> - if (std::distance(dataBegin, dataEnd) < 4)\n>> + uint32_t infoDataSize;\n>> + if (!reader.read(infoDataSize))\n>> return {};\n>> \n>> - uint32_t infoDataSize = readPOD(dataBegin, 0, dataEnd);\n>> -\n>> - std::vector::const_iterator it = dataBegin + 4;\n>> -\n>> - if (static_cast(std::distance(it, dataEnd)) < infoDataSize)\n>> + const auto *p = reader.consume(infoDataSize);\n>> + if (!p)\n>> return {};\n>> \n>> - ByteStreamBuffer buffer(&*it, infoDataSize);\n>> + ByteStreamBuffer buffer(reinterpret_cast(p), infoDataSize);\n>> ControlInfoMap map = cs->deserialize(buffer);\n>> + if (buffer.overflow())\n>> + return {};\n>> \n>> - return map;\n>> -}\n>> -\n>> -template<>\n>> -ControlInfoMap\n>> -IPADataSerializer::deserialize(const std::vector &data,\n>> - ControlSerializer *cs)\n>> -{\n>> - return deserialize(data.cbegin(), data.end(), cs);\n>> -}\n>> -\n>> -template<>\n>> -ControlInfoMap\n>> -IPADataSerializer::deserialize(const std::vector &data,\n>> - [[maybe_unused]] const std::vector &fds,\n>> - ControlSerializer *cs)\n>> -{\n>> - return deserialize(data.cbegin(), data.end(), cs);\n>> -}\n>> -\n>> -template<>\n>> -ControlInfoMap\n>> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - [[maybe_unused]] std::vector::const_iterator fdsBegin,\n>> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> - ControlSerializer *cs)\n>> -{\n>> - return deserialize(dataBegin, dataEnd, cs);\n>> + return std::move(map);\n>> }\n>> \n>> /*\n>> @@ -542,27 +339,23 @@ IPADataSerializer::serialize(const SharedFD &data,\n>> }\n>> \n>> template<>\n>> -SharedFD IPADataSerializer::deserialize([[maybe_unused]] std::vector::const_iterator dataBegin,\n>> - [[maybe_unused]] std::vector::const_iterator dataEnd,\n>> - std::vector::const_iterator fdsBegin,\n>> - std::vector::const_iterator fdsEnd,\n>> - [[maybe_unused]] ControlSerializer *cs)\n>> +std::optional\n>> +IPADataSerializer::deserialize(SeriReader &reader,\n>> + [[maybe_unused]] ControlSerializer *cs)\n>> {\n>> - ASSERT(std::distance(dataBegin, dataEnd) >= 4);\n>> + uint32_t valid;\n>> \n>> - uint32_t valid = readPOD(dataBegin, 0, dataEnd);\n>> + if (!reader.read(valid))\n>> + return {};\n>> \n>> - ASSERT(!(valid && std::distance(fdsBegin, fdsEnd) < 1));\n>> + if (!valid)\n>> + return SharedFD{};\n>> \n>> - return valid ? *fdsBegin : SharedFD();\n>> -}\n>> + const auto *fd = reader.nextFd();\n>> + if (!fd)\n>> + return {};\n>> \n>> -template<>\n>> -SharedFD IPADataSerializer::deserialize(const std::vector &data,\n>> - const std::vector &fds,\n>> - [[maybe_unused]] ControlSerializer *cs)\n>> -{\n>> - return deserialize(data.cbegin(), data.end(), fds.cbegin(), fds.end());\n>> + return *fd;\n>> }\n>> \n>> /*\n>> @@ -594,30 +387,19 @@ IPADataSerializer::serialize(const FrameBuffer::Plane &data,\n>> }\n>> \n>> template<>\n>> -FrameBuffer::Plane\n>> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - std::vector::const_iterator fdsBegin,\n>> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> +std::optional\n>> +IPADataSerializer::deserialize(SeriReader &reader,\n>> [[maybe_unused]] ControlSerializer *cs)\n>> {\n>> - FrameBuffer::Plane ret;\n>> -\n>> - ret.fd = IPADataSerializer::deserialize(dataBegin, dataBegin + 4,\n>> - fdsBegin, fdsBegin + 1);\n>> - ret.offset = readPOD(dataBegin, 4, dataEnd);\n>> - ret.length = readPOD(dataBegin, 8, dataEnd);\n>> + auto fd = IPADataSerializer::deserialize(reader);\n>> + if (!fd)\n>> + return {};\n>> \n>> - return ret;\n>> -}\n>> + uint32_t offset, length;\n>> + if (!reader.read(offset, length))\n>> + return {};\n>> \n>> -template<>\n>> -FrameBuffer::Plane\n>> -IPADataSerializer::deserialize(const std::vector &data,\n>> - const std::vector &fds,\n>> - ControlSerializer *cs)\n>> -{\n>> - return deserialize(data.cbegin(), data.end(), fds.cbegin(), fds.end(), cs);\n>> + return { { std::move(*fd), offset, length } };\n>> }\n>> \n>> #endif /* __DOXYGEN__ */\n>> diff --git a/src/libcamera/ipc_pipe.cpp b/src/libcamera/ipc_pipe.cpp\n>> index 548299d05..71d0c2cb4 100644\n>> --- a/src/libcamera/ipc_pipe.cpp\n>> +++ b/src/libcamera/ipc_pipe.cpp\n>> @@ -148,6 +148,11 @@ IPCUnixSocket::Payload IPCMessage::payload() const\n>> * \\brief Returns a const reference to the vector containing file descriptors\n>> */\n>> \n>> +/**\n>> + * \\fn IPCMessage::reader() const\n>> + * \\brief Returns a `SeriReader` instance for parsing the message\n>> + */\n>> +\n>> /**\n>> * \\class IPCPipe\n>> * \\brief IPC message pipe for IPA isolation\n>> diff --git a/test/ipc/unixsocket_ipc.cpp b/test/ipc/unixsocket_ipc.cpp\n>> index df7d9c2b4..c0e174ba9 100644\n>> --- a/test/ipc/unixsocket_ipc.cpp\n>> +++ b/test/ipc/unixsocket_ipc.cpp\n>> @@ -102,7 +102,14 @@ private:\n>> }\n>> \n>> case CmdSetAsync: {\n>> - value_ = IPADataSerializer::deserialize(ipcMessage.data());\n>> + SeriReader reader = ipcMessage.reader();\n>> + auto value = IPADataSerializer::deserialize(reader);\n>> + if (!value) {\n>> + cerr << \"Failed to deserialize value\" << endl;\n>> + stop(-ENODATA);\n>> + } else {\n>> + value_ = *value;\n>> + }\n>> break;\n>> }\n>> }\n>> @@ -155,7 +162,14 @@ protected:\n>> return ret;\n>> }\n>> \n>> - return IPADataSerializer::deserialize(buf.data());\n>> + SeriReader reader = buf.reader();\n>> + auto value = IPADataSerializer::deserialize(reader);\n>> + if (!value) {\n>> + cerr << \"Failed to deserialize value\" << endl;\n>> + return -ENODATA;\n>> + }\n>> +\n>> + return *value;\n>> }\n>> \n>> int exit()\n>> diff --git a/test/serialization/generated_serializer/generated_serializer_test.cpp b/test/serialization/generated_serializer/generated_serializer_test.cpp\n>> index dd6968850..0640e741d 100644\n>> --- a/test/serialization/generated_serializer/generated_serializer_test.cpp\n>> +++ b/test/serialization/generated_serializer/generated_serializer_test.cpp\n>> @@ -43,7 +43,7 @@ if (struct1.field != struct2.field) { \\\n>> }\n>> \n>> \n>> - ipa::test::TestStruct t, u;\n>> + ipa::test::TestStruct t;\n>> \n>> t.m = {\n>> { \"a\", \"z\" },\n>> @@ -71,8 +71,13 @@ if (struct1.field != struct2.field) { \\\n>> \n>> std::tie(serialized, ignore) =\n>> IPADataSerializer::serialize(t);\n>> + SeriReader reader(serialized);\n>> \n>> - u = IPADataSerializer::deserialize(serialized);\n>> + auto optu = IPADataSerializer::deserialize(reader);\n>> + if (!optu)\n>> + return TestFail;\n>> +\n>> + auto &u = *optu;\n>> \n>> if (!equals(t.m, u.m))\n>> return TestFail;\n>> @@ -91,12 +96,16 @@ if (struct1.field != struct2.field) { \\\n>> \n>> /* Test vector of generated structs */\n>> std::vector v = { t, u };\n>> - std::vector w;\n>> \n>> std::tie(serialized, ignore) =\n>> IPADataSerializer>::serialize(v);\n>> + reader = SeriReader(serialized);\n>> +\n>> + auto optw = IPADataSerializer>::deserialize(reader);\n>> + if (!optw)\n>> + return TestFail;\n>> \n>> - w = IPADataSerializer>::deserialize(serialized);\n>> + auto &w = *optw;\n>> \n>> if (!equals(v[0].m, w[0].m) ||\n>> !equals(v[1].m, w[1].m))\n>> diff --git a/test/serialization/ipa_data_serializer_test.cpp b/test/serialization/ipa_data_serializer_test.cpp\n>> index afea93a6c..3873808a6 100644\n>> --- a/test/serialization/ipa_data_serializer_test.cpp\n>> +++ b/test/serialization/ipa_data_serializer_test.cpp\n>> @@ -52,7 +52,9 @@ int testPodSerdes(T in)\n>> std::vector fds;\n>> \n>> std::tie(buf, fds) = IPADataSerializer::serialize(in);\n>> - T out = IPADataSerializer::deserialize(buf, fds);\n>> + SeriReader reader(buf, fds);\n>> +\n>> + auto out = IPADataSerializer::deserialize(reader);\n>> if (in == out)\n>> return TestPass;\n>> \n>> @@ -71,7 +73,9 @@ int testVectorSerdes(const std::vector &in,\n>> std::vector fds;\n>> \n>> std::tie(buf, fds) = IPADataSerializer>::serialize(in, cs);\n>> - std::vector out = IPADataSerializer>::deserialize(buf, fds, cs);\n>> + SeriReader reader(buf, fds);\n>> +\n>> + auto out = IPADataSerializer>::deserialize(reader, cs);\n>> if (in == out)\n>> return TestPass;\n>> \n>> @@ -91,7 +95,9 @@ int testMapSerdes(const std::map &in,\n>> std::vector fds;\n>> \n>> std::tie(buf, fds) = IPADataSerializer>::serialize(in, cs);\n>> - std::map out = IPADataSerializer>::deserialize(buf, fds, cs);\n>> + SeriReader reader(buf, fds);\n>> +\n>> + auto out = IPADataSerializer>::deserialize(reader, cs);\n>> if (in == out)\n>> return TestPass;\n>> \n>> @@ -171,17 +177,27 @@ private:\n>> std::tie(listBuf, std::ignore) =\n>> IPADataSerializer::serialize(list, &cs);\n>> \n>> - const ControlInfoMap infoMapOut =\n>> - IPADataSerializer::deserialize(infoMapBuf, &cs);\n>> + SeriReader listReader(listBuf);\n>> + SeriReader infoMapReader(infoMapBuf);\n>> +\n>> + auto infoMapOut = IPADataSerializer::deserialize(infoMapReader, &cs);\n>> + if (!infoMapOut) {\n>> + cerr << \"`ControlInfoMap` cannot be deserialized\" << endl;\n>> + return TestFail;\n>> + }\n>> \n>> - ControlList listOut = IPADataSerializer::deserialize(listBuf, &cs);\n>> + auto listOut = IPADataSerializer::deserialize(listReader, &cs);\n>> + if (!listOut) {\n>> + cerr << \"`ControlList` cannot be deserialized\" << endl;\n>> + return TestFail;\n>> + }\n>> \n>> - if (!SerializationTest::equals(infoMap, infoMapOut)) {\n>> + if (!SerializationTest::equals(infoMap, *infoMapOut)) {\n>> cerr << \"Deserialized map doesn't match original\" << endl;\n>> return TestFail;\n>> }\n>> \n>> - if (!SerializationTest::equals(list, listOut)) {\n>> + if (!SerializationTest::equals(list, *listOut)) {\n>> cerr << \"Deserialized list doesn't match original\" << endl;\n>> return TestFail;\n>> }\n>> diff --git a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n>> index 9a3aadbd2..843260b4b 100644\n>> --- a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n>> +++ b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n>> @@ -107,13 +107,13 @@ namespace {{ns}} {\n>> {% if interface_event.methods|length > 0 %}\n>> void {{proxy_name}}::recvMessage(const IPCMessage &data)\n>> {\n>> - size_t dataSize = data.data().size();\n>> + SeriReader reader = data.reader();\n>> {{cmd_event_enum_name}} _cmd = static_cast<{{cmd_event_enum_name}}>(data.header().cmd);\n>> \n>> switch (_cmd) {\n>> {%- for method in interface_event.methods %}\n>> case {{cmd_event_enum_name}}::{{method.mojom_name|cap}}: {\n>> - {{method.mojom_name}}IPC(data.data().cbegin(), dataSize, data.fds());\n>> + {{method.mojom_name}}IPC(reader);\n>> break;\n>> }\n>> {%- endfor %}\n>> @@ -211,18 +211,23 @@ void {{proxy_name}}::recvMessage(const IPCMessage &data)\n>> return;\n>> {%- endif %}\n>> }\n>> +{% if has_output %}\n>> + SeriReader _outputReader = _ipcOutputBuf.reader();\n>> +{% endif -%}\n>> {% if method|method_return_value != \"void\" %}\n>> - {{method|method_return_value}} _retValue = IPADataSerializer<{{method|method_return_value}}>::deserialize(_ipcOutputBuf.data(), 0);\n>> -\n>> -{{proxy_funcs.deserialize_call(method|method_param_outputs, '_ipcOutputBuf.data()', '_ipcOutputBuf.fds()', init_offset = method|method_return_value|byte_width|int)}}\n>> + auto _retValue = IPADataSerializer<{{method|method_return_value}}>::deserialize(_outputReader);\n>> + ASSERT(_retValue);\n>> +{% endif -%}\n>> \n>> - return _retValue;\n>> +{% if has_output %}\n> \n> I think this should be {% if method|method_param_outputs|length > 0 %} because\n> has_output also covers `method|method_return_value != \"void\"`, so you'd be\n> deserializing the single _retValue twice.\n\nThe generated code looks ok to me, and as far as I can see the return value is not\npart of `method_param_outputs`, so the `proxy_funcs.deserialize_call()` should expand\nto nothing because it receives an empty list.\n\n\n> \n> Or maybe an ASSERT(_outputReader.empty()) might be valuable after\n> ASSERT(_retValue) above.\n> \n>> + {{proxy_funcs.deserialize_call(method|method_param_outputs, '_outputReader')}}\n>> + ASSERT(_outputReader.empty());\n>> +{% endif -%}\n>> \n>> -{% elif method|method_param_outputs|length > 0 %}\n>> -{{proxy_funcs.deserialize_call(method|method_param_outputs, '_ipcOutputBuf.data()', '_ipcOutputBuf.fds()')}}\n>> +{% if method|method_return_value != \"void\" %}\n>> + return std::move(*_retValue);\n>> {% endif -%}\n>> }\n>> -\n>> {% endfor %}\n>> \n>> {% for method in interface_event.methods %}\n>> @@ -232,12 +237,9 @@ void {{proxy_name}}::recvMessage(const IPCMessage &data)\n>> {{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n>> }\n>> \n>> -void {{proxy_name}}::{{method.mojom_name}}IPC(\n>> - [[maybe_unused]] std::vector::const_iterator data,\n>> - [[maybe_unused]] size_t dataSize,\n>> - [[maybe_unused]] const std::vector &fds)\n>> +void {{proxy_name}}::{{method.mojom_name}}IPC([[maybe_unused]] SeriReader &reader)\n>> {\n>> -{{proxy_funcs.deserialize_call(method.parameters, 'data', 'fds', false, true, true, 'dataSize')}}\n>> + {{proxy_funcs.deserialize_call(method.parameters, 'reader', false, true)}}\n> \n> I think this is one level too much indentation? Same for the other places you\n> add this call.\n\nThe generated code looks good to me. I think the indentiation of this macro call\ndoes not matter because it has `{%-`, so the preceeding whitespaces will be stripped.\n\n\n> \n>> {{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n>> }\n>> {% endfor %}\n>> diff --git a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n>> index a0312a7c1..945a4ded9 100644\n>> --- a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n>> +++ b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n>> @@ -53,10 +53,7 @@ private:\n>> {% endfor %}\n>> {% for method in interface_event.methods %}\n>> {{proxy_funcs.func_sig(proxy_name, method, \"Thread\", false)|indent(8, true)}};\n>> - void {{method.mojom_name}}IPC(\n>> - std::vector::const_iterator data,\n>> - size_t dataSize,\n>> - const std::vector &fds);\n>> + void {{method.mojom_name}}IPC(SeriReader &reader);\n>> {% endfor %}\n>> \n>> /* Helper class to invoke async functions in another thread. */\n>> diff --git a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n>> index 1f990d3f9..de6378be0 100644\n>> --- a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n>> +++ b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n>> @@ -71,6 +71,7 @@ public:\n>> }\n>> \n>> IPCMessage _ipcMessage(_message);\n>> + SeriReader _reader = _ipcMessage.reader();\n>> \n>> {{cmd_enum_name}} _cmd = static_cast<{{cmd_enum_name}}>(_ipcMessage.header().cmd);\n>> \n>> @@ -85,7 +86,9 @@ public:\n>> {%- if method.mojom_name == \"configure\" %}\n>> controlSerializer_.reset();\n>> {%- endif %}\n>> - {{proxy_funcs.deserialize_call(method|method_param_inputs, '_ipcMessage.data()', '_ipcMessage.fds()', false, true)|indent(16, true)}}\n>> + {{proxy_funcs.deserialize_call(method|method_param_inputs, '_reader', false, true)|indent(16, true)}}\n>> + ASSERT(_reader.empty());\n>> +\n>> {% for param in method|method_param_outputs %}\n>> {{param|name}} {{param.mojom_name}};\n>> {% endfor %}\n>> diff --git a/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl b/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl\n>> index 01e2567ca..b6835ca35 100644\n>> --- a/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl\n>> +++ b/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl\n>> @@ -64,15 +64,6 @@\n>> );\n>> {%- endfor %}\n>> \n>> -{%- if params|length > 1 %}\n>> -{%- for param in params %}\n>> - appendPOD({{buf}}, {{param.mojom_name}}Buf.size());\n>> -{%- if param|has_fd %}\n>> - appendPOD({{buf}}, {{param.mojom_name}}Fds.size());\n>> -{%- endif %}\n>> -{%- endfor %}\n>> -{%- endif %}\n>> -\n>> {%- for param in params %}\n>> {{buf}}.insert({{buf}}.end(), {{param.mojom_name}}Buf.begin(), {{param.mojom_name}}Buf.end());\n>> {%- endfor %}\n>> @@ -84,104 +75,28 @@\n>> {%- endfor %}\n>> {%- endmacro -%}\n>> \n>> -\n>> -{#\n>> - # \\brief Deserialize a single object from data buffer and fd vector\n>> - #\n>> - # \\param pointer If true, deserialize the object into a dereferenced pointer\n>> - # \\param iter If true, treat \\a buf as an iterator instead of a vector\n>> - # \\param data_size Variable that holds the size of the vector referenced by \\a buf\n>> - #\n>> - # Generate code to deserialize a single object, as specified in \\a param,\n>> - # from \\a buf data buffer and \\a fds fd vector.\n>> - # This code is meant to be used by macro deserialize_call.\n>> - #}\n>> -{%- macro deserialize_param(param, pointer, loop, buf, fds, iter, data_size) -%}\n>> -{{\"*\" if pointer}}{{param.mojom_name}} =\n>> -IPADataSerializer<{{param|name_full}}>::deserialize(\n>> - {{buf}}{{- \".cbegin()\" if not iter}} + {{param.mojom_name}}Start,\n>> -{%- if loop.last and not iter %}\n>> - {{buf}}.cend()\n>> -{%- elif not iter %}\n>> - {{buf}}.cbegin() + {{param.mojom_name}}Start + {{param.mojom_name}}BufSize\n>> -{%- elif iter and loop.length == 1 %}\n>> - {{buf}} + {{data_size}}\n>> -{%- else %}\n>> - {{buf}} + {{param.mojom_name}}Start + {{param.mojom_name}}BufSize\n>> -{%- endif -%}\n>> -{{- \",\" if param|has_fd}}\n>> -{%- if param|has_fd %}\n>> - {{fds}}.cbegin() + {{param.mojom_name}}FdStart,\n>> -{%- if loop.last %}\n>> - {{fds}}.cend()\n>> -{%- else %}\n>> - {{fds}}.cbegin() + {{param.mojom_name}}FdStart + {{param.mojom_name}}FdsSize\n>> -{%- endif -%}\n>> -{%- endif -%}\n>> -{{- \",\" if param|needs_control_serializer}}\n>> -{%- if param|needs_control_serializer %}\n>> - &controlSerializer_\n>> -{%- endif -%}\n>> -);\n>> -{%- endmacro -%}\n>> -\n>> -\n>> {#\n>> - # \\brief Deserialize multiple objects from data buffer and fd vector\n>> + # \\brief Deserialize multiple objects\n>> #\n>> # \\param pointer If true, deserialize objects into pointers, and adds a null check.\n>> # \\param declare If true, declare the objects in addition to deserialization.\n>> - # \\param iter if true, treat \\a buf as an iterator instead of a vector\n>> - # \\param data_size Variable that holds the size of the vector referenced by \\a buf\n>> - #\n>> - # Generate code to deserialize multiple objects, as specified in \\a params\n>> - # (which are the parameters to some function), from \\a buf data buffer and\n>> - # \\a fds fd vector.\n>> - # This code is meant to be used by the proxy, for deserializing after IPC calls.\n>> #\n>> - # \\todo Avoid intermediate vectors\n>> + # Generate code to deserialize multiple objects, as specified in \\a params,\n>> + # from \\a reader. This code is meant to be used by the proxy, for deserializing\n>> + # after IPC calls.\n>> #}\n>> -{%- macro deserialize_call(params, buf, fds, pointer = true, declare = false, iter = false, data_size = '', init_offset = 0) -%}\n>> -{% set ns = namespace(size_offset = init_offset) %}\n>> -{%- if params|length > 1 %}\n>> +{%- macro deserialize_call(params, reader, pointer = true, declare = false) -%}\n>> {%- for param in params %}\n>> - [[maybe_unused]] const size_t {{param.mojom_name}}BufSize = readPOD({{buf}}, {{ns.size_offset}}\n>> -{%- if iter -%}\n>> -, {{buf}} + {{data_size}}\n>> -{%- endif -%}\n>> -);\n>> - {%- set ns.size_offset = ns.size_offset + 4 %}\n>> -{%- if param|has_fd %}\n>> - [[maybe_unused]] const size_t {{param.mojom_name}}FdsSize = readPOD({{buf}}, {{ns.size_offset}}\n>> -{%- if iter -%}\n>> -, {{buf}} + {{data_size}}\n>> -{%- endif -%}\n>> -);\n>> - {%- set ns.size_offset = ns.size_offset + 4 %}\n>> -{%- endif %}\n>> -{%- endfor %}\n>> -{%- endif %}\n>> -{% for param in params %}\n>> -{%- if loop.first %}\n>> - const size_t {{param.mojom_name}}Start = {{ns.size_offset}};\n>> -{%- else %}\n>> - const size_t {{param.mojom_name}}Start = {{loop.previtem.mojom_name}}Start + {{loop.previtem.mojom_name}}BufSize;\n>> -{%- endif %}\n>> -{%- endfor %}\n>> -{% for param in params|with_fds %}\n>> -{%- if loop.first %}\n>> - const size_t {{param.mojom_name}}FdStart = 0;\n>> + auto param_{{param.mojom_name}} = IPADataSerializer<{{param|name_full}}>::deserialize({{reader}}, &controlSerializer_);\n>> + ASSERT(param_{{param.mojom_name}});\n>> +\n>> +{%- if pointer %}\n>> + if ({{param.mojom_name}})\n>> + *{{param.mojom_name}} = std::move(*param_{{param.mojom_name}});\n>> +{%- elif declare %}\n>> + {{param|name}} &{{param.mojom_name}} = *param_{{param.mojom_name}};\n>> {%- else %}\n>> - const size_t {{param.mojom_name}}FdStart = {{loop.previtem.mojom_name}}FdStart + {{loop.previtem.mojom_name}}FdsSize;\n>> + {{param.mojom_name}} = std::move(*param_{{param.mojom_name}});\n>> {%- endif %}\n>> -{%- endfor %}\n>> -{% for param in params %}\n>> - {%- if pointer %}\n>> - if ({{param.mojom_name}}) {\n>> -{{deserialize_param(param, pointer, loop, buf, fds, iter, data_size)|indent(16, True)}}\n>> - }\n>> - {%- else %}\n>> - {{param|name + \" \" if declare}}{{deserialize_param(param, pointer, loop, buf, fds, iter, data_size)|indent(8)}}\n>> - {%- endif %}\n> \n> This is beautiful.\n> \n> \n> What an amazing patch. I'm excited for v2.\n> \n> Thanks,\n> \n> Paul\n> \n> \n>> {% endfor %}\n>> {%- endmacro -%}\n>> diff --git a/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl b/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl\n>> index e316dd88a..9e9dd0ca6 100644\n>> --- a/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl\n>> +++ b/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl\n>> @@ -2,22 +2,6 @@\n>> # SPDX-License-Identifier: LGPL-2.1-or-later\n>> # Copyright (C) 2020, Google Inc.\n>> -#}\n>> -{#\n>> - # \\brief Verify that there is enough bytes to deserialize\n>> - #\n>> - # Generate code that verifies that \\a size is not greater than \\a dataSize.\n>> - # Otherwise log an error with \\a name and \\a typename.\n>> - #}\n>> -{%- macro check_data_size(size, dataSize, name, typename) %}\n>> - if ({{dataSize}} < {{size}}) {\n>> - LOG(IPADataSerializer, Error)\n>> - << \"Failed to deserialize \" << \"{{name}}\"\n>> - << \": not enough {{typename}}, expected \"\n>> - << ({{size}}) << \", got \" << ({{dataSize}});\n>> - return ret;\n>> - }\n>> -{%- endmacro %}\n>> -\n>> \n>> {#\n>> # \\brief Serialize a field into return vector\n>> @@ -42,15 +26,10 @@\n>> retData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n>> retFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n>> {%- elif field|is_controls %}\n>> - if (data.{{field.mojom_name}}.size() > 0) {\n>> - std::vector {{field.mojom_name}};\n>> - std::tie({{field.mojom_name}}, std::ignore) =\n>> - IPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n>> - appendPOD(retData, {{field.mojom_name}}.size());\n>> - retData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n>> - } else {\n>> - appendPOD(retData, 0);\n>> - }\n>> + std::vector {{field.mojom_name}};\n>> + std::tie({{field.mojom_name}}, std::ignore) =\n>> + IPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n>> + retData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n>> {%- elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n>> std::vector {{field.mojom_name}};\n>> {%- if field|has_fd %}\n>> @@ -65,10 +44,6 @@\n>> IPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n>> {%- else %}\n>> IPADataSerializer<{{field|name_full}}>::serialize(data.{{field.mojom_name}}, cs);\n>> - {%- endif %}\n>> - appendPOD(retData, {{field.mojom_name}}.size());\n>> - {%- if field|has_fd %}\n>> - appendPOD(retData, {{field.mojom_name}}Fds.size());\n>> {%- endif %}\n>> retData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n>> {%- if field|has_fd %}\n>> @@ -79,89 +54,6 @@\n>> {%- endif %}\n>> {%- endmacro %}\n>> \n>> -\n>> -{#\n>> - # \\brief Deserialize a field into return struct\n>> - #\n>> - # Generate code to deserialize \\a field into object ret.\n>> - # This code is meant to be used by the IPADataSerializer specialization.\n>> - #}\n>> -{%- macro deserializer_field(field, loop) %}\n>> -{% if field|is_pod or field|is_enum %}\n>> - {%- set field_size = (field|bit_width|int / 8)|int %}\n>> - {{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n>> - ret.{{field.mojom_name}} = IPADataSerializer<{{field|name_full}}>::deserialize(m, m + {{field_size}});\n>> - {%- if not loop.last %}\n>> - m += {{field_size}};\n>> - dataSize -= {{field_size}};\n>> - {%- endif %}\n>> -{% elif field|is_fd %}\n>> - {%- set field_size = 4 %}\n>> - {{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n>> - ret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field_size}}, n, n + 1, cs);\n>> - {%- if not loop.last %}\n>> - m += {{field_size}};\n>> - dataSize -= {{field_size}};\n>> - n += ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n>> - fdsSize -= ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n>> - {%- endif %}\n>> -{% elif field|is_controls %}\n>> - {%- set field_size = 4 %}\n>> - {{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n>> - const size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n>> - m += {{field_size}};\n>> - dataSize -= {{field_size}};\n>> - {%- set field_size = field.mojom_name + 'Size' -%}\n>> - {{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n>> - if ({{field.mojom_name}}Size > 0)\n>> - ret.{{field.mojom_name}} =\n>> - IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n>> - {%- if not loop.last %}\n>> - m += {{field_size}};\n>> - dataSize -= {{field_size}};\n>> - {%- endif %}\n>> -{% elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n>> - {%- set field_size = 4 %}\n>> - {{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n>> - const size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n>> - m += {{field_size}};\n>> - dataSize -= {{field_size}};\n>> - {%- if field|has_fd %}\n>> - {%- set field_size = 4 %}\n>> - {{- check_data_size(field_size, 'dataSize', field.mojom_name + 'FdsSize', 'data')}}\n>> - const size_t {{field.mojom_name}}FdsSize = readPOD(m, 0, dataEnd);\n>> - m += {{field_size}};\n>> - dataSize -= {{field_size}};\n>> - {{- check_data_size(field.mojom_name + 'FdsSize', 'fdsSize', field.mojom_name, 'fds')}}\n>> - {%- endif %}\n>> - {%- set field_size = field.mojom_name + 'Size' -%}\n>> - {{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n>> - ret.{{field.mojom_name}} =\n>> - {%- if field|is_str %}\n>> - IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size);\n>> - {%- elif field|has_fd and (field|is_array or field|is_map) %}\n>> - IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n>> - {%- elif field|has_fd and (not (field|is_array or field|is_map)) %}\n>> - IPADataSerializer<{{field|name_full}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n>> - {%- elif (not field|has_fd) and (field|is_array or field|is_map) %}\n>> - IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n>> - {%- else %}\n>> - IPADataSerializer<{{field|name_full}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n>> - {%- endif %}\n>> - {%- if not loop.last %}\n>> - m += {{field_size}};\n>> - dataSize -= {{field_size}};\n>> - {%- if field|has_fd %}\n>> - n += {{field.mojom_name}}FdsSize;\n>> - fdsSize -= {{field.mojom_name}}FdsSize;\n>> - {%- endif %}\n>> - {%- endif %}\n>> -{% else %}\n>> - /* Unknown deserialization for {{field.mojom_name}}. */\n>> -{%- endif %}\n>> -{%- endmacro %}\n>> -\n>> -\n>> {#\n>> # \\brief Serialize a struct\n>> #\n>> @@ -194,126 +86,30 @@\n>> \n>> \n>> {#\n>> - # \\brief Deserialize a struct that has fds\n>> + # \\brief Deserialize a struct\n>> #\n>> - # Generate code for IPADataSerializer specialization, for deserializing\n>> - # \\a struct, in the case that \\a struct has file descriptors.\n>> + # Generate code for IPADataSerializer specialization, for deserializing \\a struct.\n>> #}\n>> -{%- macro deserializer_fd(struct) %}\n>> - static {{struct|name_full}}\n>> - deserialize(std::vector &data,\n>> - std::vector &fds,\n>> -{%- if struct|needs_control_serializer %}\n>> - ControlSerializer *cs)\n>> -{%- else %}\n>> - ControlSerializer *cs = nullptr)\n>> -{%- endif %}\n>> - {\n>> - return IPADataSerializer<{{struct|name_full}}>::deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n>> - }\n>> -\n>> +{%- macro deserializer(struct) %}\n>> {# \\todo Don't inline this function #}\n>> - static {{struct|name_full}}\n>> - deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - std::vector::const_iterator fdsBegin,\n>> - std::vector::const_iterator fdsEnd,\n>> + [[nodiscard]] static std::optional<{{struct|name_full}}>\n>> + deserialize(SeriReader &reader,\n>> {%- if struct|needs_control_serializer %}\n>> ControlSerializer *cs)\n>> {%- else %}\n>> [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> {%- endif %}\n>> {\n>> - {{struct|name_full}} ret;\n>> - std::vector::const_iterator m = dataBegin;\n>> - std::vector::const_iterator n = fdsBegin;\n>> -\n>> - size_t dataSize = std::distance(dataBegin, dataEnd);\n>> - [[maybe_unused]] size_t fdsSize = std::distance(fdsBegin, fdsEnd);\n>> -{%- for field in struct.fields -%}\n>> -{{deserializer_field(field, loop)}}\n>> +{%- for field in struct.fields %}\n>> + auto {{field.mojom_name}} = IPADataSerializer<{{field|name_full}}>::deserialize(reader, cs);\n>> + if (!{{field.mojom_name}})\n>> + return {};\n>> {%- endfor %}\n>> - return ret;\n>> - }\n>> -{%- endmacro %}\n>> -\n>> -{#\n>> - # \\brief Deserialize a struct that has fds, using non-fd\n>> - #\n>> - # Generate code for IPADataSerializer specialization, for deserializing\n>> - # \\a struct, in the case that \\a struct has no file descriptors but requires\n>> - # deserializers with file descriptors.\n>> - #}\n>> -{%- macro deserializer_fd_simple(struct) %}\n>> - static {{struct|name_full}}\n>> - deserialize(std::vector &data,\n>> - [[maybe_unused]] std::vector &fds,\n>> - ControlSerializer *cs = nullptr)\n>> - {\n>> - return IPADataSerializer<{{struct|name_full}}>::deserialize(data.cbegin(), data.cend(), cs);\n>> - }\n>> -\n>> - static {{struct|name_full}}\n>> - deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> - [[maybe_unused]] std::vector::const_iterator fdsBegin,\n>> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> - ControlSerializer *cs = nullptr)\n>> - {\n>> - return IPADataSerializer<{{struct|name_full}}>::deserialize(dataBegin, dataEnd, cs);\n>> - }\n>> -{%- endmacro %}\n>> -\n>> -\n>> -{#\n>> - # \\brief Deserialize a struct that has no fds\n>> - #\n>> - # Generate code for IPADataSerializer specialization, for deserializing\n>> - # \\a struct, in the case that \\a struct does not have file descriptors.\n>> - #}\n>> -{%- macro deserializer_no_fd(struct) %}\n>> - static {{struct|name_full}}\n>> - deserialize(std::vector &data,\n>> -{%- if struct|needs_control_serializer %}\n>> - ControlSerializer *cs)\n>> -{%- else %}\n>> - ControlSerializer *cs = nullptr)\n>> -{%- endif %}\n>> - {\n>> - return IPADataSerializer<{{struct|name_full}}>::deserialize(data.cbegin(), data.cend(), cs);\n>> - }\n>> \n>> -{# \\todo Don't inline this function #}\n>> - static {{struct|name_full}}\n>> - deserialize(std::vector::const_iterator dataBegin,\n>> - std::vector::const_iterator dataEnd,\n>> -{%- if struct|needs_control_serializer %}\n>> - ControlSerializer *cs)\n>> -{%- else %}\n>> - [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> -{%- endif %}\n>> - {\n>> - {{struct|name_full}} ret;\n>> - std::vector::const_iterator m = dataBegin;\n>> -\n>> - size_t dataSize = std::distance(dataBegin, dataEnd);\n>> -{%- for field in struct.fields -%}\n>> -{{deserializer_field(field, loop)}}\n>> + return { {\n>> +{%- for field in struct.fields %}\n>> + std::move(*{{field.mojom_name}}),\n>> {%- endfor %}\n>> - return ret;\n>> + } };\n>> }\n>> {%- endmacro %}\n>> -\n>> -{#\n>> - # \\brief Deserialize a struct\n>> - #\n>> - # Generate code for IPADataSerializer specialization, for deserializing \\a struct.\n>> - #}\n>> -{%- macro deserializer(struct) %}\n>> -{%- if struct|has_fd %}\n>> -{{deserializer_fd(struct)}}\n>> -{%- else %}\n>> -{{deserializer_no_fd(struct)}}\n>> -{{deserializer_fd_simple(struct)}}\n>> -{%- endif %}\n>> -{%- endmacro %}\n>> -- \n>> 2.49.0\n>>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A1E8BBD78E\n\tfor ;\n\tMon, 19 May 2025 08:51:16 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id DCD1B68D7A;\n\tMon, 19 May 2025 10:51:15 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 82E48616A3\n\tfor ;\n\tMon, 19 May 2025 10:51:14 +0200 (CEST)","from [192.168.33.12] (185.221.142.248.nat.pool.zt.hu\n\t[185.221.142.248])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 901E0496;\n\tMon, 19 May 2025 10:50:54 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"rkJZEFPn\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1747644654;\n\tbh=kcr98cAaiHbhMQ9Ar0VfZOAvxPIGOrPWjZBFjJnxk+c=;\n\th=Date:Subject:To:References:From:Cc:In-Reply-To:From;\n\tb=rkJZEFPnPWcdLFBaxwliWoojlb8JlmJeH5TQN4569tWouXsgfjKiQf7dBc+KuqqEt\n\tzUMD0b87iKktreRgFJuCzGXEe1KKfhschlI9kXW/o4GjYCFgVmunfpFFSaatGx330A\n\tXX3ky84NkJSZ6hBl1tpjCrrYONfo3p0j/GSKoUoI=","Message-ID":"<9b8e5e89-fcf6-4661-a428-3d22c1fa0a37@ideasonboard.com>","Date":"Mon, 19 May 2025 10:51:11 +0200","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [RFC PATCH v1 8/8] utils: codegen: ipc: Simplify deserialization","To":"Paul Elder ","References":"<20250515120012.3127231-1-barnabas.pocze@ideasonboard.com>\n\t<20250515120012.3127231-9-barnabas.pocze@ideasonboard.com>\n\t<174741664081.476729.1844022348649238166@calcite>","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Content-Language":"en-US, hu-HU","Cc":"libcamera-devel@lists.libcamera.org","In-Reply-To":"<174741664081.476729.1844022348649238166@calcite>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":34299,"web_url":"https://patchwork.libcamera.org/comment/34299/","msgid":"","date":"2025-05-20T15:18:58","subject":"Re: [RFC PATCH v1 8/8] utils: codegen: ipc: Simplify deserialization","submitter":{"id":143,"url":"https://patchwork.libcamera.org/api/people/143/","name":"Jacopo Mondi","email":"jacopo.mondi@ideasonboard.com"},"content":"Hi Barnabás\n\nOn Thu, May 15, 2025 at 02:00:12PM +0200, Barnabás Pőcze wrote:\n> First, introduce the `SeriReader` type, which is a collection\n> of bytes and file descriptors, with the appropriate methods to\n> consume the first couples bytes / file descriptors.\n>\n> Then a new method is added to `IPCMessage` that returns an appropriately\n> constructed `SeriReader` for parsing the message contents.\n>\n> Then three of the four `deserialize()` overloads are removed, the\n> remaining one is converted to have a single `SeriReader` and an\n> optional `ControlSerializer` as arguments.\n>\n> The remaining `deserialize()` function is also changed to return an\n> `std::optional` to be able to report deserialization failure.\n>\n> There is also a more fundamental change in the serialization: previously,\n> the number of bytes taken up by an item has been written before the serialized\n> bytes (and conditionally the number of file descriptors) when the item is\n> serialized as part of a struct, array, map, function parameter list. This\n> is changed: the number of bytes and file descriptors are *not* serialized\n> into the final buffer. This affords some simplification of the serialization\n> related code paths, but most importantly, it greatly simplifies and unifies\n> how an object is (de)serialized because the deserialization of every object\n> becomes completely self-contained.\n>\n> As a consequence of that, strings now include their lengths as part of the\n> string serialization, and it is not left to an \"upper\" layer.\n>\n> Another consequence is that an \"out parameter\" of a remote function call\n> must be deserialized if a later out parameter is needed, even if itself\n> is not. This does not appear to be a great limitation since in all\n> situations presently none of the out parameters are ignored.\n>\n> Finally, the code generation templates are adapted to the above changes.\n> This allows the simplification of the deserialization templates as now\n> calling `IPADataSerializer::deserialize(reader, &controlSerializer_)`\n> is appropriate for any type.\n\nWow, there is quite some to unpack here.\n\nI wonder if the above changes related to the API, such as using\noptional<> might be broken out to make things easier for review, but I\npresume it might be hard.\n\nAnyway, first question first: a class for \"Deserializing\" (reading\nfrom an array of Bytes) has been added, while the \"Serialization\"\n(marshalling data to an array of Bytes) is still left to each\nspecialization of the IPADataSerializer<> class.\n\nWas this by choice ?\n\nThe question also comes from the fact we'll need a serializer for all\nthe work to be done on ControlList in the public API. Do you plan to\nuse SeriReader in that context as well ?\n\n>\n> Bug: https://bugs.libcamera.org/show_bug.cgi?id=269\n> Signed-off-by: Barnabás Pőcze \n> ---\n> .../libcamera/internal/ipa_data_serializer.h | 228 +++--------\n> include/libcamera/internal/ipc_pipe.h | 3 +\n> include/libcamera/internal/serialization.h | 91 +++++\n> src/libcamera/ipa_data_serializer.cpp | 356 ++++--------------\n> src/libcamera/ipc_pipe.cpp | 5 +\n> test/ipc/unixsocket_ipc.cpp | 18 +-\n> .../generated_serializer_test.cpp | 17 +-\n> .../ipa_data_serializer_test.cpp | 32 +-\n> .../module_ipa_proxy.cpp.tmpl | 30 +-\n> .../module_ipa_proxy.h.tmpl | 5 +-\n> .../module_ipa_proxy_worker.cpp.tmpl | 5 +-\n> .../libcamera_templates/proxy_functions.tmpl | 113 +-----\n> .../libcamera_templates/serializer.tmpl | 238 +-----------\n> 13 files changed, 316 insertions(+), 825 deletions(-)\n> create mode 100644 include/libcamera/internal/serialization.h\n>\n> diff --git a/include/libcamera/internal/ipa_data_serializer.h b/include/libcamera/internal/ipa_data_serializer.h\n> index 564f59e25..0d03729a1 100644\n> --- a/include/libcamera/internal/ipa_data_serializer.h\n> +++ b/include/libcamera/internal/ipa_data_serializer.h\n> @@ -7,6 +7,7 @@\n>\n> #pragma once\n>\n> +#include \n> #include \n> #include \n> #include \n> @@ -23,6 +24,7 @@\n> #include \n>\n> #include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/serialization.h\"\n>\n> namespace libcamera {\n>\n> @@ -39,26 +41,6 @@ void appendPOD(std::vector &vec, T val)\n> \tmemcpy(&*(vec.end() - byteWidth), &val, byteWidth);\n> }\n>\n> -template -\t std::enable_if_t> * = nullptr>\n> -T readPOD(std::vector::const_iterator it, size_t pos,\n> -\t std::vector::const_iterator end)\n> -{\n> -\tASSERT(pos + it < end);\n> -\n> -\tT ret = 0;\n> -\tmemcpy(&ret, &(*(it + pos)), sizeof(ret));\n> -\n> -\treturn ret;\n> -}\n> -\n> -template -\t std::enable_if_t> * = nullptr>\n> -T readPOD(std::vector &vec, size_t pos)\n> -{\n> -\treturn readPOD(vec.cbegin(), pos, vec.end());\n> -}\n> -\n> } /* namespace */\n>\n> template\n> @@ -68,20 +50,8 @@ public:\n> \tstatic std::tuple, std::vector>\n> \tserialize(const T &data, ControlSerializer *cs = nullptr);\n>\n> -\tstatic T deserialize(const std::vector &data,\n> -\t\t\t ControlSerializer *cs = nullptr);\n> -\tstatic T deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t ControlSerializer *cs = nullptr);\n> -\n> -\tstatic T deserialize(const std::vector &data,\n> -\t\t\t const std::vector &fds,\n> -\t\t\t ControlSerializer *cs = nullptr);\n> -\tstatic T deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t std::vector::const_iterator fdsBegin,\n> -\t\t\t std::vector::const_iterator fdsEnd,\n> -\t\t\t ControlSerializer *cs = nullptr);\n> +\t[[nodiscard]] static std::optional\n> +\tdeserialize(SeriReader &reader, ControlSerializer *cs = nullptr);\n> };\n>\n> #ifndef __DOXYGEN__\n> @@ -121,9 +91,6 @@ public:\n> \t\t\tstd::tie(dvec, fvec) =\n> \t\t\t\tIPADataSerializer::serialize(it, cs);\n>\n> -\t\t\tappendPOD(dataVec, dvec.size());\n> -\t\t\tappendPOD(dataVec, fvec.size());\n> -\n\nYou should update the comment that describes the serialized format,\nfor all specializations where the serialized formats is changed to\nremove the in-line sizes\n\n\n> \t\t\tdataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n> \t\t\tfdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n> \t\t}\n> @@ -131,52 +98,25 @@ public:\n> \t\treturn { dataVec, fdsVec };\n> \t}\n>\n> -\tstatic std::vector deserialize(std::vector &data, ControlSerializer *cs = nullptr)\n> +\t[[nodiscard]] static std::optional>\n> +\tdeserialize(SeriReader &reader, ControlSerializer *cs = nullptr)\n> \t{\n> -\t\treturn deserialize(data.cbegin(), data.cend(), cs);\n> -\t}\n> +\t\tuint32_t vecLen;\n> +\t\tif (!reader.read(vecLen))\n> +\t\t\treturn {};\n\nTook me a while to understand this one. You're reading the first 4\nbytes integer that contains the serialized vector size.\n\nThe API of:\n\n\ttemplate\n\t[[nodiscard]] bool read(Ts &...xs) {}\n\nit's a bit alien to me, as\n1) It returns a pointer to the serialized data, and that's fine, but\n2) Loads the content of the serialized data (of size sizeof...(xs))\n into ...(xs)\n\nAs an aging C programmer I would have expected an API that uses a\npointer for an output argument, but this would certainly make\nimpractical to use the (Ts &...xs) magic...\n\n>\n> -\tstatic std::vector deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t\t\t ControlSerializer *cs = nullptr)\n> -\t{\n> -\t\tstd::vector fds;\n> -\t\treturn deserialize(dataBegin, dataEnd, fds.cbegin(), fds.cend(), cs);\n> -\t}\n> +\t\tstd::vector ret;\n> +\t\tret.reserve(vecLen);\n\nIs\n std::vector ret(vecLen)\n\ndifferent ?\n\n>\n> -\tstatic std::vector deserialize(std::vector &data, std::vector &fds,\n> -\t\t\t\t\t ControlSerializer *cs = nullptr)\n> -\t{\n> -\t\treturn deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n> -\t}\n> -\n> -\tstatic std::vector deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t\t\t std::vector::const_iterator fdsBegin,\n> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> -\t\t\t\t\t ControlSerializer *cs = nullptr)\n> -\t{\n> -\t\tuint32_t vecLen = readPOD(dataBegin, 0, dataEnd);\n> -\t\tstd::vector ret(vecLen);\n> -\n> -\t\tstd::vector::const_iterator dataIter = dataBegin + 4;\n> -\t\tstd::vector::const_iterator fdIter = fdsBegin;\n> \t\tfor (uint32_t i = 0; i < vecLen; i++) {\n> -\t\t\tuint32_t sizeofData = readPOD(dataIter, 0, dataEnd);\n> -\t\t\tuint32_t sizeofFds = readPOD(dataIter, 4, dataEnd);\n> -\t\t\tdataIter += 8;\n> -\n> -\t\t\tret[i] = IPADataSerializer::deserialize(dataIter,\n> -\t\t\t\t\t\t\t\t dataIter + sizeofData,\n> -\t\t\t\t\t\t\t\t fdIter,\n> -\t\t\t\t\t\t\t\t fdIter + sizeofFds,\n> -\t\t\t\t\t\t\t\t cs);\n> -\n> -\t\t\tdataIter += sizeofData;\n> -\t\t\tfdIter += sizeofFds;\n> +\t\t\tauto item = IPADataSerializer::deserialize(reader, cs);\n> +\t\t\tif (!item)\n> +\t\t\t\treturn {};\n> +\n> +\t\t\tret.emplace_back(std::move(*item));\n\nDoes std::move() make any practical difference ? The data have to\ncopied to one container to the other anyway, don't they ?\n\n> \t\t}\n>\n> -\t\treturn ret;\n> +\t\treturn std::move(ret);\n\nHere as well, does move() make any difference ? I presume it does if\nthe type V underlying the vector is move-constructable ?\n\n> \t}\n> };\n>\n> @@ -218,18 +158,12 @@ public:\n> \t\t\tstd::tie(dvec, fvec) =\n> \t\t\t\tIPADataSerializer::serialize(it.first, cs);\n>\n> -\t\t\tappendPOD(dataVec, dvec.size());\n> -\t\t\tappendPOD(dataVec, fvec.size());\n> -\n> \t\t\tdataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n> \t\t\tfdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n>\n> \t\t\tstd::tie(dvec, fvec) =\n> \t\t\t\tIPADataSerializer::serialize(it.second, cs);\n>\n> -\t\t\tappendPOD(dataVec, dvec.size());\n> -\t\t\tappendPOD(dataVec, fvec.size());\n> -\n\nPlease update the comment here as well\n\n> \t\t\tdataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n> \t\t\tfdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n> \t\t}\n> @@ -237,63 +171,25 @@ public:\n> \t\treturn { dataVec, fdsVec };\n> \t}\n>\n> -\tstatic std::map deserialize(std::vector &data, ControlSerializer *cs = nullptr)\n> -\t{\n> -\t\treturn deserialize(data.cbegin(), data.cend(), cs);\n> -\t}\n> -\n> -\tstatic std::map deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t\t\t ControlSerializer *cs = nullptr)\n> +\t[[nodiscard]] static std::optional>\n> +\tdeserialize(SeriReader &reader, ControlSerializer *cs = nullptr)\n> \t{\n> -\t\tstd::vector fds;\n> -\t\treturn deserialize(dataBegin, dataEnd, fds.cbegin(), fds.cend(), cs);\n> -\t}\n> +\t\tuint32_t mapLen;\n> +\t\tif (!reader.read(mapLen))\n> +\t\t\treturn {};\n>\n> -\tstatic std::map deserialize(std::vector &data, std::vector &fds,\n> -\t\t\t\t\t ControlSerializer *cs = nullptr)\n> -\t{\n> -\t\treturn deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n> -\t}\n> -\n> -\tstatic std::map deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t\t\t std::vector::const_iterator fdsBegin,\n> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> -\t\t\t\t\t ControlSerializer *cs = nullptr)\n> -\t{\n> \t\tstd::map ret;\n>\n> -\t\tuint32_t mapLen = readPOD(dataBegin, 0, dataEnd);\n> -\n> -\t\tstd::vector::const_iterator dataIter = dataBegin + 4;\n> -\t\tstd::vector::const_iterator fdIter = fdsBegin;\n> \t\tfor (uint32_t i = 0; i < mapLen; i++) {\n> -\t\t\tuint32_t sizeofData = readPOD(dataIter, 0, dataEnd);\n> -\t\t\tuint32_t sizeofFds = readPOD(dataIter, 4, dataEnd);\n> -\t\t\tdataIter += 8;\n> -\n> -\t\t\tK key = IPADataSerializer::deserialize(dataIter,\n> -\t\t\t\t\t\t\t\t dataIter + sizeofData,\n> -\t\t\t\t\t\t\t\t fdIter,\n> -\t\t\t\t\t\t\t\t fdIter + sizeofFds,\n> -\t\t\t\t\t\t\t\t cs);\n> -\n> -\t\t\tdataIter += sizeofData;\n> -\t\t\tfdIter += sizeofFds;\n> -\t\t\tsizeofData = readPOD(dataIter, 0, dataEnd);\n> -\t\t\tsizeofFds = readPOD(dataIter, 4, dataEnd);\n> -\t\t\tdataIter += 8;\n> -\n> -\t\t\tconst V value = IPADataSerializer::deserialize(dataIter,\n> -\t\t\t\t\t\t\t\t\t dataIter + sizeofData,\n> -\t\t\t\t\t\t\t\t\t fdIter,\n> -\t\t\t\t\t\t\t\t\t fdIter + sizeofFds,\n> -\t\t\t\t\t\t\t\t\t cs);\n> -\t\t\tret.insert({ key, value });\n> -\n> -\t\t\tdataIter += sizeofData;\n> -\t\t\tfdIter += sizeofFds;\n> +\t\t\tauto key = IPADataSerializer::deserialize(reader, cs);\n> +\t\t\tif (!key)\n> +\t\t\t\treturn {};\n> +\n> +\t\t\tauto value = IPADataSerializer::deserialize(reader, cs);\n> +\t\t\tif (!value)\n> +\t\t\t\treturn {};\n> +\n> +\t\t\tret.try_emplace(std::move(*key), std::move(*value));\n\nThe number of lines saved is impressive!\n\n> \t\t}\n>\n> \t\treturn ret;\n> @@ -314,33 +210,14 @@ public:\n> \t\treturn { dataVec, {} };\n> \t}\n>\n> -\tstatic Flags deserialize(std::vector &data,\n> -\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> -\t{\n> -\t\treturn deserialize(data.cbegin(), data.cend());\n> -\t}\n> -\n> -\tstatic Flags deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +\t[[nodiscard]] static std::optional>\n> +\tdeserialize(SeriReader &reader, [[maybe_unused]] ControlSerializer *cs = nullptr)\n> \t{\n> -\t\treturn Flags{ static_cast(readPOD(dataBegin, 0, dataEnd)) };\n> -\t}\n> +\t\tuint32_t value;\n> +\t\tif (!reader.read(value))\n> +\t\t\treturn {};\n>\n> -\tstatic Flags deserialize(std::vector &data,\n> -\t\t\t\t [[maybe_unused]] std::vector &fds,\n> -\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> -\t{\n> -\t\treturn deserialize(data.cbegin(), data.cend());\n> -\t}\n> -\n> -\tstatic Flags deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> -\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> -\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> -\t{\n> -\t\treturn deserialize(dataBegin, dataEnd);\n> +\t\treturn Flags{ static_cast(value) };\n> \t}\n> };\n>\n> @@ -360,33 +237,14 @@ public:\n> \t\treturn { dataVec, {} };\n> \t}\n>\n> -\tstatic E deserialize(std::vector &data,\n> -\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +\t[[nodiscard]] static std::optional\n> +\tdeserialize(SeriReader &reader, [[maybe_unused]] ControlSerializer *cs = nullptr)\n> \t{\n> -\t\treturn deserialize(data.cbegin(), data.cend());\n> -\t}\n> +\t\tU value;\n> +\t\tif (!reader.read(value))\n> +\t\t\treturn {};\n>\n> -\tstatic E deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> -\t{\n> -\t\treturn static_cast(readPOD(dataBegin, 0, dataEnd));\n> -\t}\n> -\n> -\tstatic E deserialize(std::vector &data,\n> -\t\t\t [[maybe_unused]] std::vector &fds,\n> -\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> -\t{\n> -\t\treturn deserialize(data.cbegin(), data.cend());\n> -\t}\n> -\n> -\tstatic E deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> -\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> -\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> -\t{\n> -\t\treturn deserialize(dataBegin, dataEnd);\n> +\t\treturn static_cast(value);\n> \t}\n> };\n>\n> diff --git a/include/libcamera/internal/ipc_pipe.h b/include/libcamera/internal/ipc_pipe.h\n> index 418c4622f..2b6cde042 100644\n> --- a/include/libcamera/internal/ipc_pipe.h\n> +++ b/include/libcamera/internal/ipc_pipe.h\n> @@ -14,6 +14,7 @@\n> #include \n>\n> #include \"libcamera/internal/ipc_unixsocket.h\"\n> +#include \"libcamera/internal/serialization.h\"\n>\n> namespace libcamera {\n>\n> @@ -40,6 +41,8 @@ public:\n> \tconst std::vector &data() const { return data_; }\n> \tconst std::vector &fds() const { return fds_; }\n>\n> +\tSeriReader reader() const { return { data(), fds() }; }\n> +\n> private:\n> \tHeader header_;\n>\n> diff --git a/include/libcamera/internal/serialization.h b/include/libcamera/internal/serialization.h\n> new file mode 100644\n> index 000000000..daa7e5438\n> --- /dev/null\n> +++ b/include/libcamera/internal/serialization.h\n> @@ -0,0 +1,91 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2025, Ideas On Board Oy\n> + *\n> + * Data (de)serialization helper structures\n> + */\n> +#pragma once\n> +\n\nMissing a few includes\n\n> +#include \n> +#include \n> +#include \n> +\n> +#include \n> +#include \n> +\n> +namespace libcamera {\n> +\n> +#ifndef __DOXYGEN__\n> +class SeriReader\n> +{\n> +public:\n> +\tSeriReader(Span data, Span fds = {})\n> +\t\t: data_(data),\n> +\t\t fds_(fds)\n> +\t{\n> +\t}\n> +\n> +\tSeriReader(Span data, Span fds = {})\n> +\t\t: data_(reinterpret_cast(data.data()),\n> +\t\t\treinterpret_cast(data.data() + data.size())),\n\nah so uint8_t is not directly convertible to a byte ?\n\n> +\t\t fds_(fds)\n> +\t{\n> +\t}\n> +\n> +\t[[nodiscard]] const std::byte *consume(std::size_t s)\n> +\t{\n> +\t\tif (data_.size() < s)\n> +\t\t\treturn nullptr;\n> +\n> +\t\tconst auto *p = data_.data();\n> +\t\tdata_ = data_.subspan(s);\n> +\n> +\t\treturn p;\n> +\t}\n> +\n> +\ttemplate\n> +\t[[nodiscard]] bool read(Ts &...xs)\n\nMy head didn't explode just because you already showed me this trick\n\n> +\t{\n> +\t\tstatic_assert((std::is_trivially_copyable_v && ...));\n> +\n> +\t\tconst auto *p = consume((sizeof(xs) + ...));\n\nSo we read the whole space required to deserialize multiple xs\n\n> +\t\tif (p)\n> +\t\t\t((memcpy(&xs, p, sizeof(xs)), p += sizeof(xs)), ...);\n\nAnd we copy them one by one\n\nbrilliant\n\n> +\n> +\t\treturn p;\n\n\n\n> +\t}\n> +\n> +\ttemplate\n> +\t[[nodiscard]] auto read()\n> +\t{\n> +\t\tstd::tuple xs;\n> +\t\tbool ok = std::apply([&](auto &...vs) {\n> +\t\t\treturn read(vs...);\n> +\t\t}, xs);\n> +\n> +\t\tif constexpr (sizeof...(Ts) == 1)\n> +\t\t\treturn ok ? std::optional(std::get<0>(xs)) : std::nullopt;\n> +\t\telse\n> +\t\t\treturn ok ? std::optional(xs) : std::nullopt;\n\nIf this last version doesn't work for sizeof...(Ts) == 1\n(I know that, I tried removing the if branch) does it mean that\n\n\t\tif constexpr (sizeof...(Ts) == 1)\n\nis compile-time evaluated and the compiler knows what option to pick.\n\nNow the main question, do you foresee an use for template argument\npacks ? As far as I understood it, all the exiting code works with a\nsingle argument, or have I missed something ?\n\nI'll stop the review here for now, enough material to ponder up for me :)\n\nThanks\n j\n\n> +\t}\n> +\n> +\t[[nodiscard]] const SharedFD *nextFd()\n> +\t{\n> +\t\tif (fds_.empty())\n> +\t\t\treturn nullptr;\n> +\n> +\t\tconst auto *p = fds_.data();\n> +\t\tfds_ = fds_.subspan(1);\n> +\n> +\t\treturn p;\n> +\t}\n> +\n> +\t[[nodiscard]] bool empty() const { return data_.empty() && fds_.empty(); }\n> +\n> +private:\n> +\tSpan data_;\n> +\tSpan fds_;\n> +};\n> +#endif\n> +\n> +} /* namespace libcamera */\n> diff --git a/src/libcamera/ipa_data_serializer.cpp b/src/libcamera/ipa_data_serializer.cpp\n> index 0537f785b..979a1e0db 100644\n> --- a/src/libcamera/ipa_data_serializer.cpp\n> +++ b/src/libcamera/ipa_data_serializer.cpp\n> @@ -50,42 +50,6 @@ namespace {\n> * generated IPA proxies.\n> */\n>\n> -/**\n> - * \\fn template T readPOD(std::vector::iterator it, size_t pos,\n> - * \t\t\t\t std::vector::iterator end)\n> - * \\brief Read POD from byte vector, in little-endian order\n> - * \\tparam T Type of POD to read\n> - * \\param[in] it Iterator of byte vector to read from\n> - * \\param[in] pos Index in byte vector to read from\n> - * \\param[in] end Iterator marking end of byte vector\n> - *\n> - * This function is meant to be used by the IPA data serializer, and the\n> - * generated IPA proxies.\n> - *\n> - * If the \\a pos plus the byte-width of the desired POD is past \\a end, it is\n> - * a fata error will occur, as it means there is insufficient data for\n> - * deserialization, which should never happen.\n> - *\n> - * \\return The POD read from \\a it at index \\a pos\n> - */\n> -\n> -/**\n> - * \\fn template T readPOD(std::vector &vec, size_t pos)\n> - * \\brief Read POD from byte vector, in little-endian order\n> - * \\tparam T Type of POD to read\n> - * \\param[in] vec Byte vector to read from\n> - * \\param[in] pos Index in vec to start reading from\n> - *\n> - * This function is meant to be used by the IPA data serializer, and the\n> - * generated IPA proxies.\n> - *\n> - * If the \\a pos plus the byte-width of the desired POD is past the end of\n> - * \\a vec, a fatal error will occur, as it means there is insufficient data\n> - * for deserialization, which should never happen.\n> - *\n> - * \\return The POD read from \\a vec at index \\a pos\n> - */\n> -\n> } /* namespace */\n>\n> /**\n> @@ -106,80 +70,13 @@ namespace {\n>\n> /**\n> * \\fn template IPADataSerializer::deserialize(\n> - * \tconst std::vector &data,\n> + * \tSeriReader &reader,\n> * \tControlSerializer *cs = nullptr)\n> - * \\brief Deserialize byte vector into an object\n> + * \\brief Deserialize bytes and file descriptors vector into an object\n> * \\tparam T Type of object to deserialize to\n> - * \\param[in] data Byte vector to deserialize from\n> + * \\param[in] reader Source of bytes and file descriptors\n> * \\param[in] cs ControlSerializer\n> *\n> - * This version of deserialize() can be used if the object type \\a T and its\n> - * members don't have any SharedFD.\n> - *\n> - * \\a cs is only necessary if the object type \\a T or its members contain\n> - * ControlList or ControlInfoMap.\n> - *\n> - * \\return The deserialized object\n> - */\n> -\n> -/**\n> - * \\fn template IPADataSerializer::deserialize(\n> - * \tstd::vector::const_iterator dataBegin,\n> - * \tstd::vector::const_iterator dataEnd,\n> - * \tControlSerializer *cs = nullptr)\n> - * \\brief Deserialize byte vector into an object\n> - * \\tparam T Type of object to deserialize to\n> - * \\param[in] dataBegin Begin iterator of byte vector to deserialize from\n> - * \\param[in] dataEnd End iterator of byte vector to deserialize from\n> - * \\param[in] cs ControlSerializer\n> - *\n> - * This version of deserialize() can be used if the object type \\a T and its\n> - * members don't have any SharedFD.\n> - *\n> - * \\a cs is only necessary if the object type \\a T or its members contain\n> - * ControlList or ControlInfoMap.\n> - *\n> - * \\return The deserialized object\n> - */\n> -\n> -/**\n> - * \\fn template IPADataSerializer::deserialize(\n> - * \tconst std::vector &data,\n> - * \tconst std::vector &fds,\n> - * \tControlSerializer *cs = nullptr)\n> - * \\brief Deserialize byte vector and fd vector into an object\n> - * \\tparam T Type of object to deserialize to\n> - * \\param[in] data Byte vector to deserialize from\n> - * \\param[in] fds Fd vector to deserialize from\n> - * \\param[in] cs ControlSerializer\n> - *\n> - * This version of deserialize() (or the iterator version) must be used if\n> - * the object type \\a T or its members contain SharedFD.\n> - *\n> - * \\a cs is only necessary if the object type \\a T or its members contain\n> - * ControlList or ControlInfoMap.\n> - *\n> - * \\return The deserialized object\n> - */\n> -\n> -/**\n> - * \\fn template IPADataSerializer::deserialize(\n> - * \tstd::vector::const_iterator dataBegin,\n> - * \tstd::vector::const_iterator dataEnd,\n> - * \tstd::vector::const_iterator fdsBegin,\n> - * \tstd::vector::const_iterator fdsEnd,\n> - * \tControlSerializer *cs = nullptr)\n> - * \\brief Deserialize byte vector and fd vector into an object\n> - * \\tparam T Type of object to deserialize to\n> - * \\param[in] dataBegin Begin iterator of byte vector to deserialize from\n> - * \\param[in] dataEnd End iterator of byte vector to deserialize from\n> - * \\param[in] fdsBegin Begin iterator of fd vector to deserialize from\n> - * \\param[in] fdsEnd End iterator of fd vector to deserialize from\n> - * \\param[in] cs ControlSerializer\n> - *\n> - * This version of deserialize() (or the vector version) must be used if\n> - * the object type \\a T or its members contain SharedFD.\n> - *\n> * \\a cs is only necessary if the object type \\a T or its members contain\n> * ControlList or ControlInfoMap.\n> *\n> @@ -202,37 +99,11 @@ IPADataSerializer::serialize(const type &data,\t\t\t\\\n> }\t\t\t\t\t\t\t\t\t\\\n> \t\t\t\t\t\t\t\t\t\\\n> template<>\t\t\t\t\t\t\t\t\\\n> -type IPADataSerializer::deserialize(std::vector::const_iterator dataBegin, \\\n> -\t\t\t\t\t std::vector::const_iterator dataEnd, \\\n> -\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs) \\\n> +std::optional IPADataSerializer::deserialize(SeriReader &reader, \\\n> +\t\t\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs) \\\n> {\t\t\t\t\t\t\t\t\t\\\n> -\treturn readPOD(dataBegin, 0, dataEnd);\t\t\t\\\n> +\treturn reader.read();\t\t\t\t\t\\\n> }\t\t\t\t\t\t\t\t\t\\\n> -\t\t\t\t\t\t\t\t\t\\\n> -template<>\t\t\t\t\t\t\t\t\\\n> -type IPADataSerializer::deserialize(const std::vector &data, \\\n> -\t\t\t\t\t ControlSerializer *cs)\t\\\n> -{\t\t\t\t\t\t\t\t\t\\\n> -\treturn deserialize(data.cbegin(), data.end(), cs);\t\t\\\n> -}\t\t\t\t\t\t\t\t\t\\\n> -\t\t\t\t\t\t\t\t\t\\\n> -template<>\t\t\t\t\t\t\t\t\\\n> -type IPADataSerializer::deserialize(const std::vector &data, \\\n> -\t\t\t\t\t [[maybe_unused]] const std::vector &fds, \\\n> -\t\t\t\t\t ControlSerializer *cs)\t\\\n> -{\t\t\t\t\t\t\t\t\t\\\n> -\treturn deserialize(data.cbegin(), data.end(), cs);\t\t\\\n> -}\t\t\t\t\t\t\t\t\t\\\n> -\t\t\t\t\t\t\t\t\t\\\n> -template<>\t\t\t\t\t\t\t\t\\\n> -type IPADataSerializer::deserialize(std::vector::const_iterator dataBegin, \\\n> -\t\t\t\t\t std::vector::const_iterator dataEnd, \\\n> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin, \\\n> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd, \\\n> -\t\t\t\t\t ControlSerializer *cs)\t\\\n> -{\t\t\t\t\t\t\t\t\t\\\n> -\treturn deserialize(dataBegin, dataEnd, cs);\t\t\t\\\n> -}\n>\n> DEFINE_POD_SERIALIZER(bool)\n> DEFINE_POD_SERIALIZER(uint8_t)\n> @@ -256,44 +127,29 @@ std::tuple, std::vector>\n> IPADataSerializer::serialize(const std::string &data,\n> \t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n> {\n> -\treturn { { data.cbegin(), data.end() }, {} };\n> -}\n> +\tstd::vector dataVec;\n>\n> -template<>\n> -std::string\n> -IPADataSerializer::deserialize(const std::vector &data,\n> -\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n> -{\n> -\treturn { data.cbegin(), data.cend() };\n> -}\n> +\tASSERT(data.size() <= std::numeric_limits::max());\n> +\tappendPOD(dataVec, data.size());\n> +\tdataVec.insert(dataVec.end(), data.c_str(), data.c_str() + data.size());\n>\n> -template<>\n> -std::string\n> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n> -{\n> -\treturn { dataBegin, dataEnd };\n> +\treturn { dataVec, {} };\n> }\n>\n> template<>\n> -std::string\n> -IPADataSerializer::deserialize(const std::vector &data,\n> -\t\t\t\t\t [[maybe_unused]] const std::vector &fds,\n> +std::optional\n> +IPADataSerializer::deserialize(SeriReader &reader,\n> \t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n> {\n> -\treturn { data.cbegin(), data.cend() };\n> -}\n> +\tuint32_t length;\n> +\tif (!reader.read(length))\n> +\t\treturn {};\n>\n> -template<>\n> -std::string\n> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> -\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n> -{\n> -\treturn { dataBegin, dataEnd };\n> +\tconst auto *p = reader.consume(length);\n> +\tif (!p)\n> +\t\treturn {};\n> +\n> +\treturn { { reinterpret_cast(p), std::size_t(length) } };\n> }\n>\n> /*\n> @@ -356,73 +212,43 @@ IPADataSerializer::serialize(const ControlList &data, ControlSerial\n> }\n>\n> template<>\n> -ControlList\n> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n> +std::optional\n> +IPADataSerializer::deserialize(SeriReader &reader,\n> \t\t\t\t\t ControlSerializer *cs)\n> {\n> \tif (!cs)\n> \t\tLOG(IPADataSerializer, Fatal)\n> \t\t\t<< \"ControlSerializer not provided for deserialization of ControlList\";\n>\n> -\tif (std::distance(dataBegin, dataEnd) < 8)\n> -\t\treturn {};\n> -\n> -\tuint32_t infoDataSize = readPOD(dataBegin, 0, dataEnd);\n> -\tuint32_t listDataSize = readPOD(dataBegin, 4, dataEnd);\n> -\n> -\tstd::vector::const_iterator it = dataBegin + 8;\n> -\n> -\tif (infoDataSize + listDataSize < infoDataSize ||\n> -\t static_cast(std::distance(it, dataEnd)) < infoDataSize + listDataSize)\n> +\tuint32_t infoDataSize, listDataSize;\n> +\tif (!reader.read(infoDataSize, listDataSize))\n> \t\treturn {};\n>\n> \tif (infoDataSize > 0) {\n> -\t\tByteStreamBuffer buffer(&*it, infoDataSize);\n> +\t\tconst auto *p = reader.consume(infoDataSize);\n> +\t\tif (!p)\n> +\t\t\treturn {};\n> +\n> +\t\tByteStreamBuffer buffer(reinterpret_cast(p), infoDataSize);\n> \t\tControlInfoMap map = cs->deserialize(buffer);\n> \t\t/* It's fine if map is empty. */\n> \t\tif (buffer.overflow()) {\n> \t\t\tLOG(IPADataSerializer, Error)\n> \t\t\t\t<< \"Failed to deserialize ControlLists's ControlInfoMap: buffer overflow\";\n> -\t\t\treturn ControlList();\n> +\t\t\treturn {};\n> \t\t}\n> \t}\n>\n> -\tit += infoDataSize;\n> -\tByteStreamBuffer buffer(&*it, listDataSize);\n> +\tconst auto *p = reader.consume(listDataSize);\n> +\n> +\tByteStreamBuffer buffer(reinterpret_cast(p), listDataSize);\n> \tControlList list = cs->deserialize(buffer);\n> -\tif (buffer.overflow())\n> +\tif (buffer.overflow()) {\n> \t\tLOG(IPADataSerializer, Error) << \"Failed to deserialize ControlList: buffer overflow\";\n> +\t\treturn {};\n> +\t}\n>\n> -\treturn list;\n> -}\n> -\n> -template<>\n> -ControlList\n> -IPADataSerializer::deserialize(const std::vector &data,\n> -\t\t\t\t\t ControlSerializer *cs)\n> -{\n> -\treturn deserialize(data.cbegin(), data.end(), cs);\n> -}\n> -\n> -template<>\n> -ControlList\n> -IPADataSerializer::deserialize(const std::vector &data,\n> -\t\t\t\t\t [[maybe_unused]] const std::vector &fds,\n> -\t\t\t\t\t ControlSerializer *cs)\n> -{\n> -\treturn deserialize(data.cbegin(), data.end(), cs);\n> -}\n> -\n> -template<>\n> -ControlList\n> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> -\t\t\t\t\t ControlSerializer *cs)\n> -{\n> -\treturn deserialize(dataBegin, dataEnd, cs);\n> +\treturn std::move(list);\n> }\n>\n> /*\n> @@ -458,57 +284,28 @@ IPADataSerializer::serialize(const ControlInfoMap &map,\n> }\n>\n> template<>\n> -ControlInfoMap\n> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n> +std::optional\n> +IPADataSerializer::deserialize(SeriReader &reader,\n> \t\t\t\t\t ControlSerializer *cs)\n> {\n> \tif (!cs)\n> \t\tLOG(IPADataSerializer, Fatal)\n> \t\t\t<< \"ControlSerializer not provided for deserialization of ControlInfoMap\";\n>\n> -\tif (std::distance(dataBegin, dataEnd) < 4)\n> +\tuint32_t infoDataSize;\n> +\tif (!reader.read(infoDataSize))\n> \t\treturn {};\n>\n> -\tuint32_t infoDataSize = readPOD(dataBegin, 0, dataEnd);\n> -\n> -\tstd::vector::const_iterator it = dataBegin + 4;\n> -\n> -\tif (static_cast(std::distance(it, dataEnd)) < infoDataSize)\n> +\tconst auto *p = reader.consume(infoDataSize);\n> +\tif (!p)\n> \t\treturn {};\n>\n> -\tByteStreamBuffer buffer(&*it, infoDataSize);\n> +\tByteStreamBuffer buffer(reinterpret_cast(p), infoDataSize);\n> \tControlInfoMap map = cs->deserialize(buffer);\n> +\tif (buffer.overflow())\n> +\t\treturn {};\n>\n> -\treturn map;\n> -}\n> -\n> -template<>\n> -ControlInfoMap\n> -IPADataSerializer::deserialize(const std::vector &data,\n> -\t\t\t\t\t ControlSerializer *cs)\n> -{\n> -\treturn deserialize(data.cbegin(), data.end(), cs);\n> -}\n> -\n> -template<>\n> -ControlInfoMap\n> -IPADataSerializer::deserialize(const std::vector &data,\n> -\t\t\t\t\t [[maybe_unused]] const std::vector &fds,\n> -\t\t\t\t\t ControlSerializer *cs)\n> -{\n> -\treturn deserialize(data.cbegin(), data.end(), cs);\n> -}\n> -\n> -template<>\n> -ControlInfoMap\n> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> -\t\t\t\t\t ControlSerializer *cs)\n> -{\n> -\treturn deserialize(dataBegin, dataEnd, cs);\n> +\treturn std::move(map);\n> }\n>\n> /*\n> @@ -542,27 +339,23 @@ IPADataSerializer::serialize(const SharedFD &data,\n> }\n>\n> template<>\n> -SharedFD IPADataSerializer::deserialize([[maybe_unused]] std::vector::const_iterator dataBegin,\n> -\t\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator dataEnd,\n> -\t\t\t\t\t\t std::vector::const_iterator fdsBegin,\n> -\t\t\t\t\t\t std::vector::const_iterator fdsEnd,\n> -\t\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n> +std::optional\n> +IPADataSerializer::deserialize(SeriReader &reader,\n> +\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n> {\n> -\tASSERT(std::distance(dataBegin, dataEnd) >= 4);\n> +\tuint32_t valid;\n>\n> -\tuint32_t valid = readPOD(dataBegin, 0, dataEnd);\n> +\tif (!reader.read(valid))\n> +\t\treturn {};\n>\n> -\tASSERT(!(valid && std::distance(fdsBegin, fdsEnd) < 1));\n> +\tif (!valid)\n> +\t\treturn SharedFD{};\n>\n> -\treturn valid ? *fdsBegin : SharedFD();\n> -}\n> +\tconst auto *fd = reader.nextFd();\n> +\tif (!fd)\n> +\t\treturn {};\n>\n> -template<>\n> -SharedFD IPADataSerializer::deserialize(const std::vector &data,\n> -\t\t\t\t\t\t const std::vector &fds,\n> -\t\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n> -{\n> -\treturn deserialize(data.cbegin(), data.end(), fds.cbegin(), fds.end());\n> +\treturn *fd;\n> }\n>\n> /*\n> @@ -594,30 +387,19 @@ IPADataSerializer::serialize(const FrameBuffer::Plane &data,\n> }\n>\n> template<>\n> -FrameBuffer::Plane\n> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n> -\t\t\t\t\t\t std::vector::const_iterator dataEnd,\n> -\t\t\t\t\t\t std::vector::const_iterator fdsBegin,\n> -\t\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> +std::optional\n> +IPADataSerializer::deserialize(SeriReader &reader,\n> \t\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n> {\n> -\tFrameBuffer::Plane ret;\n> -\n> -\tret.fd = IPADataSerializer::deserialize(dataBegin, dataBegin + 4,\n> -\t\t\t\t\t\t\t fdsBegin, fdsBegin + 1);\n> -\tret.offset = readPOD(dataBegin, 4, dataEnd);\n> -\tret.length = readPOD(dataBegin, 8, dataEnd);\n> +\tauto fd = IPADataSerializer::deserialize(reader);\n> +\tif (!fd)\n> +\t\treturn {};\n>\n> -\treturn ret;\n> -}\n> +\tuint32_t offset, length;\n> +\tif (!reader.read(offset, length))\n> +\t\treturn {};\n>\n> -template<>\n> -FrameBuffer::Plane\n> -IPADataSerializer::deserialize(const std::vector &data,\n> -\t\t\t\t\t\t const std::vector &fds,\n> -\t\t\t\t\t\t ControlSerializer *cs)\n> -{\n> -\treturn deserialize(data.cbegin(), data.end(), fds.cbegin(), fds.end(), cs);\n> +\treturn { { std::move(*fd), offset, length } };\n> }\n>\n> #endif /* __DOXYGEN__ */\n> diff --git a/src/libcamera/ipc_pipe.cpp b/src/libcamera/ipc_pipe.cpp\n> index 548299d05..71d0c2cb4 100644\n> --- a/src/libcamera/ipc_pipe.cpp\n> +++ b/src/libcamera/ipc_pipe.cpp\n> @@ -148,6 +148,11 @@ IPCUnixSocket::Payload IPCMessage::payload() const\n> * \\brief Returns a const reference to the vector containing file descriptors\n> */\n>\n> +/**\n> + * \\fn IPCMessage::reader() const\n> + * \\brief Returns a `SeriReader` instance for parsing the message\n> + */\n> +\n> /**\n> * \\class IPCPipe\n> * \\brief IPC message pipe for IPA isolation\n> diff --git a/test/ipc/unixsocket_ipc.cpp b/test/ipc/unixsocket_ipc.cpp\n> index df7d9c2b4..c0e174ba9 100644\n> --- a/test/ipc/unixsocket_ipc.cpp\n> +++ b/test/ipc/unixsocket_ipc.cpp\n> @@ -102,7 +102,14 @@ private:\n> \t\t}\n>\n> \t\tcase CmdSetAsync: {\n> -\t\t\tvalue_ = IPADataSerializer::deserialize(ipcMessage.data());\n> +\t\t\tSeriReader reader = ipcMessage.reader();\n> +\t\t\tauto value = IPADataSerializer::deserialize(reader);\n> +\t\t\tif (!value) {\n> +\t\t\t\tcerr << \"Failed to deserialize value\" << endl;\n> +\t\t\t\tstop(-ENODATA);\n> +\t\t\t} else {\n> +\t\t\t\tvalue_ = *value;\n> +\t\t\t}\n> \t\t\tbreak;\n> \t\t}\n> \t\t}\n> @@ -155,7 +162,14 @@ protected:\n> \t\t\treturn ret;\n> \t\t}\n>\n> -\t\treturn IPADataSerializer::deserialize(buf.data());\n> +\t\tSeriReader reader = buf.reader();\n> +\t\tauto value = IPADataSerializer::deserialize(reader);\n> +\t\tif (!value) {\n> +\t\t\tcerr << \"Failed to deserialize value\" << endl;\n> +\t\t\treturn -ENODATA;\n> +\t\t}\n> +\n> +\t\treturn *value;\n> \t}\n>\n> \tint exit()\n> diff --git a/test/serialization/generated_serializer/generated_serializer_test.cpp b/test/serialization/generated_serializer/generated_serializer_test.cpp\n> index dd6968850..0640e741d 100644\n> --- a/test/serialization/generated_serializer/generated_serializer_test.cpp\n> +++ b/test/serialization/generated_serializer/generated_serializer_test.cpp\n> @@ -43,7 +43,7 @@ if (struct1.field != struct2.field) {\t\t\t\t\\\n> }\n>\n>\n> -\t\tipa::test::TestStruct t, u;\n> +\t\tipa::test::TestStruct t;\n>\n> \t\tt.m = {\n> \t\t\t{ \"a\", \"z\" },\n> @@ -71,8 +71,13 @@ if (struct1.field != struct2.field) {\t\t\t\t\\\n>\n> \t\tstd::tie(serialized, ignore) =\n> \t\t\tIPADataSerializer::serialize(t);\n> +\t\tSeriReader reader(serialized);\n>\n> -\t\tu = IPADataSerializer::deserialize(serialized);\n> +\t\tauto optu = IPADataSerializer::deserialize(reader);\n> +\t\tif (!optu)\n> +\t\t\treturn TestFail;\n> +\n> +\t\tauto &u = *optu;\n>\n> \t\tif (!equals(t.m, u.m))\n> \t\t\treturn TestFail;\n> @@ -91,12 +96,16 @@ if (struct1.field != struct2.field) {\t\t\t\t\\\n>\n> \t\t/* Test vector of generated structs */\n> \t\tstd::vector v = { t, u };\n> -\t\tstd::vector w;\n>\n> \t\tstd::tie(serialized, ignore) =\n> \t\t\tIPADataSerializer>::serialize(v);\n> +\t\treader = SeriReader(serialized);\n> +\n> +\t\tauto optw = IPADataSerializer>::deserialize(reader);\n> +\t\tif (!optw)\n> +\t\t\treturn TestFail;\n>\n> -\t\tw = IPADataSerializer>::deserialize(serialized);\n> +\t\tauto &w = *optw;\n>\n> \t\tif (!equals(v[0].m, w[0].m) ||\n> \t\t !equals(v[1].m, w[1].m))\n> diff --git a/test/serialization/ipa_data_serializer_test.cpp b/test/serialization/ipa_data_serializer_test.cpp\n> index afea93a6c..3873808a6 100644\n> --- a/test/serialization/ipa_data_serializer_test.cpp\n> +++ b/test/serialization/ipa_data_serializer_test.cpp\n> @@ -52,7 +52,9 @@ int testPodSerdes(T in)\n> \tstd::vector fds;\n>\n> \tstd::tie(buf, fds) = IPADataSerializer::serialize(in);\n> -\tT out = IPADataSerializer::deserialize(buf, fds);\n> +\tSeriReader reader(buf, fds);\n> +\n> +\tauto out = IPADataSerializer::deserialize(reader);\n> \tif (in == out)\n> \t\treturn TestPass;\n>\n> @@ -71,7 +73,9 @@ int testVectorSerdes(const std::vector &in,\n> \tstd::vector fds;\n>\n> \tstd::tie(buf, fds) = IPADataSerializer>::serialize(in, cs);\n> -\tstd::vector out = IPADataSerializer>::deserialize(buf, fds, cs);\n> +\tSeriReader reader(buf, fds);\n> +\n> +\tauto out = IPADataSerializer>::deserialize(reader, cs);\n> \tif (in == out)\n> \t\treturn TestPass;\n>\n> @@ -91,7 +95,9 @@ int testMapSerdes(const std::map &in,\n> \tstd::vector fds;\n>\n> \tstd::tie(buf, fds) = IPADataSerializer>::serialize(in, cs);\n> -\tstd::map out = IPADataSerializer>::deserialize(buf, fds, cs);\n> +\tSeriReader reader(buf, fds);\n> +\n> +\tauto out = IPADataSerializer>::deserialize(reader, cs);\n> \tif (in == out)\n> \t\treturn TestPass;\n>\n> @@ -171,17 +177,27 @@ private:\n> \t\tstd::tie(listBuf, std::ignore) =\n> \t\t\tIPADataSerializer::serialize(list, &cs);\n>\n> -\t\tconst ControlInfoMap infoMapOut =\n> -\t\t\tIPADataSerializer::deserialize(infoMapBuf, &cs);\n> +\t\tSeriReader listReader(listBuf);\n> +\t\tSeriReader infoMapReader(infoMapBuf);\n> +\n> +\t\tauto infoMapOut = IPADataSerializer::deserialize(infoMapReader, &cs);\n> +\t\tif (!infoMapOut) {\n> +\t\t\tcerr << \"`ControlInfoMap` cannot be deserialized\" << endl;\n> +\t\t\treturn TestFail;\n> +\t\t}\n>\n> -\t\tControlList listOut = IPADataSerializer::deserialize(listBuf, &cs);\n> +\t\tauto listOut = IPADataSerializer::deserialize(listReader, &cs);\n> +\t\tif (!listOut) {\n> +\t\t\tcerr << \"`ControlList` cannot be deserialized\" << endl;\n> +\t\t\treturn TestFail;\n> +\t\t}\n>\n> -\t\tif (!SerializationTest::equals(infoMap, infoMapOut)) {\n> +\t\tif (!SerializationTest::equals(infoMap, *infoMapOut)) {\n> \t\t\tcerr << \"Deserialized map doesn't match original\" << endl;\n> \t\t\treturn TestFail;\n> \t\t}\n>\n> -\t\tif (!SerializationTest::equals(list, listOut)) {\n> +\t\tif (!SerializationTest::equals(list, *listOut)) {\n> \t\t\tcerr << \"Deserialized list doesn't match original\" << endl;\n> \t\t\treturn TestFail;\n> \t\t}\n> diff --git a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> index 9a3aadbd2..843260b4b 100644\n> --- a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> +++ b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> @@ -107,13 +107,13 @@ namespace {{ns}} {\n> {% if interface_event.methods|length > 0 %}\n> void {{proxy_name}}::recvMessage(const IPCMessage &data)\n> {\n> -\tsize_t dataSize = data.data().size();\n> +\tSeriReader reader = data.reader();\n> \t{{cmd_event_enum_name}} _cmd = static_cast<{{cmd_event_enum_name}}>(data.header().cmd);\n>\n> \tswitch (_cmd) {\n> {%- for method in interface_event.methods %}\n> \tcase {{cmd_event_enum_name}}::{{method.mojom_name|cap}}: {\n> -\t\t{{method.mojom_name}}IPC(data.data().cbegin(), dataSize, data.fds());\n> +\t\t{{method.mojom_name}}IPC(reader);\n> \t\tbreak;\n> \t}\n> {%- endfor %}\n> @@ -211,18 +211,23 @@ void {{proxy_name}}::recvMessage(const IPCMessage &data)\n> \t\treturn;\n> {%- endif %}\n> \t}\n> +{% if has_output %}\n> +\tSeriReader _outputReader = _ipcOutputBuf.reader();\n> +{% endif -%}\n> {% if method|method_return_value != \"void\" %}\n> -\t{{method|method_return_value}} _retValue = IPADataSerializer<{{method|method_return_value}}>::deserialize(_ipcOutputBuf.data(), 0);\n> -\n> -{{proxy_funcs.deserialize_call(method|method_param_outputs, '_ipcOutputBuf.data()', '_ipcOutputBuf.fds()', init_offset = method|method_return_value|byte_width|int)}}\n> +\tauto _retValue = IPADataSerializer<{{method|method_return_value}}>::deserialize(_outputReader);\n> +\tASSERT(_retValue);\n> +{% endif -%}\n>\n> -\treturn _retValue;\n> +{% if has_output %}\n> +\t{{proxy_funcs.deserialize_call(method|method_param_outputs, '_outputReader')}}\n> +\tASSERT(_outputReader.empty());\n> +{% endif -%}\n>\n> -{% elif method|method_param_outputs|length > 0 %}\n> -{{proxy_funcs.deserialize_call(method|method_param_outputs, '_ipcOutputBuf.data()', '_ipcOutputBuf.fds()')}}\n> +{% if method|method_return_value != \"void\" %}\n> +\treturn std::move(*_retValue);\n> {% endif -%}\n> }\n> -\n> {% endfor %}\n>\n> {% for method in interface_event.methods %}\n> @@ -232,12 +237,9 @@ void {{proxy_name}}::recvMessage(const IPCMessage &data)\n> \t{{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n> }\n>\n> -void {{proxy_name}}::{{method.mojom_name}}IPC(\n> -\t[[maybe_unused]] std::vector::const_iterator data,\n> -\t[[maybe_unused]] size_t dataSize,\n> -\t[[maybe_unused]] const std::vector &fds)\n> +void {{proxy_name}}::{{method.mojom_name}}IPC([[maybe_unused]] SeriReader &reader)\n> {\n> -{{proxy_funcs.deserialize_call(method.parameters, 'data', 'fds', false, true, true, 'dataSize')}}\n> +\t{{proxy_funcs.deserialize_call(method.parameters, 'reader', false, true)}}\n> \t{{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n> }\n> {% endfor %}\n> diff --git a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> index a0312a7c1..945a4ded9 100644\n> --- a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> +++ b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> @@ -53,10 +53,7 @@ private:\n> {% endfor %}\n> {% for method in interface_event.methods %}\n> {{proxy_funcs.func_sig(proxy_name, method, \"Thread\", false)|indent(8, true)}};\n> -\tvoid {{method.mojom_name}}IPC(\n> -\t\tstd::vector::const_iterator data,\n> -\t\tsize_t dataSize,\n> -\t\tconst std::vector &fds);\n> +\tvoid {{method.mojom_name}}IPC(SeriReader &reader);\n> {% endfor %}\n>\n> \t/* Helper class to invoke async functions in another thread. */\n> diff --git a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> index 1f990d3f9..de6378be0 100644\n> --- a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> +++ b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> @@ -71,6 +71,7 @@ public:\n> \t\t}\n>\n> \t\tIPCMessage _ipcMessage(_message);\n> +\t\tSeriReader _reader = _ipcMessage.reader();\n>\n> \t\t{{cmd_enum_name}} _cmd = static_cast<{{cmd_enum_name}}>(_ipcMessage.header().cmd);\n>\n> @@ -85,7 +86,9 @@ public:\n> {%- if method.mojom_name == \"configure\" %}\n> \t\t\tcontrolSerializer_.reset();\n> {%- endif %}\n> -\t\t{{proxy_funcs.deserialize_call(method|method_param_inputs, '_ipcMessage.data()', '_ipcMessage.fds()', false, true)|indent(16, true)}}\n> +\t\t\t{{proxy_funcs.deserialize_call(method|method_param_inputs, '_reader', false, true)|indent(16, true)}}\n> +\t\t\tASSERT(_reader.empty());\n> +\n> {% for param in method|method_param_outputs %}\n> \t\t\t{{param|name}} {{param.mojom_name}};\n> {% endfor %}\n> diff --git a/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl b/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> index 01e2567ca..b6835ca35 100644\n> --- a/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> +++ b/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> @@ -64,15 +64,6 @@\n> );\n> {%- endfor %}\n>\n> -{%- if params|length > 1 %}\n> -{%- for param in params %}\n> -\tappendPOD({{buf}}, {{param.mojom_name}}Buf.size());\n> -{%- if param|has_fd %}\n> -\tappendPOD({{buf}}, {{param.mojom_name}}Fds.size());\n> -{%- endif %}\n> -{%- endfor %}\n> -{%- endif %}\n> -\n> {%- for param in params %}\n> \t{{buf}}.insert({{buf}}.end(), {{param.mojom_name}}Buf.begin(), {{param.mojom_name}}Buf.end());\n> {%- endfor %}\n> @@ -84,104 +75,28 @@\n> {%- endfor %}\n> {%- endmacro -%}\n>\n> -\n> -{#\n> - # \\brief Deserialize a single object from data buffer and fd vector\n> - #\n> - # \\param pointer If true, deserialize the object into a dereferenced pointer\n> - # \\param iter If true, treat \\a buf as an iterator instead of a vector\n> - # \\param data_size Variable that holds the size of the vector referenced by \\a buf\n> - #\n> - # Generate code to deserialize a single object, as specified in \\a param,\n> - # from \\a buf data buffer and \\a fds fd vector.\n> - # This code is meant to be used by macro deserialize_call.\n> - #}\n> -{%- macro deserialize_param(param, pointer, loop, buf, fds, iter, data_size) -%}\n> -{{\"*\" if pointer}}{{param.mojom_name}} =\n> -IPADataSerializer<{{param|name_full}}>::deserialize(\n> -\t{{buf}}{{- \".cbegin()\" if not iter}} + {{param.mojom_name}}Start,\n> -{%- if loop.last and not iter %}\n> -\t{{buf}}.cend()\n> -{%- elif not iter %}\n> -\t{{buf}}.cbegin() + {{param.mojom_name}}Start + {{param.mojom_name}}BufSize\n> -{%- elif iter and loop.length == 1 %}\n> -\t{{buf}} + {{data_size}}\n> -{%- else %}\n> -\t{{buf}} + {{param.mojom_name}}Start + {{param.mojom_name}}BufSize\n> -{%- endif -%}\n> -{{- \",\" if param|has_fd}}\n> -{%- if param|has_fd %}\n> -\t{{fds}}.cbegin() + {{param.mojom_name}}FdStart,\n> -{%- if loop.last %}\n> -\t{{fds}}.cend()\n> -{%- else %}\n> -\t{{fds}}.cbegin() + {{param.mojom_name}}FdStart + {{param.mojom_name}}FdsSize\n> -{%- endif -%}\n> -{%- endif -%}\n> -{{- \",\" if param|needs_control_serializer}}\n> -{%- if param|needs_control_serializer %}\n> -\t&controlSerializer_\n> -{%- endif -%}\n> -);\n> -{%- endmacro -%}\n> -\n> -\n> {#\n> - # \\brief Deserialize multiple objects from data buffer and fd vector\n> + # \\brief Deserialize multiple objects\n> #\n> # \\param pointer If true, deserialize objects into pointers, and adds a null check.\n> # \\param declare If true, declare the objects in addition to deserialization.\n> - # \\param iter if true, treat \\a buf as an iterator instead of a vector\n> - # \\param data_size Variable that holds the size of the vector referenced by \\a buf\n> - #\n> - # Generate code to deserialize multiple objects, as specified in \\a params\n> - # (which are the parameters to some function), from \\a buf data buffer and\n> - # \\a fds fd vector.\n> - # This code is meant to be used by the proxy, for deserializing after IPC calls.\n> #\n> - # \\todo Avoid intermediate vectors\n> + # Generate code to deserialize multiple objects, as specified in \\a params,\n> + # from \\a reader. This code is meant to be used by the proxy, for deserializing\n> + # after IPC calls.\n> #}\n> -{%- macro deserialize_call(params, buf, fds, pointer = true, declare = false, iter = false, data_size = '', init_offset = 0) -%}\n> -{% set ns = namespace(size_offset = init_offset) %}\n> -{%- if params|length > 1 %}\n> +{%- macro deserialize_call(params, reader, pointer = true, declare = false) -%}\n> {%- for param in params %}\n> -\t[[maybe_unused]] const size_t {{param.mojom_name}}BufSize = readPOD({{buf}}, {{ns.size_offset}}\n> -{%- if iter -%}\n> -, {{buf}} + {{data_size}}\n> -{%- endif -%}\n> -);\n> -\t{%- set ns.size_offset = ns.size_offset + 4 %}\n> -{%- if param|has_fd %}\n> -\t[[maybe_unused]] const size_t {{param.mojom_name}}FdsSize = readPOD({{buf}}, {{ns.size_offset}}\n> -{%- if iter -%}\n> -, {{buf}} + {{data_size}}\n> -{%- endif -%}\n> -);\n> -\t{%- set ns.size_offset = ns.size_offset + 4 %}\n> -{%- endif %}\n> -{%- endfor %}\n> -{%- endif %}\n> -{% for param in params %}\n> -{%- if loop.first %}\n> -\tconst size_t {{param.mojom_name}}Start = {{ns.size_offset}};\n> -{%- else %}\n> -\tconst size_t {{param.mojom_name}}Start = {{loop.previtem.mojom_name}}Start + {{loop.previtem.mojom_name}}BufSize;\n> -{%- endif %}\n> -{%- endfor %}\n> -{% for param in params|with_fds %}\n> -{%- if loop.first %}\n> -\tconst size_t {{param.mojom_name}}FdStart = 0;\n> +\tauto param_{{param.mojom_name}} = IPADataSerializer<{{param|name_full}}>::deserialize({{reader}}, &controlSerializer_);\n> +\tASSERT(param_{{param.mojom_name}});\n> +\n> +{%- if pointer %}\n> +\tif ({{param.mojom_name}})\n> +\t\t*{{param.mojom_name}} = std::move(*param_{{param.mojom_name}});\n> +{%- elif declare %}\n> +\t{{param|name}} &{{param.mojom_name}} = *param_{{param.mojom_name}};\n> {%- else %}\n> -\tconst size_t {{param.mojom_name}}FdStart = {{loop.previtem.mojom_name}}FdStart + {{loop.previtem.mojom_name}}FdsSize;\n> +\t{{param.mojom_name}} = std::move(*param_{{param.mojom_name}});\n> {%- endif %}\n> -{%- endfor %}\n> -{% for param in params %}\n> -\t{%- if pointer %}\n> -\tif ({{param.mojom_name}}) {\n> -{{deserialize_param(param, pointer, loop, buf, fds, iter, data_size)|indent(16, True)}}\n> -\t}\n> -\t{%- else %}\n> -\t{{param|name + \" \" if declare}}{{deserialize_param(param, pointer, loop, buf, fds, iter, data_size)|indent(8)}}\n> -\t{%- endif %}\n> {% endfor %}\n> {%- endmacro -%}\n> diff --git a/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl b/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl\n> index e316dd88a..9e9dd0ca6 100644\n> --- a/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl\n> +++ b/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl\n> @@ -2,22 +2,6 @@\n> # SPDX-License-Identifier: LGPL-2.1-or-later\n> # Copyright (C) 2020, Google Inc.\n> -#}\n> -{#\n> - # \\brief Verify that there is enough bytes to deserialize\n> - #\n> - # Generate code that verifies that \\a size is not greater than \\a dataSize.\n> - # Otherwise log an error with \\a name and \\a typename.\n> - #}\n> -{%- macro check_data_size(size, dataSize, name, typename) %}\n> -\t\tif ({{dataSize}} < {{size}}) {\n> -\t\t\tLOG(IPADataSerializer, Error)\n> -\t\t\t\t<< \"Failed to deserialize \" << \"{{name}}\"\n> -\t\t\t\t<< \": not enough {{typename}}, expected \"\n> -\t\t\t\t<< ({{size}}) << \", got \" << ({{dataSize}});\n> -\t\t\treturn ret;\n> -\t\t}\n> -{%- endmacro %}\n> -\n>\n> {#\n> # \\brief Serialize a field into return vector\n> @@ -42,15 +26,10 @@\n> \t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> \t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> {%- elif field|is_controls %}\n> -\t\tif (data.{{field.mojom_name}}.size() > 0) {\n> -\t\t\tstd::vector {{field.mojom_name}};\n> -\t\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> -\t\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> -\t\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> -\t\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> -\t\t} else {\n> -\t\t\tappendPOD(retData, 0);\n> -\t\t}\n> +\t\tstd::vector {{field.mojom_name}};\n> +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> {%- elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> \t\tstd::vector {{field.mojom_name}};\n> \t{%- if field|has_fd %}\n> @@ -65,10 +44,6 @@\n> \t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> \t{%- else %}\n> \t\t\tIPADataSerializer<{{field|name_full}}>::serialize(data.{{field.mojom_name}}, cs);\n> -\t{%- endif %}\n> -\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> -\t{%- if field|has_fd %}\n> -\t\tappendPOD(retData, {{field.mojom_name}}Fds.size());\n> \t{%- endif %}\n> \t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> \t{%- if field|has_fd %}\n> @@ -79,89 +54,6 @@\n> {%- endif %}\n> {%- endmacro %}\n>\n> -\n> -{#\n> - # \\brief Deserialize a field into return struct\n> - #\n> - # Generate code to deserialize \\a field into object ret.\n> - # This code is meant to be used by the IPADataSerializer specialization.\n> - #}\n> -{%- macro deserializer_field(field, loop) %}\n> -{% if field|is_pod or field|is_enum %}\n> -\t{%- set field_size = (field|bit_width|int / 8)|int %}\n> -\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> -\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name_full}}>::deserialize(m, m + {{field_size}});\n> -\t{%- if not loop.last %}\n> -\t\tm += {{field_size}};\n> -\t\tdataSize -= {{field_size}};\n> -\t{%- endif %}\n> -{% elif field|is_fd %}\n> -\t{%- set field_size = 4 %}\n> -\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> -\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field_size}}, n, n + 1, cs);\n> -\t{%- if not loop.last %}\n> -\t\tm += {{field_size}};\n> -\t\tdataSize -= {{field_size}};\n> -\t\tn += ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> -\t\tfdsSize -= ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> -\t{%- endif %}\n> -{% elif field|is_controls %}\n> -\t{%- set field_size = 4 %}\n> -\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> -\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> -\t\tm += {{field_size}};\n> -\t\tdataSize -= {{field_size}};\n> -\t{%- set field_size = field.mojom_name + 'Size' -%}\n> -\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> -\t\tif ({{field.mojom_name}}Size > 0)\n> -\t\t\tret.{{field.mojom_name}} =\n> -\t\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> -\t{%- if not loop.last %}\n> -\t\tm += {{field_size}};\n> -\t\tdataSize -= {{field_size}};\n> -\t{%- endif %}\n> -{% elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> -\t{%- set field_size = 4 %}\n> -\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> -\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> -\t\tm += {{field_size}};\n> -\t\tdataSize -= {{field_size}};\n> -\t{%- if field|has_fd %}\n> -\t{%- set field_size = 4 %}\n> -\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'FdsSize', 'data')}}\n> -\t\tconst size_t {{field.mojom_name}}FdsSize = readPOD(m, 0, dataEnd);\n> -\t\tm += {{field_size}};\n> -\t\tdataSize -= {{field_size}};\n> -\t\t{{- check_data_size(field.mojom_name + 'FdsSize', 'fdsSize', field.mojom_name, 'fds')}}\n> -\t{%- endif %}\n> -\t{%- set field_size = field.mojom_name + 'Size' -%}\n> -\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> -\t\tret.{{field.mojom_name}} =\n> -\t{%- if field|is_str %}\n> -\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size);\n> -\t{%- elif field|has_fd and (field|is_array or field|is_map) %}\n> -\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> -\t{%- elif field|has_fd and (not (field|is_array or field|is_map)) %}\n> -\t\t\tIPADataSerializer<{{field|name_full}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> -\t{%- elif (not field|has_fd) and (field|is_array or field|is_map) %}\n> -\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> -\t{%- else %}\n> -\t\t\tIPADataSerializer<{{field|name_full}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> -\t{%- endif %}\n> -\t{%- if not loop.last %}\n> -\t\tm += {{field_size}};\n> -\t\tdataSize -= {{field_size}};\n> -\t{%- if field|has_fd %}\n> -\t\tn += {{field.mojom_name}}FdsSize;\n> -\t\tfdsSize -= {{field.mojom_name}}FdsSize;\n> -\t{%- endif %}\n> -\t{%- endif %}\n> -{% else %}\n> -\t\t/* Unknown deserialization for {{field.mojom_name}}. */\n> -{%- endif %}\n> -{%- endmacro %}\n> -\n> -\n> {#\n> # \\brief Serialize a struct\n> #\n> @@ -194,126 +86,30 @@\n>\n>\n> {#\n> - # \\brief Deserialize a struct that has fds\n> + # \\brief Deserialize a struct\n> #\n> - # Generate code for IPADataSerializer specialization, for deserializing\n> - # \\a struct, in the case that \\a struct has file descriptors.\n> + # Generate code for IPADataSerializer specialization, for deserializing \\a struct.\n> #}\n> -{%- macro deserializer_fd(struct) %}\n> -\tstatic {{struct|name_full}}\n> -\tdeserialize(std::vector &data,\n> -\t\t std::vector &fds,\n> -{%- if struct|needs_control_serializer %}\n> -\t\t ControlSerializer *cs)\n> -{%- else %}\n> -\t\t ControlSerializer *cs = nullptr)\n> -{%- endif %}\n> -\t{\n> -\t\treturn IPADataSerializer<{{struct|name_full}}>::deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n> -\t}\n> -\n> +{%- macro deserializer(struct) %}\n> {# \\todo Don't inline this function #}\n> -\tstatic {{struct|name_full}}\n> -\tdeserialize(std::vector::const_iterator dataBegin,\n> -\t\t std::vector::const_iterator dataEnd,\n> -\t\t std::vector::const_iterator fdsBegin,\n> -\t\t std::vector::const_iterator fdsEnd,\n> +\t[[nodiscard]] static std::optional<{{struct|name_full}}>\n> +\tdeserialize(SeriReader &reader,\n> {%- if struct|needs_control_serializer %}\n> \t\t ControlSerializer *cs)\n> {%- else %}\n> \t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> {%- endif %}\n> \t{\n> -\t\t{{struct|name_full}} ret;\n> -\t\tstd::vector::const_iterator m = dataBegin;\n> -\t\tstd::vector::const_iterator n = fdsBegin;\n> -\n> -\t\tsize_t dataSize = std::distance(dataBegin, dataEnd);\n> -\t\t[[maybe_unused]] size_t fdsSize = std::distance(fdsBegin, fdsEnd);\n> -{%- for field in struct.fields -%}\n> -{{deserializer_field(field, loop)}}\n> +{%- for field in struct.fields %}\n> +\t\tauto {{field.mojom_name}} = IPADataSerializer<{{field|name_full}}>::deserialize(reader, cs);\n> +\t\tif (!{{field.mojom_name}})\n> +\t\t\treturn {};\n> {%- endfor %}\n> -\t\treturn ret;\n> -\t}\n> -{%- endmacro %}\n> -\n> -{#\n> - # \\brief Deserialize a struct that has fds, using non-fd\n> - #\n> - # Generate code for IPADataSerializer specialization, for deserializing\n> - # \\a struct, in the case that \\a struct has no file descriptors but requires\n> - # deserializers with file descriptors.\n> - #}\n> -{%- macro deserializer_fd_simple(struct) %}\n> -\tstatic {{struct|name_full}}\n> -\tdeserialize(std::vector &data,\n> -\t\t [[maybe_unused]] std::vector &fds,\n> -\t\t ControlSerializer *cs = nullptr)\n> -\t{\n> -\t\treturn IPADataSerializer<{{struct|name_full}}>::deserialize(data.cbegin(), data.cend(), cs);\n> -\t}\n> -\n> -\tstatic {{struct|name_full}}\n> -\tdeserialize(std::vector::const_iterator dataBegin,\n> -\t\t std::vector::const_iterator dataEnd,\n> -\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> -\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> -\t\t ControlSerializer *cs = nullptr)\n> -\t{\n> -\t\treturn IPADataSerializer<{{struct|name_full}}>::deserialize(dataBegin, dataEnd, cs);\n> -\t}\n> -{%- endmacro %}\n> -\n> -\n> -{#\n> - # \\brief Deserialize a struct that has no fds\n> - #\n> - # Generate code for IPADataSerializer specialization, for deserializing\n> - # \\a struct, in the case that \\a struct does not have file descriptors.\n> - #}\n> -{%- macro deserializer_no_fd(struct) %}\n> -\tstatic {{struct|name_full}}\n> -\tdeserialize(std::vector &data,\n> -{%- if struct|needs_control_serializer %}\n> -\t\t ControlSerializer *cs)\n> -{%- else %}\n> -\t\t ControlSerializer *cs = nullptr)\n> -{%- endif %}\n> -\t{\n> -\t\treturn IPADataSerializer<{{struct|name_full}}>::deserialize(data.cbegin(), data.cend(), cs);\n> -\t}\n>\n> -{# \\todo Don't inline this function #}\n> -\tstatic {{struct|name_full}}\n> -\tdeserialize(std::vector::const_iterator dataBegin,\n> -\t\t std::vector::const_iterator dataEnd,\n> -{%- if struct|needs_control_serializer %}\n> -\t\t ControlSerializer *cs)\n> -{%- else %}\n> -\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> -{%- endif %}\n> -\t{\n> -\t\t{{struct|name_full}} ret;\n> -\t\tstd::vector::const_iterator m = dataBegin;\n> -\n> -\t\tsize_t dataSize = std::distance(dataBegin, dataEnd);\n> -{%- for field in struct.fields -%}\n> -{{deserializer_field(field, loop)}}\n> +\t\treturn { {\n> +{%- for field in struct.fields %}\n> +\t\t\tstd::move(*{{field.mojom_name}}),\n> {%- endfor %}\n> -\t\treturn ret;\n> +\t\t} };\n> \t}\n> {%- endmacro %}\n> -\n> -{#\n> - # \\brief Deserialize a struct\n> - #\n> - # Generate code for IPADataSerializer specialization, for deserializing \\a struct.\n> - #}\n> -{%- macro deserializer(struct) %}\n> -{%- if struct|has_fd %}\n> -{{deserializer_fd(struct)}}\n> -{%- else %}\n> -{{deserializer_no_fd(struct)}}\n> -{{deserializer_fd_simple(struct)}}\n> -{%- endif %}\n> -{%- endmacro %}\n> --\n> 2.49.0\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C57DEC31E9\n\tfor ;\n\tTue, 20 May 2025 15:19:03 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id E6BDB68D85;\n\tTue, 20 May 2025 17:19:02 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 40051614DE\n\tfor ;\n\tTue, 20 May 2025 17:19:01 +0200 (CEST)","from ideasonboard.com (93-61-96-190.ip145.fastwebnet.it\n\t[93.61.96.190])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 4449474C;\n\tTue, 20 May 2025 17:18:40 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"GHqZc7Zb\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1747754320;\n\tbh=MV0VcvLY8lqKmHRLEV/ZbLcA+1zuUT284zO5IeY8nZg=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=GHqZc7ZbdwW3+A6it9gPgU2bPg1C09oAIBpMV4ujZFhBbeCPh/sMZTdUCqXJLIQUD\n\tWQBobDIXBcdD9HQZmhdpzB6vobl1wVaUkpjzG7Dh+Q4vSC7jBzjT0N1Hvxu2ZAax07\n\t8wlEH/lN9YkQ3vjjqWyKsvWdvue09l3F2DlIW7kg=","Date":"Tue, 20 May 2025 17:18:58 +0200","From":"Jacopo Mondi ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"libcamera-devel@lists.libcamera.org, \n\tPaul Elder ","Subject":"Re: [RFC PATCH v1 8/8] utils: codegen: ipc: Simplify deserialization","Message-ID":"","References":"<20250515120012.3127231-1-barnabas.pocze@ideasonboard.com>\n\t<20250515120012.3127231-9-barnabas.pocze@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20250515120012.3127231-9-barnabas.pocze@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":34300,"web_url":"https://patchwork.libcamera.org/comment/34300/","msgid":"","date":"2025-05-21T07:44:33","subject":"Re: [RFC PATCH v1 8/8] utils: codegen: ipc: Simplify deserialization","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"Hi\n\n\n2025. 05. 20. 17:18 keltezéssel, Jacopo Mondi írta:\n> Hi Barnabás\n> \n> On Thu, May 15, 2025 at 02:00:12PM +0200, Barnabás Pőcze wrote:\n>> First, introduce the `SeriReader` type, which is a collection\n>> of bytes and file descriptors, with the appropriate methods to\n>> consume the first couples bytes / file descriptors.\n>>\n>> Then a new method is added to `IPCMessage` that returns an appropriately\n>> constructed `SeriReader` for parsing the message contents.\n>>\n>> Then three of the four `deserialize()` overloads are removed, the\n>> remaining one is converted to have a single `SeriReader` and an\n>> optional `ControlSerializer` as arguments.\n>>\n>> The remaining `deserialize()` function is also changed to return an\n>> `std::optional` to be able to report deserialization failure.\n>>\n>> There is also a more fundamental change in the serialization: previously,\n>> the number of bytes taken up by an item has been written before the serialized\n>> bytes (and conditionally the number of file descriptors) when the item is\n>> serialized as part of a struct, array, map, function parameter list. This\n>> is changed: the number of bytes and file descriptors are *not* serialized\n>> into the final buffer. This affords some simplification of the serialization\n>> related code paths, but most importantly, it greatly simplifies and unifies\n>> how an object is (de)serialized because the deserialization of every object\n>> becomes completely self-contained.\n>>\n>> As a consequence of that, strings now include their lengths as part of the\n>> string serialization, and it is not left to an \"upper\" layer.\n>>\n>> Another consequence is that an \"out parameter\" of a remote function call\n>> must be deserialized if a later out parameter is needed, even if itself\n>> is not. This does not appear to be a great limitation since in all\n>> situations presently none of the out parameters are ignored.\n>>\n>> Finally, the code generation templates are adapted to the above changes.\n>> This allows the simplification of the deserialization templates as now\n>> calling `IPADataSerializer::deserialize(reader, &controlSerializer_)`\n>> is appropriate for any type.\n> \n> Wow, there is quite some to unpack here.\n> \n> I wonder if the above changes related to the API, such as using\n> optional<> might be broken out to make things easier for review, but I\n> presume it might be hard.\n> \n> Anyway, first question first: a class for \"Deserializing\" (reading\n> from an array of Bytes) has been added, while the \"Serialization\"\n> (marshalling data to an array of Bytes) is still left to each\n> specialization of the IPADataSerializer<> class.\n> \n> Was this by choice ?\n\nI would like to add something like \"SeriWriter\" to do away with\nreturning vectors everywhere.\n\n\n> \n> The question also comes from the fact we'll need a serializer for all\n> the work to be done on ControlList in the public API. Do you plan to\n> use SeriReader in that context as well ?\n\nNot necessarily, some parts of it can be reused, but this class specifically\ncreated to be used in `IPADataSerializer`.\n\n\n> \n>>\n>> Bug: https://bugs.libcamera.org/show_bug.cgi?id=269\n>> Signed-off-by: Barnabás Pőcze \n>> ---\n>> .../libcamera/internal/ipa_data_serializer.h | 228 +++--------\n>> include/libcamera/internal/ipc_pipe.h | 3 +\n>> include/libcamera/internal/serialization.h | 91 +++++\n>> src/libcamera/ipa_data_serializer.cpp | 356 ++++--------------\n>> src/libcamera/ipc_pipe.cpp | 5 +\n>> test/ipc/unixsocket_ipc.cpp | 18 +-\n>> .../generated_serializer_test.cpp | 17 +-\n>> .../ipa_data_serializer_test.cpp | 32 +-\n>> .../module_ipa_proxy.cpp.tmpl | 30 +-\n>> .../module_ipa_proxy.h.tmpl | 5 +-\n>> .../module_ipa_proxy_worker.cpp.tmpl | 5 +-\n>> .../libcamera_templates/proxy_functions.tmpl | 113 +-----\n>> .../libcamera_templates/serializer.tmpl | 238 +-----------\n>> 13 files changed, 316 insertions(+), 825 deletions(-)\n>> create mode 100644 include/libcamera/internal/serialization.h\n>>\n>> diff --git a/include/libcamera/internal/ipa_data_serializer.h b/include/libcamera/internal/ipa_data_serializer.h\n>> index 564f59e25..0d03729a1 100644\n>> --- a/include/libcamera/internal/ipa_data_serializer.h\n>> +++ b/include/libcamera/internal/ipa_data_serializer.h\n>> @@ -7,6 +7,7 @@\n>>\n>> #pragma once\n>>\n>> +#include \n>> #include \n>> #include \n>> #include \n>> @@ -23,6 +24,7 @@\n>> #include \n>>\n>> #include \"libcamera/internal/control_serializer.h\"\n>> +#include \"libcamera/internal/serialization.h\"\n>>\n>> namespace libcamera {\n>>\n>> @@ -39,26 +41,6 @@ void appendPOD(std::vector &vec, T val)\n>> \tmemcpy(&*(vec.end() - byteWidth), &val, byteWidth);\n>> }\n>>\n>> -template> -\t std::enable_if_t> * = nullptr>\n>> -T readPOD(std::vector::const_iterator it, size_t pos,\n>> -\t std::vector::const_iterator end)\n>> -{\n>> -\tASSERT(pos + it < end);\n>> -\n>> -\tT ret = 0;\n>> -\tmemcpy(&ret, &(*(it + pos)), sizeof(ret));\n>> -\n>> -\treturn ret;\n>> -}\n>> -\n>> -template> -\t std::enable_if_t> * = nullptr>\n>> -T readPOD(std::vector &vec, size_t pos)\n>> -{\n>> -\treturn readPOD(vec.cbegin(), pos, vec.end());\n>> -}\n>> -\n>> } /* namespace */\n>>\n>> template\n>> @@ -68,20 +50,8 @@ public:\n>> \tstatic std::tuple, std::vector>\n>> \tserialize(const T &data, ControlSerializer *cs = nullptr);\n>>\n>> -\tstatic T deserialize(const std::vector &data,\n>> -\t\t\t ControlSerializer *cs = nullptr);\n>> -\tstatic T deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t ControlSerializer *cs = nullptr);\n>> -\n>> -\tstatic T deserialize(const std::vector &data,\n>> -\t\t\t const std::vector &fds,\n>> -\t\t\t ControlSerializer *cs = nullptr);\n>> -\tstatic T deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t std::vector::const_iterator fdsBegin,\n>> -\t\t\t std::vector::const_iterator fdsEnd,\n>> -\t\t\t ControlSerializer *cs = nullptr);\n>> +\t[[nodiscard]] static std::optional\n>> +\tdeserialize(SeriReader &reader, ControlSerializer *cs = nullptr);\n>> };\n>>\n>> #ifndef __DOXYGEN__\n>> @@ -121,9 +91,6 @@ public:\n>> \t\t\tstd::tie(dvec, fvec) =\n>> \t\t\t\tIPADataSerializer::serialize(it, cs);\n>>\n>> -\t\t\tappendPOD(dataVec, dvec.size());\n>> -\t\t\tappendPOD(dataVec, fvec.size());\n>> -\n> \n> You should update the comment that describes the serialized format,\n> for all specializations where the serialized formats is changed to\n> remove the in-line sizes\n> \n> \n>> \t\t\tdataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n>> \t\t\tfdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n>> \t\t}\n>> @@ -131,52 +98,25 @@ public:\n>> \t\treturn { dataVec, fdsVec };\n>> \t}\n>>\n>> -\tstatic std::vector deserialize(std::vector &data, ControlSerializer *cs = nullptr)\n>> +\t[[nodiscard]] static std::optional>\n>> +\tdeserialize(SeriReader &reader, ControlSerializer *cs = nullptr)\n>> \t{\n>> -\t\treturn deserialize(data.cbegin(), data.cend(), cs);\n>> -\t}\n>> +\t\tuint32_t vecLen;\n>> +\t\tif (!reader.read(vecLen))\n>> +\t\t\treturn {};\n> \n> Took me a while to understand this one. You're reading the first 4\n> bytes integer that contains the serialized vector size.\n> \n> The API of:\n> \n> \ttemplate\n> \t[[nodiscard]] bool read(Ts &...xs) {}\n> \n> it's a bit alien to me, as\n> 1) It returns a pointer to the serialized data, and that's fine, but\n> 2) Loads the content of the serialized data (of size sizeof...(xs))\n> into ...(xs)\n> \n> As an aging C programmer I would have expected an API that uses a\n> pointer for an output argument, but this would certainly make\n> impractical to use the (Ts &...xs) magic...\n\nPointers could be used, I just like references better (e.g. no nullability issues).\n\n\n> \n>>\n>> -\tstatic std::vector deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t\t\t ControlSerializer *cs = nullptr)\n>> -\t{\n>> -\t\tstd::vector fds;\n>> -\t\treturn deserialize(dataBegin, dataEnd, fds.cbegin(), fds.cend(), cs);\n>> -\t}\n>> +\t\tstd::vector ret;\n>> +\t\tret.reserve(vecLen);\n> \n> Is\n> std::vector ret(vecLen)\n> \n> different ?\n\nYes, that will fill the vector with `vecLen` default constructed elements,\nwhile `reserve()` only allocates storage for `vecLen` elements.\n\n\n> \n>>\n>> -\tstatic std::vector deserialize(std::vector &data, std::vector &fds,\n>> -\t\t\t\t\t ControlSerializer *cs = nullptr)\n>> -\t{\n>> -\t\treturn deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n>> -\t}\n>> -\n>> -\tstatic std::vector deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t\t\t std::vector::const_iterator fdsBegin,\n>> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> -\t\t\t\t\t ControlSerializer *cs = nullptr)\n>> -\t{\n>> -\t\tuint32_t vecLen = readPOD(dataBegin, 0, dataEnd);\n>> -\t\tstd::vector ret(vecLen);\n>> -\n>> -\t\tstd::vector::const_iterator dataIter = dataBegin + 4;\n>> -\t\tstd::vector::const_iterator fdIter = fdsBegin;\n>> \t\tfor (uint32_t i = 0; i < vecLen; i++) {\n>> -\t\t\tuint32_t sizeofData = readPOD(dataIter, 0, dataEnd);\n>> -\t\t\tuint32_t sizeofFds = readPOD(dataIter, 4, dataEnd);\n>> -\t\t\tdataIter += 8;\n>> -\n>> -\t\t\tret[i] = IPADataSerializer::deserialize(dataIter,\n>> -\t\t\t\t\t\t\t\t dataIter + sizeofData,\n>> -\t\t\t\t\t\t\t\t fdIter,\n>> -\t\t\t\t\t\t\t\t fdIter + sizeofFds,\n>> -\t\t\t\t\t\t\t\t cs);\n>> -\n>> -\t\t\tdataIter += sizeofData;\n>> -\t\t\tfdIter += sizeofFds;\n>> +\t\t\tauto item = IPADataSerializer::deserialize(reader, cs);\n>> +\t\t\tif (!item)\n>> +\t\t\t\treturn {};\n>> +\n>> +\t\t\tret.emplace_back(std::move(*item));\n> \n> Does std::move() make any practical difference ? The data have to\n> copied to one container to the other anyway, don't they ?\n\nIf you have a non-trivially copyable type, then yes. For example,\na vector of `SharedFD` does benefit from the `std::move()` here.\n(And by extension everything that has `SharedFd`, e.g. `IPABuffer` in `core.mojom`.)\n\n\n> \n>> \t\t}\n>>\n>> -\t\treturn ret;\n>> +\t\treturn std::move(ret);\n> \n> Here as well, does move() make any difference ? I presume it does if\n> the type V underlying the vector is move-constructable ?\n\nAs it turns out, this move is redundant, `return ret` will move implicitly,\nso no difference here.\n\n\n> \n>> \t}\n>> };\n>>\n>> @@ -218,18 +158,12 @@ public:\n>> \t\t\tstd::tie(dvec, fvec) =\n>> \t\t\t\tIPADataSerializer::serialize(it.first, cs);\n>>\n>> -\t\t\tappendPOD(dataVec, dvec.size());\n>> -\t\t\tappendPOD(dataVec, fvec.size());\n>> -\n>> \t\t\tdataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n>> \t\t\tfdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n>>\n>> \t\t\tstd::tie(dvec, fvec) =\n>> \t\t\t\tIPADataSerializer::serialize(it.second, cs);\n>>\n>> -\t\t\tappendPOD(dataVec, dvec.size());\n>> -\t\t\tappendPOD(dataVec, fvec.size());\n>> -\n> \n> Please update the comment here as well\n> \n>> \t\t\tdataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n>> \t\t\tfdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n>> \t\t}\n>> @@ -237,63 +171,25 @@ public:\n>> \t\treturn { dataVec, fdsVec };\n>> \t}\n>>\n>> -\tstatic std::map deserialize(std::vector &data, ControlSerializer *cs = nullptr)\n>> -\t{\n>> -\t\treturn deserialize(data.cbegin(), data.cend(), cs);\n>> -\t}\n>> -\n>> -\tstatic std::map deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t\t\t ControlSerializer *cs = nullptr)\n>> +\t[[nodiscard]] static std::optional>\n>> +\tdeserialize(SeriReader &reader, ControlSerializer *cs = nullptr)\n>> \t{\n>> -\t\tstd::vector fds;\n>> -\t\treturn deserialize(dataBegin, dataEnd, fds.cbegin(), fds.cend(), cs);\n>> -\t}\n>> +\t\tuint32_t mapLen;\n>> +\t\tif (!reader.read(mapLen))\n>> +\t\t\treturn {};\n>>\n>> -\tstatic std::map deserialize(std::vector &data, std::vector &fds,\n>> -\t\t\t\t\t ControlSerializer *cs = nullptr)\n>> -\t{\n>> -\t\treturn deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n>> -\t}\n>> -\n>> -\tstatic std::map deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t\t\t std::vector::const_iterator fdsBegin,\n>> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> -\t\t\t\t\t ControlSerializer *cs = nullptr)\n>> -\t{\n>> \t\tstd::map ret;\n>>\n>> -\t\tuint32_t mapLen = readPOD(dataBegin, 0, dataEnd);\n>> -\n>> -\t\tstd::vector::const_iterator dataIter = dataBegin + 4;\n>> -\t\tstd::vector::const_iterator fdIter = fdsBegin;\n>> \t\tfor (uint32_t i = 0; i < mapLen; i++) {\n>> -\t\t\tuint32_t sizeofData = readPOD(dataIter, 0, dataEnd);\n>> -\t\t\tuint32_t sizeofFds = readPOD(dataIter, 4, dataEnd);\n>> -\t\t\tdataIter += 8;\n>> -\n>> -\t\t\tK key = IPADataSerializer::deserialize(dataIter,\n>> -\t\t\t\t\t\t\t\t dataIter + sizeofData,\n>> -\t\t\t\t\t\t\t\t fdIter,\n>> -\t\t\t\t\t\t\t\t fdIter + sizeofFds,\n>> -\t\t\t\t\t\t\t\t cs);\n>> -\n>> -\t\t\tdataIter += sizeofData;\n>> -\t\t\tfdIter += sizeofFds;\n>> -\t\t\tsizeofData = readPOD(dataIter, 0, dataEnd);\n>> -\t\t\tsizeofFds = readPOD(dataIter, 4, dataEnd);\n>> -\t\t\tdataIter += 8;\n>> -\n>> -\t\t\tconst V value = IPADataSerializer::deserialize(dataIter,\n>> -\t\t\t\t\t\t\t\t\t dataIter + sizeofData,\n>> -\t\t\t\t\t\t\t\t\t fdIter,\n>> -\t\t\t\t\t\t\t\t\t fdIter + sizeofFds,\n>> -\t\t\t\t\t\t\t\t\t cs);\n>> -\t\t\tret.insert({ key, value });\n>> -\n>> -\t\t\tdataIter += sizeofData;\n>> -\t\t\tfdIter += sizeofFds;\n>> +\t\t\tauto key = IPADataSerializer::deserialize(reader, cs);\n>> +\t\t\tif (!key)\n>> +\t\t\t\treturn {};\n>> +\n>> +\t\t\tauto value = IPADataSerializer::deserialize(reader, cs);\n>> +\t\t\tif (!value)\n>> +\t\t\t\treturn {};\n>> +\n>> +\t\t\tret.try_emplace(std::move(*key), std::move(*value));\n> \n> The number of lines saved is impressive!\n> \n>> \t\t}\n>>\n>> \t\treturn ret;\n>> @@ -314,33 +210,14 @@ public:\n>> \t\treturn { dataVec, {} };\n>> \t}\n>>\n>> -\tstatic Flags deserialize(std::vector &data,\n>> -\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> -\t{\n>> -\t\treturn deserialize(data.cbegin(), data.cend());\n>> -\t}\n>> -\n>> -\tstatic Flags deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> +\t[[nodiscard]] static std::optional>\n>> +\tdeserialize(SeriReader &reader, [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> \t{\n>> -\t\treturn Flags{ static_cast(readPOD(dataBegin, 0, dataEnd)) };\n>> -\t}\n>> +\t\tuint32_t value;\n>> +\t\tif (!reader.read(value))\n>> +\t\t\treturn {};\n>>\n>> -\tstatic Flags deserialize(std::vector &data,\n>> -\t\t\t\t [[maybe_unused]] std::vector &fds,\n>> -\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> -\t{\n>> -\t\treturn deserialize(data.cbegin(), data.cend());\n>> -\t}\n>> -\n>> -\tstatic Flags deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n>> -\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> -\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> -\t{\n>> -\t\treturn deserialize(dataBegin, dataEnd);\n>> +\t\treturn Flags{ static_cast(value) };\n>> \t}\n>> };\n>>\n>> @@ -360,33 +237,14 @@ public:\n>> \t\treturn { dataVec, {} };\n>> \t}\n>>\n>> -\tstatic E deserialize(std::vector &data,\n>> -\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> +\t[[nodiscard]] static std::optional\n>> +\tdeserialize(SeriReader &reader, [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> \t{\n>> -\t\treturn deserialize(data.cbegin(), data.cend());\n>> -\t}\n>> +\t\tU value;\n>> +\t\tif (!reader.read(value))\n>> +\t\t\treturn {};\n>>\n>> -\tstatic E deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> -\t{\n>> -\t\treturn static_cast(readPOD(dataBegin, 0, dataEnd));\n>> -\t}\n>> -\n>> -\tstatic E deserialize(std::vector &data,\n>> -\t\t\t [[maybe_unused]] std::vector &fds,\n>> -\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> -\t{\n>> -\t\treturn deserialize(data.cbegin(), data.cend());\n>> -\t}\n>> -\n>> -\tstatic E deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n>> -\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> -\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> -\t{\n>> -\t\treturn deserialize(dataBegin, dataEnd);\n>> +\t\treturn static_cast(value);\n>> \t}\n>> };\n>>\n>> diff --git a/include/libcamera/internal/ipc_pipe.h b/include/libcamera/internal/ipc_pipe.h\n>> index 418c4622f..2b6cde042 100644\n>> --- a/include/libcamera/internal/ipc_pipe.h\n>> +++ b/include/libcamera/internal/ipc_pipe.h\n>> @@ -14,6 +14,7 @@\n>> #include \n>>\n>> #include \"libcamera/internal/ipc_unixsocket.h\"\n>> +#include \"libcamera/internal/serialization.h\"\n>>\n>> namespace libcamera {\n>>\n>> @@ -40,6 +41,8 @@ public:\n>> \tconst std::vector &data() const { return data_; }\n>> \tconst std::vector &fds() const { return fds_; }\n>>\n>> +\tSeriReader reader() const { return { data(), fds() }; }\n>> +\n>> private:\n>> \tHeader header_;\n>>\n>> diff --git a/include/libcamera/internal/serialization.h b/include/libcamera/internal/serialization.h\n>> new file mode 100644\n>> index 000000000..daa7e5438\n>> --- /dev/null\n>> +++ b/include/libcamera/internal/serialization.h\n>> @@ -0,0 +1,91 @@\n>> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n>> +/*\n>> + * Copyright (C) 2025, Ideas On Board Oy\n>> + *\n>> + * Data (de)serialization helper structures\n>> + */\n>> +#pragma once\n>> +\n> \n> Missing a few includes\n> \n>> +#include \n>> +#include \n>> +#include \n>> +\n>> +#include \n>> +#include \n>> +\n>> +namespace libcamera {\n>> +\n>> +#ifndef __DOXYGEN__\n>> +class SeriReader\n>> +{\n>> +public:\n>> +\tSeriReader(Span data, Span fds = {})\n>> +\t\t: data_(data),\n>> +\t\t fds_(fds)\n>> +\t{\n>> +\t}\n>> +\n>> +\tSeriReader(Span data, Span fds = {})\n>> +\t\t: data_(reinterpret_cast(data.data()),\n>> +\t\t\treinterpret_cast(data.data() + data.size())),\n> \n> ah so uint8_t is not directly convertible to a byte ?\n\n`std::byte` is its own thing, so not convertible; and I prefer that because it's\nharder to misuse than `uint8_t` (~ `unsigned char`).\n\n\n> \n>> +\t\t fds_(fds)\n>> +\t{\n>> +\t}\n>> +\n>> +\t[[nodiscard]] const std::byte *consume(std::size_t s)\n>> +\t{\n>> +\t\tif (data_.size() < s)\n>> +\t\t\treturn nullptr;\n>> +\n>> +\t\tconst auto *p = data_.data();\n>> +\t\tdata_ = data_.subspan(s);\n>> +\n>> +\t\treturn p;\n>> +\t}\n>> +\n>> +\ttemplate\n>> +\t[[nodiscard]] bool read(Ts &...xs)\n> \n> My head didn't explode just because you already showed me this trick\n> \n>> +\t{\n>> +\t\tstatic_assert((std::is_trivially_copyable_v && ...));\n>> +\n>> +\t\tconst auto *p = consume((sizeof(xs) + ...));\n> \n> So we read the whole space required to deserialize multiple xs\n> \n>> +\t\tif (p)\n>> +\t\t\t((memcpy(&xs, p, sizeof(xs)), p += sizeof(xs)), ...);\n> \n> And we copy them one by one\n> \n> brilliant\n> \n>> +\n>> +\t\treturn p;\n> \n> \n> \n>> +\t}\n>> +\n>> +\ttemplate\n>> +\t[[nodiscard]] auto read()\n>> +\t{\n>> +\t\tstd::tuple xs;\n>> +\t\tbool ok = std::apply([&](auto &...vs) {\n>> +\t\t\treturn read(vs...);\n>> +\t\t}, xs);\n>> +\n>> +\t\tif constexpr (sizeof...(Ts) == 1)\n>> +\t\t\treturn ok ? std::optional(std::get<0>(xs)) : std::nullopt;\n>> +\t\telse\n>> +\t\t\treturn ok ? std::optional(xs) : std::nullopt;\n> \n> If this last version doesn't work for sizeof...(Ts) == 1\n> (I know that, I tried removing the if branch) does it mean that\n> \n> \t\tif constexpr (sizeof...(Ts) == 1)\n> \n> is compile-time evaluated and the compiler knows what option to pick.\n\nYes.\n\n\n> \n> Now the main question, do you foresee an use for template argument\n> packs ? As far as I understood it, all the exiting code works with a\n> single argument, or have I missed something ?\n\nThis was written before I converted things, and as it turns out, this\noverload of `read()` is not really used, so it may be removed.\n\n\n> \n> I'll stop the review here for now, enough material to ponder up for me :)\n> \n> Thanks\n> j\n> \n>> +\t}\n>> +\n>> +\t[[nodiscard]] const SharedFD *nextFd()\n>> +\t{\n>> +\t\tif (fds_.empty())\n>> +\t\t\treturn nullptr;\n>> +\n>> +\t\tconst auto *p = fds_.data();\n>> +\t\tfds_ = fds_.subspan(1);\n>> +\n>> +\t\treturn p;\n>> +\t}\n>> +\n>> +\t[[nodiscard]] bool empty() const { return data_.empty() && fds_.empty(); }\n>> +\n>> +private:\n>> +\tSpan data_;\n>> +\tSpan fds_;\n>> +};\n>> +#endif\n>> +\n>> +} /* namespace libcamera */\n>> diff --git a/src/libcamera/ipa_data_serializer.cpp b/src/libcamera/ipa_data_serializer.cpp\n>> index 0537f785b..979a1e0db 100644\n>> --- a/src/libcamera/ipa_data_serializer.cpp\n>> +++ b/src/libcamera/ipa_data_serializer.cpp\n>> @@ -50,42 +50,6 @@ namespace {\n>> * generated IPA proxies.\n>> */\n>>\n>> -/**\n>> - * \\fn template T readPOD(std::vector::iterator it, size_t pos,\n>> - * \t\t\t\t std::vector::iterator end)\n>> - * \\brief Read POD from byte vector, in little-endian order\n>> - * \\tparam T Type of POD to read\n>> - * \\param[in] it Iterator of byte vector to read from\n>> - * \\param[in] pos Index in byte vector to read from\n>> - * \\param[in] end Iterator marking end of byte vector\n>> - *\n>> - * This function is meant to be used by the IPA data serializer, and the\n>> - * generated IPA proxies.\n>> - *\n>> - * If the \\a pos plus the byte-width of the desired POD is past \\a end, it is\n>> - * a fata error will occur, as it means there is insufficient data for\n>> - * deserialization, which should never happen.\n>> - *\n>> - * \\return The POD read from \\a it at index \\a pos\n>> - */\n>> -\n>> -/**\n>> - * \\fn template T readPOD(std::vector &vec, size_t pos)\n>> - * \\brief Read POD from byte vector, in little-endian order\n>> - * \\tparam T Type of POD to read\n>> - * \\param[in] vec Byte vector to read from\n>> - * \\param[in] pos Index in vec to start reading from\n>> - *\n>> - * This function is meant to be used by the IPA data serializer, and the\n>> - * generated IPA proxies.\n>> - *\n>> - * If the \\a pos plus the byte-width of the desired POD is past the end of\n>> - * \\a vec, a fatal error will occur, as it means there is insufficient data\n>> - * for deserialization, which should never happen.\n>> - *\n>> - * \\return The POD read from \\a vec at index \\a pos\n>> - */\n>> -\n>> } /* namespace */\n>>\n>> /**\n>> @@ -106,80 +70,13 @@ namespace {\n>>\n>> /**\n>> * \\fn template IPADataSerializer::deserialize(\n>> - * \tconst std::vector &data,\n>> + * \tSeriReader &reader,\n>> * \tControlSerializer *cs = nullptr)\n>> - * \\brief Deserialize byte vector into an object\n>> + * \\brief Deserialize bytes and file descriptors vector into an object\n>> * \\tparam T Type of object to deserialize to\n>> - * \\param[in] data Byte vector to deserialize from\n>> + * \\param[in] reader Source of bytes and file descriptors\n>> * \\param[in] cs ControlSerializer\n>> *\n>> - * This version of deserialize() can be used if the object type \\a T and its\n>> - * members don't have any SharedFD.\n>> - *\n>> - * \\a cs is only necessary if the object type \\a T or its members contain\n>> - * ControlList or ControlInfoMap.\n>> - *\n>> - * \\return The deserialized object\n>> - */\n>> -\n>> -/**\n>> - * \\fn template IPADataSerializer::deserialize(\n>> - * \tstd::vector::const_iterator dataBegin,\n>> - * \tstd::vector::const_iterator dataEnd,\n>> - * \tControlSerializer *cs = nullptr)\n>> - * \\brief Deserialize byte vector into an object\n>> - * \\tparam T Type of object to deserialize to\n>> - * \\param[in] dataBegin Begin iterator of byte vector to deserialize from\n>> - * \\param[in] dataEnd End iterator of byte vector to deserialize from\n>> - * \\param[in] cs ControlSerializer\n>> - *\n>> - * This version of deserialize() can be used if the object type \\a T and its\n>> - * members don't have any SharedFD.\n>> - *\n>> - * \\a cs is only necessary if the object type \\a T or its members contain\n>> - * ControlList or ControlInfoMap.\n>> - *\n>> - * \\return The deserialized object\n>> - */\n>> -\n>> -/**\n>> - * \\fn template IPADataSerializer::deserialize(\n>> - * \tconst std::vector &data,\n>> - * \tconst std::vector &fds,\n>> - * \tControlSerializer *cs = nullptr)\n>> - * \\brief Deserialize byte vector and fd vector into an object\n>> - * \\tparam T Type of object to deserialize to\n>> - * \\param[in] data Byte vector to deserialize from\n>> - * \\param[in] fds Fd vector to deserialize from\n>> - * \\param[in] cs ControlSerializer\n>> - *\n>> - * This version of deserialize() (or the iterator version) must be used if\n>> - * the object type \\a T or its members contain SharedFD.\n>> - *\n>> - * \\a cs is only necessary if the object type \\a T or its members contain\n>> - * ControlList or ControlInfoMap.\n>> - *\n>> - * \\return The deserialized object\n>> - */\n>> -\n>> -/**\n>> - * \\fn template IPADataSerializer::deserialize(\n>> - * \tstd::vector::const_iterator dataBegin,\n>> - * \tstd::vector::const_iterator dataEnd,\n>> - * \tstd::vector::const_iterator fdsBegin,\n>> - * \tstd::vector::const_iterator fdsEnd,\n>> - * \tControlSerializer *cs = nullptr)\n>> - * \\brief Deserialize byte vector and fd vector into an object\n>> - * \\tparam T Type of object to deserialize to\n>> - * \\param[in] dataBegin Begin iterator of byte vector to deserialize from\n>> - * \\param[in] dataEnd End iterator of byte vector to deserialize from\n>> - * \\param[in] fdsBegin Begin iterator of fd vector to deserialize from\n>> - * \\param[in] fdsEnd End iterator of fd vector to deserialize from\n>> - * \\param[in] cs ControlSerializer\n>> - *\n>> - * This version of deserialize() (or the vector version) must be used if\n>> - * the object type \\a T or its members contain SharedFD.\n>> - *\n>> * \\a cs is only necessary if the object type \\a T or its members contain\n>> * ControlList or ControlInfoMap.\n>> *\n>> @@ -202,37 +99,11 @@ IPADataSerializer::serialize(const type &data,\t\t\t\\\n>> }\t\t\t\t\t\t\t\t\t\\\n>> \t\t\t\t\t\t\t\t\t\\\n>> template<>\t\t\t\t\t\t\t\t\\\n>> -type IPADataSerializer::deserialize(std::vector::const_iterator dataBegin, \\\n>> -\t\t\t\t\t std::vector::const_iterator dataEnd, \\\n>> -\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs) \\\n>> +std::optional IPADataSerializer::deserialize(SeriReader &reader, \\\n>> +\t\t\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs) \\\n>> {\t\t\t\t\t\t\t\t\t\\\n>> -\treturn readPOD(dataBegin, 0, dataEnd);\t\t\t\\\n>> +\treturn reader.read();\t\t\t\t\t\\\n>> }\t\t\t\t\t\t\t\t\t\\\n>> -\t\t\t\t\t\t\t\t\t\\\n>> -template<>\t\t\t\t\t\t\t\t\\\n>> -type IPADataSerializer::deserialize(const std::vector &data, \\\n>> -\t\t\t\t\t ControlSerializer *cs)\t\\\n>> -{\t\t\t\t\t\t\t\t\t\\\n>> -\treturn deserialize(data.cbegin(), data.end(), cs);\t\t\\\n>> -}\t\t\t\t\t\t\t\t\t\\\n>> -\t\t\t\t\t\t\t\t\t\\\n>> -template<>\t\t\t\t\t\t\t\t\\\n>> -type IPADataSerializer::deserialize(const std::vector &data, \\\n>> -\t\t\t\t\t [[maybe_unused]] const std::vector &fds, \\\n>> -\t\t\t\t\t ControlSerializer *cs)\t\\\n>> -{\t\t\t\t\t\t\t\t\t\\\n>> -\treturn deserialize(data.cbegin(), data.end(), cs);\t\t\\\n>> -}\t\t\t\t\t\t\t\t\t\\\n>> -\t\t\t\t\t\t\t\t\t\\\n>> -template<>\t\t\t\t\t\t\t\t\\\n>> -type IPADataSerializer::deserialize(std::vector::const_iterator dataBegin, \\\n>> -\t\t\t\t\t std::vector::const_iterator dataEnd, \\\n>> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin, \\\n>> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd, \\\n>> -\t\t\t\t\t ControlSerializer *cs)\t\\\n>> -{\t\t\t\t\t\t\t\t\t\\\n>> -\treturn deserialize(dataBegin, dataEnd, cs);\t\t\t\\\n>> -}\n>>\n>> DEFINE_POD_SERIALIZER(bool)\n>> DEFINE_POD_SERIALIZER(uint8_t)\n>> @@ -256,44 +127,29 @@ std::tuple, std::vector>\n>> IPADataSerializer::serialize(const std::string &data,\n>> \t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n>> {\n>> -\treturn { { data.cbegin(), data.end() }, {} };\n>> -}\n>> +\tstd::vector dataVec;\n>>\n>> -template<>\n>> -std::string\n>> -IPADataSerializer::deserialize(const std::vector &data,\n>> -\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n>> -{\n>> -\treturn { data.cbegin(), data.cend() };\n>> -}\n>> +\tASSERT(data.size() <= std::numeric_limits::max());\n>> +\tappendPOD(dataVec, data.size());\n>> +\tdataVec.insert(dataVec.end(), data.c_str(), data.c_str() + data.size());\n>>\n>> -template<>\n>> -std::string\n>> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n>> -{\n>> -\treturn { dataBegin, dataEnd };\n>> +\treturn { dataVec, {} };\n>> }\n>>\n>> template<>\n>> -std::string\n>> -IPADataSerializer::deserialize(const std::vector &data,\n>> -\t\t\t\t\t [[maybe_unused]] const std::vector &fds,\n>> +std::optional\n>> +IPADataSerializer::deserialize(SeriReader &reader,\n>> \t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n>> {\n>> -\treturn { data.cbegin(), data.cend() };\n>> -}\n>> +\tuint32_t length;\n>> +\tif (!reader.read(length))\n>> +\t\treturn {};\n>>\n>> -template<>\n>> -std::string\n>> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n>> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> -\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n>> -{\n>> -\treturn { dataBegin, dataEnd };\n>> +\tconst auto *p = reader.consume(length);\n>> +\tif (!p)\n>> +\t\treturn {};\n>> +\n>> +\treturn { { reinterpret_cast(p), std::size_t(length) } };\n>> }\n>>\n>> /*\n>> @@ -356,73 +212,43 @@ IPADataSerializer::serialize(const ControlList &data, ControlSerial\n>> }\n>>\n>> template<>\n>> -ControlList\n>> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n>> +std::optional\n>> +IPADataSerializer::deserialize(SeriReader &reader,\n>> \t\t\t\t\t ControlSerializer *cs)\n>> {\n>> \tif (!cs)\n>> \t\tLOG(IPADataSerializer, Fatal)\n>> \t\t\t<< \"ControlSerializer not provided for deserialization of ControlList\";\n>>\n>> -\tif (std::distance(dataBegin, dataEnd) < 8)\n>> -\t\treturn {};\n>> -\n>> -\tuint32_t infoDataSize = readPOD(dataBegin, 0, dataEnd);\n>> -\tuint32_t listDataSize = readPOD(dataBegin, 4, dataEnd);\n>> -\n>> -\tstd::vector::const_iterator it = dataBegin + 8;\n>> -\n>> -\tif (infoDataSize + listDataSize < infoDataSize ||\n>> -\t static_cast(std::distance(it, dataEnd)) < infoDataSize + listDataSize)\n>> +\tuint32_t infoDataSize, listDataSize;\n>> +\tif (!reader.read(infoDataSize, listDataSize))\n>> \t\treturn {};\n>>\n>> \tif (infoDataSize > 0) {\n>> -\t\tByteStreamBuffer buffer(&*it, infoDataSize);\n>> +\t\tconst auto *p = reader.consume(infoDataSize);\n>> +\t\tif (!p)\n>> +\t\t\treturn {};\n>> +\n>> +\t\tByteStreamBuffer buffer(reinterpret_cast(p), infoDataSize);\n>> \t\tControlInfoMap map = cs->deserialize(buffer);\n>> \t\t/* It's fine if map is empty. */\n>> \t\tif (buffer.overflow()) {\n>> \t\t\tLOG(IPADataSerializer, Error)\n>> \t\t\t\t<< \"Failed to deserialize ControlLists's ControlInfoMap: buffer overflow\";\n>> -\t\t\treturn ControlList();\n>> +\t\t\treturn {};\n>> \t\t}\n>> \t}\n>>\n>> -\tit += infoDataSize;\n>> -\tByteStreamBuffer buffer(&*it, listDataSize);\n>> +\tconst auto *p = reader.consume(listDataSize);\n>> +\n>> +\tByteStreamBuffer buffer(reinterpret_cast(p), listDataSize);\n>> \tControlList list = cs->deserialize(buffer);\n>> -\tif (buffer.overflow())\n>> +\tif (buffer.overflow()) {\n>> \t\tLOG(IPADataSerializer, Error) << \"Failed to deserialize ControlList: buffer overflow\";\n>> +\t\treturn {};\n>> +\t}\n>>\n>> -\treturn list;\n>> -}\n>> -\n>> -template<>\n>> -ControlList\n>> -IPADataSerializer::deserialize(const std::vector &data,\n>> -\t\t\t\t\t ControlSerializer *cs)\n>> -{\n>> -\treturn deserialize(data.cbegin(), data.end(), cs);\n>> -}\n>> -\n>> -template<>\n>> -ControlList\n>> -IPADataSerializer::deserialize(const std::vector &data,\n>> -\t\t\t\t\t [[maybe_unused]] const std::vector &fds,\n>> -\t\t\t\t\t ControlSerializer *cs)\n>> -{\n>> -\treturn deserialize(data.cbegin(), data.end(), cs);\n>> -}\n>> -\n>> -template<>\n>> -ControlList\n>> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n>> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> -\t\t\t\t\t ControlSerializer *cs)\n>> -{\n>> -\treturn deserialize(dataBegin, dataEnd, cs);\n>> +\treturn std::move(list);\n>> }\n>>\n>> /*\n>> @@ -458,57 +284,28 @@ IPADataSerializer::serialize(const ControlInfoMap &map,\n>> }\n>>\n>> template<>\n>> -ControlInfoMap\n>> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n>> +std::optional\n>> +IPADataSerializer::deserialize(SeriReader &reader,\n>> \t\t\t\t\t ControlSerializer *cs)\n>> {\n>> \tif (!cs)\n>> \t\tLOG(IPADataSerializer, Fatal)\n>> \t\t\t<< \"ControlSerializer not provided for deserialization of ControlInfoMap\";\n>>\n>> -\tif (std::distance(dataBegin, dataEnd) < 4)\n>> +\tuint32_t infoDataSize;\n>> +\tif (!reader.read(infoDataSize))\n>> \t\treturn {};\n>>\n>> -\tuint32_t infoDataSize = readPOD(dataBegin, 0, dataEnd);\n>> -\n>> -\tstd::vector::const_iterator it = dataBegin + 4;\n>> -\n>> -\tif (static_cast(std::distance(it, dataEnd)) < infoDataSize)\n>> +\tconst auto *p = reader.consume(infoDataSize);\n>> +\tif (!p)\n>> \t\treturn {};\n>>\n>> -\tByteStreamBuffer buffer(&*it, infoDataSize);\n>> +\tByteStreamBuffer buffer(reinterpret_cast(p), infoDataSize);\n>> \tControlInfoMap map = cs->deserialize(buffer);\n>> +\tif (buffer.overflow())\n>> +\t\treturn {};\n>>\n>> -\treturn map;\n>> -}\n>> -\n>> -template<>\n>> -ControlInfoMap\n>> -IPADataSerializer::deserialize(const std::vector &data,\n>> -\t\t\t\t\t ControlSerializer *cs)\n>> -{\n>> -\treturn deserialize(data.cbegin(), data.end(), cs);\n>> -}\n>> -\n>> -template<>\n>> -ControlInfoMap\n>> -IPADataSerializer::deserialize(const std::vector &data,\n>> -\t\t\t\t\t [[maybe_unused]] const std::vector &fds,\n>> -\t\t\t\t\t ControlSerializer *cs)\n>> -{\n>> -\treturn deserialize(data.cbegin(), data.end(), cs);\n>> -}\n>> -\n>> -template<>\n>> -ControlInfoMap\n>> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n>> -\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> -\t\t\t\t\t ControlSerializer *cs)\n>> -{\n>> -\treturn deserialize(dataBegin, dataEnd, cs);\n>> +\treturn std::move(map);\n>> }\n>>\n>> /*\n>> @@ -542,27 +339,23 @@ IPADataSerializer::serialize(const SharedFD &data,\n>> }\n>>\n>> template<>\n>> -SharedFD IPADataSerializer::deserialize([[maybe_unused]] std::vector::const_iterator dataBegin,\n>> -\t\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator dataEnd,\n>> -\t\t\t\t\t\t std::vector::const_iterator fdsBegin,\n>> -\t\t\t\t\t\t std::vector::const_iterator fdsEnd,\n>> -\t\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n>> +std::optional\n>> +IPADataSerializer::deserialize(SeriReader &reader,\n>> +\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n>> {\n>> -\tASSERT(std::distance(dataBegin, dataEnd) >= 4);\n>> +\tuint32_t valid;\n>>\n>> -\tuint32_t valid = readPOD(dataBegin, 0, dataEnd);\n>> +\tif (!reader.read(valid))\n>> +\t\treturn {};\n>>\n>> -\tASSERT(!(valid && std::distance(fdsBegin, fdsEnd) < 1));\n>> +\tif (!valid)\n>> +\t\treturn SharedFD{};\n>>\n>> -\treturn valid ? *fdsBegin : SharedFD();\n>> -}\n>> +\tconst auto *fd = reader.nextFd();\n>> +\tif (!fd)\n>> +\t\treturn {};\n>>\n>> -template<>\n>> -SharedFD IPADataSerializer::deserialize(const std::vector &data,\n>> -\t\t\t\t\t\t const std::vector &fds,\n>> -\t\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n>> -{\n>> -\treturn deserialize(data.cbegin(), data.end(), fds.cbegin(), fds.end());\n>> +\treturn *fd;\n>> }\n>>\n>> /*\n>> @@ -594,30 +387,19 @@ IPADataSerializer::serialize(const FrameBuffer::Plane &data,\n>> }\n>>\n>> template<>\n>> -FrameBuffer::Plane\n>> -IPADataSerializer::deserialize(std::vector::const_iterator dataBegin,\n>> -\t\t\t\t\t\t std::vector::const_iterator dataEnd,\n>> -\t\t\t\t\t\t std::vector::const_iterator fdsBegin,\n>> -\t\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> +std::optional\n>> +IPADataSerializer::deserialize(SeriReader &reader,\n>> \t\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs)\n>> {\n>> -\tFrameBuffer::Plane ret;\n>> -\n>> -\tret.fd = IPADataSerializer::deserialize(dataBegin, dataBegin + 4,\n>> -\t\t\t\t\t\t\t fdsBegin, fdsBegin + 1);\n>> -\tret.offset = readPOD(dataBegin, 4, dataEnd);\n>> -\tret.length = readPOD(dataBegin, 8, dataEnd);\n>> +\tauto fd = IPADataSerializer::deserialize(reader);\n>> +\tif (!fd)\n>> +\t\treturn {};\n>>\n>> -\treturn ret;\n>> -}\n>> +\tuint32_t offset, length;\n>> +\tif (!reader.read(offset, length))\n>> +\t\treturn {};\n>>\n>> -template<>\n>> -FrameBuffer::Plane\n>> -IPADataSerializer::deserialize(const std::vector &data,\n>> -\t\t\t\t\t\t const std::vector &fds,\n>> -\t\t\t\t\t\t ControlSerializer *cs)\n>> -{\n>> -\treturn deserialize(data.cbegin(), data.end(), fds.cbegin(), fds.end(), cs);\n>> +\treturn { { std::move(*fd), offset, length } };\n>> }\n>>\n>> #endif /* __DOXYGEN__ */\n>> diff --git a/src/libcamera/ipc_pipe.cpp b/src/libcamera/ipc_pipe.cpp\n>> index 548299d05..71d0c2cb4 100644\n>> --- a/src/libcamera/ipc_pipe.cpp\n>> +++ b/src/libcamera/ipc_pipe.cpp\n>> @@ -148,6 +148,11 @@ IPCUnixSocket::Payload IPCMessage::payload() const\n>> * \\brief Returns a const reference to the vector containing file descriptors\n>> */\n>>\n>> +/**\n>> + * \\fn IPCMessage::reader() const\n>> + * \\brief Returns a `SeriReader` instance for parsing the message\n>> + */\n>> +\n>> /**\n>> * \\class IPCPipe\n>> * \\brief IPC message pipe for IPA isolation\n>> diff --git a/test/ipc/unixsocket_ipc.cpp b/test/ipc/unixsocket_ipc.cpp\n>> index df7d9c2b4..c0e174ba9 100644\n>> --- a/test/ipc/unixsocket_ipc.cpp\n>> +++ b/test/ipc/unixsocket_ipc.cpp\n>> @@ -102,7 +102,14 @@ private:\n>> \t\t}\n>>\n>> \t\tcase CmdSetAsync: {\n>> -\t\t\tvalue_ = IPADataSerializer::deserialize(ipcMessage.data());\n>> +\t\t\tSeriReader reader = ipcMessage.reader();\n>> +\t\t\tauto value = IPADataSerializer::deserialize(reader);\n>> +\t\t\tif (!value) {\n>> +\t\t\t\tcerr << \"Failed to deserialize value\" << endl;\n>> +\t\t\t\tstop(-ENODATA);\n>> +\t\t\t} else {\n>> +\t\t\t\tvalue_ = *value;\n>> +\t\t\t}\n>> \t\t\tbreak;\n>> \t\t}\n>> \t\t}\n>> @@ -155,7 +162,14 @@ protected:\n>> \t\t\treturn ret;\n>> \t\t}\n>>\n>> -\t\treturn IPADataSerializer::deserialize(buf.data());\n>> +\t\tSeriReader reader = buf.reader();\n>> +\t\tauto value = IPADataSerializer::deserialize(reader);\n>> +\t\tif (!value) {\n>> +\t\t\tcerr << \"Failed to deserialize value\" << endl;\n>> +\t\t\treturn -ENODATA;\n>> +\t\t}\n>> +\n>> +\t\treturn *value;\n>> \t}\n>>\n>> \tint exit()\n>> diff --git a/test/serialization/generated_serializer/generated_serializer_test.cpp b/test/serialization/generated_serializer/generated_serializer_test.cpp\n>> index dd6968850..0640e741d 100644\n>> --- a/test/serialization/generated_serializer/generated_serializer_test.cpp\n>> +++ b/test/serialization/generated_serializer/generated_serializer_test.cpp\n>> @@ -43,7 +43,7 @@ if (struct1.field != struct2.field) {\t\t\t\t\\\n>> }\n>>\n>>\n>> -\t\tipa::test::TestStruct t, u;\n>> +\t\tipa::test::TestStruct t;\n>>\n>> \t\tt.m = {\n>> \t\t\t{ \"a\", \"z\" },\n>> @@ -71,8 +71,13 @@ if (struct1.field != struct2.field) {\t\t\t\t\\\n>>\n>> \t\tstd::tie(serialized, ignore) =\n>> \t\t\tIPADataSerializer::serialize(t);\n>> +\t\tSeriReader reader(serialized);\n>>\n>> -\t\tu = IPADataSerializer::deserialize(serialized);\n>> +\t\tauto optu = IPADataSerializer::deserialize(reader);\n>> +\t\tif (!optu)\n>> +\t\t\treturn TestFail;\n>> +\n>> +\t\tauto &u = *optu;\n>>\n>> \t\tif (!equals(t.m, u.m))\n>> \t\t\treturn TestFail;\n>> @@ -91,12 +96,16 @@ if (struct1.field != struct2.field) {\t\t\t\t\\\n>>\n>> \t\t/* Test vector of generated structs */\n>> \t\tstd::vector v = { t, u };\n>> -\t\tstd::vector w;\n>>\n>> \t\tstd::tie(serialized, ignore) =\n>> \t\t\tIPADataSerializer>::serialize(v);\n>> +\t\treader = SeriReader(serialized);\n>> +\n>> +\t\tauto optw = IPADataSerializer>::deserialize(reader);\n>> +\t\tif (!optw)\n>> +\t\t\treturn TestFail;\n>>\n>> -\t\tw = IPADataSerializer>::deserialize(serialized);\n>> +\t\tauto &w = *optw;\n>>\n>> \t\tif (!equals(v[0].m, w[0].m) ||\n>> \t\t !equals(v[1].m, w[1].m))\n>> diff --git a/test/serialization/ipa_data_serializer_test.cpp b/test/serialization/ipa_data_serializer_test.cpp\n>> index afea93a6c..3873808a6 100644\n>> --- a/test/serialization/ipa_data_serializer_test.cpp\n>> +++ b/test/serialization/ipa_data_serializer_test.cpp\n>> @@ -52,7 +52,9 @@ int testPodSerdes(T in)\n>> \tstd::vector fds;\n>>\n>> \tstd::tie(buf, fds) = IPADataSerializer::serialize(in);\n>> -\tT out = IPADataSerializer::deserialize(buf, fds);\n>> +\tSeriReader reader(buf, fds);\n>> +\n>> +\tauto out = IPADataSerializer::deserialize(reader);\n>> \tif (in == out)\n>> \t\treturn TestPass;\n>>\n>> @@ -71,7 +73,9 @@ int testVectorSerdes(const std::vector &in,\n>> \tstd::vector fds;\n>>\n>> \tstd::tie(buf, fds) = IPADataSerializer>::serialize(in, cs);\n>> -\tstd::vector out = IPADataSerializer>::deserialize(buf, fds, cs);\n>> +\tSeriReader reader(buf, fds);\n>> +\n>> +\tauto out = IPADataSerializer>::deserialize(reader, cs);\n>> \tif (in == out)\n>> \t\treturn TestPass;\n>>\n>> @@ -91,7 +95,9 @@ int testMapSerdes(const std::map &in,\n>> \tstd::vector fds;\n>>\n>> \tstd::tie(buf, fds) = IPADataSerializer>::serialize(in, cs);\n>> -\tstd::map out = IPADataSerializer>::deserialize(buf, fds, cs);\n>> +\tSeriReader reader(buf, fds);\n>> +\n>> +\tauto out = IPADataSerializer>::deserialize(reader, cs);\n>> \tif (in == out)\n>> \t\treturn TestPass;\n>>\n>> @@ -171,17 +177,27 @@ private:\n>> \t\tstd::tie(listBuf, std::ignore) =\n>> \t\t\tIPADataSerializer::serialize(list, &cs);\n>>\n>> -\t\tconst ControlInfoMap infoMapOut =\n>> -\t\t\tIPADataSerializer::deserialize(infoMapBuf, &cs);\n>> +\t\tSeriReader listReader(listBuf);\n>> +\t\tSeriReader infoMapReader(infoMapBuf);\n>> +\n>> +\t\tauto infoMapOut = IPADataSerializer::deserialize(infoMapReader, &cs);\n>> +\t\tif (!infoMapOut) {\n>> +\t\t\tcerr << \"`ControlInfoMap` cannot be deserialized\" << endl;\n>> +\t\t\treturn TestFail;\n>> +\t\t}\n>>\n>> -\t\tControlList listOut = IPADataSerializer::deserialize(listBuf, &cs);\n>> +\t\tauto listOut = IPADataSerializer::deserialize(listReader, &cs);\n>> +\t\tif (!listOut) {\n>> +\t\t\tcerr << \"`ControlList` cannot be deserialized\" << endl;\n>> +\t\t\treturn TestFail;\n>> +\t\t}\n>>\n>> -\t\tif (!SerializationTest::equals(infoMap, infoMapOut)) {\n>> +\t\tif (!SerializationTest::equals(infoMap, *infoMapOut)) {\n>> \t\t\tcerr << \"Deserialized map doesn't match original\" << endl;\n>> \t\t\treturn TestFail;\n>> \t\t}\n>>\n>> -\t\tif (!SerializationTest::equals(list, listOut)) {\n>> +\t\tif (!SerializationTest::equals(list, *listOut)) {\n>> \t\t\tcerr << \"Deserialized list doesn't match original\" << endl;\n>> \t\t\treturn TestFail;\n>> \t\t}\n>> diff --git a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n>> index 9a3aadbd2..843260b4b 100644\n>> --- a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n>> +++ b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n>> @@ -107,13 +107,13 @@ namespace {{ns}} {\n>> {% if interface_event.methods|length > 0 %}\n>> void {{proxy_name}}::recvMessage(const IPCMessage &data)\n>> {\n>> -\tsize_t dataSize = data.data().size();\n>> +\tSeriReader reader = data.reader();\n>> \t{{cmd_event_enum_name}} _cmd = static_cast<{{cmd_event_enum_name}}>(data.header().cmd);\n>>\n>> \tswitch (_cmd) {\n>> {%- for method in interface_event.methods %}\n>> \tcase {{cmd_event_enum_name}}::{{method.mojom_name|cap}}: {\n>> -\t\t{{method.mojom_name}}IPC(data.data().cbegin(), dataSize, data.fds());\n>> +\t\t{{method.mojom_name}}IPC(reader);\n>> \t\tbreak;\n>> \t}\n>> {%- endfor %}\n>> @@ -211,18 +211,23 @@ void {{proxy_name}}::recvMessage(const IPCMessage &data)\n>> \t\treturn;\n>> {%- endif %}\n>> \t}\n>> +{% if has_output %}\n>> +\tSeriReader _outputReader = _ipcOutputBuf.reader();\n>> +{% endif -%}\n>> {% if method|method_return_value != \"void\" %}\n>> -\t{{method|method_return_value}} _retValue = IPADataSerializer<{{method|method_return_value}}>::deserialize(_ipcOutputBuf.data(), 0);\n>> -\n>> -{{proxy_funcs.deserialize_call(method|method_param_outputs, '_ipcOutputBuf.data()', '_ipcOutputBuf.fds()', init_offset = method|method_return_value|byte_width|int)}}\n>> +\tauto _retValue = IPADataSerializer<{{method|method_return_value}}>::deserialize(_outputReader);\n>> +\tASSERT(_retValue);\n>> +{% endif -%}\n>>\n>> -\treturn _retValue;\n>> +{% if has_output %}\n>> +\t{{proxy_funcs.deserialize_call(method|method_param_outputs, '_outputReader')}}\n>> +\tASSERT(_outputReader.empty());\n>> +{% endif -%}\n>>\n>> -{% elif method|method_param_outputs|length > 0 %}\n>> -{{proxy_funcs.deserialize_call(method|method_param_outputs, '_ipcOutputBuf.data()', '_ipcOutputBuf.fds()')}}\n>> +{% if method|method_return_value != \"void\" %}\n>> +\treturn std::move(*_retValue);\n>> {% endif -%}\n>> }\n>> -\n>> {% endfor %}\n>>\n>> {% for method in interface_event.methods %}\n>> @@ -232,12 +237,9 @@ void {{proxy_name}}::recvMessage(const IPCMessage &data)\n>> \t{{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n>> }\n>>\n>> -void {{proxy_name}}::{{method.mojom_name}}IPC(\n>> -\t[[maybe_unused]] std::vector::const_iterator data,\n>> -\t[[maybe_unused]] size_t dataSize,\n>> -\t[[maybe_unused]] const std::vector &fds)\n>> +void {{proxy_name}}::{{method.mojom_name}}IPC([[maybe_unused]] SeriReader &reader)\n>> {\n>> -{{proxy_funcs.deserialize_call(method.parameters, 'data', 'fds', false, true, true, 'dataSize')}}\n>> +\t{{proxy_funcs.deserialize_call(method.parameters, 'reader', false, true)}}\n>> \t{{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n>> }\n>> {% endfor %}\n>> diff --git a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n>> index a0312a7c1..945a4ded9 100644\n>> --- a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n>> +++ b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n>> @@ -53,10 +53,7 @@ private:\n>> {% endfor %}\n>> {% for method in interface_event.methods %}\n>> {{proxy_funcs.func_sig(proxy_name, method, \"Thread\", false)|indent(8, true)}};\n>> -\tvoid {{method.mojom_name}}IPC(\n>> -\t\tstd::vector::const_iterator data,\n>> -\t\tsize_t dataSize,\n>> -\t\tconst std::vector &fds);\n>> +\tvoid {{method.mojom_name}}IPC(SeriReader &reader);\n>> {% endfor %}\n>>\n>> \t/* Helper class to invoke async functions in another thread. */\n>> diff --git a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n>> index 1f990d3f9..de6378be0 100644\n>> --- a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n>> +++ b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n>> @@ -71,6 +71,7 @@ public:\n>> \t\t}\n>>\n>> \t\tIPCMessage _ipcMessage(_message);\n>> +\t\tSeriReader _reader = _ipcMessage.reader();\n>>\n>> \t\t{{cmd_enum_name}} _cmd = static_cast<{{cmd_enum_name}}>(_ipcMessage.header().cmd);\n>>\n>> @@ -85,7 +86,9 @@ public:\n>> {%- if method.mojom_name == \"configure\" %}\n>> \t\t\tcontrolSerializer_.reset();\n>> {%- endif %}\n>> -\t\t{{proxy_funcs.deserialize_call(method|method_param_inputs, '_ipcMessage.data()', '_ipcMessage.fds()', false, true)|indent(16, true)}}\n>> +\t\t\t{{proxy_funcs.deserialize_call(method|method_param_inputs, '_reader', false, true)|indent(16, true)}}\n>> +\t\t\tASSERT(_reader.empty());\n>> +\n>> {% for param in method|method_param_outputs %}\n>> \t\t\t{{param|name}} {{param.mojom_name}};\n>> {% endfor %}\n>> diff --git a/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl b/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl\n>> index 01e2567ca..b6835ca35 100644\n>> --- a/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl\n>> +++ b/utils/codegen/ipc/generators/libcamera_templates/proxy_functions.tmpl\n>> @@ -64,15 +64,6 @@\n>> );\n>> {%- endfor %}\n>>\n>> -{%- if params|length > 1 %}\n>> -{%- for param in params %}\n>> -\tappendPOD({{buf}}, {{param.mojom_name}}Buf.size());\n>> -{%- if param|has_fd %}\n>> -\tappendPOD({{buf}}, {{param.mojom_name}}Fds.size());\n>> -{%- endif %}\n>> -{%- endfor %}\n>> -{%- endif %}\n>> -\n>> {%- for param in params %}\n>> \t{{buf}}.insert({{buf}}.end(), {{param.mojom_name}}Buf.begin(), {{param.mojom_name}}Buf.end());\n>> {%- endfor %}\n>> @@ -84,104 +75,28 @@\n>> {%- endfor %}\n>> {%- endmacro -%}\n>>\n>> -\n>> -{#\n>> - # \\brief Deserialize a single object from data buffer and fd vector\n>> - #\n>> - # \\param pointer If true, deserialize the object into a dereferenced pointer\n>> - # \\param iter If true, treat \\a buf as an iterator instead of a vector\n>> - # \\param data_size Variable that holds the size of the vector referenced by \\a buf\n>> - #\n>> - # Generate code to deserialize a single object, as specified in \\a param,\n>> - # from \\a buf data buffer and \\a fds fd vector.\n>> - # This code is meant to be used by macro deserialize_call.\n>> - #}\n>> -{%- macro deserialize_param(param, pointer, loop, buf, fds, iter, data_size) -%}\n>> -{{\"*\" if pointer}}{{param.mojom_name}} =\n>> -IPADataSerializer<{{param|name_full}}>::deserialize(\n>> -\t{{buf}}{{- \".cbegin()\" if not iter}} + {{param.mojom_name}}Start,\n>> -{%- if loop.last and not iter %}\n>> -\t{{buf}}.cend()\n>> -{%- elif not iter %}\n>> -\t{{buf}}.cbegin() + {{param.mojom_name}}Start + {{param.mojom_name}}BufSize\n>> -{%- elif iter and loop.length == 1 %}\n>> -\t{{buf}} + {{data_size}}\n>> -{%- else %}\n>> -\t{{buf}} + {{param.mojom_name}}Start + {{param.mojom_name}}BufSize\n>> -{%- endif -%}\n>> -{{- \",\" if param|has_fd}}\n>> -{%- if param|has_fd %}\n>> -\t{{fds}}.cbegin() + {{param.mojom_name}}FdStart,\n>> -{%- if loop.last %}\n>> -\t{{fds}}.cend()\n>> -{%- else %}\n>> -\t{{fds}}.cbegin() + {{param.mojom_name}}FdStart + {{param.mojom_name}}FdsSize\n>> -{%- endif -%}\n>> -{%- endif -%}\n>> -{{- \",\" if param|needs_control_serializer}}\n>> -{%- if param|needs_control_serializer %}\n>> -\t&controlSerializer_\n>> -{%- endif -%}\n>> -);\n>> -{%- endmacro -%}\n>> -\n>> -\n>> {#\n>> - # \\brief Deserialize multiple objects from data buffer and fd vector\n>> + # \\brief Deserialize multiple objects\n>> #\n>> # \\param pointer If true, deserialize objects into pointers, and adds a null check.\n>> # \\param declare If true, declare the objects in addition to deserialization.\n>> - # \\param iter if true, treat \\a buf as an iterator instead of a vector\n>> - # \\param data_size Variable that holds the size of the vector referenced by \\a buf\n>> - #\n>> - # Generate code to deserialize multiple objects, as specified in \\a params\n>> - # (which are the parameters to some function), from \\a buf data buffer and\n>> - # \\a fds fd vector.\n>> - # This code is meant to be used by the proxy, for deserializing after IPC calls.\n>> #\n>> - # \\todo Avoid intermediate vectors\n>> + # Generate code to deserialize multiple objects, as specified in \\a params,\n>> + # from \\a reader. This code is meant to be used by the proxy, for deserializing\n>> + # after IPC calls.\n>> #}\n>> -{%- macro deserialize_call(params, buf, fds, pointer = true, declare = false, iter = false, data_size = '', init_offset = 0) -%}\n>> -{% set ns = namespace(size_offset = init_offset) %}\n>> -{%- if params|length > 1 %}\n>> +{%- macro deserialize_call(params, reader, pointer = true, declare = false) -%}\n>> {%- for param in params %}\n>> -\t[[maybe_unused]] const size_t {{param.mojom_name}}BufSize = readPOD({{buf}}, {{ns.size_offset}}\n>> -{%- if iter -%}\n>> -, {{buf}} + {{data_size}}\n>> -{%- endif -%}\n>> -);\n>> -\t{%- set ns.size_offset = ns.size_offset + 4 %}\n>> -{%- if param|has_fd %}\n>> -\t[[maybe_unused]] const size_t {{param.mojom_name}}FdsSize = readPOD({{buf}}, {{ns.size_offset}}\n>> -{%- if iter -%}\n>> -, {{buf}} + {{data_size}}\n>> -{%- endif -%}\n>> -);\n>> -\t{%- set ns.size_offset = ns.size_offset + 4 %}\n>> -{%- endif %}\n>> -{%- endfor %}\n>> -{%- endif %}\n>> -{% for param in params %}\n>> -{%- if loop.first %}\n>> -\tconst size_t {{param.mojom_name}}Start = {{ns.size_offset}};\n>> -{%- else %}\n>> -\tconst size_t {{param.mojom_name}}Start = {{loop.previtem.mojom_name}}Start + {{loop.previtem.mojom_name}}BufSize;\n>> -{%- endif %}\n>> -{%- endfor %}\n>> -{% for param in params|with_fds %}\n>> -{%- if loop.first %}\n>> -\tconst size_t {{param.mojom_name}}FdStart = 0;\n>> +\tauto param_{{param.mojom_name}} = IPADataSerializer<{{param|name_full}}>::deserialize({{reader}}, &controlSerializer_);\n>> +\tASSERT(param_{{param.mojom_name}});\n>> +\n>> +{%- if pointer %}\n>> +\tif ({{param.mojom_name}})\n>> +\t\t*{{param.mojom_name}} = std::move(*param_{{param.mojom_name}});\n>> +{%- elif declare %}\n>> +\t{{param|name}} &{{param.mojom_name}} = *param_{{param.mojom_name}};\n>> {%- else %}\n>> -\tconst size_t {{param.mojom_name}}FdStart = {{loop.previtem.mojom_name}}FdStart + {{loop.previtem.mojom_name}}FdsSize;\n>> +\t{{param.mojom_name}} = std::move(*param_{{param.mojom_name}});\n>> {%- endif %}\n>> -{%- endfor %}\n>> -{% for param in params %}\n>> -\t{%- if pointer %}\n>> -\tif ({{param.mojom_name}}) {\n>> -{{deserialize_param(param, pointer, loop, buf, fds, iter, data_size)|indent(16, True)}}\n>> -\t}\n>> -\t{%- else %}\n>> -\t{{param|name + \" \" if declare}}{{deserialize_param(param, pointer, loop, buf, fds, iter, data_size)|indent(8)}}\n>> -\t{%- endif %}\n>> {% endfor %}\n>> {%- endmacro -%}\n>> diff --git a/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl b/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl\n>> index e316dd88a..9e9dd0ca6 100644\n>> --- a/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl\n>> +++ b/utils/codegen/ipc/generators/libcamera_templates/serializer.tmpl\n>> @@ -2,22 +2,6 @@\n>> # SPDX-License-Identifier: LGPL-2.1-or-later\n>> # Copyright (C) 2020, Google Inc.\n>> -#}\n>> -{#\n>> - # \\brief Verify that there is enough bytes to deserialize\n>> - #\n>> - # Generate code that verifies that \\a size is not greater than \\a dataSize.\n>> - # Otherwise log an error with \\a name and \\a typename.\n>> - #}\n>> -{%- macro check_data_size(size, dataSize, name, typename) %}\n>> -\t\tif ({{dataSize}} < {{size}}) {\n>> -\t\t\tLOG(IPADataSerializer, Error)\n>> -\t\t\t\t<< \"Failed to deserialize \" << \"{{name}}\"\n>> -\t\t\t\t<< \": not enough {{typename}}, expected \"\n>> -\t\t\t\t<< ({{size}}) << \", got \" << ({{dataSize}});\n>> -\t\t\treturn ret;\n>> -\t\t}\n>> -{%- endmacro %}\n>> -\n>>\n>> {#\n>> # \\brief Serialize a field into return vector\n>> @@ -42,15 +26,10 @@\n>> \t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n>> \t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n>> {%- elif field|is_controls %}\n>> -\t\tif (data.{{field.mojom_name}}.size() > 0) {\n>> -\t\t\tstd::vector {{field.mojom_name}};\n>> -\t\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n>> -\t\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n>> -\t\t\tappendPOD(retData, {{field.mojom_name}}.size());\n>> -\t\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n>> -\t\t} else {\n>> -\t\t\tappendPOD(retData, 0);\n>> -\t\t}\n>> +\t\tstd::vector {{field.mojom_name}};\n>> +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n>> +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n>> +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n>> {%- elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n>> \t\tstd::vector {{field.mojom_name}};\n>> \t{%- if field|has_fd %}\n>> @@ -65,10 +44,6 @@\n>> \t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n>> \t{%- else %}\n>> \t\t\tIPADataSerializer<{{field|name_full}}>::serialize(data.{{field.mojom_name}}, cs);\n>> -\t{%- endif %}\n>> -\t\tappendPOD(retData, {{field.mojom_name}}.size());\n>> -\t{%- if field|has_fd %}\n>> -\t\tappendPOD(retData, {{field.mojom_name}}Fds.size());\n>> \t{%- endif %}\n>> \t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n>> \t{%- if field|has_fd %}\n>> @@ -79,89 +54,6 @@\n>> {%- endif %}\n>> {%- endmacro %}\n>>\n>> -\n>> -{#\n>> - # \\brief Deserialize a field into return struct\n>> - #\n>> - # Generate code to deserialize \\a field into object ret.\n>> - # This code is meant to be used by the IPADataSerializer specialization.\n>> - #}\n>> -{%- macro deserializer_field(field, loop) %}\n>> -{% if field|is_pod or field|is_enum %}\n>> -\t{%- set field_size = (field|bit_width|int / 8)|int %}\n>> -\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n>> -\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name_full}}>::deserialize(m, m + {{field_size}});\n>> -\t{%- if not loop.last %}\n>> -\t\tm += {{field_size}};\n>> -\t\tdataSize -= {{field_size}};\n>> -\t{%- endif %}\n>> -{% elif field|is_fd %}\n>> -\t{%- set field_size = 4 %}\n>> -\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n>> -\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field_size}}, n, n + 1, cs);\n>> -\t{%- if not loop.last %}\n>> -\t\tm += {{field_size}};\n>> -\t\tdataSize -= {{field_size}};\n>> -\t\tn += ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n>> -\t\tfdsSize -= ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n>> -\t{%- endif %}\n>> -{% elif field|is_controls %}\n>> -\t{%- set field_size = 4 %}\n>> -\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n>> -\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n>> -\t\tm += {{field_size}};\n>> -\t\tdataSize -= {{field_size}};\n>> -\t{%- set field_size = field.mojom_name + 'Size' -%}\n>> -\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n>> -\t\tif ({{field.mojom_name}}Size > 0)\n>> -\t\t\tret.{{field.mojom_name}} =\n>> -\t\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n>> -\t{%- if not loop.last %}\n>> -\t\tm += {{field_size}};\n>> -\t\tdataSize -= {{field_size}};\n>> -\t{%- endif %}\n>> -{% elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n>> -\t{%- set field_size = 4 %}\n>> -\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n>> -\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n>> -\t\tm += {{field_size}};\n>> -\t\tdataSize -= {{field_size}};\n>> -\t{%- if field|has_fd %}\n>> -\t{%- set field_size = 4 %}\n>> -\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'FdsSize', 'data')}}\n>> -\t\tconst size_t {{field.mojom_name}}FdsSize = readPOD(m, 0, dataEnd);\n>> -\t\tm += {{field_size}};\n>> -\t\tdataSize -= {{field_size}};\n>> -\t\t{{- check_data_size(field.mojom_name + 'FdsSize', 'fdsSize', field.mojom_name, 'fds')}}\n>> -\t{%- endif %}\n>> -\t{%- set field_size = field.mojom_name + 'Size' -%}\n>> -\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n>> -\t\tret.{{field.mojom_name}} =\n>> -\t{%- if field|is_str %}\n>> -\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size);\n>> -\t{%- elif field|has_fd and (field|is_array or field|is_map) %}\n>> -\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n>> -\t{%- elif field|has_fd and (not (field|is_array or field|is_map)) %}\n>> -\t\t\tIPADataSerializer<{{field|name_full}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n>> -\t{%- elif (not field|has_fd) and (field|is_array or field|is_map) %}\n>> -\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n>> -\t{%- else %}\n>> -\t\t\tIPADataSerializer<{{field|name_full}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n>> -\t{%- endif %}\n>> -\t{%- if not loop.last %}\n>> -\t\tm += {{field_size}};\n>> -\t\tdataSize -= {{field_size}};\n>> -\t{%- if field|has_fd %}\n>> -\t\tn += {{field.mojom_name}}FdsSize;\n>> -\t\tfdsSize -= {{field.mojom_name}}FdsSize;\n>> -\t{%- endif %}\n>> -\t{%- endif %}\n>> -{% else %}\n>> -\t\t/* Unknown deserialization for {{field.mojom_name}}. */\n>> -{%- endif %}\n>> -{%- endmacro %}\n>> -\n>> -\n>> {#\n>> # \\brief Serialize a struct\n>> #\n>> @@ -194,126 +86,30 @@\n>>\n>>\n>> {#\n>> - # \\brief Deserialize a struct that has fds\n>> + # \\brief Deserialize a struct\n>> #\n>> - # Generate code for IPADataSerializer specialization, for deserializing\n>> - # \\a struct, in the case that \\a struct has file descriptors.\n>> + # Generate code for IPADataSerializer specialization, for deserializing \\a struct.\n>> #}\n>> -{%- macro deserializer_fd(struct) %}\n>> -\tstatic {{struct|name_full}}\n>> -\tdeserialize(std::vector &data,\n>> -\t\t std::vector &fds,\n>> -{%- if struct|needs_control_serializer %}\n>> -\t\t ControlSerializer *cs)\n>> -{%- else %}\n>> -\t\t ControlSerializer *cs = nullptr)\n>> -{%- endif %}\n>> -\t{\n>> -\t\treturn IPADataSerializer<{{struct|name_full}}>::deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n>> -\t}\n>> -\n>> +{%- macro deserializer(struct) %}\n>> {# \\todo Don't inline this function #}\n>> -\tstatic {{struct|name_full}}\n>> -\tdeserialize(std::vector::const_iterator dataBegin,\n>> -\t\t std::vector::const_iterator dataEnd,\n>> -\t\t std::vector::const_iterator fdsBegin,\n>> -\t\t std::vector::const_iterator fdsEnd,\n>> +\t[[nodiscard]] static std::optional<{{struct|name_full}}>\n>> +\tdeserialize(SeriReader &reader,\n>> {%- if struct|needs_control_serializer %}\n>> \t\t ControlSerializer *cs)\n>> {%- else %}\n>> \t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> {%- endif %}\n>> \t{\n>> -\t\t{{struct|name_full}} ret;\n>> -\t\tstd::vector::const_iterator m = dataBegin;\n>> -\t\tstd::vector::const_iterator n = fdsBegin;\n>> -\n>> -\t\tsize_t dataSize = std::distance(dataBegin, dataEnd);\n>> -\t\t[[maybe_unused]] size_t fdsSize = std::distance(fdsBegin, fdsEnd);\n>> -{%- for field in struct.fields -%}\n>> -{{deserializer_field(field, loop)}}\n>> +{%- for field in struct.fields %}\n>> +\t\tauto {{field.mojom_name}} = IPADataSerializer<{{field|name_full}}>::deserialize(reader, cs);\n>> +\t\tif (!{{field.mojom_name}})\n>> +\t\t\treturn {};\n>> {%- endfor %}\n>> -\t\treturn ret;\n>> -\t}\n>> -{%- endmacro %}\n>> -\n>> -{#\n>> - # \\brief Deserialize a struct that has fds, using non-fd\n>> - #\n>> - # Generate code for IPADataSerializer specialization, for deserializing\n>> - # \\a struct, in the case that \\a struct has no file descriptors but requires\n>> - # deserializers with file descriptors.\n>> - #}\n>> -{%- macro deserializer_fd_simple(struct) %}\n>> -\tstatic {{struct|name_full}}\n>> -\tdeserialize(std::vector &data,\n>> -\t\t [[maybe_unused]] std::vector &fds,\n>> -\t\t ControlSerializer *cs = nullptr)\n>> -\t{\n>> -\t\treturn IPADataSerializer<{{struct|name_full}}>::deserialize(data.cbegin(), data.cend(), cs);\n>> -\t}\n>> -\n>> -\tstatic {{struct|name_full}}\n>> -\tdeserialize(std::vector::const_iterator dataBegin,\n>> -\t\t std::vector::const_iterator dataEnd,\n>> -\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n>> -\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n>> -\t\t ControlSerializer *cs = nullptr)\n>> -\t{\n>> -\t\treturn IPADataSerializer<{{struct|name_full}}>::deserialize(dataBegin, dataEnd, cs);\n>> -\t}\n>> -{%- endmacro %}\n>> -\n>> -\n>> -{#\n>> - # \\brief Deserialize a struct that has no fds\n>> - #\n>> - # Generate code for IPADataSerializer specialization, for deserializing\n>> - # \\a struct, in the case that \\a struct does not have file descriptors.\n>> - #}\n>> -{%- macro deserializer_no_fd(struct) %}\n>> -\tstatic {{struct|name_full}}\n>> -\tdeserialize(std::vector &data,\n>> -{%- if struct|needs_control_serializer %}\n>> -\t\t ControlSerializer *cs)\n>> -{%- else %}\n>> -\t\t ControlSerializer *cs = nullptr)\n>> -{%- endif %}\n>> -\t{\n>> -\t\treturn IPADataSerializer<{{struct|name_full}}>::deserialize(data.cbegin(), data.cend(), cs);\n>> -\t}\n>>\n>> -{# \\todo Don't inline this function #}\n>> -\tstatic {{struct|name_full}}\n>> -\tdeserialize(std::vector::const_iterator dataBegin,\n>> -\t\t std::vector::const_iterator dataEnd,\n>> -{%- if struct|needs_control_serializer %}\n>> -\t\t ControlSerializer *cs)\n>> -{%- else %}\n>> -\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n>> -{%- endif %}\n>> -\t{\n>> -\t\t{{struct|name_full}} ret;\n>> -\t\tstd::vector::const_iterator m = dataBegin;\n>> -\n>> -\t\tsize_t dataSize = std::distance(dataBegin, dataEnd);\n>> -{%- for field in struct.fields -%}\n>> -{{deserializer_field(field, loop)}}\n>> +\t\treturn { {\n>> +{%- for field in struct.fields %}\n>> +\t\t\tstd::move(*{{field.mojom_name}}),\n>> {%- endfor %}\n>> -\t\treturn ret;\n>> +\t\t} };\n>> \t}\n>> {%- endmacro %}\n>> -\n>> -{#\n>> - # \\brief Deserialize a struct\n>> - #\n>> - # Generate code for IPADataSerializer specialization, for deserializing \\a struct.\n>> - #}\n>> -{%- macro deserializer(struct) %}\n>> -{%- if struct|has_fd %}\n>> -{{deserializer_fd(struct)}}\n>> -{%- else %}\n>> -{{deserializer_no_fd(struct)}}\n>> -{{deserializer_fd_simple(struct)}}\n>> -{%- endif %}\n>> -{%- endmacro %}\n>> --\n>> 2.49.0\n>>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8C3F3BD78E\n\tfor ;\n\tWed, 21 May 2025 07:44:41 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id AB6C368D8A;\n\tWed, 21 May 2025 09:44:39 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 74DE768C90\n\tfor ;\n\tWed, 21 May 2025 09:44:37 +0200 (CEST)","from [192.168.33.23] (185.221.141.226.nat.pool.zt.hu\n\t[185.221.141.226])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id CB4EB446;\n\tWed, 21 May 2025 09:44:15 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"nXklBNAR\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1747813456;\n\tbh=h1j5onZZ0DjWXMHMlebarE/gv1iBoZK7jl95mVI4aDM=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=nXklBNARyUIjpieLHrdFyagp9oL+UBkCL638GYtQy8zMkiliP0l5x8cmjrJa5ei9z\n\tvqT+O7e5f+kV4md4EWSw1CpLUwE887nP5EgbCRsWz5Oeivspt2JlLCFarSw1WBPKK5\n\tAHDC8bWaln9Dz9BUvPfb0f5YQ/nKj6gKWHSM/jgo=","Message-ID":"","Date":"Wed, 21 May 2025 09:44:33 +0200","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [RFC PATCH v1 8/8] utils: codegen: ipc: Simplify deserialization","To":"Jacopo Mondi ","Cc":"libcamera-devel@lists.libcamera.org,\n\tPaul Elder ","References":"<20250515120012.3127231-1-barnabas.pocze@ideasonboard.com>\n\t<20250515120012.3127231-9-barnabas.pocze@ideasonboard.com>\n\t","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Content-Language":"en-US, hu-HU","In-Reply-To":"","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":34309,"web_url":"https://patchwork.libcamera.org/comment/34309/","msgid":"<174784715838.14042.17418405103980303587@calcite>","date":"2025-05-21T17:05:58","subject":"Re: [RFC PATCH v1 8/8] utils: codegen: ipc: Simplify deserialization","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"content":"\n\n> >> - static std::map deserialize(std::vector::const_iterator dataBegin,\n> >> - std::vector::const_iterator dataEnd,\n> >> - std::vector::const_iterator fdsBegin,\n> >> - [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> >> - ControlSerializer *cs = nullptr)\n> >> - {\n> >> std::map ret;\n> >> \n> >> - uint32_t mapLen = readPOD(dataBegin, 0, dataEnd);\n> >> -\n> >> - std::vector::const_iterator dataIter = dataBegin + 4;\n> >> - std::vector::const_iterator fdIter = fdsBegin;\n> >> for (uint32_t i = 0; i < mapLen; i++) {\n> >> - uint32_t sizeofData = readPOD(dataIter, 0, dataEnd);\n> >> - uint32_t sizeofFds = readPOD(dataIter, 4, dataEnd);\n> >> - dataIter += 8;\n> >> -\n> >> - K key = IPADataSerializer::deserialize(dataIter,\n> >> - dataIter + sizeofData,\n> >> - fdIter,\n> >> - fdIter + sizeofFds,\n> >> - cs);\n> >> -\n> >> - dataIter += sizeofData;\n> >> - fdIter += sizeofFds;\n> >> - sizeofData = readPOD(dataIter, 0, dataEnd);\n> >> - sizeofFds = readPOD(dataIter, 4, dataEnd);\n> >> - dataIter += 8;\n> >> -\n> >> - const V value = IPADataSerializer::deserialize(dataIter,\n> >> - dataIter + sizeofData,\n> >> - fdIter,\n> >> - fdIter + sizeofFds,\n> >> - cs);\n> >> - ret.insert({ key, value });\n> >> -\n> >> - dataIter += sizeofData;\n> >> - fdIter += sizeofFds;\n> >> + auto key = IPADataSerializer::deserialize(reader, cs);\n> >> + if (!key)\n> >> + return {};\n> >> +\n> >> + auto value = IPADataSerializer::deserialize(reader, cs);\n> >> + if (!value)\n> >> + return {};\n> >> +\n> >> + ret.try_emplace(std::move(*key), std::move(*value));\n> > \n> > I'm curious, what's the reason for using try_emplace() here?\n> \n> Alternatives:\n> * operator[]\n> * requires default constructibility\n> * must default construct and then (move) assign\n> * emplace()\n> * may need to allocate and construct a node (even if the key exists)\n> * insert_or_assign()\n> * could be used if the latest value is desired\n> * must generate conditional code for (move) assignment\n> \n> So my rule of thumb is to basically always use `try_emplace()` or `insert_or_assign()`,\n> but I think I will modify the map deserialization code to reject duplicate keys,\n> so that leaves `try_emplace()` as the best option in my opinion.\n\nAh, I see.\n\n\n\n> > \n> >> +\n> >> + template\n> >> + [[nodiscard]] auto read()\n> >> + {\n> >> + std::tuple xs;\n> >> + bool ok = std::apply([&](auto &...vs) {\n> >> + return read(vs...);\n> >> + }, xs);\n> >> +\n> >> + if constexpr (sizeof...(Ts) == 1)\n> >> + return ok ? std::optional(std::get<0>(xs)) : std::nullopt;\n> >> + else\n> >> + return ok ? std::optional(xs) : std::nullopt;\n> > \n> > This means that each std::nullopt is of different types no? afaict you only use\n> > this function when sizeof...(Ts) == 1.\n> \n> Well, `std::nullopt` is always `std::nullopt`, but you're right that the conditional\n> operator will resolve to different types. I thought it is best for usability to have\n> `read()` return `std::optional` and `read()` return `std::optional>`.\n> But yes, this is overload not really used, so it could be be removed.\n\nAh not the nullopt but the std::optional type will be different, ok.\n\n\n\n> >> diff --git a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> >> index 9a3aadbd2..843260b4b 100644\n> >> --- a/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> >> +++ b/utils/codegen/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> >> @@ -107,13 +107,13 @@ namespace {{ns}} {\n> >> {% if interface_event.methods|length > 0 %}\n> >> void {{proxy_name}}::recvMessage(const IPCMessage &data)\n> >> {\n> >> - size_t dataSize = data.data().size();\n> >> + SeriReader reader = data.reader();\n> >> {{cmd_event_enum_name}} _cmd = static_cast<{{cmd_event_enum_name}}>(data.header().cmd);\n> >> \n> >> switch (_cmd) {\n> >> {%- for method in interface_event.methods %}\n> >> case {{cmd_event_enum_name}}::{{method.mojom_name|cap}}: {\n> >> - {{method.mojom_name}}IPC(data.data().cbegin(), dataSize, data.fds());\n> >> + {{method.mojom_name}}IPC(reader);\n> >> break;\n> >> }\n> >> {%- endfor %}\n> >> @@ -211,18 +211,23 @@ void {{proxy_name}}::recvMessage(const IPCMessage &data)\n> >> return;\n> >> {%- endif %}\n> >> }\n> >> +{% if has_output %}\n> >> + SeriReader _outputReader = _ipcOutputBuf.reader();\n> >> +{% endif -%}\n> >> {% if method|method_return_value != \"void\" %}\n> >> - {{method|method_return_value}} _retValue = IPADataSerializer<{{method|method_return_value}}>::deserialize(_ipcOutputBuf.data(), 0);\n> >> -\n> >> -{{proxy_funcs.deserialize_call(method|method_param_outputs, '_ipcOutputBuf.data()', '_ipcOutputBuf.fds()', init_offset = method|method_return_value|byte_width|int)}}\n> >> + auto _retValue = IPADataSerializer<{{method|method_return_value}}>::deserialize(_outputReader);\n> >> + ASSERT(_retValue);\n> >> +{% endif -%}\n> >> \n> >> - return _retValue;\n> >> +{% if has_output %}\n> > \n> > I think this should be {% if method|method_param_outputs|length > 0 %} because\n> > has_output also covers `method|method_return_value != \"void\"`, so you'd be\n> > deserializing the single _retValue twice.\n> \n> The generated code looks ok to me, and as far as I can see the return value is not\n> part of `method_param_outputs`, so the `proxy_funcs.deserialize_call()` should expand\n> to nothing because it receives an empty list.\n\nIndeed, you are right.\n\n> \n> \n> > \n> > Or maybe an ASSERT(_outputReader.empty()) might be valuable after\n> > ASSERT(_retValue) above.\n> > \n> >> + {{proxy_funcs.deserialize_call(method|method_param_outputs, '_outputReader')}}\n> >> + ASSERT(_outputReader.empty());\n> >> +{% endif -%}\n> >> \n> >> -{% elif method|method_param_outputs|length > 0 %}\n> >> -{{proxy_funcs.deserialize_call(method|method_param_outputs, '_ipcOutputBuf.data()', '_ipcOutputBuf.fds()')}}\n> >> +{% if method|method_return_value != \"void\" %}\n> >> + return std::move(*_retValue);\n> >> {% endif -%}\n> >> }\n> >> -\n> >> {% endfor %}\n> >> \n> >> {% for method in interface_event.methods %}\n> >> @@ -232,12 +237,9 @@ void {{proxy_name}}::recvMessage(const IPCMessage &data)\n> >> {{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n> >> }\n> >> \n> >> -void {{proxy_name}}::{{method.mojom_name}}IPC(\n> >> - [[maybe_unused]] std::vector::const_iterator data,\n> >> - [[maybe_unused]] size_t dataSize,\n> >> - [[maybe_unused]] const std::vector &fds)\n> >> +void {{proxy_name}}::{{method.mojom_name}}IPC([[maybe_unused]] SeriReader &reader)\n> >> {\n> >> -{{proxy_funcs.deserialize_call(method.parameters, 'data', 'fds', false, true, true, 'dataSize')}}\n> >> + {{proxy_funcs.deserialize_call(method.parameters, 'reader', false, true)}}\n> > \n> > I think this is one level too much indentation? Same for the other places you\n> > add this call.\n> \n> The generated code looks good to me. I think the indentiation of this macro call\n> does not matter because it has `{%-`, so the preceeding whitespaces will be stripped.\n\nOk.\n\n\n\nI just saw Jacopo's review and was reminded that the serialization format\ndocumentation needs to updated. afaict that's the only significant thing left.\n\n\nThanks,\n\nPaul","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id DA61BBD78E\n\tfor ;\n\tWed, 21 May 2025 17:06:04 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 4C71668D92;\n\tWed, 21 May 2025 19:06:03 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3CF4368C91\n\tfor ;\n\tWed, 21 May 2025 19:06:01 +0200 (CEST)","from pyrite.rasen.tech (unknown [149.232.183.6])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 9EA706AF;\n\tWed, 21 May 2025 19:05:39 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Nj25V8qn\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1747847139;\n\tbh=4c2F/vkuTMatgkX9g4lY4J36btljp8hKirBtMcq0zdA=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=Nj25V8qnyu9+2OmYl6bYYjLQ/ftJsN82Th2HJZxsANJcrRQmYygI67mfjwv5xT69h\n\tS2Y88YPGCgS7KavNTGRkIXc5rRrp7zx/KGNx/QTd4/jKmDFK+pe7lPap0DNAwtAN0j\n\tw7zZMrjOBQUgpRSFw6yo0VoIGa0FjkdELM6cCveo=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<9b8e5e89-fcf6-4661-a428-3d22c1fa0a37@ideasonboard.com>","References":"<20250515120012.3127231-1-barnabas.pocze@ideasonboard.com>\n\t<20250515120012.3127231-9-barnabas.pocze@ideasonboard.com>\n\t<174741664081.476729.1844022348649238166@calcite>\n\t<9b8e5e89-fcf6-4661-a428-3d22c1fa0a37@ideasonboard.com>","Subject":"Re: [RFC PATCH v1 8/8] utils: codegen: ipc: Simplify deserialization","From":"Paul Elder ","Cc":"libcamera-devel@lists.libcamera.org","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Date":"Wed, 21 May 2025 19:05:58 +0200","Message-ID":"<174784715838.14042.17418405103980303587@calcite>","User-Agent":"alot/0.10","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]