{"id":10720,"url":"https://patchwork.libcamera.org/api/patches/10720/?format=json","web_url":"https://patchwork.libcamera.org/patch/10720/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20201224081534.41601-7-paul.elder@ideasonboard.com>","date":"2020-12-24T08:15:31","name":"[libcamera-devel,v6,6/9] libcamera: Add IPADataSerializer","commit_ref":null,"pull_url":null,"state":"changes-requested","archived":false,"hash":"e0c7beb0a90119ef6a278eb4acab10b400d48490","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/?format=json","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"delegate":{"id":17,"url":"https://patchwork.libcamera.org/api/users/17/?format=json","username":"epaul","first_name":"Paul","last_name":"Elder","email":"paul.elder@ideasonboard.com"},"mbox":"https://patchwork.libcamera.org/patch/10720/mbox/","series":[{"id":1546,"url":"https://patchwork.libcamera.org/api/series/1546/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=1546","date":"2020-12-24T08:15:25","name":"IPA isolation: Part 1: Core components","version":6,"mbox":"https://patchwork.libcamera.org/series/1546/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/10720/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/10720/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 0EC94C0F1A\n\tfor ;\n\tThu, 24 Dec 2020 08:16:03 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D1764615AF;\n\tThu, 24 Dec 2020 09:16:02 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 6C01760525\n\tfor ;\n\tThu, 24 Dec 2020 09:16:01 +0100 (CET)","from pyrite.rasen.tech (unknown\n\t[IPv6:2400:4051:61:600:2c71:1b79:d06d:5032])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 8E2D4A1D;\n\tThu, 24 Dec 2020 09:15:59 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"aiqS9MU+\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1608797761;\n\tbh=1vWrzxN36u0lfl+udZ6GzdTqw48MfTnfF2BPn6KIFwQ=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=aiqS9MU+a03BrRzFPEAMigaX0vpk3OeNSlv/GbCQhsPJRDSkj7FkBwT77hZwCX3DG\n\tKcafM9WCx8uJUK9OsUFiUnpsp5raM7GTHP4xZHstajlIrUE1Wiqo1Rh9IHAHVX2Crz\n\t+NvrL2RySaLye5pV9IhjEw9hc6qT75dFbAtvjTVM=","From":"Paul Elder ","To":"libcamera-devel@lists.libcamera.org","Date":"Thu, 24 Dec 2020 17:15:31 +0900","Message-Id":"<20201224081534.41601-7-paul.elder@ideasonboard.com>","X-Mailer":"git-send-email 2.27.0","In-Reply-To":"<20201224081534.41601-1-paul.elder@ideasonboard.com>","References":"<20201224081534.41601-1-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Subject":"[libcamera-devel] [PATCH v6 6/9] libcamera: Add IPADataSerializer","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Add an IPADataSerializer which implments de/serialization of built-in\n(PODs, vector, map, string) and libcamera data structures. This is\nintended to be used by the proxy and the proxy worker in the IPC layer.\n\nSigned-off-by: Paul Elder \nReviewed-by: Jacopo Mondi \n\n---\nChanges in v6:\n- remove (de)serializers for IPASettings, CameraSensorInfo, IPAStream,\n and IPABuffer as they are now defined in core.mojom and generated\n- ControlList (de)serializer no longer needs a ControlInfoMap\n- ControlList (de)serializer now checks the ControlSerializer if the\n ControlList's ControlInfoMap is cached\n- add some todos for optimization and hardening\n\nChanges in v5:\n- fix style of: {{}, {}} -> { {}, {} }\n- add documentation on serialization formats (not in doxygen, though)\n- compress readPOD\n - use memcpy\n - use ASSERT\n- remove ifdef DOXYGEN guard from base IPADataSerializer\n- remove integer overflow risk when calculating vector offsets\n- use const iterators and const references\n- remove const ControlList and const ControlInfoMap specializations\n\nChanges in v4:\n- rename readUInt/appendUInt to readPOD/appendPOD\n - put them in anonymous namespace\n - and add length check\n - fatal if failure, because it means not enough data to deserialize,\n which is technically a segfault\n - use the new readPOD/appendPOD with length protections\n - expand on their docs correspondingly\n- change snake_case to camelCase\n- add optimizations to the hand-written de/serializers\n- reserve vector length where trivially possible\n- remove unnecessary IPADataSerializer:: explicit calls (if\n they're calling a specialization from the same specialization)\n\nChanges in v3:\n- reimplement append/readUInt with memcpy (intead of bit shifting)\n- change DECLARE_INTEGRAL_SERIALIZER with DECLARE_POD_SERIALIZER\n - use this for int64_t, bool, float, and double\n- fix comment style\n\nChanges in v2:\n- added serializers for all integer types, bool, and string\n- use string serializer for IPASettings serializer\n- add documentation\n- add serializer for const ControlList\n---\n .../libcamera/internal/ipa_data_serializer.h | 701 ++++++++++++++++++\n src/libcamera/ipa_data_serializer.cpp | 185 +++++\n src/libcamera/meson.build | 1 +\n 3 files changed, 887 insertions(+)\n create mode 100644 include/libcamera/internal/ipa_data_serializer.h\n create mode 100644 src/libcamera/ipa_data_serializer.cpp","diff":"diff --git a/include/libcamera/internal/ipa_data_serializer.h b/include/libcamera/internal/ipa_data_serializer.h\nnew file mode 100644\nindex 00000000..5468a0b7\n--- /dev/null\n+++ b/include/libcamera/internal/ipa_data_serializer.h\n@@ -0,0 +1,701 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2020, Google Inc.\n+ *\n+ * ipa_data_serializer.h - Image Processing Algorithm data serializer\n+ */\n+#ifndef __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_H__\n+#define __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_H__\n+\n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+\n+#include \n+#include \n+#include \n+#include \n+\n+#include \"libcamera/internal/byte_stream_buffer.h\"\n+#include \"libcamera/internal/camera_sensor.h\"\n+#include \"libcamera/internal/control_serializer.h\"\n+#include \"libcamera/internal/log.h\"\n+\n+namespace libcamera {\n+\n+LOG_DECLARE_CATEGORY(IPADataSerializer)\n+\n+namespace {\n+\n+template\n+void appendPOD(std::vector &vec,\n+\t typename std::enable_if, T>::type val)\n+{\n+\tsize_t byteWidth = sizeof(val);\n+\tvec.resize(vec.size() + byteWidth);\n+\tmemcpy(&*(vec.end() - byteWidth), &val, byteWidth);\n+}\n+\n+template\n+typename std::enable_if, T>::type\n+readPOD(std::vector::const_iterator it, size_t pos,\n+\tstd::vector::const_iterator end)\n+{\n+\tASSERT(pos + it < end);\n+\n+\tT ret = 0;\n+\tmemcpy(&ret, &(*(it + pos)), sizeof(ret));\n+\n+\treturn ret;\n+}\n+\n+template\n+typename std::enable_if, T>::type\n+readPOD(std::vector &vec, size_t pos)\n+{\n+\treturn readPOD(vec.cbegin(), pos, vec.end());\n+}\n+\n+} /* namespace */\n+\n+template\n+class IPADataSerializer\n+{\n+public:\n+\tstatic std::tuple, std::vector>\n+\tserialize(const T &data, ControlSerializer *cs);\n+\n+\tstatic T deserialize(const std::vector &data,\n+\t\t\t ControlSerializer *cs);\n+\tstatic T deserialize(std::vector::const_iterator dataBegin,\n+\t\t\t std::vector::const_iterator dataEnd,\n+\t\t\t ControlSerializer *cs);\n+\n+\tstatic T deserialize(const std::vector &data,\n+\t\t\t const std::vector &fds,\n+\t\t\t ControlSerializer *cs);\n+\tstatic T deserialize(std::vector::const_iterator dataBegin,\n+\t\t\t std::vector::const_iterator dataEnd,\n+\t\t\t std::vector::const_iterator fdsBegin,\n+\t\t\t std::vector::const_iterator fdsEnd,\n+\t\t\t ControlSerializer *cs);\n+};\n+\n+#ifndef __DOXYGEN__\n+\n+/*\n+ * Serialization format for vector of type V:\n+ *\n+ * 4 bytes - uint32_t Length of vector, in number of elements\n+ *\n+ * For every element in the vector:\n+ *\n+ * 4 bytes - uint32_t Size of element, in bytes\n+ * 4 bytes - uint32_t Size of fds for the element, in number of fds\n+ * X bytes - Serialized element\n+ */\n+template\n+class IPADataSerializer>\n+{\n+public:\n+\tstatic std::tuple, std::vector>\n+\tserialize(const std::vector &data, ControlSerializer *cs = nullptr)\n+\t{\n+\t\tstd::vector dataVec;\n+\t\tstd::vector fdsVec;\n+\n+\t\t/* Serialize the length. */\n+\t\tuint32_t vecLen = data.size();\n+\t\tappendPOD(dataVec, vecLen);\n+\n+\t\t/* Serialize the members. */\n+\t\tfor (auto const &it : data) {\n+\t\t\tstd::vector dvec;\n+\t\t\tstd::vector fvec;\n+\n+\t\t\tstd::tie(dvec, fvec) =\n+\t\t\t\tIPADataSerializer::serialize(it, cs);\n+\n+\t\t\tappendPOD(dataVec, dvec.size());\n+\t\t\tappendPOD(dataVec, fvec.size());\n+\n+\t\t\tdataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n+\t\t\tfdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n+\t\t}\n+\n+\t\treturn { dataVec, fdsVec };\n+\t}\n+\n+\tstatic std::vector deserialize(std::vector &data, ControlSerializer *cs = nullptr)\n+\t{\n+\t\treturn deserialize(data.cbegin(), data.end(), cs);\n+\t}\n+\n+\tstatic std::vector deserialize(std::vector::const_iterator dataBegin,\n+\t\t\t\t\t std::vector::const_iterator dataEnd,\n+\t\t\t\t\t ControlSerializer *cs = nullptr)\n+\t{\n+\t\tstd::vector fds;\n+\t\treturn deserialize(dataBegin, dataEnd, fds.cbegin(), fds.end(), cs);\n+\t}\n+\n+\tstatic std::vector deserialize(std::vector &data, std::vector &fds,\n+\t\t\t\t\t ControlSerializer *cs = nullptr)\n+\t{\n+\t\treturn deserialize(data.cbegin(), data.end(), fds.cbegin(), fds.end(), cs);\n+\t}\n+\n+\tstatic std::vector deserialize(std::vector::const_iterator dataBegin,\n+\t\t\t\t\t std::vector::const_iterator dataEnd,\n+\t\t\t\t\t std::vector::const_iterator fdsBegin,\n+\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n+\t\t\t\t\t ControlSerializer *cs = nullptr)\n+\t{\n+\t\tuint32_t vecLen = readPOD(dataBegin, 0, dataEnd);\n+\t\tstd::vector ret(vecLen);\n+\n+\t\tstd::vector::const_iterator dataIter = dataBegin + 4;\n+\t\tstd::vector::const_iterator fdIter = fdsBegin;\n+\t\tfor (uint32_t i = 0; i < vecLen; i++) {\n+\t\t\tuint32_t sizeofData = readPOD(dataIter, 0, dataEnd);\n+\t\t\tuint32_t sizeofFds = readPOD(dataIter, 4, dataEnd);\n+\t\t\tdataIter += 8;\n+\n+\t\t\tret[i] = IPADataSerializer::deserialize(dataIter,\n+\t\t\t\t\t\t\t\t dataIter + sizeofData,\n+\t\t\t\t\t\t\t\t fdIter,\n+\t\t\t\t\t\t\t\t fdIter + sizeofFds,\n+\t\t\t\t\t\t\t\t cs);\n+\n+\t\t\tdataIter += sizeofData;\n+\t\t\tfdIter += sizeofFds;\n+\t\t}\n+\n+\t\treturn ret;\n+\t}\n+};\n+\n+/*\n+ * Serialization format for map of key type K and value type V:\n+ *\n+ * 4 bytes - uint32_t Length of map, in number of pairs\n+ *\n+ * For every pair in the map:\n+ *\n+ * 4 bytes - uint32_t Size of key, in bytes\n+ * 4 bytes - uint32_t Size of fds for the key, in number of fds\n+ * X bytes - Serialized key\n+ * 4 bytes - uint32_t Size of value, in bytes\n+ * 4 bytes - uint32_t Size of fds for the value, in number of fds\n+ * X bytes - Serialized value\n+ */\n+template\n+class IPADataSerializer>\n+{\n+public:\n+\tstatic std::tuple, std::vector>\n+\tserialize(const std::map &data, ControlSerializer *cs = nullptr)\n+\t{\n+\t\tstd::vector dataVec;\n+\t\tstd::vector fdsVec;\n+\n+\t\t/* Serialize the length. */\n+\t\tuint32_t mapLen = data.size();\n+\t\tappendPOD(dataVec, mapLen);\n+\n+\t\t/* Serialize the members. */\n+\t\tfor (auto const &it : data) {\n+\t\t\tstd::vector dvec;\n+\t\t\tstd::vector fvec;\n+\n+\t\t\tstd::tie(dvec, fvec) =\n+\t\t\t\tIPADataSerializer::serialize(it.first, cs);\n+\n+\t\t\tappendPOD(dataVec, dvec.size());\n+\t\t\tappendPOD(dataVec, fvec.size());\n+\n+\t\t\tdataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n+\t\t\tfdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n+\n+\t\t\tstd::tie(dvec, fvec) =\n+\t\t\t\tIPADataSerializer::serialize(it.second, cs);\n+\n+\t\t\tappendPOD(dataVec, dvec.size());\n+\t\t\tappendPOD(dataVec, fvec.size());\n+\n+\t\t\tdataVec.insert(dataVec.end(), dvec.begin(), dvec.end());\n+\t\t\tfdsVec.insert(fdsVec.end(), fvec.begin(), fvec.end());\n+\t\t}\n+\n+\t\treturn { dataVec, fdsVec };\n+\t}\n+\n+\tstatic std::map deserialize(std::vector &data, ControlSerializer *cs = nullptr)\n+\t{\n+\t\treturn deserialize(data.cbegin(), data.end(), cs);\n+\t}\n+\n+\tstatic std::map deserialize(std::vector::const_iterator dataBegin,\n+\t\t\t\t\t std::vector::const_iterator dataEnd,\n+\t\t\t\t\t ControlSerializer *cs = nullptr)\n+\t{\n+\t\tstd::vector fds;\n+\t\treturn deserialize(dataBegin, dataEnd, fds.cbegin(), fds.end(), cs);\n+\t}\n+\n+\tstatic std::map deserialize(std::vector &data, std::vector &fds,\n+\t\t\t\t\t ControlSerializer *cs = nullptr)\n+\t{\n+\t\treturn deserialize(data.cbegin(), data.end(), fds.cbegin(), fds.end(), cs);\n+\t}\n+\n+\tstatic std::map deserialize(std::vector::const_iterator dataBegin,\n+\t\t\t\t\t std::vector::const_iterator dataEnd,\n+\t\t\t\t\t std::vector::const_iterator fdsBegin,\n+\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n+\t\t\t\t\t ControlSerializer *cs = nullptr)\n+\t{\n+\t\tstd::map ret;\n+\n+\t\tuint32_t mapLen = readPOD(dataBegin, 0, dataEnd);\n+\n+\t\tstd::vector::const_iterator dataIter = dataBegin + 4;\n+\t\tstd::vector::const_iterator fdIter = fdsBegin;\n+\t\tfor (uint32_t i = 0; i < mapLen; i++) {\n+\t\t\tuint32_t sizeofData = readPOD(dataIter, 0, dataEnd);\n+\t\t\tuint32_t sizeofFds = readPOD(dataIter, 4, dataEnd);\n+\t\t\tdataIter += 8;\n+\n+\t\t\tK key = IPADataSerializer::deserialize(dataIter,\n+\t\t\t\t\t\t\t\t dataIter + sizeofData,\n+\t\t\t\t\t\t\t\t fdIter,\n+\t\t\t\t\t\t\t\t fdIter + sizeofFds,\n+\t\t\t\t\t\t\t\t cs);\n+\n+\t\t\tdataIter += sizeofData;\n+\t\t\tfdIter += sizeofFds;\n+\t\t\tsizeofData = readPOD(dataIter, 0, dataEnd);\n+\t\t\tsizeofFds = readPOD(dataIter, 4, dataEnd);\n+\t\t\tdataIter += 8;\n+\n+\t\t\tconst V value = IPADataSerializer::deserialize(dataIter,\n+\t\t\t\t\t\t\t\t\t dataIter + sizeofData,\n+\t\t\t\t\t\t\t\t\t fdIter,\n+\t\t\t\t\t\t\t\t\t fdIter + sizeofFds,\n+\t\t\t\t\t\t\t\t\t cs);\n+\t\t\tret.insert({key, value});\n+\n+\t\t\tdataIter += sizeofData;\n+\t\t\tfdIter += sizeofFds;\n+\t\t}\n+\n+\t\treturn ret;\n+\t}\n+};\n+\n+#define DECLARE_POD_SERIALIZER(type)\t\t\t\t\t\\\n+template<>\t\t\t\t\t\t\t\t\\\n+class IPADataSerializer\t\t\t\t\t\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+public:\t\t\t\t\t\t\t\t\t\\\n+\tstatic std::tuple, std::vector>\t\\\n+\tserialize(const type data,\t\t\t\t\t\\\n+\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\t\\\n+\t{\t\t\t\t\t\t\t\t\\\n+\t\tstd::vector dataVec;\t\t\t\t\\\n+\t\tdataVec.reserve(sizeof(type));\t\t\t\t\\\n+\t\tappendPOD(dataVec, data);\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\t\treturn { dataVec, {} };\t\t\t\t\t\\\n+\t}\t\t\t\t\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\tstatic type deserialize(std::vector &data,\t\t\\\n+\t\t\t\t[[maybe_unused]] ControlSerializer *cs = nullptr)\\\n+\t{\t\t\t\t\t\t\t\t\\\n+\t\treturn deserialize(data.cbegin(), data.end());\t\t\\\n+\t}\t\t\t\t\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\tstatic type deserialize(std::vector::const_iterator dataBegin,\\\n+\t\t\t\t[[maybe_unused]] std::vector::const_iterator dataEnd,\\\n+\t\t\t\t[[maybe_unused]] ControlSerializer *cs = nullptr)\\\n+\t{\t\t\t\t\t\t\t\t\\\n+\t\treturn readPOD(dataBegin, 0, dataEnd);\t\t\\\n+\t}\t\t\t\t\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\tstatic type deserialize(std::vector &data,\t\t\\\n+\t\t\t\t[[maybe_unused]] std::vector &fds,\\\n+\t\t\t\t[[maybe_unused]] ControlSerializer *cs = nullptr)\\\n+\t{\t\t\t\t\t\t\t\t\\\n+\t\treturn deserialize(data.cbegin(), data.end());\t\t\\\n+\t}\t\t\t\t\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\tstatic type deserialize(std::vector::const_iterator dataBegin,\\\n+\t\t\t\tstd::vector::const_iterator dataEnd,\\\n+\t\t\t\t[[maybe_unused]] std::vector::const_iterator fdsBegin,\\\n+\t\t\t\t[[maybe_unused]] std::vector::const_iterator fdsEnd,\\\n+\t\t\t\t[[maybe_unused]] ControlSerializer *cs = nullptr)\\\n+\t{\t\t\t\t\t\t\t\t\\\n+\t\treturn deserialize(dataBegin, dataEnd);\t\t\t\\\n+\t}\t\t\t\t\t\t\t\t\\\n+};\n+\n+DECLARE_POD_SERIALIZER(bool)\n+DECLARE_POD_SERIALIZER(uint8_t)\n+DECLARE_POD_SERIALIZER(uint16_t)\n+DECLARE_POD_SERIALIZER(uint32_t)\n+DECLARE_POD_SERIALIZER(uint64_t)\n+DECLARE_POD_SERIALIZER(int8_t)\n+DECLARE_POD_SERIALIZER(int16_t)\n+DECLARE_POD_SERIALIZER(int32_t)\n+DECLARE_POD_SERIALIZER(int64_t)\n+DECLARE_POD_SERIALIZER(float)\n+DECLARE_POD_SERIALIZER(double)\n+\n+/*\n+ * Strings are serialized simply by converting by {string.cbegin(), string.end()}.\n+ * The size of the string is recorded by the container (struct, vector, map, or\n+ * function parameter serdes).\n+ */\n+template<>\n+class IPADataSerializer\n+{\n+public:\n+\tstatic std::tuple, std::vector>\n+\tserialize(const std::string &data, [[maybe_unused]] ControlSerializer *cs = nullptr)\n+\t{\n+\t\treturn { { data.cbegin(), data.end() }, {} };\n+\t}\n+\n+\tstatic std::string deserialize(std::vector &data,\n+\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n+\t{\n+\t\treturn { data.cbegin(), data.cend() };\n+\t}\n+\n+\tstatic std::string deserialize(std::vector::const_iterator dataBegin,\n+\t\t\t\t std::vector::const_iterator dataEnd,\n+\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n+\t{\n+\t\treturn { dataBegin, dataEnd };\n+\t}\n+\n+\tstatic std::string deserialize(std::vector &data,\n+\t\t\t\t [[maybe_unused]] std::vector &fds,\n+\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n+\t{\n+\t\treturn { data.cbegin(), data.cend() };\n+\t}\n+\n+\tstatic std::string deserialize(std::vector::const_iterator dataBegin,\n+\t\t\t\t std::vector::const_iterator dataEnd,\n+\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n+\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n+\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n+\t{\n+\t\treturn { dataBegin, dataEnd };\n+\t}\n+};\n+\n+/*\n+ * FileDescriptors are serialized into a single byte that tells if the\n+ * FileDescriptor is valid or not. If it is valid, then for serialization\n+ * the fd will be written to the fd vector, or for deserialization the\n+ * fd vector const_iterator will be valid.\n+ *\n+ * This validity is necessary so that we don't send -1 fd over sendmsg(). It\n+ * also allows us to simply send the entire fd vector into the deserializer\n+ * and it will be recursively consumed as necessary.\n+ */\n+template<>\n+class IPADataSerializer\n+{\n+public:\n+\tstatic std::tuple, std::vector>\n+\tserialize(const FileDescriptor &data, [[maybe_unused]] ControlSerializer *cs = nullptr)\n+\t{\n+\t\tstd::vector dataVec = { data.isValid() };\n+\t\tstd::vector fdVec;\n+\t\tif (data.isValid())\n+\t\t\tfdVec.push_back(data.fd());\n+\n+\t\treturn { dataVec, fdVec };\n+\t}\n+\n+\tstatic FileDescriptor deserialize(std::vector &data, std::vector &fds,\n+\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n+\t{\n+\t\treturn deserialize(data.cbegin(), data.end(), fds.cbegin(), fds.end());\n+\t}\n+\n+\tstatic FileDescriptor deserialize(std::vector::const_iterator dataBegin,\n+\t\t\t\t\t std::vector::const_iterator dataEnd,\n+\t\t\t\t\t std::vector::const_iterator fdsBegin,\n+\t\t\t\t\t std::vector::const_iterator fdsEnd,\n+\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n+\t{\n+\t\tASSERT(std::distance(dataBegin, dataEnd) >= 1);\n+\n+\t\tbool valid = !!(*dataBegin);\n+\n+\t\tASSERT(!(valid && std::distance(fdsBegin, fdsEnd) < 1));\n+\n+\t\treturn valid ? FileDescriptor(*fdsBegin) : FileDescriptor();\n+\t}\n+};\n+\n+/*\n+ * ControlList is serialized as:\n+ *\n+ * 4 bytes - uint32_t Size of serialized ControlInfoMap, in bytes\n+ * 4 bytes - uint32_t Size of serialized ControlList, in bytes\n+ * X bytes - Serialized ControlInfoMap (using ControlSerializer)\n+ * X bytes - Serialized ControlList (using ControlSerializer)\n+ *\n+ * If data.infoMap() is nullptr, then the default controls::controls will\n+ * be used. The serialized ControlInfoMap will have zero length.\n+ */\n+template<>\n+class IPADataSerializer\n+{\n+public:\n+\tstatic std::tuple, std::vector>\n+\tserialize(const ControlList &data, ControlSerializer *cs)\n+\t{\n+\t\tif (!cs)\n+\t\t\tLOG(IPADataSerializer, Fatal)\n+\t\t\t\t<< \"ControlSerializer not provided for serialization of ControlList\";\n+\n+\t\tsize_t size;\n+\t\tstd::vector infoData;\n+\t\tint ret;\n+\n+\t\tif (data.infoMap() && !cs->isCached(data.infoMap())) {\n+\t\t\tsize = cs->binarySize(*data.infoMap());\n+\t\t\tinfoData.resize(size);\n+\t\t\tByteStreamBuffer buffer(infoData.data(), infoData.size());\n+\t\t\tret = cs->serialize(*data.infoMap(), buffer);\n+\n+\t\t\tif (ret < 0 || buffer.overflow()) {\n+\t\t\t\tLOG(IPADataSerializer, Error) << \"Failed to serialize ControlList's ControlInfoMap\";\n+\t\t\t\treturn { {}, {} };\n+\t\t\t}\n+\t\t}\n+\n+\t\tsize = cs->binarySize(data);\n+\t\tstd::vector listData(size);\n+\t\tByteStreamBuffer buffer(listData.data(), listData.size());\n+\t\tret = cs->serialize(data, buffer);\n+\n+\t\tif (ret < 0 || buffer.overflow()) {\n+\t\t\tLOG(IPADataSerializer, Error) << \"Failed to serialize ControlList\";\n+\t\t\treturn { {}, {} };\n+\t\t}\n+\n+\t\tstd::vector dataVec;\n+\t\tdataVec.reserve(8 + infoData.size() + listData.size());\n+\t\tappendPOD(dataVec, infoData.size());\n+\t\tappendPOD(dataVec, listData.size());\n+\t\tdataVec.insert(dataVec.end(), infoData.begin(), infoData.end());\n+\t\tdataVec.insert(dataVec.end(), listData.begin(), listData.end());\n+\n+\t\treturn { dataVec, {} };\n+\t}\n+\n+\tstatic ControlList deserialize(std::vector &data, ControlSerializer *cs)\n+\t{\n+\t\treturn deserialize(data.cbegin(), data.end(), cs);\n+\t}\n+\n+\tstatic ControlList deserialize(std::vector::const_iterator dataBegin,\n+\t\t\t\t std::vector::const_iterator dataEnd,\n+\t\t\t\t ControlSerializer *cs)\n+\t{\n+\t\tif (!cs)\n+\t\t\tLOG(IPADataSerializer, Fatal)\n+\t\t\t\t<< \"ControlSerializer not provided for deserialization of ControlList\";\n+\n+\t\tif (!std::distance(dataBegin, dataEnd))\n+\t\t\treturn {};\n+\n+\t\tuint32_t infoDataSize = readPOD(dataBegin, 0, dataEnd);\n+\t\tuint32_t listDataSize = readPOD(dataBegin, 4, dataEnd);\n+\n+\t\tstd::vector::const_iterator it = dataBegin + 8;\n+\n+\t\tstd::vector infoData(it, it + infoDataSize);\n+\t\tstd::vector listData(it + infoDataSize, it + infoDataSize + listDataSize);\n+\n+\t\tif (infoDataSize > 0) {\n+\t\t\tByteStreamBuffer buffer(const_cast(infoData.data()), infoData.size());\n+\t\t\tControlInfoMap map = cs->deserialize(buffer);\n+\t\t\t/* It's fine if map is empty. */\n+\t\t\tif (buffer.overflow()) {\n+\t\t\t\tLOG(IPADataSerializer, Error)\n+\t\t\t\t\t<< \"Failed to deserialize ControlLists's ControlInfoMap: buffer overflow\";\n+\t\t\t\treturn ControlList();\n+\t\t\t}\n+\t\t}\n+\n+\t\tByteStreamBuffer buffer(const_cast(listData.data()), listData.size());\n+\t\tControlList list = cs->deserialize(buffer);\n+\t\tif (buffer.overflow())\n+\t\t\tLOG(IPADataSerializer, Error) << \"Failed to deserialize ControlList: buffer overflow\";\n+\t\tif (list.empty())\n+\t\t\tLOG(IPADataSerializer, Error) << \"Failed to deserialize ControlList: empty list\";\n+\n+\t\treturn list;\n+\t}\n+\n+\tstatic ControlList deserialize(std::vector &data,\n+\t\t\t\t [[maybe_unused]] std::vector &fds,\n+\t\t\t\t ControlSerializer *cs)\n+\t{\n+\t\treturn deserialize(data.cbegin(), data.end(), cs);\n+\t}\n+\n+\tstatic ControlList deserialize(std::vector::const_iterator dataBegin,\n+\t\t\t\t std::vector::const_iterator dataEnd,\n+\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n+\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n+\t\t\t\t ControlSerializer *cs)\n+\t{\n+\t\treturn deserialize(dataBegin, dataEnd, cs);\n+\t}\n+};\n+\n+/*\n+ * const ControlInfoMap is serialized as:\n+ *\n+ * 4 bytes - uint32_t Size of serialized ControlInfoMap, in bytes\n+ * X bytes - Serialized ControlInfoMap (using ControlSerializer)\n+ */\n+template<>\n+class IPADataSerializer\n+{\n+public:\n+\tstatic std::tuple, std::vector>\n+\tserialize(const ControlInfoMap &map, ControlSerializer *cs)\n+\t{\n+\t\tif (!cs)\n+\t\t\tLOG(IPADataSerializer, Fatal)\n+\t\t\t\t<< \"ControlSerializer not provided for serialization of ControlInfoMap\";\n+\n+\t\tsize_t size = cs->binarySize(map);\n+\t\tstd::vector infoData(size);\n+\t\tByteStreamBuffer buffer(infoData.data(), infoData.size());\n+\t\tint ret = cs->serialize(map, buffer);\n+\n+\t\tif (ret < 0 || buffer.overflow()) {\n+\t\t\tLOG(IPADataSerializer, Error) << \"Failed to serialize ControlInfoMap\";\n+\t\t\treturn { {}, {} };\n+\t\t}\n+\n+\t\tstd::vector dataVec;\n+\t\tappendPOD(dataVec, infoData.size());\n+\t\tdataVec.insert(dataVec.end(), infoData.begin(), infoData.end());\n+\n+\t\treturn { dataVec, {} };\n+\t}\n+\n+\tstatic ControlInfoMap deserialize(std::vector &data,\n+\t\t\t\t\t ControlSerializer *cs)\n+\t{\n+\t\treturn deserialize(data.cbegin(), data.end(), cs);\n+\t}\n+\n+\tstatic ControlInfoMap deserialize(std::vector::const_iterator dataBegin,\n+\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator dataEnd,\n+\t\t\t\t\t ControlSerializer *cs)\n+\t{\n+\t\tif (!cs)\n+\t\t\tLOG(IPADataSerializer, Fatal)\n+\t\t\t\t<< \"ControlSerializer not provided for deserialization of ControlInfoMap\";\n+\n+\t\tuint32_t infoDataSize = readPOD(dataBegin, 0, dataEnd);\n+\n+\t\tstd::vector::const_iterator it = dataBegin + 4;\n+\n+\t\tstd::vector infoData(it, it + infoDataSize);\n+\n+\t\tByteStreamBuffer buffer(const_cast(infoData.data()), infoData.size());\n+\t\tControlInfoMap map = cs->deserialize(buffer);\n+\n+\t\treturn map;\n+\t}\n+\n+\tstatic ControlInfoMap deserialize(std::vector &data,\n+\t\t\t\t\t [[maybe_unused]] std::vector &fds,\n+\t\t\t\t\t ControlSerializer *cs)\n+\t{\n+\t\treturn deserialize(data.cbegin(), data.end(), cs);\n+\t}\n+\n+\tstatic ControlInfoMap deserialize(std::vector::const_iterator dataBegin,\n+\t\t\t\t\t std::vector::const_iterator dataEnd,\n+\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n+\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n+\t\t\t\t\t ControlSerializer *cs)\n+\t{\n+\t\treturn deserialize(dataBegin, dataEnd, cs);\n+\t}\n+};\n+\n+/*\n+ * FrameBuffer::Plane is serialized as:\n+ *\n+ * 1 byte - FileDescriptor\n+ * 4 bytes - uint32_t Length\n+ */\n+template<>\n+class IPADataSerializer\n+{\n+public:\n+\tstatic std::tuple, std::vector>\n+\tserialize(const FrameBuffer::Plane &data, [[maybe_unused]] ControlSerializer *cs = nullptr)\n+\t{\n+\t\tstd::vector dataVec;\n+\t\tstd::vector fdsVec;\n+\n+\t\tstd::vector fdBuf;\n+\t\tstd::vector fdFds;\n+\t\tstd::tie(fdBuf, fdFds) =\n+\t\t\tIPADataSerializer::serialize(data.fd);\n+\t\tdataVec.insert(dataVec.end(), fdBuf.begin(), fdBuf.end());\n+\t\tfdsVec.insert(fdsVec.end(), fdFds.begin(), fdFds.end());\n+\n+\t\tappendPOD(dataVec, data.length);\n+\n+\t\treturn { dataVec, fdsVec };\n+\t}\n+\n+\tstatic FrameBuffer::Plane deserialize(std::vector &data,\n+\t\t\t\t\t std::vector &fds,\n+\t\t\t\t\t ControlSerializer *cs = nullptr)\n+\t{\n+\t\treturn deserialize(data.cbegin(), data.end(), fds.cbegin(), fds.end(), cs);\n+\t}\n+\n+\tstatic FrameBuffer::Plane deserialize(std::vector::const_iterator dataBegin,\n+\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator dataEnd,\n+\t\t\t\t\t std::vector::const_iterator fdsBegin,\n+\t\t\t\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n+\t\t\t\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n+\t{\n+\t\tFrameBuffer::Plane ret;\n+\n+\t\tret.fd = IPADataSerializer::deserialize(dataBegin, dataBegin + 1,\n+\t\t\t\t\t\t\t\t\tfdsBegin, fdsBegin + 1);\n+\t\tret.length = readPOD(dataBegin, 1, dataEnd);\n+\n+\t\treturn ret;\n+\t}\n+};\n+\n+#endif /* __DOXYGEN__ */\n+\n+} /* namespace libcamera */\n+\n+#endif /* __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_H__ */\ndiff --git a/src/libcamera/ipa_data_serializer.cpp b/src/libcamera/ipa_data_serializer.cpp\nnew file mode 100644\nindex 00000000..34287d6a\n--- /dev/null\n+++ b/src/libcamera/ipa_data_serializer.cpp\n@@ -0,0 +1,185 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2020, Google Inc.\n+ *\n+ * ipa_data_serializer.cpp - Image Processing Algorithm data serializer\n+ */\n+\n+#include \"libcamera/internal/ipa_data_serializer.h\"\n+\n+#include \"libcamera/internal/log.h\"\n+\n+/**\n+ * \\file ipa_data_serializer.h\n+ * \\brief IPA Data Serializer\n+ */\n+\n+namespace libcamera {\n+\n+LOG_DEFINE_CATEGORY(IPADataSerializer)\n+\n+/**\n+ * \\class IPADataSerializer\n+ * \\brief IPA Data Serializer\n+ *\n+ * Static template class that provides functions for serializing and\n+ * deserializing IPA data.\n+ *\n+ * \\todo Switch to Span instead of byte and fd vector\n+ *\n+ * \\todo Harden the vector and map deserializer\n+ *\n+ * \\todo For FileDescriptors, instead of storing a validity flag, store an\n+ * index into the fd array. This will allow us to use views instead of copying.\n+ */\n+\n+namespace {\n+\n+/**\n+ * \\fn template void appendPOD(std::vector &vec, T val)\n+ * \\brief Append POD to end of byte vector, in little-endian order\n+ * \\tparam T Type of POD to append\n+ * \\param[in] vec Byte vector to append to\n+ * \\param[in] val Value to append\n+ *\n+ * This function is meant to be used by the IPA data serializer, and the\n+ * generated IPA proxies.\n+ */\n+\n+/**\n+ * \\fn template T readPOD(std::vector::iterator it, size_t pos,\n+ * \t\t\t\t std::vector::iterator end)\n+ * \\brief Read POD from byte vector, in little-endian order\n+ * \\tparam T Type of POD to read\n+ * \\param[in] it Iterator of byte vector to read from\n+ * \\param[in] pos Index in byte vector to read from\n+ * \\param[in] end Iterator marking end of byte vector\n+ *\n+ * This function is meant to be used by the IPA data serializer, and the\n+ * generated IPA proxies.\n+ *\n+ * If the \\a pos plus the byte-width of the desired POD is past \\a end, it is\n+ * a fata error will occur, as it means there is insufficient data for\n+ * deserialization, which should never happen.\n+ *\n+ * \\return The POD read from \\a it at index \\a pos\n+ */\n+\n+/**\n+ * \\fn template T readPOD(std::vector &vec, size_t pos)\n+ * \\brief Read POD from byte vector, in little-endian order\n+ * \\tparam T Type of POD to read\n+ * \\param[in] vec Byte vector to read from\n+ * \\param[in] pos Index in vec to start reading from\n+ *\n+ * This function is meant to be used by the IPA data serializer, and the\n+ * generated IPA proxies.\n+ *\n+ * If the \\a pos plus the byte-width of the desired POD is past the end of\n+ * \\a vec, a fatal error will occur, as it means there is insufficient data\n+ * for deserialization, which should never happen.\n+ *\n+ * \\return The POD read from \\a vec at index \\a pos\n+ */\n+\n+} /* namespace */\n+\n+/**\n+ * \\fn template IPADataSerializer::serialize(\n+ * \tT data,\n+ * \tControlSerializer *cs = nullptr)\n+ * \\brief Serialize an object into byte vector and fd vector\n+ * \\tparam T Type of object to serialize\n+ * \\param[in] data Object to serialize\n+ * \\param[in] cs ControlSerializer\n+ *\n+ * \\a cs is only necessary if the object type \\a T or its members contain\n+ * ControlList or ControlInfoMap.\n+ *\n+ * \\return Tuple of byte vector and fd vector, that is the serialized form\n+ * of \\a data\n+ */\n+\n+/**\n+ * \\fn template IPADataSerializer::deserialize(\n+ * \tconst std::vector &data,\n+ * \tControlSerializer *cs = nullptr)\n+ * \\brief Deserialize byte vector into an object\n+ * \\tparam T Type of object to deserialize to\n+ * \\param[in] data Byte vector to deserialize from\n+ * \\param[in] cs ControlSerializer\n+ *\n+ * This version of deserialize() can be used if the object type \\a T and its\n+ * members don't have any FileDescriptor.\n+ *\n+ * \\a cs is only necessary if the object type \\a T or its members contain\n+ * ControlList or ControlInfoMap.\n+ *\n+ * \\return The deserialized object\n+ */\n+\n+/**\n+ * \\fn template IPADataSerializer::deserialize(\n+ * \tstd::vector::const_iterator dataBegin,\n+ * \tstd::vector::const_iterator dataEnd,\n+ * \tControlSerializer *cs = nullptr)\n+ * \\brief Deserialize byte vector into an object\n+ * \\tparam T Type of object to deserialize to\n+ * \\param[in] dataBegin Begin iterator of byte vector to deserialize from\n+ * \\param[in] dataEnd End iterator of byte vector to deserialize from\n+ * \\param[in] cs ControlSerializer\n+ *\n+ * This version of deserialize() can be used if the object type \\a T and its\n+ * members don't have any FileDescriptor.\n+ *\n+ * \\a cs is only necessary if the object type \\a T or its members contain\n+ * ControlList or ControlInfoMap.\n+ *\n+ * \\return The deserialized object\n+ */\n+\n+/**\n+ * \\fn template IPADataSerializer::deserialize(\n+ * \tconst std::vector &data,\n+ * \tconst std::vector &fds,\n+ * \tControlSerializer *cs = nullptr)\n+ * \\brief Deserialize byte vector and fd vector into an object\n+ * \\tparam T Type of object to deserialize to\n+ * \\param[in] data Byte vector to deserialize from\n+ * \\param[in] fds Fd vector to deserialize from\n+ * \\param[in] cs ControlSerializer\n+ *\n+ * This version of deserialize() (or the iterator version) must be used if\n+ * the object type \\a T or its members contain FileDescriptor.\n+ *\n+ * \\a cs is only necessary if the object type \\a T or its members contain\n+ * ControlList or ControlInfoMap.\n+ *\n+ * \\return The deserialized object\n+ */\n+\n+/**\n+ * \\fn template IPADataSerializer::deserialize(\n+ * \tstd::vector::const_iterator dataBegin,\n+ * \tstd::vector::const_iterator dataEnd,\n+ * \tstd::vector::const_iterator fdsBegin,\n+ * \tstd::vector::const_iterator fdsEnd,\n+ * \tControlSerializer *cs = nullptr)\n+ * \\brief Deserialize byte vector and fd vector into an object\n+ * \\tparam T Type of object to deserialize to\n+ * \\param[in] dataBegin Begin iterator of byte vector to deserialize from\n+ * \\param[in] dataEnd End iterator of byte vector to deserialize from\n+ * \\param[in] fdsBegin Begin iterator of fd vector to deserialize from\n+ * \\param[in] fdsEnd End iterator of fd vector to deserialize from\n+ * \\param[in] cs ControlSerializer\n+ *\n+ * This version of deserialize() (or the vector version) must be used if\n+ * the object type \\a T or its members contain FileDescriptor.\n+ *\n+ * \\a cs is only necessary if the object type \\a T or its members contain\n+ * ControlList or ControlInfoMap.\n+ *\n+ * \\return The deserialized object\n+ */\n+\n+} /* namespace libcamera */\ndiff --git a/src/libcamera/meson.build b/src/libcamera/meson.build\nindex 387d5d88..1291f1b3 100644\n--- a/src/libcamera/meson.build\n+++ b/src/libcamera/meson.build\n@@ -25,6 +25,7 @@ libcamera_sources = files([\n 'geometry.cpp',\n 'ipa_context_wrapper.cpp',\n 'ipa_controls.cpp',\n+ 'ipa_data_serializer.cpp',\n 'ipa_interface.cpp',\n 'ipa_manager.cpp',\n 'ipa_module.cpp',\n","prefixes":["libcamera-devel","v6","6/9"]}