From patchwork Thu Dec 24 08:15:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 10714 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CEC85C0F1A for ; Thu, 24 Dec 2020 08:15:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2A318615B0; Thu, 24 Dec 2020 09:15:47 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="b57pQzIG"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3BF0960525 for ; Thu, 24 Dec 2020 09:15:46 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CF716A1D; Thu, 24 Dec 2020 09:15:44 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1608797745; bh=HtqeFfzc1zO90hjWrIx7bPcOFMJUELDK4ldIatUgFIw=; h=From:To:Cc:Subject:Date:From; b=b57pQzIGjjVaZvpMvS47AP3Zr5PidFatkc2YlkxJN5NiMe3Bp0EkIeNZutZ2JnxDX NGDIsqMulIi9CIczBrcy9fZBJwDbWMRyt9fgmZa8RLN0B0U7Tuzrq7xPNhIpHY6aDA 8jZ8YLXwHKJcTHIzPbv/Vyb9tFIvfkfYgdc7wISI= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Dec 2020 17:15:25 +0900 Message-Id: <20201224081534.41601-1-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 0/9] IPA isolation: Part 1: Core components X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" v6 is split in three parts, core components, conversion + plumbing, and tests + documentation. This is part 1, and implements IPA isolation. To restate the problem, we have two goals: - to be able to run IPAs isolated in a separate process - the isolation must be transparent to both the pipeline handler and the IPA During design of the IPC mechanism, we realized that we could support both custom fuctions and custom data structures, which would be a lot nicer than the tedious manual de/serialization that we had before with IPAOperationData. The architecture of the whole thing is as follows: pipeline handler -> IPAProxyRPi --thread--> IPARPi \ \-> IPCPipe --IPC--> IPAProxyRPiWorker -> IPARPi Where the pipeline author defines the interface between the pipeline handler and the IPA, as well as any data structures. Based on this, headers, de/serializers, and the proxy (including the worker) will be generated. Patches 2/9 to 9/9 add the meat of this IPC and custom IPA interface mechanisms, including the code generation templates, serializers, and IPC message pipes. 1/9 is new in v6, and implements caching ControlInfoMaps in ControlSerializer. This allows us to avoid reserializing ControlInfoMaps every time we send a ControlList over IPC. In v6 we newly support custom start() definitions. The biggest change in v6 is that we now support defining structs in mojom for core libcamera structs, as is shown in core.mojom in 9/9. We can also designate individually for every struct if we want a header definition as well, or just a serializer to be generated. Defining consts in the mojom file is also supported as of v6, meaning that the rkisp1 and vimc headers can be removed (in Part 2), since those headers only define consts and enums. Changes in v6: - ControlInfoMap caching in ControlSerializer - support custom start() - support defining libcamera structs in core.mojom - support consts in mojom - namespacing is required in mojom files - expand documentation on mojom, in core.mojom Changes in v5: - rename IPAIPC to IPCPipe - IPCPipe passes IPCMessages, instead of byte vector + fd vector - rename the generated files so the naming is consistent - make IPADataSerializer use const references and const iterators - removal of IPA wrapper test moved earlier - squash all patches related to replacing the C IPA interface with the customizable C++ IPA interface - squash all patches related to making the IPAProxy one-per-pipeline Changes in v4: - add IPA guide - add test for header and serializer generation - fix style in generated code and IPADataSerializer - add segfault protections in read/append helpers in de/serializer - rename generated header and serializer - rename IPA callback interface to IPA event interface Changes in v3: - add support for namespaces in the mojom file - note that they individually must not collide with existing namespaces (enforced by the C++ compiler) - move IPAIPCUnixSocket helper functions away from global and into the class - add SPDX and copyright to python scripts - add parser.py to wrap the mojo parser - make IPADataSerializer definition a bit cleaner with reimplemented POD manipulation functions - expand mojom documentation in core.mojom Changes in v2: - upgrade documentation - fix documentation compiling - plumb the new IPC system through the other pipelines - fix the issue where editing jinja templates wouldn't trigger recompile - enforce IPA interface rules in code generator - fix previously-untriggered code generation bugs - add de/serializers for all primitive types, string, and const ControlList - customized the RPi IPA interface - add licenses - add tests Paul Elder (9): libcamera: control_serializer: Save serialized ControlInfoMap in a cache utils: ipc: add templates for code generation for IPC mechanism utils: ipc: add generator script utils: ipc: add parser script Documentation: skip generating documentation for generated code libcamera: Add IPADataSerializer libcamera: Add IPCPipe libcamera: Add IPCPipe implementation based on unix socket ipa: Add core.mojom Documentation/Doxyfile.in | 6 +- .../libcamera/internal/control_serializer.h | 1 + .../libcamera/internal/ipa_data_serializer.h | 701 ++++++++++++++++++ include/libcamera/internal/ipc_pipe.h | 70 ++ .../libcamera/internal/ipc_pipe_unixsocket.h | 50 ++ include/libcamera/ipa/core.mojom | 208 ++++++ src/libcamera/control_serializer.cpp | 30 + src/libcamera/ipa_data_serializer.cpp | 185 +++++ src/libcamera/ipc_pipe.cpp | 211 ++++++ src/libcamera/ipc_pipe_unixsocket.cpp | 147 ++++ src/libcamera/meson.build | 3 + utils/ipc/generate.py | 29 + .../core_ipa_interface.h.tmpl | 37 + .../core_ipa_serializer.h.tmpl | 47 ++ .../definition_functions.tmpl | 53 ++ .../libcamera_templates/meson.build | 14 + .../module_ipa_interface.h.tmpl | 87 +++ .../module_ipa_proxy.cpp.tmpl | 232 ++++++ .../module_ipa_proxy.h.tmpl | 126 ++++ .../module_ipa_proxy_worker.cpp.tmpl | 224 ++++++ .../module_ipa_serializer.h.tmpl | 47 ++ .../libcamera_templates/proxy_functions.tmpl | 192 +++++ .../libcamera_templates/serializer.tmpl | 313 ++++++++ .../generators/mojom_libcamera_generator.py | 511 +++++++++++++ utils/ipc/parser.py | 20 + 25 files changed, 3543 insertions(+), 1 deletion(-) create mode 100644 include/libcamera/internal/ipa_data_serializer.h create mode 100644 include/libcamera/internal/ipc_pipe.h create mode 100644 include/libcamera/internal/ipc_pipe_unixsocket.h create mode 100644 include/libcamera/ipa/core.mojom create mode 100644 src/libcamera/ipa_data_serializer.cpp create mode 100644 src/libcamera/ipc_pipe.cpp create mode 100644 src/libcamera/ipc_pipe_unixsocket.cpp create mode 100755 utils/ipc/generate.py create mode 100644 utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/definition_functions.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/meson.build create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/proxy_functions.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/serializer.tmpl create mode 100644 utils/ipc/generators/mojom_libcamera_generator.py create mode 100755 utils/ipc/parser.py