Patch Detail
Show a patch.
GET /api/1.1/patches/9384/?format=api
{ "id": 9384, "url": "https://patchwork.libcamera.org/api/1.1/patches/9384/?format=api", "web_url": "https://patchwork.libcamera.org/patch/9384/", "project": { "id": 1, "url": "https://patchwork.libcamera.org/api/1.1/projects/1/?format=api", "name": "libcamera", "link_name": "libcamera", "list_id": "libcamera_core", "list_email": "libcamera-devel@lists.libcamera.org", "web_url": "", "scm_url": "", "webscm_url": "" }, "msgid": "<20200826110926.67192-1-paul.elder@ideasonboard.com>", "date": "2020-08-26T11:09:09", "name": "[libcamera-devel,RFC,00/17] IPA isolation example, with IPC, fully plumbed", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": false, "hash": "3973b04c7f5b848523794ed0756b86e13ed8600c", "submitter": { "id": 17, "url": "https://patchwork.libcamera.org/api/1.1/people/17/?format=api", "name": "Paul Elder", "email": "paul.elder@ideasonboard.com" }, "delegate": null, "mbox": "https://patchwork.libcamera.org/patch/9384/mbox/", "series": [], "comments": "https://patchwork.libcamera.org/api/patches/9384/comments/", "check": "pending", "checks": "https://patchwork.libcamera.org/api/patches/9384/checks/", "tags": {}, "headers": { "Return-Path": "<libcamera-devel-bounces@lists.libcamera.org>", "X-Original-To": "parsemail@patchwork.libcamera.org", "Delivered-To": "parsemail@patchwork.libcamera.org", "Received": [ "from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5A486BD87E\n\tfor <parsemail@patchwork.libcamera.org>;\n\tWed, 26 Aug 2020 11:09:51 +0000 (UTC)", "from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C852960387;\n\tWed, 26 Aug 2020 13:09:50 +0200 (CEST)", "from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 007546037B\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 26 Aug 2020 13:09:48 +0200 (CEST)", "from pyrite.rasen.tech (unknown\n\t[IPv6:2400:4051:61:600:2c71:1b79:d06d:5032])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 1D27A53C;\n\tWed, 26 Aug 2020 13:09:46 +0200 (CEST)" ], "Authentication-Results": "lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"kwBY8NT6\"; dkim-atps=neutral", "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1598440188;\n\tbh=JKGQ0aNsCCImSDkXe6TC+QKcy8nlAtgvKnfqGrQqZwU=;\n\th=From:To:Cc:Subject:Date:From;\n\tb=kwBY8NT6mwa2lNwsxKpnCsa35693QYo+2IpK+Ps/tm95XJlzTwNiziq8hY6ywe6Ir\n\tFaDtCKimD5j/9G2Vi3c/4RTYBIaiEnf6/AoYpVn8KSwGY04zKURJR3B+31E0NsysJg\n\tW2mCYVehVq1YwHRR1voca9mm1BgFGVA3DCBbD1uI=", "From": "Paul Elder <paul.elder@ideasonboard.com>", "To": "libcamera-devel@lists.libcamera.org", "Date": "Wed, 26 Aug 2020 20:09:09 +0900", "Message-Id": "<20200826110926.67192-1-paul.elder@ideasonboard.com>", "X-Mailer": "git-send-email 2.27.0", "MIME-Version": "1.0", "Subject": "[libcamera-devel] [RFC PATCH 00/17] IPA isolation example, with IPC,\n\tfully plumbed", "X-BeenThere": "libcamera-devel@lists.libcamera.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "<libcamera-devel.lists.libcamera.org>", "List-Unsubscribe": "<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>", "List-Archive": "<https://lists.libcamera.org/pipermail/libcamera-devel/>", "List-Post": "<mailto:libcamera-devel@lists.libcamera.org>", "List-Help": "<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>", "List-Subscribe": "<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>", "Content-Type": "text/plain; charset=\"us-ascii\"", "Content-Transfer-Encoding": "7bit", "Errors-To": "libcamera-devel-bounces@lists.libcamera.org", "Sender": "\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>" }, "content": "This patchset implements IPA isolation, and is fully plumbed and\nrunnable on the raspberrypi pipeline handler. This patchset is not\norganized as a proper patchset, but rather such that it is easier to\nunderstand what components there are, and how they will fit together.\n\nTo restate the problem, we have two goals:\n- to be able to run IPAs isolated in a separate process\n- the isolation must be transparent to both the pipeline handler and\n the IPA\nDuring design of the IPC mechanism, we realized that we could support\nboth custom fuctions and custom data structures, which would be a lot\nnicer than the tedious manual de/serialization that we had before with\nIPAOperationData. This patchset shows how this can be achieved.\n\n\nThe architecture of the whole thing is as follows:\n\npipeline handler -> IPAProxyRPi --thread--> IPARPi\n \\\n \\-> IPAIPC --IPA--> IPAProxyRPiWorker -> IPARPi\n\nThe pipeline handler author will write an IPA interface definition, in\nmojo format. This will declare what functions the IPA shall implement,\nwhat callbacks the IPA can signal, and what custom data structures can\nbe passed (in addition to primitives and libcamera data structures). An\nexample of this can be seen in 1/17. This IPA interface will become the\nIPAInterface specific to the pipeline handler. In the example in this\nRFC, that will be IPARPiInterface. In the diagram above, IPAProxyRPi and\nIPARPi both implement IPARPiInterface.\n\nA compiler will generate a header for the custom data structures, as\nwell as a header for the IPA interface. This can also be seen in 1/17.\n\nPatches 2/17 and 3/17 show how the pipeline author and IPA author would\nexpect to use the IPA interface and data structures defined in the mojo\nfile. I think it is quite straightforward and transparent.\n\nPatch 4/17 shows the hand-written IPADataSerializer for built-in and\nlibcamera types, as well as the IPADataSerializer for the custom data\ntypes, which will be generated. These will be used by the proxies, which\ncan be seen in 5/17.\n\nEach pipeline shall have its own proxy. This is to allow custom data\ntypes and functions. The proxy will be generated from some proxy\ntemplate, and crossed with the IPA interface as defined in the mojo\nfile. The proxy decides whether to call the IPA function directly, or\nvia IPC. Thus, all IPA modules will be wrapped in an IPAProxy. The proxy\nis also responsible for de/serializing the data, before sending it\nthrough the IPC mechanism. The IPC mechanism is designed to be\nswap-outabble, and simply exposes an async and synchronous version of\nsend(), and a callback signal. These can be seen in 6/17, along with an\nimplementation using UnixSocket.\n\n7/17 enables this whole infrastructure in meson. Note that this only\nimplements IPC for raspberrypi, so make sure to disable the other\npipeline handlers. This can be done like:\n\nmeson -Dpipelines=raspberrypi build\n\nAlso, to test isolation, the following needs to be applied:\n\n return nullptr;\n})\n\nAnd don't forget to set permission on the dma heap:\n\nsudo chmod 766 /dev/dma_heap/linux,cma \n\n8/17 is a collection of tests that I implmented during debug. If there\nis any value in them, I'll polish them and merge them. Otherwise I'll\ndiscard them.\n\nThe rest of the patches are all small tweaks in other parts of libcamera\nto get this whole IPC infrastructure to run.\n\n\nPaul Elder (17):\n IPA: IPC: raspberrypi: Add data definition and generated header\n libcamera: pipeline: raspberrypi: Use generated IPARPiInterface\n libcamera: IPA: raspberrypi: Use generated IPARPiInterface\n IPA: IPC: add IPADataSerializer\n IPA: IPC: add IPAProxyRPi and IPAProxyRPiWorker\n IPA: IPC: add IPAIPC\n meson: Enable IPC\n tests: IPC: Add various tests for the IPC framework\n libcamera: ProcessManager: make ProcessManager lifetime explicitly\n managed\n libcamera: IPAModule: Replace ipa_context with IPAInterface\n libcamera: IPAProxy: Remove stop() override\n libcamera: IPAProxy: Add isolate parameter to create()\n libcamera: PipelineHandler: Remove IPA from base class\n libcamera: IPAInterface: Remove all functions from IPAInterface\n libcamera: IPAInterface: make ipaCreate return IPAInterface\n libcamera: IPAManager: Fetch IPAProxy corresponding to pipeline\n libcamera: IPAManager: add isolation flag to proxy creation\n\n .../libcamera/internal/ipa_data_serializer.h | 816 ++++++++++++++++++\n include/libcamera/internal/ipa_ipc.h | 73 ++\n .../libcamera/internal/ipa_ipc_unixsocket.h | 116 +++\n include/libcamera/internal/ipa_module.h | 4 +-\n include/libcamera/internal/ipa_proxy.h | 24 +-\n .../internal/ipa_proxy_raspberrypi.h | 117 +++\n include/libcamera/internal/pipeline_handler.h | 1 -\n include/libcamera/internal/process.h | 27 +\n include/libcamera/ipa/ipa_interface.h | 23 +-\n include/libcamera/ipa/raspberrypi.h | 61 +-\n include/libcamera/ipa/raspberrypi.mojom | 97 +++\n .../libcamera/ipa/raspberrypi_serializer.h | 487 +++++++++++\n include/libcamera/ipa/raspberrypi_wrapper.h | 136 +++\n src/ipa/libipa/meson.build | 4 +-\n src/ipa/raspberrypi/raspberrypi.cpp | 107 ++-\n src/libcamera/camera_manager.cpp | 2 +\n src/libcamera/ipa_data_serializer.cpp | 29 +\n src/libcamera/ipa_ipc.cpp | 58 ++\n src/libcamera/ipa_ipc_unixsocket.cpp | 177 ++++\n src/libcamera/ipa_manager.cpp | 7 +-\n src/libcamera/ipa_module.cpp | 16 +-\n src/libcamera/meson.build | 5 +-\n .../pipeline/raspberrypi/raspberrypi.cpp | 125 +--\n src/libcamera/process.cpp | 46 +-\n src/libcamera/proxy/ipa_proxy_raspberrypi.cpp | 381 ++++++++\n src/libcamera/proxy/meson.build | 5 +-\n .../worker/ipa_proxy_raspberrypi_worker.cpp | 355 ++++++++\n src/libcamera/proxy/worker/meson.build | 3 +-\n test/ipa/meson.build | 4 +-\n test/ipc/meson.build | 3 +-\n test/ipc/unixsocket_fds.cpp | 284 ++++++\n .../control_serialization_ipa.cpp | 85 ++\n test/serialization/meson.build | 5 +-\n .../rpi_action_serialization.cpp | 141 +++\n .../rpi_config_serialization.cpp | 162 ++++\n 35 files changed, 3778 insertions(+), 208 deletions(-)\n create mode 100644 include/libcamera/internal/ipa_data_serializer.h\n create mode 100644 include/libcamera/internal/ipa_ipc.h\n create mode 100644 include/libcamera/internal/ipa_ipc_unixsocket.h\n create mode 100644 include/libcamera/internal/ipa_proxy_raspberrypi.h\n create mode 100644 include/libcamera/ipa/raspberrypi.mojom\n create mode 100644 include/libcamera/ipa/raspberrypi_serializer.h\n create mode 100644 include/libcamera/ipa/raspberrypi_wrapper.h\n create mode 100644 src/libcamera/ipa_data_serializer.cpp\n create mode 100644 src/libcamera/ipa_ipc.cpp\n create mode 100644 src/libcamera/ipa_ipc_unixsocket.cpp\n create mode 100644 src/libcamera/proxy/ipa_proxy_raspberrypi.cpp\n create mode 100644 src/libcamera/proxy/worker/ipa_proxy_raspberrypi_worker.cpp\n create mode 100644 test/ipc/unixsocket_fds.cpp\n create mode 100644 test/serialization/control_serialization_ipa.cpp\n create mode 100644 test/serialization/rpi_action_serialization.cpp\n create mode 100644 test/serialization/rpi_config_serialization.cpp", "diff": "diff --git a/src/libcamera/ipa_manager.cpp\nb/src/libcamera/ipa_manager.cpp\nindex 26458153..be4fd6db 100644\n--- a/src/libcamera/ipa_manager.cpp\n+++ b/src/libcamera/ipa_manager.cpp\n@@ -292,7 +292,7 @@ std::unique_ptr<IPAProxy>\nIPAManager::createIPA(PipelineHandler *pipe,\n }\n \n std::unique_ptr<IPAProxy> proxy =\n- pf->create(m, !self_->isSignatureValid(m));\n+ pf->create(m, true);\n if (!proxy->isValid()) {\n LOG(IPAManager, Error) << \"Failed to load proxy\";\n", "prefixes": [ "libcamera-devel", "RFC", "00/17" ] }