From patchwork Fri Nov 6 10:37:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 10384 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E6DCCBDB89 for ; Fri, 6 Nov 2020 10:38:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B0DCD62D45; Fri, 6 Nov 2020 11:38:39 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BRUfp16Y"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EA0E962D22 for ; Fri, 6 Nov 2020 11:38:37 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 77E91A19; Fri, 6 Nov 2020 11:38:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1604659116; bh=YE8b4weRmvrJ45nSwtz1mzfWzhGU7R4XBOtzXbdLhjU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BRUfp16YxDf/ufSz3ynMR8b+lxUGa2Mp1SehpS7rismv4Oihm32xYXWmPyRBMoear Y51K2yk8E5m3LxGMuimTxjrtoDc9+62lV4kGGN08Fg1ZRm4mQ8vnjBFz/DT3jOPELJ D3XOzHkU1sO+8bi1eSEuzEYZpjioZHaBjVLSbX20= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 6 Nov 2020 19:37:06 +0900 Message-Id: <20201106103707.49660-37-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201106103707.49660-1-paul.elder@ideasonboard.com> References: <20201106103707.49660-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 36/37] Documentation: Add IPA writers guide X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a guide about writing IPAs. Signed-off-by: Paul Elder --- So far it only covers stuff about the IPA interface... New in v4 --- Documentation/guides/ipa.rst | 112 +++++++++++++++++++++++++++++++++++ Documentation/index.rst | 1 + Documentation/meson.build | 1 + 3 files changed, 114 insertions(+) create mode 100644 Documentation/guides/ipa.rst diff --git a/Documentation/guides/ipa.rst b/Documentation/guides/ipa.rst new file mode 100644 index 00000000..4b270eb7 --- /dev/null +++ b/Documentation/guides/ipa.rst @@ -0,0 +1,112 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +IPA Writers Guide +================= + +IPA are Image Processing Algorithm modules. They provide functionality that +the pipeline handler can use for image processing. + +The IPA interface and protocol +------------------------------ + +The IPA interface defines the interface between the pipeline handler and the +IPA. Specifically, it defines the functions that the IPA exposes that the +pipeline handler can call, and the Signals that the pipeline handler can +connect to to receive data from the IPA asyncrhonously. In addition, it +contains any custom data structures that the pipeline handler and IPA may +pass to each other. + +The IPA protocol refers to the agreement between the pipeline handler and the +IPA regarding the expected response(s) from the IPA for given calls to the IPA. +This protocol doesn't need to be declared anywhere in code, but it would be +useful to document it, as there may be different IPAs for some pipeline handler. + +The IPA interface must be defined in a mojom file. The interface includes the +functions that the pipeline handler can call from the IPA, Signals in the +pipeline handler that the IPA can emit, and any data structures that are to +be passed between the pipeline handler and the IPA. All IPA of a given pipeline +handler use the same IPA interface. The IPA interface definition is thus +likely to be written by the pipeline handler author, based on how they imagine +the pipeline handler will interact with the IPA. + +Any custom structs that the IPA interfaces use must be defined in in a file +named {pipeline_name}.mojom under include/libcamera/ipa/ using the mojo +mojo interface definition language. + +It is recommended to use namespacing, to avoid potential collisions with +libcamera types. Use mojom's module directive for this. + +The Main IPA interface +---------------------- + +The IPA interface is split in two parts, the Main IPA interface, which +describes the functions that the pipeline handler can call from the IPA, +and the Event IPA interface, which describes the Signals in the pipeline +handler that the IPA can emit. Both must be defined. This section focuses +on the Main IPA interface. + +The main interface must be named as IPA{pipeline_name}Interface. + +At minimum, the following three functions must be present (and implemented): +- init(IPASettings settings) => (int32 ret); +- start() => (int32 ret); +- stop(); + +All three of these functions are synchronous. + +All input parameters will become const references, except for arithmetic types, +which will be passed by value. Output parameters will become pointers, unless +there is only one primitive output parameter, in which case it will become a +a regular return value. + +const is not allowed inside of arrays and maps. mojo arrays will become C++ +std::vector<>. + +By default, all methods defined in the main interface are synchronous. This +means that in the case of IPC (ie. isolated IPA), the function call will not +return until the return value or output parameters are ready. To specify an +asynchronous function, the [async] attribute can be used. Asynchronous +methods must not have any return value or output parameters, since in the +case of IPC the call needs to return immediately. + +It is also possible that the IPA will not be run in isolation. In this case, +the IPA thread will not exist until start() is called. This means that in the +case of no isolation, asynchronous calls cannot be made before start(). Since +the IPA interface must be the same regardless of isolation, the same +restriction applies to the case of isolation, and any function that will be +called before start() must be synchronous. + +In addition, any call made after start() and before stop() must be +asynchronous. The motivation for this is to avoid damaging real time +performance of the pipeline handler. If the pipeline handler wants some data +from the IPA, the IPA should return the data asynchronously via an event +(see "The Event IPA interface"). + +In addition the following must be defined in {pipeline_name}.h in the +libcamera namespace: + +static ControlInfoMap {pipeline_name}::controls; + +It may be empty. This will be the default ControlInfoMap used when +serializing and deserializing any ControlLists if it is not specified +in the ControlList. + + +The Event IPA interface +----------------------- + +The event IPA interface describes the Signals in the pipeline handler that the +IPA can emit. It must be defined. If there are no event functions, then it may +be empty. These emissions are meant to notify the pipeline handler of some +event, such as reqeust data is ready, and *must not* be used to drive the +camera pipeline from the IPA. + +The event interface must be named as IPA{pipeline_name}EventInterface. + +Methods defined in the event interface are implictly asynchronous. +They thus can't return any value. Specifying the [async] tag is not +necessary. + +Methods defined in the event interface will become Signals in the IPA +interface. The IPA can emit signals, while the pipeline handler can connect +slots to them. diff --git a/Documentation/index.rst b/Documentation/index.rst index ff697d4f..8bc8922e 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -16,4 +16,5 @@ Developer Guide Application Writer's Guide Pipeline Handler Writer's Guide + IPA Writer's guide Tracing guide diff --git a/Documentation/meson.build b/Documentation/meson.build index 59107151..17f3b9d7 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -52,6 +52,7 @@ if sphinx.found() 'docs.rst', 'index.rst', 'guides/introduction.rst', + 'guides/ipa.rst', 'guides/application-developer.rst', 'guides/pipeline-handler.rst', 'guides/tracing.rst',