{"id":2315,"url":"https://patchwork.libcamera.org/api/patches/2315/?format=json","web_url":"https://patchwork.libcamera.org/patch/2315/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20191108205409.18845-20-laurent.pinchart@ideasonboard.com>","date":"2019-11-08T20:54:04","name":"[libcamera-devel,v2,19/24] ipa: Define a plain C API","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"06f1bec0da343869b05f2ae4e311712ca1fc0c89","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/?format=json","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/2315/mbox/","series":[{"id":568,"url":"https://patchwork.libcamera.org/api/series/568/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=568","date":"2019-11-08T20:53:45","name":"Control serialization and IPA C API","version":2,"mbox":"https://patchwork.libcamera.org/series/568/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/2315/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/2315/checks/","tags":{},"headers":{"Return-Path":"","Received":["from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2AFF561511\n\tfor ;\n\tFri, 8 Nov 2019 21:54:29 +0100 (CET)","from pendragon.bb.dnainternet.fi (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id CC38F2D1\n\tfor ;\n\tFri, 8 Nov 2019 21:54:28 +0100 (CET)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1573246468;\n\tbh=CSTBNOQS+tReWgD9u8nPG595a0IYkFh6xwVf04Fcb50=;\n\th=From:To:Subject:Date:In-Reply-To:References:From;\n\tb=lCq5PEFSlj+k6rI/45Yc3geQbFdnEQ5PfDy11/dGHrqMS8w7brYEbrCDx9BcHxjEB\n\tSOvx8MVocVrY3iGZ8R0jeJP5ROWLewfEgpb+zzSB2uZwnfEqT9ujPFkpbG+VojNBxi\n\tVgdOJ8VTvwfarH9hX7S7Fn/PEbmzr169SbEdyzaE=","From":"Laurent Pinchart ","To":"libcamera-devel@lists.libcamera.org","Date":"Fri, 8 Nov 2019 22:54:04 +0200","Message-Id":"<20191108205409.18845-20-laurent.pinchart@ideasonboard.com>","X-Mailer":"git-send-email 2.23.0","In-Reply-To":"<20191108205409.18845-1-laurent.pinchart@ideasonboard.com>","References":"<20191108205409.18845-1-laurent.pinchart@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v2 19/24] ipa: Define a plain C API","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Fri, 08 Nov 2019 20:54:29 -0000"},"content":"The C++ objects that are expected to convey data through the IPA API\nwill have associated methods that would require IPAs to link to\nlibcamera. Even though the libcamera license allows this, suppliers of\nclosed-source IPAs may have a different interpretation. To ease their\nmind and clearly separate vendor code and libcamera code, define a plain\nC IPA API. The corresponding C objects are stored in plain C structures\nor have their binary format documented, removing the need for linking to\nlibcamera code on the IPA side.\n\nThe C API adds three new C structures, ipa_context, ipa_context_ops and\nipa_callback_ops. The ipa_context_ops and ipa_callback_ops contain\nfunction pointers for all the IPA interface methods and signals,\nrespectively. The ipa_context represents a context of operation for the\nIPA, and is passed to the IPA oparations. The IPAInterface class is\nretained as it is easier to use than a plain C API for pipeline\nhandlers, and wrappers will be developed to translate between the C and\nC++ APIs.\n\nSwitching to the C API internally will be done in a second step.\n\nSigned-off-by: Laurent Pinchart \nSigned-off-by: Jacopo Mondi \n---\n include/ipa/ipa_interface.h | 77 +++++++++\n src/libcamera/ipa_interface.cpp | 295 ++++++++++++++++++++++++++++----\n 2 files changed, 340 insertions(+), 32 deletions(-)","diff":"diff --git a/include/ipa/ipa_interface.h b/include/ipa/ipa_interface.h\nindex 35dc4b4e3165..3a423e37671f 100644\n--- a/include/ipa/ipa_interface.h\n+++ b/include/ipa/ipa_interface.h\n@@ -7,6 +7,82 @@\n #ifndef __LIBCAMERA_IPA_INTERFACE_H__\n #define __LIBCAMERA_IPA_INTERFACE_H__\n \n+#include \n+#include \n+\n+#ifdef __cplusplus\n+extern \"C\" {\n+#endif\n+\n+struct ipa_context {\n+\tconst struct ipa_context_ops *ops;\n+};\n+\n+struct ipa_stream {\n+\tunsigned int id;\n+\tunsigned int pixel_format;\n+\tunsigned int width;\n+\tunsigned int height;\n+};\n+\n+struct ipa_control_info_map {\n+\tunsigned int id;\n+\tconst uint8_t *data;\n+\tsize_t size;\n+};\n+\n+struct ipa_buffer_plane {\n+\tint dmabuf;\n+\tsize_t length;\n+};\n+\n+struct ipa_buffer {\n+\tunsigned int id;\n+\tunsigned int num_planes;\n+\tstruct ipa_buffer_plane planes[3];\n+};\n+\n+struct ipa_control_list {\n+\tconst uint8_t *data;\n+\tunsigned int size;\n+};\n+\n+struct ipa_operation_data {\n+\tunsigned int operation;\n+\tconst uint32_t *data;\n+\tunsigned int num_data;\n+\tconst struct ipa_control_list *lists;\n+\tunsigned int num_lists;\n+};\n+\n+struct ipa_callback_ops {\n+\tvoid (*queue_frame_action)(void *cb_ctx, unsigned int frame,\n+\t\t\t\t struct ipa_operation_data &data);\n+};\n+\n+struct ipa_context_ops {\n+\tvoid (*destroy)(struct ipa_context *ctx);\n+\tvoid (*init)(struct ipa_context *ctx);\n+\tvoid (*register_callbacks)(struct ipa_context *ctx,\n+\t\t\t\t const struct ipa_callback_ops *callbacks,\n+\t\t\t\t void *cb_ctx);\n+\tvoid (*configure)(struct ipa_context *ctx,\n+\t\t\t const struct ipa_stream *streams,\n+\t\t\t unsigned int num_streams,\n+\t\t\t const struct ipa_control_info_map *maps,\n+\t\t\t unsigned int num_maps);\n+\tvoid (*map_buffers)(struct ipa_context *ctx,\n+\t\t\t const struct ipa_buffer *buffers,\n+\t\t\t size_t num_buffers);\n+\tvoid (*unmap_buffers)(struct ipa_context *ctx, const unsigned int *ids,\n+\t\t\t size_t num_buffers);\n+\tvoid (*process_event)(struct ipa_context *ctx,\n+\t\t\t const struct ipa_operation_data *data);\n+};\n+\n+#ifdef __cplusplus\n+}\n+\n #include \n #include \n \n@@ -53,5 +129,6 @@ public:\n };\n \n } /* namespace libcamera */\n+#endif\n \n #endif /* __LIBCAMERA_IPA_INTERFACE_H__ */\ndiff --git a/src/libcamera/ipa_interface.cpp b/src/libcamera/ipa_interface.cpp\nindex 0571b8e6bf8b..89fce0e8347f 100644\n--- a/src/libcamera/ipa_interface.cpp\n+++ b/src/libcamera/ipa_interface.cpp\n@@ -11,27 +11,254 @@\n * \\file ipa_interface.h\n * \\brief Image Processing Algorithm interface\n *\n- * Pipeline handlers communicate with IPAs through a C++ interface defined by\n- * the IPAInterface class. The class defines high-level methods and signals to\n- * configure the IPA, notify it of events, and receive actions back from the\n- * IPA.\n- *\n- * Pipeline handlers define the set of events and actions used to communicate\n- * with their IPA. These are collectively referred to as IPA operations and\n- * define the pipeline handler-specific IPA protocol. Each operation defines the\n- * data that it carries, and how the data is encoded in the IPAOperationData\n- * structure.\n- *\n- * IPAs can be isolated in a separate process. This implies that all arguments\n- * to the IPA interface may need to be transferred over IPC. The IPA interface\n- * thus uses serialisable data types only. The IPA interface defines custom data\n- * structures that mirror core libcamera structures when the latter are not\n- * suitable, such as IPAStream to carry StreamConfiguration data.\n+ * Every pipeline handler in libcamera may attach some or all of its cameras to\n+ * an Image Processing Algorithm (IPA) module. An IPA module is developed for a\n+ * specific pipeline handler and each pipeline handler may be compatible with\n+ * multiple IPA implementations, both open and closed source. To support this,\n+ * libcamera communicates with IPA modules through a standard plain C interface.\n+ *\n+ * IPA modules shall expose a public function named ipaCreate() with the\n+ * following prototype.\n+ *\n+ * \\code{.c}\n+ * struct ipa_context *ipaCreate();\n+ * \\endcode\n+ *\n+ * The ipaCreate() function creates an instance of an IPA context, which models\n+ * a context of execution for the IPA. IPA modules shall support creating one\n+ * context per camera, as required by their associated pipeline handler.\n+ *\n+ * The IPA module context operations are defined in the struct ipa_context_ops.\n+ * They model a low-level interface to configure the IPA, notify it of events,\n+ * and receive IPA actions through callbacks. An IPA module stores a pointer to\n+ * the operations corresponding to its context in the ipa_context::ops field.\n+ * That pointer is immutable for the lifetime of the context, and may differ\n+ * between different contexts created by the same IPA module.\n+ *\n+ * The IPA interface defines base data types and functions to exchange data. On\n+ * top of this, each pipeline handler is responsible for defining the set of\n+ * events and actions used to communicate with their IPA. These are collectively\n+ * referred to as IPA operations and define the pipeline handler-specific IPA\n+ * protocol. Each operation defines the data that it carries, and how that data\n+ * is encoded in the ipa_context_ops functions arguments.\n+ *\n+ * \\todo Add reference to how pipelines shall document their protocol.\n+ *\n+ * IPAs can be isolated in a separate process. This implies that arguments to\n+ * the IPA interface functions may need to be transferred over IPC. All\n+ * arguments use Plain Old Data types and are documented either in the form of C\n+ * data types, or as a textual description of byte arrays for types that can't\n+ * be expressed using C data types (such as arrays of mixed data types). IPA\n+ * modules can thus use the C API without calling into libcamera to access the\n+ * data passed to the IPA context operations.\n *\n * Due to IPC, synchronous communication between pipeline handlers and IPAs can\n * be costly. For that reason, the interface operates asynchronously. This\n- * implies that methods don't return a status, and that both methods and signals\n- * may copy their arguments.\n+ * implies that methods don't return a status, and that all methods may copy\n+ * their arguments.\n+ *\n+ * The IPAInterface class is a C++ representation of the ipa_context_ops, using\n+ * C++ data classes provided by libcamera. This is the API exposed to pipeline\n+ * handlers to communicate with IPA modules. IPA modules may use the\n+ * IPAInterface API internally if they want to benefit from the data and helper\n+ * classes offered by libcamera.\n+ */\n+\n+/**\n+ * \\struct ipa_context\n+ * \\brief IPA module context of execution\n+ *\n+ * This structure models a context of execution for an IPA module. It is\n+ * instantiated by the IPA module ipaCreate() function. IPA modules allocate\n+ * context instances in an implementation-defined way, contexts shall thus be\n+ * destroyed using the ipa_operation::destroy function only.\n+ *\n+ * The ipa_context structure provides a pointer to the IPA context operations.\n+ * It shall otherwise be treated as a constant black-box cookie and passed\n+ * unmodified to the functions defined in struct ipa_context_ops.\n+ *\n+ * IPA modules are expected to extend struct ipa_context by inheriting from it,\n+ * either through structure embedding to model inheritance in plain C, or\n+ * through C++ class inheritance. A simple example of the latter is available\n+ * in the IPAContextWrapper class implementation.\n+ *\n+ * \\var ipa_context::ops\n+ * \\brief The IPA context operations\n+ */\n+\n+/**\n+ * \\struct ipa_stream\n+ * \\brief Stream information for the IPA context operations\n+ *\n+ * \\var ipa_stream::id\n+ * \\brief Identifier for the stream, defined by the IPA protocol\n+ *\n+ * \\var ipa_stream::pixel_format\n+ * \\brief The stream pixel format, as defined by the PixelFormat class\n+ *\n+ * \\var ipa_stream::width\n+ * \\brief The stream width in pixels\n+ *\n+ * \\var ipa_stream::height\n+ * \\brief The stream height in pixels\n+ */\n+\n+/**\n+ * \\struct ipa_control_info_map\n+ * \\brief ControlInfoMap description for the IPA context operations\n+ *\n+ * \\var ipa_control_info_map::id\n+ * \\brief Identifier for the ControlInfoMap, defined by the IPA protocol\n+ *\n+ * \\var ipa_control_info_map::data\n+ * \\brief Pointer to a control packet for the ControlInfoMap\n+ * \\sa ipa_controls.h\n+ *\n+ * \\var ipa_control_info_map::size\n+ * \\brief The size of the control packet in bytes\n+ */\n+\n+/**\n+ * \\struct ipa_buffer_plane\n+ * \\brief A plane for an ipa_buffer\n+ *\n+ * \\var ipa_buffer_plane::dmabuf\n+ * \\brief The dmabuf file descriptor for the plane (-1 for unused planes)\n+ *\n+ * \\var ipa_buffer_plane::length\n+ * \\brief The plane length in bytes (0 for unused planes)\n+ */\n+\n+/**\n+ * \\struct ipa_buffer\n+ * \\brief Buffer information for the IPA context operations\n+ *\n+ * \\var ipa_buffer::id\n+ * \\brief The buffer unique ID (see \\ref libcamera::IPABuffer::id)\n+ *\n+ * \\var ipa_buffer::num_planes\n+ * \\brief The number of used planes in the ipa_buffer::planes array\n+ *\n+ * \\var ipa_buffer::planes\n+ * \\brief The buffer planes (up to 3)\n+ */\n+\n+/**\n+ * \\struct ipa_control_list\n+ * \\brief ControlList description for the IPA context operations\n+ *\n+ * \\var ipa_control_list::data\n+ * \\brief Pointer to a control packet for the ControlList\n+ * \\sa ipa_controls.h\n+ *\n+ * \\var ipa_control_list::size\n+ * \\brief The size of the control packet in bytes\n+ */\n+\n+/**\n+ * \\struct ipa_operation_data\n+ * \\brief IPA operation data for the IPA context operations\n+ * \\sa libcamera::IPAOperationData\n+ *\n+ * \\var ipa_operation_data::operation\n+ * \\brief IPA protocol operation\n+ *\n+ * \\var ipa_operation_data::data\n+ * \\brief Pointer to the operation data array\n+ *\n+ * \\var ipa_operation_data::num_data\n+ * \\brief Number of entries in the ipa_operation_data::data array\n+ *\n+ * \\var ipa_operation_data::lists\n+ * \\brief Pointer to an array of ipa_control_list\n+ *\n+ * \\var ipa_operation_data::num_lists\n+ * \\brief Number of entries in the ipa_control_list array\n+ */\n+\n+/**\n+ * \\struct ipa_callback_ops\n+ * \\brief IPA context operations as a set of function pointers\n+ */\n+\n+/**\n+ * \\var ipa_callback_ops::queue_frame_action\n+ * \\brief Queue an action associated with a frame to the pipeline handler\n+ * \\param[in] cb_ctx The callback context registered with\n+ * ipa_context_ops::register_callbacks\n+ * \\param[in] frame The frame number\n+ *\n+ * \\sa libcamera::IPAInterface::queueFrameAction\n+ */\n+\n+/**\n+ * \\struct ipa_context_ops\n+ * \\brief IPA context operations as a set of function pointers\n+ *\n+ * To allow for isolation of IPA modules in separate processes, the functions\n+ * defined in the ipa_context_ops structure return only data related to the\n+ * libcamera side of the operations. In particular, error related to the\n+ * libcamera side of the IPC may be returned. Data returned by the IPA,\n+ * including status information, shall be provided through callbacks from the\n+ * IPA to libcamera.\n+ */\n+\n+/**\n+ * \\var ipa_context_ops::destroy\n+ * \\brief Destroy the IPA context created by the module's ipaCreate() function\n+ * \\param[in] ctx The IPA context\n+ */\n+\n+/**\n+ * \\var ipa_context_ops::init\n+ * \\brief Initialise the IPA context\n+ * \\param[in] ctx The IPA context\n+ *\n+ * \\sa libcamera::IPAInterface::init()\n+ */\n+\n+/**\n+ * \\var ipa_context_ops::register_callbacks\n+ * \\brief Register callback operation from the IPA to the pipeline handler\n+ * \\param[in] ctx The IPA context\n+ * \\param[in] callback The IPA callback operations\n+ * \\param[in] cb_ctx The callback context, passed to all callback operations\n+ */\n+\n+/**\n+ * \\var ipa_context_ops::configure\n+ * \\brief Configure the IPA stream and sensor settings\n+ * \\param[in] ctx The IPA context\n+ *\n+ * \\sa libcamera::IPAInterface::configure()\n+ */\n+\n+/**\n+ * \\var ipa_context_ops::map_buffers\n+ * \\brief Map buffers shared between the pipeline handler and the IPA\n+ * \\param[in] ctx The IPA context\n+ * \\param[in] buffers The buffers to map\n+ * \\param[in] num_buffers The number of entries in the \\a buffers array\n+ *\n+ * \\sa libcamera::IPAInterface::mapBuffers()\n+ */\n+\n+/**\n+ * \\var ipa_context_ops::unmap_buffers\n+ * \\brief Unmap buffers shared by the pipeline to the IPA\n+ * \\param[in] ctx The IPA context\n+ * \\param[in] ids The IDs of the buffers to unmap\n+ * \\param[in] num_buffers The number of entries in the \\a ids array\n+ *\n+ * \\sa libcamera::IPAInterface::unmapBuffers()\n+ */\n+\n+/**\n+ * \\var ipa_context_ops::process_event\n+ * \\brief Process an event from the pipeline handler\n+ * \\param[in] ctx The IPA context\n+ *\n+ * \\sa libcamera::IPAInterface::processEvent()\n */\n \n namespace libcamera {\n@@ -122,26 +349,30 @@ namespace libcamera {\n \n /**\n * \\class IPAInterface\n- * \\brief Interface for IPA implementation\n+ * \\brief C++ Interface for IPA implementation\n *\n- * Every pipeline handler in libcamera may attach all or some of its cameras to\n- * an Image Processing Algorithm (IPA) module. An IPA module is developed for a\n- * specific pipeline handler and each pipeline handler may have multiple\n- * compatible IPA implementations, both open and closed source.\n+ * This pure virtual class defines a C++ API corresponding to the ipa_context,\n+ * ipa_context_ops and ipa_callback_ops API. It is used by pipeline handlers to\n+ * interact with IPA modules, and may be used internally in IPA modules if\n+ * desired to benefit from the data and helper classes provided by libcamera.\n+ *\n+ * Functions defined in the ipa_context_ops structure are mapped to IPAInterface\n+ * methods, while functions defined in the ipa_callback_ops are mapped to\n+ * IPAInterface signals. As with the C API, the IPA C++ interface uses\n+ * serializable data types only. It reuses structures defined by the C API, or\n+ * defines corresponding classes using C++ containers when required.\n *\n- * To allow for multiple IPA modules for the same pipeline handler, a standard\n- * interface for the pipeline handler and IPA communication is needed.\n- * IPAInterface is this interface.\n+ * Due to process isolation all arguments to the IPAInterface methods and\n+ * signals may need to be transferred over IPC. The class thus uses serializable\n+ * data types only. The IPA C++ interface defines custom data structures that\n+ * mirror core libcamera structures when the latter are not suitable, such as\n+ * IPAStream to carry StreamConfiguration data.\n *\n- * The interface defines base data types and methods to exchange data. On top of\n- * this, each pipeline handler is responsible for defining specific operations\n- * that make up its IPA protocol, shared by all IPA modules compatible with the\n- * pipeline handler.\n+ * As for the functions defined in struct ipa_context_ops, the methods defined\n+ * by this class shall not return data from the IPA.\n *\n * The pipeline handler shall use the IPAManager to locate a compatible\n * IPAInterface. The interface may then be used to interact with the IPA module.\n- *\n- * \\todo Add reference to how pipelines shall document their protocol.\n */\n \n /**\n","prefixes":["libcamera-devel","v2","19/24"]}