From patchwork Sun Oct 13 00:13:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 2177 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 318EA60BE5 for ; Sun, 13 Oct 2019 02:13:49 +0200 (CEST) Received: from pendragon.bb.dnainternet.fi (dfj612yhrgyx302h3jwwy-3.rev.dnainternet.fi [IPv6:2001:14ba:21f5:5b00:ce28:277f:58d7:3ca4]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B099C71D for ; Sun, 13 Oct 2019 02:13:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1570925628; bh=iGAXYSGQz/pHbqbijFxRC3fjxxbT3TQFthgEcNAJP9o=; h=From:To:Subject:Date:In-Reply-To:References:From; b=AvwUT6jekB1yTE0ntviHMVsLXJs8D/GU6J3STaGuOHISHgKzJyL0TvYzY9cT3ym0U zOKngzYaQLz73Esg8NZGwkQXq6Ut7x4EAwfkpCpc85VwTP81omQVJyGl5q2sgmU7b+ 1XaO71jiVqMPGt2m82hZLkMcmlnBazsS3bvksndo= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Sun, 13 Oct 2019 03:13:37 +0300 Message-Id: <20191013001340.14266-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191013001340.14266-1-laurent.pinchart@ideasonboard.com> References: <20191013001340.14266-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH/RFC v2 1/4] libcamera: buffer: Add const accessor to Buffer planes X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Oct 2019 00:13:49 -0000 In order to inspect planes of a const Buffer, add a const accessor. Signed-off-by: Laurent Pinchart Reviewed-by: Niklas Söderlund --- include/libcamera/buffer.h | 1 + src/libcamera/buffer.cpp | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/include/libcamera/buffer.h b/include/libcamera/buffer.h index 7b657509ab5f..7a070205f0b5 100644 --- a/include/libcamera/buffer.h +++ b/include/libcamera/buffer.h @@ -43,6 +43,7 @@ private: class BufferMemory final { public: + const std::vector &planes() const { return planes_; } std::vector &planes() { return planes_; } private: diff --git a/src/libcamera/buffer.cpp b/src/libcamera/buffer.cpp index 4407201bd81c..960eeb8f7d19 100644 --- a/src/libcamera/buffer.cpp +++ b/src/libcamera/buffer.cpp @@ -181,6 +181,12 @@ void *Plane::mem() * image format is multi-planar. */ +/** + * \fn BufferMemory::planes() const + * \brief Retrieve the planes within the buffer + * \return A const reference to a vector holding all Planes within the buffer + */ + /** * \fn BufferMemory::planes() * \brief Retrieve the planes within the buffer From patchwork Sun Oct 13 00:13:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 2178 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8AB5460BE5 for ; Sun, 13 Oct 2019 02:13:49 +0200 (CEST) Received: from pendragon.bb.dnainternet.fi (dfj612yhrgyx302h3jwwy-3.rev.dnainternet.fi [IPv6:2001:14ba:21f5:5b00:ce28:277f:58d7:3ca4]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1065133A for ; Sun, 13 Oct 2019 02:13:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1570925629; bh=yVOqlYdV1uXW2d1XbXyLIsZugVM3FUNiWhsdcssGcVI=; h=From:To:Subject:Date:In-Reply-To:References:From; b=mfK5hkZZmRbc4SugNsbeGp1WeGPkGmLWXm4Dmoj8+mAjJTawrIN+m2Xt1sjdRUk7o R5mUIQ4kjxiwDj+/nMeOPOtsNcxzbRwBWbsAzfBdhsLGSoqZ49ncDxmLx+Fky7jgAz Rm1qXcP6ewpOWD2HP8DFcki/lJnmQm6G0C1Vzb3Q= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Sun, 13 Oct 2019 03:13:38 +0300 Message-Id: <20191013001340.14266-3-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191013001340.14266-1-laurent.pinchart@ideasonboard.com> References: <20191013001340.14266-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH/RFC v2 2/4] ipa: Define a plain C API X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Oct 2019 00:13:49 -0000 The C++ objects that are expected to convey data through the IPA API will have associated methods that would require IPAs to link to libcamera. Even though the libcamera license allows this, suppliers of closed-source IPAs may have a different interpretation. To ease their mind and clearly separate vendor code and libcamera code, define a plain C IPA API. The corresponding C objects are stored in plain C structures or have their binary format documented, removing the need for linking to libcamera code on the IPA side. The C API adds three new C structures, ipa_context, ipa_context_ops and ipa_callback_ops. The ipa_context_ops and ipa_callback_ops contain function pointers for all the IPA interface methods and signals, respectively. The ipa_context represents a context of operation for the IPA, and is passed to the IPA oparations. The IPAInterface class is retained as it is easier to use than a plain C API for pipeline handlers, and wrappers will be developed to translate between the C and C++ APIs. Switching to the C API internally will be done in a second step. Signed-off-by: Laurent Pinchart --- include/ipa/ipa_interface.h | 44 ++++++ src/libcamera/ipa_interface.cpp | 228 +++++++++++++++++++++++++++----- 2 files changed, 242 insertions(+), 30 deletions(-) diff --git a/include/ipa/ipa_interface.h b/include/ipa/ipa_interface.h index dfb1bcbee516..acb491e3958c 100644 --- a/include/ipa/ipa_interface.h +++ b/include/ipa/ipa_interface.h @@ -7,6 +7,49 @@ #ifndef __LIBCAMERA_IPA_INTERFACE_H__ #define __LIBCAMERA_IPA_INTERFACE_H__ +#include + +#ifdef __cplusplus +extern "C" { +#endif + +struct ipa_context { + const struct ipa_context_ops *ops; +}; + +struct ipa_buffer_plane { + int dmabuf; + size_t length; +}; + +struct ipa_buffer { + unsigned int id; + unsigned int num_planes; + struct ipa_buffer_plane planes[3]; +}; + +struct ipa_callback_ops { + void (*queue_frame_action)(void *cb_ctx, unsigned int frame); +}; + +struct ipa_context_ops { + void (*destroy)(struct ipa_context *ctx); + void (*init)(struct ipa_context *ctx); + void (*register_callbacks)(struct ipa_context *ctx, + const struct ipa_callback_ops *callbacks, + void *cb_ctx); + void (*configure)(struct ipa_context *ctx); + void (*map_buffers)(struct ipa_context *ctx, + const struct ipa_buffer *buffers, + size_t num_buffers); + void (*unmap_buffers)(struct ipa_context *ctx, const unsigned int *ids, + size_t num_buffers); + void (*process_event)(struct ipa_context *ctx); +}; + +#ifdef __cplusplus +} + #include #include @@ -53,5 +96,6 @@ public: }; } /* namespace libcamera */ +#endif #endif /* __LIBCAMERA_IPA_INTERFACE_H__ */ diff --git a/src/libcamera/ipa_interface.cpp b/src/libcamera/ipa_interface.cpp index 0571b8e6bf8b..1df430d721f0 100644 --- a/src/libcamera/ipa_interface.cpp +++ b/src/libcamera/ipa_interface.cpp @@ -11,27 +11,191 @@ * \file ipa_interface.h * \brief Image Processing Algorithm interface * - * Pipeline handlers communicate with IPAs through a C++ interface defined by - * the IPAInterface class. The class defines high-level methods and signals to - * configure the IPA, notify it of events, and receive actions back from the - * IPA. + * Every pipeline handler in libcamera may attach all or some of its cameras to + * an Image Processing Algorithm (IPA) module. An IPA module is developed for a + * specific pipeline handler and each pipeline handler may be compatible with + * multiple IPA implementations, both open and closed source. To support this, + * libcamera communicates with IPA modules through a standard plain C interface. * - * Pipeline handlers define the set of events and actions used to communicate - * with their IPA. These are collectively referred to as IPA operations and - * define the pipeline handler-specific IPA protocol. Each operation defines the - * data that it carries, and how the data is encoded in the IPAOperationData - * structure. + * IPA modules shall expose a public function named ipaCreate() with the + * following prototype. * - * IPAs can be isolated in a separate process. This implies that all arguments - * to the IPA interface may need to be transferred over IPC. The IPA interface - * thus uses serialisable data types only. The IPA interface defines custom data - * structures that mirror core libcamera structures when the latter are not - * suitable, such as IPAStream to carry StreamConfiguration data. + * \code{.c} + * struct ipa_context *ipaCreate(); + * \endcode + * + * The ipaCreate() function creates an instance of an IPA context, which models + * a context of execution for the IPA. IPA modules shall support creating one + * context per camera, as required by their associated pipeline handler. + * + * The IPA module context operations are defined in the struct ipa_context_ops. + * They model a low-level interface to configure the IPA, notify it of events, + * and receive IPA actions through callbacks. An IPA module stores a pointer to + * the operations corresponding to its context in the ipa_context::ops field. + * That pointer is immutable for the lifetime of the context, and may differ + * between different contexts created by the same IPA module. + * + * The IPA interface defines base data types and functions to exchange data. On + * top of this, each pipeline handler is responsible for defining the set of + * events and actions used to communicate with their IPA. These are collectively + * referred to as IPA operations and define the pipeline handler-specific IPA + * protocol. Each operation defines the data that it carries, and how that data + * is encoded in the ipa_context_ops functions arguments. + * + * \todo Add reference to how pipelines shall document their protocol. + * + * IPAs can be isolated in a separate process. This implies that arguments to + * the IPA interface functions may need to be transferred over IPC. All + * arguments use Plain Old Data types and are documented either in the form of C + * data types, or as a textual description of byte arrays for types that can't + * be expressed using C data types (such as arrays of mixed data types). IPA + * modules can thus use the C API without calling into libcamera to access the + * data passed to the IPA context operations. * * Due to IPC, synchronous communication between pipeline handlers and IPAs can * be costly. For that reason, the interface operates asynchronously. This - * implies that methods don't return a status, and that both methods and signals - * may copy their arguments. + * implies that methods don't return a status, and that all methods may copy + * their arguments. + * + * The IPAInterface class is a C++ representation of the ipa_context_ops, using + * C++ data classes provided by libcamera. This is the API exposed to pipeline + * handlers to communicate with IPA modules. IPA modules may use the + * IPAInterface API internally if they want to benefit from the data and helper + * classes offered by libcamera. + */ + +/** + * \struct ipa_context + * \brief IPA module context of execution + * + * This structure models a context of execution for an IPA module. It is + * instantiated by the IPA module ipaCreate() function. IPA modules allocate + * context instances in an implementation-defined way, contexts shall thus be + * destroyed using the ipa_operation::destroy function only. + * + * The ipa_context structure provides a pointer to the IPA context operations. + * It shall otherwise be treated as a constant black-box cookie and passed + * unmodified to the functions defined in struct ipa_context_ops. + * + * IPA modules are expected to extend struct ipa_context by inheriting from it, + * either through structure embedding to model inheritance in plain C, or + * through C++ class inheritance. A simple example of the latter is available + * in the IPAContextWrapper class implementation. + * + * \var ipa_context::ops + * \brief The IPA context operations + */ + +/** + * \struct ipa_buffer_plane + * \brief A plane for an ipa_buffer + * + * \var ipa_buffer_plane::dmabuf + * \brief The dmabuf file descriptor for the plane + * + * \var ipa_buffer_plane::length + * \brief The plane length in bytes + */ + +/** + * \struct ipa_buffer + * \brief Buffer information for the IPA interface + * + * \sa libcamera::IPABuffer + * + * \var ipa_buffer::id + * \brief The buffer unique ID + * + * \var ipa_buffer::num_planes + * \brief The number of used planes in the ipa_buffer::planes array + * + * \var ipa_buffer::planes + * \brief The buffer planes (up to 3) + */ + +/** + * \struct ipa_callback_ops + * \brief IPA context operations as a set of function pointers + */ + +/** + * \var ipa_callback_ops::queue_frame_action + * \brief Queue an action associated with a frame to the pipeline handler + * \param[in] cb_ctx The callback context registered with + * ipa_context_ops::register_callbacks + * \param[in] frame The frame number + * + * \sa libcamera::IPAInterface::queueFrameAction + */ + +/** + * \struct ipa_context_ops + * \brief IPA context operations as a set of function pointers + * + * To allow for isolation of IPA modules in separate processes, the functions + * defined in the ipa_context_ops structure return only data related to the + * libcamera side of the operations. In particular, error related to the + * libcamera side of the IPC may be returned. Data returned by the IPA, + * including status information, shall be provided through callbacks from the + * IPA to libcamera. + */ + +/** + * \var ipa_context_ops::destroy + * \brief Destroy the IPA context created by the module's ipaCreate() function + * \param[in] ctx The IPA context + */ + +/** + * \var ipa_context_ops::init + * \brief Initialise the IPA context + * \param[in] ctx The IPA context + * + * \sa libcamera::IPAInterface::init() + */ + +/** + * \var ipa_context_ops::register_callbacks + * \brief Register callback operation from the IPA to the pipeline handler + * \param[in] ctx The IPA context + * \param[in] callback The IPA callback operations + * \param[in] cb_ctx The callback context, passed to all callback operations + */ + +/** + * \var ipa_context_ops::configure + * \brief Configure the IPA stream and sensor settings + * \param[in] ctx The IPA context + * + * \sa libcamera::IPAInterface::configure() + */ + +/** + * \var ipa_context_ops::map_buffers + * \brief Map buffers shared between the pipeline handler and the IPA + * \param[in] ctx The IPA context + * \param[in] buffers The buffers to map + * \param[in] num_buffers The number of entries in the \a buffers array + * + * \sa libcamera::IPAInterface::mapBuffers() + */ + +/** + * \var ipa_context_ops::unmap_buffers + * \brief Unmap buffers shared by the pipeline to the IPA + * \param[in] ctx The IPA context + * \param[in] ids The IDs of the buffers to unmap + * \param[in] num_buffers The number of entries in the \a ids array + * + * \sa libcamera::IPAInterface::unmapBuffers() + */ + +/** + * \var ipa_context_ops::process_event + * \brief Process an event from the pipeline handler + * \param[in] ctx The IPA context + * + * \sa libcamera::IPAInterface::processEvent() */ namespace libcamera { @@ -122,26 +286,30 @@ namespace libcamera { /** * \class IPAInterface - * \brief Interface for IPA implementation + * \brief C++ Interface for IPA implementation * - * Every pipeline handler in libcamera may attach all or some of its cameras to - * an Image Processing Algorithm (IPA) module. An IPA module is developed for a - * specific pipeline handler and each pipeline handler may have multiple - * compatible IPA implementations, both open and closed source. + * This pure virtual class defines a C++ API corresponding to the ipa_context, + * ipa_context_ops and ipa_callback_ops API. It is used by pipeline handlers to + * interact with IPA modules, and may be used internally in IPA modules if + * desired to benefit from the data and helper classes provided by libcamera. * - * To allow for multiple IPA modules for the same pipeline handler, a standard - * interface for the pipeline handler and IPA communication is needed. - * IPAInterface is this interface. + * Functions defined in the ipa_context_ops structure are mapped to IPAInterface + * methods, while functions defined in the ipa_callback_ops are mapped to + * IPAInterface signals. As with the C API, the IPA C++ interface uses + * serialisable data types only. It reuses structures defined by the C API, or + * defines corresponding classes using C++ containers when required. * - * The interface defines base data types and methods to exchange data. On top of - * this, each pipeline handler is responsible for defining specific operations - * that make up its IPA protocol, shared by all IPA modules compatible with the - * pipeline handler. + * Due to process isolation all arguments to the IPAInterface methods and + * signals may need to be transferred over IPC. The class thus uses serialisable + * data types only. The IPA C++ interface defines custom data structures that + * mirror core libcamera structures when the latter are not suitable, such as + * IPAStream to carry StreamConfiguration data. + * + * As for the functions defined in struct ipa_context_ops, the methods defined + * by this class shall not return data from the IPA. * * The pipeline handler shall use the IPAManager to locate a compatible * IPAInterface. The interface may then be used to interact with the IPA module. - * - * \todo Add reference to how pipelines shall document their protocol. */ /** From patchwork Sun Oct 13 00:13:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 2179 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DAB3360BE5 for ; Sun, 13 Oct 2019 02:13:49 +0200 (CEST) Received: from pendragon.bb.dnainternet.fi (dfj612yhrgyx302h3jwwy-3.rev.dnainternet.fi [IPv6:2001:14ba:21f5:5b00:ce28:277f:58d7:3ca4]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7935371D for ; Sun, 13 Oct 2019 02:13:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1570925629; bh=Ifvi8HgbVuwfFdOQ3quLeuYuHBQuz7w47ZrqWBjcHDQ=; h=From:To:Subject:Date:In-Reply-To:References:From; b=EQ/9wlLrcY8U0GytsbjVqz+g5gy1rtYQcIiqineoGuZXiyCNSB1l0BnvelyQIwOA/ P64XtjZUsPhXgUCdZQpUDv9ET+4aTIfmQJPyxJSSEZpdDe/5Tn1GjtnOX3F4Uz8U/z epYPVwI4xnU57f/cdZ6rYTQiYmBppYfqsrVw8fDQ= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Sun, 13 Oct 2019 03:13:39 +0300 Message-Id: <20191013001340.14266-4-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191013001340.14266-1-laurent.pinchart@ideasonboard.com> References: <20191013001340.14266-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH/RFC v2 3/4] ipa: Switch to the plain C API X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Oct 2019 00:13:50 -0000 From: Jacopo Mondi Switch IPA communication to the plain C API. As the IPAInterface class is easier to use for pipeline handlers than a plain C API, retain it and an IPAContextWrapper that translate between the C++ and the C APIs. On the IPA module side usage of IPAInterface may be desired for IPAs implemented in C++ that want to link to libcamera. For those IPAs, a new IPAInterfaceWrapper helper class is introduce to wrap the IPAInterface implemented internally by the IPA module into an ipa_context, ipa_context_ops and ipa_callback_ops. Signed-off-by: Jacopo Mondi Signed-off-by: Laurent Pinchart --- Documentation/Doxyfile.in | 1 + Documentation/meson.build | 2 + src/ipa/ipa_vimc.cpp | 6 +- src/ipa/libipa/ipa_interface_wrapper.cpp | 151 ++++++++++++++++++ src/ipa/libipa/ipa_interface_wrapper.h | 45 ++++++ src/ipa/libipa/meson.build | 11 ++ src/ipa/meson.build | 3 + src/libcamera/include/ipa_context_wrapper.h | 38 +++++ src/libcamera/include/ipa_module.h | 5 +- src/libcamera/include/meson.build | 1 + src/libcamera/ipa_context_wrapper.cpp | 131 +++++++++++++++ src/libcamera/ipa_manager.cpp | 67 +++++++- src/libcamera/ipa_module.cpp | 23 +-- src/libcamera/meson.build | 1 + .../proxy/worker/ipa_proxy_linux_worker.cpp | 8 +- 15 files changed, 473 insertions(+), 20 deletions(-) create mode 100644 src/ipa/libipa/ipa_interface_wrapper.cpp create mode 100644 src/ipa/libipa/ipa_interface_wrapper.h create mode 100644 src/ipa/libipa/meson.build create mode 100644 src/libcamera/include/ipa_context_wrapper.h create mode 100644 src/libcamera/ipa_context_wrapper.cpp diff --git a/Documentation/Doxyfile.in b/Documentation/Doxyfile.in index 5237cf60854f..018fd668a5a8 100644 --- a/Documentation/Doxyfile.in +++ b/Documentation/Doxyfile.in @@ -793,6 +793,7 @@ WARN_LOGFILE = INPUT = "@TOP_SRCDIR@/include/ipa" \ "@TOP_SRCDIR@/include/libcamera" \ + "@TOP_SRCDIR@/src/ipa/libipa" \ "@TOP_SRCDIR@/src/libcamera" \ "@TOP_BUILDDIR@/include/libcamera" \ "@TOP_BUILDDIR@/src/libcamera" diff --git a/Documentation/meson.build b/Documentation/meson.build index 4ff3fbeb0674..9136506f5d9c 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -24,6 +24,8 @@ if doxygen.found() libcamera_ipa_api, libcamera_headers, libcamera_sources, + libipa_headers, + libipa_sources, ], output : 'api-html', command : [doxygen, doxyfile], diff --git a/src/ipa/ipa_vimc.cpp b/src/ipa/ipa_vimc.cpp index 620dc25f456e..f43fc6ab6cd1 100644 --- a/src/ipa/ipa_vimc.cpp +++ b/src/ipa/ipa_vimc.cpp @@ -17,6 +17,8 @@ #include #include +#include "libipa/ipa_interface_wrapper.h" + #include "log.h" namespace libcamera { @@ -108,9 +110,9 @@ const struct IPAModuleInfo ipaModuleInfo = { LICENSE, }; -IPAInterface *ipaCreate() +struct ipa_context *ipaCreate() { - return new IPAVimc(); + return new IPAInterfaceWrapper(new IPAVimc()); } }; diff --git a/src/ipa/libipa/ipa_interface_wrapper.cpp b/src/ipa/libipa/ipa_interface_wrapper.cpp new file mode 100644 index 000000000000..d7c4905e72ee --- /dev/null +++ b/src/ipa/libipa/ipa_interface_wrapper.cpp @@ -0,0 +1,151 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Google Inc. + * + * ipa_interface_wrapper.cpp - Image Processing Algorithm interface wrapper + */ + +#include "ipa_interface_wrapper.h" + +#include + +#include + +/** + * \file ipa_interface_wrapper.h + * \brief Image Processing Algorithm interface wrapper + */ + +namespace libcamera { + +/** + * \class IPAInterfaceWrapper + * \brief Wrap an IPAInterface and expose it as an ipa_context + * + * This class implements the ipa_context API based on a provided IPAInterface. + * It helps IPAs that implement the IPAInterface API to provide the external + * ipa_context API. + * + * To use the wrapper, an IPA module simple creates a new instance of its + * IPAInterface implementation, and passes it to the constructor of the + * IPAInterfaceWrapper. As IPAInterfaceWrapper inherits from ipa_context, the + * constructed wrapper can then be directly returned from the IPA module's + * ipaCreate() function. + * + * \code{.cpp} + * class MyIPA : public IPAInterface + * { + * ... + * }; + * + * struct ipa_context *ipaCreate() + * { + * return new IPAInterfaceWrapper(new MyIPA()); + * } + * \endcode + */ + +/** + * \brief Construct an IPAInterfaceWrapper wrapping \a interface + * \param[in] interface The interface to wrap + */ +IPAInterfaceWrapper::IPAInterfaceWrapper(IPAInterface *interface) + : ipa_(interface), callbacks_(nullptr), cb_ctx_(nullptr) +{ + ops = &operations_; + + ipa_->queueFrameAction.connect(this, &IPAInterfaceWrapper::queueFrameAction); +} + +void IPAInterfaceWrapper::destroy(struct ipa_context *_ctx) +{ + IPAInterfaceWrapper *ctx = static_cast(_ctx); + + delete ctx->ipa_; + delete ctx; +} + +void IPAInterfaceWrapper::init(struct ipa_context *_ctx) +{ + IPAInterfaceWrapper *ctx = static_cast(_ctx); + + ctx->ipa_->init(); +} + +void IPAInterfaceWrapper::register_callbacks(struct ipa_context *_ctx, + const struct ipa_callback_ops *callbacks, + void *cb_ctx) +{ + IPAInterfaceWrapper *ctx = static_cast(_ctx); + + ctx->callbacks_ = callbacks; + ctx->cb_ctx_ = cb_ctx; +} + +void IPAInterfaceWrapper::configure(struct ipa_context *_ctx) +{ +} + +void IPAInterfaceWrapper::map_buffers(struct ipa_context *_ctx, + const struct ipa_buffer *_buffers, + size_t num_buffers) +{ + IPAInterfaceWrapper *ctx = static_cast(_ctx); + std::vector buffers(num_buffers); + + for (unsigned int i = 0; i < num_buffers; ++i) { + const struct ipa_buffer &_buffer = _buffers[i]; + IPABuffer &buffer = buffers[i]; + std::vector &planes = buffer.memory.planes(); + + buffer.id = _buffer.id; + + planes.resize(_buffer.num_planes); + for (unsigned int j = 0; j < _buffer.num_planes; ++j) { + planes[j].setDmabuf(_buffer.planes[j].dmabuf, + _buffer.planes[j].length); + /** \todo Create a Dmabuf class to implement RAII. */ + ::close(_buffer.planes[j].dmabuf); + } + } + + ctx->ipa_->mapBuffers(buffers); +} + +void IPAInterfaceWrapper::unmap_buffers(struct ipa_context *_ctx, + const unsigned int *_ids, + size_t num_buffers) +{ + IPAInterfaceWrapper *ctx = static_cast(_ctx); + std::vector ids(_ids, _ids + num_buffers); + ctx->ipa_->unmapBuffers(ids); +} + +void IPAInterfaceWrapper::process_event(struct ipa_context *_ctx) +{ +} + +void IPAInterfaceWrapper::queueFrameAction(unsigned int frame, + const IPAOperationData &data) +{ + if (callbacks_) + callbacks_->queue_frame_action(cb_ctx_, frame); +} + +#ifndef __DOXYGEN__ +/* + * This construct confuses Doygen and makes it believe that all members of the + * operations is a member of IPAInterfaceWrapper. It must thus be hidden. + */ +const struct ipa_context_ops IPAInterfaceWrapper::operations_ = { + .destroy = &IPAInterfaceWrapper::destroy, + .init = &IPAInterfaceWrapper::init, + .register_callbacks = &IPAInterfaceWrapper::register_callbacks, + .configure = &IPAInterfaceWrapper::configure, + .map_buffers = &IPAInterfaceWrapper::map_buffers, + .unmap_buffers = &IPAInterfaceWrapper::unmap_buffers, + .process_event = &IPAInterfaceWrapper::process_event, +}; +#endif + +}; /* namespace libcamera */ diff --git a/src/ipa/libipa/ipa_interface_wrapper.h b/src/ipa/libipa/ipa_interface_wrapper.h new file mode 100644 index 000000000000..a7c70cdc2d83 --- /dev/null +++ b/src/ipa/libipa/ipa_interface_wrapper.h @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Google Inc. + * + * ipa_interface_wrapper.h - Image Processing Algorithm interface wrapper + */ +#ifndef __LIBCAMERA_IPA_INTERFACE_WRAPPER_H__ +#define __LIBCAMERA_IPA_INTERFACE_WRAPPER_H__ + +#include + +namespace libcamera { + +class IPAInterfaceWrapper : public ipa_context +{ +public: + IPAInterfaceWrapper(IPAInterface *interface); + +private: + static void destroy(struct ipa_context *ctx); + static void init(struct ipa_context *ctx); + static void register_callbacks(struct ipa_context *ctx, + const struct ipa_callback_ops *callbacks, + void *cb_ctx); + static void configure(struct ipa_context *ctx); + static void map_buffers(struct ipa_context *ctx, + const struct ipa_buffer *c_buffers, + size_t num_buffers); + static void unmap_buffers(struct ipa_context *ctx, + const unsigned int *ids, + size_t num_buffers); + static void process_event(struct ipa_context *ctx); + + static const struct ipa_context_ops operations_; + + void queueFrameAction(unsigned int frame, const IPAOperationData &data); + + IPAInterface *ipa_; + const struct ipa_callback_ops *callbacks_; + void *cb_ctx_; +}; + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_IPA_INTERFACE_WRAPPER_H__ */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build new file mode 100644 index 000000000000..03b046e2203e --- /dev/null +++ b/src/ipa/libipa/meson.build @@ -0,0 +1,11 @@ +libipa_headers = files([ + 'ipa_interface_wrapper.h', +]) + +libipa_sources = files([ + 'ipa_interface_wrapper.cpp', +]) + +libipa = static_library('ipa', libipa_sources, + include_directories : ipa_includes, + dependencies : libcamera_dep) diff --git a/src/ipa/meson.build b/src/ipa/meson.build index 4f2a45771201..421803243e32 100644 --- a/src/ipa/meson.build +++ b/src/ipa/meson.build @@ -10,11 +10,14 @@ ipa_includes = [ libcamera_internal_includes, ] +subdir('libipa') + foreach t : ipa_vimc_sources ipa = shared_module(t[0], 'ipa_vimc.cpp', name_prefix : '', include_directories : ipa_includes, dependencies : libcamera_dep, + link_with : libipa, install : true, install_dir : ipa_install_dir, cpp_args : '-DLICENSE="' + t[1] + '"') diff --git a/src/libcamera/include/ipa_context_wrapper.h b/src/libcamera/include/ipa_context_wrapper.h new file mode 100644 index 000000000000..f758bb07abea --- /dev/null +++ b/src/libcamera/include/ipa_context_wrapper.h @@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Google Inc. + * + * ipa_context_wrapper.h - Image Processing Algorithm context wrapper + */ +#ifndef __LIBCAMERA_IPA_CONTEXT_WRAPPER_H__ +#define __LIBCAMERA_IPA_CONTEXT_WRAPPER_H__ + +#include + +namespace libcamera { + +class IPAContextWrapper final : public IPAInterface +{ +public: + IPAContextWrapper(struct ipa_context *context); + ~IPAContextWrapper(); + + int init() override; + void configure(const std::map &streamConfig, + const std::map &entityControls) override; + + void mapBuffers(const std::vector &buffers) override; + void unmapBuffers(const std::vector &ids) override; + + virtual void processEvent(const IPAOperationData &data) override; + +private: + static void queue_frame_action(void *ctx, unsigned int frame); + static const struct ipa_callback_ops callbacks_; + + struct ipa_context *ctx_; +}; + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_IPA_CONTEXT_WRAPPER_H__ */ diff --git a/src/libcamera/include/ipa_module.h b/src/libcamera/include/ipa_module.h index 97737587ab3a..2028b76a1913 100644 --- a/src/libcamera/include/ipa_module.h +++ b/src/libcamera/include/ipa_module.h @@ -7,7 +7,6 @@ #ifndef __LIBCAMERA_IPA_MODULE_H__ #define __LIBCAMERA_IPA_MODULE_H__ -#include #include #include @@ -30,7 +29,7 @@ public: bool load(); - std::unique_ptr createInstance(); + struct ipa_context *createContext(); bool match(PipelineHandler *pipe, uint32_t minVersion, uint32_t maxVersion) const; @@ -45,7 +44,7 @@ private: bool loaded_; void *dlHandle_; - typedef IPAInterface *(*IPAIntfFactory)(void); + typedef struct ipa_context *(*IPAIntfFactory)(void); IPAIntfFactory ipaCreate_; int loadIPAModuleInfo(); diff --git a/src/libcamera/include/meson.build b/src/libcamera/include/meson.build index 2c74d29bd925..f9b7e9b09a60 100644 --- a/src/libcamera/include/meson.build +++ b/src/libcamera/include/meson.build @@ -7,6 +7,7 @@ libcamera_headers = files([ 'device_enumerator_udev.h', 'event_dispatcher_poll.h', 'formats.h', + 'ipa_context_wrapper.h', 'ipa_manager.h', 'ipa_module.h', 'ipa_proxy.h', diff --git a/src/libcamera/ipa_context_wrapper.cpp b/src/libcamera/ipa_context_wrapper.cpp new file mode 100644 index 000000000000..355911791d43 --- /dev/null +++ b/src/libcamera/ipa_context_wrapper.cpp @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Google Inc. + * + * ipa_context_wrapper.cpp - Image Processing Algorithm context wrapper + */ + +#include "ipa_context_wrapper.h" + +#include + +/** + * \file ipa_context_wrapper.h + * \brief Image Processing Algorithm context wrapper + */ + +namespace libcamera { + +/** + * \class IPAContextWrapper + * \brief Wrap an ipa_context and expose it as an IPAInterface + * + * The IPAContextWrapper class wraps an ipa_context, provided by an IPA module, and + * exposes an IPAInterface. This mechanism is used for IPAs that are not + * isolated in a separate process to allow direct calls from pipeline handler + * using the IPAInterface API instead of the lower-level ipa_context API. + * + * The IPAInterface methods are converted to the ipa_context API by serialising + * all C++ arguments into plain C structures or byte arrays that contain no + * pointer, as required by the ipa_context API. + */ + +/** + * \brief Construct an IPAContextWrapper instance that wraps the \a context + * \param[in] context The IPA module context + * + * Ownership of the \a context is passed to the IPAContextWrapper. The context remains + * valid for the whole lifetime of the wrapper and is destroyed automatically + * with it. + */ +IPAContextWrapper::IPAContextWrapper(struct ipa_context *context) + : ctx_(context) +{ +} + +IPAContextWrapper::~IPAContextWrapper() +{ + if (ctx_) + ctx_->ops->destroy(ctx_); +} + +int IPAContextWrapper::init() +{ + if (!ctx_) + return 0; + + ctx_->ops->register_callbacks(ctx_, &IPAContextWrapper::callbacks_, this); + ctx_->ops->init(ctx_); + + return 0; +} + +void IPAContextWrapper::configure(const std::map &streamConfig, + const std::map &entityControls) +{ + if (!ctx_) + return; + + ctx_->ops->configure(ctx_); +} + +void IPAContextWrapper::mapBuffers(const std::vector &buffers) +{ + if (!ctx_) + return; + + struct ipa_buffer c_buffers[buffers.size()]; + + for (unsigned int i = 0; i < buffers.size(); ++i) { + struct ipa_buffer &c_buffer = c_buffers[i]; + const IPABuffer &buffer = buffers[i]; + const std::vector &planes = buffer.memory.planes(); + + c_buffer.id = buffer.id; + c_buffer.num_planes = planes.size(); + + for (unsigned int j = 0; j < planes.size(); ++j) { + const Plane &plane = planes[j]; + c_buffer.planes[j].dmabuf = plane.dmabuf(); + c_buffer.planes[j].length = plane.length(); + } + } + + ctx_->ops->map_buffers(ctx_, c_buffers, buffers.size()); +} + +void IPAContextWrapper::unmapBuffers(const std::vector &ids) +{ + if (!ctx_) + return; + + ctx_->ops->unmap_buffers(ctx_, ids.data(), ids.size()); +} + +void IPAContextWrapper::processEvent(const IPAOperationData &data) +{ + if (!ctx_) + return; + + ctx_->ops->process_event(ctx_); +} + +void IPAContextWrapper::queue_frame_action(void *ctx, unsigned int frame) +{ + IPAContextWrapper *_this = static_cast(ctx); + IPAOperationData data; + + _this->queueFrameAction.emit(frame, data); +} + +#ifndef __DOXYGEN__ +/* + * This construct confuses Doygen and makes it believe that all members of the + * operations is a member of IPAInterfaceWrapper. It must thus be hidden. + */ +const struct ipa_callback_ops IPAContextWrapper::callbacks_ = { + .queue_frame_action = &IPAContextWrapper::queue_frame_action, +}; +#endif + +} /* namespace libcamera */ diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp index f3180c0739cb..47b131d75da4 100644 --- a/src/libcamera/ipa_manager.cpp +++ b/src/libcamera/ipa_manager.cpp @@ -12,6 +12,7 @@ #include #include +#include "ipa_context_wrapper.h" #include "ipa_module.h" #include "ipa_proxy.h" #include "log.h" @@ -30,6 +31,66 @@ LOG_DEFINE_CATEGORY(IPAManager) /** * \class IPAManager * \brief Manager for IPA modules + * + * The IPA module manager discovers IPA modules from disk, queries and loads + * them, and creates IPA contexts. It supports isolation of the modules in a + * separate process with IPC communication and offers a unified IPAInterface + * view of the IPA contexts to pipeline handlers regardless of whether the + * modules are isolated or loaded in the same process. + * + * Module isolation is based on the module licence. Open-source modules are + * loaded without isolation, while closed-source module are forcefully isolated. + * The isolation mechanism ensures that no code from a closed-source module is + * ever run in the libcamera process. + * + * To create an IPA context, pipeline handlers call the IPAManager::ipaCreate() + * method. For a directly loaded module, the manager calls the module's + * ipaCreate() function directly and wraps the returned context in an + * IPAContextWrapper that exposes an IPAInterface. + * + * ~~~~ + * +---------------+ + * | Pipeline | + * | Handler | + * +---------------+ + * | + * v + * +---------------+ +---------------+ + * | IPA | | Open Source | + * | Interface | | IPA Module | + * | - - - - - - - | | - - - - - - - | + * | IPA Context | ipa_context_ops | ipa_context | + * | Wrapper | ----------------> | | + * +---------------+ +---------------+ + * ~~~~ + * + * For an isolated module, the manager instantiates an IPAProxy which spawns a + * new process for an IPA proxy worker. The worker loads the IPA module and + * creates the IPA context. The IPAProxy alse exposes an IPAInterface. + * + * ~~~~ + * +---------------+ +---------------+ + * | Pipeline | | Closed Source | + * | Handler | | IPA Module | + * +---------------+ | - - - - - - - | + * | | ipa_context | + * v | | + * +---------------+ +---------------+ + * | IPA | ipa_context_ops ^ + * | Interface | | + * | - - - - - - - | +---------------+ + * | IPA Proxy | operations | IPA Proxy | + * | | ----------------> | Worker | + * +---------------+ over IPC +---------------+ + * ~~~~ + * + * The IPAInterface implemented by the IPAContextWrapper or IPAProxy is + * returned to the pipeline handler, and all interactions with the IPA context + * go the same interface regardless of process isolation. + * + * In all cases the data passed to the IPAInterface methods is serialised to + * Plain Old Data, either for the purpose of passing it to the IPA context + * plain C API, or to transmit the data to the isolated process through IPC. */ IPAManager::IPAManager() @@ -199,7 +260,11 @@ std::unique_ptr IPAManager::createIPA(PipelineHandler *pipe, if (!m->load()) return nullptr; - return m->createInstance(); + struct ipa_context *ctx = m->createContext(); + if (!ctx) + return nullptr; + + return utils::make_unique(ctx); } } /* namespace libcamera */ diff --git a/src/libcamera/ipa_module.cpp b/src/libcamera/ipa_module.cpp index 99d308efd47b..2c355ea8b5e5 100644 --- a/src/libcamera/ipa_module.cpp +++ b/src/libcamera/ipa_module.cpp @@ -385,13 +385,13 @@ const std::string &IPAModule::path() const /** * \brief Load the IPA implementation factory from the shared object * - * The IPA module shared object implements an IPAInterface class to be used + * The IPA module shared object implements an ipa_context object to be used * by pipeline handlers. This method loads the factory function from the - * shared object. Later, createInstance() can be called to instantiate the - * IPAInterface. + * shared object. Later, createContext() can be called to instantiate the + * ipa_context. * * This method only needs to be called successfully once, after which - * createInstance() can be called as many times as IPAInterface instances are + * createContext() can be called as many times as ipa_context instances are * needed. * * Calling this function on an invalid module (as returned by isValid()) is @@ -433,24 +433,25 @@ bool IPAModule::load() } /** - * \brief Instantiate an IPAInterface + * \brief Instantiate an IPA context * - * After loading the IPA module with load(), this method creates an - * instance of the IPA module interface. + * After loading the IPA module with load(), this method creates an instance of + * the IPA module context. Ownership of the context is passed to the caller, and + * the context shall be destroyed by calling the \ref ipa_context_ops::destroy + * "ipa_context::ops::destroy()" function. * * Calling this function on a module that has not yet been loaded, or an * invalid module (as returned by load() and isValid(), respectively) is * an error. * - * \return The IPA implementation as a new IPAInterface instance on success, - * or nullptr on error + * \return The IPA context on success, or nullptr on error */ -std::unique_ptr IPAModule::createInstance() +struct ipa_context *IPAModule::createContext() { if (!valid_ || !loaded_) return nullptr; - return std::unique_ptr(ipaCreate_()); + return ipaCreate_(); } /** diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index d329820b9582..775ceff9dace 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -14,6 +14,7 @@ libcamera_sources = files([ 'event_notifier.cpp', 'formats.cpp', 'geometry.cpp', + 'ipa_context_wrapper.cpp', 'ipa_interface.cpp', 'ipa_manager.cpp', 'ipa_module.cpp', diff --git a/src/libcamera/proxy/worker/ipa_proxy_linux_worker.cpp b/src/libcamera/proxy/worker/ipa_proxy_linux_worker.cpp index a10761e52b32..07380c16e2d5 100644 --- a/src/libcamera/proxy/worker/ipa_proxy_linux_worker.cpp +++ b/src/libcamera/proxy/worker/ipa_proxy_linux_worker.cpp @@ -72,9 +72,9 @@ int main(int argc, char **argv) } socket.readyRead.connect(&readyRead); - std::unique_ptr ipa = ipam->createInstance(); - if (!ipa) { - LOG(IPAProxyLinuxWorker, Error) << "Failed to create IPA interface"; + struct ipa_context *ipac = ipam->createContext(); + if (!ipac) { + LOG(IPAProxyLinuxWorker, Error) << "Failed to create IPA context"; return EXIT_FAILURE; } @@ -85,5 +85,7 @@ int main(int argc, char **argv) while (1) dispatcher->processEvents(); + ipac->ops->destroy(ipac); + return 0; } From patchwork Sun Oct 13 00:13:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 2180 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3F06560BE5 for ; Sun, 13 Oct 2019 02:13:50 +0200 (CEST) Received: from pendragon.bb.dnainternet.fi (dfj612yhrgyx302h3jwwy-3.rev.dnainternet.fi [IPv6:2001:14ba:21f5:5b00:ce28:277f:58d7:3ca4]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D307333A for ; Sun, 13 Oct 2019 02:13:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1570925630; bh=WIH2FNXIlC1GM2+5g/DM4cX0pyVtFIf0+aAyL2YJED8=; h=From:To:Subject:Date:In-Reply-To:References:From; b=VRJc3is7kGTi2C8bYifd1Wbk2wJ4kTxFU90hJvDB05N9kDKmGdP/iVAfoRFOu1TwD YB7fO/2TgFkP3UEPw5D6s3jNEUs7IKfyujf4n4m8x7brCH3ajOfMpGvOYVLv7TIzGD nRtYjZdG8QiHKgpK4Os1aElxf/nzalHWJ/OpM+JU= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Sun, 13 Oct 2019 03:13:40 +0300 Message-Id: <20191013001340.14266-5-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191013001340.14266-1-laurent.pinchart@ideasonboard.com> References: <20191013001340.14266-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH/RFC v2 4/4] ipa: Allow short-circuiting the ipa_context_ops X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Oct 2019 00:13:50 -0000 When an IPA module is loaded without isolation and implements the IPAInterface internally, going through ipa_context_ops is a waste of time. Add an operation to retrieve the IPAInterface, and use it directly in the IPAContextWrapper. Signed-off-by: Laurent Pinchart Reviewed-by: Jacopo Mondi Reviewed-by: Niklas Söderlund --- include/ipa/ipa_interface.h | 1 + src/ipa/libipa/ipa_interface_wrapper.cpp | 8 ++++++ src/ipa/libipa/ipa_interface_wrapper.h | 1 + src/libcamera/include/ipa_context_wrapper.h | 3 +++ src/libcamera/ipa_context_wrapper.cpp | 29 ++++++++++++++++++++- src/libcamera/ipa_interface.cpp | 18 +++++++++++++ 6 files changed, 59 insertions(+), 1 deletion(-) diff --git a/include/ipa/ipa_interface.h b/include/ipa/ipa_interface.h index acb491e3958c..c104b8805e7e 100644 --- a/include/ipa/ipa_interface.h +++ b/include/ipa/ipa_interface.h @@ -34,6 +34,7 @@ struct ipa_callback_ops { struct ipa_context_ops { void (*destroy)(struct ipa_context *ctx); + void *(*get_interface)(struct ipa_context *ctx); void (*init)(struct ipa_context *ctx); void (*register_callbacks)(struct ipa_context *ctx, const struct ipa_callback_ops *callbacks, diff --git a/src/ipa/libipa/ipa_interface_wrapper.cpp b/src/ipa/libipa/ipa_interface_wrapper.cpp index d7c4905e72ee..baed3acd2b01 100644 --- a/src/ipa/libipa/ipa_interface_wrapper.cpp +++ b/src/ipa/libipa/ipa_interface_wrapper.cpp @@ -65,6 +65,13 @@ void IPAInterfaceWrapper::destroy(struct ipa_context *_ctx) delete ctx; } +void *IPAInterfaceWrapper::get_interface(struct ipa_context *_ctx) +{ + IPAInterfaceWrapper *ctx = static_cast(_ctx); + + return ctx->ipa_; +} + void IPAInterfaceWrapper::init(struct ipa_context *_ctx) { IPAInterfaceWrapper *ctx = static_cast(_ctx); @@ -139,6 +146,7 @@ void IPAInterfaceWrapper::queueFrameAction(unsigned int frame, */ const struct ipa_context_ops IPAInterfaceWrapper::operations_ = { .destroy = &IPAInterfaceWrapper::destroy, + .get_interface = &IPAInterfaceWrapper::get_interface, .init = &IPAInterfaceWrapper::init, .register_callbacks = &IPAInterfaceWrapper::register_callbacks, .configure = &IPAInterfaceWrapper::configure, diff --git a/src/ipa/libipa/ipa_interface_wrapper.h b/src/ipa/libipa/ipa_interface_wrapper.h index a7c70cdc2d83..c4b9d4b668d8 100644 --- a/src/ipa/libipa/ipa_interface_wrapper.h +++ b/src/ipa/libipa/ipa_interface_wrapper.h @@ -18,6 +18,7 @@ public: private: static void destroy(struct ipa_context *ctx); + static void *get_interface(struct ipa_context *ctx); static void init(struct ipa_context *ctx); static void register_callbacks(struct ipa_context *ctx, const struct ipa_callback_ops *callbacks, diff --git a/src/libcamera/include/ipa_context_wrapper.h b/src/libcamera/include/ipa_context_wrapper.h index f758bb07abea..218768eaa8be 100644 --- a/src/libcamera/include/ipa_context_wrapper.h +++ b/src/libcamera/include/ipa_context_wrapper.h @@ -30,7 +30,10 @@ private: static void queue_frame_action(void *ctx, unsigned int frame); static const struct ipa_callback_ops callbacks_; + void queueFrameAction(unsigned int frame, const IPAOperationData &data); + struct ipa_context *ctx_; + IPAInterface *intf_; }; } /* namespace libcamera */ diff --git a/src/libcamera/ipa_context_wrapper.cpp b/src/libcamera/ipa_context_wrapper.cpp index 355911791d43..7e58a7971391 100644 --- a/src/libcamera/ipa_context_wrapper.cpp +++ b/src/libcamera/ipa_context_wrapper.cpp @@ -41,6 +41,12 @@ namespace libcamera { IPAContextWrapper::IPAContextWrapper(struct ipa_context *context) : ctx_(context) { + if (ctx_ && ctx_->ops->get_interface) { + intf_ = reinterpret_cast(ctx_->ops->get_interface(ctx_)); + intf_->queueFrameAction.connect(this, &IPAContextWrapper::queueFrameAction); + } else { + intf_ = nullptr; + } } IPAContextWrapper::~IPAContextWrapper() @@ -51,6 +57,9 @@ IPAContextWrapper::~IPAContextWrapper() int IPAContextWrapper::init() { + if (intf_) + return intf_->init(); + if (!ctx_) return 0; @@ -63,6 +72,9 @@ int IPAContextWrapper::init() void IPAContextWrapper::configure(const std::map &streamConfig, const std::map &entityControls) { + if (intf_) + return intf_->configure(streamConfig, entityControls); + if (!ctx_) return; @@ -71,6 +83,9 @@ void IPAContextWrapper::configure(const std::map &strea void IPAContextWrapper::mapBuffers(const std::vector &buffers) { + if (intf_) + return intf_->mapBuffers(buffers); + if (!ctx_) return; @@ -96,6 +111,9 @@ void IPAContextWrapper::mapBuffers(const std::vector &buffers) void IPAContextWrapper::unmapBuffers(const std::vector &ids) { + if (intf_) + return intf_->unmapBuffers(ids); + if (!ctx_) return; @@ -104,18 +122,27 @@ void IPAContextWrapper::unmapBuffers(const std::vector &ids) void IPAContextWrapper::processEvent(const IPAOperationData &data) { + if (intf_) + return intf_->processEvent(data); + if (!ctx_) return; ctx_->ops->process_event(ctx_); } +void IPAContextWrapper::queueFrameAction(unsigned int frame, + const IPAOperationData &data) +{ + IPAInterface::queueFrameAction.emit(frame, data); +} + void IPAContextWrapper::queue_frame_action(void *ctx, unsigned int frame) { IPAContextWrapper *_this = static_cast(ctx); IPAOperationData data; - _this->queueFrameAction.emit(frame, data); + _this->queueFrameAction(frame, data); } #ifndef __DOXYGEN__ diff --git a/src/libcamera/ipa_interface.cpp b/src/libcamera/ipa_interface.cpp index 1df430d721f0..a0647b112466 100644 --- a/src/libcamera/ipa_interface.cpp +++ b/src/libcamera/ipa_interface.cpp @@ -62,6 +62,12 @@ * handlers to communicate with IPA modules. IPA modules may use the * IPAInterface API internally if they want to benefit from the data and helper * classes offered by libcamera. + * + * When an IPA module is loaded directly into the libcamera process and uses + * the IPAInterface API internally, short-circuiting the path to the + * ipa_context_ops and back to IPAInterface is desirable. To support this, IPA + * modules may implement the ipa_context_ops::get_interface function to return a + * pointer to their internal IPAInterface. */ /** @@ -146,6 +152,18 @@ * \param[in] ctx The IPA context */ +/** + * \var ipa_context_ops::get_interface + * \brief Retrieve the IPAInterface implemented by the ipa_context (optional) + * \param[in] ctx The IPA context + * + * IPA modules may implement this function to expose their internal + * IPAInterface, if any. When implemented, libcamera may at its sole discretion + * call it and then bypass the ipa_context_ops API by calling the IPAInterface + * methods directly. IPA modules shall still implement and support the full + * ipa_context_ops API. + */ + /** * \var ipa_context_ops::init * \brief Initialise the IPA context