From patchwork Wed Aug 26 11:09:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9385 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5CF69BD87E for ; Wed, 26 Aug 2020 11:09:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2872A628BB; Wed, 26 Aug 2020 13:09:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="TYfwQ0rY"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 36715628BB for ; Wed, 26 Aug 2020 13:09:51 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5B2AC9CE; Wed, 26 Aug 2020 13:09:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440190; bh=F5tkcbjIQGxN2c/bWCViW9R213l+wmoPQMh9FZwRSTY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TYfwQ0rYP92WjFOrtGD9QOXwpG0geUDokuJ7SIA7XbqNlOATD+2Kn0I3LDfIlJphl CG3jblHg9SILx3nhOsGQ8r5a/U5aIpKT5GeBQgV7yQ0Xfs+LFAMkcYpzgQowDR4kPa YhWAwnCma+iaRzwMCT3H7TWMv2/f/+OXINrn57kI= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:10 +0900 Message-Id: <20200826110926.67192-2-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 01/17] IPA: IPC: raspberrypi: Add data definition and generated header X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch shows what the data definition for the IPA interface for raspberrypi might look like, as well as the corresponding generated header, with the data definition file as the source. This first patch is meant to show what the pipeline/IPA author would have to write/agree upon, and what the corresponding generated header would look like. The data defition file, as seen in raspberrypi.mojom, uses the mojo syntax [1]. To define the IPA interface, functions must be defined as shown in interface RPiIPAInterface. These functions are for the pipeline handler to call. For functions that the IPA calls, via signals, are defined in interface RPiIPACallbackInterface (we'll probably pick up which interface is for which by magic names and extract the pipeline/IPA name). Any functions and any data structures can be used. Data structures that that are specific to the pipeline/IPA must be defined in the same file (eg. RPiConfigureParams, RPiEventParams, RPiActionParams, etc). Any other data structures (eg. IPAStream, IPABuffer, ControlInfoMap) must have de/serializers implemented. I have implemented de/serializers for a few data structures that I think are important. There are some rules to the interface definition: - any functions that the pipeline calls after start() must not return anything (eg. processEvent) - any functions that the IPA calls, ever, must not return anything (eg. queueFrameAction) - any input values, if not POD, will have const and reference in the generated function prototype - if there is only one output and it is a POD, then it will be a simple return value. Otherwise the output(s) will be via pointer output parameter(s). - maps and arrays are allowed (they will be translated to std::map and std::vector, respectively) raspberrypi_wrapper.h shows the generated header. For this RFC, it is hand-generated by me, but I'll make a compiler to automatically generate it, given the mojo data definition file. I think this generated file is fairly straightforward. It contains the structs and enums as defined in the mojo file. This header is meant to be included by pipelines and IPAs so that they can know how to access the data structures passed around in the IPA interface. It will also be used by the generated IPA proxy to figure out how to de/serialize the nested structures. In raspberrypi.h we first move the enum away into the mojo definition and the generated header. The last hunk is meant to be generated by the compiler as well, though I'm not sure how best to fit it in, since the rest of this header is hand-written. RPiCMD will be generated based on the IPA interface defined in the mojo file. Note that both the main functions and the callbacks share the same enumspace. The IPARPiInterface, which implements and IPAInterface, is defined as well, also based on the IPA interface defined in the mojo file. It is a virtual class, since both IPAProxyRPi and IPARPi must implement it since that's how the stack will work: pipeline handler -> IPAProxyRPi --thread--> IPARPi \ \-> IPAIPC --IPA--> IPAProxyRPiWorker -> IPARPi Signed-off-by: Paul Elder --- include/libcamera/ipa/raspberrypi.h | 61 ++++++--- include/libcamera/ipa/raspberrypi.mojom | 97 ++++++++++++++ include/libcamera/ipa/raspberrypi_wrapper.h | 136 ++++++++++++++++++++ 3 files changed, 276 insertions(+), 18 deletions(-) create mode 100644 include/libcamera/ipa/raspberrypi.mojom create mode 100644 include/libcamera/ipa/raspberrypi_wrapper.h diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index ca62990e..fa7efdbc 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -9,24 +9,7 @@ #include #include - -enum RPiConfigParameters { - RPI_IPA_CONFIG_LS_TABLE = (1 << 0), - RPI_IPA_CONFIG_STAGGERED_WRITE = (1 << 1), - RPI_IPA_CONFIG_SENSOR = (1 << 2), -}; - -enum RPiOperations { - RPI_IPA_ACTION_V4L2_SET_STAGGERED = 1, - RPI_IPA_ACTION_V4L2_SET_ISP, - RPI_IPA_ACTION_STATS_METADATA_COMPLETE, - RPI_IPA_ACTION_RUN_ISP, - RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME, - RPI_IPA_ACTION_EMBEDDED_COMPLETE, - RPI_IPA_EVENT_SIGNAL_STAT_READY, - RPI_IPA_EVENT_SIGNAL_ISP_PREPARE, - RPI_IPA_EVENT_QUEUE_REQUEST, -}; +#include enum RPiIpaMask { ID = 0x0ffff, @@ -40,6 +23,10 @@ enum RPiIpaMask { namespace libcamera { +struct RPiConfigureParams; +struct RPiEventParams; +struct RPiActionParams; + /* List of controls handled by the Raspberry Pi IPA */ static const ControlInfoMap RPiControls = { { &controls::AeEnable, ControlInfo(false, true) }, @@ -59,6 +46,44 @@ static const ControlInfoMap RPiControls = { { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, }; +// TODO this needs to be automatically generated too +enum RPiCMD { + CMD_INIT = 1, + CMD_EXIT, + CMD_START, + CMD_STOP, + CMD_CONFIGURE, + CMD_MAPBUFFERS, + CMD_UNMAPBUFFERS, + CMD_PROCESSEVENT, + CMD_QUEUEFRAMEACTION, +}; + +// This needs to be here so that the rpi proxy and worker can include it +// Still needs to inherit from IPAInterface for IPAModule to work +// TODO This needs to be automatically generated... so put in a different +// header too, probably +class IPARPiInterface : public IPAInterface +{ +public: + virtual ~IPARPiInterface() {}; + + virtual int init(const IPASettings &settings) = 0; + virtual int start() = 0; + virtual void stop() = 0; + + virtual void configure(const CameraSensorInfo &sensorInfo, + const std::map &streamConfig, + const std::map &entityControls, + const RPiConfigureParams &data, + RPiConfigureParams *response) = 0; + virtual void mapBuffers(const std::vector &buffers) = 0; + virtual void unmapBuffers(const std::vector &ids) = 0; + virtual void processEvent(const RPiEventParams &event) = 0; + + Signal queueFrameAction; +}; + } /* namespace libcamera */ #endif /* __LIBCAMERA_IPA_INTERFACE_RASPBERRYPI_H__ */ diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom new file mode 100644 index 00000000..3baceb37 --- /dev/null +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -0,0 +1,97 @@ +// automatically add const and & to all inputs (that are not primitives) +// automatically add pointer to all outputs (that are not primitives) +// if return value is single and primitive, then return as ret value +// if return value is multiple or non-primitive, then return as output param +interface RPiIPAInterface { + init(IPASettings settings) => (int32); + start() => (int32); + stop() => (); + + configure(CameraSensorInfo sensorInfo, + map streamConfig, + map entityControls, + RPiConfigureParams ipaConfig) + => (RPiConfigureParams result); + + // arrays get turned into vectors + mapBuffers(array buffers) => (int32); + unmapBuffers(array ids) => (); + + processEvent(RPiEventParams data) => (); +} + +// these shall not return anything +interface RPiIPACallbackInterface { + queueFrameAction(uint32, RPiActionPayload) => (); +} + +enum RPiConfigParameters { + RPI_IPA_CONFIG_LS_TABLE, + RPI_IPA_CONFIG_STAGGERED_WRITE, + RPI_IPA_CONFIG_SENSOR, + RPI_IPA_CONFIG_SEND_FD, +}; + +enum RPiEvents { + RPI_IPA_EVENT_SIGNAL_STAT_READY, + RPI_IPA_EVENT_SIGNAL_ISP_PREPARE, + RPI_IPA_EVENT_QUEUE_REQUEST, + RPI_IPA_EVENT_SEND_FD, +}; + +enum RPiActions { + RPI_IPA_ACTION_V4L2_SET_STAGGERED, + RPI_IPA_ACTION_V4L2_SET_ISP, + RPI_IPA_ACTION_STATS_METADATA_COMPLETE, + RPI_IPA_ACTION_RUN_ISP, + RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME, + RPI_IPA_ACTION_EMBEDDED_COMPLETE, +}; + +// Custom Data containers + +struct RPiStaggeredWritePayload { + uint32 gainDelay; + uint32 exposureDelay; + uint32 sensorMetadata; +}; + +struct RPiIspPreparePayload { + uint32 embeddedbufferId; + uint32 bayerbufferId; +}; + +struct RPiStatsCompletePayload { + uint32 bufferId; + ControlList controls; +}; + +struct RPiConfigurePayload { + RPiConfigParameters op; + FileDescriptor lsTableHandle; + int32 lsTableHandleStatic; + RPiStaggeredWritePayload staggeredWriteResult; + ControlList controls; + int32 bufferFd; +}; + +// First level payload + +struct RPiConfigureParams { + array payload; +}; + +struct RPiEventParams { + RPiEvents ev; + uint32 bufferId; + RPiIspPreparePayload ispPrepare; + ControlList controls; + int32 bufferFd; +}; + +struct RPiActionParams { + RPiActions op; + uint32 bufferId; + RPiStatsCompletePayload statsComplete; + ControlList controls; +}; diff --git a/include/libcamera/ipa/raspberrypi_wrapper.h b/include/libcamera/ipa/raspberrypi_wrapper.h new file mode 100644 index 00000000..ba44526d --- /dev/null +++ b/include/libcamera/ipa/raspberrypi_wrapper.h @@ -0,0 +1,136 @@ +// automatically generated by custom compiler + +#ifndef __LIBCAMERA_IPA_INTERFACE_RASPBERRYPI_FB_H__ +#define __LIBCAMERA_IPA_INTERFACE_RASPBERRYPI_FB_H__ + +#include "libcamera/internal/byte_stream_buffer.h" +#include "libcamera/internal/control_serializer.h" +#include "libcamera/internal/ipa_data_serializer.h" + +#include +#include + +#include +#include + +namespace libcamera { + +enum RPiConfigParameters { + RPI_IPA_CONFIG_LS_TABLE = 1, + RPI_IPA_CONFIG_STAGGERED_WRITE = 2, + RPI_IPA_CONFIG_SENSOR = 3, + RPI_IPA_CONFIG_SEND_FD = 4, +}; + +enum RPiEvents { + RPI_IPA_EVENT_SIGNAL_STAT_READY = 1, + RPI_IPA_EVENT_SIGNAL_ISP_PREPARE = 2, + RPI_IPA_EVENT_QUEUE_REQUEST = 3, + RPI_IPA_EVENT_SEND_FD = 4, +}; + +enum RPiActions { + RPI_IPA_ACTION_V4L2_SET_STAGGERED = 1, + RPI_IPA_ACTION_V4L2_SET_ISP = 2, + RPI_IPA_ACTION_STATS_METADATA_COMPLETE = 3, + RPI_IPA_ACTION_RUN_ISP = 4, + RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME = 5, + RPI_IPA_ACTION_EMBEDDED_COMPLETE = 6, +}; + +struct RPiStaggeredWritePayload +{ +public: + RPiStaggeredWritePayload() : gainDelay_(0), exposureDelay_(0), sensorMetadata_(0) {} + ~RPiStaggeredWritePayload() {} + + RPiStaggeredWritePayload(uint32_t gainDelay, uint32_t exposureDelay, uint32_t sensorMetadata) : gainDelay_(gainDelay), exposureDelay_(exposureDelay), sensorMetadata_(sensorMetadata) {} + + uint32_t gainDelay_; + uint32_t exposureDelay_; + uint32_t sensorMetadata_; +}; + +struct RPiIspPreparePayload +{ +public: + RPiIspPreparePayload() : embeddedbufferId_(0), bayerbufferId_(0) {} + ~RPiIspPreparePayload() {} + + RPiIspPreparePayload(uint32_t embeddedbufferId, uint32_t bayerbufferId) : embeddedbufferId_(embeddedbufferId), bayerbufferId_(bayerbufferId) {} + + uint32_t embeddedbufferId_; + uint32_t bayerbufferId_; +}; + +struct RPiStatsCompletePayload +{ +public: + RPiStatsCompletePayload() : bufferId_(0) {} + ~RPiStatsCompletePayload() {} + + RPiStatsCompletePayload(uint32_t bufferId, ControlList &controls) : bufferId_(bufferId), controls_(controls) {} + + uint32_t bufferId_; + ControlList controls_; +}; + +struct RPiConfigurePayload +{ +public: + RPiConfigurePayload() : op_(static_cast(0)), lsTableHandle_(-1), bufferFd_(0) {} + ~RPiConfigurePayload() {} + + RPiConfigurePayload(enum RPiConfigParameters op, uint32_t lsTableHandle, uint32_t lsTableHandleStatic, RPiStaggeredWritePayload staggeredWriteResult, ControlList controls, int32_t bufferFd) : op_(op), lsTableHandle_(lsTableHandle), lsTableHandleStatic_(lsTableHandleStatic), staggeredWriteResult_(staggeredWriteResult), controls_(controls), bufferFd_(bufferFd) {} + + enum RPiConfigParameters op_; + FileDescriptor lsTableHandle_; + int32_t lsTableHandleStatic_; + RPiStaggeredWritePayload staggeredWriteResult_; + ControlList controls_; + int32_t bufferFd_; +}; + +struct RPiConfigureParams +{ +public: + RPiConfigureParams() {} + ~RPiConfigureParams() {} + + RPiConfigureParams(std::vector payload) : payload_(payload) {} + + std::vector payload_; +}; + +struct RPiEventParams +{ +public: + RPiEventParams() : ev_(static_cast(0)), bufferId_(0), bufferFd_(0) {} + ~RPiEventParams() {} + + RPiEventParams(enum RPiEvents ev, uint32_t bufferId, RPiIspPreparePayload ispPrepare, ControlList controls, int32_t bufferFd) : ev_(ev), bufferId_(bufferId), ispPrepare_(ispPrepare), controls_(controls), bufferFd_(bufferFd) {} + + enum RPiEvents ev_; + uint32_t bufferId_; + RPiIspPreparePayload ispPrepare_; + ControlList controls_; + int32_t bufferFd_; +}; + +struct RPiActionParams +{ +public: + RPiActionParams() : op_(static_cast(0)), bufferId_(0) {} + ~RPiActionParams() {} + + RPiActionParams(enum RPiActions op, uint32_t bufferId, RPiStatsCompletePayload statsComplete, ControlList controls) : op_(op), bufferId_(bufferId), statsComplete_(statsComplete), controls_(controls) {} + + enum RPiActions op_; + uint32_t bufferId_; + RPiStatsCompletePayload statsComplete_; + ControlList controls_; +}; + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_IPA_INTERFACE_RASPBERRYPI_FB_H__ */ From patchwork Wed Aug 26 11:09:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9386 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ADC45BD87E for ; Wed, 26 Aug 2020 11:09:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7A3A6628FB; Wed, 26 Aug 2020 13:09:54 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ivwyO3IG"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 595FB628F3 for ; Wed, 26 Aug 2020 13:09:53 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 989DAB1A; Wed, 26 Aug 2020 13:09:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440193; bh=/0ZrAwhrufuHQsETJWT0Htdeae5oD1X0A00R0YVDKoE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ivwyO3IG2qzUSLINhYC8pBXFZHI45XeMeIKYULMeEHwSioe2nExbeCfPMZf1F4bav 6HRd/sLf5eL9aNQPyUlDHTOm3wpeOTbIBBc4CE29Bb1JwR10/JI3oWKsZO9LmP2Zny rSpf2LDhp0Sfa1kykrdNNxTTUzvw7XaJa31DVDdE= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:11 +0900 Message-Id: <20200826110926.67192-3-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 02/17] libcamera: pipeline: raspberrypi: Use generated IPARPiInterface X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch shows how the pipeline would use the generated IPA interface. raspberrypi_wrapper.h is included, since that contains the definitions of all the custom data structures. ipa_proxy_raspberrypi.h is also included, so that the pipeline handler can create the IPAProxyRPi. Due to the custom IPA interface, the generic IPAProxy can no longer be used. This will be elaborated on in a few patches. Other than that, the rest of the patch should be straightforward. The only changes are just fiddling the code to use the new structures. Most noteworthy is that the pipeline handler does not know nor care if it is communicating with the IPA isolated or direct. This is one of our main goals. Signed-off-by: Paul Elder --- .../pipeline/raspberrypi/raspberrypi.cpp | 125 ++++++++++-------- 1 file changed, 69 insertions(+), 56 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 42c9caa0..214dba8f 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include @@ -35,6 +36,8 @@ #include "dma_heaps.h" #include "staggered_ctrl.h" +#include "libcamera/internal/ipa_proxy_raspberrypi.h" + namespace libcamera { LOG_DEFINE_CATEGORY(RPI) @@ -296,7 +299,7 @@ public: int loadIPA(); int configureIPA(); - void queueFrameAction(unsigned int frame, const IPAOperationData &action); + void queueFrameAction(unsigned int frame, const RPiActionParams &action); /* bufferComplete signal handlers. */ void unicamBufferDequeue(FrameBuffer *buffer); @@ -307,6 +310,8 @@ public: void handleStreamBuffer(FrameBuffer *buffer, const RPiStream *stream); void handleState(); + std::unique_ptr ipa_; + CameraSensor *sensor_; /* Array of Unicam and ISP device streams and associated buffers/streams. */ RPiDevice unicam_; @@ -1094,7 +1099,9 @@ void RPiCameraData::frameStarted(uint32_t sequence) int RPiCameraData::loadIPA() { - ipa_ = IPAManager::createIPA(pipe_, 1, 1); + std::unique_ptr ptr = IPAManager::createIPA(pipe_, 1, 1); + ipa_ = std::unique_ptr{static_cast(std::move(ptr).release())}; + if (!ipa_) return -ENOENT; @@ -1110,8 +1117,8 @@ int RPiCameraData::loadIPA() int RPiCameraData::configureIPA() { std::map streamConfig; - std::map entityControls; - IPAOperationData ipaConfig = {}; + std::map entityControls; + RPiConfigureParams ipaConfig; /* Get the device format to pass to the IPA. */ V4L2DeviceFormat sensorFormat; @@ -1136,8 +1143,11 @@ int RPiCameraData::configureIPA() return -ENOMEM; /* Allow the IPA to mmap the LS table via the file descriptor. */ - ipaConfig.operation = RPI_IPA_CONFIG_LS_TABLE; - ipaConfig.data = { static_cast(lsTable_.fd()) }; + RPiConfigurePayload payload; + payload.op_ = RPI_IPA_CONFIG_LS_TABLE; + payload.lsTableHandle_ = lsTable_; + payload.lsTableHandleStatic_ = lsTable_.fd(); + ipaConfig.payload_.push_back(payload); } CameraSensorInfo sensorInfo = {}; @@ -1148,60 +1158,66 @@ int RPiCameraData::configureIPA() } /* Ready the IPA - it must know about the sensor resolution. */ - IPAOperationData result; + RPiConfigureParams results; ipa_->configure(sensorInfo, streamConfig, entityControls, ipaConfig, - &result); + &results); - if (result.operation & RPI_IPA_CONFIG_STAGGERED_WRITE) { - /* - * Setup our staggered control writer with the sensor default - * gain and exposure delays. - */ - if (!staggeredCtrl_) { - staggeredCtrl_.init(unicam_[Unicam::Image].dev(), - { { V4L2_CID_ANALOGUE_GAIN, result.data[0] }, - { V4L2_CID_EXPOSURE, result.data[1] } }); - sensorMetadata_ = result.data[2]; - } + for (RPiConfigurePayload &result : results.payload_) { + if (result.op_ == RPI_IPA_CONFIG_STAGGERED_WRITE) { - /* Configure the H/V flip controls based on the sensor rotation. */ - ControlList ctrls(unicam_[Unicam::Image].dev()->controls()); - int32_t rotation = sensor_->properties().get(properties::Rotation); - ctrls.set(V4L2_CID_HFLIP, static_cast(!!rotation)); - ctrls.set(V4L2_CID_VFLIP, static_cast(!!rotation)); - unicam_[Unicam::Image].dev()->setControls(&ctrls); - } + /* + * Setup our staggered control writer with the sensor default + * gain and exposure delays. + */ + if (!staggeredCtrl_) { + staggeredCtrl_.init(unicam_[Unicam::Image].dev(), + { { V4L2_CID_ANALOGUE_GAIN, result.staggeredWriteResult_.gainDelay_ }, + { V4L2_CID_EXPOSURE, result.staggeredWriteResult_.exposureDelay_ } }); + sensorMetadata_ = result.staggeredWriteResult_.sensorMetadata_; + } - if (result.operation & RPI_IPA_CONFIG_SENSOR) { - const ControlList &ctrls = result.controls[0]; - if (!staggeredCtrl_.set(ctrls)) - LOG(RPI, Error) << "V4L2 staggered set failed"; + /* Configure the H/V flip controls based on the sensor rotation. */ + ControlList ctrls(unicam_[Unicam::Image].dev()->controls()); + int32_t rotation = sensor_->properties().get(properties::Rotation); + ctrls.set(V4L2_CID_HFLIP, static_cast(!!rotation)); + ctrls.set(V4L2_CID_VFLIP, static_cast(!!rotation)); + unicam_[Unicam::Image].dev()->setControls(&ctrls); + } + + if (result.op_ == RPI_IPA_CONFIG_SENSOR) { + const ControlList &ctrls = result.controls_; + if (!staggeredCtrl_.set(ctrls)) + LOG(RPI, Error) << "V4L2 staggered set failed"; + } } return 0; } void RPiCameraData::queueFrameAction([[maybe_unused]] unsigned int frame, - const IPAOperationData &action) + const RPiActionParams &action) { /* * The following actions can be handled when the pipeline handler is in * a stopped state. */ - switch (action.operation) { + switch (action.op_) { case RPI_IPA_ACTION_V4L2_SET_STAGGERED: { - const ControlList &controls = action.controls[0]; + const ControlList &controls = action.controls_; if (!staggeredCtrl_.set(controls)) LOG(RPI, Error) << "V4L2 staggered set failed"; goto done; } case RPI_IPA_ACTION_V4L2_SET_ISP: { - ControlList controls = action.controls[0]; + ControlList controls = action.controls_; isp_[Isp::Input].dev()->setControls(&controls); goto done; } + + default: + break; } if (state_ == State::Stopped) @@ -1211,20 +1227,20 @@ void RPiCameraData::queueFrameAction([[maybe_unused]] unsigned int frame, * The following actions must not be handled when the pipeline handler * is in a stopped state. */ - switch (action.operation) { + switch (action.op_) { case RPI_IPA_ACTION_STATS_METADATA_COMPLETE: { - unsigned int bufferId = action.data[0]; + unsigned int bufferId = action.statsComplete_.bufferId_; FrameBuffer *buffer = isp_[Isp::Stats].getBuffers()->at(bufferId).get(); handleStreamBuffer(buffer, &isp_[Isp::Stats]); /* Fill the Request metadata buffer with what the IPA has provided */ - requestQueue_.front()->metadata() = std::move(action.controls[0]); + requestQueue_.front()->metadata() = std::move(action.statsComplete_.controls_); state_ = State::IpaComplete; break; } case RPI_IPA_ACTION_EMBEDDED_COMPLETE: { - unsigned int bufferId = action.data[0]; + unsigned int bufferId = action.bufferId_; FrameBuffer *buffer = unicam_[Unicam::Embedded].getBuffers()->at(bufferId).get(); handleStreamBuffer(buffer, &unicam_[Unicam::Embedded]); break; @@ -1232,20 +1248,17 @@ void RPiCameraData::queueFrameAction([[maybe_unused]] unsigned int frame, case RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME: case RPI_IPA_ACTION_RUN_ISP: { - unsigned int bufferId = action.data[0]; + unsigned int bufferId = action.bufferId_; FrameBuffer *buffer = unicam_[Unicam::Image].getBuffers()->at(bufferId).get(); - LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << buffer->cookie() - << ", timestamp: " << buffer->metadata().timestamp; - isp_[Isp::Input].dev()->queueBuffer(buffer); - dropFrame_ = (action.operation == RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME) ? true : false; + dropFrame_ = (action.op_ == RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME) ? true : false; ispOutputCount_ = 0; break; } default: - LOG(RPI, Error) << "Unknown action " << action.operation; + LOG(RPI, Error) << "Unknown action " << action.op_; break; } @@ -1345,10 +1358,10 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) /* If this is a stats output, hand it to the IPA now. */ if (stream == &isp_[Isp::Stats]) { - IPAOperationData op; - op.operation = RPI_IPA_EVENT_SIGNAL_STAT_READY; - op.data = { RPiIpaMask::STATS | buffer->cookie() }; - ipa_->processEvent(op); + RPiEventParams ev; + ev.ev_ = RPI_IPA_EVENT_SIGNAL_STAT_READY; + ev.bufferId_ = { RPiIpaMask::STATS | buffer->cookie() }; + ipa_->processEvent(ev); } handleState(); @@ -1491,7 +1504,7 @@ void RPiCameraData::checkRequestCompleted() void RPiCameraData::tryRunPipeline() { FrameBuffer *bayerBuffer, *embeddedBuffer; - IPAOperationData op; + RPiEventParams ev; /* If any of our request or buffer queues are empty, we cannot proceed. */ if (state_ != State::Idle || requestQueue_.empty() || @@ -1546,9 +1559,9 @@ void RPiCameraData::tryRunPipeline() * queue the ISP output buffer listed in the request to start the HW * pipeline. */ - op.operation = RPI_IPA_EVENT_QUEUE_REQUEST; - op.controls = { request->controls() }; - ipa_->processEvent(op); + ev.ev_ = RPI_IPA_EVENT_QUEUE_REQUEST; + ev.controls_ = { request->controls() }; + ipa_->processEvent(ev); /* Queue up any ISP buffers passed into the request. */ for (auto &stream : isp_) { @@ -1567,10 +1580,10 @@ void RPiCameraData::tryRunPipeline() << " Bayer buffer id: " << bayerBuffer->cookie() << " Embedded buffer id: " << embeddedBuffer->cookie(); - op.operation = RPI_IPA_EVENT_SIGNAL_ISP_PREPARE; - op.data = { RPiIpaMask::EMBEDDED_DATA | embeddedBuffer->cookie(), - RPiIpaMask::BAYER_DATA | bayerBuffer->cookie() }; - ipa_->processEvent(op); + ev.ev_ = RPI_IPA_EVENT_SIGNAL_ISP_PREPARE; + ev.ispPrepare_.embeddedbufferId_ = RPiIpaMask::EMBEDDED_DATA | embeddedBuffer->cookie(); + ev.ispPrepare_.bayerbufferId_ = RPiIpaMask::BAYER_DATA | bayerBuffer->cookie(); + ipa_->processEvent(ev); } void RPiCameraData::tryFlushQueues() From patchwork Wed Aug 26 11:09:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9387 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 09F64BD87E for ; Wed, 26 Aug 2020 11:09:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C979E628F9; Wed, 26 Aug 2020 13:09:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CmK8ib4s"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B8C206037B for ; Wed, 26 Aug 2020 13:09:55 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D649F9CE; Wed, 26 Aug 2020 13:09:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440195; bh=sjG9vxRkOt3lgltbA6fyd10Wfr0WcwxEZFAHE+5wVbs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CmK8ib4swlSXI2CQ3Q4vpRuT+PHzl/1ioBLZ++0SlhZrOEtmIAkZYLdVT1z+6j2r8 Ys+psb36XHmDkrjivYDsO1Asnsj7nYb3S0v+3Qw1TanjAnhS4Uxs8crzsJn10zLx3j m2dECUfixLLXvzWDrOztlRZGgKD1cPcIy6CtdGXc= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:12 +0900 Message-Id: <20200826110926.67192-4-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 03/17] libcamera: IPA: raspberrypi: Use generated IPARPiInterface X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch shows how the IPA would use the generated IPA interface. Like the patch for the pipeline, this should be straightforward. It mainly switches to using the new data structures. A couple things that are noteworthy: ipaCreate returns an IPAInterface, instead of a struct ipa_context. This means we don't need the C interface anymore! File descriptors *cannot* be sent via ControlList and be expected to still work on the other side. This is because ControlList is expected to be sent during the stream. Every time an fd is sent across the process boundary, it is practically duped, which means we'll eventually run out of fds. Keeping a map of fds cannot be maintained practically either, since the map on the other side of the process boundary will not be notified of changes, among other nasty issues. Another big nasty issue is that we'll probably have a memory leak due to having so many fds open pointing to the same resource. The solution that I've come up with is to send one fd as a FileDescriptor from the pipeline handler that the IPA can use, and another fd as an int32, that the IPA can send back to the pipeline handler, and expect the pipeline handler to be able to use it. This is something to keep in mind during design of the IPA interface. Signed-off-by: Paul Elder --- src/ipa/raspberrypi/raspberrypi.cpp | 107 ++++++++++++++++------------ 1 file changed, 62 insertions(+), 45 deletions(-) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 4557016c..2c088742 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -19,6 +19,7 @@ #include #include #include +#include #include #include @@ -60,7 +61,7 @@ namespace libcamera { LOG_DEFINE_CATEGORY(IPARPI) -class IPARPi : public IPAInterface +class IPARPi : public IPARPiInterface { public: IPARPi() @@ -82,12 +83,12 @@ public: void configure(const CameraSensorInfo &sensorInfo, const std::map &streamConfig, - const std::map &entityControls, - const IPAOperationData &data, - IPAOperationData *response) override; + const std::map &entityControls, + const RPiConfigureParams &data, + RPiConfigureParams *response) override; void mapBuffers(const std::vector &buffers) override; void unmapBuffers(const std::vector &ids) override; - void processEvent(const IPAOperationData &event) override; + void processEvent(const RPiEventParams &event) override; private: void setMode(const CameraSensorInfo &sensorInfo); @@ -143,6 +144,11 @@ private: unsigned int mistrust_count_; /* LS table allocation passed in from the pipeline handler. */ FileDescriptor lsTableHandle_; + /* + * LS table allocation passed in from the pipeline handler, + * in the context of the pipeline handler. + */ + int32_t lsTableHandlePH_; void *lsTable_; }; @@ -192,15 +198,13 @@ void IPARPi::setMode(const CameraSensorInfo &sensorInfo) void IPARPi::configure(const CameraSensorInfo &sensorInfo, [[maybe_unused]] const std::map &streamConfig, - const std::map &entityControls, - const IPAOperationData &ipaConfig, - IPAOperationData *result) + const std::map &entityControls, + const RPiConfigureParams &ipaConfig, + RPiConfigureParams *result) { if (entityControls.empty()) return; - result->operation = 0; - unicam_ctrls_ = entityControls.at(0); isp_ctrls_ = entityControls.at(1); /* Setup a metadata ControlList to output metadata. */ @@ -222,11 +226,13 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, helper_->GetDelays(exposureDelay, gainDelay); sensorMetadata = helper_->SensorEmbeddedDataPresent(); - result->data.push_back(gainDelay); - result->data.push_back(exposureDelay); - result->data.push_back(sensorMetadata); + RPiConfigurePayload payload = {}; + payload.op_ = RPI_IPA_CONFIG_STAGGERED_WRITE; + payload.staggeredWriteResult_.gainDelay_ = gainDelay; + payload.staggeredWriteResult_.exposureDelay_ = exposureDelay; + payload.staggeredWriteResult_.sensorMetadata_ = sensorMetadata; - result->operation |= RPI_IPA_CONFIG_STAGGERED_WRITE; + result->payload_.push_back(payload); } /* Re-assemble camera mode using the sensor info. */ @@ -274,15 +280,23 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, if (agcStatus.shutter_time != 0.0 && agcStatus.analogue_gain != 0.0) { ControlList ctrls(unicam_ctrls_); applyAGC(&agcStatus, ctrls); - result->controls.push_back(ctrls); - result->operation |= RPI_IPA_CONFIG_SENSOR; + RPiConfigurePayload payload = {}; + payload.op_ = RPI_IPA_CONFIG_SENSOR; + payload.controls_ = ctrls; + + result->payload_.push_back(payload); } lastMode_ = mode_; /* Store the lens shading table pointer and handle if available. */ - if (ipaConfig.operation & RPI_IPA_CONFIG_LS_TABLE) { + auto lens = std::find_if(ipaConfig.payload_.begin(), + ipaConfig.payload_.end(), + [] (const RPiConfigurePayload &p) { + return p.op_ == RPI_IPA_CONFIG_LS_TABLE; + }); + if (lens != ipaConfig.payload_.end()) { /* Remove any previous table, if there was one. */ if (lsTable_) { munmap(lsTable_, MAX_LS_GRID_SIZE); @@ -290,7 +304,8 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, } /* Map the LS table buffer into user space. */ - lsTableHandle_ = FileDescriptor(ipaConfig.data[0]); + lsTableHandle_ = FileDescriptor(lens->lsTableHandle_); + lsTableHandlePH_ = lens->lsTableHandleStatic_; if (lsTableHandle_.isValid()) { lsTable_ = mmap(nullptr, MAX_LS_GRID_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, lsTableHandle_.fd(), 0); @@ -335,11 +350,11 @@ void IPARPi::unmapBuffers(const std::vector &ids) } } -void IPARPi::processEvent(const IPAOperationData &event) +void IPARPi::processEvent(const RPiEventParams &event) { - switch (event.operation) { + switch (event.ev_) { case RPI_IPA_EVENT_SIGNAL_STAT_READY: { - unsigned int bufferId = event.data[0]; + unsigned int bufferId = event.bufferId_; if (++check_count_ != frame_count_) /* assert here? */ LOG(IPARPI, Error) << "WARNING: Prepare/Process mismatch!!!"; @@ -348,17 +363,17 @@ void IPARPi::processEvent(const IPAOperationData &event) reportMetadata(); - IPAOperationData op; - op.operation = RPI_IPA_ACTION_STATS_METADATA_COMPLETE; - op.data = { bufferId & RPiIpaMask::ID }; - op.controls = { libcameraMetadata_ }; + RPiActionParams op; + op.op_ = RPI_IPA_ACTION_STATS_METADATA_COMPLETE; + op.statsComplete_.bufferId_ = { bufferId & RPiIpaMask::ID }; + op.statsComplete_.controls_ = { libcameraMetadata_ }; queueFrameAction.emit(0, op); break; } case RPI_IPA_EVENT_SIGNAL_ISP_PREPARE: { - unsigned int embeddedbufferId = event.data[0]; - unsigned int bayerbufferId = event.data[1]; + unsigned int embeddedbufferId = event.ispPrepare_.embeddedbufferId_; + unsigned int bayerbufferId = event.ispPrepare_.bayerbufferId_; /* * At start-up, or after a mode-switch, we may want to @@ -368,23 +383,23 @@ void IPARPi::processEvent(const IPAOperationData &event) prepareISP(embeddedbufferId); /* Ready to push the input buffer into the ISP. */ - IPAOperationData op; + RPiActionParams op; if (++frame_count_ > hide_count_) - op.operation = RPI_IPA_ACTION_RUN_ISP; + op.op_ = RPI_IPA_ACTION_RUN_ISP; else - op.operation = RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME; - op.data = { bayerbufferId & RPiIpaMask::ID }; + op.op_ = RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME; + op.bufferId_ = { bayerbufferId & RPiIpaMask::ID }; queueFrameAction.emit(0, op); break; } case RPI_IPA_EVENT_QUEUE_REQUEST: { - queueRequest(event.controls[0]); + queueRequest(event.controls_); break; } default: - LOG(IPARPI, Error) << "Unknown event " << event.operation; + LOG(IPARPI, Error) << "Unknown event " << event.ev_; break; } } @@ -489,6 +504,8 @@ void IPARPi::queueRequest(const ControlList &controls) /* Clear the return metadata buffer. */ libcameraMetadata_.clear(); + LOG(IPARPI, Info) << "Request ctrl length: " << controls.size(); + for (auto const &ctrl : controls) { LOG(IPARPI, Info) << "Request ctrl: " << controls::controls.at(ctrl.first)->name() @@ -698,9 +715,9 @@ void IPARPi::queueRequest(const ControlList &controls) void IPARPi::returnEmbeddedBuffer(unsigned int bufferId) { - IPAOperationData op; - op.operation = RPI_IPA_ACTION_EMBEDDED_COMPLETE; - op.data = { bufferId & RPiIpaMask::ID }; + RPiActionParams op; + op.op_ = RPI_IPA_ACTION_EMBEDDED_COMPLETE; + op.bufferId_ = { bufferId & RPiIpaMask::ID }; queueFrameAction.emit(0, op); } @@ -763,9 +780,9 @@ void IPARPi::prepareISP(unsigned int bufferId) applyDPC(dpcStatus, ctrls); if (!ctrls.empty()) { - IPAOperationData op; - op.operation = RPI_IPA_ACTION_V4L2_SET_ISP; - op.controls.push_back(ctrls); + RPiActionParams op; + op.op_ = RPI_IPA_ACTION_V4L2_SET_ISP; + op.controls_ = ctrls; queueFrameAction.emit(0, op); } } @@ -823,9 +840,9 @@ void IPARPi::processStats(unsigned int bufferId) ControlList ctrls(unicam_ctrls_); applyAGC(&agcStatus, ctrls); - IPAOperationData op; - op.operation = RPI_IPA_ACTION_V4L2_SET_STAGGERED; - op.controls.push_back(ctrls); + RPiActionParams op; + op.op_ = RPI_IPA_ACTION_V4L2_SET_STAGGERED; + op.controls_ = ctrls; queueFrameAction.emit(0, op); } } @@ -1056,7 +1073,7 @@ void IPARPi::applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls) .grid_width = w, .grid_stride = w, .grid_height = h, - .dmabuf = lsTableHandle_.fd(), + .dmabuf = lsTableHandlePH_, .ref_transform = 0, .corner_sampled = 1, .gain_format = GAIN_FORMAT_U4P10 @@ -1136,9 +1153,9 @@ const struct IPAModuleInfo ipaModuleInfo = { "raspberrypi", }; -struct ipa_context *ipaCreate() +IPAInterface *ipaCreate() { - return new IPAInterfaceWrapper(std::make_unique()); + return new IPARPi(); } }; /* extern "C" */ From patchwork Wed Aug 26 11:09:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9388 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AC099BD87E for ; Wed, 26 Aug 2020 11:09:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 76FEA628F9; Wed, 26 Aug 2020 13:09:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DuvBpjE1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8DD2C6037B for ; Wed, 26 Aug 2020 13:09:58 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2000B53C; Wed, 26 Aug 2020 13:09:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440198; bh=3I8Uvtq2XBRUqzTBotDWXcSdqJubJfUIb63/4k3Coo4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DuvBpjE1JlFfuHAAHw0ayeUuqsw1p8u1VkAuBnlnLkfZ7olTJ6go49DR3GAyC0U3C lOrOUF/e/wIOdJ+W86Pol9VylPs+dRUmgCEzmO8RZDxTkj39p2lBtGWdZ1mm82oEWz sd+ZFMQ8Hngn6ddQF+pSIMDdIQBwe5udEGp4Gv7U= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:13 +0900 Message-Id: <20200826110926.67192-5-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 04/17] IPA: IPC: add IPADataSerializer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch adds IPADataSerializer. It is a template class that implements de/serialization. ipa_data_serializer.h implements de/serialization for libcamera classes and primitives. This is handwritten, and if there is any demand for any more stuctures to be added, they must be added by hand. There was some debate as to whether this should use some well-known wire format, such as mojo or flatbuffers. As for a pre-made de/serializer, such as flatbuffers that I used in the last RFC, we noticed that I had to wrap the serialization anyway, so I decided I might as well just do the whole thing and throw flatbuffers out the window. To my knowledge, there is no serdes framework that can actually use generate serdes code for a pre-defined class, except mojo, which we can't use :) So since I had to implement manual serialization anyway, I could've followed a pre-existing standard. I didn't, but this can be changed. To get this RFC out I just did a simple wire format. I also think that the only merit to following a standard format is for performance, since we're going to be wrapping both sides of the IPC pipe in IPAProxy and IPAProxyWorker, so it's not like the world is going to see the wire format. raspberrypi_serializer.h will be generated based on the mojo file seen in the last patch. Other than that, it is conceptually the same as ipa_data_serializer.h. This header will be included by the generated IPA proxy and proxy worker so that they can de/serialize the data when then call into IPC. Signed-off-by: Paul Elder --- .../libcamera/internal/ipa_data_serializer.h | 816 ++++++++++++++++++ .../libcamera/ipa/raspberrypi_serializer.h | 487 +++++++++++ src/libcamera/ipa_data_serializer.cpp | 29 + 3 files changed, 1332 insertions(+) create mode 100644 include/libcamera/internal/ipa_data_serializer.h create mode 100644 include/libcamera/ipa/raspberrypi_serializer.h create mode 100644 src/libcamera/ipa_data_serializer.cpp diff --git a/include/libcamera/internal/ipa_data_serializer.h b/include/libcamera/internal/ipa_data_serializer.h new file mode 100644 index 00000000..368a398b --- /dev/null +++ b/include/libcamera/internal/ipa_data_serializer.h @@ -0,0 +1,816 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * ipa_data_serializer.h - Image Processing Algorithm data serializer + */ +#ifndef __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_H__ +#define __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_H__ + +#include +#include +#include +#include + +#include +#include +#include +#include + +#include "libcamera/internal/byte_stream_buffer.h" +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/control_serializer.h" +#include "libcamera/internal/log.h" + +#include + +template std::ostream &operator<<(std::ostream &stream, const std::vector &vec) +{ + stream << "{ "; + for (const T &v : vec) + stream << std::hex << std::setfill('0') << std::setw(2) << static_cast(v) << ", "; + + stream << " }"; + return stream; +} + +namespace libcamera { + +LOG_DECLARE_CATEGORY(IPADataSerializer) + +static void appendUint32(std::vector &vec, uint32_t val) +{ + for (int i = 0; i < 4; i++) + vec.push_back(static_cast((val >> 8*i) & 0xff)); +} + +static void appendUint64(std::vector &vec, uint64_t val) +{ + for (int i = 0; i < 8; i++) + vec.push_back(static_cast((val >> 8*i) & 0xff)); +} + +static uint32_t extractUint32(std::vector::iterator it) +{ + uint32_t ret = 0; + for (int i = 0; i < 4; i++) + ret |= *(it + i) << 8*i; + return ret; +} + +static uint64_t extractUint64(std::vector::iterator it) +{ + uint32_t ret = 0; + for (int i = 0; i < 8; i++) + ret |= *(it + i) << 8*i; + return ret; +} + +template +class IPADataSerializer +{ +}; + +template +class IPADataSerializer> +{ +public: + static std::tuple, std::vector> + serialize(const std::vector &data, ControlSerializer *cs = nullptr) + { + std::vector data_vec; + std::vector fds_vec; + + // serialize the length + uint32_t vec_len = data.size(); + appendUint32(data_vec, vec_len); + + // serialize the members + for (auto it = data.begin(); it != data.end(); ++it) { + std::vector dvec; + std::vector fvec; + + std::tie(dvec, fvec) = + IPADataSerializer::serialize(*it, cs); + + appendUint32(data_vec, dvec.size()); + appendUint32(data_vec, fvec.size()); + + data_vec.insert(data_vec.end(), dvec.begin(), dvec.end()); + fds_vec.insert(fds_vec.end(), fvec.begin(), fvec.end()); + } + + return {data_vec, fds_vec}; + } + + static std::vector deserialize(std::vector &data, ControlSerializer *cs = nullptr) + { + return IPADataSerializer>::deserialize(data.begin(), data.end(), cs); + } + + static std::vector deserialize(std::vector::iterator it1, + std::vector::iterator it2, + ControlSerializer *cs = nullptr) + { + std::vector fds; + return IPADataSerializer>::deserialize(it1, it2, + fds.begin(), fds.end(), + cs); + } + + static std::vector deserialize(std::vector &data, std::vector &fds, + ControlSerializer *cs = nullptr) + { + return IPADataSerializer>::deserialize(data.begin(), data.end(), + fds.begin(), fds.end(), + cs); + } + + static std::vector deserialize(std::vector::iterator data_it1, + [[maybe_unused]] std::vector::iterator data_it2, + std::vector::iterator fds_it1, + [[maybe_unused]] std::vector::iterator fds_it2, + ControlSerializer *cs = nullptr) + { + uint32_t vec_len = extractUint32(data_it1); + std::vector ret(vec_len); + + std::vector::iterator data_it = data_it1 + 4; + std::vector::iterator fd_it = fds_it1; + for (uint32_t i = 0; i < vec_len; i++) { + uint32_t sizeof_data = extractUint32(data_it); + uint32_t sizeof_fds = extractUint32(data_it + 4); + + ret[i] = IPADataSerializer::deserialize(data_it + 8, + data_it + 8 + sizeof_data, + fd_it, + fd_it + sizeof_fds, + cs); + + data_it += 8 + sizeof_data; + fd_it += sizeof_fds; + } + + return ret; + } +}; + +template +class IPADataSerializer> +{ +public: + static std::tuple, std::vector> + serialize(const std::map &data, ControlSerializer *cs = nullptr) + { + std::vector data_vec; + std::vector fds_vec; + + // serialize the length + uint32_t map_len = data.size(); + appendUint32(data_vec, map_len); + + // serialize the members + for (auto it = data.begin(); it != data.end(); ++it) { + std::vector dvec; + std::vector fvec; + + std::tie(dvec, fvec) = + IPADataSerializer::serialize(it->first, cs); + + appendUint32(data_vec, dvec.size()); + appendUint32(data_vec, fvec.size()); + + data_vec.insert(data_vec.end(), dvec.begin(), dvec.end()); + fds_vec.insert(fds_vec.end(), fvec.begin(), fvec.end()); + + std::tie(dvec, fvec) = + IPADataSerializer::serialize(it->second, cs); + + appendUint32(data_vec, dvec.size()); + appendUint32(data_vec, fvec.size()); + + data_vec.insert(data_vec.end(), dvec.begin(), dvec.end()); + fds_vec.insert(fds_vec.end(), fvec.begin(), fvec.end()); + } + + return {data_vec, fds_vec}; + } + + static std::map deserialize(std::vector &data, ControlSerializer *cs = nullptr) + { + return IPADataSerializer>::deserialize(data.begin(), data.end(), cs); + } + + static std::map deserialize(std::vector::iterator it1, + std::vector::iterator it2, + ControlSerializer *cs = nullptr) + { + std::vector fds; + return IPADataSerializer>::deserialize(it1, it2, + fds.begin(), fds.end(), + cs); + } + + static std::map deserialize(std::vector &data, std::vector &fds, + ControlSerializer *cs = nullptr) + { + return IPADataSerializer>::deserialize(data.begin(), data.end(), + fds.begin(), fds.end(), + cs); + } + + static std::map deserialize(std::vector::iterator data_it1, + [[maybe_unused]] std::vector::iterator data_it2, + std::vector::iterator fds_it1, + [[maybe_unused]] std::vector::iterator fds_it2, + ControlSerializer *cs = nullptr) + { + std::map ret; + + uint32_t map_len = extractUint32(data_it1); + + std::vector::iterator data_it = data_it1 + 4; + std::vector::iterator fd_it = fds_it1; + for (uint32_t i = 0; i < map_len; i++) { + uint32_t sizeof_data = extractUint32(data_it); + uint32_t sizeof_fds = extractUint32(data_it + 4); + + K key = IPADataSerializer::deserialize(data_it + 8, + data_it + 8 + sizeof_data, + fd_it, + fd_it + sizeof_fds, + cs); + + data_it += 8 + sizeof_data; + fd_it += sizeof_fds; + sizeof_data = extractUint32(data_it); + sizeof_fds = extractUint32(data_it + 4); + + const V value = IPADataSerializer::deserialize(data_it + 8, + data_it + 8 + sizeof_data, + fd_it, + fd_it + sizeof_fds, + cs); + ret.insert({key, value}); + + data_it += 8 + sizeof_data; + fd_it += sizeof_fds; + } + + return ret; + } +}; + +// TODO implement this for all primitives +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const unsigned int data, [[maybe_unused]] ControlSerializer *cs = nullptr) + { + std::vector data_vec; + appendUint32(data_vec, data); + + return {data_vec, {}}; + } + + static unsigned int deserialize(std::vector &data, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data.begin(), data.end()); + } + + static unsigned int deserialize(std::vector::iterator it1, + [[maybe_unused]] std::vector::iterator it2, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + return extractUint32(it1); + } + + static unsigned int deserialize(std::vector &data, + [[maybe_unused]] std::vector &fds, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data.begin(), data.end()); + } + + static unsigned int deserialize(std::vector::iterator data_it1, + std::vector::iterator data_it2, + [[maybe_unused]] std::vector::iterator fds_it1, + [[maybe_unused]] std::vector::iterator fds_it2, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data_it1, data_it2); + } +}; + +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const FileDescriptor &data, [[maybe_unused]] ControlSerializer *cs = nullptr) + { + std::vector data_vec = { data.isValid() }; + std::vector fd_vec; + if (data.isValid()) + fd_vec.push_back(data.fd()); + + return {data_vec, fd_vec}; + } + + static FileDescriptor deserialize(std::vector &data, std::vector &fds, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data.begin(), data.end(), + fds.begin(), fds.end()); + } + + static FileDescriptor deserialize(std::vector::iterator data_it1, + std::vector::iterator data_it2, + std::vector::iterator fds_it1, + std::vector::iterator fds_it2, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + if (std::distance(data_it1, data_it2) < 1) + LOG(IPADataSerializer, Fatal) + << "Invalid data to deserialize FileDescriptor"; + + bool valid = *data_it1; + + if (valid && std::distance(fds_it1, fds_it2) < 1) + LOG(IPADataSerializer, Fatal) + << "Invalid fds to deserialize FileDescriptor"; + + return valid ? FileDescriptor(*fds_it1) : FileDescriptor(); + } +}; + +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const IPASettings &data, [[maybe_unused]] ControlSerializer *cs = nullptr) + { + std::vector data_vec(data.configurationFile.begin(), + data.configurationFile.end()); + + return {data_vec, {}}; + } + + static IPASettings deserialize(std::vector &data, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data.begin(), data.end()); + } + + static IPASettings deserialize(std::vector::iterator it1, + std::vector::iterator it2, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + std::string str(it1, it2); + + IPASettings ret; + ret.configurationFile = str; + + return ret; + } + + static IPASettings deserialize(std::vector &data, + [[maybe_unused]] std::vector &fds, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data.begin(), data.end()); + } + + static IPASettings deserialize(std::vector::iterator data_it1, + std::vector::iterator data_it2, + [[maybe_unused]] std::vector::iterator fds_it1, + [[maybe_unused]] std::vector::iterator fds_it2, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data_it1, data_it2); + } +}; + +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const CameraSensorInfo &data, [[maybe_unused]] ControlSerializer *cs = nullptr) + { + std::vector data_vec; + + uint32_t str_len = data.model.size(); + appendUint32(data_vec, str_len); + + data_vec.insert(data_vec.end(), data.model.begin(), data.model.end()); + + appendUint32(data_vec, data.bitsPerPixel); + + appendUint32(data_vec, data.activeAreaSize.width); + appendUint32(data_vec, data.activeAreaSize.height); + + appendUint32(data_vec, static_cast(data.analogCrop.x)); + appendUint32(data_vec, static_cast(data.analogCrop.y)); + appendUint32(data_vec, data.analogCrop.width); + appendUint32(data_vec, data.analogCrop.height); + + appendUint32(data_vec, data.outputSize.width); + appendUint32(data_vec, data.outputSize.height); + + appendUint64(data_vec, data.pixelRate); + + appendUint32(data_vec, data.lineLength); + + return {data_vec, {}}; + } + + static CameraSensorInfo deserialize(std::vector &data, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data.begin(), data.end()); + } + + static CameraSensorInfo deserialize(std::vector::iterator it1, + [[maybe_unused]] std::vector::iterator it2, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + CameraSensorInfo ret; + + uint32_t str_len = extractUint32(it1); + std::string str(it1 + 4, it1 + 4 + str_len); + ret.model = str; + + std::vector::iterator it = it1 + 4 + str_len; + + ret.bitsPerPixel = extractUint32(it); + + ret.activeAreaSize.width = extractUint32(it + 4); + ret.activeAreaSize.height = extractUint32(it + 8); + + ret.analogCrop.x = static_cast(extractUint32(it + 12)); + ret.analogCrop.y = static_cast(extractUint32(it + 16)); + ret.analogCrop.width = extractUint32(it + 20); + ret.analogCrop.height = extractUint32(it + 24); + + ret.outputSize.width = extractUint32(it + 28); + ret.outputSize.height = extractUint32(it + 32); + + ret.pixelRate = extractUint64(it + 36); + + ret.lineLength = extractUint64(it + 44); + + return ret; + } + + static CameraSensorInfo deserialize(std::vector &data, + [[maybe_unused]] std::vector &fds, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data.begin(), data.end()); + } + + static CameraSensorInfo deserialize(std::vector::iterator data_it1, + std::vector::iterator data_it2, + [[maybe_unused]] std::vector::iterator fds_it1, + [[maybe_unused]] std::vector::iterator fds_it2, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data_it1, data_it2); + } +}; + +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const IPAStream &data, [[maybe_unused]] ControlSerializer *cs = nullptr) + { + std::vector data_vec; + + appendUint32(data_vec, data.pixelFormat); + + appendUint32(data_vec, data.size.width); + appendUint32(data_vec, data.size.height); + + return {data_vec, {}}; + } + + static IPAStream deserialize(std::vector &data, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data.begin(), data.end()); + } + + static IPAStream deserialize(std::vector::iterator it1, + [[maybe_unused]] std::vector::iterator it2, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + IPAStream ret; + + ret.pixelFormat = extractUint32(it1); + + ret.size.width = extractUint32(it1 + 4); + ret.size.height = extractUint32(it1 + 8); + + return ret; + } + + static IPAStream deserialize(std::vector &data, + [[maybe_unused]] std::vector &fds, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data.begin(), data.end()); + } + + static IPAStream deserialize(std::vector::iterator data_it1, + std::vector::iterator data_it2, + [[maybe_unused]] std::vector::iterator fds_it1, + [[maybe_unused]] std::vector::iterator fds_it2, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data_it1, data_it2); + } +}; + +template<> +class IPADataSerializer +{ +public: + // map arg will be generated, since it's per-pipeline anyway + static std::tuple, std::vector> + serialize(const ControlList &data, const ControlInfoMap &map, + ControlSerializer *cs) + { + if (!cs) + LOG(IPADataSerializer, Fatal) + << "ControlSerializer not provided for serialization of ControlList"; + + size_t size = cs->binarySize(map); + std::vector infoData(size); + ByteStreamBuffer buffer(infoData.data(), infoData.size()); + int ret = cs->serialize(map, buffer); + + if (ret < 0 || buffer.overflow()) { + std::cerr << "Failed to serialize ControlList's ControlInfoMap" << std::endl; + return {{}, {}}; + } + + size = cs->binarySize(data); + std::vector listData(size); + buffer = ByteStreamBuffer(listData.data(), listData.size()); + ret = cs->serialize(data, buffer); + + if (ret < 0 || buffer.overflow()) { + std::cerr << "Failed to serialize ControlList" << std::endl; + return {{}, {}}; + } + + std::vector data_vec; + appendUint32(data_vec, infoData.size()); + appendUint32(data_vec, listData.size()); + data_vec.insert(data_vec.end(), infoData.begin(), infoData.end()); + data_vec.insert(data_vec.end(), listData.begin(), listData.end()); + + return {data_vec, {}}; + } + + static ControlList deserialize(std::vector &data, ControlSerializer *cs) + { + return IPADataSerializer::deserialize(data.begin(), data.end(), cs); + } + + static ControlList deserialize(std::vector::iterator it1, + [[maybe_unused]] std::vector::iterator it2, + ControlSerializer *cs) + { + if (!cs) + LOG(IPADataSerializer, Fatal) + << "ControlSerializer not provided for deserialization of ControlList"; + + uint32_t infoDataSize = extractUint32(it1); + uint32_t listDataSize = extractUint32(it1 + 4); + + std::vector::iterator it = it1 + 8; + + std::vector infoData(it, it + infoDataSize); + std::vector listData(it + infoDataSize, it + infoDataSize + listDataSize); + + ByteStreamBuffer buffer(const_cast(infoData.data()), infoData.size()); + ControlInfoMap map = cs->deserialize(buffer); + if (map.empty() || buffer.overflow()) { + std::cerr << "Failed to deserialize ControlLists's ControlInfoMap" << std::endl; + return ControlList(); + } + + buffer = ByteStreamBuffer(const_cast(listData.data()), listData.size()); + ControlList list = cs->deserialize(buffer); + if (buffer.overflow()) + std::cerr << "Failed to deserialize ControlList: buffer overflow" << std::endl; + if (list.empty()) + std::cerr << "Failed to deserialize ControlList: empty list" << std::endl; + + return list; + } + + static ControlList deserialize(std::vector &data, + [[maybe_unused]] std::vector &fds, + ControlSerializer *cs) + { + return IPADataSerializer::deserialize(data.begin(), data.end(), cs); + } + + static ControlList deserialize(std::vector::iterator data_it1, + std::vector::iterator data_it2, + [[maybe_unused]] std::vector::iterator fds_it1, + [[maybe_unused]] std::vector::iterator fds_it2, + ControlSerializer *cs) + { + return IPADataSerializer::deserialize(data_it1, data_it2, cs); + } +}; + +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const ControlInfoMap &map, ControlSerializer *cs) + { + if (!cs) + LOG(IPADataSerializer, Fatal) + << "ControlSerializer not provided for serialization of ControlInfoMap"; + + size_t size = cs->binarySize(map); + std::vector infoData(size); + ByteStreamBuffer buffer(infoData.data(), infoData.size()); + int ret = cs->serialize(map, buffer); + + if (ret < 0 || buffer.overflow()) + return {{}, {}}; + + std::vector data_vec; + appendUint32(data_vec, infoData.size()); + data_vec.insert(data_vec.end(), infoData.begin(), infoData.end()); + + return {data_vec, {}}; + } + + static const ControlInfoMap deserialize(std::vector &data, + ControlSerializer *cs) + { + return IPADataSerializer::deserialize(data.begin(), data.end(), cs); + } + + static const ControlInfoMap deserialize(std::vector::iterator it1, + [[maybe_unused]] std::vector::iterator it2, + ControlSerializer *cs) + { + if (!cs) + LOG(IPADataSerializer, Fatal) + << "ControlSerializer not provided for deserialization of ControlInfoMap"; + + uint32_t infoDataSize = extractUint32(it1); + + std::vector::iterator it = it1 + 4; + + std::vector infoData(it, it + infoDataSize); + + ByteStreamBuffer buffer(const_cast(infoData.data()), infoData.size()); + const ControlInfoMap map = cs->deserialize(buffer); + + /* + ControlInfoMap::Map ctrls; + for (auto pair : map) + ctrls.emplace(controls::controls.at(pair.first->id()), + pair.second); + + ControlInfoMap ret = std::move(ctrls); + return ret; + */ + + return map; + } + + static const ControlInfoMap deserialize(std::vector &data, + [[maybe_unused]] std::vector &fds, + ControlSerializer *cs) + { + return IPADataSerializer::deserialize(data.begin(), data.end(), cs); + } + + static const ControlInfoMap deserialize(std::vector::iterator data_it1, + std::vector::iterator data_it2, + [[maybe_unused]] std::vector::iterator fds_it1, + [[maybe_unused]] std::vector::iterator fds_it2, + ControlSerializer *cs) + { + return IPADataSerializer::deserialize(data_it1, data_it2, cs); + } +}; + +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const FrameBuffer::Plane &data, [[maybe_unused]] ControlSerializer *cs = nullptr) + { + std::vector data_vec; + std::vector fds_vec; + + // fd + std::vector fdBuf; + std::vector fdFds; + std::tie(fdBuf, fdFds) = + IPADataSerializer::serialize(data.fd); + data_vec.insert(data_vec.end(), fdBuf.begin(), fdBuf.end()); + fds_vec.insert(fds_vec.end(), fdFds.begin(), fdFds.end()); + + // length + appendUint32(data_vec, data.length); + + return {data_vec, fds_vec}; + } + + static FrameBuffer::Plane deserialize(std::vector &data, + std::vector &fds, + ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data.begin(), data.end(), + fds.begin(), fds.end(), + cs); + } + + static FrameBuffer::Plane deserialize(std::vector::iterator data_it1, + [[maybe_unused]] std::vector::iterator data_it2, + std::vector::iterator fds_it1, + [[maybe_unused]] std::vector::iterator fds_it2, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + FrameBuffer::Plane ret; + + ret.fd = IPADataSerializer::deserialize(data_it1, data_it1 + 1, + fds_it1, fds_it1 + 1); + ret.length = extractUint32(data_it1 + 1); + + return ret; + } +}; + + +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const IPABuffer &data, ControlSerializer *cs = nullptr) + { + std::vector data_vec; + + appendUint32(data_vec, data.id); + + std::vector planes_data_vec; + std::vector planes_fds_vec; + std::tie(planes_data_vec, planes_fds_vec) = + IPADataSerializer>::serialize(data.planes, cs); + + data_vec.insert(data_vec.end(), planes_data_vec.begin(), planes_data_vec.end()); + + return {data_vec, planes_fds_vec}; + } + + static IPABuffer deserialize(std::vector &data, + std::vector &fds, + ControlSerializer *cs = nullptr) + { + return IPADataSerializer::deserialize(data.begin(), data.end(), + fds.begin(), fds.end(), cs); + } + + static IPABuffer deserialize(std::vector::iterator data_it1, + std::vector::iterator data_it2, + std::vector::iterator fds_it1, + std::vector::iterator fds_it2, + ControlSerializer *cs = nullptr) + { + IPABuffer ret; + + ret.id = extractUint32(data_it1); + + ret.planes = + IPADataSerializer>::deserialize( + data_it1 + 4, data_it2, fds_it1, fds_it2, cs); + + return ret; + } +}; + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_H__ */ diff --git a/include/libcamera/ipa/raspberrypi_serializer.h b/include/libcamera/ipa/raspberrypi_serializer.h new file mode 100644 index 00000000..01d69986 --- /dev/null +++ b/include/libcamera/ipa/raspberrypi_serializer.h @@ -0,0 +1,487 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * raspberrypi_serializer.h - Image Processing Algorithm data serializer for raspberry pi + */ + +// automatically generated by custom compiler + +#include +#include + +#include "libcamera/internal/ipa_data_serializer.h" + +#ifndef __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_RASPBERRYPI_H__ +#define __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_RASPBERRYPI_H__ + +namespace libcamera { + +LOG_DECLARE_CATEGORY(IPADataSerializer) + +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const RPiStaggeredWritePayload data, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + std::vector ret_data; + + // scalar gainDelay_ + appendUint32(ret_data, data.gainDelay_); + + // scalar exposureDelay_ + appendUint32(ret_data, data.exposureDelay_); + + // scalar sensorMetadata_ + appendUint32(ret_data, data.sensorMetadata_); + + return {ret_data, {}}; + } + + static RPiStaggeredWritePayload deserialize(std::vector &data, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + RPiStaggeredWritePayload ret; + std::vector::iterator m = data.begin(); + + // scalar gainDelay_ + ret.gainDelay_ = extractUint32(m); + m += 4; + + // scalar exposureDelay_ + ret.exposureDelay_ = extractUint32(m); + m += 4; + + // scalar sensorMetadata_ + ret.sensorMetadata_ = extractUint32(m); + + return ret; + } + + static RPiStaggeredWritePayload deserialize(std::vector::iterator it1, + std::vector::iterator it2, + ControlSerializer *cs = nullptr) + { + std::vector data(it1, it2); + return IPADataSerializer::deserialize(data, cs); + } +}; + +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const RPiIspPreparePayload data, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + std::vector ret_data; + + // scalar embeddedbufferId_ + appendUint32(ret_data, data.embeddedbufferId_); + + // scalar bayerbufferId_ + appendUint32(ret_data, data.bayerbufferId_); + + return {ret_data, {}}; + } + + static RPiIspPreparePayload deserialize(std::vector &data, + [[maybe_unused]] ControlSerializer *cs = nullptr) + { + RPiIspPreparePayload ret; + std::vector::iterator m = data.begin(); + + // scalar embeddedbufferId_ + ret.embeddedbufferId_ = extractUint32(m); + m += 4; + + // scalar bayerbufferId_ + ret.bayerbufferId_ = extractUint32(m); + + return ret; + } + + static RPiIspPreparePayload deserialize(std::vector::iterator it1, + std::vector::iterator it2, + ControlSerializer *cs = nullptr) + { + std::vector data(it1, it2); + return IPADataSerializer::deserialize(data, cs); + } +}; + +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const RPiStatsCompletePayload data, + ControlSerializer *cs) + { + std::vector ret_data; + + // scalar bufferId_ + appendUint32(ret_data, data.bufferId_); + + // ControlList controls_ + if (data.controls_.size() > 0) { + std::vector controls; + std::tie(controls, std::ignore) = + IPADataSerializer::serialize(data.controls_, + data.controls_.infoMap() ? *data.controls_.infoMap() : RPiControls, + cs); + appendUint32(ret_data, controls.size()); + ret_data.insert(ret_data.end(), controls.begin(), controls.end()); + } else { + appendUint32(ret_data, 0); + } + + + return {ret_data, {}}; + } + + static RPiStatsCompletePayload deserialize(std::vector &data, + ControlSerializer *cs) + { + RPiStatsCompletePayload ret; + std::vector::iterator m = data.begin(); + + // scalar bufferId_ + ret.bufferId_ = extractUint32(m); + m += 4; + + // ControlList controls_ + size_t controlsSize = extractUint32(m); + if (controlsSize > 0) + ret.controls_ = + IPADataSerializer::deserialize(m + 4, m + 4 + controlsSize, cs); + + return ret; + } + + static RPiStatsCompletePayload deserialize(std::vector::iterator it1, + std::vector::iterator it2, + ControlSerializer *cs) + { + std::vector data(it1, it2); + return IPADataSerializer::deserialize(data, cs); + } +}; + +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const RPiConfigurePayload data, + ControlSerializer *cs) + { + std::vector ret_data; + std::vector ret_fds; + + // scalar op_ + appendUint32(ret_data, data.op_); + + // fd lsTableHandle_ + std::vector lsTableHandle; + std::vector lsTableHandleFds; + std::tie(lsTableHandle, lsTableHandleFds) = + IPADataSerializer::serialize(data.lsTableHandle_); + ret_data.insert(ret_data.end(), lsTableHandle.begin(), lsTableHandle.end()); + ret_fds.insert(ret_fds.end(), lsTableHandleFds.begin(), lsTableHandleFds.end()); + + // scalar lsTableHandleStatic_ + appendUint32(ret_data, static_cast(data.lsTableHandleStatic_)); + + // struct staggeredWriteResult_ + std::vector staggeredWriteResult; + std::tie(staggeredWriteResult, std::ignore) = + IPADataSerializer::serialize(data.staggeredWriteResult_, cs); + appendUint32(ret_data, staggeredWriteResult.size()); + ret_data.insert(ret_data.end(), staggeredWriteResult.begin(), staggeredWriteResult.end()); + + // ControlList controls_ + if (data.controls_.size() > 0) { + std::vector controls; + std::tie(controls, std::ignore) = + IPADataSerializer::serialize(data.controls_, + data.controls_.infoMap() ? *data.controls_.infoMap() : RPiControls, + cs); + appendUint32(ret_data, controls.size()); + ret_data.insert(ret_data.end(), controls.begin(), controls.end()); + } else { + appendUint32(ret_data, 0); + } + + // scalar bufferFd_ + appendUint32(ret_data, static_cast(data.bufferFd_)); + + return {ret_data, ret_fds}; + } + + static RPiConfigurePayload deserialize(std::vector &data, + std::vector &fds, + ControlSerializer *cs) + { + RPiConfigurePayload ret; + std::vector::iterator m = data.begin(); + std::vector::iterator n = fds.begin(); + + // scalar op_ + ret.op_ = static_cast(extractUint32(m)); + m += 4; + + // fd lsTableHandle_ + ret.lsTableHandle_ = IPADataSerializer::deserialize(m, m + 1, n, n + 1); + m += 1; + n += ret.lsTableHandle_.isValid() ? 1 : 0; + + // scalar lsTableHandleStatic_ + ret.lsTableHandleStatic_ = static_cast(extractUint32(m)); + m += 4; + + // struct staggeredWriteResult_ + size_t staggeredWriteResultSize = extractUint32(m); + ret.staggeredWriteResult_ = + IPADataSerializer::deserialize(m + 4, m + 4 + staggeredWriteResultSize, cs); + m += 4 + staggeredWriteResultSize; + + // ControlList controls_ + size_t controlsSize = extractUint32(m); + if (controlsSize > 0) + ret.controls_ = + IPADataSerializer::deserialize(m + 4, m + 4 + controlsSize, cs); + m += 4 + controlsSize; + + // scalar bufferFd_ + ret.bufferFd_ = static_cast(extractUint32(m)); + + return ret; + } + + static RPiConfigurePayload deserialize(std::vector::iterator data_it1, + std::vector::iterator data_it2, + std::vector::iterator fds_it1, + std::vector::iterator fds_it2, + ControlSerializer *cs) + { + std::vector data(data_it1, data_it2); + std::vector fds(fds_it1, fds_it2); + return IPADataSerializer::deserialize(data, fds, cs); + } +}; + +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const RPiConfigureParams data, + ControlSerializer *cs) + { + std::vector ret_data; + std::vector ret_fds; + + // vector payload_ + // only member, so don't need size + std::tie(ret_data, ret_fds) = + IPADataSerializer>::serialize(data.payload_, cs); + + return {ret_data, ret_fds}; + } + + static RPiConfigureParams deserialize(std::vector &data, + std::vector &fds, + ControlSerializer *cs) + { + RPiConfigureParams ret; + + // vector payload_ + ret.payload_ = + IPADataSerializer>::deserialize(data, fds, cs); + + return ret; + } + + static RPiConfigureParams deserialize(std::vector::iterator data_it1, + std::vector::iterator data_it2, + std::vector::iterator fds_it1, + std::vector::iterator fds_it2, + ControlSerializer *cs) + { + std::vector data(data_it1, data_it2); + std::vector fds(fds_it1, fds_it2); + return IPADataSerializer::deserialize(data, fds, cs); + } +}; + +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const RPiEventParams data, + ControlSerializer *cs) + { + std::vector ret_data; + + // scalar ev_ + appendUint32(ret_data, data.ev_); + + // scalar bufferId_ + appendUint32(ret_data, data.bufferId_); + + // struct ispPrepare_ + std::vector ispPrepare; + std::tie(ispPrepare, std::ignore) = + IPADataSerializer::serialize(data.ispPrepare_, cs); + appendUint32(ret_data, ispPrepare.size()); + ret_data.insert(ret_data.end(), ispPrepare.begin(), ispPrepare.end()); + + // ControlList controls_ + if (data.controls_.size() > 0) { + std::vector controls; + std::tie(controls, std::ignore) = + IPADataSerializer::serialize(data.controls_, + data.controls_.infoMap() ? *data.controls_.infoMap() : RPiControls, + cs); + appendUint32(ret_data, controls.size()); + ret_data.insert(ret_data.end(), controls.begin(), controls.end()); + } else { + appendUint32(ret_data, 0); + } + + // scalar bufferFd_ + appendUint32(ret_data, static_cast(data.bufferFd_)); + + return {ret_data, {}}; + } + + static RPiEventParams deserialize(std::vector &data, + ControlSerializer *cs) + { + RPiEventParams ret; + std::vector::iterator m = data.begin(); + + // scalar ev_ + ret.ev_ = static_cast(extractUint32(m)); + m += 4; + + // scalar bufferId_ + ret.bufferId_ = extractUint32(m); + m += 4; + + // struct ispPrepare_ + size_t ispPrepareSize = extractUint32(m); + ret.ispPrepare_ = + IPADataSerializer::deserialize(m + 4, m + 4 + ispPrepareSize, cs); + m += 4 + ispPrepareSize; + + // ControlList controls_ + size_t controlsSize = extractUint32(m); + if (controlsSize > 0) + ret.controls_ = + IPADataSerializer::deserialize(m + 4, m + 4 + controlsSize, cs); + m += 4 + controlsSize; + + // scalar bufferFd_ + ret.bufferFd_ = static_cast(extractUint32(m)); + + return ret; + } + + static RPiEventParams deserialize(std::vector::iterator it1, + std::vector::iterator it2, + ControlSerializer *cs) + { + std::vector data(it1, it2); + return IPADataSerializer::deserialize(data, cs); + } + +}; + +template<> +class IPADataSerializer +{ +public: + static std::tuple, std::vector> + serialize(const RPiActionParams data, + ControlSerializer *cs) + { + std::vector ret_data; + + // scalar op_ + appendUint32(ret_data, data.op_); + + // scalar bufferId_ + appendUint32(ret_data, data.bufferId_); + + // struct statsComplete_ + std::vector statsComplete; + std::tie(statsComplete, std::ignore) = + IPADataSerializer::serialize(data.statsComplete_, cs); + appendUint32(ret_data, statsComplete.size()); + ret_data.insert(ret_data.end(), statsComplete.begin(), statsComplete.end()); + + // ControlList controls_ + if (data.controls_.size() > 0) { + std::vector controls; + std::tie(controls, std::ignore) = + IPADataSerializer::serialize(data.controls_, + data.controls_.infoMap() ? *data.controls_.infoMap() : RPiControls, + cs); + appendUint32(ret_data, controls.size()); + ret_data.insert(ret_data.end(), controls.begin(), controls.end()); + } else { + appendUint32(ret_data, 0); + } + + return {ret_data, {}}; + } + + static RPiActionParams deserialize(std::vector &data, + ControlSerializer *cs) + { + RPiActionParams ret; + std::vector::iterator m = data.begin(); + + // scalar op_ + ret.op_ = static_cast(extractUint32(m)); + m += 4; + + // scalar bufferId_ + ret.bufferId_ = extractUint32(m); + m += 4; + + // struct statsComplete_ + size_t statsCompleteSize = extractUint32(m); + ret.statsComplete_ = + IPADataSerializer::deserialize(m + 4, m + 4 + statsCompleteSize, cs); + m += 4 + statsCompleteSize; + + // ControlList controls_ + size_t controlsSize = extractUint32(m); + if (controlsSize > 0) + ret.controls_ = + IPADataSerializer::deserialize(m + 4, m + 4 + controlsSize, cs); + + return ret; + } + + static RPiActionParams deserialize(std::vector::iterator it1, + std::vector::iterator it2, + ControlSerializer *cs) + { + std::vector data(it1, it2); + return IPADataSerializer::deserialize(data, cs); + } + +}; + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_RASPBERRYPI_H__ */ diff --git a/src/libcamera/ipa_data_serializer.cpp b/src/libcamera/ipa_data_serializer.cpp new file mode 100644 index 00000000..86332abc --- /dev/null +++ b/src/libcamera/ipa_data_serializer.cpp @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * ipa_data_serializer.cpp - Image Processing Algorithm data serializer + */ + +#include "libcamera/internal/ipa_data_serializer.h" + +#include "libcamera/internal/log.h" + +/** + * \file ipa_ipa_data_serializer.h + * \brief IPA Data Serializer + */ + +namespace libcamera { + +LOG_DEFINE_CATEGORY(IPADataSerializer) + +/** + * \class IPADataSerializer + * \brief IPA Data Serializer + * + */ + +// TODO the rest of the documentation + +} /* namespace libcamera */ From patchwork Wed Aug 26 11:09:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9389 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4E259BD87E for ; Wed, 26 Aug 2020 11:10:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1A773628F9; Wed, 26 Aug 2020 13:10:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FCqb1Hz2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 43794628E6 for ; Wed, 26 Aug 2020 13:10:01 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 188719CE; Wed, 26 Aug 2020 13:09:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440201; bh=YjoOuE7v0XfTU6DmTt/8gVf6mg1AqOc4JKPOkzG9Ipg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FCqb1Hz2GSbLlSmRj7t1CoPVa0tWFmmaMpQoQduAhwOVsAtn2kSGRPgMzJhykn27J It3T3Mq3MVbPflv6xDNpShJdSC1Y+/U2Drsoh1cEqdWF0xyVJtWJ1P+A/v/s9+OsHS pZnGbXLa738DfHW7+cDFBsPlhpHcV+I1Kl902m2g= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:14 +0900 Message-Id: <20200826110926.67192-6-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 05/17] IPA: IPC: add IPAProxyRPi and IPAProxyRPiWorker X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch implements IPAProxyRPi and the corresponding proxy worker. Both of these are meant to be generated based on some template proxy, and the RPi IPA interface mojo definition. The IPAProxyRPi implements an IPARPiInterface, and internally does a switch on if isolation is enabled or not, and does either a thread call into the IPARPi directly, or sends a message over IPC. The IPC mechanism is implemented in an IPAIPC, and can be swapped out (though swapping out isn't enabled yet, as we only have one IPC mechanism for now). For this reason, the proxy (and worker) is responsible for calling the de/serializer. On the other side, the proxy worker listens on the socket (in the case of IPAIPCUnixSocket), and does the switch-case on the cmd to call the appropriate function from the linked IPARPi. For callbacks, since they're implemented as signals, we simply propagate the signal emission (except when crossing the process boundary, which has a send-receive). Signed-off-by: Paul Elder --- .../internal/ipa_proxy_raspberrypi.h | 117 ++++++ src/libcamera/proxy/ipa_proxy_raspberrypi.cpp | 381 ++++++++++++++++++ .../worker/ipa_proxy_raspberrypi_worker.cpp | 355 ++++++++++++++++ 3 files changed, 853 insertions(+) create mode 100644 include/libcamera/internal/ipa_proxy_raspberrypi.h create mode 100644 src/libcamera/proxy/ipa_proxy_raspberrypi.cpp create mode 100644 src/libcamera/proxy/worker/ipa_proxy_raspberrypi_worker.cpp diff --git a/include/libcamera/internal/ipa_proxy_raspberrypi.h b/include/libcamera/internal/ipa_proxy_raspberrypi.h new file mode 100644 index 00000000..bc2001ad --- /dev/null +++ b/include/libcamera/internal/ipa_proxy_raspberrypi.h @@ -0,0 +1,117 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * ipa_proxy_raspberrypi.h - Image Processing Algorithm proxy for Raspberry pi + */ +#ifndef __LIBCAMERA_INTERNAL_IPA_PROXY_RASPBERRYPI_H__ +#define __LIBCAMERA_INTERNAL_IPA_PROXY_RASPBERRYPI_H__ + +// automatically generated by custom compiler + +// TODO move this to a proxy header directory + +#include +#include +#include + +#include "libcamera/internal/ipa_ipc.h" +#include "libcamera/internal/ipa_proxy.h" +#include "libcamera/internal/ipc_unixsocket.h" +#include "libcamera/internal/thread.h" + +namespace libcamera { + +class IPAProxyRPi : public IPAProxy, public IPARPiInterface +{ +public: + IPAProxyRPi(IPAModule *ipam, bool isolate); + ~IPAProxyRPi(); + + int init(const IPASettings &settings) override; + int start() override; + void stop() override; + void configure(const CameraSensorInfo &sensorInfo, + const std::map &streamConfig, + const std::map &entityControls, + const RPiConfigureParams &ipaConfig, + RPiConfigureParams *result) override; + void mapBuffers(const std::vector &buffers) override; + void unmapBuffers(const std::vector &ids) override; + void processEvent(const RPiEventParams &event) override; + + Signal queueFrameAction; + +private: + void recvIPC(std::vector &data, std::vector &fds); + + int initThread(const IPASettings &settings); + int startThread(); + void stopThread(); + void configureThread(const CameraSensorInfo &sensorInfo, + const std::map &streamConfig, + const std::map &entityControls, + const RPiConfigureParams &ipaConfig, + RPiConfigureParams *result); + void mapBuffersThread(const std::vector &buffers); + void unmapBuffersThread(const std::vector &ids); + void processEventThread(const RPiEventParams &event); + void frameActionHandlerThread(unsigned int frame, const RPiActionParams &data); + + int initIPC(const IPASettings &settings); + int startIPC(); + void stopIPC(); + void configureIPC(const CameraSensorInfo &sensorInfo, + const std::map &streamConfig, + const std::map &entityControls, + const RPiConfigureParams &ipaConfig, + RPiConfigureParams *result); + void mapBuffersIPC(const std::vector &buffers); + void unmapBuffersIPC(const std::vector &ids); + void processEventIPC(const RPiEventParams &event); + void frameActionHandlerIPC(std::vector &data, + std::vector &fds); + + /* Helper class to invoke processEvent() in another thread. */ + class ThreadProxy : public Object + { + public: + void setIPA(IPARPiInterface *ipa) + { + ipa_ = ipa; + } + + int start() + { + return ipa_->start(); + } + + void stop() + { + ipa_->stop(); + } + + void processEvent(const RPiEventParams &event) + { + ipa_->processEvent(event); + } + + private: + IPARPiInterface *ipa_; + }; + + bool running_; + Thread thread_; + ThreadProxy proxy_; + std::unique_ptr ipa_; + + const bool isolate_; + + std::unique_ptr ipc_; + + ControlSerializer controlSerializer_; +}; + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_INTERNAL_IPA_PROXY_RASPBERRYPI_H__ */ diff --git a/src/libcamera/proxy/ipa_proxy_raspberrypi.cpp b/src/libcamera/proxy/ipa_proxy_raspberrypi.cpp new file mode 100644 index 00000000..6526fb1d --- /dev/null +++ b/src/libcamera/proxy/ipa_proxy_raspberrypi.cpp @@ -0,0 +1,381 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * ipa_proxy_raspberrypi.cpp - Image Processing Algorithm proxy for Raspberry pi + */ + +// automatically generated by custom compiler + +#include + +#include +#include +#include +#include +#include + +#include "libcamera/internal/control_serializer.h" +#include "libcamera/internal/ipa_ipc.h" +#include "libcamera/internal/ipa_data_serializer.h" +#include "libcamera/internal/ipa_module.h" +#include "libcamera/internal/ipa_proxy.h" +#include "libcamera/internal/ipc_unixsocket.h" +#include "libcamera/internal/log.h" +#include "libcamera/internal/process.h" +#include "libcamera/internal/thread.h" + +#include "libcamera/internal/ipa_proxy_raspberrypi.h" + +// just inline this at code generation +#define CALL_THREAD_OR_IPC(isolate, thread_func, ipc_func, ...) \ + (isolate ? ipc_func(__VA_ARGS__) : thread_func(__VA_ARGS__)) + +namespace libcamera { + +LOG_DECLARE_CATEGORY(IPAProxy) + +// TODO yeah we really need to move this into a header somewhere +void appendUInt32(std::vector &vec, uint32_t val) +{ + uint8_t arr[] = {static_cast(val & 0xff), + static_cast((val >> 8) & 0xff), + static_cast((val >> 16) & 0xff), + static_cast((val >> 24) & 0xff)}; + vec.insert(vec.end(), arr, arr + 4); +} + +IPAProxyRPi::IPAProxyRPi(IPAModule *ipam, bool isolate) + : IPAProxy(ipam), running_(false), + isolate_(isolate) +{ + LOG(IPAProxy, Debug) + << "initializing dummy proxy: loading IPA from " + << ipam->path(); + + if (isolate_) { + const std::string proxy_worker_path = resolvePath("ipa_proxy_raspberrypi"); + if (proxy_worker_path.empty()) { + LOG(IPAProxy, Error) + << "Failed to get proxy worker path"; + return; + } + + IPAIPCFactory *ipcf = nullptr; + for (IPAIPCFactory *factory : IPAIPCFactory::factories()) { + if (!strcmp(factory->name().c_str(), "IPAIPCUnixSocket")) { + ipcf = factory; + break; + } + } + + if (!ipcf) { + LOG(IPAProxy, Error) << "Failed to get IPAIPC factory"; + return; + } + + ipc_ = ipcf->create(ipam->path().c_str(), proxy_worker_path.c_str()); + if (!ipc_->isValid()) { + LOG(IPAProxy, Error) << "Failed to create IPAIPC"; + return; + } + + ipc_->recvIPC.connect(this, &IPAProxyRPi::recvIPC); + + valid_ = true; + return; + } + + if (!ipam->load()) + return; + + IPAInterface *ipai = ipam->createInterface(); + if (!ipai) { + LOG(IPAProxy, Error) + << "Failed to create IPA context for " << ipam->path(); + return; + } + + ipa_ = std::unique_ptr(dynamic_cast(ipai)); + proxy_.setIPA(ipa_.get()); + + /* + * Proxy the queueFrameAction signal to dispatch it in the caller's + * thread. + */ + ipa_->queueFrameAction.connect(this, &IPAProxyRPi::frameActionHandlerThread); + + valid_ = true; +} + +IPAProxyRPi::~IPAProxyRPi() +{ + if (isolate_) + ipc_->sendAsync(CMD_EXIT, {}, {}); +} + +void IPAProxyRPi::recvIPC(std::vector &data, std::vector &fds) +{ + uint32_t cmd = (data[0] & 0xff) | + ((data[1] & 0xff) << 8) | + ((data[2] & 0xff) << 16) | + ((data[3] & 0xff) << 24); + + /* Need to skip another 4 bytes for the sequence number. */ + std::vector vec(data.begin() + 8, data.end()); + + switch (cmd) { + case CMD_QUEUEFRAMEACTION: { + frameActionHandlerIPC(vec, fds); + break; + } + } +} + +int IPAProxyRPi::init(const IPASettings &settings) +{ + return CALL_THREAD_OR_IPC(isolate_, initThread, initIPC, settings); +} + +int IPAProxyRPi::start() +{ + return CALL_THREAD_OR_IPC(isolate_, startThread, startIPC); +} + +void IPAProxyRPi::stop() +{ + CALL_THREAD_OR_IPC(isolate_, stopThread, stopIPC); +} + +void IPAProxyRPi::configure(const CameraSensorInfo &sensorInfo, + const std::map &streamConfig, + const std::map &entityControls, + const RPiConfigureParams &ipaConfig, + RPiConfigureParams *result) +{ + CALL_THREAD_OR_IPC(isolate_, configureThread, configureIPC, + sensorInfo, streamConfig, entityControls, ipaConfig, result); +} + +void IPAProxyRPi::mapBuffers(const std::vector &buffers) +{ + CALL_THREAD_OR_IPC(isolate_, mapBuffersThread, mapBuffersIPC, buffers); +} + +void IPAProxyRPi::unmapBuffers(const std::vector &ids) +{ + CALL_THREAD_OR_IPC(isolate_, unmapBuffersThread, unmapBuffersIPC, ids); +} + +void IPAProxyRPi::processEvent(const RPiEventParams &event) +{ + CALL_THREAD_OR_IPC(isolate_, processEventThread, processEventIPC, event); +} + +/* Thread functions */ + +int IPAProxyRPi::initThread(const IPASettings &settings) +{ + int ret = ipa_->init(settings); + if (ret) + return ret; + + proxy_.moveToThread(&thread_); + + return 0; +} + +int IPAProxyRPi::startThread() +{ + running_ = true; + thread_.start(); + + return proxy_.invokeMethod(&ThreadProxy::start, ConnectionTypeBlocking); +} + +void IPAProxyRPi::stopThread() +{ + if (!running_) + return; + + running_ = false; + + proxy_.invokeMethod(&ThreadProxy::stop, ConnectionTypeBlocking); + + thread_.exit(); + thread_.wait(); +} + +void IPAProxyRPi::configureThread(const CameraSensorInfo &sensorInfo, + const std::map &streamConfig, + const std::map &entityControls, + const RPiConfigureParams &ipaConfig, + RPiConfigureParams *result) +{ + ipa_->configure(sensorInfo, streamConfig, entityControls, ipaConfig, + result); +} + +void IPAProxyRPi::mapBuffersThread(const std::vector &buffers) +{ + ipa_->mapBuffers(buffers); +} + +void IPAProxyRPi::unmapBuffersThread(const std::vector &ids) +{ + ipa_->unmapBuffers(ids); +} + +void IPAProxyRPi::processEventThread(const RPiEventParams &event) +{ + if (!running_) + return; + + /* Dispatch the processEvent() call to the thread. */ + proxy_.invokeMethod(&ThreadProxy::processEvent, ConnectionTypeQueued, + event); +} + +void IPAProxyRPi::frameActionHandlerThread(unsigned int frame, const RPiActionParams &data) +{ + queueFrameAction.emit(frame, data); +} + +/* IPC functions */ + +int IPAProxyRPi::initIPC(const IPASettings &settings) +{ + std::vector buf; + std::tie(buf, std::ignore) = IPADataSerializer::serialize(settings); + std::vector resultBuf; + + int ret = ipc_->sendSync(CMD_INIT, buf, {}, &resultBuf); + if (ret < 0) { + LOG(IPAProxy, Error) << "Failed to call init"; + return ret; + } + LOG(IPAProxy, Info) << "Done calling init"; + + /* Maybe we should cast this properly? */ + return static_cast(resultBuf[0]); +} + +int IPAProxyRPi::startIPC() +{ + std::vector resultBuf; + + int ret = ipc_->sendSync(CMD_START, {}, {}, &resultBuf); + if (ret < 0) { + LOG(IPAProxy, Error) << "Failed to call start"; + return ret; + } + LOG(IPAProxy, Info) << "Done calling start"; + + /* Maybe we should cast this properly? */ + return static_cast(resultBuf[0]); +} + +void IPAProxyRPi::stopIPC() +{ + int ret = ipc_->sendSync(CMD_STOP, {}, {}); + if (ret < 0) + LOG(IPAProxy, Error) << "Failed to call stop"; + LOG(IPAProxy, Info) << "Done calling stop"; +} + +void IPAProxyRPi::configureIPC(const CameraSensorInfo &sensorInfo, + const std::map &streamConfig, + const std::map &entityControls, + const RPiConfigureParams &ipaConfig, + RPiConfigureParams *result) +{ + std::vector sensorInfoBuf; + std::tie(sensorInfoBuf, std::ignore) = + IPADataSerializer::serialize(sensorInfo); + std::vector streamConfigBuf; + std::tie(streamConfigBuf, std::ignore) = + IPADataSerializer>::serialize(streamConfig); + std::vector entityControlsBuf; + std::tie(entityControlsBuf, std::ignore) = + IPADataSerializer>::serialize(entityControls, &controlSerializer_); + std::vector ipaConfigBuf; + std::vector ipaConfigFds; + std::tie(ipaConfigBuf, ipaConfigFds) = + IPADataSerializer::serialize(ipaConfig, &controlSerializer_); + + std::vector resultBuf; + std::vector resultFds; + + std::vector input; + appendUInt32(input, sensorInfoBuf.size()); + appendUInt32(input, streamConfigBuf.size()); + appendUInt32(input, entityControlsBuf.size()); + appendUInt32(input, ipaConfigBuf.size()); + appendUInt32(input, ipaConfigFds.size()); + input.insert(input.end(), sensorInfoBuf.begin(), sensorInfoBuf.end()); + input.insert(input.end(), streamConfigBuf.begin(), streamConfigBuf.end()); + input.insert(input.end(), entityControlsBuf.begin(), entityControlsBuf.end()); + input.insert(input.end(), ipaConfigBuf.begin(), ipaConfigBuf.end()); + + std::vector fds; + fds.insert(fds.end(), ipaConfigFds.begin(), ipaConfigFds.end()); + + int ret = ipc_->sendSync(CMD_CONFIGURE, input, fds, &resultBuf, &resultFds); + if (ret < 0) { + LOG(IPAProxy, Error) << "Failed to call configure"; + return; + } + LOG(IPAProxy, Info) << "Done calling configure"; + + *result = IPADataSerializer::deserialize(resultBuf, resultFds, &controlSerializer_); +} + +void IPAProxyRPi::mapBuffersIPC(const std::vector &buffers) +{ + std::vector buffersBuf; + std::vector fdsBuf; + std::tie(buffersBuf, fdsBuf) = + IPADataSerializer>::serialize(buffers); + + int ret = ipc_->sendSync(CMD_MAPBUFFERS, buffersBuf, fdsBuf); + if (ret < 0) + LOG(IPAProxy, Error) << "Failed to call mapBuffers"; + LOG(IPAProxy, Info) << "Done calling mapBuffers"; +} + +void IPAProxyRPi::unmapBuffersIPC(const std::vector &ids) +{ + std::vector idsBuf; + std::tie(idsBuf, std::ignore) = + IPADataSerializer>::serialize(ids); + int ret = ipc_->sendSync(CMD_UNMAPBUFFERS, idsBuf, {}); + if (ret < 0) + LOG(IPAProxy, Error) << "Failed to call unmapBuffers"; + LOG(IPAProxy, Info) << "Done calling unmapBuffers"; +} + +void IPAProxyRPi::processEventIPC(const RPiEventParams &event) +{ + std::vector eventBuf; + std::tie(eventBuf, std::ignore) = + IPADataSerializer::serialize(event, &controlSerializer_); + int ret = ipc_->sendAsync(CMD_PROCESSEVENT, eventBuf, {}); + if (ret < 0) + LOG(IPAProxy, Error) << "Failed to call processEvent"; + LOG(IPAProxy, Info) << "Done calling processEvent"; +} + +void IPAProxyRPi::frameActionHandlerIPC(std::vector &data, + [[maybe_unused]] std::vector &fds) +{ + uint32_t frame = data[0] | (data[1] << 8) | (data[2] << 16) | (data[3] << 24); + RPiActionParams params = + IPADataSerializer::deserialize(data.begin() + 4, data.end(), &controlSerializer_); + + queueFrameAction.emit(frame, params); +} + + +REGISTER_IPA_PROXY(IPAProxyRPi) + +} /* namespace libcamera */ diff --git a/src/libcamera/proxy/worker/ipa_proxy_raspberrypi_worker.cpp b/src/libcamera/proxy/worker/ipa_proxy_raspberrypi_worker.cpp new file mode 100644 index 00000000..ad917756 --- /dev/null +++ b/src/libcamera/proxy/worker/ipa_proxy_raspberrypi_worker.cpp @@ -0,0 +1,355 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * ipa_proxy_raspberrypi_worker.cpp - Image Processing Algorithm proxy worker for Raspberry pi + */ + +// automatically generated by custom compiler + +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/control_serializer.h" +#include "libcamera/internal/ipa_data_serializer.h" +#include "libcamera/internal/ipa_module.h" +#include "libcamera/internal/ipc_unixsocket.h" +#include "libcamera/internal/log.h" +#include "libcamera/internal/thread.h" + +using namespace libcamera; + +LOG_DEFINE_CATEGORY(IPAProxyRPiWorker) + +struct CallData { + IPCUnixSocket::Payload *response; + bool done; +}; + +IPARPiInterface *ipa_; +IPCUnixSocket socket_; +std::map callData_; + +ControlSerializer controlSerializer_; + +bool exit_ = false; + +void writeHeader(IPCUnixSocket::Payload &payload, uint32_t cmd, uint32_t seq) +{ + uint8_t cmd_arr[] = {static_cast(cmd & 0xff), + static_cast((cmd >> 8 ) & 0xff), + static_cast((cmd >> 16) & 0xff), + static_cast((cmd >> 24) & 0xff)}; + uint8_t seq_arr[] = {static_cast(seq & 0xff), + static_cast((seq >> 8 ) & 0xff), + static_cast((seq >> 16) & 0xff), + static_cast((seq >> 24) & 0xff)}; + payload.data.insert(payload.data.begin(), cmd_arr, cmd_arr+4); + payload.data.insert(payload.data.begin() + 4, seq_arr, seq_arr+4); +} + +// cmd, seq +std::tuple readHeader(IPCUnixSocket::Payload &payload) +{ + uint32_t cmd = (payload.data[0] & 0xff) | + ((payload.data[1] & 0xff) << 8) | + ((payload.data[2] & 0xff) << 16) | + ((payload.data[3] & 0xff) << 24); + uint32_t seq = (payload.data[4] & 0xff) | + ((payload.data[5] & 0xff) << 8) | + ((payload.data[6] & 0xff) << 16) | + ((payload.data[7] & 0xff) << 24); + + return {cmd, seq}; +} + +void eraseHeader(IPCUnixSocket::Payload &payload) +{ + payload.data.erase(payload.data.begin(), payload.data.begin() + 8); +} + +void writeUInt32(IPCUnixSocket::Payload &payload, uint32_t val, uint32_t pos) +{ + if (pos + 4 > payload.data.size()) + payload.data.resize(pos + 4); + + uint8_t arr[] = {static_cast(val & 0xff), + static_cast((val >> 8 ) & 0xff), + static_cast((val >> 16) & 0xff), + static_cast((val >> 24) & 0xff)}; + + std::copy(arr, arr + 4, payload.data.begin() + pos); +} + +uint32_t readUInt32(IPCUnixSocket::Payload &payload, uint32_t pos) +{ + if (pos + 4 > payload.data.size()) + return 0; + + return (payload.data[pos] & 0xff) | + ((payload.data[pos + 1] & 0xff) << 8) | + ((payload.data[pos + 2] & 0xff) << 16) | + ((payload.data[pos + 3] & 0xff) << 24); +} + +void readyRead(IPCUnixSocket *socket) +{ + IPCUnixSocket::Payload message, response; + int ret = socket->receive(&message); + if (ret) { + LOG(IPAProxyRPiWorker, Error) + << "Receive message failed" << ret; + return; + } + + uint32_t cmd, seq; + std::tie(cmd, seq) = readHeader(message); + eraseHeader(message); + + switch (cmd) { + case CMD_INIT: { + IPASettings settings = IPADataSerializer::deserialize(message.data); + + int ret = ipa_->init(settings); + writeHeader(response, cmd, seq); + /* Maybe we should do this cast properly? */ + response.data.push_back(static_cast(ret)); + + ret = socket_.send(response); + if (ret < 0) { + LOG(IPAProxyRPiWorker, Error) + << "Reply to init() failed" << ret; + } + + LOG(IPAProxyRPiWorker, Info) << "Done replying to init()"; + break; + } + + case CMD_EXIT: { + exit_ = true; + break; + } + + case CMD_START: { + int ret = ipa_->start(); + writeHeader(response, cmd, seq); + /* Maybe we should do this cast properly? */ + response.data.push_back(static_cast(ret)); + + ret = socket_.send(response); + if (ret < 0) { + LOG(IPAProxyRPiWorker, Error) + << "Reply to start() failed" << ret; + } + + LOG(IPAProxyRPiWorker, Info) << "Done replying to start()"; + break; + } + + case CMD_STOP: { + ipa_->stop(); + + writeHeader(response, cmd, seq); + int ret = socket_.send(response); + if (ret < 0) { + LOG(IPAProxyRPiWorker, Error) + << "Reply to stop() failed" << ret; + } + + LOG(IPAProxyRPiWorker, Info) << "Done replying to stop()"; + break; + } + + case CMD_CONFIGURE: { + LOG(IPAProxyRPiWorker, Info) << "Starting configure()"; + + size_t sensorInfoSize = readUInt32(message, 0); + size_t streamConfigSize = readUInt32(message, 4); + size_t entityControlsSize = readUInt32(message, 8); + + // TODO make this pattern more like the data ones + // - this will be fixed in the compiler + // remember to fix these offsets too + size_t ipaConfigFdsSize = readUInt32(message, 16); + + size_t sensorInfoStart = 20; + size_t streamConfigStart = sensorInfoStart + sensorInfoSize; + size_t entityControlsStart = streamConfigStart + streamConfigSize; + size_t ipaConfigStart = entityControlsStart + entityControlsSize; + + LOG(IPAProxyRPiWorker, Info) << " deserializing sensorInfo..."; + struct CameraSensorInfo sensorInfo = + IPADataSerializer::deserialize( + message.data.begin() + sensorInfoStart, + message.data.begin() + streamConfigStart); + LOG(IPAProxyRPiWorker, Info) << " deserializing streamConfig..."; + std::map streamConfig = + IPADataSerializer>::deserialize( + message.data.begin() + streamConfigStart, + message.data.begin() + entityControlsStart); + LOG(IPAProxyRPiWorker, Info) << " deserializing entityControls..."; + const std::map entityControls = + IPADataSerializer>::deserialize( + message.data.begin() + entityControlsStart, + message.data.begin() + ipaConfigStart, + &controlSerializer_); + LOG(IPAProxyRPiWorker, Info) << " deserializing ipaConfig..."; + RPiConfigureParams ipaConfig = + IPADataSerializer::deserialize( + message.data.begin() + ipaConfigStart, + message.data.end(), + message.fds.begin(), + message.fds.begin() + ipaConfigFdsSize, + &controlSerializer_); + RPiConfigureParams results; + + LOG(IPAProxyRPiWorker, Info) << "Calling IPA's configure()"; + ipa_->configure(sensorInfo, streamConfig, entityControls, ipaConfig, &results); + LOG(IPAProxyRPiWorker, Info) << "Done calling IPA's configure()"; + + std::vector resultsSerialized; + std::vector resultsFds; + std::tie(resultsSerialized, resultsFds) = + IPADataSerializer::serialize(results, &controlSerializer_); + + writeHeader(response, cmd, seq); + response.data.insert(response.data.end(), + resultsSerialized.begin(), resultsSerialized.end()); + response.fds.insert(response.fds.end(), + resultsFds.begin(), resultsFds.end()); + LOG(IPAProxyRPiWorker, Info) << "Sending response to configure()"; + int ret = socket_.send(response); + if (ret < 0) { + LOG(IPAProxyRPiWorker, Error) + << "Reply to configure() failed" << ret; + } + + LOG(IPAProxyRPiWorker, Info) << "Done replying to configure()"; + break; + } + + case CMD_MAPBUFFERS: { + std::vector ipaBuffers = + IPADataSerializer>::deserialize(message.data, message.fds); + ipa_->mapBuffers(ipaBuffers); + + writeHeader(response, cmd, seq); + int ret = socket_.send(response); + if (ret < 0) { + LOG(IPAProxyRPiWorker, Error) + << "Reply to mapBuffers() failed" << ret; + } + + LOG(IPAProxyRPiWorker, Info) << "Done replying to mapBuffers()"; + break; + } + + case CMD_UNMAPBUFFERS: { + std::vector ids = + IPADataSerializer>::deserialize(message.data); + ipa_->unmapBuffers(ids); + + writeHeader(response, cmd, seq); + int ret = socket_.send(response); + if (ret < 0) { + LOG(IPAProxyRPiWorker, Error) + << "Reply to unmapBuffers() failed" << ret; + } + + LOG(IPAProxyRPiWorker, Info) << "Done replying to unmapBuffers()"; + break; + } + + case CMD_PROCESSEVENT: { + RPiEventParams ev = + IPADataSerializer::deserialize(message.data, &controlSerializer_); + ipa_->processEvent(ev); + + LOG(IPAProxyRPiWorker, Info) << "Done processEvent()ing"; + break; + } + + } + + return; +} + +void queueFrameAction(unsigned int frame, const RPiActionParams &action) +{ + LOG(IPAProxyRPiWorker, Info) << "queueFrameAction triggered"; + IPCUnixSocket::Payload message; + writeUInt32(message, frame, 0); + std::vector actionVector; + std::tie(actionVector, std::ignore) = + IPADataSerializer::serialize(action, &controlSerializer_); + + message.data.insert(message.data.end(), actionVector.begin(), actionVector.end()); + writeHeader(message, CMD_QUEUEFRAMEACTION, 0); + + socket_.send(message); + LOG(IPAProxyRPiWorker, Info) << "queueFrameAction done"; + return; +} + +int main(int argc, char **argv) +{ + /* Uncomment this for debugging. */ + std::string logPath = "/tmp/libcamera.worker." + + std::to_string(getpid()) + ".log"; + logSetFile(logPath.c_str()); + //logSetTarget(LoggingTargetSyslog); + + if (argc < 3) { + LOG(IPAProxyRPiWorker, Debug) + << "Tried to start worker with no args"; + return EXIT_FAILURE; + } + + int fd = std::stoi(argv[2]); + LOG(IPAProxyRPiWorker, Debug) + << "Starting worker for IPA module " << argv[1] + << " with IPC fd = " << fd; + + std::unique_ptr ipam = std::make_unique(argv[1]); + if (!ipam->isValid() || !ipam->load()) { + LOG(IPAProxyRPiWorker, Error) + << "IPAModule " << argv[1] << " should be valid but isn't"; + return EXIT_FAILURE; + } + + if (socket_.bind(fd) < 0) { + LOG(IPAProxyRPiWorker, Error) << "IPC socket binding failed"; + return EXIT_FAILURE; + } + socket_.readyRead.connect(&readyRead); + + ipa_ = dynamic_cast(ipam->createInterface()); + if (!ipa_) { + LOG(IPAProxyRPiWorker, Error) << "Failed to create IPA context"; + return EXIT_FAILURE; + } + + ipa_->queueFrameAction.connect(&queueFrameAction); + + LOG(IPAProxyRPiWorker, Debug) << "Proxy worker successfully started"; + + /* \todo upgrade listening loop */ + EventDispatcher *dispatcher = Thread::current()->eventDispatcher(); + while (!exit_) + dispatcher->processEvents(); + + delete ipa_; + socket_.close(); + + return 0; +} From patchwork Wed Aug 26 11:09:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9390 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B4E2BBD87E for ; Wed, 26 Aug 2020 11:10:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7FDF96290C; Wed, 26 Aug 2020 13:10:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HdUWV35N"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D582B60387 for ; Wed, 26 Aug 2020 13:10:03 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A12FC53C; Wed, 26 Aug 2020 13:10:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440203; bh=az/aNo+37b8u5rH27+RMNMIt3F9hMSEcRSEgHSFh6BY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HdUWV35NF5BdmbJuUNAxZhr4mkwaqmgt8Ddh2JxIxRvgv1HKiT/q9MXQF8CHHA995 m79/twQNZPUGCVCYyHziBirpqlcyKN+tty6jk2RkqVdU2aNo6ErnpgrmqusxL5t7i6 lhC5Y/yjc0PDCsp/Hf4RdSH/H+pTBHGM1HgSQdHg= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:15 +0900 Message-Id: <20200826110926.67192-7-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 06/17] IPA: IPC: add IPAIPC X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch adds the abstract IPAIPC class and an implementation that uses UnixSocket, IPAIPCUnixSocket. IPAIPC exposes two functions to send a message synchronously, for IPA setup calls such as init(), mapBuffers(), configure(), etc, and to send a message asynchronously, for streaming calls such as processEvent(). It also exposes a signal for callbacks. Just like the callforwards, the IPAIPC has no knowledge of the individual IPA functions, therefore it is the responsibililty of the specific IPA proxy to direct the callback to the specific callback handler in the pipeline handler. An IPAIPCFactory and REGISTER_IPA_IPC facility is also implemented, similar to how pipeline handlers and IPA proxies are declared and created. IPAIPCUnixSocket is an implementation of IPAIPC that uses UnixSocket. For now, IPAProxyRPi fetches IPAIPCUnixSocket and uses it, but this can be changed so that the IPAManager fetches it based on some setting somewhere, and feeds the IPAIPC to the proxy. Signed-off-by: Paul Elder --- include/libcamera/internal/ipa_ipc.h | 73 ++++++++ .../libcamera/internal/ipa_ipc_unixsocket.h | 116 ++++++++++++ src/libcamera/ipa_ipc.cpp | 58 ++++++ src/libcamera/ipa_ipc_unixsocket.cpp | 177 ++++++++++++++++++ 4 files changed, 424 insertions(+) create mode 100644 include/libcamera/internal/ipa_ipc.h create mode 100644 include/libcamera/internal/ipa_ipc_unixsocket.h create mode 100644 src/libcamera/ipa_ipc.cpp create mode 100644 src/libcamera/ipa_ipc_unixsocket.cpp diff --git a/include/libcamera/internal/ipa_ipc.h b/include/libcamera/internal/ipa_ipc.h new file mode 100644 index 00000000..bd7b9bf2 --- /dev/null +++ b/include/libcamera/internal/ipa_ipc.h @@ -0,0 +1,73 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * ipa_ipc.h - Image Processing Algorithm IPC module for IPA proxies + */ +#ifndef __LIBCAMERA_INTERNAL_IPA_IPC_H__ +#define __LIBCAMERA_INTERNAL_IPA_IPC_H__ + +#include + +namespace libcamera { + +class IPAIPC +{ +public: + IPAIPC([[maybe_unused]] const char *ipa_module_path, + [[maybe_unused]] const char *ipa_proxy_worker_path); + virtual ~IPAIPC(); + + bool isValid() const { return valid_; } + + virtual int sendSync(uint32_t cmd, + const std::vector &data_in, + const std::vector &fds_in, + std::vector *data_out = nullptr, + std::vector *fds_out = nullptr) = 0; + + virtual int sendAsync(uint32_t cmd, + const std::vector &data_in, + const std::vector &fds_in) = 0; + + Signal &, std::vector &> recvIPC; + +protected: + bool valid_; +}; + +class IPAIPCFactory +{ +public: + IPAIPCFactory(const char *name); + virtual ~IPAIPCFactory() {} + + virtual std::unique_ptr create(const char *ipa_module_path, + const char *ipa_proxy_worker_path) = 0; + + const std::string &name() const { return name_; } + + static void registerType(IPAIPCFactory *factory); + static std::vector &factories(); + +private: + std::string name_; +}; + +#define REGISTER_IPA_IPC(ipc) \ +class ipc##Factory final : public IPAIPCFactory \ +{ \ +public: \ + ipc##Factory() : IPAIPCFactory(#ipc) {} \ + std::unique_ptr create(const char *ipa_module_path, \ + const char *ipa_proxy_worker_path)\ + { \ + return std::make_unique(ipa_module_path, \ + ipa_proxy_worker_path); \ + } \ +}; \ +static ipc##Factory global_##ipc##Factory; + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_INTERNAL_IPA_IPC_H__ */ diff --git a/include/libcamera/internal/ipa_ipc_unixsocket.h b/include/libcamera/internal/ipa_ipc_unixsocket.h new file mode 100644 index 00000000..4acb27aa --- /dev/null +++ b/include/libcamera/internal/ipa_ipc_unixsocket.h @@ -0,0 +1,116 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * ipa_ipc_unixsocket.h - Image Processing Algorithm IPC module using unix socket + */ +#ifndef __LIBCAMERA_INTERNAL_IPA_IPC_UNIXSOCKET_H__ +#define __LIBCAMERA_INTERNAL_IPA_IPC_UNIXSOCKET_H__ + +// TODO move this to an ipaipc header directory + +#include + +#include + +#include "libcamera/internal/ipa_module.h" +#include "libcamera/internal/ipc_unixsocket.h" + +namespace libcamera { + +class Process; + +void writeHeader(IPCUnixSocket::Payload &payload, uint32_t cmd, uint32_t seq) +{ + uint8_t cmd_arr[] = {static_cast(cmd & 0xff), + static_cast(cmd & (0xff << 8)), + static_cast(cmd & (0xff << 16)), + static_cast(cmd & (0xff << 24))}; + uint8_t seq_arr[] = {static_cast(seq & 0xff), + static_cast(seq & (0xff << 8)), + static_cast(seq & (0xff << 16)), + static_cast(seq & (0xff << 24))}; + payload.data.insert(payload.data.begin(), cmd_arr, cmd_arr+4); + payload.data.insert(payload.data.begin() + 4, seq_arr, seq_arr+4); +} + +std::tuple readHeader(IPCUnixSocket::Payload &payload) +{ + uint32_t cmd = (payload.data[0] & 0xff) | + ((payload.data[1] & 0xff) << 8) | + ((payload.data[2] & 0xff) << 16) | + ((payload.data[3] & 0xff) << 24); + uint32_t seq = (payload.data[4] & 0xff) | + ((payload.data[5] & 0xff) << 8) | + ((payload.data[6] & 0xff) << 16) | + ((payload.data[7] & 0xff) << 24); + + return {cmd, seq}; +} + +void eraseHeader(IPCUnixSocket::Payload &payload) +{ + payload.data.erase(payload.data.begin(), payload.data.begin() + 8); +} + +void writeUInt32(IPCUnixSocket::Payload &payload, uint32_t val, uint32_t pos) +{ + if (pos + 4 > payload.data.size()) + return; + + uint8_t arr[] = {static_cast(val & 0xff), + static_cast(val & (0xff << 8)), + static_cast(val & (0xff << 16)), + static_cast(val & (0xff << 24))}; + std::copy(arr, arr + 4, payload.data.begin() + pos); +} + +uint32_t readUInt32(IPCUnixSocket::Payload &payload, uint32_t pos) +{ + if (pos + 4 > payload.data.size()) + return 0; + + return payload.data[pos] & (payload.data[pos + 1] << 8) & + (payload.data[pos + 2] << 16) & (payload.data[pos + 3] << 24); +} + + +class IPAIPCUnixSocket : public IPAIPC +{ +public: + IPAIPCUnixSocket(const char *ipa_module_path, const char *ipa_proxy_worker_path); + ~IPAIPCUnixSocket(); + + bool isValid() const { return valid_; } + + int sendSync(uint32_t cmd, + const std::vector &data_in, + const std::vector &fds_in, + std::vector *data_out, + std::vector *fds_out) override; + + int sendAsync(uint32_t cmd, + const std::vector &data_in, + const std::vector &fds_in) override; + +private: + struct CallData { + IPCUnixSocket::Payload *response; + bool done; + }; + + void readyRead(IPCUnixSocket *socket); + int call(const IPCUnixSocket::Payload &message, IPCUnixSocket::Payload *response, uint32_t seq); + + uint32_t seq_; + + Process *proc_; + + IPCUnixSocket *socket_; + + std::map callData_; +}; + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_INTERNAL_IPA_IPC_UNIXSOCKET_H__ */ diff --git a/src/libcamera/ipa_ipc.cpp b/src/libcamera/ipa_ipc.cpp new file mode 100644 index 00000000..142f4bc3 --- /dev/null +++ b/src/libcamera/ipa_ipc.cpp @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * ipa_ipc.cpp - Image Processing Algorithm IPC module for IPA proxies + */ + +#include + +#include "libcamera/internal/ipc_unixsocket.h" +#include "libcamera/internal/log.h" +#include "libcamera/internal/process.h" +#include "libcamera/internal/thread.h" + +#include +#include + +#include "libcamera/internal/ipa_ipc.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(IPAIPC) + +IPAIPC::IPAIPC([[maybe_unused]] const char *ipa_module_path, + [[maybe_unused]] const char *ipa_proxy_worker_path) + : valid_(false) +{ +} + +IPAIPC::~IPAIPC() +{ +} + +IPAIPCFactory::IPAIPCFactory(const char *name) + : name_(name) +{ + registerType(this); +} + +void IPAIPCFactory::registerType(IPAIPCFactory *factory) +{ + std::vector &factories = IPAIPCFactory::factories(); + + factories.push_back(factory); + + LOG(IPAIPC, Debug) + << "Registered IPC \"" << factory->name() << "\""; +} + +std::vector &IPAIPCFactory::factories() +{ + static std::vector factories; + return factories; +} + +// TODO documentation, obviously + +} /* namespace libcamera */ diff --git a/src/libcamera/ipa_ipc_unixsocket.cpp b/src/libcamera/ipa_ipc_unixsocket.cpp new file mode 100644 index 00000000..aabc81bd --- /dev/null +++ b/src/libcamera/ipa_ipc_unixsocket.cpp @@ -0,0 +1,177 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * ipa_ipc_unixsocket.cpp - Image Processing Algorithm IPC module using unix socket + */ + +#include + +#include "libcamera/internal/ipc_unixsocket.h" +#include "libcamera/internal/log.h" +#include "libcamera/internal/process.h" +#include "libcamera/internal/thread.h" + +#include +#include + +#include "libcamera/internal/ipa_ipc.h" +#include "libcamera/internal/ipa_ipc_unixsocket.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(IPAIPC) + +IPAIPCUnixSocket::IPAIPCUnixSocket(const char *ipa_module_path, + const char *ipa_proxy_worker_path) + : IPAIPC(ipa_module_path, ipa_proxy_worker_path), seq_(0), + proc_(nullptr), socket_(nullptr) +{ + std::vector fds; + std::vector args; + args.push_back(ipa_module_path); + + socket_ = new IPCUnixSocket(); + int fd = socket_->create(); + if (fd < 0) { + LOG(IPAIPC, Error) + << "Failed to create socket"; + return; + } + socket_->readyRead.connect(this, &IPAIPCUnixSocket::readyRead); + args.push_back(std::to_string(fd)); + fds.push_back(fd); + + proc_ = new Process(); + int ret = proc_->start(ipa_proxy_worker_path, args, fds); + if (ret) { + LOG(IPAIPC, Error) + << "Failed to start proxy worker process"; + return; + } + + valid_ = true; +} + +IPAIPCUnixSocket::~IPAIPCUnixSocket() +{ + delete proc_; + delete socket_; +} + +int IPAIPCUnixSocket::sendSync(uint32_t cmd, + const std::vector &data_in, + const std::vector &fds_in, + std::vector *data_out, + std::vector *fds_out) +{ + IPCUnixSocket::Payload message, response; + int ret; + + /* It's fine if seq_ overflows; that'll just be the new epoch. */ + seq_++; + writeHeader(message, cmd, seq_); + message.data.insert(message.data.end(), data_in.begin(), data_in.end()); + + message.fds = const_cast &>(fds_in); + + ret = call(message, &response, seq_); + if (ret) { + LOG(IPAIPC, Error) << "Failed to call sync"; + callData_.erase(seq_); + return ret; + } + + if (data_out) + data_out->insert(data_out->end(), response.data.begin(), response.data.end()); + + if (fds_out) + fds_out->insert(fds_out->end(), response.fds.begin(), response.fds.end()); + + return 0; +} + +int IPAIPCUnixSocket::sendAsync(uint32_t cmd, + const std::vector &data_in, + const std::vector &fds_in) +{ + IPCUnixSocket::Payload message; + int ret; + + writeHeader(message, cmd, 0); + message.data.insert(message.data.end(), data_in.begin(), data_in.end()); + + message.fds = const_cast &>(fds_in); + + ret = socket_->send(message); + if (ret) { + LOG(IPAIPC, Error) << "Failed to call async"; + return ret; + } + + return 0; +} + +void IPAIPCUnixSocket::readyRead(IPCUnixSocket *socket) +{ + IPCUnixSocket::Payload message; + int ret = socket->receive(&message); + if (ret) { + LOG(IPAIPC, Error) << "Receive message failed" << ret; + return; + } + + uint32_t cmd, seq; + std::tie(cmd, seq) = readHeader(message); + + auto callData = callData_.find(seq); + if (callData != callData_.end()) { + eraseHeader(message); + /* Is there any way to avoid this copy? */ + *callData->second.response = message; + callData->second.done = true; + return; + } + + /* + * Received unexpected data, this means it's a call from the IPA. + * We can't return anything to the IPA (gotta keep them under *our* + * control, plus returning would require blocking the caller, and we + * can't afford to do that). Let the proxy do switch-case on cmd. + */ + recvIPC.emit(message.data, message.fds); + + return; +} + +int IPAIPCUnixSocket::call(const IPCUnixSocket::Payload &message, IPCUnixSocket::Payload *response, uint32_t seq) +{ + Timer timeout; + int ret; + + callData_[seq].response = response; + callData_[seq].done = false; + + ret = socket_->send(message); + if (ret) + return ret; + + timeout.start(200); + while (!callData_[seq].done) { + if (!timeout.isRunning()) { + LOG(IPAIPC, Error) << "Call timeout!"; + callData_.erase(seq); + return -ETIMEDOUT; + } + + Thread::current()->eventDispatcher()->processEvents(); + } + + callData_.erase(seq); + + return 0; +} + +REGISTER_IPA_IPC(IPAIPCUnixSocket) + +} /* namespace libcamera */ From patchwork Wed Aug 26 11:09:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9391 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5A0A3BD87E for ; Wed, 26 Aug 2020 11:10:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2799B628FB; Wed, 26 Aug 2020 13:10:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="vru6uA2s"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0D3FD62907 for ; Wed, 26 Aug 2020 13:10:06 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4EB59B1A; Wed, 26 Aug 2020 13:10:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440205; bh=ngP6sy7t2uHFyW6y24HqCIEjGtzPS1EMbL7VH5eleDg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vru6uA2skR6ITDL+/u/15XX/Z+k+zLPW6jWWVvC0WMoGXPz14zI69eCXOiYQYAz/v eO1E0AC4TtK0qvmUFn+R7KBMbffqd7aHWEA4nBnTMOBtiW7bFcvWw622+TmFGKbH7c dGNQeU7hmkH3QHEKwBFuCGbawlPZo/mgW9QTG93Y= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:16 +0900 Message-Id: <20200826110926.67192-8-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 07/17] meson: Enable IPC X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Enable compilation of all components necessary for IPA over IPC, and disable components that prevent it from working. Note that this patch is not complete. Instead of commenting out compilation units, for example, they should simply be removed. This patch is merely to get IPC runnable for the purpose of the RFC. Signed-off-by: Paul Elder --- src/ipa/libipa/meson.build | 4 ++-- src/libcamera/meson.build | 5 ++++- src/libcamera/proxy/meson.build | 5 +++-- src/libcamera/proxy/worker/meson.build | 3 ++- 4 files changed, 11 insertions(+), 6 deletions(-) diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index 22626405..8335fff6 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -1,11 +1,11 @@ # SPDX-License-Identifier: CC0-1.0 libipa_headers = files([ - 'ipa_interface_wrapper.h', + #'ipa_interface_wrapper.h', ]) libipa_sources = files([ - 'ipa_interface_wrapper.cpp', + #'ipa_interface_wrapper.cpp', ]) libipa_includes = include_directories('..') diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index af2f3d95..8c6a33ac 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -21,8 +21,11 @@ libcamera_sources = files([ 'formats.cpp', 'framebuffer_allocator.cpp', 'geometry.cpp', - 'ipa_context_wrapper.cpp', + #'ipa_context_wrapper.cpp', 'ipa_controls.cpp', + 'ipa_data_serializer.cpp', + 'ipa_ipc.cpp', + 'ipa_ipc_unixsocket.cpp', 'ipa_interface.cpp', 'ipa_manager.cpp', 'ipa_module.cpp', diff --git a/src/libcamera/proxy/meson.build b/src/libcamera/proxy/meson.build index bd804750..11101882 100644 --- a/src/libcamera/proxy/meson.build +++ b/src/libcamera/proxy/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_sources += files([ - 'ipa_proxy_linux.cpp', - 'ipa_proxy_thread.cpp', + #'ipa_proxy_linux.cpp', + 'ipa_proxy_raspberrypi.cpp', + #'ipa_proxy_thread.cpp', ]) diff --git a/src/libcamera/proxy/worker/meson.build b/src/libcamera/proxy/worker/meson.build index ac0310a7..8da38d5e 100644 --- a/src/libcamera/proxy/worker/meson.build +++ b/src/libcamera/proxy/worker/meson.build @@ -1,7 +1,8 @@ # SPDX-License-Identifier: CC0-1.0 ipa_proxy_sources = [ - ['ipa_proxy_linux', 'ipa_proxy_linux_worker.cpp'] + #['ipa_proxy_linux', 'ipa_proxy_linux_worker.cpp'], + ['ipa_proxy_raspberrypi', 'ipa_proxy_raspberrypi_worker.cpp'] ] proxy_install_dir = join_paths(get_option('libexecdir'), 'libcamera') From patchwork Wed Aug 26 11:09:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9392 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C222DBD87E for ; Wed, 26 Aug 2020 11:10:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8E6496290A; Wed, 26 Aug 2020 13:10:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bPfcS5Kb"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9DE3060387 for ; Wed, 26 Aug 2020 13:10:08 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8C92353C; Wed, 26 Aug 2020 13:10:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440208; bh=udlIiFlJQhkwZsEE1j6AMeT/KpZdrmswIKiPql6/6hc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bPfcS5KbqcmhULLyKx95J6H1s+8h1Yh7NwLO6kMDjF4bNrbnDrPHhJe/72Jiq6GzZ w5NwdI9GUif07hvNdSehpCqdOXxwzNTsz5Cvph0tW7ejFonlFVBLgS+ZweeDrClya+ wBz3crjNQIIBTr6CwjYGovH0VZfLRvXHWnFHzlpE= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:17 +0900 Message-Id: <20200826110926.67192-9-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 08/17] tests: IPC: Add various tests for the IPC framework X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch includes tests that I used while debugging. I just kind of plopped them here. If there's any value in these, I'll brush them up and keep them, otherwise I'll discard them. - unixsocket_fds confirms that passing fds over the process boundary increases the fd count - control_serialization_ipa tests de/serializing ControlList and ControlInfoMap using IPADataSerializer - rpi_action_serialization tests de/serialization of RPiActionParams - rpi_config_serialization tests de/serialization of RPiConfigureParams Signed-off-by: Paul Elder --- test/ipa/meson.build | 4 +- test/ipc/meson.build | 3 +- test/ipc/unixsocket_fds.cpp | 284 ++++++++++++++++++ .../control_serialization_ipa.cpp | 85 ++++++ test/serialization/meson.build | 5 +- .../rpi_action_serialization.cpp | 141 +++++++++ .../rpi_config_serialization.cpp | 162 ++++++++++ 7 files changed, 680 insertions(+), 4 deletions(-) create mode 100644 test/ipc/unixsocket_fds.cpp create mode 100644 test/serialization/control_serialization_ipa.cpp create mode 100644 test/serialization/rpi_action_serialization.cpp create mode 100644 test/serialization/rpi_config_serialization.cpp diff --git a/test/ipa/meson.build b/test/ipa/meson.build index ba672f3f..b89c6b66 100644 --- a/test/ipa/meson.build +++ b/test/ipa/meson.build @@ -2,8 +2,8 @@ ipa_test = [ ['ipa_module_test', 'ipa_module_test.cpp'], - ['ipa_interface_test', 'ipa_interface_test.cpp'], - ['ipa_wrappers_test', 'ipa_wrappers_test.cpp'], + #['ipa_interface_test', 'ipa_interface_test.cpp'], + #['ipa_wrappers_test', 'ipa_wrappers_test.cpp'], ] foreach t : ipa_test diff --git a/test/ipc/meson.build b/test/ipc/meson.build index 650df1d6..e960aa82 100644 --- a/test/ipc/meson.build +++ b/test/ipc/meson.build @@ -1,7 +1,8 @@ # SPDX-License-Identifier: CC0-1.0 ipc_tests = [ - [ 'unixsocket', 'unixsocket.cpp' ], + [ 'unixsocket_fds', 'unixsocket_fds.cpp' ], + [ 'unixsocket', 'unixsocket.cpp' ], ] foreach t : ipc_tests diff --git a/test/ipc/unixsocket_fds.cpp b/test/ipc/unixsocket_fds.cpp new file mode 100644 index 00000000..8e9c6864 --- /dev/null +++ b/test/ipc/unixsocket_fds.cpp @@ -0,0 +1,284 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * unixsocket_fds.cpp - Unix socket IPC test with file descriptors + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "libcamera/internal/ipc_unixsocket.h" +#include "libcamera/internal/thread.h" +#include "libcamera/internal/utils.h" + +#include "test.h" + +#define CMD_CLOSE 0 +#define CMD_REVERSE 1 +#define CMD_LEN_CALC 2 +#define CMD_LEN_CMP 3 +#define CMD_JOIN 4 + +using namespace std; +using namespace libcamera; + +class UnixSocketTestFdSlave +{ +public: + UnixSocketTestFdSlave() + : exitCode_(EXIT_FAILURE), exit_(false) + { + dispatcher_ = Thread::current()->eventDispatcher(); + ipc_.readyRead.connect(this, &UnixSocketTestFdSlave::readyRead); + } + + int run(int fd) + { + if (ipc_.bind(fd)) { + cerr << "Failed to connect to IPC channel" << endl; + return EXIT_FAILURE; + } + + while (!exit_) + dispatcher_->processEvents(); + + ipc_.close(); + + return exitCode_; + } + +private: + void readyRead(IPCUnixSocket *ipc) + { + IPCUnixSocket::Payload message, response; + int ret; + + ret = ipc->receive(&message); + if (ret) { + cerr << "Receive message failed: " << ret << endl; + return; + } + + int fd = message.fds[0]; + + cerr << "slave: received " << fd << endl; + + response.fds = { fd }; + + ret = ipc_.send(response); + if (ret < 0) { + cerr << "Reply failed" << endl; + stop(ret); + } + } + + void stop(int code) + { + exitCode_ = code; + exit_ = true; + } + + IPCUnixSocket ipc_; + EventDispatcher *dispatcher_; + int exitCode_; + bool exit_; +}; + +class UnixSocketTestFd : public Test +{ +protected: + int slaveStart(int fd) + { + pid_ = fork(); + + if (pid_ == -1) + return TestFail; + + if (!pid_) { + std::string arg = std::to_string(fd); + execl("/proc/self/exe", "/proc/self/exe", + arg.c_str(), nullptr); + + /* Only get here if exec fails. */ + exit(TestFail); + } + + return TestPass; + } + + int slaveStop() + { + int status; + + if (pid_ < 0) + return TestFail; + + if (waitpid(pid_, &status, 0) < 0) + return TestFail; + + if (!WIFEXITED(status) || WEXITSTATUS(status)) + return TestFail; + + return TestPass; + } + + int testSendNewFd() + { + IPCUnixSocket::Payload message, response; + int ret; + + int fd = open("/dev/null", O_RDONLY); + if (fd < 0) { + ret = -errno; + cerr << "Failed to open /dev/null: " << strerror(-ret) << endl; + return ret; + } + + cerr << "master: opened and sending fd " << fd << endl; + + message.fds = { fd }; + + ret = call(message, &response); + if (ret) + return ret; + + cerr << "master: received fd " << response.fds[0] << endl; + + return 0; + } + + int testSendOldFd(int fd) + { + IPCUnixSocket::Payload message, response; + int ret; + + cerr << "master: sending fd " << fd << endl; + + message.fds = { fd }; + + ret = call(message, &response); + if (ret) + return ret; + + cerr << "master: received fd " << response.fds[0] << endl; + + return 0; + } + + int init() + { + callResponse_ = nullptr; + return 0; + } + + int run() + { + int slavefd = ipc_.create(); + if (slavefd < 0) + return TestFail; + + if (slaveStart(slavefd)) { + cerr << "Failed to start slave" << endl; + return TestFail; + } + + ipc_.readyRead.connect(this, &UnixSocketTestFd::readyRead); + + for (int i = 0; i < 10; i++) + testSendNewFd(); + + for (int i = 0; i < 10; i++) + testSendOldFd(20); + + /* Close slave connection. */ + IPCUnixSocket::Payload close; + close.data.push_back(CMD_CLOSE); + if (ipc_.send(close)) { + cerr << "Closing IPC channel failed" << endl; + return TestFail; + } + + ipc_.close(); + if (slaveStop()) { + cerr << "Failed to stop slave" << endl; + return TestFail; + } + + return TestPass; + } + +private: + int call(const IPCUnixSocket::Payload &message, IPCUnixSocket::Payload *response) + { + Timer timeout; + int ret; + + callDone_ = false; + callResponse_ = response; + + ret = ipc_.send(message); + if (ret) + return ret; + + timeout.start(200); + while (!callDone_) { + if (!timeout.isRunning()) { + cerr << "Call timeout!" << endl; + callResponse_ = nullptr; + return -ETIMEDOUT; + } + + Thread::current()->eventDispatcher()->processEvents(); + } + + callResponse_ = nullptr; + + return 0; + } + + void readyRead(IPCUnixSocket *ipc) + { + if (!callResponse_) { + cerr << "Read ready without expecting data, fail." << endl; + return; + } + + if (ipc->receive(callResponse_)) { + cerr << "Receive message failed" << endl; + return; + } + + callDone_ = true; + } + + pid_t pid_; + IPCUnixSocket ipc_; + bool callDone_; + IPCUnixSocket::Payload *callResponse_; +}; + +/* + * Can't use TEST_REGISTER() as single binary needs to act as both proxy + * master and slave. + */ +int main(int argc, char **argv) +{ + if (argc == 2) { + int ipcfd = std::stoi(argv[1]); + UnixSocketTestFdSlave slave; + return slave.run(ipcfd); + } + + return UnixSocketTestFd().execute(); +} diff --git a/test/serialization/control_serialization_ipa.cpp b/test/serialization/control_serialization_ipa.cpp new file mode 100644 index 00000000..88489326 --- /dev/null +++ b/test/serialization/control_serialization_ipa.cpp @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * control_serialization_ipa.cpp - Serialize and deserialize controls with IPADataSerializer + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/ipa_data_serializer.h" +#include "libcamera/internal/ipa_manager.h" +#include "libcamera/internal/ipa_module.h" +#include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/thread.h" + +#include "serialization_test.h" +#include "test.h" + +using namespace std; +using namespace libcamera; + +class ControlSerializationIPATest : public CameraTest, public Test +{ +public: + ControlSerializationIPATest() + : CameraTest("imx219") + { + } + +protected: + int init() override + { + return status_; + } + + int run() override + { + ControlSerializer cs; + + /* Create a control list with three controls. */ + const ControlInfoMap &infoMap = camera_->controls(); + ControlList list(infoMap); + + list.set(controls::Brightness, 0.5f); + list.set(controls::Contrast, 1.2f); + list.set(controls::Saturation, 0.2f); + + vector infoMapBuf; + tie(infoMapBuf, ignore) = IPADataSerializer::serialize(infoMap, &cs); + + vector listBuf; + tie(listBuf, ignore) = IPADataSerializer::serialize(list, *list.infoMap(), &cs); + + + const ControlInfoMap infoMapOut = IPADataSerializer::deserialize(infoMapBuf, &cs); + + ControlList listOut = IPADataSerializer::deserialize(listBuf, &cs); + + + if (!SerializationTest::equals(infoMap, infoMapOut)) { + cerr << "Deserialized map doesn't match original" << endl; + return TestFail; + } + + if (!SerializationTest::equals(list, listOut)) { + cerr << "Deserialized list doesn't match original" << endl; + return TestFail; + } + + return TestPass; + } +}; + +TEST_REGISTER(ControlSerializationIPATest) diff --git a/test/serialization/meson.build b/test/serialization/meson.build index a9d9cbcb..ecd07adf 100644 --- a/test/serialization/meson.build +++ b/test/serialization/meson.build @@ -1,7 +1,10 @@ # SPDX-License-Identifier: CC0-1.0 serialization_tests = [ - [ 'control_serialization', 'control_serialization.cpp' ], + [ 'control_serialization_ipa', 'control_serialization_ipa.cpp' ], + [ 'control_serialization', 'control_serialization.cpp' ], + [ 'rpi_config_serialization', 'rpi_config_serialization.cpp' ], + [ 'rpi_action_serialization', 'rpi_action_serialization.cpp' ], ] foreach t : serialization_tests diff --git a/test/serialization/rpi_action_serialization.cpp b/test/serialization/rpi_action_serialization.cpp new file mode 100644 index 00000000..de9db03f --- /dev/null +++ b/test/serialization/rpi_action_serialization.cpp @@ -0,0 +1,141 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * rpi_action_serialization.cpp - Serialize and deserialize raspberrypi IPA action data + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/ipa_data_serializer.h" +#include "libcamera/internal/ipa_manager.h" +#include "libcamera/internal/ipa_module.h" +#include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/thread.h" + +#include "serialization_test.h" +#include "test.h" + +using namespace std; +using namespace libcamera; + +class RPiActionSerializationTest : public CameraTest, public Test +{ +public: + RPiActionSerializationTest() + : CameraTest("imx219") + { + } + +protected: + int init() override + { + return status_; + } + + int run() override + { + ControlSerializer cs; + + RPiConfigureParams ipaConfig; + + const ControlInfoMap &infoMap1 = camera_->controls(); + ControlList list1(infoMap1); + list1.set(controls::Brightness, 0.5f); + list1.set(controls::Contrast, 1.2f); + list1.set(controls::Saturation, 0.2f); + + const ControlInfoMap &infoMap2 = camera_->controls(); + ControlList list2(infoMap2); + list2.set(controls::Brightness, -0.8f); + list2.set(controls::Contrast, 2.1f); + list2.set(controls::Saturation, 1.1f); + + RPiStatsCompletePayload s; + s.bufferId_ = 111; + s.controls_ = list2; + + RPiActionParams action; + action.op_ = RPI_IPA_ACTION_RUN_ISP; + action.bufferId_ = 222; + action.statsComplete_ = s; + action.controls_ = list1; + + + vector buf; + tie(buf, std::ignore) = IPADataSerializer::serialize(action, &cs); + + RPiActionParams actionOut = IPADataSerializer::deserialize(buf, &cs); + + + if (!equals(action, actionOut)) { + cerr << "Deserialized action doesn't match original" << endl; + return TestFail; + } else { + cerr << "Pass!" << endl; + } + + return TestPass; + } + +private: + bool equals(const RPiStatsCompletePayload &lhs, const RPiStatsCompletePayload &rhs) + { + bool matches = true; + if (lhs.bufferId_ != rhs.bufferId_) + matches = false; + + if (!SerializationTest::equals(lhs.controls_, rhs.controls_)) { + matches = false; + cerr << "controls in stats complete not matching" << endl; + } + + return matches; + } + + bool equals(const RPiActionParams &lhs, const RPiActionParams &rhs) + { + bool matches = true; + + if (lhs.op_ != rhs.op_) { + matches = false; + cerr << "op_ expected " << lhs.op_ << " got " << rhs.op_; + } + + if (lhs.bufferId_ != rhs.bufferId_) { + matches = false; + cerr << "bufferId_ expected " << lhs.bufferId_ << " got " << rhs.bufferId_; + } + + if (!equals(lhs.statsComplete_, rhs.statsComplete_)) { + matches = false; + cerr << "statsComplete_ expected {" + << lhs.statsComplete_.bufferId_ << ", and some controls}" + << " got {" + << rhs.statsComplete_.bufferId_ << ", and some controls}" << endl; + } + + if (!SerializationTest::equals(lhs.controls_, rhs.controls_)) { + matches = false; + cerr << "controls not matching" << endl; + } + + return matches; + } + +}; + +TEST_REGISTER(RPiActionSerializationTest) diff --git a/test/serialization/rpi_config_serialization.cpp b/test/serialization/rpi_config_serialization.cpp new file mode 100644 index 00000000..390579f1 --- /dev/null +++ b/test/serialization/rpi_config_serialization.cpp @@ -0,0 +1,162 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * rpi_config_serialization.cpp - Serialize and deserialize raspberrypi IPA configuration data + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/ipa_data_serializer.h" +#include "libcamera/internal/ipa_manager.h" +#include "libcamera/internal/ipa_module.h" +#include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/thread.h" + +#include "serialization_test.h" +#include "test.h" + +using namespace std; +using namespace libcamera; + +class RPiConfigSerializationTest : public CameraTest, public Test +{ +public: + RPiConfigSerializationTest() + : CameraTest("imx219") + { + } + +protected: + int init() override + { + return status_; + } + + int run() override + { + ControlSerializer cs; + + RPiConfigureParams ipaConfig; + + const ControlInfoMap &infoMap = camera_->controls(); + ControlList list(infoMap); + list.set(controls::Brightness, 0.5f); + list.set(controls::Contrast, 1.2f); + list.set(controls::Saturation, 0.2f); + + + RPiConfigurePayload p1; + p1.op_ = RPI_IPA_CONFIG_LS_TABLE; + p1.lsTableHandle_ = FileDescriptor(222); + + RPiConfigurePayload p2; + p2.op_ = RPI_IPA_CONFIG_STAGGERED_WRITE; + p2.staggeredWriteResult_ = RPiStaggeredWritePayload(6, 6, 6); + + RPiConfigurePayload p3; + p3.op_ = RPI_IPA_CONFIG_SENSOR; + p3.controls_ = list; + + ipaConfig.payload_.push_back(p1); + ipaConfig.payload_.push_back(p2); + ipaConfig.payload_.push_back(p3); + + vector buf; + vector fds; + tie(buf, fds) = IPADataSerializer::serialize(ipaConfig, &cs); + + RPiConfigureParams ipaConfigOut = IPADataSerializer::deserialize(buf, fds, &cs); + + + if (!equals(ipaConfig, ipaConfigOut)) { + cerr << "Deserialized config doesn't match original" << endl; + return TestFail; + } else { + cerr << "Pass!" << endl; + } + + return TestPass; + } + +private: + bool equals(const RPiStaggeredWritePayload &lhs, const RPiStaggeredWritePayload &rhs) + { + return lhs.gainDelay_ == rhs.gainDelay_ && + lhs.exposureDelay_ == rhs.exposureDelay_ && + lhs.sensorMetadata_ == rhs.sensorMetadata_; + } + + bool equals(const RPiConfigurePayload &lhs, const RPiConfigurePayload &rhs) + { + bool matches = true; + + if (lhs.op_ != rhs.op_) { + matches = false; + cerr << "expected " << lhs.op_ << " got " << rhs.op_; + } + + if (lhs.lsTableHandle_.fd() != rhs.lsTableHandle_.fd()) { + matches = false; + cerr << "expected " << lhs.lsTableHandle_.fd() << " got " << rhs.lsTableHandle_.fd(); + } + + if (lhs.bufferFd_ != rhs.bufferFd_) { + matches = false; + cerr << "expected " << lhs.bufferFd_ << " got " << rhs.bufferFd_; + } + + if (!equals(lhs.staggeredWriteResult_, rhs.staggeredWriteResult_)) { + matches = false; + cerr << "expected {" + << lhs.staggeredWriteResult_.gainDelay_ << ", " + << lhs.staggeredWriteResult_.exposureDelay_ << ", " + << lhs.staggeredWriteResult_.sensorMetadata_ << "} " + << " got {" + << rhs.staggeredWriteResult_.gainDelay_ << ", " + << rhs.staggeredWriteResult_.exposureDelay_ << ", " + << rhs.staggeredWriteResult_.sensorMetadata_ << "} "; + } + + if (!SerializationTest::equals(lhs.controls_, rhs.controls_)) { + matches = false; + } + + return matches; + } + + bool equals(const RPiConfigureParams &lhs, const RPiConfigureParams &rhs) + { + bool matches = true; + + if (lhs.payload_.size() != rhs.payload_.size()) { + cerr << "non-matching size" << endl; + return false; + } + + size_t len = lhs.payload_.size(); + for (unsigned int i = 0; i < len; i++) { + cerr << "[" << i << "]: " << endl; + if (!equals(lhs.payload_[i], rhs.payload_[i])) + matches = false; + } + + return matches; + } + +}; + +TEST_REGISTER(RPiConfigSerializationTest) From patchwork Wed Aug 26 11:09:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9393 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4077ABD87E for ; Wed, 26 Aug 2020 11:10:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 05A3A6290C; Wed, 26 Aug 2020 13:10:12 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="tajoIYXd"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CB365628DD for ; Wed, 26 Aug 2020 13:10:10 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1E1FA9CE; Wed, 26 Aug 2020 13:10:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440210; bh=hZ5z9Pn83/b9ODQDApDcjez+uT4fueazDjez05BOfEM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tajoIYXdWANBZnKEgAbV+Jh3DLocWxo+xffjKeiUAgBWoNlJoCnqU+xxoTqUg6rZx O6e/r5QxK59udx90V04JhX7ScpFUVwxx1U6y5qxi1F6hZKWTaVBv9NbevhCybNwEJL WVcSlBIGJUFKNgqyNYVTLVBSiakrHZkrf/HLsfEA= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:18 +0900 Message-Id: <20200826110926.67192-10-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 09/17] libcamera: ProcessManager: make ProcessManager lifetime explicitly managed X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If any Process instances are destroyed after the ProcessManager is destroyed, then a segfault will occur. Fix this by making the lifetime of the ProcessManager explicit, and make the CameraManager construct and deconstruct (automatically, via a unique pointer) the ProcessManager. Signed-off-by: Paul Elder --- include/libcamera/internal/process.h | 27 ++++++++++++++++ src/libcamera/camera_manager.cpp | 2 ++ src/libcamera/process.cpp | 46 ++++++++++++---------------- 3 files changed, 48 insertions(+), 27 deletions(-) diff --git a/include/libcamera/internal/process.h b/include/libcamera/internal/process.h index 36595106..df697c7f 100644 --- a/include/libcamera/internal/process.h +++ b/include/libcamera/internal/process.h @@ -7,6 +7,7 @@ #ifndef __LIBCAMERA_INTERNAL_PROCESS_H__ #define __LIBCAMERA_INTERNAL_PROCESS_H__ +#include #include #include @@ -50,6 +51,32 @@ private: friend class ProcessManager; }; +class ProcessManager +{ +public: + ProcessManager(); + ~ProcessManager(); + + void registerProcess(Process *proc); + + static ProcessManager *instance(); + + int writePipe() const; + + const struct sigaction &oldsa() const; + +private: + static ProcessManager *self_; + + void sighandler(EventNotifier *notifier); + + std::list processes_; + + struct sigaction oldsa_; + EventNotifier *sigEvent_; + int pipe_[2]; +}; + } /* namespace libcamera */ #endif /* __LIBCAMERA_INTERNAL_PROCESS_H__ */ diff --git a/src/libcamera/camera_manager.cpp b/src/libcamera/camera_manager.cpp index 47d56256..b8d3ccc8 100644 --- a/src/libcamera/camera_manager.cpp +++ b/src/libcamera/camera_manager.cpp @@ -18,6 +18,7 @@ #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/log.h" #include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/process.h" #include "libcamera/internal/thread.h" #include "libcamera/internal/utils.h" @@ -67,6 +68,7 @@ private: std::unique_ptr enumerator_; IPAManager ipaManager_; + ProcessManager processManager_; }; CameraManager::Private::Private(CameraManager *cm) diff --git a/src/libcamera/process.cpp b/src/libcamera/process.cpp index 994190dc..72b5afe2 100644 --- a/src/libcamera/process.cpp +++ b/src/libcamera/process.cpp @@ -41,28 +41,6 @@ LOG_DEFINE_CATEGORY(Process) * The ProcessManager singleton keeps track of all created Process instances, * and manages the signal handling involved in terminating processes. */ -class ProcessManager -{ -public: - void registerProcess(Process *proc); - - static ProcessManager *instance(); - - int writePipe() const; - - const struct sigaction &oldsa() const; - -private: - void sighandler(EventNotifier *notifier); - ProcessManager(); - ~ProcessManager(); - - std::list processes_; - - struct sigaction oldsa_; - EventNotifier *sigEvent_; - int pipe_[2]; -}; namespace { @@ -127,8 +105,20 @@ void ProcessManager::registerProcess(Process *proc) processes_.push_back(proc); } +ProcessManager *ProcessManager::self_ = nullptr; + +/** + * \brief Construct a ProcessManager instance + * + * The ProcessManager class is meant to only be instantiated once, by the + * CameraManager. + */ ProcessManager::ProcessManager() { + if (self_) + LOG(Process, Fatal) + << "Multiple ProcessManager objects are not allowed"; + sigaction(SIGCHLD, NULL, &oldsa_); struct sigaction sa; @@ -145,6 +135,8 @@ ProcessManager::ProcessManager() << "Failed to initialize pipe for signal handling"; sigEvent_ = new EventNotifier(pipe_[0], EventNotifier::Read); sigEvent_->activated.connect(this, &ProcessManager::sighandler); + + self_ = this; } ProcessManager::~ProcessManager() @@ -153,21 +145,21 @@ ProcessManager::~ProcessManager() delete sigEvent_; close(pipe_[0]); close(pipe_[1]); + + self_ = nullptr; } /** * \brief Retrieve the Process manager instance * - * The ProcessManager is a singleton and can't be constructed manually. This - * method shall instead be used to retrieve the single global instance of the - * manager. + * The ProcessManager is constructed by the CameraManager. This function shall + * be used to retrieve the single instance of the manager. * * \return The Process manager instance */ ProcessManager *ProcessManager::instance() { - static ProcessManager processManager; - return &processManager; + return self_; } /** From patchwork Wed Aug 26 11:09:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9394 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CA4E4BD87E for ; Wed, 26 Aug 2020 11:10:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 97F2162907; Wed, 26 Aug 2020 13:10:13 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="usvnx86F"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 032A1628F3 for ; Wed, 26 Aug 2020 13:10:13 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3BFAB53C; Wed, 26 Aug 2020 13:10:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440212; bh=EwePtZp5fuiDyD3xiuPpKAkPxwi5TXYBBJzzBeP9v5E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=usvnx86FRYnhOGlCi8h/Zy4rlmS7slW8XmiGBPnpxfNr/wvwQaprUDPFc7qJmpgBH UmhKsdXlsfLh1xpSA2i2a6H3oAvvCPTGudvYml4vFZwj+96WKn8gNWeuZkJC6Pbb5P fVuJhGThXn4H5oZ0rGwxbNKVVSKSwz89rKAHNW4Y= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:19 +0900 Message-Id: <20200826110926.67192-11-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 10/17] libcamera: IPAModule: Replace ipa_context with IPAInterface X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the new IPC infrastructure, we no longer need the C interface as provided by struct ipa_context. Make ipaCreate_() and createInterface() return IPAInterface. Signed-off-by: Paul Elder --- include/libcamera/internal/ipa_module.h | 4 ++-- src/libcamera/ipa_module.cpp | 16 +--------------- 2 files changed, 3 insertions(+), 17 deletions(-) diff --git a/include/libcamera/internal/ipa_module.h b/include/libcamera/internal/ipa_module.h index c2df2476..19fc5827 100644 --- a/include/libcamera/internal/ipa_module.h +++ b/include/libcamera/internal/ipa_module.h @@ -33,7 +33,7 @@ public: bool load(); - struct ipa_context *createContext(); + IPAInterface *createInterface(); bool match(PipelineHandler *pipe, uint32_t minVersion, uint32_t maxVersion) const; @@ -52,7 +52,7 @@ private: bool loaded_; void *dlHandle_; - typedef struct ipa_context *(*IPAIntfFactory)(); + typedef IPAInterface *(*IPAIntfFactory)(void); IPAIntfFactory ipaCreate_; }; diff --git a/src/libcamera/ipa_module.cpp b/src/libcamera/ipa_module.cpp index de512a7f..008325c4 100644 --- a/src/libcamera/ipa_module.cpp +++ b/src/libcamera/ipa_module.cpp @@ -438,21 +438,7 @@ bool IPAModule::load() return true; } -/** - * \brief Instantiate an IPA context - * - * After loading the IPA module with load(), this method creates an instance of - * the IPA module context. Ownership of the context is passed to the caller, and - * the context shall be destroyed by calling the \ref ipa_context_ops::destroy - * "ipa_context::ops::destroy()" function. - * - * Calling this function on a module that has not yet been loaded, or an - * invalid module (as returned by load() and isValid(), respectively) is - * an error. - * - * \return The IPA context on success, or nullptr on error - */ -struct ipa_context *IPAModule::createContext() +IPAInterface *IPAModule::createInterface() { if (!valid_ || !loaded_) return nullptr; From patchwork Wed Aug 26 11:09:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9395 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3BB17BD87E for ; Wed, 26 Aug 2020 11:10:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0997A628FB; Wed, 26 Aug 2020 13:10:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="tjxNlJeJ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6F1D6628DD for ; Wed, 26 Aug 2020 13:10:15 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6AA189CE; Wed, 26 Aug 2020 13:10:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440214; bh=3Tmzp3GTPNTq8PBgF2z8zvV2ry6agt6dvbi+OPuAp20=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tjxNlJeJ1prUZyypOJqhXNsfcrD97HTf0OogP97uDjW5mRPaB7884Dw9mjuem9f3V yZvwIiApfWCNPmmyx8h7n5N6jd3X0rZxeRJMlKNr8lboLhuHy5rhpkr4j/6KxQN/EA xaNxaCCF2VcZUxRXexjhB5PTCLiYOVnrtNttAV2w= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:20 +0900 Message-Id: <20200826110926.67192-12-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 11/17] libcamera: IPAProxy: Remove stop() override X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Since stop() is part of the IPA interface, and the IPA interface is now generated based on the data definition file per pipeline, this no longer needs to be overrided by the base IPAProxy. Remove it. Signed-off-by: Paul Elder --- include/libcamera/internal/ipa_proxy.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/include/libcamera/internal/ipa_proxy.h b/include/libcamera/internal/ipa_proxy.h index b429ce5a..aec8f04f 100644 --- a/include/libcamera/internal/ipa_proxy.h +++ b/include/libcamera/internal/ipa_proxy.h @@ -27,8 +27,6 @@ public: std::string configurationFile(const std::string &file) const; - void stop() override = 0; - protected: std::string resolvePath(const std::string &file) const; From patchwork Wed Aug 26 11:09:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9396 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A6382BD87E for ; Wed, 26 Aug 2020 11:10:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 71037628DD; Wed, 26 Aug 2020 13:10:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="YR6hPKRj"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9666A628F3 for ; Wed, 26 Aug 2020 13:10:17 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DA32E53C; Wed, 26 Aug 2020 13:10:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440217; bh=ILA9qGSrVUo7ScoEwEt/hWzj4MgFRkfuDHEvjOeEnOo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YR6hPKRjqjoir4i5in4fz+UtN5acU0G1zWRcvxfgz8y3i9cSxNqgcFIL0xyISGVcP Q8mkwllXZWAmqdRR5EbuO2ZmonLsI0nShXAnZ6rViIVKIfIZeIotJuue71+K8zP2k+ rZJFsPN5USwBMJqQ6xfGZwMtxUdPUnPVkYGHUcJQ= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:21 +0900 Message-Id: <20200826110926.67192-13-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 12/17] libcamera: IPAProxy: Add isolate parameter to create() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Since IPAProxy implementations now always encapsulate IPA modules, add a parameter to create() to signal if the proxy should isolate the IPA or not. Signed-off-by: Paul Elder --- include/libcamera/internal/ipa_proxy.h | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/include/libcamera/internal/ipa_proxy.h b/include/libcamera/internal/ipa_proxy.h index aec8f04f..1903150e 100644 --- a/include/libcamera/internal/ipa_proxy.h +++ b/include/libcamera/internal/ipa_proxy.h @@ -42,7 +42,7 @@ public: IPAProxyFactory(const char *name); virtual ~IPAProxyFactory() {} - virtual std::unique_ptr create(IPAModule *ipam) = 0; + virtual std::unique_ptr create(IPAModule *ipam, bool isolate) = 0; const std::string &name() const { return name_; } @@ -53,16 +53,16 @@ private: std::string name_; }; -#define REGISTER_IPA_PROXY(proxy) \ -class proxy##Factory final : public IPAProxyFactory \ -{ \ -public: \ - proxy##Factory() : IPAProxyFactory(#proxy) {} \ - std::unique_ptr create(IPAModule *ipam) \ - { \ - return std::make_unique(ipam); \ - } \ -}; \ +#define REGISTER_IPA_PROXY(proxy) \ +class proxy##Factory final : public IPAProxyFactory \ +{ \ +public: \ + proxy##Factory() : IPAProxyFactory(#proxy) {} \ + std::unique_ptr create(IPAModule *ipam, bool isolate) \ + { \ + return std::make_unique(ipam, isolate); \ + } \ +}; \ static proxy##Factory global_##proxy##Factory; } /* namespace libcamera */ From patchwork Wed Aug 26 11:09:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9397 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 07D9ABD87E for ; Wed, 26 Aug 2020 11:10:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C8B4C62912; Wed, 26 Aug 2020 13:10:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="MRE/3/fV"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BC8AA628F9 for ; Wed, 26 Aug 2020 13:10:19 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0AFE3B1A; Wed, 26 Aug 2020 13:10:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440219; bh=sHQZxUf464cUUxWvh5t0rO7Wh4wGbbpd0kdCkgnFrhk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MRE/3/fVe31poWTnNbkGJWaWUBlm8k7RgTFRS5hHef8xZKemFvyQW6sMr8ErVmZ7S n7S2WQIYwUWgC4+YL4R5a3Dk3t5O/M3NViB3Ul3gbnO1OVXCBUqt786B1KDtM5u/d3 2XRthtGYOGAf9iyrAPCP2lOarO5HOmXn1jtV2kxg= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:22 +0900 Message-Id: <20200826110926.67192-14-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 13/17] libcamera: PipelineHandler: Remove IPA from base class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Since pipline handlers now have their own IPA interface types, it can no longer be defined in the base class, and each pipline handler implementation must declare it and its type themselves. Remove it from the base class. Signed-off-by: Paul Elder --- include/libcamera/internal/pipeline_handler.h | 1 - 1 file changed, 1 deletion(-) diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h index a4e1b529..2018bef0 100644 --- a/include/libcamera/internal/pipeline_handler.h +++ b/include/libcamera/internal/pipeline_handler.h @@ -47,7 +47,6 @@ public: std::list queuedRequests_; ControlInfoMap controlInfo_; ControlList properties_; - std::unique_ptr ipa_; private: CameraData(const CameraData &) = delete; From patchwork Wed Aug 26 11:09:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9398 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7B153BD87E for ; Wed, 26 Aug 2020 11:10:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 44EEB628BB; Wed, 26 Aug 2020 13:10:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Z0j+vBg6"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 07217628DD for ; Wed, 26 Aug 2020 13:10:22 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 490669CE; Wed, 26 Aug 2020 13:10:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440221; bh=wzGdd1JnYp6naLTSildYQuu4PfeQ/BwsU5DGLaEbhsI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Z0j+vBg6z0ADfOpH9T3abVO/UunOHy1aA6Or/EHNOtn9L81aA1cHQaE9KY9kC9XD6 Q6fX3GKauRyWZx+yMznh5tTcksSlBmCDaH85lYVvwOWga1d6Rt+F36QdFLGC6PC4MC nON7E4WzrRgm1PFWVLy1XP19jpZEzVMEQy3Giecc= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:23 +0900 Message-Id: <20200826110926.67192-15-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 14/17] libcamera: IPAInterface: Remove all functions from IPAInterface X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that all the functions in the IPA interface are defined in the data definition file and a specialized IPAInterface is generated per pipeline handler, remove all the functions from the case IPAInterface. Signed-off-by: Paul Elder --- include/libcamera/ipa/ipa_interface.h | 16 ---------------- 1 file changed, 16 deletions(-) diff --git a/include/libcamera/ipa/ipa_interface.h b/include/libcamera/ipa/ipa_interface.h index 5016ec25..cbe325ea 100644 --- a/include/libcamera/ipa/ipa_interface.h +++ b/include/libcamera/ipa/ipa_interface.h @@ -151,22 +151,6 @@ class IPAInterface { public: virtual ~IPAInterface() {} - - virtual int init(const IPASettings &settings) = 0; - virtual int start() = 0; - virtual void stop() = 0; - - virtual void configure(const CameraSensorInfo &sensorInfo, - const std::map &streamConfig, - const std::map &entityControls, - const IPAOperationData &ipaConfig, - IPAOperationData *result) = 0; - - virtual void mapBuffers(const std::vector &buffers) = 0; - virtual void unmapBuffers(const std::vector &ids) = 0; - - virtual void processEvent(const IPAOperationData &data) = 0; - Signal queueFrameAction; }; } /* namespace libcamera */ From patchwork Wed Aug 26 11:09:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9399 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D9FA2BD87E for ; Wed, 26 Aug 2020 11:10:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A63386290A; Wed, 26 Aug 2020 13:10:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="JBpHAgEN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 54902628DD for ; Wed, 26 Aug 2020 13:10:24 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8745A53C; Wed, 26 Aug 2020 13:10:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440224; bh=OcghZ9l+sP+9kXxJlJJSLmso5fjqijVhZW9Op5VXJ4c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JBpHAgENY9xXP8/l48ov7njNYebiMkPzdALZhopnQwunCR8vLXTlFmKbVWb4GFS0+ foeTA+CY4Hvq6y46rs5Y3UifFgAa/ijTIFBpUibQoULgCcZ/ZyVaSwFrwvhymVhleW R3QWFVi0okG99PPYe+mBfPmZkLBUbBxXlNJr7Qhg= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:24 +0900 Message-Id: <20200826110926.67192-16-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 15/17] libcamera: IPAInterface: make ipaCreate return IPAInterface X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the new IPC infrastructure, we no longer need the C interface as provided by struct ipa_context. Make ipaCreate return IPAinterface. Signed-off-by: Paul Elder --- include/libcamera/ipa/ipa_interface.h | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/include/libcamera/ipa/ipa_interface.h b/include/libcamera/ipa/ipa_interface.h index cbe325ea..6efe2e99 100644 --- a/include/libcamera/ipa/ipa_interface.h +++ b/include/libcamera/ipa/ipa_interface.h @@ -110,8 +110,6 @@ struct ipa_context_ops { const struct ipa_operation_data *data); }; -struct ipa_context *ipaCreate(); - #ifdef __cplusplus } @@ -154,6 +152,11 @@ public: }; } /* namespace libcamera */ + +extern "C" { +libcamera::IPAInterface *ipaCreate(); +} + #endif #endif /* __LIBCAMERA_IPA_INTERFACE_H__ */ From patchwork Wed Aug 26 11:09:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9400 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4B662BD87E for ; Wed, 26 Aug 2020 11:10:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 190D9628F9; Wed, 26 Aug 2020 13:10:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="N1Qme77h"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 81176628BB for ; Wed, 26 Aug 2020 13:10:26 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C41C39CE; Wed, 26 Aug 2020 13:10:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440226; bh=mENnW9UmAMtZuYE04IPh/wzybnVDFzxLOipDE9iDS6o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=N1Qme77hrS5hQKxuPc0VRka56llPY77DRXabqr+m2erWGz4jsTYPpkknuw9IpAiK7 ozWN9VEAKJJ68DaBUaVkt95Csu2p0kk02JTa0nvUFIOnb2N67rUnyDTh3j5PaRzk9p LrMiWdfl0K0E9zwFuiNrIVtLvtAdKZU9MiI3XPaY= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:25 +0900 Message-Id: <20200826110926.67192-17-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 16/17] libcamera: IPAManager: Fetch IPAProxy corresponding to pipeline X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that each pipeline handler has its own IPAProxy implementation, make the IPAManager fetch the IPAProxy based on the pipeline handler name. Also, since the IPAProxy is used regardless of isolation or no isolation, remove the isolation check from the proxy selection. Signed-off-by: Paul Elder --- src/libcamera/ipa_manager.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp index 046fd5c6..2d0ea242 100644 --- a/src/libcamera/ipa_manager.cpp +++ b/src/libcamera/ipa_manager.cpp @@ -275,8 +275,8 @@ std::unique_ptr IPAManager::createIPA(PipelineHandler *pipe, * * \todo Implement a better proxy selection */ - const char *proxyName = self_->isSignatureValid(m) - ? "IPAProxyThread" : "IPAProxyLinux"; + std::string pipeName(pipe->name()); + const char *proxyName = pipeName.replace(0, 15, "IPAProxy").c_str(); IPAProxyFactory *pf = nullptr; for (IPAProxyFactory *factory : IPAProxyFactory::factories()) { From patchwork Wed Aug 26 11:09:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9401 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A209ABD87E for ; Wed, 26 Aug 2020 11:10:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6EEC0628DD; Wed, 26 Aug 2020 13:10:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pj1k5Tit"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BF924628BB for ; Wed, 26 Aug 2020 13:10:28 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0D9F253C; Wed, 26 Aug 2020 13:10:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1598440228; bh=QjT7FPHIivPKBBoHciyl3yu0BFsUF5GXIgvLXLy20Ng=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pj1k5Tit/gv+Sq7Wfys4sPUInA/pkKh9q5ur/sAeAmmr0Rzpgi5+9p73hupCZ48Zq Xbs58mRKeM6ynH17U9uQ3+c1v6BLMoyINc6jimCaT/zGV5DX8dJUGiCkZFVIEbDZvS MDNcELEWjS5DzF4/ul0AvT5e723Ch1j/E5FoN60E= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 26 Aug 2020 20:09:26 +0900 Message-Id: <20200826110926.67192-18-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200826110926.67192-1-paul.elder@ideasonboard.com> References: <20200826110926.67192-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 17/17] libcamera: IPAManager: add isolation flag to proxy creation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the IPA proxy is created, it needs to know whether to isolate or not. Feed the flag at creation of the IPA proxy. Signed-off-by: Paul Elder --- src/libcamera/ipa_manager.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp index 2d0ea242..26458153 100644 --- a/src/libcamera/ipa_manager.cpp +++ b/src/libcamera/ipa_manager.cpp @@ -291,7 +291,8 @@ std::unique_ptr IPAManager::createIPA(PipelineHandler *pipe, return nullptr; } - std::unique_ptr proxy = pf->create(m); + std::unique_ptr proxy = + pf->create(m, !self_->isSignatureValid(m)); if (!proxy->isValid()) { LOG(IPAManager, Error) << "Failed to load proxy"; return nullptr;