From patchwork Wed Apr 6 14:17:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 15636 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0181CC3256 for ; Wed, 6 Apr 2022 14:17:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AEE2A6563F; Wed, 6 Apr 2022 16:17:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1649254647; bh=2CgfBk3kfXhtt73drBveowytw0xH0Y+dlWm2vrcsysI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=A56GBqyG63n/oxPNDxB7ugWQ7vdeGOL3tNyRd93JvUpC4qB1hSyXX/9Q4OQ48PdTE VYI1GbD7vh/b/ljXrIaazE664hq8WEhY/9B7lXmWUbnVDDOkm7vvAL6+ki5tQyNb/B PWlv9PwUgXIZIoWydDdT/PV9mmIh5d/nIQcjGq6SkGlq8zQqEyTwxVpyykykt3XGC3 sW0FEkYiuh5E2SCM31wyXRpEtOVT8T6GqzpAOLx+78wyXD7vzG4FJExfq6CBAfUAuG tV4xSVpfzbQJw+hoBkTrkBJMR8WJsdrhiNcnKWSlUzHEtdh4hZbZoM6R6BgeqHHPBn hIRsGNNM8w6mw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2291A633A4 for ; Wed, 6 Apr 2022 16:17:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="daxKXdzu"; dkim-atps=neutral Received: from perceval.ideasonboard.com (unknown [27.57.186.178]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id ACF4D482; Wed, 6 Apr 2022 16:17:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1649254644; bh=2CgfBk3kfXhtt73drBveowytw0xH0Y+dlWm2vrcsysI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=daxKXdzuIBrxB8HejPlSMdkRkcVvjJlGhb8rfFdMBSkv+e9acmqGt+tXsFy4U7cW+ vnle+7uiWpwOS23EG/DsmwhH+L08fbXGaKZVWJwlGXstWzetxd6/6GcmQInmwgAinh OEbsGVl9bGuHYQGRzbANTvjO4/eGQBej77tnH9lw= To: libcamera-devel@lists.libcamera.org Date: Wed, 6 Apr 2022 19:47:04 +0530 Message-Id: <20220406141709.164794-2-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220406141709.164794-1-umang.jain@ideasonboard.com> References: <20220406141709.164794-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 1/6] ipa: ipu3: Replace event-based ops with dedicated functions X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The IPAIPU3 interface currently uses event-type based structures in order to communicate with the pipeline-handler (and vice-versa). Replace the event based structures with dedicated functions associated to each operation. The translated naming scheme of operations to dedicated functions: ActionSetSensorControls => setSensorControls ActionParamFilled => paramsBufferReady ActionMetadataReady => metadataReady EventProcessControls => queueRequest() EventStatReady => processStatsBuffer() EventFillParams => fillParamsBuffer() Signed-off-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder --- include/libcamera/ipa/ipu3.mojom | 36 ++----- src/ipa/ipu3/ipu3-ipa-design-guide.rst | 27 +++--- src/ipa/ipu3/ipu3.cpp | 129 +++++++++++-------------- src/libcamera/pipeline/ipu3/ipu3.cpp | 122 ++++++++++------------- 4 files changed, 130 insertions(+), 184 deletions(-) diff --git a/include/libcamera/ipa/ipu3.mojom b/include/libcamera/ipa/ipu3.mojom index 18cdc744..d1b1c6b8 100644 --- a/include/libcamera/ipa/ipu3.mojom +++ b/include/libcamera/ipa/ipu3.mojom @@ -8,32 +8,6 @@ module ipa.ipu3; import "include/libcamera/ipa/core.mojom"; -enum IPU3Operations { - ActionSetSensorControls = 1, - ActionParamFilled = 2, - ActionMetadataReady = 3, - EventProcessControls = 4, - EventStatReady = 5, - EventFillParams = 6, -}; - -struct IPU3Event { - IPU3Operations op; - uint32 frame; - int64 frameTimestamp; - uint32 bufferId; - libcamera.ControlList controls; - libcamera.ControlList sensorControls; - libcamera.ControlList lensControls; -}; - -struct IPU3Action { - IPU3Operations op; - libcamera.ControlList controls; - libcamera.ControlList sensorControls; - libcamera.ControlList lensControls; -}; - struct IPAConfigInfo { libcamera.IPACameraSensorInfo sensorInfo; libcamera.ControlInfoMap sensorControls; @@ -56,9 +30,15 @@ interface IPAIPU3Interface { mapBuffers(array buffers); unmapBuffers(array ids); - [async] processEvent(IPU3Event ev); + [async] queueRequest(uint32 frame, libcamera.ControlList controls); + [async] fillParamsBuffer(uint32 frame, uint32 bufferId); + [async] processStatsBuffer(uint32 frame, int64 frameTimestamp, + uint32 bufferId, libcamera.ControlList sensorControls); }; interface IPAIPU3EventInterface { - queueFrameAction(uint32 frame, IPU3Action action); + setSensorControls(uint32 frame, libcamera.ControlList sensorControls, + libcamera.ControlList lensControls); + paramsBufferReady(uint32 frame); + metadataReady(uint32 frame, libcamera.ControlList metadata); }; diff --git a/src/ipa/ipu3/ipu3-ipa-design-guide.rst b/src/ipa/ipu3/ipu3-ipa-design-guide.rst index 89c71108..e724fdda 100644 --- a/src/ipa/ipu3/ipu3-ipa-design-guide.rst +++ b/src/ipa/ipu3/ipu3-ipa-design-guide.rst @@ -25,7 +25,8 @@ from applications, and managing events from the pipeline handler. └─┬───┬───┬──────┬────┬────┬────┬─┴────▼─┬──┘ 1: init() │ │ │ │ ▲ │ ▲ │ ▲ │ ▲ │ 2: configure() │1 │2 │3 │4│ │4│ │4│ │4│ │5 3: mapBuffers(), start() - ▼ ▼ ▼ ▼ │ ▼ │ ▼ │ ▼ │ ▼ 4: processEvent() + │ │ │ │ │ │ │ │ │ │ │ │ 4: (▼) queueRequest(), fillParamsBuffer(), processStatsBuffer() + ▼ ▼ ▼ ▼ │ ▼ │ ▼ │ ▼ │ ▼ (▲) setSensorControls, paramsBufferReady, metadataReady Signals ┌──────────────────┴────┴────┴────┴─────────┐ 5: stop(), unmapBuffers() │ IPU3 IPA │ │ ┌───────────────────────┐ │ @@ -100,8 +101,9 @@ There are three main interactions with the algorithms for the IPU3 IPA to operate when running: - configure() -- processEvent(``EventFillParams``) -- processEvent(``EventStatReady``) +- queueRequest() +- fillParamsBuffer() +- processStatsBuffer() The configuration phase allows the pipeline-handler to inform the IPA of the current stream configurations, which is then passed into each @@ -114,8 +116,8 @@ Pre-frame preparation When configured, the IPA is notified by the pipeline handler of the Camera ``start()`` event, after which incoming requests will be queued for processing, requiring a parameter buffer (``ipu3_uapi_params``) to -be populated for the ImgU. This is given to the IPA through the -``EventFillParams`` event, and then passed directly to each algorithm +be populated for the ImgU. This is given to the IPA through +``fillParamsBuffer()``, and then passed directly to each algorithm through the ``prepare()`` call allowing the ISP configuration to be updated for the needs of each component that the algorithm is responsible for. @@ -125,7 +127,7 @@ structure that it modifies, and it should take care to ensure that any structure set by a use flag is fully initialised to suitable values. The parameter buffer is returned to the pipeline handler through the -``ActionParamFilled`` event, and from there queued to the ImgU along +``paramsBufferReady`` signal, and from there queued to the ImgU along with a raw frame captured with the CIO2. Post-frame completion @@ -133,8 +135,8 @@ Post-frame completion When the capture of an image is completed, and successfully processed through the ImgU, the generated statistics buffer -(``ipu3_uapi_stats_3a``) is given to the IPA through the -``EventStatReady`` event. This provides the IPA with an opportunity to +(``ipu3_uapi_stats_3a``) is given to the IPA through +``processStatsBuffer()``. This provides the IPA with an opportunity to examine the results of the ISP and run the calculations required by each algorithm on the new data. The algorithms may require context from the operations of other algorithms, for example, the AWB might choose to use @@ -145,11 +147,14 @@ before they are needed. The ordering of the algorithm processing is determined by their placement in the ``IPU3::algorithms_`` ordered list. +Finally, the IPA metadata for the completed frame is returned back via +the ``metadataReady`` signal. + Sensor Controls ~~~~~~~~~~~~~~~ The AutoExposure and AutoGain (AGC) algorithm differs slightly from the others as it requires operating directly on the sensor, as opposed to -through the ImgU ISP. To support this, there is a dedicated action -`ActionSetSensorControls` to allow the IPA to request controls to be set -on the camera sensor through the pipeline handler. +through the ImgU ISP. To support this, there is a ``setSensorControls`` +signal to allow the IPA to request controls to be set on the camera +sensor through the pipeline handler. diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 50b52d8b..9a59f80f 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -65,8 +65,9 @@ namespace ipa::ipu3 { * The IPU3 Pipeline defines an IPU3-specific interface for communication * between the PipelineHandler and the IPA module. * - * We extend the IPAIPU3Interface to implement our algorithms and handle events - * from the IPU3 PipelineHandler to satisfy requests from the application. + * We extend the IPAIPU3Interface to implement our algorithms and handle + * calls from the IPU3 PipelineHandler to satisfy requests from the + * application. * * At initialisation time, a CameraSensorHelper is instantiated to support * camera-specific calculations, while the default controls are computed, and @@ -81,14 +82,14 @@ namespace ipa::ipu3 { * parameter buffer, and adapting the settings of the sensor attached to the * IPU3 CIO2 through sensor-specific V4L2 controls. * - * When the event \a EventFillParams occurs we populate the ImgU parameter - * buffer with settings to configure the device in preparation for handling the - * frame queued in the Request. + * In fillParamsBuffer(), we populate the ImgU parameter buffer with + * settings to configure the device in preparation for handling the frame + * queued in the Request. * * When the frame has completed processing, the ImgU will generate a statistics - * buffer which is given to the IPA as part of the \a EventStatReady event. At - * this event we run the algorithms to parse the statistics and cache any - * results for the next \a EventFillParams event. + * buffer which is given to the IPA with processStatsBuffer(). In this we run the + * algorithms to parse the statistics and cache any results for the next + * fillParamsBuffer() call. * * The individual algorithms are split into modular components that are called * iteratively to allow them to process statistics from the ImgU in a defined @@ -143,14 +144,18 @@ public: void mapBuffers(const std::vector &buffers) override; void unmapBuffers(const std::vector &ids) override; - void processEvent(const IPU3Event &event) override; + void queueRequest(const uint32_t frame, const ControlList &controls) override; + void fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) override; + void processStatsBuffer(const uint32_t frame, const int64_t frameTimestamp, + const uint32_t bufferId, + const ControlList &sensorControls) override; private: void updateControls(const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls); void updateSessionConfiguration(const ControlInfoMap &sensorControls); - void processControls(unsigned int frame, const ControlList &controls); + void fillParams(unsigned int frame, ipu3_uapi_params *params); void parseStatistics(unsigned int frame, int64_t frameTimestamp, @@ -505,73 +510,61 @@ void IPAIPU3::unmapBuffers(const std::vector &ids) } /** - * \brief Process an event generated by the pipeline handler - * \param[in] event The event sent from pipeline handler - * - * The expected event handling over the lifetime of a Request has - * the following sequence: - * - * - EventProcessControls : Handle controls from a new Request - * - EventFillParams : Prepare the ISP to process the Request - * - EventStatReady : Process statistics after ISP completion + * \brief Fill and return a buffer with ISP processing parameters for a frame + * \param[in] frame The frame number + * \param[in] bufferId ID of the parameter buffer to fill */ -void IPAIPU3::processEvent(const IPU3Event &event) +void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) { - switch (event.op) { - case EventProcessControls: { - processControls(event.frame, event.controls); - break; + auto it = buffers_.find(bufferId); + if (it == buffers_.end()) { + LOG(IPAIPU3, Error) << "Could not find param buffer!"; + return; } - case EventFillParams: { - auto it = buffers_.find(event.bufferId); - if (it == buffers_.end()) { - LOG(IPAIPU3, Error) << "Could not find param buffer!"; - return; - } - Span mem = it->second.planes()[0]; - ipu3_uapi_params *params = - reinterpret_cast(mem.data()); + Span mem = it->second.planes()[0]; + ipu3_uapi_params *params = + reinterpret_cast(mem.data()); - fillParams(event.frame, params); - break; - } - case EventStatReady: { - auto it = buffers_.find(event.bufferId); - if (it == buffers_.end()) { - LOG(IPAIPU3, Error) << "Could not find stats buffer!"; - return; - } + fillParams(frame, params); +} - Span mem = it->second.planes()[0]; - const ipu3_uapi_stats_3a *stats = - reinterpret_cast(mem.data()); +/** + * \brief Process statistics after ISP completion + * \param[in] frame The frame number + * \param[in] frameTimestamp Timestamp of the frame + * \param[in] bufferId ID of the statistics buffer + * \param[in] sensorControls Sensor controls + */ +void IPAIPU3::processStatsBuffer(const uint32_t frame, const int64_t frameTimestamp, + const uint32_t bufferId, const ControlList &sensorControls) +{ + auto it = buffers_.find(bufferId); + if (it == buffers_.end()) { + LOG(IPAIPU3, Error) << "Could not find stats buffer!"; + return; + } - int32_t exposure = event.sensorControls.get(V4L2_CID_EXPOSURE).get(); - int32_t gain = event.sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get(); + Span mem = it->second.planes()[0]; + const ipu3_uapi_stats_3a *stats = + reinterpret_cast(mem.data()); - context_.frameContext.sensor.exposure = exposure; - context_.frameContext.sensor.gain = camHelper_->gain(gain); + context_.frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); + context_.frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); - parseStatistics(event.frame, event.frameTimestamp, stats); - break; - } - default: - LOG(IPAIPU3, Error) << "Unknown event " << event.op; - break; - } + parseStatistics(frame, frameTimestamp, stats); } /** - * \brief Process a control list for a request from the application + * \brief Queue a request and process the control list from the application * \param[in] frame The number of the frame which will be processed next * \param[in] controls The controls for the \a frame * * Parse the request to handle any IPA-managed controls that were set from the * application such as manual sensor settings. */ -void IPAIPU3::processControls([[maybe_unused]] unsigned int frame, - [[maybe_unused]] const ControlList &controls) +void IPAIPU3::queueRequest(const uint32_t frame, + [[maybe_unused]] const ControlList &controls) { /* \todo Start processing for 'frame' based on 'controls'. */ } @@ -600,10 +593,7 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) for (auto const &algo : algorithms_) algo->prepare(context_, params); - IPU3Action op; - op.op = ActionParamFilled; - - queueFrameAction.emit(frame, op); + paramsBufferReady.emit(frame); } /** @@ -647,11 +637,7 @@ void IPAIPU3::parseStatistics(unsigned int frame, * likely want to avoid putting platform specific metadata in. */ - IPU3Action op; - op.op = ActionMetadataReady; - op.controls = ctrls; - - queueFrameAction.emit(frame, op); + metadataReady.emit(frame, ctrls); } /** @@ -663,23 +649,18 @@ void IPAIPU3::parseStatistics(unsigned int frame, */ void IPAIPU3::setControls(unsigned int frame) { - IPU3Action op; - op.op = ActionSetSensorControls; - int32_t exposure = context_.frameContext.agc.exposure; int32_t gain = camHelper_->gainCode(context_.frameContext.agc.gain); ControlList ctrls(sensorCtrls_); ctrls.set(V4L2_CID_EXPOSURE, exposure); ctrls.set(V4L2_CID_ANALOGUE_GAIN, gain); - op.sensorControls = ctrls; ControlList lensCtrls(lensCtrls_); lensCtrls.set(V4L2_CID_FOCUS_ABSOLUTE, static_cast(context_.frameContext.af.focus)); - op.lensControls = lensCtrls; - queueFrameAction.emit(frame, op); + setSensorControls.emit(frame, ctrls, lensCtrls); } } /* namespace ipa::ipu3 */ diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 60e01917..59d7e9c0 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -86,8 +86,10 @@ public: ControlInfoMap ipaControls_; private: - void queueFrameAction(unsigned int id, - const ipa::ipu3::IPU3Action &action); + void metadataReady(unsigned int id, const ControlList &metadata); + void paramsFilled(unsigned int id); + void setSensorControls(unsigned int id, const ControlList &sensorControls, + const ControlList &lensControls); }; class IPU3CameraConfiguration : public CameraConfiguration @@ -871,11 +873,7 @@ void IPU3CameraData::queuePendingRequests() info->rawBuffer = rawBuffer; - ipa::ipu3::IPU3Event ev; - ev.op = ipa::ipu3::EventProcessControls; - ev.frame = info->id; - ev.controls = request->controls(); - ipa_->processEvent(ev); + ipa_->queueRequest(info->id, request->controls()); pendingRequests_.pop(); processingRequests_.push(request); @@ -1218,7 +1216,9 @@ int IPU3CameraData::loadIPA() if (!ipa_) return -ENOENT; - ipa_->queueFrameAction.connect(this, &IPU3CameraData::queueFrameAction); + ipa_->setSensorControls.connect(this, &IPU3CameraData::setSensorControls); + ipa_->paramsBufferReady.connect(this, &IPU3CameraData::paramsFilled); + ipa_->metadataReady.connect(this, &IPU3CameraData::metadataReady); /* * Pass the sensor info to the IPA to initialize controls. @@ -1253,69 +1253,58 @@ int IPU3CameraData::loadIPA() return 0; } -void IPU3CameraData::queueFrameAction(unsigned int id, - const ipa::ipu3::IPU3Action &action) +void IPU3CameraData::setSensorControls([[maybe_unused]] unsigned int id, + const ControlList &sensorControls, + const ControlList &lensControls) { - switch (action.op) { - case ipa::ipu3::ActionSetSensorControls: { - const ControlList &sensorControls = action.sensorControls; - delayedCtrls_->push(sensorControls); + delayedCtrls_->push(sensorControls); - CameraLens *focusLens = cio2_.sensor()->focusLens(); - if (!focusLens) - break; - - const ControlList lensControls = action.lensControls; - if (!lensControls.contains(V4L2_CID_FOCUS_ABSOLUTE)) - break; + CameraLens *focusLens = cio2_.sensor()->focusLens(); + if (!focusLens) + return; - const ControlValue &focusValue = - lensControls.get(V4L2_CID_FOCUS_ABSOLUTE); + if (!lensControls.contains(V4L2_CID_FOCUS_ABSOLUTE)) + return; - focusLens->setFocusPosition(focusValue.get()); + const ControlValue &focusValue = lensControls.get(V4L2_CID_FOCUS_ABSOLUTE); - break; - } - case ipa::ipu3::ActionParamFilled: { - IPU3Frames::Info *info = frameInfos_.find(id); - if (!info) - break; - - /* Queue all buffers from the request aimed for the ImgU. */ - for (auto it : info->request->buffers()) { - const Stream *stream = it.first; - FrameBuffer *outbuffer = it.second; + focusLens->setFocusPosition(focusValue.get()); +} - if (stream == &outStream_) - imgu_->output_->queueBuffer(outbuffer); - else if (stream == &vfStream_) - imgu_->viewfinder_->queueBuffer(outbuffer); - } +void IPU3CameraData::paramsFilled(unsigned int id) +{ + IPU3Frames::Info *info = frameInfos_.find(id); + if (!info) + return; - imgu_->param_->queueBuffer(info->paramBuffer); - imgu_->stat_->queueBuffer(info->statBuffer); - imgu_->input_->queueBuffer(info->rawBuffer); + /* Queue all buffers from the request aimed for the ImgU. */ + for (auto it : info->request->buffers()) { + const Stream *stream = it.first; + FrameBuffer *outbuffer = it.second; - break; + if (stream == &outStream_) + imgu_->output_->queueBuffer(outbuffer); + else if (stream == &vfStream_) + imgu_->viewfinder_->queueBuffer(outbuffer); } - case ipa::ipu3::ActionMetadataReady: { - IPU3Frames::Info *info = frameInfos_.find(id); - if (!info) - break; - Request *request = info->request; - request->metadata().merge(action.controls); + imgu_->param_->queueBuffer(info->paramBuffer); + imgu_->stat_->queueBuffer(info->statBuffer); + imgu_->input_->queueBuffer(info->rawBuffer); +} - info->metadataProcessed = true; - if (frameInfos_.tryComplete(info)) - pipe()->completeRequest(request); +void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) +{ + IPU3Frames::Info *info = frameInfos_.find(id); + if (!info) + return; - break; - } - default: - LOG(IPU3, Error) << "Unknown action " << action.op; - break; - } + Request *request = info->request; + request->metadata().merge(metadata); + + info->metadataProcessed = true; + if (frameInfos_.tryComplete(info)) + pipe()->completeRequest(request); } /* ----------------------------------------------------------------------------- @@ -1390,11 +1379,7 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) if (request->findBuffer(&rawStream_)) pipe()->completeBuffer(request, buffer); - ipa::ipu3::IPU3Event ev; - ev.op = ipa::ipu3::EventFillParams; - ev.frame = info->id; - ev.bufferId = info->paramBuffer->cookie(); - ipa_->processEvent(ev); + ipa_->fillParamsBuffer(info->id, info->paramBuffer->cookie()); } void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) @@ -1438,13 +1423,8 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer) return; } - ipa::ipu3::IPU3Event ev; - ev.op = ipa::ipu3::EventStatReady; - ev.frame = info->id; - ev.bufferId = info->statBuffer->cookie(); - ev.frameTimestamp = request->metadata().get(controls::SensorTimestamp); - ev.sensorControls = info->effectiveSensorControls; - ipa_->processEvent(ev); + ipa_->processStatsBuffer(info->id, request->metadata().get(controls::SensorTimestamp), + info->statBuffer->cookie(), info->effectiveSensorControls); } /* From patchwork Wed Apr 6 14:17:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 15637 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 52C99C3256 for ; Wed, 6 Apr 2022 14:17:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1C77165647; Wed, 6 Apr 2022 16:17:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1649254651; bh=wd10aGLKpxSp/InQAIJ4yIW1zodreFRdlgdg/HVS/EU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=TGq8SaQDkfSf9CQ6bXvu43RqUhwXdxj+gY8Edq9saE+AlhMUNVx8lj+9mvPqrHSzW cj+iZlQdNwaz5j2pNXvDVtNP9nDKM6N+InRWpQcTDdE0T89caZ11XcIM33SfkeLhTL Wvg31LAymPHQde2nl3HFmem+vSiPX0HKTll9D5XyLFTS3humkj3fqG4vJ897y4+dz3 u/+RPbsNIMWxxqcE3SYpwNTNmcpeBPVF+QsRvIBQNJN/aMr5ZNt5pILC8jh1sX1v3c CCpVwWWJqw82AyvIHQKbNmFYmVCGoAFsvz/6/ZM8R8K7e8mLMkK/zL6M/VPr8NPOFf S9ffBR5ODDO7Q== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C1D72633A4 for ; Wed, 6 Apr 2022 16:17:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HF3Txxgr"; dkim-atps=neutral Received: from perceval.ideasonboard.com (unknown [27.57.186.178]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C5428482; Wed, 6 Apr 2022 16:17:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1649254649; bh=wd10aGLKpxSp/InQAIJ4yIW1zodreFRdlgdg/HVS/EU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HF3TxxgrEiDgZNZtLu044m7KrgIV88kgPhqOicJ+ExdrZyzuTzuY48udPGpzCJKcg ex6c3Cm2vcG+B37n+0TEHFZ0mkF9Qz9yYhDOl38dSwZQzv7FPAYykdH4HeOvqruKYv 9WpuHu9UadGdFcR8UZ6au1AKNMZkNLcSB+H+oZgY= To: libcamera-devel@lists.libcamera.org Date: Wed, 6 Apr 2022 19:47:05 +0530 Message-Id: <20220406141709.164794-3-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220406141709.164794-1-umang.jain@ideasonboard.com> References: <20220406141709.164794-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 2/6] ipa: ipu3: Inline fillParams() in fillParamsBuffer() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Since we have moved away from switch/case on the operation ID, there's little reason to split the operation in two functions. Signed-off-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder --- src/ipa/ipu3/ipu3.cpp | 47 +++++++++++++++++-------------------------- 1 file changed, 18 insertions(+), 29 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 9a59f80f..8c9a20f5 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -156,7 +156,6 @@ private: ControlInfoMap *ipaControls); void updateSessionConfiguration(const ControlInfoMap &sensorControls); - void fillParams(unsigned int frame, ipu3_uapi_params *params); void parseStatistics(unsigned int frame, int64_t frameTimestamp, const ipu3_uapi_stats_3a *stats); @@ -513,6 +512,9 @@ void IPAIPU3::unmapBuffers(const std::vector &ids) * \brief Fill and return a buffer with ISP processing parameters for a frame * \param[in] frame The frame number * \param[in] bufferId ID of the parameter buffer to fill + * + * Algorithms are expected to fill the IPU3 parameter buffer for the next + * frame given their most recent processing of the ImgU statistics. */ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) { @@ -526,7 +528,21 @@ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) ipu3_uapi_params *params = reinterpret_cast(mem.data()); - fillParams(frame, params); + /* + * The incoming params buffer may contain uninitialised data, or the + * parameters of previously queued frames. Clearing the entire buffer + * may be an expensive operation, and the kernel will only read from + * structures which have their associated use-flag set. + * + * It is the responsibility of the algorithms to set the use flags + * accordingly for any data structure they update during prepare(). + */ + params->use = {}; + + for (auto const &algo : algorithms_) + algo->prepare(context_, params); + + paramsBufferReady.emit(frame); } /** @@ -569,33 +585,6 @@ void IPAIPU3::queueRequest(const uint32_t frame, /* \todo Start processing for 'frame' based on 'controls'. */ } -/** - * \brief Fill the ImgU parameter buffer for the next frame - * \param[in] frame The number of the latest frame processed - * \param[out] params The parameter buffer to fill - * - * Algorithms are expected to fill the IPU3 parameter buffer for the next - * frame given their most recent processing of the ImgU statistics. - */ -void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) -{ - /* - * The incoming params buffer may contain uninitialised data, or the - * parameters of previously queued frames. Clearing the entire buffer - * may be an expensive operation, and the kernel will only read from - * structures which have their associated use-flag set. - * - * It is the responsibility of the algorithms to set the use flags - * accordingly for any data structure they update during prepare(). - */ - params->use = {}; - - for (auto const &algo : algorithms_) - algo->prepare(context_, params); - - paramsBufferReady.emit(frame); -} - /** * \brief Process the statistics generated by the ImgU * \param[in] frame The number of the latest frame processed From patchwork Wed Apr 6 14:17:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 15638 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1A1F6C3256 for ; Wed, 6 Apr 2022 14:17:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C040B65644; Wed, 6 Apr 2022 16:17:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1649254653; bh=DCDdXzR5EXqotv0wLejnkY0Hx6RbJS0oVZfHYjTbqGo=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=1IPpvbccylA+xx88CsjIBiZGtOt0kbeyLHMxW7F2LpmKg1blWESdzIcgq6LyxXZpB +bB/HyH6SgiqcfQahee7UUhB4+6Be4m9CZsmCEBuXL2VogVywjXjpuj6YDjzCzW08W MZ+T/LrXLA5XmkZY53Ppf9HAYnrIJ2BFVkJ1t8ud7AhOMIdb6cqp7LWmotyddDxizT nJFOHmJT+nuJHcR2FlaVEfg+2HsBV+ZsS2MgTWVgFRdg739IZpj1T9WQPlEv63a5xd QtSZxxHLXba7Vj+vlbKTbMFODqjMvrbEhrFuKvovppeaQPi7d6QwjUZgSETaDOC+lb f1jMU9I0SyOzw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 25BE9633A4 for ; Wed, 6 Apr 2022 16:17:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gv621aeT"; dkim-atps=neutral Received: from perceval.ideasonboard.com (unknown [27.57.186.178]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BD2BA482; Wed, 6 Apr 2022 16:17:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1649254651; bh=DCDdXzR5EXqotv0wLejnkY0Hx6RbJS0oVZfHYjTbqGo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gv621aeTDtQTRIijaN6a9rL8LjAltJs8bgMTVxmNIRJCoQi0hvZdSVmUFZ4h4sLVK a0+wOqBaYqRaZCfzPqtbXUTL4a2G9RyzwOjFMxltJxbgVqMHOqQ4R9W7DrRezErWAD oOxzFekLwcGUA14blZnZjHir9apXn/Sj7zvjFiv4= To: libcamera-devel@lists.libcamera.org Date: Wed, 6 Apr 2022 19:47:06 +0530 Message-Id: <20220406141709.164794-4-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220406141709.164794-1-umang.jain@ideasonboard.com> References: <20220406141709.164794-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 3/6] ipa: ipu3: Inline parseStatistics() into processStatsBuffer() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Since we have moved away from switch/case on the operation ID, there's little reason to split the operation in two functions. Signed-off-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder --- src/ipa/ipu3/ipu3.cpp | 54 +++++++++++++++---------------------------- 1 file changed, 19 insertions(+), 35 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 8c9a20f5..569c1311 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -156,9 +156,6 @@ private: ControlInfoMap *ipaControls); void updateSessionConfiguration(const ControlInfoMap &sensorControls); - void parseStatistics(unsigned int frame, - int64_t frameTimestamp, - const ipu3_uapi_stats_3a *stats); bool validateSensorControls(); void setControls(unsigned int frame); @@ -546,11 +543,15 @@ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) } /** - * \brief Process statistics after ISP completion + * \brief Process the statistics generated by the ImgU * \param[in] frame The frame number * \param[in] frameTimestamp Timestamp of the frame * \param[in] bufferId ID of the statistics buffer * \param[in] sensorControls Sensor controls + * + * Parse the most recently processed image statistics from the ImgU. The + * statistics are passed to each algorithm module to run their calculations and + * update their state accordingly. */ void IPAIPU3::processStatsBuffer(const uint32_t frame, const int64_t frameTimestamp, const uint32_t bufferId, const ControlList &sensorControls) @@ -568,37 +569,6 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, const int64_t frameTimest context_.frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); context_.frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); - parseStatistics(frame, frameTimestamp, stats); -} - -/** - * \brief Queue a request and process the control list from the application - * \param[in] frame The number of the frame which will be processed next - * \param[in] controls The controls for the \a frame - * - * Parse the request to handle any IPA-managed controls that were set from the - * application such as manual sensor settings. - */ -void IPAIPU3::queueRequest(const uint32_t frame, - [[maybe_unused]] const ControlList &controls) -{ - /* \todo Start processing for 'frame' based on 'controls'. */ -} - -/** - * \brief Process the statistics generated by the ImgU - * \param[in] frame The number of the latest frame processed - * \param[in] frameTimestamp The current frame timestamp - * \param[in] stats The IPU3 statistics and ISP results - * - * Parse the most recently processed image statistics from the ImgU. The - * statistics are passed to each algorithm module to run their calculations and - * update their state accordingly. - */ -void IPAIPU3::parseStatistics(unsigned int frame, - [[maybe_unused]] int64_t frameTimestamp, - const ipu3_uapi_stats_3a *stats) -{ double lineDuration = context_.configuration.sensor.lineDuration.get(); int32_t vBlank = context_.configuration.sensor.defVBlank; ControlList ctrls(controls::controls); @@ -629,6 +599,20 @@ void IPAIPU3::parseStatistics(unsigned int frame, metadataReady.emit(frame, ctrls); } +/** + * \brief Queue a request and process the control list from the application + * \param[in] frame The number of the frame which will be processed next + * \param[in] controls The controls for the \a frame + * + * Parse the request to handle any IPA-managed controls that were set from the + * application such as manual sensor settings. + */ +void IPAIPU3::queueRequest(const uint32_t frame, + [[maybe_unused]] const ControlList &controls) +{ + /* \todo Start processing for 'frame' based on 'controls'. */ +} + /** * \brief Handle sensor controls for a given \a frame number * \param[in] frame The frame on which the sensor controls should be set From patchwork Wed Apr 6 14:17:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 15639 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7F7C3C3256 for ; Wed, 6 Apr 2022 14:17:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4A11765649; Wed, 6 Apr 2022 16:17:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1649254656; bh=OguJDJPfaEXUOIkbJ7xjx5mE7uSsWhyhsfTUwBVetwM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Mzq7midEXfuxmMaZ4SWiGla18RthZbquVl8H+mIotCe0JxHG0wA6UV1MJpecaKRTs 0kZydeIWSLqahPVfqTlqS2m43QNSEqFbaX9OfjVQVGY2jwnxzfR61jzl+pm4Voqr6b bxvHgn+N2HKUxawHU6PrHJFUwVhCoEbBcgupG5OWpqIFydFNsWe5Cl0zaRVklSuoYp LSU4agn2bvtVdaecW9jcyIumJxCWeu5NSCUtdRkpDUr49jOFpMcJ4e7H5SaTGveuP/ /fMArkObaRcRWNV1Xtzur9Z5JImKprvZIlRPYaX4jSg/I+6qKnm8Wdra4dc0/zj9g5 2xOOBJBDpK3Rw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ED43C633A4 for ; Wed, 6 Apr 2022 16:17:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cZSMxoGw"; dkim-atps=neutral Received: from perceval.ideasonboard.com (unknown [27.57.186.178]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 098E5482; Wed, 6 Apr 2022 16:17:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1649254654; bh=OguJDJPfaEXUOIkbJ7xjx5mE7uSsWhyhsfTUwBVetwM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cZSMxoGwUIsGN+pvxyHli1ru6Lb5fEXUw00ppXnZB4JMzbPNBlZvTahiylibHFIOZ iB3egoueImhwXZqjvYLGwGsSeK6SWhRT8vrvZ54O/pbpCqkUptfGufduzNrfaJQQw9 lHLmalyK9LhrN7C0wHaORqVMuEqW6mJMnYFoiX6U= To: libcamera-devel@lists.libcamera.org Date: Wed, 6 Apr 2022 19:47:07 +0530 Message-Id: <20220406141709.164794-5-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220406141709.164794-1-umang.jain@ideasonboard.com> References: <20220406141709.164794-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [IPAIPU3 PATCH v5 4/6] ipu3: Use the new IPAIPU3 interface X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the new IPAIPU3 interface that migrates the event-based ops to dedication functions. Signed-off-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder --- ipu3.cpp | 101 ++++++++++++++++++++++--------------------------------- 1 file changed, 40 insertions(+), 61 deletions(-) diff --git a/ipu3.cpp b/ipu3.cpp index 6ab86e0..b6d2129 100644 --- a/ipu3.cpp +++ b/ipu3.cpp @@ -49,13 +49,16 @@ public: void mapBuffers(const std::vector &buffers) override; void unmapBuffers(const std::vector &ids) override; - void processEvent(const IPU3Event &event) override; + void queueRequest(const uint32_t frame, const ControlList &controls) override; + void fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) override; + void processStatsBuffer(const uint32_t frame, const int64_t frameTimestamp, + const uint32_t bufferId, + const ControlList &sensorControls) override; private: void updateControls(const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls); - void processControls(unsigned int frame, const ControlList &metadata); void runAiq(unsigned int frame); void fillParams(unsigned int frame, ipu3_uapi_params *params); void parseStatistics(unsigned int frame, @@ -321,54 +324,41 @@ void IPAIPU3::unmapBuffers(const std::vector &ids) } } -void IPAIPU3::processEvent(const IPU3Event &event) +void IPAIPU3::queueRequest([[maybe_unused]] unsigned int frame, + [[maybe_unused]] const ControlList &controls) { - switch (event.op) { - case EventProcessControls: { - processControls(event.frame, event.controls); - break; - } - case EventStatReady: { - auto it = buffers_.find(event.bufferId); - if (it == buffers_.end()) { - LOG(IPAIPU3, Error) << "Could not find stats buffer!"; - return; - } - - Span mem = it->second.maps()[0]; - const ipu3_uapi_stats_3a *stats = - reinterpret_cast(mem.data()); - - parseStatistics(event.frame, - event.frameTimestamp, - stats, - event.sensorControls); - break; - } - case EventFillParams: { - auto it = buffers_.find(event.bufferId); - if (it == buffers_.end()) { - LOG(IPAIPU3, Error) << "Could not find param buffer!"; - return; - } - - Span mem = it->second.maps()[0]; - ipu3_uapi_params *params = - reinterpret_cast(mem.data()); - - fillParams(event.frame, params); - break; - } - default: - LOG(IPAIPU3, Error) << "Unknown event " << event.op; - break; + /* \todo Start processing for 'frame' based on 'controls'. */ +} + +void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) +{ + auto it = buffers_.find(bufferId); + if (it == buffers_.end()) { + LOG(IPAIPU3, Error) << "Could not find params buffer"; + return; } + + Span mem = it->second.maps()[0]; + ipu3_uapi_params *params = + reinterpret_cast(mem.data()); + + fillParams(frame, params); } -void IPAIPU3::processControls([[maybe_unused]] unsigned int frame, - [[maybe_unused]] const ControlList &controls) +void IPAIPU3::processStatsBuffer(const uint32_t frame, const int64_t frameTimestamp, + const uint32_t bufferId, const ControlList &sensorControls) { - /* \todo Start processing for 'frame' based on 'controls'. */ + auto it = buffers_.find(bufferId); + if (it == buffers_.end()) { + LOG(IPAIPU3, Error) << "Could not find stats buffer"; + return; + } + + Span mem = it->second.maps()[0]; + const ipu3_uapi_stats_3a *stats = + reinterpret_cast(mem.data()); + + parseStatistics(frame, frameTimestamp, stats, sensorControls); } void IPAIPU3::runAiq([[maybe_unused]] unsigned int frame) @@ -424,10 +414,7 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) setControls(frame); - IPU3Action op; - op.op = ActionParamFilled; - - queueFrameAction.emit(frame, op); + paramsBufferReady.emit(frame); } void IPAIPU3::parseStatistics(unsigned int frame, @@ -488,27 +475,19 @@ void IPAIPU3::parseStatistics(unsigned int frame, (sensorInfo_.pixelRate / 1e6); ctrls.set(controls::FrameDuration, frameDuration); - IPU3Action op; - op.op = ActionMetadataReady; - op.controls = ctrls; - - queueFrameAction.emit(frame, op); + metadataReady.emit(frame, ctrls); } void IPAIPU3::setControls(unsigned int frame) { - IPU3Action op; - op.op = ActionSetSensorControls; - ControlList sensorCtrls(ctrls_); sensorCtrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); sensorCtrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); - op.sensorControls = sensorCtrls; - - op.lensControls.set(V4L2_CID_FOCUS_ABSOLUTE, lensPosition_); + ControlList lensCtrls(ctrls_); + lensCtrls.set(V4L2_CID_FOCUS_ABSOLUTE, lensPosition_); - queueFrameAction.emit(frame, op); + setSensorControls.emit(frame, sensorCtrls, lensCtrls); } } /* namespace ipa::ipu3 */ From patchwork Wed Apr 6 14:17:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 15640 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DA9A0C3256 for ; Wed, 6 Apr 2022 14:17:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A2DE865646; Wed, 6 Apr 2022 16:17:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1649254659; bh=Rf9JkFUqZrQKSWL4TWMesOvFCqEe7TSKyBCPUM4MhwU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Gms3GtEwXfDTLe92atPE1Sv7jyjJdIOiGGX68LuIWZVOpj+Y6jfhUTzyUGgCsixCN xcwm99l53iLjnm88XOqBxICRFQncNeKTaX4h6LGYBK8TIlOwi2N8kOSD7rWiuZ4fM9 kdpD2LBs/8dMpPvFLSTapulXSC9e9Qtt9ZRO6jiKz0gBMr2UWJTvHAl95hyeKhK3TG tTki2tdP2kZTftR0IMg4emX2XLo7VojYqZI0MxLzs6uy89W5OSlkQsrdMZLSndc6dE uFJSxftEW789rUcQY08COy9fXR+qjTG2npEUytn/MBwMOcjNQbOkQcJQR6vb4AZJun +2n+oRZCE15jg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E1999633A4 for ; Wed, 6 Apr 2022 16:17:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="SxEFZJyN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (unknown [27.57.186.178]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 19EF2482; Wed, 6 Apr 2022 16:17:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1649254657; bh=Rf9JkFUqZrQKSWL4TWMesOvFCqEe7TSKyBCPUM4MhwU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SxEFZJyNb5l8J0MceVPF8vgvTO8TyT10EYib2lDIxif3r0rMLrV2VI6Xw7CDd6GdU 6tCCEAhihnEOJJwBum8QBJFPkFvDjunEbLY2+STMquOQMpch9hoiIiCxEX3660ZSmI nMUt2NCRtS4NQ4sGggBUgYEtjkD70XRrAYDGkhqg= To: libcamera-devel@lists.libcamera.org Date: Wed, 6 Apr 2022 19:47:08 +0530 Message-Id: <20220406141709.164794-6-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220406141709.164794-1-umang.jain@ideasonboard.com> References: <20220406141709.164794-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [IPAIPU3 PATCH v5 5/6] ipu3: Inline fillParams() in fillParamsBuffer() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Since we have moved away from switch/case on the operation ID, there's little reason to split the operation in two functions. Signed-off-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder --- ipu3.cpp | 52 +++++++++++++++++++++++----------------------------- 1 file changed, 23 insertions(+), 29 deletions(-) diff --git a/ipu3.cpp b/ipu3.cpp index b6d2129..0543a2e 100644 --- a/ipu3.cpp +++ b/ipu3.cpp @@ -60,7 +60,6 @@ private: const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls); void runAiq(unsigned int frame); - void fillParams(unsigned int frame, ipu3_uapi_params *params); void parseStatistics(unsigned int frame, int64_t frameTimestamp, const ipu3_uapi_stats_3a *stats, @@ -342,7 +341,29 @@ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) ipu3_uapi_params *params = reinterpret_cast(mem.data()); - fillParams(frame, params); + /* Prepare parameters buffer. */ + memset(params, 0, sizeof(*params)); + + /* + * Call into the AIQ object, and set up the library with any requested + * controls or settings from the incoming request. + * + * (statistics are fed into the library as a separate event + * when available) + * + * - Run algorithms + * + * - Fill params buffer with the results of the algorithms. + */ + runAiq(frame); + + aiq::AiqResults& latestResults = resultsHistory_.latest(); + aic_.updateRuntimeParams(latestResults); + aic_.run(params); + + setControls(frame); + + paramsBufferReady.emit(frame); } void IPAIPU3::processStatsBuffer(const uint32_t frame, const int64_t frameTimestamp, @@ -390,33 +411,6 @@ void IPAIPU3::runAiq([[maybe_unused]] unsigned int frame) lensPosition_ = latestResults.af()->next_lens_position; } -void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) -{ - /* Prepare parameters buffer. */ - memset(params, 0, sizeof(*params)); - - /* - * Call into the AIQ object, and set up the library with any requested - * controls or settings from the incoming request. - * - * (statistics are fed into the library as a separate event - * when available) - * - * - Run algorithms - * - * - Fill params buffer with the results of the algorithms. - */ - runAiq(frame); - - aiq::AiqResults& latestResults = resultsHistory_.latest(); - aic_.updateRuntimeParams(latestResults); - aic_.run(params); - - setControls(frame); - - paramsBufferReady.emit(frame); -} - void IPAIPU3::parseStatistics(unsigned int frame, int64_t frameTimestamp, const ipu3_uapi_stats_3a *stats, From patchwork Wed Apr 6 14:17:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 15641 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 65DEEC3256 for ; Wed, 6 Apr 2022 14:17:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2982165644; Wed, 6 Apr 2022 16:17:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1649254662; bh=hM+QtYApyJn6xFKXIdi7rcRUtSQGmYDu2MruGUIJ7Ww=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=qQqcQ4Onl3ked7bvCxnGFDtJYfS7a/jUK5rwkXN5Bl5cfGD0inlqgar6E40RrSpvC dNYiQTGk3tnEuHH2cEWMYtx/e2sRfMum/6Ze8WqDr/sP0TFEqoP+FWaxvmvzreT/eR ZQhMb0BFsL1l66CouSWomSSV19VhfbqdkzawbJavhkerDAn1N+H4VZwqmCVKpl/BEf pgvhfWYO1MVPDcWIrRg5ysLj8YoHjPGaryuMIgwfI7o4C8Py41nbj+75WNJGRGxtmY 4Mb9sjZSjjx4GCfgMybCBfjXg8/WfOKjSMcddAYglu4vRhjTgUk68muxfKJzM+Fr28 AijpUn++YX7wA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C05BC633A4 for ; Wed, 6 Apr 2022 16:17:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="EkkeVIJQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (unknown [27.57.186.178]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EC12E482; Wed, 6 Apr 2022 16:17:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1649254660; bh=hM+QtYApyJn6xFKXIdi7rcRUtSQGmYDu2MruGUIJ7Ww=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EkkeVIJQvg3Azvi4yboccaDWsNtP86H/lqYPXc59j88Annew+xQbMJkLqT6MweLu9 jIJ9RSHGM9CLObQL3rnkKAgRuSWUa3826tbv/a+FYSZvttuR7weliLPQF9X05NX4DG ItBqvndc96UIm6X/bNpQ7iiJgvjIylSnZPVchdAY= To: libcamera-devel@lists.libcamera.org Date: Wed, 6 Apr 2022 19:47:09 +0530 Message-Id: <20220406141709.164794-7-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220406141709.164794-1-umang.jain@ideasonboard.com> References: <20220406141709.164794-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [IPAIPU3 PATCH v5 6/6] ipa: ipu3: Inline parseStatistics() into processStatsBuffer() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Since we have moved away from switch/case on the operation ID, there's little reason to split the operation in two functions. Signed-off-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder --- ipu3.cpp | 74 ++++++++++++++++++++++++-------------------------------- 1 file changed, 31 insertions(+), 43 deletions(-) diff --git a/ipu3.cpp b/ipu3.cpp index 0543a2e..7ecd377 100644 --- a/ipu3.cpp +++ b/ipu3.cpp @@ -60,10 +60,6 @@ private: const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls); void runAiq(unsigned int frame); - void parseStatistics(unsigned int frame, - int64_t frameTimestamp, - const ipu3_uapi_stats_3a *stats, - const ControlList& sensorCtrls); void setControls(unsigned int frame); @@ -379,43 +375,6 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, const int64_t frameTimest const ipu3_uapi_stats_3a *stats = reinterpret_cast(mem.data()); - parseStatistics(frame, frameTimestamp, stats, sensorControls); -} - -void IPAIPU3::runAiq([[maybe_unused]] unsigned int frame) -{ - /* Run algorithms into/using this context structure */ - resultsHistory_.extendOne(); - aiq::AiqResults& latestResults = resultsHistory_.latest(); - - /* Todo: Refactor AiqInputParameters interface to set following parameters. */ - aiqInputParams_.afParams.lens_position = lensPosition_; - aiqInputParams_.afParams.lens_movement_start_timestamp = lensMovementStartTime_; - - aiq_.run2a(frame, aiqInputParams_, latestResults); - - exposure_ = latestResults.ae()->exposures[0].sensor_exposure->coarse_integration_time; - gain_ = latestResults.ae()->exposures[0].sensor_exposure->analog_gain_code_global; - - /* - * Af algorithm compares the timestamp of start of the lens movement and - * that of the statistics generated to estimate whether next lens - * position should be produced. - * \todo use the lens movement start time reported by the pipeline handler. - */ - if (lensPosition_ != latestResults.af()->next_lens_position) { - utils::time_point time = utils::clock::now(); - uint64_t usecs = std::chrono::duration_cast(time.time_since_epoch()).count(); - lensMovementStartTime_ = usecs; - } - lensPosition_ = latestResults.af()->next_lens_position; -} - -void IPAIPU3::parseStatistics(unsigned int frame, - int64_t frameTimestamp, - const ipu3_uapi_stats_3a *stats, - const ControlList& sensorCtrls) -{ ControlList ctrls(controls::controls); /* \todo React to statistics and update internal state machine. */ @@ -438,11 +397,11 @@ void IPAIPU3::parseStatistics(unsigned int frame, int32_t effectiveGain = 0; ControlValue ctrlValue; - ctrlValue = sensorCtrls.get(V4L2_CID_EXPOSURE); + ctrlValue = sensorControls.get(V4L2_CID_EXPOSURE); if (!ctrlValue.isNone()) effectiveExpo = ctrlValue.get(); - ctrlValue = sensorCtrls.get(V4L2_CID_ANALOGUE_GAIN); + ctrlValue = sensorControls.get(V4L2_CID_ANALOGUE_GAIN); if (!ctrlValue.isNone()) effectiveGain = ctrlValue.get(); @@ -472,6 +431,35 @@ void IPAIPU3::parseStatistics(unsigned int frame, metadataReady.emit(frame, ctrls); } +void IPAIPU3::runAiq([[maybe_unused]] unsigned int frame) +{ + /* Run algorithms into/using this context structure */ + resultsHistory_.extendOne(); + aiq::AiqResults& latestResults = resultsHistory_.latest(); + + /* Todo: Refactor AiqInputParameters interface to set following parameters. */ + aiqInputParams_.afParams.lens_position = lensPosition_; + aiqInputParams_.afParams.lens_movement_start_timestamp = lensMovementStartTime_; + + aiq_.run2a(frame, aiqInputParams_, latestResults); + + exposure_ = latestResults.ae()->exposures[0].sensor_exposure->coarse_integration_time; + gain_ = latestResults.ae()->exposures[0].sensor_exposure->analog_gain_code_global; + + /* + * Af algorithm compares the timestamp of start of the lens movement and + * that of the statistics generated to estimate whether next lens + * position should be produced. + * \todo use the lens movement start time reported by the pipeline handler. + */ + if (lensPosition_ != latestResults.af()->next_lens_position) { + utils::time_point time = utils::clock::now(); + uint64_t usecs = std::chrono::duration_cast(time.time_since_epoch()).count(); + lensMovementStartTime_ = usecs; + } + lensPosition_ = latestResults.af()->next_lens_position; +} + void IPAIPU3::setControls(unsigned int frame) { ControlList sensorCtrls(ctrls_);