From patchwork Thu Jun 30 13:38:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 16477 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4F663BE173 for ; Thu, 30 Jun 2022 13:39:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CA231656B1; Thu, 30 Jun 2022 15:39:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656596393; bh=oK0Vulz7dFwEAwnbt3WG2svQdV6Rb+H55QXELyLuJ7Q=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=iQOqFhY7M3ts1YoqBicY1CRsYb0uHbnxxqa7z1R16LGFo8wtwQLuHVpLHBx74Gwpo dci+IwjXdbBY6jgeYXpEg8kjDCqMwVSfGjEbOIYoBqLOJC5kkpYmAa/8yBDZPA5dLb 2MZF62/FVAF1KNc6YqvIpwqe0bPrQu0x/xR8kFnc1wKvWnW84kocgmpjUeVQL63xNV NI6RtkZOaEy93mKkO0ukwBc6GPoTMlhdN+NSoBTKbVK4TU0pslqibrtBxSxYlfqSv0 QYFpn8hkXZs15jHFZglI56AQZKNop1qEMWyIMNW1ManF6o9w8aOrsuV37ylQvwAHwn RfK1s78+RNLjg== Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DD3F9656AD for ; Thu, 30 Jun 2022 15:39:45 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by mail.gandi.net (Postfix) with ESMTPSA id E1F5B240005; Thu, 30 Jun 2022 13:39:43 +0000 (UTC) To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Jun 2022 15:38:58 +0200 Message-Id: <20220630133902.321099-20-jacopo@jmondi.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220630133902.321099-1-jacopo@jmondi.org> References: <20220630133902.321099-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 19/23] ipa: ipu3: Rework IPAIPU3::processStatsBuffer() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Before removing V4L2 controls from the IPAIPU3::processStatsBuffer() interface, rationalize the existing code a bit. 1) we have a context_.configuration which collects the per-capture session configuration data. At the same time the sensor and lens controls info are kept in class member variables. Move all per-capture session parameters to context_.configuration. 2) IPAIPU3::setControls() operates on the active state, which is implicitly retrieved from the global context. Pass it explicitly to make it clear. 3) Now that we have the full IPACameraSensorInfo in the current context remove vblank from IPASessionConfiguration::sensor as it can be retrieved from IPASessionConfiguration::sensor.info. 4) Rationalize the code of IPAIPU3::processStatsBuffer() to organize it in a logical flow of operations: - Update the per-frame context with the sensor's controls values - Map the stats buffer - Run algorithms with the per-frame context and stats - setControls() using the active state as computed by algorithms - Populate the metadata buffer Signed-off-by: Jacopo Mondi --- src/ipa/ipu3/ipa_context.cpp | 14 ++++++-- src/ipa/ipu3/ipa_context.h | 9 ++++- src/ipa/ipu3/ipu3.cpp | 68 +++++++++++++++++++----------------- 3 files changed, 55 insertions(+), 36 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 06fdf2a1efc7..f09d061e2d19 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -126,8 +126,18 @@ namespace libcamera::ipa::ipu3 { * \var IPASessionConfiguration::sensor.lineDuration * \brief Line duration in microseconds * - * \var IPASessionConfiguration::sensor.vBlank - * \brief The vertical blanking expressed in number of lines + * \var IPASessionConfiguration::sensor.info + * \brief The IPACameraSensorInfo valid for the session + * + * \var IPASessionConfiguration::sensor.controls + * \brief The ControlInfoMap of camera sensor control limits valid for the + * session + * + * \var IPASessionConfiguration::lens + * \brief Lens-specific configuration of the IPA + * + * \var IPASessionConfiguration::lens.controls + * \brief The ControlInfoMap of lens control limits valid for the session */ /** diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index a5b878ab7792..35baa17e8708 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -17,6 +17,8 @@ #include #include +#include + namespace libcamera { namespace ipa::ipu3 { @@ -43,9 +45,14 @@ struct IPASessionConfiguration { } agc; struct { - int32_t vBlank; utils::Duration lineDuration; + ControlInfoMap controls; + IPACameraSensorInfo info; } sensor; + + struct { + ControlInfoMap controls; + } lens; }; struct IPAActiveState { diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index eb97c8be5431..f2e28c06a2c2 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -152,16 +152,11 @@ private: bool validateSensorControls(const ControlInfoMap &sensorControls); - void setControls(unsigned int frame); + void setControls(unsigned int frame, const IPAActiveState &state); void calculateBdsGrid(const Size &bdsOutputSize); std::map buffers_; - ControlInfoMap sensorCtrls_; - ControlInfoMap lensCtrls_; - - IPACameraSensorInfo sensorInfo_; - /* Interface to the Camera Helper */ std::unique_ptr camHelper_; @@ -179,7 +174,6 @@ private: void IPAIPU3::updateSessionConfiguration(const IPAConfigInfo &info) { const IPACameraSensorInfo &sensorInfo = info.sensorInfo; - context_.configuration.sensor.vBlank = sensorInfo.vblank; context_.configuration.sensor.lineDuration = sensorInfo.lineLength * 1.0s / sensorInfo.pixelRate; @@ -199,6 +193,11 @@ void IPAIPU3::updateSessionConfiguration(const IPAConfigInfo &info) const ControlInfo &gain = sensorControls.at(&controls::internal::AnalogueGain); context_.configuration.agc.minAnalogueGain = gain.min().get(); context_.configuration.agc.maxAnalogueGain = gain.max().get(); + + /* Store the sensor and lens configuration in the current session. */ + context_.configuration.sensor.info = info.sensorInfo; + context_.configuration.sensor.controls = info.sensorControls; + context_.configuration.lens.controls = info.lensControls; } /** @@ -262,7 +261,7 @@ int IPAIPU3::start() * Set the sensors V4L2 controls before the first frame to ensure that * we have an expected and known configuration from the start. */ - setControls(0); + setControls(0, context_.activeState); return 0; } @@ -379,10 +378,6 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) return ret; } - sensorInfo_ = configInfo.sensorInfo; - lensCtrls_ = configInfo.lensControls; - sensorCtrls_ = configInfo.sensorControls; - return 0; } @@ -472,10 +467,10 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, return; } - Span mem = it->second.planes()[0]; - const ipu3_uapi_stats_3a *stats = - reinterpret_cast(mem.data()); - + /* + * Update the per-frame context storing the sensor exposure and + * gain for later use by algorithms. + */ IPAFrameContext &frameContext = context_.frameContexts[frame % kMaxFrameContexts]; if (frameContext.frame != frame) @@ -484,23 +479,28 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); - double lineDuration = context_.configuration.sensor.lineDuration.get(); - int32_t vBlank = context_.configuration.sensor.vBlank; - ControlList ctrls(controls::controls); + /* Run algorithms on the statistics and per-frame context. */ + Span mem = it->second.planes()[0]; + const ipu3_uapi_stats_3a *stats = reinterpret_cast(mem.data()); for (auto const &algo : algorithms_) algo->process(context_, &frameContext, stats); - setControls(frame); - - int64_t frameDuration = (vBlank + sensorInfo_.outputSize.height) * lineDuration; - ctrls.set(controls::FrameDuration, frameDuration); + /* Set sensor controls using the newly computed values. */ + setControls(frame, context_.activeState); - ctrls.set(controls::AnalogueGain, frameContext.sensor.gain); + /* Prepare metadata for the frame. */ + const IPASessionConfiguration &ipaConfig = context_.configuration; + const IPACameraSensorInfo &sensorInfo = ipaConfig.sensor.info; + double lineDuration = ipaConfig.sensor.lineDuration.get(); + int64_t frameDuration = (sensorInfo.vblank + sensorInfo.outputSize.height) * lineDuration; - ctrls.set(controls::ColourTemperature, context_.activeState.awb.temperatureK); + ControlList metadata(controls::controls); - ctrls.set(controls::ExposureTime, frameContext.sensor.exposure * lineDuration); + metadata.set(controls::FrameDuration, frameDuration); + metadata.set(controls::AnalogueGain, frameContext.sensor.gain); + metadata.set(controls::ColourTemperature, context_.activeState.awb.temperatureK); + metadata.set(controls::ExposureTime, frameContext.sensor.exposure * lineDuration); /* * \todo The Metadata provides a path to getting extended data @@ -510,7 +510,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, * likely want to avoid putting platform specific metadata in. */ - metadataReady.emit(frame, ctrls); + metadataReady.emit(frame, metadata); } /** @@ -530,22 +530,24 @@ void IPAIPU3::queueRequest(const uint32_t frame, const ControlList &controls) /** * \brief Handle sensor controls for a given \a frame number * \param[in] frame The frame on which the sensor controls should be set + * \param[in] state The IPA active state which contains the control values as + * computed by the algorithms * * Send the desired sensor control values to the pipeline handler to request * that they are applied on the camera sensor. */ -void IPAIPU3::setControls(unsigned int frame) +void IPAIPU3::setControls(unsigned int frame, const IPAActiveState &state) { - int32_t exposure = context_.activeState.agc.exposure; - int32_t gain = camHelper_->gainCode(context_.activeState.agc.gain); + int32_t exposure = state.agc.exposure; + int32_t gain = camHelper_->gainCode(state.agc.gain); - ControlList ctrls(sensorCtrls_); + ControlList ctrls(controls::controls); ctrls.set(V4L2_CID_EXPOSURE, exposure); ctrls.set(V4L2_CID_ANALOGUE_GAIN, gain); - ControlList lensCtrls(lensCtrls_); + ControlList lensCtrls(controls::controls); lensCtrls.set(V4L2_CID_FOCUS_ABSOLUTE, - static_cast(context_.activeState.af.focus)); + static_cast(state.af.focus)); setSensorControls.emit(frame, ctrls, lensCtrls); }