From patchwork Mon Jun 27 16:27:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 16396 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9B996BD808 for ; Mon, 27 Jun 2022 16:28:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0C23D65649; Mon, 27 Jun 2022 18:28:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656347286; bh=P9aa6O5zAz3cqNS1dxpJn9f54hgJ61tTUiMnkK+R99U=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=X9/OdMiA0wxKeZAWPs+JYM+ulTDATiBmqLtXeRPpInPMwB7QrPFN3Y3HZ4O3UZH82 YJ1xv10uS9wakIWiyqF2lFHM1Fm5TxhOmB4nLv7dB0ZsRS7kbzIAjTaTUUGhEt55OZ la00oRSmzkzPibBbNP0OzQ4WNPp2CzmoXCCc6nf+rHYB7KEzfRHcKURPhnqfqMWPpu AyQ+Sbl83ryJJVbsVsFjmVcvnnM5z232ACrt0aK1pTNZGqIjTCAUYEVcQF093d/DI3 +pFWkNwkzTKoE9byW0Ys2i0HYffBy3gB7vjOWOqwmxUXoNAse/8nQiQJYYI4ZlU79A dETL5pKLbK8kw== Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [217.70.183.200]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B4D0965650 for ; Mon, 27 Jun 2022 18:27:55 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by mail.gandi.net (Postfix) with ESMTPSA id 0B57C20008; Mon, 27 Jun 2022 16:27:54 +0000 (UTC) To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Jun 2022 18:27:32 +0200 Message-Id: <20220627162732.33160-16-jacopo@jmondi.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220627162732.33160-1-jacopo@jmondi.org> References: <20220627162732.33160-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 15/15] ipa: ipu3: Rework IPAIPU3::processStatsBuffer() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Before removing V4L2 controls from the IPAIPU3::processStatsBuffer() interface, rationalize the existing code a bit. 1) we have a context_.configuration which collects the per-capture session configuration data. At the same time the sensor and lens controls info are kept in class member variables. Move all per-capture session parameters to context_.configuration 2) IPAIPU3::setControls() operates on the active state, which is implicitly retrieved from the global context. Pass it explicitely to make it clear. 3) Now that we have the full IPACameraSensorInfo in the current context remove vblank from IPASessionConfiguration::sensor as it can be retrieved from IPASessionConfiguration::sensor.info. 4) Rationalize the code of IPAIPU3::processStatsBuffer() to organize it in a logical flow of operations: - Update the per-frame context with the sensor's controls values - Map the stats buffer - Run algorithms with the the per-frame context and stats - setControls() using the active state as computed by algorithms - Populate the metadata buffer Signed-off-by: Jacopo Mondi --- src/ipa/ipu3/ipa_context.cpp | 14 ++++++- src/ipa/ipu3/ipa_context.h | 9 ++++- src/ipa/ipu3/ipu3.cpp | 73 +++++++++++++++++++----------------- 3 files changed, 59 insertions(+), 37 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 06fdf2a1efc7..f09d061e2d19 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -126,8 +126,18 @@ namespace libcamera::ipa::ipu3 { * \var IPASessionConfiguration::sensor.lineDuration * \brief Line duration in microseconds * - * \var IPASessionConfiguration::sensor.vBlank - * \brief The vertical blanking expressed in number of lines + * \var IPASessionConfiguration::sensor.info + * \brief The IPACameraSensorInfo valid for the session + * + * \var IPASessionConfiguration::sensor.controls + * \brief The ControlInfoMap of camera sensor control limits valid for the + * session + * + * \var IPASessionConfiguration::lens + * \brief Lens-specific configuration of the IPA + * + * \var IPASessionConfiguration::lens.controls + * \brief The ControlInfoMap of lens control limits valid for the session */ /** diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index a5b878ab7792..35baa17e8708 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -17,6 +17,8 @@ #include #include +#include + namespace libcamera { namespace ipa::ipu3 { @@ -43,9 +45,14 @@ struct IPASessionConfiguration { } agc; struct { - int32_t vBlank; utils::Duration lineDuration; + ControlInfoMap controls; + IPACameraSensorInfo info; } sensor; + + struct { + ControlInfoMap controls; + } lens; }; struct IPAActiveState { diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index b67ff1948bc2..791c03e5a2d3 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -152,16 +152,11 @@ private: bool validateConfiguration(const IPAConfigInfo &config); - void setControls(unsigned int frame); + void setControls(unsigned int frame, const IPAActiveState &state); void calculateBdsGrid(const Size &bdsOutputSize); std::map buffers_; - ControlInfoMap sensorCtrls_; - ControlInfoMap lensCtrls_; - - IPACameraSensorInfo sensorInfo_; - /* Interface to the Camera Helper */ std::unique_ptr camHelper_; @@ -179,7 +174,6 @@ private: void IPAIPU3::updateSessionConfiguration(const IPAConfigInfo &info) { const IPACameraSensorInfo &sensorInfo = info.sensorInfo; - context_.configuration.sensor.vBlank = sensorInfo.vblank; context_.configuration.sensor.lineDuration = sensorInfo.lineLength * 1.0s / sensorInfo.pixelRate; @@ -202,6 +196,11 @@ void IPAIPU3::updateSessionConfiguration(const IPAConfigInfo &info) context_.configuration.agc.minAnalogueGain = camHelper_->gain(minGain); context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain); + + /* Store the sensor and lens configuration in the current session. */ + context_.configuration.sensor.info = info.sensorInfo; + context_.configuration.sensor.controls = info.sensorControls; + context_.configuration.lens.controls = info.lensControls; } /** @@ -265,7 +264,7 @@ int IPAIPU3::start() * Set the sensors V4L2 controls before the first frame to ensure that * we have an expected and known configuration from the start. */ - setControls(0); + setControls(0, context_.activeState); return 0; } @@ -387,9 +386,9 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) return ret; } - sensorInfo_ = configInfo.sensorInfo; - lensCtrls_ = configInfo.lensControls; - sensorCtrls_ = configInfo.sensorControls; + context_.configuration.sensor.controls = configInfo.sensorControls; + context_.configuration.sensor.info = configInfo.sensorInfo; + context_.configuration.lens.controls = configInfo.lensControls; return 0; } @@ -480,35 +479,39 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, return; } - Span mem = it->second.planes()[0]; - const ipu3_uapi_stats_3a *stats = - reinterpret_cast(mem.data()); - + /* + * Update the per-frame context storing the sensor exposure and + * gain for later use by algorithms. + */ IPAFrameContext &frameContext = context_.frameContexts[frame % kMaxFrameContexts]; - if (frameContext.frame != frame) LOG(IPAIPU3, Warning) << "Frame " << frame << " does not match its frame context"; frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); - double lineDuration = context_.configuration.sensor.lineDuration.get(); - int32_t vBlank = context_.configuration.sensor.vBlank; - ControlList ctrls(controls::controls); - + /* Run algorithms on the statistics and per-frame context. */ + Span mem = it->second.planes()[0]; + const ipu3_uapi_stats_3a *stats = + reinterpret_cast(mem.data()); for (auto const &algo : algorithms_) algo->process(context_, &frameContext, stats); - setControls(frame); + /* Set sensor controls using the newly computed values. */ + setControls(frame, context_.activeState); - int64_t frameDuration = (vBlank + sensorInfo_.outputSize.height) * lineDuration; - ctrls.set(controls::FrameDuration, frameDuration); + /* Prepare metadata for the frame. */ + const IPASessionConfiguration &ipaConfig = context_.configuration; + const IPACameraSensorInfo &sensorInfo = ipaConfig.sensor.info; + double lineDuration = ipaConfig.sensor.lineDuration.get(); + int64_t frameDuration = (sensorInfo.vblank + sensorInfo.outputSize.height) * lineDuration; - ctrls.set(controls::AnalogueGain, frameContext.sensor.gain); + ControlList metadata(controls::controls); - ctrls.set(controls::ColourTemperature, context_.activeState.awb.temperatureK); - - ctrls.set(controls::ExposureTime, frameContext.sensor.exposure * lineDuration); + metadata.set(controls::FrameDuration, frameDuration); + metadata.set(controls::AnalogueGain, frameContext.sensor.gain); + metadata.set(controls::ColourTemperature, context_.activeState.awb.temperatureK); + metadata.set(controls::ExposureTime, frameContext.sensor.exposure * lineDuration); /* * \todo The Metadata provides a path to getting extended data @@ -518,7 +521,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, * likely want to avoid putting platform specific metadata in. */ - metadataReady.emit(frame, ctrls); + metadataReady.emit(frame, metadata); } /** @@ -538,22 +541,24 @@ void IPAIPU3::queueRequest(const uint32_t frame, const ControlList &controls) /** * \brief Handle sensor controls for a given \a frame number * \param[in] frame The frame on which the sensor controls should be set + * \param[in] state The IPA active state which contains the control values as + * computed by the algorithms * * Send the desired sensor control values to the pipeline handler to request * that they are applied on the camera sensor. */ -void IPAIPU3::setControls(unsigned int frame) +void IPAIPU3::setControls(unsigned int frame, const IPAActiveState &state) { - int32_t exposure = context_.activeState.agc.exposure; - int32_t gain = camHelper_->gainCode(context_.activeState.agc.gain); + int32_t exposure = state.agc.exposure; + int32_t gain = camHelper_->gainCode(state.agc.gain); - ControlList ctrls(sensorCtrls_); + ControlList ctrls(controls::controls); ctrls.set(V4L2_CID_EXPOSURE, exposure); ctrls.set(V4L2_CID_ANALOGUE_GAIN, gain); - ControlList lensCtrls(lensCtrls_); + ControlList lensCtrls(controls::controls); lensCtrls.set(V4L2_CID_FOCUS_ABSOLUTE, - static_cast(context_.activeState.af.focus)); + static_cast(state.af.focus)); setSensorControls.emit(frame, ctrls, lensCtrls); }