From patchwork Fri May 6 09:53:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 15810 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 430E4C3256 for ; Fri, 6 May 2022 09:53:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0462C65643; Fri, 6 May 2022 11:53:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1651830809; bh=IFeCJ3yN3MrRoQpo1uDUprI+hmvYjx9F9SXDZ9enxK8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=BdrtPQ9EfktgGLgboJZEmx+YOSViVlKS7dc+4oHYLCIBALm6DOmbzY84V6AcxwYiF JUcvTfmtK0ZWsrs6KHNZNlNwPrxzDyeSYyjJBchfZ9MVsmOeyWVRM/eH/dHR6zgDHQ 3E4f0N1HTztuQYIadi5vpAfhnwFRcZXzlaP9npFeWaRmJQw3Y2535baKbZQ70oZQAc nMI+K6etYO9nPYSacG/USNEWTCSYcoHGnvTZ8ozjIhf/mIdZlBHR2UkOhWOLpoU9W4 9Rc4aVzYwH++BvCe74NqeK1VaQHzyT67b+CsFA/srPPPI629cfzdDHuAiPHPxPgJmU X3pRA4ER2+Y+Q== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1EE7C65643 for ; Fri, 6 May 2022 11:53:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="tgZaxrir"; dkim-atps=neutral Received: from perceval.ideasonboard.com (unknown [103.251.226.91]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E21A5487; Fri, 6 May 2022 11:53:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1651830807; bh=IFeCJ3yN3MrRoQpo1uDUprI+hmvYjx9F9SXDZ9enxK8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tgZaxrir2jwUnv9UCiKwKJc6oZG9Tt436i5gMFv4aK/MuIHakfx4PVN/1nmgdrroy oVlXZ2NzLsunc+pzhuK53F3xYOgnjaFcsM087rNhYIjJJdUrFxpvSe4qc+TYGsqQFB DI1Rx8V6KkRFEznPZICHtMy9bCxQFpESA0RZhp8c= To: libcamera-devel@lists.libcamera.org Date: Fri, 6 May 2022 15:23:07 +0530 Message-Id: <20220506095307.78370-4-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220506095307.78370-1-umang.jain@ideasonboard.com> References: <20220506095307.78370-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/3] ipa: ipu3: Introduce a IPAFrameContext queue X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Introduce a queue of IPAFrameContext which shall maintain the data required to track each frame. Currently, it is storing only the sensor controls values applied for the frame but new structures can be introduced in IPAFrameContext to extend any frame-related contexts (either in-processing or new frames being queued). For example, this patch provides the foundation for the extension of IPAFrameContext to store incoming request controls. Signed-off-by: Umang Jain --- src/ipa/ipu3/algorithms/agc.cpp | 2 +- src/ipa/ipu3/ipa_context.cpp | 16 ++++++++++++++-- src/ipa/ipu3/ipa_context.h | 8 +++++++- src/ipa/ipu3/ipu3.cpp | 31 ++++++++++++++++++++++++++----- 4 files changed, 48 insertions(+), 9 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index fdeec09d..4784af00 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -187,7 +187,7 @@ void Agc::computeExposure(IPAContext &context, double yGain, double iqMeanGain) { const IPASessionConfiguration &configuration = context.configuration; - IPAFrameContext &frameContext = context.frameContext; + IPAFrameContext &frameContext = context.frameContextQueue.front(); /* Get the effective exposure and gain applied on the sensor. */ uint32_t exposure = frameContext.sensor.exposure; double analogueGain = frameContext.sensor.gain; diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 06eb2776..fb48bc9b 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -58,8 +58,8 @@ namespace libcamera::ipa::ipu3 { * \var IPAContext::configuration * \brief The IPA session configuration, immutable during the session * - * \var IPAContext::frameContext - * \brief The frame context for the frame being processed + * \var IPAContext::frameContextQueue + * \brief FIFO container of IPAFrameContext for all the frames being queued * * \var IPAContext::activeState * \brief The current state of IPA algorithms @@ -182,6 +182,15 @@ namespace libcamera::ipa::ipu3 { * struct ipu3_uapi_gamma_corr_lut for further details. */ +/** + * \brief Construct a IPAFrameContext instance + */ +IPAFrameContext::IPAFrameContext(uint32_t frame) + : frame(frame) +{ + sensor = {}; +} + /** * \var IPAFrameContext::sensor * \brief Effective sensor values that were applied for the frame @@ -191,6 +200,9 @@ namespace libcamera::ipa::ipu3 { * * \var IPAFrameContext::sensor.gain * \brief Analogue gain multiplier + * + * \var IPAFrameContext::frame + * \brief The frame number associated with this IPAFrameContext */ } /* namespace libcamera::ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 8d681131..20cccc97 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -8,6 +8,8 @@ #pragma once +#include + #include #include @@ -71,17 +73,21 @@ struct IPAActiveState { }; struct IPAFrameContext { + IPAFrameContext(uint32_t frame); + struct { uint32_t exposure; double gain; } sensor; + + uint32_t frame; }; struct IPAContext { IPASessionConfiguration configuration; IPAActiveState activeState; - IPAFrameContext frameContext; + std::queue frameContextQueue; }; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 268c8f61..1b566f14 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -350,6 +350,8 @@ int IPAIPU3::start() */ void IPAIPU3::stop() { + while (!context_.frameContextQueue.empty()) + context_.frameContextQueue.pop(); } /** @@ -458,7 +460,7 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo, /* Clean IPAActiveState at each reconfiguration. */ context_.activeState = {}; - context_.frameContext = {}; + context_.frameContextQueue = {}; if (!validateSensorControls()) { LOG(IPAIPU3, Error) << "Sensor control validation failed."; @@ -510,6 +512,13 @@ void IPAIPU3::unmapBuffers(const std::vector &ids) void IPAIPU3::frameCompleted([[maybe_unused]] const uint32_t frame) { + while (!context_.frameContextQueue.empty()) { + auto &fc = context_.frameContextQueue.front(); + if (fc.frame <= frame) + context_.frameContextQueue.pop(); + else + break; + } } /** @@ -574,8 +583,18 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, const ipu3_uapi_stats_3a *stats = reinterpret_cast(mem.data()); - context_.frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); - context_.frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + auto &frameContext = context_.frameContextQueue.front(); + + /* + * An assert might be too harsh here. We want to know the cases + * where the front of the queue (implies the current frame in processing, + * diverges from the frame parameter of this function + * \todo Identify those cases - e.g. frame drop? + */ + ASSERT(context_.frameContextQueue.front().frame == frame); + + frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); + frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); double lineDuration = context_.configuration.sensor.lineDuration.get(); int32_t vBlank = context_.configuration.sensor.defVBlank; @@ -590,11 +609,11 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, int64_t frameDuration = (vBlank + sensorInfo_.outputSize.height) * lineDuration; ctrls.set(controls::FrameDuration, frameDuration); - ctrls.set(controls::AnalogueGain, context_.frameContext.sensor.gain); + ctrls.set(controls::AnalogueGain, frameContext.sensor.gain); ctrls.set(controls::ColourTemperature, context_.activeState.awb.temperatureK); - ctrls.set(controls::ExposureTime, context_.frameContext.sensor.exposure * lineDuration); + ctrls.set(controls::ExposureTime, frameContext.sensor.exposure * lineDuration); /* * \todo The Metadata provides a path to getting extended data @@ -621,6 +640,8 @@ void IPAIPU3::queueRequest([[maybe_unused]] const uint32_t frame, [[maybe_unused]] const ControlList &controls) { /* \todo Start processing for 'frame' based on 'controls'. */ + + context_.frameContextQueue.emplace(frame); } /**