From patchwork Tue May 17 19:18:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 15941 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 26B46C0F2A for ; Tue, 17 May 2022 19:18:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AF7A965660; Tue, 17 May 2022 21:18:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1652815125; bh=27si9dP945uWLD3Hwhd2DzgcUm0eVf4husfdps57WQQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=iYk+qCOwET3T9LTSjTB2gkC0gw6SkzGt6viEN8k/Hp4r1dE8WtDlB5Gkmc8Jv7AD/ dr2pCOfDr5tftXgxhJybDnLn7yCLcp6wcEbJP+T2MjALKfzZokTsBTAExRtlyz55Zs l64X45hEYTJ59Cs8xcsL5DtFP6gtmmfTO41nTUMGL4OFXcIceY7yNSf+VHNrLLFk67 zelwy9ZnjMf0dWl5kHH5YdHpTeRftwBZTC6anGndv0Irilmw7ALAjwNr2RTi+8i5T3 lXFRUJUhNwlkQ5wWaIcdqDWJ0r+Xn/vlP8h0yBww6Cwaax8Nus7VRFyCyoV0sHx5Yf oZBkwiHWVvTUA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 850256041E for ; Tue, 17 May 2022 21:18:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="NQeMyG50"; dkim-atps=neutral Received: from perceval.ideasonboard.com (unknown [45.131.31.124]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 552ED8E6; Tue, 17 May 2022 21:18:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1652815123; bh=27si9dP945uWLD3Hwhd2DzgcUm0eVf4husfdps57WQQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NQeMyG50aXpxCJfNN5Tk8OuzxVBIGXwg9CAWnQckUeeHo7Xlikd1WeevSiOsEOqnC BxOmZjXoMvnRg2ZMi2S7KaVyBevMTwOnaH2nTtL/JvCAMGgQEQryQ1i/RWevw/iBXr /c07xScheAyYhZ3aS9IFSq2DXbSP3qKr4tujC8BA= To: libcamera-devel@lists.libcamera.org Date: Tue, 17 May 2022 21:18:32 +0200 Message-Id: <20220517191833.333122-3-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220517191833.333122-1-umang.jain@ideasonboard.com> References: <20220517191833.333122-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/3] ipa: libipa: Add frame context pointer in process() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Currently we have a single structure of IPAFrameContext but subsequently, we shall have a ring buffer (or similar) container to keep IPAFrameContext structures for each frame. It would be a hassle to query out the frame context required for process() (since they will reside in a ring buffer) by the IPA for each process. Hence, prepare the process() libipa template to accept a particular IPAFrameContext early on. As for this patch, we shall pass in the pointer as nullptr, so that the changes compile and keep working as-is. Signed-off-by: Umang Jain Reviewed-by: Jacopo Mondi Reviewed-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/af.cpp | 4 +++- src/ipa/ipu3/algorithms/af.h | 3 ++- src/ipa/ipu3/algorithms/agc.cpp | 4 +++- src/ipa/ipu3/algorithms/agc.h | 3 ++- src/ipa/ipu3/algorithms/algorithm.h | 4 +++- src/ipa/ipu3/algorithms/awb.cpp | 3 ++- src/ipa/ipu3/algorithms/awb.h | 3 ++- src/ipa/ipu3/algorithms/tone_mapping.cpp | 3 ++- src/ipa/ipu3/algorithms/tone_mapping.h | 3 ++- src/ipa/ipu3/ipu3.cpp | 2 +- src/ipa/libipa/algorithm.cpp | 1 + src/ipa/libipa/algorithm.h | 4 +++- src/ipa/rkisp1/algorithms/agc.cpp | 4 +++- src/ipa/rkisp1/algorithms/agc.h | 3 ++- src/ipa/rkisp1/algorithms/algorithm.h | 4 +++- src/ipa/rkisp1/algorithms/awb.cpp | 4 +++- src/ipa/rkisp1/algorithms/awb.h | 3 ++- src/ipa/rkisp1/rkisp1.cpp | 2 +- 18 files changed, 40 insertions(+), 17 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index 8a5a6b1a..d07521a0 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -406,6 +406,7 @@ bool Af::afIsOutOfFocus(IPAContext context) /** * \brief Determine the max contrast image and lens position. * \param[in] context The IPA context. + * \param[in] frameContext The current frame context * \param[in] stats The statistics buffer of IPU3. * * Ideally, a clear image also has a relatively higher contrast. So, every @@ -419,7 +420,8 @@ bool Af::afIsOutOfFocus(IPAContext context) * * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing */ -void Af::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) +void Af::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, + const ipu3_uapi_stats_3a *stats) { /* Evaluate the AF buffer length */ uint32_t afRawBufferLen = context.configuration.af.afGrid.width * diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index b85cf941..ccf015f3 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -32,7 +32,8 @@ public: void prepare(IPAContext &context, ipu3_uapi_params *params) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; + void process(IPAContext &context, IPAFrameContext *frameContext, + const ipu3_uapi_stats_3a *stats) override; private: void afCoarseScan(IPAContext &context); diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index fdeec09d..383a8deb 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -318,12 +318,14 @@ double Agc::estimateLuminance(IPAActiveState &activeState, /** * \brief Process IPU3 statistics, and run AGC operations * \param[in] context The shared IPA context + * \param[in] frameContext The current frame context * \param[in] stats The IPU3 statistics and ISP results * * Identify the current image brightness, and use that to estimate the optimal * new exposure and gain for the scene. */ -void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) +void Agc::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, + const ipu3_uapi_stats_3a *stats) { /* * Estimate the gain needed to have the proportion of pixels in a given diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 31420841..219a1a96 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -28,7 +28,8 @@ public: ~Agc() = default; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; + void process(IPAContext &context, IPAFrameContext *frameContext, + const ipu3_uapi_stats_3a *stats) override; private: double measureBrightness(const ipu3_uapi_stats_3a *stats, diff --git a/src/ipa/ipu3/algorithms/algorithm.h b/src/ipa/ipu3/algorithms/algorithm.h index d2eecc78..234b2bd7 100644 --- a/src/ipa/ipu3/algorithms/algorithm.h +++ b/src/ipa/ipu3/algorithms/algorithm.h @@ -17,7 +17,9 @@ namespace libcamera { namespace ipa::ipu3 { -using Algorithm = libcamera::ipa::Algorithm; +using Algorithm = libcamera::ipa::Algorithm; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index ab6924eb..5c232d92 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -387,7 +387,8 @@ void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) /** * \copydoc libcamera::ipa::Algorithm::process */ -void Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) +void Awb::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, + const ipu3_uapi_stats_3a *stats) { calculateWBGains(stats); diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index ab4b0a33..9a50a985 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -40,7 +40,8 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, ipu3_uapi_params *params) override; - void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; + void process(IPAContext &context, IPAFrameContext *frameContext, + const ipu3_uapi_stats_3a *stats) override; private: /* \todo Make these structs available to all the ISPs ? */ diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 7c78d0d9..f86e79b2 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -72,12 +72,13 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, /** * \brief Calculate the tone mapping look up table * \param context The shared IPA context + * \param frameContext The current frame context * \param stats The IPU3 statistics and ISP results * * The tone mapping look up table is generated as an inverse power curve from * our gamma setting. */ -void ToneMapping::process(IPAContext &context, +void ToneMapping::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) { /* diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index b727ab1e..d7d48006 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -20,7 +20,8 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, ipu3_uapi_params *params) override; - void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; + void process(IPAContext &context, IPAFrameContext *frameContext, + const ipu3_uapi_stats_3a *stats) override; private: double gamma_; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 3b4fc911..16e5028f 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -576,7 +576,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, ControlList ctrls(controls::controls); for (auto const &algo : algorithms_) - algo->process(context_, stats); + algo->process(context_, nullptr, stats); setControls(frame); diff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/libipa/algorithm.cpp index 398d5372..cce2ed62 100644 --- a/src/ipa/libipa/algorithm.cpp +++ b/src/ipa/libipa/algorithm.cpp @@ -64,6 +64,7 @@ namespace ipa { * \fn Algorithm::process() * \brief Process ISP statistics, and run algorithm operations * \param[in] context The shared IPA context + * \param[in] frameContext The current frame's context * \param[in] stats The IPA statistics and ISP results * * This function is called while camera is running for every frame processed by diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h index 766aee5d..032a05b5 100644 --- a/src/ipa/libipa/algorithm.h +++ b/src/ipa/libipa/algorithm.h @@ -10,7 +10,8 @@ namespace libcamera { namespace ipa { -template +template class Algorithm { public: @@ -28,6 +29,7 @@ public: } virtual void process([[maybe_unused]] Context &context, + [[maybe_unused]] FrameContext *frameContext, [[maybe_unused]] const Stats *stats) { } diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 5f4c3f93..b5a184d9 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -280,7 +280,9 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const * Identify the current image brightness, and use that to estimate the optimal * new exposure and gain for the scene. */ -void Agc::process(IPAContext &context, const rkisp1_stat_buffer *stats) +void Agc::process(IPAContext &context, + [[maybe_unused]] IPAFrameContext *frameContext, + const rkisp1_stat_buffer *stats) { const rkisp1_cif_isp_stat *params = &stats->params; ASSERT(stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP); diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index ce1adf27..22c02779 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -29,7 +29,8 @@ public: int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; void prepare(IPAContext &context, rkisp1_params_cfg *params) override; - void process(IPAContext &context, const rkisp1_stat_buffer *stats) override; + void process(IPAContext &context, IPAFrameContext *frameContext, + const rkisp1_stat_buffer *stats) override; private: void computeExposure(IPAContext &Context, double yGain, double iqMeanGain); diff --git a/src/ipa/rkisp1/algorithms/algorithm.h b/src/ipa/rkisp1/algorithms/algorithm.h index d46c3188..68e3a44e 100644 --- a/src/ipa/rkisp1/algorithms/algorithm.h +++ b/src/ipa/rkisp1/algorithms/algorithm.h @@ -19,7 +19,9 @@ namespace libcamera { namespace ipa::rkisp1 { -using Algorithm = libcamera::ipa::Algorithm; +using Algorithm = libcamera::ipa::Algorithm; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index be4585c6..88441382 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -119,7 +119,9 @@ void Awb::prepare(IPAContext &context, rkisp1_params_cfg *params) /** * \copydoc libcamera::ipa::Algorithm::process */ -void Awb::process([[maybe_unused]] IPAContext &context, const rkisp1_stat_buffer *stats) +void Awb::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] IPAFrameContext *frameCtx, + const rkisp1_stat_buffer *stats) { const rkisp1_cif_isp_stat *params = &stats->params; const rkisp1_cif_isp_awb_stat *awb = ¶ms->awb; diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 11946643..7647842f 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -23,7 +23,8 @@ public: int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; void prepare(IPAContext &context, rkisp1_params_cfg *params) override; - void process(IPAContext &context, const rkisp1_stat_buffer *stats) override; + void process(IPAContext &context, IPAFrameContext *frameCtx, + const rkisp1_stat_buffer *stats) override; private: uint32_t estimateCCT(double red, double green, double blue); diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index ef1f0d56..c818a6d7 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -272,7 +272,7 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId unsigned int aeState = 0; for (auto const &algo : algorithms_) - algo->process(context_, stats); + algo->process(context_, nullptr, stats); setControls(frame);