From patchwork Tue Nov 23 15:04:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14714 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AB258BF415 for ; Tue, 23 Nov 2021 15:04:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0D6A46042A; Tue, 23 Nov 2021 16:04:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FXX+6afl"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 01C0860228 for ; Tue, 23 Nov 2021 16:04:28 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8D4DFF95; Tue, 23 Nov 2021 16:04:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637679867; bh=6qkF2Tg7yCai9Xl8kh9FitL+Os2hwaUjtO8oJRvJdwQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FXX+6aflMGpeT0cgX+Pv1xkrno0zEG7dULkdIDjdG3efuY3S6govV51uuW7XwWfzb 9I141oHS8BzejMJ89n3OEf+Lxv2vjrfgQpLXiRkXuODF6/qdH7yvURjwIVkA5TV59X OlN6oTJb2slTmCtNzSlZj7srS1Ne1IYu5iIJkIcg= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 16:04:13 +0100 Message-Id: <20211123150423.125524-2-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 01/11] libipa: Correct IMX219 in CameraSensorHelper X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The equation is badly reported in the CameraSensorHelper, as m1 and c0 are inverted. Correct it to have a proper gain calculation. Signed-off-by: Jean-Michel Hautbois Acked-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/libipa/camera_sensor_helper.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/libipa/camera_sensor_helper.cpp b/src/ipa/libipa/camera_sensor_helper.cpp index 0b0eb503..5d052612 100644 --- a/src/ipa/libipa/camera_sensor_helper.cpp +++ b/src/ipa/libipa/camera_sensor_helper.cpp @@ -290,7 +290,7 @@ class CameraSensorHelperImx219 : public CameraSensorHelper public: CameraSensorHelperImx219() { - analogueGainConstants_ = { AnalogueGainLinear, 0, -1, 256, 256 }; + analogueGainConstants_ = { AnalogueGainLinear, 0, 256, -1, 256 }; } }; REGISTER_CAMERA_SENSOR_HELPER("imx219", CameraSensorHelperImx219) From patchwork Tue Nov 23 15:04:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14715 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 801C4BF415 for ; Tue, 23 Nov 2021 15:04:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 390CE603EE; Tue, 23 Nov 2021 16:04:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GEF2daVi"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 155FE6022F for ; Tue, 23 Nov 2021 16:04:28 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BFD8A11BB; Tue, 23 Nov 2021 16:04:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637679867; bh=x12yORGu01s1Kvrnd7axewwidSalFhXjHeeTGktmx7c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GEF2daVivEdFalCWbcs9WH/+W6I0yiBO4MeD6MyRfLnKi1Coqhvs/2kbSpzT+Oc7Z js/dfVsux+Gblu9geS8VraUKZ8fdHVMt08v8XvXeHPB9SXp6VIYfzqM3go2aipmL0A 69F9UFj3x1oJpRNWTy8yqAT6iwYKSMY1xYlcBrZc= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 16:04:14 +0100 Message-Id: <20211123150423.125524-3-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 02/11] ipa: rkisp1: Pass IPASettings at init call X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the IPA is initialized, it will need to know the sensor model used in order to properly call CameraSensorHelper for the analogue gain. Modify the init definition in the pipeline handler and in the IPA as well as the mojo interface to pass the IPASettings. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- include/libcamera/ipa/rkisp1.mojom | 4 +++- src/ipa/rkisp1/rkisp1.cpp | 5 +++-- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- 3 files changed, 7 insertions(+), 4 deletions(-) diff --git a/include/libcamera/ipa/rkisp1.mojom b/include/libcamera/ipa/rkisp1.mojom index cae757ea..a6991d4f 100644 --- a/include/libcamera/ipa/rkisp1.mojom +++ b/include/libcamera/ipa/rkisp1.mojom @@ -29,7 +29,9 @@ struct RkISP1Action { }; interface IPARkISP1Interface { - init(uint32 hwRevision) => (int32 ret); + init(libcamera.IPASettings settings, + uint32 hwRevision) + => (int32 ret); start() => (int32 ret); stop(); diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index bf2c13b6..7ecbf8ae 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -34,7 +34,7 @@ namespace ipa::rkisp1 { class IPARkISP1 : public IPARkISP1Interface { public: - int init(unsigned int hwRevision) override; + int init(const IPASettings &settings, unsigned int hwRevision) override; int start() override; void stop() override {} @@ -75,7 +75,8 @@ private: unsigned int hwHistogramWeightGridsSize_; }; -int IPARkISP1::init(unsigned int hwRevision) +int IPARkISP1::init([[maybe_unused]] const IPASettings &settings, + unsigned int hwRevision) { /* \todo Add support for other revisions */ switch (hwRevision) { diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 98008862..6cdc4c03 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -319,7 +319,7 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision) ipa_->queueFrameAction.connect(this, &RkISP1CameraData::queueFrameAction); - int ret = ipa_->init(hwRevision); + int ret = ipa_->init(IPASettings{ "", sensor_->model() }, hwRevision); if (ret < 0) { LOG(RkISP1, Error) << "IPA initialization failure"; return ret; From patchwork Tue Nov 23 15:04:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14716 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 20008BF415 for ; Tue, 23 Nov 2021 15:04:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3A1BA6022F; Tue, 23 Nov 2021 16:04:35 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Dmn+Vkr4"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 42AAF60230 for ; Tue, 23 Nov 2021 16:04:28 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EE2D2993; Tue, 23 Nov 2021 16:04:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637679868; bh=RjPFSEqcQzvjL+C53oA43iFIzoqXgn3nfL7xrF1B5w0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Dmn+Vkr4gNmGPMh6cKquBMOeuSvm18SPmcn9ZXbpRB2QWuc3aZUR7iSCaV4Un+S0S XCvQWA6YE1ZdtPlFoskB36TkmyyG6pktt6qnBFkinh2pxfaD41M3C98sGzhwAGX/K5 Vib0gTBvSgrZroGpAekizt+mP91l/Q7OFZ6Nf7yU= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 16:04:15 +0100 Message-Id: <20211123150423.125524-4-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 03/11] ipa: Do not modify the sensor limits X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The driver is responsible of setting the proper limits for its controls. For instance, imx219 has an analogue gain of 1.0 when the gain code is set to 0. The minimum analogue is forced to be at least 1, which for imx219 sets it to 1.00329. Rework this for both IPU3 and RkISP1. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipu3.cpp | 4 ++-- src/ipa/rkisp1/rkisp1.cpp | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index a8d54a5d..b0c75541 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -444,11 +444,11 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo, return -EINVAL; } - minExposure_ = std::max(itExp->second.min().get(), 1); + minExposure_ = itExp->second.min().get(); maxExposure_ = itExp->second.max().get(); exposure_ = minExposure_; - minGain_ = std::max(itGain->second.min().get(), 1); + minGain_ = itGain->second.min().get(); maxGain_ = itGain->second.max().get(); gain_ = minGain_; diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 7ecbf8ae..910ad952 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -139,11 +139,11 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, autoExposure_ = true; - minExposure_ = std::max(itExp->second.min().get(), 1); + minExposure_ = itExp->second.min().get(); maxExposure_ = itExp->second.max().get(); exposure_ = minExposure_; - minGain_ = std::max(itGain->second.min().get(), 1); + minGain_ = itGain->second.min().get(); maxGain_ = itGain->second.max().get(); gain_ = minGain_; From patchwork Tue Nov 23 15:04:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14717 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 166CAC324F for ; Tue, 23 Nov 2021 15:04:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 58561603F9; Tue, 23 Nov 2021 16:04:36 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dsy/92pC"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 776486033C for ; Tue, 23 Nov 2021 16:04:28 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2A3C9F95; Tue, 23 Nov 2021 16:04:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637679868; bh=7EKJLxN6FyygVP3ODjD71RglMl4Afhijn0ZyokJr6KI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dsy/92pCFhYyRwvViT69+2MHndT/iIFm4WZKgtTbdhqWrgSXxBdqKSzpbHmVf4n29 wCu13v1bGBBTnlqmKgePfdwP7vBK8GMPa1v7UstS3dr0Qu6TWjHw0dB5j0u+NsRv+k 27MgQ6DFZnVGfWbyLgt8CuAEA0Hp6yp2SzbFlWOM= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 16:04:16 +0100 Message-Id: <20211123150423.125524-5-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 04/11] ipa: rkisp1: Instantiate CameraSensorHelper X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the AGC will run, it will use analogue gains as double values. We will need those values to be converted to apply the control. Introduce CameraSensorHelper and call it at init(). Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/rkisp1.cpp | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 910ad952..73a7f582 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -25,6 +25,8 @@ #include +#include "libipa/camera_sensor_helper.h" + namespace libcamera { LOG_DEFINE_CATEGORY(IPARkISP1) @@ -73,10 +75,12 @@ private: unsigned int hwHistBinNMax_; unsigned int hwGammaOutMaxSamples_; unsigned int hwHistogramWeightGridsSize_; + + /* Interface to the Camera Helper */ + std::unique_ptr camHelper_; }; -int IPARkISP1::init([[maybe_unused]] const IPASettings &settings, - unsigned int hwRevision) +int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) { /* \todo Add support for other revisions */ switch (hwRevision) { @@ -100,6 +104,15 @@ int IPARkISP1::init([[maybe_unused]] const IPASettings &settings, } LOG(IPARkISP1, Debug) << "Hardware revision is " << hwRevision; + + camHelper_ = CameraSensorHelperFactory::create(settings.sensorModel); + if (!camHelper_) { + LOG(IPARkISP1, Error) + << "Failed to create camera sensor helper for " + << settings.sensorModel; + return -ENODEV; + } + return 0; } From patchwork Tue Nov 23 15:04:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14718 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1B2F3BF415 for ; Tue, 23 Nov 2021 15:04:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CF58D60442; Tue, 23 Nov 2021 16:04:39 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="t8xCL0eg"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ABF126036F for ; Tue, 23 Nov 2021 16:04:28 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5DAC7993; Tue, 23 Nov 2021 16:04:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637679868; bh=5Nsxxw4E2N506pwBm9ygWYo5ljyITRor5SiSazR4TfQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=t8xCL0egxh+gfDfGqbm3qHO5SCMBqXMJ5zqMWwiZqoLEqc1ybmR3KAw+0Wc7qPQXP P7Zc55c5iumHESprDiimx/QLM+SvDp2Val0ZZaazP81nlsy1q0F2xiBresxEe1oc1R /vReW/HVKNZxkVgna4U/FGOD/tkEQWfbPdwCSz7o= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 16:04:17 +0100 Message-Id: <20211123150423.125524-6-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 05/11] ipa: rkisp1: Introduce IPAContext X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Before using any algorithm, we want the IPAContext to be ready for those. Introduce the IPAContext following the existing design from IPA::IPU3. Each algorithm will then introduce the needed fields. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/ipa_context.cpp | 58 ++++++++++++++++++++++++++++++++++ src/ipa/rkisp1/ipa_context.h | 30 ++++++++++++++++++ src/ipa/rkisp1/meson.build | 7 +++- src/ipa/rkisp1/rkisp1.cpp | 7 ++++ 4 files changed, 101 insertions(+), 1 deletion(-) create mode 100644 src/ipa/rkisp1/ipa_context.cpp create mode 100644 src/ipa/rkisp1/ipa_context.h diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp new file mode 100644 index 00000000..819b2c73 --- /dev/null +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * ipa_context.cpp - RkISP1 IPA Context + */ + +#include "ipa_context.h" + +/** + * \file ipa_context.h + * \brief Context and state information shared between the algorithms + */ + +namespace libcamera::ipa::rkisp1 { + +/** + * \struct IPASessionConfiguration + * \brief Session configuration for the IPA module + * + * The session configuration contains all IPA configuration parameters that + * remain constant during the capture session, from IPA module start to stop. + * It is typically set during the configure() operation of the IPA module, but + * may also be updated in the start() operation. + */ + +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + * + * The frame context stores data specific to a single frame processed by the + * IPA. Each frame processed by the IPA has a context associated with it, + * accessible through the IPAContext structure. + * + * \todo Detail how to access contexts for a particular frame + * + * Each of the fields in the frame context belongs to either a specific + * algorithm, or to the top-level IPA module. A field may be read by any + * algorithm, but should only be written by its owner. + */ + +/** + * \struct IPAContext + * \brief Global IPA context data shared between all algorithms + * + * \var IPAContext::configuration + * \brief The IPA session configuration, immutable during the session + * + * \var IPAContext::frameContext + * \brief The frame context for the frame being processed + * + * \todo While the frame context is supposed to be per-frame, this + * single frame context stores data related to both the current frame + * and the previous frames, with fields being updated as the algorithms + * are run. This needs to be turned into real per-frame data storage. + */ + +} /* namespace libcamera::ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h new file mode 100644 index 00000000..ff40efe3 --- /dev/null +++ b/src/ipa/rkisp1/ipa_context.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * ipa_context.h - RkISP1 IPA Context + * + */ +#ifndef __LIBCAMERA_RKISP1_IPA_CONTEXT_H__ +#define __LIBCAMERA_RKISP1_IPA_CONTEXT_H__ + +namespace libcamera { + +namespace ipa::rkisp1 { + +struct IPASessionConfiguration { +}; + +struct IPAFrameContext { +}; + +struct IPAContext { + IPASessionConfiguration configuration; + IPAFrameContext frameContext; +}; + +} /* namespace ipa::rkisp1 */ + +} /* namespace libcamera*/ + +#endif /* __LIBCAMERA_RKISP1_IPA_CONTEXT_H__ */ diff --git a/src/ipa/rkisp1/meson.build b/src/ipa/rkisp1/meson.build index f76b37f5..3683c922 100644 --- a/src/ipa/rkisp1/meson.build +++ b/src/ipa/rkisp1/meson.build @@ -2,8 +2,13 @@ ipa_name = 'ipa_rkisp1' +rkisp1_ipa_sources = files([ + 'ipa_context.cpp', + 'rkisp1.cpp', +]) + mod = shared_module(ipa_name, - ['rkisp1.cpp', libcamera_generated_ipa_headers], + [rkisp1_ipa_sources, libcamera_generated_ipa_headers], name_prefix : '', include_directories : [ipa_includes, libipa_includes], dependencies : libcamera_private, diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 73a7f582..34c3f9a2 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -25,6 +25,7 @@ #include +#include "ipa_context.h" #include "libipa/camera_sensor_helper.h" namespace libcamera { @@ -78,6 +79,9 @@ private: /* Interface to the Camera Helper */ std::unique_ptr camHelper_; + + /* Local parameter storage */ + struct IPAContext context_; }; int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) @@ -164,6 +168,9 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, << "Exposure: " << minExposure_ << "-" << maxExposure_ << " Gain: " << minGain_ << "-" << maxGain_; + /* Clean context at configuration */ + context_ = {}; + return 0; } From patchwork Tue Nov 23 15:04:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14719 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A49EFC3250 for ; Tue, 23 Nov 2021 15:04:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 40EA96022F; Tue, 23 Nov 2021 16:04:40 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="OaCBYIXs"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DBD656038A for ; Tue, 23 Nov 2021 16:04:28 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9240111BB; Tue, 23 Nov 2021 16:04:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637679868; bh=DHZVwOMhXllSyW4+iIbBha5GEfcztvqNGRywvoA7h/o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OaCBYIXsFMT93C1NW4D5LnsgjNwhvlf/f2/I3a/c/wru/CGOqaoOHgnvyFNVu6XRj jfrUEKiPK7PvHz4ldB7MWlUyWbhaD6QMsQPa7XvbibG64bYLSShLmBYSPOCKBtqHVJ I4NCoOUNf7QCiPRrAqwYkpive0cVgLfwvpnGqFHU= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 16:04:18 +0100 Message-Id: <20211123150423.125524-7-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 06/11] ipa: libipa: Introduce Algorithm class template X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The algorithms are using the same function names with specialized parameters. Instead of duplicating code, introduce a libipa Algorithm class which implements a base class with template parameters in libipa, and use it in each IPA. As we now won't need an algorithm class for each IPA, move the documentation to libipa, and make it agnostic of the IPA used. While at it, fix the IPU3::Algorithm::Awb documentation. Signed-off-by: Laurent Pinchart Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- v3: - split rkisp1 out from the patch - squash the AWB doc here --- src/ipa/ipu3/algorithms/algorithm.h | 12 ++---- src/ipa/ipu3/algorithms/awb.cpp | 9 ++++ src/ipa/ipu3/algorithms/meson.build | 1 - .../{ipu3/algorithms => libipa}/algorithm.cpp | 38 ++++++++--------- src/ipa/libipa/algorithm.h | 41 +++++++++++++++++++ src/ipa/libipa/meson.build | 1 + 6 files changed, 70 insertions(+), 32 deletions(-) rename src/ipa/{ipu3/algorithms => libipa}/algorithm.cpp (75%) create mode 100644 src/ipa/libipa/algorithm.h diff --git a/src/ipa/ipu3/algorithms/algorithm.h b/src/ipa/ipu3/algorithms/algorithm.h index 43f5d8b0..3c0ce461 100644 --- a/src/ipa/ipu3/algorithms/algorithm.h +++ b/src/ipa/ipu3/algorithms/algorithm.h @@ -9,21 +9,15 @@ #include +#include + #include "ipa_context.h" namespace libcamera { namespace ipa::ipu3 { -class Algorithm -{ -public: - virtual ~Algorithm() {} - - virtual int configure(IPAContext &context, const IPAConfigInfo &configInfo); - virtual void prepare(IPAContext &context, ipu3_uapi_params *params); - virtual void process(IPAContext &context, const ipu3_uapi_stats_3a *stats); -}; +using Algorithm = libcamera::ipa::Algorithm; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index c7bcb20e..1dc27fc9 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -193,6 +193,9 @@ Awb::Awb() Awb::~Awb() = default; +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ int Awb::configure(IPAContext &context, [[maybe_unused]] const IPAConfigInfo &configInfo) { @@ -373,6 +376,9 @@ void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) } } +/** + * \copydoc libcamera::ipa::Algorithm::process + */ void Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { calculateWBGains(stats); @@ -394,6 +400,9 @@ constexpr uint16_t Awb::threshold(float value) return value * 8191; } +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ void Awb::prepare(IPAContext &context, ipu3_uapi_params *params) { /* diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build index 3ec42f72..4db6ae1d 100644 --- a/src/ipa/ipu3/algorithms/meson.build +++ b/src/ipa/ipu3/algorithms/meson.build @@ -2,7 +2,6 @@ ipu3_ipa_algorithms = files([ 'agc.cpp', - 'algorithm.cpp', 'awb.cpp', 'blc.cpp', 'tone_mapping.cpp', diff --git a/src/ipa/ipu3/algorithms/algorithm.cpp b/src/ipa/libipa/algorithm.cpp similarity index 75% rename from src/ipa/ipu3/algorithms/algorithm.cpp rename to src/ipa/libipa/algorithm.cpp index 3e7e3018..398d5372 100644 --- a/src/ipa/ipu3/algorithms/algorithm.cpp +++ b/src/ipa/libipa/algorithm.cpp @@ -2,7 +2,7 @@ /* * Copyright (C) 2021, Ideas On Board * - * algorithm.cpp - IPU3 control algorithm interface + * algorithm.cpp - IPA control algorithm interface */ #include "algorithm.h" @@ -14,11 +14,15 @@ namespace libcamera { -namespace ipa::ipu3 { +namespace ipa { /** * \class Algorithm - * \brief The base class for all IPU3 algorithms + * \brief The base class for all IPA algorithms + * \tparam Context The type of shared IPA context + * \tparam Config The type of the IPA configuration data + * \tparam Params The type of the ISP specific parameters + * \tparam Stats The type of the IPA statistics and ISP results * * The Algorithm class defines a standard interface for IPA algorithms. By * abstracting algorithms, it makes possible the implementation of generic code @@ -26,6 +30,7 @@ namespace ipa::ipu3 { */ /** + * \fn Algorithm::configure() * \brief Configure the Algorithm given an IPAConfigInfo * \param[in] context The shared IPA context * \param[in] configInfo The IPA configuration data, received from the pipeline @@ -39,37 +44,30 @@ namespace ipa::ipu3 { * * \return 0 if successful, an error code otherwise */ -int Algorithm::configure([[maybe_unused]] IPAContext &context, - [[maybe_unused]] const IPAConfigInfo &configInfo) -{ - return 0; -} /** + * \fn Algorithm::prepare() * \brief Fill the \a params buffer with ISP processing parameters for a frame * \param[in] context The shared IPA context - * \param[out] params The IPU3 specific parameters. + * \param[out] params The ISP specific parameters. * * This function is called for every frame when the camera is running before it - * is processed by the ImgU to prepare the ImgU processing parameters for that + * is processed by the ISP to prepare the ISP processing parameters for that * frame. * * Algorithms shall fill in the parameter structure fields appropriately to - * configure the ImgU processing blocks that they are responsible for. This + * configure the ISP processing blocks that they are responsible for. This * includes setting fields and flags that enable those processing blocks. */ -void Algorithm::prepare([[maybe_unused]] IPAContext &context, - [[maybe_unused]] ipu3_uapi_params *params) -{ -} /** + * \fn Algorithm::process() * \brief Process ISP statistics, and run algorithm operations * \param[in] context The shared IPA context - * \param[in] stats The IPU3 statistics and ISP results + * \param[in] stats The IPA statistics and ISP results * * This function is called while camera is running for every frame processed by - * the ImgU, to process statistics generated from that frame by the ImgU. + * the ISP, to process statistics generated from that frame by the ISP. * Algorithms shall use this data to run calculations and update their state * accordingly. * @@ -91,11 +89,7 @@ void Algorithm::prepare([[maybe_unused]] IPAContext &context, * Care shall be taken to ensure the ordering of access to the information * such that the algorithms use up to date state as required. */ -void Algorithm::process([[maybe_unused]] IPAContext &context, - [[maybe_unused]] const ipu3_uapi_stats_3a *stats) -{ -} -} /* namespace ipa::ipu3 */ +} /* namespace ipa */ } /* namespace libcamera */ diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h new file mode 100644 index 00000000..48087127 --- /dev/null +++ b/src/ipa/libipa/algorithm.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * algorithm.h - ISP control algorithm interface + */ +#ifndef __LIBCAMERA_IPA_LIBIPA_ALGORITHM_H__ +#define __LIBCAMERA_IPA_LIBIPA_ALGORITHM_H__ + +namespace libcamera { + +namespace ipa { + +template +class Algorithm +{ +public: + virtual ~Algorithm() {} + + virtual int configure([[maybe_unused]] Context &context, + [[maybe_unused]] const Config &configInfo) + { + return 0; + } + + virtual void prepare([[maybe_unused]] Context &context, + [[maybe_unused]] Params *params) + { + } + + virtual void process([[maybe_unused]] Context &context, + [[maybe_unused]] const Stats *stats) + { + } +}; + +} /* namespace ipa */ + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_IPA_LIBIPA_ALGORITHM_H__ */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index 4d073a03..161cc5a1 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 libipa_headers = files([ + 'algorithm.h', 'camera_sensor_helper.h', 'histogram.h' ]) From patchwork Tue Nov 23 15:04:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14721 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E0648C3251 for ; Tue, 23 Nov 2021 15:04:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 943FD6049F; Tue, 23 Nov 2021 16:04:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XdBoafpJ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0CC086038C for ; Tue, 23 Nov 2021 16:04:29 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BF2E6993; Tue, 23 Nov 2021 16:04:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637679868; bh=2X7RSKsz1xR1D3Tn2++8a9J/eRWrsR8gH+DyF2WRmn0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XdBoafpJRHrukoPrvecwTeLA/9dKQ9d8tCYSOqFRrJ8QomCOpXx5wYb50wJ3kLGf2 PtUyZpW/ciTGi7/XiY8gLZQunW7JguUmUt4jlNt0VrI0FSPEmx/jcx8PbxRHuQa1oD HCJopFGPxvD7sUEuIbsNiiaxjnf5RRDz4MVEzU/Y= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 16:04:19 +0100 Message-Id: <20211123150423.125524-8-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 07/11] ipa: rkisp1: Use the Algorithm class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that libipa offers a templated class for Algorithm, use it in RkISP1. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/rkisp1/algorithms/algorithm.h | 28 +++++++++++++++++++++++++++ src/ipa/rkisp1/algorithms/meson.build | 4 ++++ src/ipa/rkisp1/meson.build | 4 ++++ src/ipa/rkisp1/rkisp1.cpp | 5 ++++- 4 files changed, 40 insertions(+), 1 deletion(-) create mode 100644 src/ipa/rkisp1/algorithms/algorithm.h create mode 100644 src/ipa/rkisp1/algorithms/meson.build diff --git a/src/ipa/rkisp1/algorithms/algorithm.h b/src/ipa/rkisp1/algorithms/algorithm.h new file mode 100644 index 00000000..dfa58727 --- /dev/null +++ b/src/ipa/rkisp1/algorithms/algorithm.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * algorithm.h - RkISP1 control algorithm interface + */ +#ifndef __LIBCAMERA_IPA_RKISP1_ALGORITHM_H__ +#define __LIBCAMERA_IPA_RKISP1_ALGORITHM_H__ + +#include + +#include + +#include + +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::rkisp1 { + +using Algorithm = libcamera::ipa::Algorithm; + +} /* namespace ipa::rkisp1 */ + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_IPA_RKISP1_ALGORITHM_H__ */ diff --git a/src/ipa/rkisp1/algorithms/meson.build b/src/ipa/rkisp1/algorithms/meson.build new file mode 100644 index 00000000..1c6c59cf --- /dev/null +++ b/src/ipa/rkisp1/algorithms/meson.build @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: CC0-1.0 + +rkisp1_ipa_algorithms = files([ +]) diff --git a/src/ipa/rkisp1/meson.build b/src/ipa/rkisp1/meson.build index 3683c922..8c822fbb 100644 --- a/src/ipa/rkisp1/meson.build +++ b/src/ipa/rkisp1/meson.build @@ -1,5 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 +subdir('algorithms') + ipa_name = 'ipa_rkisp1' rkisp1_ipa_sources = files([ @@ -7,6 +9,8 @@ rkisp1_ipa_sources = files([ 'rkisp1.cpp', ]) +rkisp1_ipa_sources += rkisp1_ipa_algorithms + mod = shared_module(ipa_name, [rkisp1_ipa_sources, libcamera_generated_ipa_headers], name_prefix : '', diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 34c3f9a2..0c54d8ec 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -25,7 +25,7 @@ #include -#include "ipa_context.h" +#include "algorithms/algorithm.h" #include "libipa/camera_sensor_helper.h" namespace libcamera { @@ -82,6 +82,9 @@ private: /* Local parameter storage */ struct IPAContext context_; + + /* Maintain the algorithms used by the IPA */ + std::list> algorithms_; }; int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) From patchwork Tue Nov 23 15:04:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14723 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0E3A0C3252 for ; Tue, 23 Nov 2021 15:04:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B21E4603FC; Tue, 23 Nov 2021 16:04:42 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="TOeuigAh"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3A00260121 for ; Tue, 23 Nov 2021 16:04:29 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E9454F95; Tue, 23 Nov 2021 16:04:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637679869; bh=e+V7V9HX7kQwW8gkHkGirWIj44MWCGIsdN2qZEqzBh0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TOeuigAhL8501sqVBuaEWohzcGht2Z007PGi1rZbJUPAf+yiRXqGj9gF7f69tsQaR qXJuz0ulGcdGlVnm9P5OwUYqKaKnKzfL4ApsiQ73hmZGtHUPCIuoGR7oT1jRfqR/dT JkfDcFUucW1FP4hj6q8GRN15tlrC0PtNwvG5K3GY= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 16:04:20 +0100 Message-Id: <20211123150423.125524-9-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 08/11] ipa: rkisp1: Report and use sensor controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The pipeline handler populates a new sensorControls ControlList, to have the effective exposure and gain values for the current frame. This is done when a statistics buffer is received. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/rkisp1.mojom | 2 ++ src/ipa/rkisp1/rkisp1.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 3 ++- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/include/libcamera/ipa/rkisp1.mojom b/include/libcamera/ipa/rkisp1.mojom index a6991d4f..c3a6d8e1 100644 --- a/include/libcamera/ipa/rkisp1.mojom +++ b/include/libcamera/ipa/rkisp1.mojom @@ -21,11 +21,13 @@ struct RkISP1Event { uint32 frame; uint32 bufferId; libcamera.ControlList controls; + libcamera.ControlList sensorControls; }; struct RkISP1Action { RkISP1Operations op; libcamera.ControlList controls; + libcamera.ControlList sensorControls; }; interface IPARkISP1Interface { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 0c54d8ec..b5aa93f8 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -312,7 +312,7 @@ void IPARkISP1::setControls(unsigned int frame) ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); - op.controls = ctrls; + op.sensorControls = ctrls; queueFrameAction.emit(frame, op); } diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 6cdc4c03..36ef6a02 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -333,7 +333,7 @@ void RkISP1CameraData::queueFrameAction(unsigned int frame, { switch (action.op) { case ipa::rkisp1::ActionV4L2Set: { - const ControlList &controls = action.controls; + const ControlList &controls = action.sensorControls; delayedCtrls_->push(controls); break; } @@ -1124,6 +1124,7 @@ void PipelineHandlerRkISP1::statReady(FrameBuffer *buffer) ev.op = ipa::rkisp1::EventSignalStatBuffer; ev.frame = info->frame; ev.bufferId = info->statBuffer->cookie(); + ev.sensorControls = data->delayedCtrls_->get(buffer->metadata().sequence); data->ipa_->processEvent(ev); } From patchwork Tue Nov 23 15:04:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14720 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3D513C324F for ; Tue, 23 Nov 2021 15:04:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D44CA60398; Tue, 23 Nov 2021 16:04:40 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Bl/0RcwK"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7564760394 for ; Tue, 23 Nov 2021 16:04:29 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 28BD2993; Tue, 23 Nov 2021 16:04:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637679869; bh=AfLesHaI81KKFBlXXjJo6bUvn4fCQHqFqc0tnw+fYQ8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Bl/0RcwKf7lZth2s36DEWYQWQ6OQFBd6oJXsO7ZIrODY/g5qAw92ZS+uTRyRuyEAL bU5urWCHRnCSBm4W66co0o+WC+RHwQzI5i+d/3HN0kejhylAz3mTTfGY1zDKCx7G6V PurrZWKFksRlYZdsK2nRhTzbPo0DZpfTRe+by4fw= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 16:04:21 +0100 Message-Id: <20211123150423.125524-10-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 09/11] ipa: rkisp1: Introduce AGC X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that we have IPAContext and Algorithm, we can implement a simple AGC based on the IPU3 one. It is very similar, except that there is no histogram used for an inter quantile mean. The RkISP1 is returning a 5x5 array (for V10) of luminance means. Estimating the relative luminance is thus a simple mean of all the blocks already calculated by the ISP. Signed-off-by: Jean-Michel Hautbois --- src/ipa/rkisp1/algorithms/agc.cpp | 265 ++++++++++++++++++++++++++ src/ipa/rkisp1/algorithms/agc.h | 55 ++++++ src/ipa/rkisp1/algorithms/meson.build | 1 + src/ipa/rkisp1/ipa_context.cpp | 44 +++++ src/ipa/rkisp1/ipa_context.h | 17 ++ src/ipa/rkisp1/rkisp1.cpp | 72 +++---- 6 files changed, 419 insertions(+), 35 deletions(-) create mode 100644 src/ipa/rkisp1/algorithms/agc.cpp create mode 100644 src/ipa/rkisp1/algorithms/agc.h diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp new file mode 100644 index 00000000..9c6d312e --- /dev/null +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -0,0 +1,265 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * agc.cpp - AGC/AEC mean-based control algorithm + */ + +#include "agc.h" + +#include +#include +#include + +#include + +#include + +/** + * \file agc.h + */ + +namespace libcamera { + +using namespace std::literals::chrono_literals; + +namespace ipa::rkisp1::algorithms { + +/** + * \class Agc + * \brief A mean-based auto-exposure algorithm + */ + +LOG_DEFINE_CATEGORY(RkISP1Agc) + +/* Limits for analogue gain values */ +static constexpr double kMinAnalogueGain = 1.0; +static constexpr double kMaxAnalogueGain = 8.0; + +/* \todo Honour the FrameDurationLimits control instead of hardcoding a limit */ +static constexpr utils::Duration kMaxShutterSpeed = 60ms; + +/* Number of frames to wait before calculating stats on minimum exposure */ +static constexpr uint32_t kNumStartupFrames = 10; + +/* + * Relative luminance target. + * + * It's a number that's chosen so that, when the camera points at a grey + * target, the resulting image brightness is considered right. + */ +static constexpr double kRelativeLuminanceTarget = 0.4; + +Agc::Agc() + : frameCount_(0), lineDuration_(0s), minShutterSpeed_(0s), + maxShutterSpeed_(0s), filteredExposure_(0s), currentExposure_(0s) +{ +} + +/** + * \brief Configure the AGC given a configInfo + * \param[in] context The shared IPA context + * \param[in] configInfo The IPA configuration data + * + * \return 0 + */ +int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) +{ + /* \todo use the IPAContext to provide the limits */ + lineDuration_ = configInfo.lineLength * 1.0s / configInfo.pixelRate; + + minShutterSpeed_ = context.configuration.agc.minShutterSpeed; + maxShutterSpeed_ = std::min(context.configuration.agc.maxShutterSpeed, + kMaxShutterSpeed); + + minAnalogueGain_ = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain); + maxAnalogueGain_ = std::min(context.configuration.agc.maxAnalogueGain, kMaxAnalogueGain); + + /* Configure the default exposure and gain. */ + context.frameContext.agc.gain = minAnalogueGain_; + context.frameContext.agc.exposure = 10ms / lineDuration_; + + return 0; +} + +/** + * \brief Apply a filter on the exposure value to limit the speed of changes + */ +void Agc::filterExposure() +{ + double speed = 0.2; + + /* Adapt instantly if we are in startup phase */ + if (frameCount_ < kNumStartupFrames) + speed = 1.0; + + if (filteredExposure_ == 0s) { + /* DG stands for digital gain.*/ + filteredExposure_ = currentExposure_; + } else { + /* + * If we are close to the desired result, go faster to avoid making + * multiple micro-adjustments. + * \todo Make this customisable? + */ + if (filteredExposure_ < 1.2 * currentExposure_ && + filteredExposure_ > 0.8 * currentExposure_) + speed = sqrt(speed); + + filteredExposure_ = speed * currentExposure_ + + filteredExposure_ * (1.0 - speed); + } + + LOG(RkISP1Agc, Debug) << "After filtering, total_exposure " << filteredExposure_; +} + +/** + * \brief Estimate the new exposure and gain values + * \param[inout] frameContext The shared IPA frame Context + * \param[in] yGain The gain calculated on the current brightness level + */ +void Agc::computeExposure(IPAFrameContext &frameContext, double yGain) +{ + /* Get the effective exposure and gain applied on the sensor. */ + uint32_t exposure = frameContext.sensor.exposure; + double analogueGain = frameContext.sensor.gain; + + /* Consider within 1% of the target as correctly exposed */ + if (std::abs(yGain - 1.0) < 0.01) + LOG(RkISP1Agc, Debug) << "We are well exposed (iqMean = " + << yGain << ")"; + + /* extracted from Rpi::Agc::computeTargetExposure */ + + /* Calculate the shutter time in seconds */ + utils::Duration currentShutter = exposure * lineDuration_; + + /* + * Update the exposure value for the next computation using the values + * of exposure and gain really used by the sensor. + */ + utils::Duration effectiveExposureValue = currentShutter * analogueGain; + + LOG(RkISP1Agc, Debug) << "Actual total exposure " << currentShutter * analogueGain + << " Shutter speed " << currentShutter + << " Gain " << analogueGain + << " Needed ev gain " << yGain; + + /* + * Calculate the current exposure value for the scene as the latest + * exposure value applied multiplied by the new estimated gain. + */ + currentExposure_ = effectiveExposureValue * yGain; + + /* Clamp the exposure value to the min and max authorized */ + utils::Duration maxTotalExposure = maxShutterSpeed_ * maxAnalogueGain_; + currentExposure_ = std::min(currentExposure_, maxTotalExposure); + LOG(RkISP1Agc, Debug) << "Target total exposure " << currentExposure_ + << ", maximum is " << maxTotalExposure; + + /* \todo: estimate if we need to desaturate */ + filterExposure(); + + /* Divide the exposure value as new exposure and gain values */ + utils::Duration exposureValue = filteredExposure_; + utils::Duration shutterTime; + + /* + * Push the shutter time up to the maximum first, and only then + * increase the gain. + */ + shutterTime = std::clamp(exposureValue / minAnalogueGain_, + minShutterSpeed_, maxShutterSpeed_); + double stepGain = std::clamp(exposureValue / shutterTime, + minAnalogueGain_, maxAnalogueGain_); + LOG(RkISP1Agc, Debug) << "Divided up shutter and gain are " + << shutterTime << " and " + << stepGain; + + /* Update the estimated exposure and gain. */ + frameContext.agc.exposure = shutterTime / lineDuration_; + frameContext.agc.gain = stepGain; +} + +/** + * \brief Estimate the relative luminance of the frame with a given gain + * \param[in] ae The RkISP1 statistics and ISP results + * \param[in] gain The gain calculated on the current brightness level + * \return The relative luminance + * + * This function estimates the average relative luminance of the frame that + * would be output by the sensor if an additional \a gain was applied. + * + * The estimation is based on the AE statistics for the current frame. Y + * averages for all cells are first multiplied by the gain, and then saturated + * to approximate the sensor behaviour at high brightness values. The + * approximation is quite rough, as it doesn't take into account non-linearities + * when approaching saturation. + * + * The values are normalized to the [0.0, 1.0] range, where 1.0 corresponds to a + * theoretical perfect reflector of 100% reference white. + * + * More detailed information can be found in: + * https://en.wikipedia.org/wiki/Relative_luminance + */ +double Agc::estimateLuminance(const rkisp1_cif_isp_ae_stat *ae, + double gain) +{ + double ySum = 0.0; + unsigned int num = 0; + + /* Sum the averages, saturated to 255. */ + for (unsigned int aeCell = 0; aeCell < numCells_; aeCell++) { + ySum += std::min(ae->exp_mean[aeCell] * gain, 255.0); + num++; + } + + /* \todo Weight with the AWB gains */ + + return ySum / num / 255; +} + +/** + * \brief Process RkISP1 statistics, and run AGC operations + * \param[in] context The shared IPA context + * \param[in] stats The RKIsp1 statistics and ISP results + * + * Identify the current image brightness, and use that to estimate the optimal + * new exposure and gain for the scene. + */ +void Agc::process(IPAContext &context, const rkisp1_stat_buffer *stats) +{ + const rkisp1_cif_isp_stat *params = &stats->params; + ASSERT(stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP); + + const rkisp1_cif_isp_ae_stat *ae = ¶ms->ae; + + /* + * Estimate the gain needed to achieve a relative luminance target. To + * account for non-linearity caused by saturation, the value needs to be + * estimated in an iterative process, as multiplying by a gain will not + * increase the relative luminance by the same factor if some image + * regions are saturated. + */ + double yGain = 1.0; + double yTarget = kRelativeLuminanceTarget; + + for (unsigned int i = 0; i < 8; i++) { + double yValue = estimateLuminance(ae, yGain); + double extra_gain = std::min(10.0, yTarget / (yValue + .001)); + + yGain *= extra_gain; + LOG(RkISP1Agc, Debug) << "Y value: " << yValue + << ", Y target: " << yTarget + << ", gives gain " << yGain; + if (extra_gain < 1.01) + break; + } + + computeExposure(context.frameContext, yGain); + frameCount_++; +} + +} /* namespace ipa::rkisp1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h new file mode 100644 index 00000000..83159743 --- /dev/null +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * agc.h - RkISP1 AGC/AEC mean-based control algorithm + */ +#ifndef __LIBCAMERA_RKISP1_ALGORITHMS_AGC_H__ +#define __LIBCAMERA_RKISP1_ALGORITHMS_AGC_H__ + +#include + +#include + +#include + +#include "algorithm.h" + +namespace libcamera { + +struct IPACameraSensorInfo; + +namespace ipa::rkisp1::algorithms { + +class Agc : public Algorithm +{ +public: + Agc(); + ~Agc() = default; + + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void process(IPAContext &context, const rkisp1_stat_buffer *stats) override; + +private: + void filterExposure(); + void computeExposure(IPAFrameContext &frameContext, double yGain); + double estimateLuminance(const rkisp1_cif_isp_ae_stat *ae, double gain); + + uint64_t frameCount_; + + utils::Duration lineDuration_; + utils::Duration minShutterSpeed_; + utils::Duration maxShutterSpeed_; + + double minAnalogueGain_; + double maxAnalogueGain_; + + utils::Duration filteredExposure_; + utils::Duration currentExposure_; +}; + +} /* namespace ipa::rkisp1::algorithms */ + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_RKISP1_ALGORITHMS_AGC_H__ */ diff --git a/src/ipa/rkisp1/algorithms/meson.build b/src/ipa/rkisp1/algorithms/meson.build index 1c6c59cf..a19c1a4f 100644 --- a/src/ipa/rkisp1/algorithms/meson.build +++ b/src/ipa/rkisp1/algorithms/meson.build @@ -1,4 +1,5 @@ # SPDX-License-Identifier: CC0-1.0 rkisp1_ipa_algorithms = files([ + 'agc.cpp', ]) diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 819b2c73..16fc248f 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -55,4 +55,48 @@ namespace libcamera::ipa::rkisp1 { * are run. This needs to be turned into real per-frame data storage. */ +/** + * \var IPASessionConfiguration::agc + * \brief AGC parameters configuration of the IPA + * + * \var IPASessionConfiguration::agc.minShutterSpeed + * \brief Minimum shutter speed supported with the configured sensor + * + * \var IPASessionConfiguration::agc.maxShutterSpeed + * \brief Maximum shutter speed supported with the configured sensor + * + * \var IPASessionConfiguration::agc.minAnalogueGain + * \brief Minimum analogue gain supported with the configured sensor + * + * \var IPASessionConfiguration::agc.maxAnalogueGain + * \brief Maximum analogue gain supported with the configured sensor + */ + +/** + * \var IPAFrameContext::agc + * \brief Context for the Automatic Gain Control algorithm + * + * The exposure and gain determined are expected to be applied to the sensor + * at the earliest opportunity. + * + * \var IPAFrameContext::agc.exposure + * \brief Exposure time expressed as a number of lines + * + * \var IPAFrameContext::agc.gain + * \brief Analogue gain multiplier + * + * The gain should be adapted to the sensor specific gain code before applying. + */ + +/** + * \var IPAFrameContext::sensor + * \brief Effective sensor values + * + * \var IPAFrameContext::sensor.exposure + * \brief Exposure time expressed as a number of lines + * + * \var IPAFrameContext::sensor.gain + * \brief Analogue gain multiplier + */ + } /* namespace libcamera::ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index ff40efe3..8e756fb1 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -8,14 +8,31 @@ #ifndef __LIBCAMERA_RKISP1_IPA_CONTEXT_H__ #define __LIBCAMERA_RKISP1_IPA_CONTEXT_H__ +#include + namespace libcamera { namespace ipa::rkisp1 { struct IPASessionConfiguration { + struct { + utils::Duration minShutterSpeed; + utils::Duration maxShutterSpeed; + double minAnalogueGain; + double maxAnalogueGain; + } agc; }; struct IPAFrameContext { + struct { + uint32_t exposure; + double gain; + } agc; + + struct { + uint32_t exposure; + double gain; + } sensor; }; struct IPAContext { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index b5aa93f8..89d98b6c 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -25,6 +25,7 @@ #include +#include "algorithms/agc.h" #include "algorithms/algorithm.h" #include "libipa/camera_sensor_helper.h" @@ -32,6 +33,8 @@ namespace libcamera { LOG_DEFINE_CATEGORY(IPARkISP1) +using namespace std::literals::chrono_literals; + namespace ipa::rkisp1 { class IPARkISP1 : public IPARkISP1Interface @@ -77,6 +80,8 @@ private: unsigned int hwGammaOutMaxSamples_; unsigned int hwHistogramWeightGridsSize_; + utils::Duration lineDuration_; + /* Interface to the Camera Helper */ std::unique_ptr camHelper_; @@ -120,6 +125,9 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) return -ENODEV; } + /* Construct our Algorithms */ + algorithms_.push_back(std::make_unique()); + return 0; } @@ -171,9 +179,29 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, << "Exposure: " << minExposure_ << "-" << maxExposure_ << " Gain: " << minGain_ << "-" << maxGain_; + lineDuration_ = info.lineLength * 1.0s / info.pixelRate; + /* Clean context at configuration */ context_ = {}; + /* + * When the AGC computes the new exposure values for a frame, it needs + * to know the limits for shutter speed and analogue gain. + * As it depends on the sensor, update it with the controls. + * + * \todo take VBLANK into account for maximum shutter speed + */ + context_.configuration.agc.minShutterSpeed = minExposure_ * lineDuration_; + context_.configuration.agc.maxShutterSpeed = maxExposure_ * lineDuration_; + context_.configuration.agc.minAnalogueGain = camHelper_->gain(minGain_); + context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain_); + + for (auto const &algo : algorithms_) { + int ret = algo->configure(context_, info); + if (ret) + return ret; + } + return 0; } @@ -218,6 +246,9 @@ void IPARkISP1::processEvent(const RkISP1Event &event) reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); + context_.frameContext.sensor.exposure = event.sensorControls.get(V4L2_CID_EXPOSURE).get(); + context_.frameContext.sensor.gain = camHelper_->gain(event.sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + updateStatistics(frame, stats); break; } @@ -262,44 +293,12 @@ void IPARkISP1::queueRequest(unsigned int frame, rkisp1_params_cfg *params, void IPARkISP1::updateStatistics(unsigned int frame, const rkisp1_stat_buffer *stats) { - const rkisp1_cif_isp_stat *params = &stats->params; unsigned int aeState = 0; - if (stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP) { - const rkisp1_cif_isp_ae_stat *ae = ¶ms->ae; - - const unsigned int target = 60; + for (auto const &algo : algorithms_) + algo->process(context_, stats); - unsigned int value = 0; - unsigned int num = 0; - for (unsigned int i = 0; i < hwAeMeanMax_; i++) { - if (ae->exp_mean[i] <= 15) - continue; - - value += ae->exp_mean[i]; - num++; - } - value /= num; - - double factor = (double)target / value; - - if (frame % 3 == 0) { - double exposure; - - exposure = factor * exposure_ * gain_ / minGain_; - exposure_ = std::clamp((uint64_t)exposure, - minExposure_, - maxExposure_); - - exposure = exposure / exposure_ * minGain_; - gain_ = std::clamp((uint64_t)exposure, - minGain_, maxGain_); - - setControls(frame + 1); - } - - aeState = fabs(factor - 1.0f) < 0.05f ? 2 : 1; - } + setControls(frame + 1); metadataReady(frame, aeState); } @@ -309,6 +308,9 @@ void IPARkISP1::setControls(unsigned int frame) RkISP1Action op; op.op = ActionV4L2Set; + exposure_ = context_.frameContext.agc.exposure; + gain_ = camHelper_->gainCode(context_.frameContext.agc.gain); + ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); From patchwork Tue Nov 23 15:04:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14722 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 643B6BF415 for ; Tue, 23 Nov 2021 15:04:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1E532604FA; Tue, 23 Nov 2021 16:04:42 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BcAFxZgX"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A934B603B5 for ; Tue, 23 Nov 2021 16:04:29 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6154EF95; Tue, 23 Nov 2021 16:04:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637679869; bh=bT+N3UBi9S+MSpCr9mmMbO2ADxWvOkhfGHF2KLhjiIQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BcAFxZgXeTdVraViU3q4vNdY4P+HROr/dmHvC5vqHQp2kHJdlxqSCvqPyouuCc5wf oHknBimo73VgZWcZYL+mfsv3dZFjF9kpGSvAY95ARj8pCTVEmgDzhvqRDxDnXMz4a3 ZWfyiwUG6U4vUIM3l/mNoPRr479FDWPX1ohZkuTY= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 16:04:22 +0100 Message-Id: <20211123150423.125524-11-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 10/11] ipa: rkisp1: agc: Introduce prepare call X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a new parameter buffer needs to be queued, we need to specify which algorithm is activated or not in the ISP. Add a simple prepare function in AGC for that, which may later evolve to take the exposure locking into account. For that function to be called, we also need to add the loop on the algorithms in IPARkISP1::queueRequest. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/rkisp1/algorithms/agc.cpp | 7 +++++++ src/ipa/rkisp1/algorithms/agc.h | 1 + src/ipa/rkisp1/rkisp1.cpp | 12 +++--------- 3 files changed, 11 insertions(+), 9 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 9c6d312e..0433813a 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -260,6 +260,13 @@ void Agc::process(IPAContext &context, const rkisp1_stat_buffer *stats) frameCount_++; } +void Agc::prepare([[maybe_unused]] IPAContext &context, + rkisp1_params_cfg *params) +{ + params->module_ens |= RKISP1_CIF_ISP_MODULE_AEC; + params->module_en_update |= RKISP1_CIF_ISP_MODULE_AEC; +} + } /* namespace ipa::rkisp1::algorithms */ } /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index 83159743..ac95dea5 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -28,6 +28,7 @@ public: ~Agc() = default; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void prepare(IPAContext &context, rkisp1_params_cfg *params) override; void process(IPAContext &context, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 89d98b6c..59e868ff 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -270,19 +270,13 @@ void IPARkISP1::processEvent(const RkISP1Event &event) } void IPARkISP1::queueRequest(unsigned int frame, rkisp1_params_cfg *params, - const ControlList &controls) + [[maybe_unused]] const ControlList &controls) { /* Prepare parameters buffer. */ memset(params, 0, sizeof(*params)); - /* Auto Exposure on/off. */ - if (controls.contains(controls::AeEnable)) { - autoExposure_ = controls.get(controls::AeEnable); - if (autoExposure_) - params->module_ens = RKISP1_CIF_ISP_MODULE_AEC; - - params->module_en_update = RKISP1_CIF_ISP_MODULE_AEC; - } + for (auto const &algo : algorithms_) + algo->prepare(context_, params); RkISP1Action op; op.op = ActionParamFilled; From patchwork Tue Nov 23 15:04:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14724 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8EA0FC3250 for ; Tue, 23 Nov 2021 15:04:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3B2A86048E; Tue, 23 Nov 2021 16:04:43 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XwC19jWC"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CAF55603BE for ; Tue, 23 Nov 2021 16:04:29 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 905F7993; Tue, 23 Nov 2021 16:04:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637679869; bh=EDLV191SXoMZhbH/vUYEizKVGiT+5OxmrFvAKXWOJSo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XwC19jWCwb2o8KyCa2lYDDZg7BZ6+n80rwV6Pk9o1B50dzBh0XSzHPLbasbwbNDpM 4ZWiIxpOsNBEzBfuCFcrC5LAMA2JnoV6NBcmRC5UuZ0yl89nhBz9vjznzVS3ZtDmBi a+JkOlIHnWbr77FcYWH31b8nVWHzLA3zNUC5h16k= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 16:04:23 +0100 Message-Id: <20211123150423.125524-12-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123150423.125524-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 11/11] ipa: rkisp1: agc: Configure the number of cells used by HW X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ISP can use 25 or 81 cells depending on its revision. Remove the cached value in IPARkISP1 and use IPASessionConfiguration to store it and pass it to AGC. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- v3: - Use the rkisp1 enum for the HW revision --- src/ipa/rkisp1/algorithms/agc.cpp | 12 +++++++++++- src/ipa/rkisp1/algorithms/agc.h | 2 ++ src/ipa/rkisp1/ipa_context.cpp | 8 ++++++++ src/ipa/rkisp1/ipa_context.h | 6 ++++++ src/ipa/rkisp1/rkisp1.cpp | 10 +++++++--- 5 files changed, 34 insertions(+), 4 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 0433813a..16966b16 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -61,7 +61,7 @@ Agc::Agc() * \param[in] context The shared IPA context * \param[in] configInfo The IPA configuration data * - * \return 0 + * \return 0 on success or a negative error code otherwise */ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { @@ -79,6 +79,16 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) context.frameContext.agc.gain = minAnalogueGain_; context.frameContext.agc.exposure = 10ms / lineDuration_; + /* + * According to the RkISP1 documentation: + * - versions <= V11 have RKISP1_CIF_ISP_AE_MEAN_MAX_V10 entries, + * - versions >= V12 have RKISP1_CIF_ISP_AE_MEAN_MAX_V12 entries. + */ + if (context.configuration.hw.revision <= RKISP1_V11) + numCells_ = RKISP1_CIF_ISP_AE_MEAN_MAX_V10; + else + numCells_ = RKISP1_CIF_ISP_AE_MEAN_MAX_V12; + return 0; } diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index ac95dea5..3ab3f347 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -45,6 +45,8 @@ private: double minAnalogueGain_; double maxAnalogueGain_; + uint32_t numCells_; + utils::Duration filteredExposure_; utils::Duration currentExposure_; }; diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 16fc248f..faa002d1 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -72,6 +72,14 @@ namespace libcamera::ipa::rkisp1 { * \brief Maximum analogue gain supported with the configured sensor */ +/** + * \var IPASessionConfiguration::hw + * \brief RkISP1-specific hardware information + * + * \var IPASessionConfiguration::hw.revision + * \brief Hardware revision of the ISP (RKISP1_V*) + */ + /** * \var IPAFrameContext::agc * \brief Context for the Automatic Gain Control algorithm diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 8e756fb1..e526a0d3 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -8,6 +8,8 @@ #ifndef __LIBCAMERA_RKISP1_IPA_CONTEXT_H__ #define __LIBCAMERA_RKISP1_IPA_CONTEXT_H__ +#include + #include namespace libcamera { @@ -21,6 +23,10 @@ struct IPASessionConfiguration { double minAnalogueGain; double maxAnalogueGain; } agc; + + struct { + rkisp1_cif_isp_version revision; + } hw; }; struct IPAFrameContext { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 59e868ff..99ccd596 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -75,7 +75,7 @@ private: uint32_t maxGain_; /* revision-specific data */ - unsigned int hwAeMeanMax_; + rkisp1_cif_isp_version hwRevision_; unsigned int hwHistBinNMax_; unsigned int hwGammaOutMaxSamples_; unsigned int hwHistogramWeightGridsSize_; @@ -97,13 +97,11 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) /* \todo Add support for other revisions */ switch (hwRevision) { case RKISP1_V10: - hwAeMeanMax_ = RKISP1_CIF_ISP_AE_MEAN_MAX_V10; hwHistBinNMax_ = RKISP1_CIF_ISP_HIST_BIN_N_MAX_V10; hwGammaOutMaxSamples_ = RKISP1_CIF_ISP_GAMMA_OUT_MAX_SAMPLES_V10; hwHistogramWeightGridsSize_ = RKISP1_CIF_ISP_HISTOGRAM_WEIGHT_GRIDS_SIZE_V10; break; case RKISP1_V12: - hwAeMeanMax_ = RKISP1_CIF_ISP_AE_MEAN_MAX_V12; hwHistBinNMax_ = RKISP1_CIF_ISP_HIST_BIN_N_MAX_V12; hwGammaOutMaxSamples_ = RKISP1_CIF_ISP_GAMMA_OUT_MAX_SAMPLES_V12; hwHistogramWeightGridsSize_ = RKISP1_CIF_ISP_HISTOGRAM_WEIGHT_GRIDS_SIZE_V12; @@ -117,6 +115,9 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) LOG(IPARkISP1, Debug) << "Hardware revision is " << hwRevision; + /* Cache the value to set it in configure. */ + hwRevision_ = static_cast(hwRevision); + camHelper_ = CameraSensorHelperFactory::create(settings.sensorModel); if (!camHelper_) { LOG(IPARkISP1, Error) @@ -184,6 +185,9 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, /* Clean context at configuration */ context_ = {}; + /* Set the hardware revision for the algorithms. */ + context_.configuration.hw.revision = hwRevision_; + /* * When the AGC computes the new exposure values for a frame, it needs * to know the limits for shutter speed and analogue gain.