From patchwork Tue Nov 23 09:14:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14685 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 595A5BDB13 for ; Tue, 23 Nov 2021 09:15:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B260360423; Tue, 23 Nov 2021 10:14:59 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="O0plcA+o"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3C0916033C for ; Tue, 23 Nov 2021 10:14:57 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C63F5F95; Tue, 23 Nov 2021 10:14:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637658896; bh=6qkF2Tg7yCai9Xl8kh9FitL+Os2hwaUjtO8oJRvJdwQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=O0plcA+o/I5qAN9B4wiN8ozKfFBvIsUfhfmtqFoWDM+vDknetqyFh47lQbQmB3gCf cH6vuMFofZ1Mx9dqEDtOu4XB1R8Jy0/B9SFCiVOreoNaegLHYd7gVmsWGZnr6kOerF 8vvFqVPwDniLZs6Djvo4LwaMe4edaQWaefG7I9P8= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 10:14:41 +0100 Message-Id: <20211123091451.67404-2-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 01/11] libipa: Correct IMX219 in CameraSensorHelper X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The equation is badly reported in the CameraSensorHelper, as m1 and c0 are inverted. Correct it to have a proper gain calculation. Signed-off-by: Jean-Michel Hautbois Acked-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/libipa/camera_sensor_helper.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/libipa/camera_sensor_helper.cpp b/src/ipa/libipa/camera_sensor_helper.cpp index 0b0eb503..5d052612 100644 --- a/src/ipa/libipa/camera_sensor_helper.cpp +++ b/src/ipa/libipa/camera_sensor_helper.cpp @@ -290,7 +290,7 @@ class CameraSensorHelperImx219 : public CameraSensorHelper public: CameraSensorHelperImx219() { - analogueGainConstants_ = { AnalogueGainLinear, 0, -1, 256, 256 }; + analogueGainConstants_ = { AnalogueGainLinear, 0, 256, -1, 256 }; } }; REGISTER_CAMERA_SENSOR_HELPER("imx219", CameraSensorHelperImx219) From patchwork Tue Nov 23 09:14:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14686 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DB55EC324F for ; Tue, 23 Nov 2021 09:15:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9CA1F603D9; Tue, 23 Nov 2021 10:15:00 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="E0H51yhS"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 539626036F for ; Tue, 23 Nov 2021 10:14:57 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 056BA11BB; Tue, 23 Nov 2021 10:14:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637658897; bh=NilDTt+2dNsnCDdzgETlFJ//2qg8ZrvXdiA9K5ZjUdQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=E0H51yhSoDqJ8OHl0+pP+AhuNMSzoISdTrPRr4LKSXio3IMNBAe4VYDOM3c/POiTB ZYS8OPunFZT0gabQDQZ6rvATeiDPrJUimVBhCuUmarnabdScbL18+nvRnhkojbz4v9 1XE9yOKoIs/e9sjgJnwBFr78LPuhnm840157pqN0= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 10:14:42 +0100 Message-Id: <20211123091451.67404-3-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 02/11] ipa: rkisp1: Pass IPASettings at init call X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the IPA is initialized, it will need to know the sensor model used in order to properly call CameraSensorHelper for the analogue gain. Modify the init definition in the pipeline handler and in the IPA as well as the mojo interface to pass the IPASettings. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/rkisp1.mojom | 4 +++- src/ipa/rkisp1/rkisp1.cpp | 5 +++-- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- 3 files changed, 7 insertions(+), 4 deletions(-) diff --git a/include/libcamera/ipa/rkisp1.mojom b/include/libcamera/ipa/rkisp1.mojom index cae757ea..a6991d4f 100644 --- a/include/libcamera/ipa/rkisp1.mojom +++ b/include/libcamera/ipa/rkisp1.mojom @@ -29,7 +29,9 @@ struct RkISP1Action { }; interface IPARkISP1Interface { - init(uint32 hwRevision) => (int32 ret); + init(libcamera.IPASettings settings, + uint32 hwRevision) + => (int32 ret); start() => (int32 ret); stop(); diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index bf2c13b6..7ecbf8ae 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -34,7 +34,7 @@ namespace ipa::rkisp1 { class IPARkISP1 : public IPARkISP1Interface { public: - int init(unsigned int hwRevision) override; + int init(const IPASettings &settings, unsigned int hwRevision) override; int start() override; void stop() override {} @@ -75,7 +75,8 @@ private: unsigned int hwHistogramWeightGridsSize_; }; -int IPARkISP1::init(unsigned int hwRevision) +int IPARkISP1::init([[maybe_unused]] const IPASettings &settings, + unsigned int hwRevision) { /* \todo Add support for other revisions */ switch (hwRevision) { diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 98008862..6cdc4c03 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -319,7 +319,7 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision) ipa_->queueFrameAction.connect(this, &RkISP1CameraData::queueFrameAction); - int ret = ipa_->init(hwRevision); + int ret = ipa_->init(IPASettings{ "", sensor_->model() }, hwRevision); if (ret < 0) { LOG(RkISP1, Error) << "IPA initialization failure"; return ret; From patchwork Tue Nov 23 09:14:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14687 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C7B65BDB13 for ; Tue, 23 Nov 2021 09:15:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 543B560442; Tue, 23 Nov 2021 10:15:03 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="H0JonW71"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7040660376 for ; Tue, 23 Nov 2021 10:14:57 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2DD04A1B; Tue, 23 Nov 2021 10:14:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637658897; bh=cDAwKOK7nZf5p1eAa/44NWbZMgG4DKa6W2900WGBzKM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H0JonW71WTKsfFlEjPNOBEVXn3wjH1DkIt/VJ4K+er7NoJiqZd/hRKIUAlJLOwjEu E3dntpqHZdFe/E/ISme7RhAkel4CRgUH//Ocs1w8UdHoTJMIX3O7coLzzljuAxIsGf u5O1dUMi2SKDBKM0L2gyRuDafEQyijQ3sXQGNc2U= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 10:14:43 +0100 Message-Id: <20211123091451.67404-4-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 03/11] ipa: Do not modify the sensor limits X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The driver is responsible of setting the proper limits for its controls. For instance, imx219 has an analogue gain of 1.0 when the gain code is set to 0. The minimum analogue is forced to be at least 1, which for imx219 sets it to 1.00329. Rework this for both IPU3 and RkISP1. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart --- src/ipa/ipu3/ipu3.cpp | 4 ++-- src/ipa/rkisp1/rkisp1.cpp | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index a8d54a5d..b0c75541 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -444,11 +444,11 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo, return -EINVAL; } - minExposure_ = std::max(itExp->second.min().get(), 1); + minExposure_ = itExp->second.min().get(); maxExposure_ = itExp->second.max().get(); exposure_ = minExposure_; - minGain_ = std::max(itGain->second.min().get(), 1); + minGain_ = itGain->second.min().get(); maxGain_ = itGain->second.max().get(); gain_ = minGain_; diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 7ecbf8ae..910ad952 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -139,11 +139,11 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, autoExposure_ = true; - minExposure_ = std::max(itExp->second.min().get(), 1); + minExposure_ = itExp->second.min().get(); maxExposure_ = itExp->second.max().get(); exposure_ = minExposure_; - minGain_ = std::max(itGain->second.min().get(), 1); + minGain_ = itGain->second.min().get(); maxGain_ = itGain->second.max().get(); gain_ = minGain_; From patchwork Tue Nov 23 09:14:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14688 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6FEF6C324F for ; Tue, 23 Nov 2021 09:15:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AC3DA60491; Tue, 23 Nov 2021 10:15:03 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="t/ynnZCw"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9D45B6038A for ; Tue, 23 Nov 2021 10:14:57 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 57720F95; Tue, 23 Nov 2021 10:14:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637658897; bh=7EKJLxN6FyygVP3ODjD71RglMl4Afhijn0ZyokJr6KI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=t/ynnZCwNIqSJ0WtZKtkV4Zb9di4mCgWCG/5QeJHRYLw1Tm4y3I8dtiMIdXGrPUet 5Ti6pNs58tW2EoXoaloqoERFSIMDvUpliOn92oSMejol852nqiGhg5Lf37Au48eo78 R6B2qKGxhN5SgoROLYojbV+yVcaOZMXD5+jDMfZ0= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 10:14:44 +0100 Message-Id: <20211123091451.67404-5-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 04/11] ipa: rkisp1: Instantiate CameraSensorHelper X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the AGC will run, it will use analogue gains as double values. We will need those values to be converted to apply the control. Introduce CameraSensorHelper and call it at init(). Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/rkisp1.cpp | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 910ad952..73a7f582 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -25,6 +25,8 @@ #include +#include "libipa/camera_sensor_helper.h" + namespace libcamera { LOG_DEFINE_CATEGORY(IPARkISP1) @@ -73,10 +75,12 @@ private: unsigned int hwHistBinNMax_; unsigned int hwGammaOutMaxSamples_; unsigned int hwHistogramWeightGridsSize_; + + /* Interface to the Camera Helper */ + std::unique_ptr camHelper_; }; -int IPARkISP1::init([[maybe_unused]] const IPASettings &settings, - unsigned int hwRevision) +int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) { /* \todo Add support for other revisions */ switch (hwRevision) { @@ -100,6 +104,15 @@ int IPARkISP1::init([[maybe_unused]] const IPASettings &settings, } LOG(IPARkISP1, Debug) << "Hardware revision is " << hwRevision; + + camHelper_ = CameraSensorHelperFactory::create(settings.sensorModel); + if (!camHelper_) { + LOG(IPARkISP1, Error) + << "Failed to create camera sensor helper for " + << settings.sensorModel; + return -ENODEV; + } + return 0; } From patchwork Tue Nov 23 09:14:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14689 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1709AC3250 for ; Tue, 23 Nov 2021 09:15:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 210496038C; Tue, 23 Nov 2021 10:15:04 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="M/fpcA3s"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D030E6038B for ; Tue, 23 Nov 2021 10:14:57 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8A779A1B; Tue, 23 Nov 2021 10:14:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637658897; bh=EQUVCyvOeCY6QseexrIrQwjZIUB0MoFO7bec3ns9atg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=M/fpcA3sqshn/J8H4yD3J5pv1vN2iwhQAEZO4EgpEuT2UX4b+vu7pgNJzsKz6FVfD 3uiJJ215OtyVmvqa2CvGJgi2wltvf3UmXoaet7dfWU67ALmm+JIEW0IItd29GP42pO G+ff4+ZSZ+gAuvjj+erEPrOvh+WhUYgHefIPWOuA= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 10:14:45 +0100 Message-Id: <20211123091451.67404-6-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 05/11] ipa: rkisp1: Introduce IPAContext X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Before using any algorithm, we want the IPAContext to be ready for those. Introduce the IPAContext following the existing design from IPA::IPU3. Each algorithm will then introduce the needed fields. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/ipa_context.cpp | 58 ++++++++++++++++++++++++++++++++++ src/ipa/rkisp1/ipa_context.h | 30 ++++++++++++++++++ src/ipa/rkisp1/meson.build | 7 +++- src/ipa/rkisp1/rkisp1.cpp | 7 ++++ 4 files changed, 101 insertions(+), 1 deletion(-) create mode 100644 src/ipa/rkisp1/ipa_context.cpp create mode 100644 src/ipa/rkisp1/ipa_context.h diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp new file mode 100644 index 00000000..819b2c73 --- /dev/null +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * ipa_context.cpp - RkISP1 IPA Context + */ + +#include "ipa_context.h" + +/** + * \file ipa_context.h + * \brief Context and state information shared between the algorithms + */ + +namespace libcamera::ipa::rkisp1 { + +/** + * \struct IPASessionConfiguration + * \brief Session configuration for the IPA module + * + * The session configuration contains all IPA configuration parameters that + * remain constant during the capture session, from IPA module start to stop. + * It is typically set during the configure() operation of the IPA module, but + * may also be updated in the start() operation. + */ + +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + * + * The frame context stores data specific to a single frame processed by the + * IPA. Each frame processed by the IPA has a context associated with it, + * accessible through the IPAContext structure. + * + * \todo Detail how to access contexts for a particular frame + * + * Each of the fields in the frame context belongs to either a specific + * algorithm, or to the top-level IPA module. A field may be read by any + * algorithm, but should only be written by its owner. + */ + +/** + * \struct IPAContext + * \brief Global IPA context data shared between all algorithms + * + * \var IPAContext::configuration + * \brief The IPA session configuration, immutable during the session + * + * \var IPAContext::frameContext + * \brief The frame context for the frame being processed + * + * \todo While the frame context is supposed to be per-frame, this + * single frame context stores data related to both the current frame + * and the previous frames, with fields being updated as the algorithms + * are run. This needs to be turned into real per-frame data storage. + */ + +} /* namespace libcamera::ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h new file mode 100644 index 00000000..ff40efe3 --- /dev/null +++ b/src/ipa/rkisp1/ipa_context.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * ipa_context.h - RkISP1 IPA Context + * + */ +#ifndef __LIBCAMERA_RKISP1_IPA_CONTEXT_H__ +#define __LIBCAMERA_RKISP1_IPA_CONTEXT_H__ + +namespace libcamera { + +namespace ipa::rkisp1 { + +struct IPASessionConfiguration { +}; + +struct IPAFrameContext { +}; + +struct IPAContext { + IPASessionConfiguration configuration; + IPAFrameContext frameContext; +}; + +} /* namespace ipa::rkisp1 */ + +} /* namespace libcamera*/ + +#endif /* __LIBCAMERA_RKISP1_IPA_CONTEXT_H__ */ diff --git a/src/ipa/rkisp1/meson.build b/src/ipa/rkisp1/meson.build index f76b37f5..3683c922 100644 --- a/src/ipa/rkisp1/meson.build +++ b/src/ipa/rkisp1/meson.build @@ -2,8 +2,13 @@ ipa_name = 'ipa_rkisp1' +rkisp1_ipa_sources = files([ + 'ipa_context.cpp', + 'rkisp1.cpp', +]) + mod = shared_module(ipa_name, - ['rkisp1.cpp', libcamera_generated_ipa_headers], + [rkisp1_ipa_sources, libcamera_generated_ipa_headers], name_prefix : '', include_directories : [ipa_includes, libipa_includes], dependencies : libcamera_private, diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 73a7f582..34c3f9a2 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -25,6 +25,7 @@ #include +#include "ipa_context.h" #include "libipa/camera_sensor_helper.h" namespace libcamera { @@ -78,6 +79,9 @@ private: /* Interface to the Camera Helper */ std::unique_ptr camHelper_; + + /* Local parameter storage */ + struct IPAContext context_; }; int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) @@ -164,6 +168,9 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, << "Exposure: " << minExposure_ << "-" << maxExposure_ << " Gain: " << minGain_ << "-" << maxGain_; + /* Clean context at configuration */ + context_ = {}; + return 0; } From patchwork Tue Nov 23 09:14:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14690 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9B715BDB13 for ; Tue, 23 Nov 2021 09:15:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 24AB0603CB; Tue, 23 Nov 2021 10:15:05 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="KwuPH8lN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 05F226038C for ; Tue, 23 Nov 2021 10:14:58 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B7729F95; Tue, 23 Nov 2021 10:14:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637658897; bh=p13rWa+3iqUgkFarjy07FZEvynT03pE9wGITsSUA6oI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KwuPH8lNGsBTB1TRCU2moOf9WgHYZ0Qf/nMQCujWw8wsTRinewaEbyjogwWvs6jEY gr+G0rGDIWctqUXx5ZPXpwNAYz1H35t74gcEDKHkMbK+9ngYlhIJRHwGSDvV/cLdac 89jb0oTJ2spvot4XgvsgqTE/xcFDGUaTN1X1lA4w= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 10:14:46 +0100 Message-Id: <20211123091451.67404-7-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 06/11] ipa: libipa: Introduce Algorithm class template X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The algorithms are using the same function names with specialized parameters. Instead of duplicating code, introduce a libipa Algorithm class which implements a base class with template parameters in libipa, and use it in each IPA. As we now won't need an algorithm class for each IPA, move the documentation to libipa, and make it agnostic of the IPA used. Signed-off-by: Laurent Pinchart Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/algorithm.h | 12 ++---- src/ipa/ipu3/algorithms/meson.build | 1 - .../{ipu3/algorithms => libipa}/algorithm.cpp | 35 ++++++---------- src/ipa/libipa/algorithm.h | 41 +++++++++++++++++++ src/ipa/libipa/meson.build | 1 + src/ipa/rkisp1/algorithms/algorithm.h | 28 +++++++++++++ src/ipa/rkisp1/algorithms/meson.build | 4 ++ src/ipa/rkisp1/meson.build | 4 ++ src/ipa/rkisp1/rkisp1.cpp | 5 ++- 9 files changed, 97 insertions(+), 34 deletions(-) rename src/ipa/{ipu3/algorithms => libipa}/algorithm.cpp (74%) create mode 100644 src/ipa/libipa/algorithm.h create mode 100644 src/ipa/rkisp1/algorithms/algorithm.h create mode 100644 src/ipa/rkisp1/algorithms/meson.build diff --git a/src/ipa/ipu3/algorithms/algorithm.h b/src/ipa/ipu3/algorithms/algorithm.h index 43f5d8b0..3c0ce461 100644 --- a/src/ipa/ipu3/algorithms/algorithm.h +++ b/src/ipa/ipu3/algorithms/algorithm.h @@ -9,21 +9,15 @@ #include +#include + #include "ipa_context.h" namespace libcamera { namespace ipa::ipu3 { -class Algorithm -{ -public: - virtual ~Algorithm() {} - - virtual int configure(IPAContext &context, const IPAConfigInfo &configInfo); - virtual void prepare(IPAContext &context, ipu3_uapi_params *params); - virtual void process(IPAContext &context, const ipu3_uapi_stats_3a *stats); -}; +using Algorithm = libcamera::ipa::Algorithm; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build index 3ec42f72..4db6ae1d 100644 --- a/src/ipa/ipu3/algorithms/meson.build +++ b/src/ipa/ipu3/algorithms/meson.build @@ -2,7 +2,6 @@ ipu3_ipa_algorithms = files([ 'agc.cpp', - 'algorithm.cpp', 'awb.cpp', 'blc.cpp', 'tone_mapping.cpp', diff --git a/src/ipa/ipu3/algorithms/algorithm.cpp b/src/ipa/libipa/algorithm.cpp similarity index 74% rename from src/ipa/ipu3/algorithms/algorithm.cpp rename to src/ipa/libipa/algorithm.cpp index 3e7e3018..888ffb57 100644 --- a/src/ipa/ipu3/algorithms/algorithm.cpp +++ b/src/ipa/libipa/algorithm.cpp @@ -2,7 +2,7 @@ /* * Copyright (C) 2021, Ideas On Board * - * algorithm.cpp - IPU3 control algorithm interface + * algorithm.cpp - IPA control algorithm interface */ #include "algorithm.h" @@ -14,11 +14,11 @@ namespace libcamera { -namespace ipa::ipu3 { +namespace ipa { /** * \class Algorithm - * \brief The base class for all IPU3 algorithms + * \brief The base class for all IPA algorithms * * The Algorithm class defines a standard interface for IPA algorithms. By * abstracting algorithms, it makes possible the implementation of generic code @@ -26,6 +26,7 @@ namespace ipa::ipu3 { */ /** + * \fn Algorithm::configure() * \brief Configure the Algorithm given an IPAConfigInfo * \param[in] context The shared IPA context * \param[in] configInfo The IPA configuration data, received from the pipeline @@ -39,37 +40,29 @@ namespace ipa::ipu3 { * * \return 0 if successful, an error code otherwise */ -int Algorithm::configure([[maybe_unused]] IPAContext &context, - [[maybe_unused]] const IPAConfigInfo &configInfo) -{ - return 0; -} /** + * \fn Algorithm::prepare() * \brief Fill the \a params buffer with ISP processing parameters for a frame * \param[in] context The shared IPA context - * \param[out] params The IPU3 specific parameters. + * \param[out] params The ISP specific parameters. * * This function is called for every frame when the camera is running before it - * is processed by the ImgU to prepare the ImgU processing parameters for that - * frame. + * is processed by the ISP. * * Algorithms shall fill in the parameter structure fields appropriately to - * configure the ImgU processing blocks that they are responsible for. This + * configure the ISP processing blocks that they are responsible for. This * includes setting fields and flags that enable those processing blocks. */ -void Algorithm::prepare([[maybe_unused]] IPAContext &context, - [[maybe_unused]] ipu3_uapi_params *params) -{ -} /** + * \fn Algorithm::process() * \brief Process ISP statistics, and run algorithm operations * \param[in] context The shared IPA context - * \param[in] stats The IPU3 statistics and ISP results + * \param[in] stats The IPA statistics and ISP results * * This function is called while camera is running for every frame processed by - * the ImgU, to process statistics generated from that frame by the ImgU. + * the ISP, to process statistics generated from that frame by the ISP. * Algorithms shall use this data to run calculations and update their state * accordingly. * @@ -91,11 +84,7 @@ void Algorithm::prepare([[maybe_unused]] IPAContext &context, * Care shall be taken to ensure the ordering of access to the information * such that the algorithms use up to date state as required. */ -void Algorithm::process([[maybe_unused]] IPAContext &context, - [[maybe_unused]] const ipu3_uapi_stats_3a *stats) -{ -} -} /* namespace ipa::ipu3 */ +} /* namespace ipa */ } /* namespace libcamera */ diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h new file mode 100644 index 00000000..74dc49c4 --- /dev/null +++ b/src/ipa/libipa/algorithm.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * algorithm.h - ISP control algorithm interface + */ +#ifndef __LIBCAMERA_IPA_LIBIPA_ALGORITHM_H__ +#define __LIBCAMERA_IPA_LIBIPA_ALGORITHM_H__ + +namespace libcamera { + +namespace ipa { + +template +class Algorithm +{ +public: + virtual ~Algorithm() {} + + virtual int configure([[maybe_unused]] Context &context, + [[maybe_unused]] const IPAConfigInfo &configInfo) + { + return 0; + } + + virtual void prepare([[maybe_unused]] Context &context, + [[maybe_unused]] Params *params) + { + } + + virtual void process([[maybe_unused]] Context &context, + [[maybe_unused]] const Stats *stats) + { + } +}; + +} /* namespace ipa */ + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_IPA_LIBIPA_ALGORITHM_H__ */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index 4d073a03..161cc5a1 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 libipa_headers = files([ + 'algorithm.h', 'camera_sensor_helper.h', 'histogram.h' ]) diff --git a/src/ipa/rkisp1/algorithms/algorithm.h b/src/ipa/rkisp1/algorithms/algorithm.h new file mode 100644 index 00000000..dfa58727 --- /dev/null +++ b/src/ipa/rkisp1/algorithms/algorithm.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * algorithm.h - RkISP1 control algorithm interface + */ +#ifndef __LIBCAMERA_IPA_RKISP1_ALGORITHM_H__ +#define __LIBCAMERA_IPA_RKISP1_ALGORITHM_H__ + +#include + +#include + +#include + +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::rkisp1 { + +using Algorithm = libcamera::ipa::Algorithm; + +} /* namespace ipa::rkisp1 */ + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_IPA_RKISP1_ALGORITHM_H__ */ diff --git a/src/ipa/rkisp1/algorithms/meson.build b/src/ipa/rkisp1/algorithms/meson.build new file mode 100644 index 00000000..1c6c59cf --- /dev/null +++ b/src/ipa/rkisp1/algorithms/meson.build @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: CC0-1.0 + +rkisp1_ipa_algorithms = files([ +]) diff --git a/src/ipa/rkisp1/meson.build b/src/ipa/rkisp1/meson.build index 3683c922..8c822fbb 100644 --- a/src/ipa/rkisp1/meson.build +++ b/src/ipa/rkisp1/meson.build @@ -1,5 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 +subdir('algorithms') + ipa_name = 'ipa_rkisp1' rkisp1_ipa_sources = files([ @@ -7,6 +9,8 @@ rkisp1_ipa_sources = files([ 'rkisp1.cpp', ]) +rkisp1_ipa_sources += rkisp1_ipa_algorithms + mod = shared_module(ipa_name, [rkisp1_ipa_sources, libcamera_generated_ipa_headers], name_prefix : '', diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 34c3f9a2..0c54d8ec 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -25,7 +25,7 @@ #include -#include "ipa_context.h" +#include "algorithms/algorithm.h" #include "libipa/camera_sensor_helper.h" namespace libcamera { @@ -82,6 +82,9 @@ private: /* Local parameter storage */ struct IPAContext context_; + + /* Maintain the algorithms used by the IPA */ + std::list> algorithms_; }; int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) From patchwork Tue Nov 23 09:14:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14693 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2C7D3BDB13 for ; Tue, 23 Nov 2021 09:15:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C8611603C4; Tue, 23 Nov 2021 10:15:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="M34pn5Ns"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2FBBB60121 for ; Tue, 23 Nov 2021 10:14:58 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E4844A1B; Tue, 23 Nov 2021 10:14:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637658898; bh=tFVCTeHjMgZumUa9l1ICA2t2tlOhFc2vD6qmehqHlX8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=M34pn5Nsnf8ViFHN626qEOpliSkwow8znJHKUcdItfdEW16RqorJB+HqFj4MQng1y 1gHs2q0w+zYeqy+XkDjJGu0r0Iu/rWyr8TiK1K4MHyScf+9c47GoHmpHU6asjoCc8Z LtedErdfdzlwR8m8B17w1cBqejoNjAsK6aUeo4co= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 10:14:47 +0100 Message-Id: <20211123091451.67404-8-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 07/11] ipa: ipu3: Document AWB Algorithm overriden functions X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that Algorithm is defined in libipa, we miss documentation for the functions implemented in IPU3::Awb. Fix it. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/awb.cpp | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index c7bcb20e..85205333 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -193,6 +193,9 @@ Awb::Awb() Awb::~Awb() = default; +/** + * \brief Configure the AWB given a configInfo + */ int Awb::configure(IPAContext &context, [[maybe_unused]] const IPAConfigInfo &configInfo) { @@ -373,6 +376,9 @@ void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) } } +/** + * \brief Process IPU3 statistics, and run algorithm operations + */ void Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { calculateWBGains(stats); @@ -394,6 +400,9 @@ constexpr uint16_t Awb::threshold(float value) return value * 8191; } +/** + * \brief Fill the \a params buffer with IPU3 processing parameters for a frame + */ void Awb::prepare(IPAContext &context, ipu3_uapi_params *params) { /* From patchwork Tue Nov 23 09:14:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14692 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8B985C3250 for ; Tue, 23 Nov 2021 09:15:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2494A6043C; Tue, 23 Nov 2021 10:15:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fMVNELaw"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5CDF460398 for ; Tue, 23 Nov 2021 10:14:58 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1A37511BB; Tue, 23 Nov 2021 10:14:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637658898; bh=Esr2kiZNj6jLvz3SXVFZnyjOJAGouO/3oyLS6zLdVDE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fMVNELawZfxBJqS/PFogiJq8Ty/P8Yn6E/ICy35gn2uX7l2bWXwmOO6QOoFjBA4WC oFHnbEDrSNDgRpF4CxBb/QSuUskUMgvMhnsmnkB/NtbCChRRHIjlDCe9WB1VMmuGdm dZB6tG5oxqH4XStWl8thPa2fLjDZKuamSZRHp2M8= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 10:14:48 +0100 Message-Id: <20211123091451.67404-9-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 08/11] ipa: rkisp1: Report and use sensor controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The pipeline handler populates a new sensorControls ControlList, to have the effective exposure and gain values for the current frame. This is done when a statistics buffer is received. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/rkisp1.mojom | 2 ++ src/ipa/rkisp1/rkisp1.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 3 ++- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/include/libcamera/ipa/rkisp1.mojom b/include/libcamera/ipa/rkisp1.mojom index a6991d4f..c3a6d8e1 100644 --- a/include/libcamera/ipa/rkisp1.mojom +++ b/include/libcamera/ipa/rkisp1.mojom @@ -21,11 +21,13 @@ struct RkISP1Event { uint32 frame; uint32 bufferId; libcamera.ControlList controls; + libcamera.ControlList sensorControls; }; struct RkISP1Action { RkISP1Operations op; libcamera.ControlList controls; + libcamera.ControlList sensorControls; }; interface IPARkISP1Interface { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 0c54d8ec..b5aa93f8 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -312,7 +312,7 @@ void IPARkISP1::setControls(unsigned int frame) ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); - op.controls = ctrls; + op.sensorControls = ctrls; queueFrameAction.emit(frame, op); } diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 6cdc4c03..36ef6a02 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -333,7 +333,7 @@ void RkISP1CameraData::queueFrameAction(unsigned int frame, { switch (action.op) { case ipa::rkisp1::ActionV4L2Set: { - const ControlList &controls = action.controls; + const ControlList &controls = action.sensorControls; delayedCtrls_->push(controls); break; } @@ -1124,6 +1124,7 @@ void PipelineHandlerRkISP1::statReady(FrameBuffer *buffer) ev.op = ipa::rkisp1::EventSignalStatBuffer; ev.frame = info->frame; ev.bufferId = info->statBuffer->cookie(); + ev.sensorControls = data->delayedCtrls_->get(buffer->metadata().sequence); data->ipa_->processEvent(ev); } From patchwork Tue Nov 23 09:14:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14691 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0B317C324F for ; Tue, 23 Nov 2021 09:15:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A147860490; Tue, 23 Nov 2021 10:15:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="qkMjGiqB"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 84D576039C for ; Tue, 23 Nov 2021 10:14:58 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 48D15A1B; Tue, 23 Nov 2021 10:14:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637658898; bh=AfLesHaI81KKFBlXXjJo6bUvn4fCQHqFqc0tnw+fYQ8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qkMjGiqBGMF5+EfTDjepUL63Wz6ESoVn9SOQR0Q0aMNp+KfIA/5KaBAKnuvQo78a6 Qnu8X8hS5UgUeic03EXQIHQAk9Gp6Ua74uMQfP/50jDaGEz9ViusXKDAZjQrJ7QML1 2PwirvXO0LLM1SxoyZAqpSl+xHgiJ6RNorm4VBZU= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 10:14:49 +0100 Message-Id: <20211123091451.67404-10-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 09/11] ipa: rkisp1: Introduce AGC X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that we have IPAContext and Algorithm, we can implement a simple AGC based on the IPU3 one. It is very similar, except that there is no histogram used for an inter quantile mean. The RkISP1 is returning a 5x5 array (for V10) of luminance means. Estimating the relative luminance is thus a simple mean of all the blocks already calculated by the ISP. Signed-off-by: Jean-Michel Hautbois --- src/ipa/rkisp1/algorithms/agc.cpp | 265 ++++++++++++++++++++++++++ src/ipa/rkisp1/algorithms/agc.h | 55 ++++++ src/ipa/rkisp1/algorithms/meson.build | 1 + src/ipa/rkisp1/ipa_context.cpp | 44 +++++ src/ipa/rkisp1/ipa_context.h | 17 ++ src/ipa/rkisp1/rkisp1.cpp | 72 +++---- 6 files changed, 419 insertions(+), 35 deletions(-) create mode 100644 src/ipa/rkisp1/algorithms/agc.cpp create mode 100644 src/ipa/rkisp1/algorithms/agc.h diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp new file mode 100644 index 00000000..9c6d312e --- /dev/null +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -0,0 +1,265 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * agc.cpp - AGC/AEC mean-based control algorithm + */ + +#include "agc.h" + +#include +#include +#include + +#include + +#include + +/** + * \file agc.h + */ + +namespace libcamera { + +using namespace std::literals::chrono_literals; + +namespace ipa::rkisp1::algorithms { + +/** + * \class Agc + * \brief A mean-based auto-exposure algorithm + */ + +LOG_DEFINE_CATEGORY(RkISP1Agc) + +/* Limits for analogue gain values */ +static constexpr double kMinAnalogueGain = 1.0; +static constexpr double kMaxAnalogueGain = 8.0; + +/* \todo Honour the FrameDurationLimits control instead of hardcoding a limit */ +static constexpr utils::Duration kMaxShutterSpeed = 60ms; + +/* Number of frames to wait before calculating stats on minimum exposure */ +static constexpr uint32_t kNumStartupFrames = 10; + +/* + * Relative luminance target. + * + * It's a number that's chosen so that, when the camera points at a grey + * target, the resulting image brightness is considered right. + */ +static constexpr double kRelativeLuminanceTarget = 0.4; + +Agc::Agc() + : frameCount_(0), lineDuration_(0s), minShutterSpeed_(0s), + maxShutterSpeed_(0s), filteredExposure_(0s), currentExposure_(0s) +{ +} + +/** + * \brief Configure the AGC given a configInfo + * \param[in] context The shared IPA context + * \param[in] configInfo The IPA configuration data + * + * \return 0 + */ +int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) +{ + /* \todo use the IPAContext to provide the limits */ + lineDuration_ = configInfo.lineLength * 1.0s / configInfo.pixelRate; + + minShutterSpeed_ = context.configuration.agc.minShutterSpeed; + maxShutterSpeed_ = std::min(context.configuration.agc.maxShutterSpeed, + kMaxShutterSpeed); + + minAnalogueGain_ = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain); + maxAnalogueGain_ = std::min(context.configuration.agc.maxAnalogueGain, kMaxAnalogueGain); + + /* Configure the default exposure and gain. */ + context.frameContext.agc.gain = minAnalogueGain_; + context.frameContext.agc.exposure = 10ms / lineDuration_; + + return 0; +} + +/** + * \brief Apply a filter on the exposure value to limit the speed of changes + */ +void Agc::filterExposure() +{ + double speed = 0.2; + + /* Adapt instantly if we are in startup phase */ + if (frameCount_ < kNumStartupFrames) + speed = 1.0; + + if (filteredExposure_ == 0s) { + /* DG stands for digital gain.*/ + filteredExposure_ = currentExposure_; + } else { + /* + * If we are close to the desired result, go faster to avoid making + * multiple micro-adjustments. + * \todo Make this customisable? + */ + if (filteredExposure_ < 1.2 * currentExposure_ && + filteredExposure_ > 0.8 * currentExposure_) + speed = sqrt(speed); + + filteredExposure_ = speed * currentExposure_ + + filteredExposure_ * (1.0 - speed); + } + + LOG(RkISP1Agc, Debug) << "After filtering, total_exposure " << filteredExposure_; +} + +/** + * \brief Estimate the new exposure and gain values + * \param[inout] frameContext The shared IPA frame Context + * \param[in] yGain The gain calculated on the current brightness level + */ +void Agc::computeExposure(IPAFrameContext &frameContext, double yGain) +{ + /* Get the effective exposure and gain applied on the sensor. */ + uint32_t exposure = frameContext.sensor.exposure; + double analogueGain = frameContext.sensor.gain; + + /* Consider within 1% of the target as correctly exposed */ + if (std::abs(yGain - 1.0) < 0.01) + LOG(RkISP1Agc, Debug) << "We are well exposed (iqMean = " + << yGain << ")"; + + /* extracted from Rpi::Agc::computeTargetExposure */ + + /* Calculate the shutter time in seconds */ + utils::Duration currentShutter = exposure * lineDuration_; + + /* + * Update the exposure value for the next computation using the values + * of exposure and gain really used by the sensor. + */ + utils::Duration effectiveExposureValue = currentShutter * analogueGain; + + LOG(RkISP1Agc, Debug) << "Actual total exposure " << currentShutter * analogueGain + << " Shutter speed " << currentShutter + << " Gain " << analogueGain + << " Needed ev gain " << yGain; + + /* + * Calculate the current exposure value for the scene as the latest + * exposure value applied multiplied by the new estimated gain. + */ + currentExposure_ = effectiveExposureValue * yGain; + + /* Clamp the exposure value to the min and max authorized */ + utils::Duration maxTotalExposure = maxShutterSpeed_ * maxAnalogueGain_; + currentExposure_ = std::min(currentExposure_, maxTotalExposure); + LOG(RkISP1Agc, Debug) << "Target total exposure " << currentExposure_ + << ", maximum is " << maxTotalExposure; + + /* \todo: estimate if we need to desaturate */ + filterExposure(); + + /* Divide the exposure value as new exposure and gain values */ + utils::Duration exposureValue = filteredExposure_; + utils::Duration shutterTime; + + /* + * Push the shutter time up to the maximum first, and only then + * increase the gain. + */ + shutterTime = std::clamp(exposureValue / minAnalogueGain_, + minShutterSpeed_, maxShutterSpeed_); + double stepGain = std::clamp(exposureValue / shutterTime, + minAnalogueGain_, maxAnalogueGain_); + LOG(RkISP1Agc, Debug) << "Divided up shutter and gain are " + << shutterTime << " and " + << stepGain; + + /* Update the estimated exposure and gain. */ + frameContext.agc.exposure = shutterTime / lineDuration_; + frameContext.agc.gain = stepGain; +} + +/** + * \brief Estimate the relative luminance of the frame with a given gain + * \param[in] ae The RkISP1 statistics and ISP results + * \param[in] gain The gain calculated on the current brightness level + * \return The relative luminance + * + * This function estimates the average relative luminance of the frame that + * would be output by the sensor if an additional \a gain was applied. + * + * The estimation is based on the AE statistics for the current frame. Y + * averages for all cells are first multiplied by the gain, and then saturated + * to approximate the sensor behaviour at high brightness values. The + * approximation is quite rough, as it doesn't take into account non-linearities + * when approaching saturation. + * + * The values are normalized to the [0.0, 1.0] range, where 1.0 corresponds to a + * theoretical perfect reflector of 100% reference white. + * + * More detailed information can be found in: + * https://en.wikipedia.org/wiki/Relative_luminance + */ +double Agc::estimateLuminance(const rkisp1_cif_isp_ae_stat *ae, + double gain) +{ + double ySum = 0.0; + unsigned int num = 0; + + /* Sum the averages, saturated to 255. */ + for (unsigned int aeCell = 0; aeCell < numCells_; aeCell++) { + ySum += std::min(ae->exp_mean[aeCell] * gain, 255.0); + num++; + } + + /* \todo Weight with the AWB gains */ + + return ySum / num / 255; +} + +/** + * \brief Process RkISP1 statistics, and run AGC operations + * \param[in] context The shared IPA context + * \param[in] stats The RKIsp1 statistics and ISP results + * + * Identify the current image brightness, and use that to estimate the optimal + * new exposure and gain for the scene. + */ +void Agc::process(IPAContext &context, const rkisp1_stat_buffer *stats) +{ + const rkisp1_cif_isp_stat *params = &stats->params; + ASSERT(stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP); + + const rkisp1_cif_isp_ae_stat *ae = ¶ms->ae; + + /* + * Estimate the gain needed to achieve a relative luminance target. To + * account for non-linearity caused by saturation, the value needs to be + * estimated in an iterative process, as multiplying by a gain will not + * increase the relative luminance by the same factor if some image + * regions are saturated. + */ + double yGain = 1.0; + double yTarget = kRelativeLuminanceTarget; + + for (unsigned int i = 0; i < 8; i++) { + double yValue = estimateLuminance(ae, yGain); + double extra_gain = std::min(10.0, yTarget / (yValue + .001)); + + yGain *= extra_gain; + LOG(RkISP1Agc, Debug) << "Y value: " << yValue + << ", Y target: " << yTarget + << ", gives gain " << yGain; + if (extra_gain < 1.01) + break; + } + + computeExposure(context.frameContext, yGain); + frameCount_++; +} + +} /* namespace ipa::rkisp1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h new file mode 100644 index 00000000..83159743 --- /dev/null +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * agc.h - RkISP1 AGC/AEC mean-based control algorithm + */ +#ifndef __LIBCAMERA_RKISP1_ALGORITHMS_AGC_H__ +#define __LIBCAMERA_RKISP1_ALGORITHMS_AGC_H__ + +#include + +#include + +#include + +#include "algorithm.h" + +namespace libcamera { + +struct IPACameraSensorInfo; + +namespace ipa::rkisp1::algorithms { + +class Agc : public Algorithm +{ +public: + Agc(); + ~Agc() = default; + + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void process(IPAContext &context, const rkisp1_stat_buffer *stats) override; + +private: + void filterExposure(); + void computeExposure(IPAFrameContext &frameContext, double yGain); + double estimateLuminance(const rkisp1_cif_isp_ae_stat *ae, double gain); + + uint64_t frameCount_; + + utils::Duration lineDuration_; + utils::Duration minShutterSpeed_; + utils::Duration maxShutterSpeed_; + + double minAnalogueGain_; + double maxAnalogueGain_; + + utils::Duration filteredExposure_; + utils::Duration currentExposure_; +}; + +} /* namespace ipa::rkisp1::algorithms */ + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_RKISP1_ALGORITHMS_AGC_H__ */ diff --git a/src/ipa/rkisp1/algorithms/meson.build b/src/ipa/rkisp1/algorithms/meson.build index 1c6c59cf..a19c1a4f 100644 --- a/src/ipa/rkisp1/algorithms/meson.build +++ b/src/ipa/rkisp1/algorithms/meson.build @@ -1,4 +1,5 @@ # SPDX-License-Identifier: CC0-1.0 rkisp1_ipa_algorithms = files([ + 'agc.cpp', ]) diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 819b2c73..16fc248f 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -55,4 +55,48 @@ namespace libcamera::ipa::rkisp1 { * are run. This needs to be turned into real per-frame data storage. */ +/** + * \var IPASessionConfiguration::agc + * \brief AGC parameters configuration of the IPA + * + * \var IPASessionConfiguration::agc.minShutterSpeed + * \brief Minimum shutter speed supported with the configured sensor + * + * \var IPASessionConfiguration::agc.maxShutterSpeed + * \brief Maximum shutter speed supported with the configured sensor + * + * \var IPASessionConfiguration::agc.minAnalogueGain + * \brief Minimum analogue gain supported with the configured sensor + * + * \var IPASessionConfiguration::agc.maxAnalogueGain + * \brief Maximum analogue gain supported with the configured sensor + */ + +/** + * \var IPAFrameContext::agc + * \brief Context for the Automatic Gain Control algorithm + * + * The exposure and gain determined are expected to be applied to the sensor + * at the earliest opportunity. + * + * \var IPAFrameContext::agc.exposure + * \brief Exposure time expressed as a number of lines + * + * \var IPAFrameContext::agc.gain + * \brief Analogue gain multiplier + * + * The gain should be adapted to the sensor specific gain code before applying. + */ + +/** + * \var IPAFrameContext::sensor + * \brief Effective sensor values + * + * \var IPAFrameContext::sensor.exposure + * \brief Exposure time expressed as a number of lines + * + * \var IPAFrameContext::sensor.gain + * \brief Analogue gain multiplier + */ + } /* namespace libcamera::ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index ff40efe3..8e756fb1 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -8,14 +8,31 @@ #ifndef __LIBCAMERA_RKISP1_IPA_CONTEXT_H__ #define __LIBCAMERA_RKISP1_IPA_CONTEXT_H__ +#include + namespace libcamera { namespace ipa::rkisp1 { struct IPASessionConfiguration { + struct { + utils::Duration minShutterSpeed; + utils::Duration maxShutterSpeed; + double minAnalogueGain; + double maxAnalogueGain; + } agc; }; struct IPAFrameContext { + struct { + uint32_t exposure; + double gain; + } agc; + + struct { + uint32_t exposure; + double gain; + } sensor; }; struct IPAContext { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index b5aa93f8..89d98b6c 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -25,6 +25,7 @@ #include +#include "algorithms/agc.h" #include "algorithms/algorithm.h" #include "libipa/camera_sensor_helper.h" @@ -32,6 +33,8 @@ namespace libcamera { LOG_DEFINE_CATEGORY(IPARkISP1) +using namespace std::literals::chrono_literals; + namespace ipa::rkisp1 { class IPARkISP1 : public IPARkISP1Interface @@ -77,6 +80,8 @@ private: unsigned int hwGammaOutMaxSamples_; unsigned int hwHistogramWeightGridsSize_; + utils::Duration lineDuration_; + /* Interface to the Camera Helper */ std::unique_ptr camHelper_; @@ -120,6 +125,9 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) return -ENODEV; } + /* Construct our Algorithms */ + algorithms_.push_back(std::make_unique()); + return 0; } @@ -171,9 +179,29 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, << "Exposure: " << minExposure_ << "-" << maxExposure_ << " Gain: " << minGain_ << "-" << maxGain_; + lineDuration_ = info.lineLength * 1.0s / info.pixelRate; + /* Clean context at configuration */ context_ = {}; + /* + * When the AGC computes the new exposure values for a frame, it needs + * to know the limits for shutter speed and analogue gain. + * As it depends on the sensor, update it with the controls. + * + * \todo take VBLANK into account for maximum shutter speed + */ + context_.configuration.agc.minShutterSpeed = minExposure_ * lineDuration_; + context_.configuration.agc.maxShutterSpeed = maxExposure_ * lineDuration_; + context_.configuration.agc.minAnalogueGain = camHelper_->gain(minGain_); + context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain_); + + for (auto const &algo : algorithms_) { + int ret = algo->configure(context_, info); + if (ret) + return ret; + } + return 0; } @@ -218,6 +246,9 @@ void IPARkISP1::processEvent(const RkISP1Event &event) reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); + context_.frameContext.sensor.exposure = event.sensorControls.get(V4L2_CID_EXPOSURE).get(); + context_.frameContext.sensor.gain = camHelper_->gain(event.sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + updateStatistics(frame, stats); break; } @@ -262,44 +293,12 @@ void IPARkISP1::queueRequest(unsigned int frame, rkisp1_params_cfg *params, void IPARkISP1::updateStatistics(unsigned int frame, const rkisp1_stat_buffer *stats) { - const rkisp1_cif_isp_stat *params = &stats->params; unsigned int aeState = 0; - if (stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP) { - const rkisp1_cif_isp_ae_stat *ae = ¶ms->ae; - - const unsigned int target = 60; + for (auto const &algo : algorithms_) + algo->process(context_, stats); - unsigned int value = 0; - unsigned int num = 0; - for (unsigned int i = 0; i < hwAeMeanMax_; i++) { - if (ae->exp_mean[i] <= 15) - continue; - - value += ae->exp_mean[i]; - num++; - } - value /= num; - - double factor = (double)target / value; - - if (frame % 3 == 0) { - double exposure; - - exposure = factor * exposure_ * gain_ / minGain_; - exposure_ = std::clamp((uint64_t)exposure, - minExposure_, - maxExposure_); - - exposure = exposure / exposure_ * minGain_; - gain_ = std::clamp((uint64_t)exposure, - minGain_, maxGain_); - - setControls(frame + 1); - } - - aeState = fabs(factor - 1.0f) < 0.05f ? 2 : 1; - } + setControls(frame + 1); metadataReady(frame, aeState); } @@ -309,6 +308,9 @@ void IPARkISP1::setControls(unsigned int frame) RkISP1Action op; op.op = ActionV4L2Set; + exposure_ = context_.frameContext.agc.exposure; + gain_ = camHelper_->gainCode(context_.frameContext.agc.gain); + ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); From patchwork Tue Nov 23 09:14:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14694 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 439CBC3251 for ; Tue, 23 Nov 2021 09:15:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7D5886044F; Tue, 23 Nov 2021 10:15:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="TGRj7Web"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B5B58603BE for ; Tue, 23 Nov 2021 10:14:58 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 756E6F95; Tue, 23 Nov 2021 10:14:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637658898; bh=bT+N3UBi9S+MSpCr9mmMbO2ADxWvOkhfGHF2KLhjiIQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TGRj7WebaPfZZXjxQk0HkWb9O5m2Y91obvBRQUlwyo1sNdQLbgo5xe0pgpzk/MAX1 xFeGWC2W6rsWDSPEBS0jEZJTNzqGsOrgxFUUKQU8RfNk6Y9aiXgdmQXXCeYgusLn/e Z4bDqka36G35V1cSam7w/IAk8yLiu4kKaX8PJEYs= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 10:14:50 +0100 Message-Id: <20211123091451.67404-11-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 10/11] ipa: rkisp1: agc: Introduce prepare call X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a new parameter buffer needs to be queued, we need to specify which algorithm is activated or not in the ISP. Add a simple prepare function in AGC for that, which may later evolve to take the exposure locking into account. For that function to be called, we also need to add the loop on the algorithms in IPARkISP1::queueRequest. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/algorithms/agc.cpp | 7 +++++++ src/ipa/rkisp1/algorithms/agc.h | 1 + src/ipa/rkisp1/rkisp1.cpp | 12 +++--------- 3 files changed, 11 insertions(+), 9 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 9c6d312e..0433813a 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -260,6 +260,13 @@ void Agc::process(IPAContext &context, const rkisp1_stat_buffer *stats) frameCount_++; } +void Agc::prepare([[maybe_unused]] IPAContext &context, + rkisp1_params_cfg *params) +{ + params->module_ens |= RKISP1_CIF_ISP_MODULE_AEC; + params->module_en_update |= RKISP1_CIF_ISP_MODULE_AEC; +} + } /* namespace ipa::rkisp1::algorithms */ } /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index 83159743..ac95dea5 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -28,6 +28,7 @@ public: ~Agc() = default; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void prepare(IPAContext &context, rkisp1_params_cfg *params) override; void process(IPAContext &context, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 89d98b6c..59e868ff 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -270,19 +270,13 @@ void IPARkISP1::processEvent(const RkISP1Event &event) } void IPARkISP1::queueRequest(unsigned int frame, rkisp1_params_cfg *params, - const ControlList &controls) + [[maybe_unused]] const ControlList &controls) { /* Prepare parameters buffer. */ memset(params, 0, sizeof(*params)); - /* Auto Exposure on/off. */ - if (controls.contains(controls::AeEnable)) { - autoExposure_ = controls.get(controls::AeEnable); - if (autoExposure_) - params->module_ens = RKISP1_CIF_ISP_MODULE_AEC; - - params->module_en_update = RKISP1_CIF_ISP_MODULE_AEC; - } + for (auto const &algo : algorithms_) + algo->prepare(context_, params); RkISP1Action op; op.op = ActionParamFilled; From patchwork Tue Nov 23 09:14:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14695 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ECAD4C3252 for ; Tue, 23 Nov 2021 09:15:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 37B47604F9; Tue, 23 Nov 2021 10:15:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XHmjOcT8"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 03A4A6033C for ; Tue, 23 Nov 2021 10:14:59 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:3c3b:9149:b:8aa9]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A1A07A1B; Tue, 23 Nov 2021 10:14:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637658898; bh=z+WpoiU7f87GQk8JuHzkZYHoyFljaFQAVxkZNfwQDGo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XHmjOcT8A0NlaG8Gy75WccK2n/GyPApfZ6W5B5o9aC9lU9vC4v9xLdRuUMscgWEBt ggdIq5MU4qZsPJi/06qaSlaZhAtoeiv8CnnHCal98LmGYxsgy9Br8Fw3SIX7REtP/L We1drFJVRHGvg7egnRLVaZlXo9GlX7z/JGV/2N3o= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Tue, 23 Nov 2021 10:14:51 +0100 Message-Id: <20211123091451.67404-12-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> References: <20211123091451.67404-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 11/11] ipa: rkisp1: agc: Configure the number of cells used by HW X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ISP can use 25 or 81 cells depending on its revision. Remove the cached value in IPARkISP1 and use IPASessionConfiguration to store it and pass it to AGC. Signed-off-by: Jean-Michel Hautbois --- src/ipa/rkisp1/algorithms/agc.cpp | 16 +++++++++++++++- src/ipa/rkisp1/algorithms/agc.h | 2 ++ src/ipa/rkisp1/ipa_context.cpp | 8 ++++++++ src/ipa/rkisp1/ipa_context.h | 4 ++++ src/ipa/rkisp1/rkisp1.cpp | 10 +++++++--- 5 files changed, 36 insertions(+), 4 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 0433813a..e5358ca3 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -61,7 +61,7 @@ Agc::Agc() * \param[in] context The shared IPA context * \param[in] configInfo The IPA configuration data * - * \return 0 + * \return 0 on success or a negative error code otherwise */ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { @@ -79,6 +79,20 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) context.frameContext.agc.gain = minAnalogueGain_; context.frameContext.agc.exposure = 10ms / lineDuration_; + switch (context.configuration.hw.revision) { + case RKISP1_V10: + numCells_ = RKISP1_CIF_ISP_AE_MEAN_MAX_V10; + break; + case RKISP1_V12: + numCells_ = RKISP1_CIF_ISP_AE_MEAN_MAX_V12; + break; + default: + LOG(RkISP1Agc, Error) + << "Hardware revision " << context.configuration.hw.revision + << " is currently not supported"; + return -ENODEV; + } + return 0; } diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index ac95dea5..3ab3f347 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -45,6 +45,8 @@ private: double minAnalogueGain_; double maxAnalogueGain_; + uint32_t numCells_; + utils::Duration filteredExposure_; utils::Duration currentExposure_; }; diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 16fc248f..cdb952be 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -72,6 +72,14 @@ namespace libcamera::ipa::rkisp1 { * \brief Maximum analogue gain supported with the configured sensor */ +/** + * \var IPASessionConfiguration::hw + * \brief RkISP1 specific hardware information + * + * \var IPASessionConfiguration::hw.revision + * \brief RkISP1 revision reported + */ + /** * \var IPAFrameContext::agc * \brief Context for the Automatic Gain Control algorithm diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 8e756fb1..316036c6 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -21,6 +21,10 @@ struct IPASessionConfiguration { double minAnalogueGain; double maxAnalogueGain; } agc; + + struct { + uint32_t revision; + } hw; }; struct IPAFrameContext { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 59e868ff..a472d111 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -75,7 +75,7 @@ private: uint32_t maxGain_; /* revision-specific data */ - unsigned int hwAeMeanMax_; + unsigned int hwRevision_; unsigned int hwHistBinNMax_; unsigned int hwGammaOutMaxSamples_; unsigned int hwHistogramWeightGridsSize_; @@ -97,13 +97,11 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) /* \todo Add support for other revisions */ switch (hwRevision) { case RKISP1_V10: - hwAeMeanMax_ = RKISP1_CIF_ISP_AE_MEAN_MAX_V10; hwHistBinNMax_ = RKISP1_CIF_ISP_HIST_BIN_N_MAX_V10; hwGammaOutMaxSamples_ = RKISP1_CIF_ISP_GAMMA_OUT_MAX_SAMPLES_V10; hwHistogramWeightGridsSize_ = RKISP1_CIF_ISP_HISTOGRAM_WEIGHT_GRIDS_SIZE_V10; break; case RKISP1_V12: - hwAeMeanMax_ = RKISP1_CIF_ISP_AE_MEAN_MAX_V12; hwHistBinNMax_ = RKISP1_CIF_ISP_HIST_BIN_N_MAX_V12; hwGammaOutMaxSamples_ = RKISP1_CIF_ISP_GAMMA_OUT_MAX_SAMPLES_V12; hwHistogramWeightGridsSize_ = RKISP1_CIF_ISP_HISTOGRAM_WEIGHT_GRIDS_SIZE_V12; @@ -117,6 +115,9 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) LOG(IPARkISP1, Debug) << "Hardware revision is " << hwRevision; + /* Cache the value to set it in configure. */ + hwRevision_ = hwRevision; + camHelper_ = CameraSensorHelperFactory::create(settings.sensorModel); if (!camHelper_) { LOG(IPARkISP1, Error) @@ -184,6 +185,9 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, /* Clean context at configuration */ context_ = {}; + /* Set the hardware revision for the algorithms. */ + context_.configuration.hw.revision = hwRevision_; + /* * When the AGC computes the new exposure values for a frame, it needs * to know the limits for shutter speed and analogue gain.