From patchwork Thu Nov 25 05:42:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14762 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C287EBF415 for ; Thu, 25 Nov 2021 05:43:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3E4586042A; Thu, 25 Nov 2021 06:43:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="isa6ijjh"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3A2E160228 for ; Thu, 25 Nov 2021 06:43:04 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:d9a5:5e40:3323:d95]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C52C3A24; Thu, 25 Nov 2021 06:43:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637818983; bh=6qkF2Tg7yCai9Xl8kh9FitL+Os2hwaUjtO8oJRvJdwQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=isa6ijjhd7rmYsvMWXtprfRQtx1dnxXrOSjbS/mR8rqYEDr5wmb6dXhY8g/UkUxGD 5AmngwQ9XQeS3iqg7Hn82J8Rw+FVLOo0vX1kCVlbgEP4kW7NY6bBNdf05rZC7rTo6y b5/Y0OOotvun2RCaKgv28OYE5W2TVSZCcXD8/n4A= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 25 Nov 2021 06:42:49 +0100 Message-Id: <20211125054259.24792-2-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> References: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 01/11] libipa: Correct IMX219 in CameraSensorHelper X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The equation is badly reported in the CameraSensorHelper, as m1 and c0 are inverted. Correct it to have a proper gain calculation. Signed-off-by: Jean-Michel Hautbois Acked-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/libipa/camera_sensor_helper.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/libipa/camera_sensor_helper.cpp b/src/ipa/libipa/camera_sensor_helper.cpp index 0b0eb503..5d052612 100644 --- a/src/ipa/libipa/camera_sensor_helper.cpp +++ b/src/ipa/libipa/camera_sensor_helper.cpp @@ -290,7 +290,7 @@ class CameraSensorHelperImx219 : public CameraSensorHelper public: CameraSensorHelperImx219() { - analogueGainConstants_ = { AnalogueGainLinear, 0, -1, 256, 256 }; + analogueGainConstants_ = { AnalogueGainLinear, 0, 256, -1, 256 }; } }; REGISTER_CAMERA_SENSOR_HELPER("imx219", CameraSensorHelperImx219) From patchwork Thu Nov 25 05:42:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14763 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1171CBF415 for ; Thu, 25 Nov 2021 05:43:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 929AD6043C; Thu, 25 Nov 2021 06:43:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bvqxsdhi"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5974A60231 for ; Thu, 25 Nov 2021 06:43:04 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:d9a5:5e40:3323:d95]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 013901010; Thu, 25 Nov 2021 06:43:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637818984; bh=YehKf1iayUezpPnKMAd/oOrQ48+itY7/+GNTbzUaglE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bvqxsdhihwqP81ntlblMo+GyckTkRCBG0lbzKBP5aUzyV6l9KItC2f9EMqTKnRyK2 S1m3Axf6tzykaGzfCEiQv9EkZ2ajlM3BEYoGESxxp93b+bWMvbMdHnlIIvHQyKyVgH hVch0kX9ROZ5KFFpfMjeYo4GiGO09b/JNJOv76fc= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 25 Nov 2021 06:42:50 +0100 Message-Id: <20211125054259.24792-3-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> References: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 02/11] ipa: rkisp1: Pass IPASettings at init call X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the IPA is initialized, it will need to know the sensor model used in order to properly call CameraSensorHelper for the analogue gain. Modify the init definition in the pipeline handler and in the IPA as well as the mojo interface to pass the IPASettings. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- include/libcamera/ipa/rkisp1.mojom | 4 +++- src/ipa/rkisp1/rkisp1.cpp | 5 +++-- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- 3 files changed, 7 insertions(+), 4 deletions(-) diff --git a/include/libcamera/ipa/rkisp1.mojom b/include/libcamera/ipa/rkisp1.mojom index cae757ea..a6991d4f 100644 --- a/include/libcamera/ipa/rkisp1.mojom +++ b/include/libcamera/ipa/rkisp1.mojom @@ -29,7 +29,9 @@ struct RkISP1Action { }; interface IPARkISP1Interface { - init(uint32 hwRevision) => (int32 ret); + init(libcamera.IPASettings settings, + uint32 hwRevision) + => (int32 ret); start() => (int32 ret); stop(); diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index bf2c13b6..7ecbf8ae 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -34,7 +34,7 @@ namespace ipa::rkisp1 { class IPARkISP1 : public IPARkISP1Interface { public: - int init(unsigned int hwRevision) override; + int init(const IPASettings &settings, unsigned int hwRevision) override; int start() override; void stop() override {} @@ -75,7 +75,8 @@ private: unsigned int hwHistogramWeightGridsSize_; }; -int IPARkISP1::init(unsigned int hwRevision) +int IPARkISP1::init([[maybe_unused]] const IPASettings &settings, + unsigned int hwRevision) { /* \todo Add support for other revisions */ switch (hwRevision) { diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 98008862..6cdc4c03 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -319,7 +319,7 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision) ipa_->queueFrameAction.connect(this, &RkISP1CameraData::queueFrameAction); - int ret = ipa_->init(hwRevision); + int ret = ipa_->init(IPASettings{ "", sensor_->model() }, hwRevision); if (ret < 0) { LOG(RkISP1, Error) << "IPA initialization failure"; return ret; From patchwork Thu Nov 25 05:42:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14764 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3AC17BF415 for ; Thu, 25 Nov 2021 05:43:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 92D4660394; Thu, 25 Nov 2021 06:43:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DyLEn4KP"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7548C60233 for ; Thu, 25 Nov 2021 06:43:04 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:d9a5:5e40:3323:d95]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 30C9A90E; Thu, 25 Nov 2021 06:43:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637818984; bh=6pybVIYFXVd1SgQSEoyDZDyd/ijoQp2Fy/E09A9+tuw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DyLEn4KPAw+EKuZGxzLiMQXlsRktXdtRP/nLk7y7bh6YpMYWazXFNgvOA/86H2a4r i+J53CVxcP/e7ApYsCIsrHBlY+6yOm/Gt3u4i9my6pEmj5FVfM5mdKjkBQm78d8IcF mK28ufWKNrmhseGSmDzJS1w4FTa9LaHvxzEqMcNU= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 25 Nov 2021 06:42:51 +0100 Message-Id: <20211125054259.24792-4-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> References: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 03/11] ipa: Do not modify the sensor limits X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The driver is responsible for setting the proper limits for its controls. The IMX219 has an analogue gain of 1.0 when the gain code is set to 0, therefore we can not clamp to a minimum gain code of 1. Rework this for both IPU3 and RkISP1, for both Exposure and Gain controls. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipu3.cpp | 4 ++-- src/ipa/rkisp1/rkisp1.cpp | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index a8d54a5d..b0c75541 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -444,11 +444,11 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo, return -EINVAL; } - minExposure_ = std::max(itExp->second.min().get(), 1); + minExposure_ = itExp->second.min().get(); maxExposure_ = itExp->second.max().get(); exposure_ = minExposure_; - minGain_ = std::max(itGain->second.min().get(), 1); + minGain_ = itGain->second.min().get(); maxGain_ = itGain->second.max().get(); gain_ = minGain_; diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 7ecbf8ae..910ad952 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -139,11 +139,11 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, autoExposure_ = true; - minExposure_ = std::max(itExp->second.min().get(), 1); + minExposure_ = itExp->second.min().get(); maxExposure_ = itExp->second.max().get(); exposure_ = minExposure_; - minGain_ = std::max(itGain->second.min().get(), 1); + minGain_ = itGain->second.min().get(); maxGain_ = itGain->second.max().get(); gain_ = minGain_; From patchwork Thu Nov 25 05:42:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14765 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1338EC324F for ; Thu, 25 Nov 2021 05:43:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9601A60398; Thu, 25 Nov 2021 06:43:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ETeXdCKD"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A039360371 for ; Thu, 25 Nov 2021 06:43:04 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:d9a5:5e40:3323:d95]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5DC32A24; Thu, 25 Nov 2021 06:43:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637818984; bh=7EKJLxN6FyygVP3ODjD71RglMl4Afhijn0ZyokJr6KI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ETeXdCKD0rh57e3xu282wPxF6mL/9KihGkNUhz9tWhHjWjAe3o47Tff5SZ80cdDzk CWh0Ws2n4Qg6e5oEyFi7cb71ISCS8SWfr2bhOK2PB9F2eu6uycW60L7dDRa8bfBl+T 6BoSruH5QHr/EuRLWncmQ5YIRQakVFzVjm7gOIAw= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 25 Nov 2021 06:42:52 +0100 Message-Id: <20211125054259.24792-5-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> References: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 04/11] ipa: rkisp1: Instantiate CameraSensorHelper X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the AGC will run, it will use analogue gains as double values. We will need those values to be converted to apply the control. Introduce CameraSensorHelper and call it at init(). Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/rkisp1.cpp | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 910ad952..73a7f582 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -25,6 +25,8 @@ #include +#include "libipa/camera_sensor_helper.h" + namespace libcamera { LOG_DEFINE_CATEGORY(IPARkISP1) @@ -73,10 +75,12 @@ private: unsigned int hwHistBinNMax_; unsigned int hwGammaOutMaxSamples_; unsigned int hwHistogramWeightGridsSize_; + + /* Interface to the Camera Helper */ + std::unique_ptr camHelper_; }; -int IPARkISP1::init([[maybe_unused]] const IPASettings &settings, - unsigned int hwRevision) +int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) { /* \todo Add support for other revisions */ switch (hwRevision) { @@ -100,6 +104,15 @@ int IPARkISP1::init([[maybe_unused]] const IPASettings &settings, } LOG(IPARkISP1, Debug) << "Hardware revision is " << hwRevision; + + camHelper_ = CameraSensorHelperFactory::create(settings.sensorModel); + if (!camHelper_) { + LOG(IPARkISP1, Error) + << "Failed to create camera sensor helper for " + << settings.sensorModel; + return -ENODEV; + } + return 0; } From patchwork Thu Nov 25 05:42:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14766 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BD4CCBF415 for ; Thu, 25 Nov 2021 05:43:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8553A603D9; Thu, 25 Nov 2021 06:43:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pS5F5FFD"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D1FCD6011E for ; Thu, 25 Nov 2021 06:43:04 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:d9a5:5e40:3323:d95]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8E31690E; Thu, 25 Nov 2021 06:43:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637818984; bh=/AillO5/wmQaq9rVhDawE0pFgfQm1NqVClMmCk2E5ig=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pS5F5FFDZDSCHDKzihE/O0KRwwmUsmr/I/aKG5dK4oMPAKmIq9koXklThtq7N3kmx SJtZ0HEnTZ5bg1Hz/uCRFedUJxic56Kse5BgiHNa6ih1oeBS8UWX89u1UTw50yxugy i0XIbIVln+L/LNms5HG+SqDP5zp7WjSr2NtvN4Xo= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 25 Nov 2021 06:42:53 +0100 Message-Id: <20211125054259.24792-6-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> References: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 05/11] ipa: rkisp1: Introduce IPAContext X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Before using any algorithm, we want the IPAContext to be ready for those. Introduce the IPAContext following the existing design from IPA::IPU3. Each algorithm will then introduce the needed fields. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- v4: use #pragma once --- src/ipa/rkisp1/ipa_context.cpp | 58 ++++++++++++++++++++++++++++++++++ src/ipa/rkisp1/ipa_context.h | 28 ++++++++++++++++ src/ipa/rkisp1/meson.build | 7 +++- src/ipa/rkisp1/rkisp1.cpp | 7 ++++ 4 files changed, 99 insertions(+), 1 deletion(-) create mode 100644 src/ipa/rkisp1/ipa_context.cpp create mode 100644 src/ipa/rkisp1/ipa_context.h diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp new file mode 100644 index 00000000..819b2c73 --- /dev/null +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * ipa_context.cpp - RkISP1 IPA Context + */ + +#include "ipa_context.h" + +/** + * \file ipa_context.h + * \brief Context and state information shared between the algorithms + */ + +namespace libcamera::ipa::rkisp1 { + +/** + * \struct IPASessionConfiguration + * \brief Session configuration for the IPA module + * + * The session configuration contains all IPA configuration parameters that + * remain constant during the capture session, from IPA module start to stop. + * It is typically set during the configure() operation of the IPA module, but + * may also be updated in the start() operation. + */ + +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + * + * The frame context stores data specific to a single frame processed by the + * IPA. Each frame processed by the IPA has a context associated with it, + * accessible through the IPAContext structure. + * + * \todo Detail how to access contexts for a particular frame + * + * Each of the fields in the frame context belongs to either a specific + * algorithm, or to the top-level IPA module. A field may be read by any + * algorithm, but should only be written by its owner. + */ + +/** + * \struct IPAContext + * \brief Global IPA context data shared between all algorithms + * + * \var IPAContext::configuration + * \brief The IPA session configuration, immutable during the session + * + * \var IPAContext::frameContext + * \brief The frame context for the frame being processed + * + * \todo While the frame context is supposed to be per-frame, this + * single frame context stores data related to both the current frame + * and the previous frames, with fields being updated as the algorithms + * are run. This needs to be turned into real per-frame data storage. + */ + +} /* namespace libcamera::ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h new file mode 100644 index 00000000..919139dd --- /dev/null +++ b/src/ipa/rkisp1/ipa_context.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * ipa_context.h - RkISP1 IPA Context + * + */ + +#pragma once + +namespace libcamera { + +namespace ipa::rkisp1 { + +struct IPASessionConfiguration { +}; + +struct IPAFrameContext { +}; + +struct IPAContext { + IPASessionConfiguration configuration; + IPAFrameContext frameContext; +}; + +} /* namespace ipa::rkisp1 */ + +} /* namespace libcamera*/ diff --git a/src/ipa/rkisp1/meson.build b/src/ipa/rkisp1/meson.build index f76b37f5..3683c922 100644 --- a/src/ipa/rkisp1/meson.build +++ b/src/ipa/rkisp1/meson.build @@ -2,8 +2,13 @@ ipa_name = 'ipa_rkisp1' +rkisp1_ipa_sources = files([ + 'ipa_context.cpp', + 'rkisp1.cpp', +]) + mod = shared_module(ipa_name, - ['rkisp1.cpp', libcamera_generated_ipa_headers], + [rkisp1_ipa_sources, libcamera_generated_ipa_headers], name_prefix : '', include_directories : [ipa_includes, libipa_includes], dependencies : libcamera_private, diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 73a7f582..34c3f9a2 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -25,6 +25,7 @@ #include +#include "ipa_context.h" #include "libipa/camera_sensor_helper.h" namespace libcamera { @@ -78,6 +79,9 @@ private: /* Interface to the Camera Helper */ std::unique_ptr camHelper_; + + /* Local parameter storage */ + struct IPAContext context_; }; int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) @@ -164,6 +168,9 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, << "Exposure: " << minExposure_ << "-" << maxExposure_ << " Gain: " << minGain_ << "-" << maxGain_; + /* Clean context at configuration */ + context_ = {}; + return 0; } From patchwork Thu Nov 25 05:42:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14767 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B060AC3250 for ; Thu, 25 Nov 2021 05:43:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8BA1360490; Thu, 25 Nov 2021 06:43:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pwdquUvL"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 08E656038A for ; Thu, 25 Nov 2021 06:43:05 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:d9a5:5e40:3323:d95]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BBE6D1010; Thu, 25 Nov 2021 06:43:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637818984; bh=nUNJZu16m8uatmAMoR5/WTY+cV6rJSZG9O0vYntEQSI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pwdquUvLZToKcIBUgBZw6NfLrPK/UpAirqUmBFog8s03zO2qCTRK+uVH4fLjtB63r KMaILouZHJWP6QiqvZeDnD9N9P5q3BMWpXcoLdxfO01DO95sBGUl/WabCbOfexKocz FFw7W46kr7XuI73iKiKAdtFsOvBDGxQYeThjkjTI= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 25 Nov 2021 06:42:54 +0100 Message-Id: <20211125054259.24792-7-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> References: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 06/11] ipa: libipa: Introduce Algorithm class template X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The algorithms are using the same function names with specialized parameters. Instead of duplicating code, introduce a libipa Algorithm class which implements a base class with template parameters in libipa, and use it in each IPA. As we now won't need an algorithm class for each IPA, move the documentation to libipa, and make it agnostic of the IPA used. While at it, fix the IPU3::Algorithm::Awb documentation. Signed-off-by: Laurent Pinchart Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain --- v4: use #pragma once --- src/ipa/ipu3/algorithms/algorithm.h | 12 ++---- src/ipa/ipu3/algorithms/awb.cpp | 9 +++++ src/ipa/ipu3/algorithms/meson.build | 1 - .../{ipu3/algorithms => libipa}/algorithm.cpp | 38 ++++++++----------- src/ipa/libipa/algorithm.h | 38 +++++++++++++++++++ src/ipa/libipa/meson.build | 1 + 6 files changed, 67 insertions(+), 32 deletions(-) rename src/ipa/{ipu3/algorithms => libipa}/algorithm.cpp (75%) create mode 100644 src/ipa/libipa/algorithm.h diff --git a/src/ipa/ipu3/algorithms/algorithm.h b/src/ipa/ipu3/algorithms/algorithm.h index 16310ab1..d2eecc78 100644 --- a/src/ipa/ipu3/algorithms/algorithm.h +++ b/src/ipa/ipu3/algorithms/algorithm.h @@ -9,21 +9,15 @@ #include +#include + #include "ipa_context.h" namespace libcamera { namespace ipa::ipu3 { -class Algorithm -{ -public: - virtual ~Algorithm() {} - - virtual int configure(IPAContext &context, const IPAConfigInfo &configInfo); - virtual void prepare(IPAContext &context, ipu3_uapi_params *params); - virtual void process(IPAContext &context, const ipu3_uapi_stats_3a *stats); -}; +using Algorithm = libcamera::ipa::Algorithm; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index c7bcb20e..1dc27fc9 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -193,6 +193,9 @@ Awb::Awb() Awb::~Awb() = default; +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ int Awb::configure(IPAContext &context, [[maybe_unused]] const IPAConfigInfo &configInfo) { @@ -373,6 +376,9 @@ void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) } } +/** + * \copydoc libcamera::ipa::Algorithm::process + */ void Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { calculateWBGains(stats); @@ -394,6 +400,9 @@ constexpr uint16_t Awb::threshold(float value) return value * 8191; } +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ void Awb::prepare(IPAContext &context, ipu3_uapi_params *params) { /* diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build index 3ec42f72..4db6ae1d 100644 --- a/src/ipa/ipu3/algorithms/meson.build +++ b/src/ipa/ipu3/algorithms/meson.build @@ -2,7 +2,6 @@ ipu3_ipa_algorithms = files([ 'agc.cpp', - 'algorithm.cpp', 'awb.cpp', 'blc.cpp', 'tone_mapping.cpp', diff --git a/src/ipa/ipu3/algorithms/algorithm.cpp b/src/ipa/libipa/algorithm.cpp similarity index 75% rename from src/ipa/ipu3/algorithms/algorithm.cpp rename to src/ipa/libipa/algorithm.cpp index 3e7e3018..398d5372 100644 --- a/src/ipa/ipu3/algorithms/algorithm.cpp +++ b/src/ipa/libipa/algorithm.cpp @@ -2,7 +2,7 @@ /* * Copyright (C) 2021, Ideas On Board * - * algorithm.cpp - IPU3 control algorithm interface + * algorithm.cpp - IPA control algorithm interface */ #include "algorithm.h" @@ -14,11 +14,15 @@ namespace libcamera { -namespace ipa::ipu3 { +namespace ipa { /** * \class Algorithm - * \brief The base class for all IPU3 algorithms + * \brief The base class for all IPA algorithms + * \tparam Context The type of shared IPA context + * \tparam Config The type of the IPA configuration data + * \tparam Params The type of the ISP specific parameters + * \tparam Stats The type of the IPA statistics and ISP results * * The Algorithm class defines a standard interface for IPA algorithms. By * abstracting algorithms, it makes possible the implementation of generic code @@ -26,6 +30,7 @@ namespace ipa::ipu3 { */ /** + * \fn Algorithm::configure() * \brief Configure the Algorithm given an IPAConfigInfo * \param[in] context The shared IPA context * \param[in] configInfo The IPA configuration data, received from the pipeline @@ -39,37 +44,30 @@ namespace ipa::ipu3 { * * \return 0 if successful, an error code otherwise */ -int Algorithm::configure([[maybe_unused]] IPAContext &context, - [[maybe_unused]] const IPAConfigInfo &configInfo) -{ - return 0; -} /** + * \fn Algorithm::prepare() * \brief Fill the \a params buffer with ISP processing parameters for a frame * \param[in] context The shared IPA context - * \param[out] params The IPU3 specific parameters. + * \param[out] params The ISP specific parameters. * * This function is called for every frame when the camera is running before it - * is processed by the ImgU to prepare the ImgU processing parameters for that + * is processed by the ISP to prepare the ISP processing parameters for that * frame. * * Algorithms shall fill in the parameter structure fields appropriately to - * configure the ImgU processing blocks that they are responsible for. This + * configure the ISP processing blocks that they are responsible for. This * includes setting fields and flags that enable those processing blocks. */ -void Algorithm::prepare([[maybe_unused]] IPAContext &context, - [[maybe_unused]] ipu3_uapi_params *params) -{ -} /** + * \fn Algorithm::process() * \brief Process ISP statistics, and run algorithm operations * \param[in] context The shared IPA context - * \param[in] stats The IPU3 statistics and ISP results + * \param[in] stats The IPA statistics and ISP results * * This function is called while camera is running for every frame processed by - * the ImgU, to process statistics generated from that frame by the ImgU. + * the ISP, to process statistics generated from that frame by the ISP. * Algorithms shall use this data to run calculations and update their state * accordingly. * @@ -91,11 +89,7 @@ void Algorithm::prepare([[maybe_unused]] IPAContext &context, * Care shall be taken to ensure the ordering of access to the information * such that the algorithms use up to date state as required. */ -void Algorithm::process([[maybe_unused]] IPAContext &context, - [[maybe_unused]] const ipu3_uapi_stats_3a *stats) -{ -} -} /* namespace ipa::ipu3 */ +} /* namespace ipa */ } /* namespace libcamera */ diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h new file mode 100644 index 00000000..766aee5d --- /dev/null +++ b/src/ipa/libipa/algorithm.h @@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * algorithm.h - ISP control algorithm interface + */ +#pragma once + +namespace libcamera { + +namespace ipa { + +template +class Algorithm +{ +public: + virtual ~Algorithm() {} + + virtual int configure([[maybe_unused]] Context &context, + [[maybe_unused]] const Config &configInfo) + { + return 0; + } + + virtual void prepare([[maybe_unused]] Context &context, + [[maybe_unused]] Params *params) + { + } + + virtual void process([[maybe_unused]] Context &context, + [[maybe_unused]] const Stats *stats) + { + } +}; + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index 4d073a03..161cc5a1 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 libipa_headers = files([ + 'algorithm.h', 'camera_sensor_helper.h', 'histogram.h' ]) From patchwork Thu Nov 25 05:42:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14768 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 12CF7C324F for ; Thu, 25 Nov 2021 05:43:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4F85460423; Thu, 25 Nov 2021 06:43:15 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="wBIvJMxH"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3E7C76038B for ; Thu, 25 Nov 2021 06:43:05 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:d9a5:5e40:3323:d95]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EC0D890E; Thu, 25 Nov 2021 06:43:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637818985; bh=8fl7Lehagj28Qb4w6d+e/+6wU0+qsFUeaY0oOnOy/gM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wBIvJMxH7sLXOjXTgSkZ6VP/KTW2hRqskRqCtDodqQSOb2A0rkrpw9EKLyM5Ubm+w t/zPbMvqclPMOGgrtW0puzZ+hyhxd9fjTvOPoAIM/wOu61qeW1YgnUdBoV7p6zBKg/ 5KtqhLhqQ0dSAItHaYeFYbTFZ5lypApH8VBFsP3Y= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 25 Nov 2021 06:42:55 +0100 Message-Id: <20211125054259.24792-8-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> References: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 07/11] ipa: rkisp1: Use the Algorithm class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that libipa offers a templated class for Algorithm, use it in RkISP1. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- v4: use #pragma once --- src/ipa/rkisp1/algorithms/algorithm.h | 26 ++++++++++++++++++++++++++ src/ipa/rkisp1/algorithms/meson.build | 4 ++++ src/ipa/rkisp1/meson.build | 4 ++++ src/ipa/rkisp1/rkisp1.cpp | 7 ++++++- 4 files changed, 40 insertions(+), 1 deletion(-) create mode 100644 src/ipa/rkisp1/algorithms/algorithm.h create mode 100644 src/ipa/rkisp1/algorithms/meson.build diff --git a/src/ipa/rkisp1/algorithms/algorithm.h b/src/ipa/rkisp1/algorithms/algorithm.h new file mode 100644 index 00000000..d46c3188 --- /dev/null +++ b/src/ipa/rkisp1/algorithms/algorithm.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * algorithm.h - RkISP1 control algorithm interface + */ + +#pragma once + +#include + +#include + +#include + +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::rkisp1 { + +using Algorithm = libcamera::ipa::Algorithm; + +} /* namespace ipa::rkisp1 */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/meson.build b/src/ipa/rkisp1/algorithms/meson.build new file mode 100644 index 00000000..1c6c59cf --- /dev/null +++ b/src/ipa/rkisp1/algorithms/meson.build @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: CC0-1.0 + +rkisp1_ipa_algorithms = files([ +]) diff --git a/src/ipa/rkisp1/meson.build b/src/ipa/rkisp1/meson.build index 3683c922..8c822fbb 100644 --- a/src/ipa/rkisp1/meson.build +++ b/src/ipa/rkisp1/meson.build @@ -1,5 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 +subdir('algorithms') + ipa_name = 'ipa_rkisp1' rkisp1_ipa_sources = files([ @@ -7,6 +9,8 @@ rkisp1_ipa_sources = files([ 'rkisp1.cpp', ]) +rkisp1_ipa_sources += rkisp1_ipa_algorithms + mod = shared_module(ipa_name, [rkisp1_ipa_sources, libcamera_generated_ipa_headers], name_prefix : '', diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 34c3f9a2..225f1847 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -25,9 +25,11 @@ #include -#include "ipa_context.h" +#include "algorithms/algorithm.h" #include "libipa/camera_sensor_helper.h" +#include "ipa_context.h" + namespace libcamera { LOG_DEFINE_CATEGORY(IPARkISP1) @@ -82,6 +84,9 @@ private: /* Local parameter storage */ struct IPAContext context_; + + /* Maintain the algorithms used by the IPA */ + std::list> algorithms_; }; int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) From patchwork Thu Nov 25 05:42:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14769 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 93AC2C3251 for ; Thu, 25 Nov 2021 05:43:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3A487604F9; Thu, 25 Nov 2021 06:43:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VjzKFp5F"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6E8DC60376 for ; Thu, 25 Nov 2021 06:43:05 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:d9a5:5e40:3323:d95]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 269E3A24; Thu, 25 Nov 2021 06:43:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637818985; bh=RXF5gd3nL/zL5elGMr1rrP0VB1zm1MMGKOQYwt6bjxI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VjzKFp5Fvnb/8ld8hCI+sweUOVY6QglfcQ0Hv0ZcGapwbcIreBMXRwBHmLdMxyOfO n2xWeetiK7dk0cfKN4PSczX7wDCdpBHgfw72xR3TH5FfQnOG2spDkx4lwNT8g6mUhQ LFP5y1FfGxnmv7Wco3inMCTBfIWz5RiXS0xyoxsY= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 25 Nov 2021 06:42:56 +0100 Message-Id: <20211125054259.24792-9-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> References: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 08/11] ipa: rkisp1: Report and use sensor controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The pipeline handler populates a new sensorControls ControlList, to have the effective exposure and gain values for the current frame. This is done when a statistics buffer is received. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/rkisp1.mojom | 2 ++ src/ipa/rkisp1/rkisp1.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 3 ++- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/include/libcamera/ipa/rkisp1.mojom b/include/libcamera/ipa/rkisp1.mojom index a6991d4f..c3a6d8e1 100644 --- a/include/libcamera/ipa/rkisp1.mojom +++ b/include/libcamera/ipa/rkisp1.mojom @@ -21,11 +21,13 @@ struct RkISP1Event { uint32 frame; uint32 bufferId; libcamera.ControlList controls; + libcamera.ControlList sensorControls; }; struct RkISP1Action { RkISP1Operations op; libcamera.ControlList controls; + libcamera.ControlList sensorControls; }; interface IPARkISP1Interface { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 225f1847..2b073258 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -314,7 +314,7 @@ void IPARkISP1::setControls(unsigned int frame) ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); - op.controls = ctrls; + op.sensorControls = ctrls; queueFrameAction.emit(frame, op); } diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 6cdc4c03..36ef6a02 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -333,7 +333,7 @@ void RkISP1CameraData::queueFrameAction(unsigned int frame, { switch (action.op) { case ipa::rkisp1::ActionV4L2Set: { - const ControlList &controls = action.controls; + const ControlList &controls = action.sensorControls; delayedCtrls_->push(controls); break; } @@ -1124,6 +1124,7 @@ void PipelineHandlerRkISP1::statReady(FrameBuffer *buffer) ev.op = ipa::rkisp1::EventSignalStatBuffer; ev.frame = info->frame; ev.bufferId = info->statBuffer->cookie(); + ev.sensorControls = data->delayedCtrls_->get(buffer->metadata().sequence); data->ipa_->processEvent(ev); } From patchwork Thu Nov 25 05:42:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14772 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BA4CAC3250 for ; Thu, 25 Nov 2021 05:43:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5BED7603F9; Thu, 25 Nov 2021 06:43:18 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Oza+8R5l"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 996FE60398 for ; Thu, 25 Nov 2021 06:43:05 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:d9a5:5e40:3323:d95]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 54FE71010; Thu, 25 Nov 2021 06:43:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637818985; bh=2fEa3PyQcdgXEzwUL/VwdXkZ9SIizmiHhEZ4ci3A55k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Oza+8R5lP4ouXdeKnvxmU5Dlcw3LtddI7mI7TL0MKJ7/UoR1FyLzSlO42nyaLytcj fkZT84UhMjgshEs8DPVczDf+r5vZqTF/dn+qfAXuDuM4Mr2kBlfPeEECP3haYoMpfH LZfSvCcUPbLN4aWWEbwV5kytJ5DdZeUpjW9PqSnI= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 25 Nov 2021 06:42:57 +0100 Message-Id: <20211125054259.24792-10-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> References: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 09/11] ipa: rkisp1: agc: Introduce HW revision in IPAContext X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ISP can use 25 or 81 cells depending on its revision. Remove the cached value in IPARkISP1 and use IPASessionConfiguration to store it and pass it to AGC later. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- v4: - Only store HW revision, and use it in next patch --- src/ipa/rkisp1/ipa_context.cpp | 8 ++++++++ src/ipa/rkisp1/ipa_context.h | 5 +++++ src/ipa/rkisp1/rkisp1.cpp | 7 +++++++ 3 files changed, 20 insertions(+) diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 819b2c73..6b53dfdf 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -55,4 +55,12 @@ namespace libcamera::ipa::rkisp1 { * are run. This needs to be turned into real per-frame data storage. */ +/** + * \var IPASessionConfiguration::hw + * \brief RkISP1-specific hardware information + * + * \var IPASessionConfiguration::hw.revision + * \brief Hardware revision of the ISP + */ + } /* namespace libcamera::ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 919139dd..9342025b 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -8,11 +8,16 @@ #pragma once +#include + namespace libcamera { namespace ipa::rkisp1 { struct IPASessionConfiguration { + struct { + rkisp1_cif_isp_version revision; + } hw; }; struct IPAFrameContext { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 2b073258..59676a70 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -74,6 +74,7 @@ private: uint32_t maxGain_; /* revision-specific data */ + rkisp1_cif_isp_version hwRevision_; unsigned int hwAeMeanMax_; unsigned int hwHistBinNMax_; unsigned int hwGammaOutMaxSamples_; @@ -114,6 +115,9 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) LOG(IPARkISP1, Debug) << "Hardware revision is " << hwRevision; + /* Cache the value to set it in configure. */ + hwRevision_ = static_cast(hwRevision); + camHelper_ = CameraSensorHelperFactory::create(settings.sensorModel); if (!camHelper_) { LOG(IPARkISP1, Error) @@ -176,6 +180,9 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, /* Clean context at configuration */ context_ = {}; + /* Set the hardware revision for the algorithms. */ + context_.configuration.hw.revision = hwRevision_; + return 0; } From patchwork Thu Nov 25 05:42:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14771 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DD2FDBF415 for ; Thu, 25 Nov 2021 05:43:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 72B5D60497; Thu, 25 Nov 2021 06:43:17 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jOKtAvXp"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C08686039C for ; Thu, 25 Nov 2021 06:43:05 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:d9a5:5e40:3323:d95]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8247190E; Thu, 25 Nov 2021 06:43:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637818985; bh=RVEAXx8vxe70ZRjU7nZOuxhfhpp5MMIowxjTXJUn+H4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jOKtAvXpk87yXXi1uokYQaspMoRkQSiH4jxMRNiQKK7rR20ZLeXahxLLBX0PZaVHT luDQ8XSWghK6NU1CvOfBmCSAs9KpSd35fnDtduEvfNZEi1RtVkRH5r+4XAcYjLNrMI sBTrALezBRFozj+cbveE7XrZrgzK6uy6rWXnpoog= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 25 Nov 2021 06:42:58 +0100 Message-Id: <20211125054259.24792-11-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> References: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 10/11] ipa: rkisp1: Introduce AGC X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that we have IPAContext and Algorithm, we can implement a simple AGC based on the IPU3 one. It is very similar, except that there is no histogram used for an inter quantile mean. The RkISP1 is returning a 5x5 array (for V10) of luminance means. Estimating the relative luminance is thus a simple mean of all the blocks already calculated by the ISP. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- v5: - use private filteredExposure_ and pass currentExposure as a member variable - Drop num and replace it with numCells_ - Make exposure and gain local variables in IPARkISP1 - Shorter the lines in processEvent() v4: - use #pragma once - Return filtered value from the function - Store line duration in IPASessionConfiguration - Use the hw revision to configure the number of AE cells --- src/ipa/rkisp1/algorithms/agc.cpp | 275 ++++++++++++++++++++++++++ src/ipa/rkisp1/algorithms/agc.h | 46 +++++ src/ipa/rkisp1/algorithms/meson.build | 1 + src/ipa/rkisp1/ipa_context.cpp | 45 +++++ src/ipa/rkisp1/ipa_context.h | 19 ++ src/ipa/rkisp1/rkisp1.cpp | 83 ++++---- 6 files changed, 425 insertions(+), 44 deletions(-) create mode 100644 src/ipa/rkisp1/algorithms/agc.cpp create mode 100644 src/ipa/rkisp1/algorithms/agc.h diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp new file mode 100644 index 00000000..31a3276f --- /dev/null +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -0,0 +1,275 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * agc.cpp - AGC/AEC mean-based control algorithm + */ + +#include "agc.h" + +#include +#include +#include + +#include + +#include + +/** + * \file agc.h + */ + +namespace libcamera { + +using namespace std::literals::chrono_literals; + +namespace ipa::rkisp1::algorithms { + +/** + * \class Agc + * \brief A mean-based auto-exposure algorithm + */ + +LOG_DEFINE_CATEGORY(RkISP1Agc) + +/* Limits for analogue gain values */ +static constexpr double kMinAnalogueGain = 1.0; +static constexpr double kMaxAnalogueGain = 8.0; + +/* \todo Honour the FrameDurationLimits control instead of hardcoding a limit */ +static constexpr utils::Duration kMaxShutterSpeed = 60ms; + +/* Number of frames to wait before calculating stats on minimum exposure */ +static constexpr uint32_t kNumStartupFrames = 10; + +/* + * Relative luminance target. + * + * It's a number that's chosen so that, when the camera points at a grey + * target, the resulting image brightness is considered right. + */ +static constexpr double kRelativeLuminanceTarget = 0.4; + +Agc::Agc() + : frameCount_(0), filteredExposure_(0s) +{ +} + +/** + * \brief Configure the AGC given a configInfo + * \param[in] context The shared IPA context + * \param[in] configInfo The IPA configuration data + * + * \return 0 + */ +int Agc::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + /* Configure the default exposure and gain. */ + context.frameContext.agc.gain = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain); + context.frameContext.agc.exposure = 10ms / context.configuration.agc.lineDuration; + + /* + * According to the RkISP1 documentation: + * - versions <= V11 have RKISP1_CIF_ISP_AE_MEAN_MAX_V10 entries, + * - versions >= V12 have RKISP1_CIF_ISP_AE_MEAN_MAX_V12 entries. + */ + if (context.configuration.hw.revision < RKISP1_V12) + numCells_ = RKISP1_CIF_ISP_AE_MEAN_MAX_V10; + else + numCells_ = RKISP1_CIF_ISP_AE_MEAN_MAX_V12; + + return 0; +} + +/** + * \brief Apply a filter on the exposure value to limit the speed of changes + */ +utils::Duration Agc::filterExposure(utils::Duration currentExposure) +{ + double speed = 0.2; + + /* Adapt instantly if we are in startup phase */ + if (frameCount_ < kNumStartupFrames) + speed = 1.0; + + if (filteredExposure_ == 0s) { + filteredExposure_ = currentExposure; + } else { + /* + * If we are close to the desired result, go faster to avoid making + * multiple micro-adjustments. + * \todo Make this customisable? + */ + if (filteredExposure_ < 1.2 * currentExposure && + filteredExposure_ > 0.8 * currentExposure) + speed = sqrt(speed); + + filteredExposure_ = speed * currentExposure + + filteredExposure_ * (1.0 - speed); + } + + LOG(RkISP1Agc, Debug) << "After filtering, total_exposure " << filteredExposure_; + + return filteredExposure_; +} + +/** + * \brief Estimate the new exposure and gain values + * \param[inout] frameContext The shared IPA frame Context + * \param[in] yGain The gain calculated on the current brightness level + */ +void Agc::computeExposure(IPAContext &context, double yGain) +{ + IPASessionConfiguration &configuration = context.configuration; + IPAFrameContext &frameContext = context.frameContext; + + /* Get the effective exposure and gain applied on the sensor. */ + uint32_t exposure = frameContext.sensor.exposure; + double analogueGain = frameContext.sensor.gain; + + utils::Duration minShutterSpeed = configuration.agc.minShutterSpeed; + utils::Duration maxShutterSpeed = std::min(configuration.agc.maxShutterSpeed, + kMaxShutterSpeed); + + double minAnalogueGain = std::max(configuration.agc.minAnalogueGain, + kMinAnalogueGain); + double maxAnalogueGain = std::min(configuration.agc.maxAnalogueGain, + kMaxAnalogueGain); + + /* Consider within 1% of the target as correctly exposed */ + if (std::abs(yGain - 1.0) < 0.01) + LOG(RkISP1Agc, Debug) << "We are well exposed (iqMean = " + << yGain << ")"; + + /* extracted from Rpi::Agc::computeTargetExposure */ + + /* Calculate the shutter time in seconds */ + utils::Duration currentShutter = exposure * configuration.agc.lineDuration; + + /* + * Update the exposure value for the next computation using the values + * of exposure and gain really used by the sensor. + */ + utils::Duration effectiveExposureValue = currentShutter * analogueGain; + + LOG(RkISP1Agc, Debug) << "Actual total exposure " << currentShutter * analogueGain + << " Shutter speed " << currentShutter + << " Gain " << analogueGain + << " Needed ev gain " << yGain; + + /* + * Calculate the current exposure value for the scene as the latest + * exposure value applied multiplied by the new estimated gain. + */ + utils::Duration currentExposure = effectiveExposureValue * yGain; + + /* Clamp the exposure value to the min and max authorized */ + utils::Duration maxTotalExposure = maxShutterSpeed * maxAnalogueGain; + currentExposure = std::min(currentExposure, maxTotalExposure); + LOG(RkISP1Agc, Debug) << "Target total exposure " << currentExposure + << ", maximum is " << maxTotalExposure; + + /* + * Divide the exposure value as new exposure and gain values + * \todo: estimate if we need to desaturate + */ + utils::Duration exposureValue = filterExposure(currentExposure); + utils::Duration shutterTime; + + /* + * Push the shutter time up to the maximum first, and only then + * increase the gain. + */ + shutterTime = std::clamp(exposureValue / minAnalogueGain, + minShutterSpeed, maxShutterSpeed); + double stepGain = std::clamp(exposureValue / shutterTime, + minAnalogueGain, maxAnalogueGain); + LOG(RkISP1Agc, Debug) << "Divided up shutter and gain are " + << shutterTime << " and " + << stepGain; + + /* Update the estimated exposure and gain. */ + frameContext.agc.exposure = shutterTime / configuration.agc.lineDuration; + frameContext.agc.gain = stepGain; +} + +/** + * \brief Estimate the relative luminance of the frame with a given gain + * \param[in] ae The RkISP1 statistics and ISP results + * \param[in] gain The gain calculated on the current brightness level + * \return The relative luminance + * + * This function estimates the average relative luminance of the frame that + * would be output by the sensor if an additional \a gain was applied. + * + * The estimation is based on the AE statistics for the current frame. Y + * averages for all cells are first multiplied by the gain, and then saturated + * to approximate the sensor behaviour at high brightness values. The + * approximation is quite rough, as it doesn't take into account non-linearities + * when approaching saturation. + * + * The values are normalized to the [0.0, 1.0] range, where 1.0 corresponds to a + * theoretical perfect reflector of 100% reference white. + * + * More detailed information can be found in: + * https://en.wikipedia.org/wiki/Relative_luminance + */ +double Agc::estimateLuminance(const rkisp1_cif_isp_ae_stat *ae, + double gain) +{ + double ySum = 0.0; + + /* Sum the averages, saturated to 255. */ + for (unsigned int aeCell = 0; aeCell < numCells_; aeCell++) + ySum += std::min(ae->exp_mean[aeCell] * gain, 255.0); + + /* \todo Weight with the AWB gains */ + + return ySum / numCells_ / 255; +} + +/** + * \brief Process RkISP1 statistics, and run AGC operations + * \param[in] context The shared IPA context + * \param[in] stats The RKIsp1 statistics and ISP results + * + * Identify the current image brightness, and use that to estimate the optimal + * new exposure and gain for the scene. + */ +void Agc::process(IPAContext &context, const rkisp1_stat_buffer *stats) +{ + const rkisp1_cif_isp_stat *params = &stats->params; + ASSERT(stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP); + + const rkisp1_cif_isp_ae_stat *ae = ¶ms->ae; + + /* + * Estimate the gain needed to achieve a relative luminance target. To + * account for non-linearity caused by saturation, the value needs to be + * estimated in an iterative process, as multiplying by a gain will not + * increase the relative luminance by the same factor if some image + * regions are saturated. + */ + double yGain = 1.0; + double yTarget = kRelativeLuminanceTarget; + + for (unsigned int i = 0; i < 8; i++) { + double yValue = estimateLuminance(ae, yGain); + double extra_gain = std::min(10.0, yTarget / (yValue + .001)); + + yGain *= extra_gain; + LOG(RkISP1Agc, Debug) << "Y value: " << yValue + << ", Y target: " << yTarget + << ", gives gain " << yGain; + if (extra_gain < 1.01) + break; + } + + computeExposure(context, yGain); + frameCount_++; +} + +} /* namespace ipa::rkisp1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h new file mode 100644 index 00000000..8ec172d7 --- /dev/null +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Ideas On Board + * + * agc.h - RkISP1 AGC/AEC mean-based control algorithm + */ + +#pragma once + +#include + +#include + +#include + +#include "algorithm.h" + +namespace libcamera { + +struct IPACameraSensorInfo; + +namespace ipa::rkisp1::algorithms { + +class Agc : public Algorithm +{ +public: + Agc(); + ~Agc() = default; + + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void process(IPAContext &context, const rkisp1_stat_buffer *stats) override; + +private: + void computeExposure(IPAContext &Context, double yGain); + utils::Duration filterExposure(utils::Duration currentExposure); + double estimateLuminance(const rkisp1_cif_isp_ae_stat *ae, double gain); + + uint64_t frameCount_; + + uint32_t numCells_; + + utils::Duration filteredExposure_; +}; + +} /* namespace ipa::rkisp1::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/meson.build b/src/ipa/rkisp1/algorithms/meson.build index 1c6c59cf..a19c1a4f 100644 --- a/src/ipa/rkisp1/algorithms/meson.build +++ b/src/ipa/rkisp1/algorithms/meson.build @@ -1,4 +1,5 @@ # SPDX-License-Identifier: CC0-1.0 rkisp1_ipa_algorithms = files([ + 'agc.cpp', ]) diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 6b53dfdf..dff00362 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -56,6 +56,24 @@ namespace libcamera::ipa::rkisp1 { */ /** + * \var IPASessionConfiguration::agc + * \brief AGC parameters configuration of the IPA + * + * \var IPASessionConfiguration::agc.minShutterSpeed + * \brief Minimum shutter speed supported with the configured sensor + * + * \var IPASessionConfiguration::agc.maxShutterSpeed + * \brief Maximum shutter speed supported with the configured sensor + * + * \var IPASessionConfiguration::agc.minAnalogueGain + * \brief Minimum analogue gain supported with the configured sensor + * + * \var IPASessionConfiguration::agc.maxAnalogueGain + * \brief Maximum analogue gain supported with the configured sensor + * + * \var IPASessionConfiguration::agc.lineDuration + * \brief Line duration in microseconds + * * \var IPASessionConfiguration::hw * \brief RkISP1-specific hardware information * @@ -63,4 +81,31 @@ namespace libcamera::ipa::rkisp1 { * \brief Hardware revision of the ISP */ +/** + * \var IPAFrameContext::agc + * \brief Context for the Automatic Gain Control algorithm + * + * The exposure and gain determined are expected to be applied to the sensor + * at the earliest opportunity. + * + * \var IPAFrameContext::agc.exposure + * \brief Exposure time expressed as a number of lines + * + * \var IPAFrameContext::agc.gain + * \brief Analogue gain multiplier + * + * The gain should be adapted to the sensor specific gain code before applying. + */ + +/** + * \var IPAFrameContext::sensor + * \brief Effective sensor values + * + * \var IPAFrameContext::sensor.exposure + * \brief Exposure time expressed as a number of lines + * + * \var IPAFrameContext::sensor.gain + * \brief Analogue gain multiplier + */ + } /* namespace libcamera::ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 9342025b..af832e6e 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -10,17 +10,36 @@ #include +#include + namespace libcamera { namespace ipa::rkisp1 { struct IPASessionConfiguration { + struct { + utils::Duration minShutterSpeed; + utils::Duration maxShutterSpeed; + double minAnalogueGain; + double maxAnalogueGain; + utils::Duration lineDuration; + } agc; + struct { rkisp1_cif_isp_version revision; } hw; }; struct IPAFrameContext { + struct { + uint32_t exposure; + double gain; + } agc; + + struct { + uint32_t exposure; + double gain; + } sensor; }; struct IPAContext { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 59676a70..1710f9a3 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -25,6 +25,7 @@ #include +#include "algorithms/agc.h" #include "algorithms/algorithm.h" #include "libipa/camera_sensor_helper.h" @@ -34,6 +35,8 @@ namespace libcamera { LOG_DEFINE_CATEGORY(IPARkISP1) +using namespace std::literals::chrono_literals; + namespace ipa::rkisp1 { class IPARkISP1 : public IPARkISP1Interface @@ -66,16 +69,13 @@ private: /* Camera sensor controls. */ bool autoExposure_; - uint32_t exposure_; uint32_t minExposure_; uint32_t maxExposure_; - uint32_t gain_; uint32_t minGain_; uint32_t maxGain_; /* revision-specific data */ rkisp1_cif_isp_version hwRevision_; - unsigned int hwAeMeanMax_; unsigned int hwHistBinNMax_; unsigned int hwGammaOutMaxSamples_; unsigned int hwHistogramWeightGridsSize_; @@ -95,13 +95,11 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) /* \todo Add support for other revisions */ switch (hwRevision) { case RKISP1_V10: - hwAeMeanMax_ = RKISP1_CIF_ISP_AE_MEAN_MAX_V10; hwHistBinNMax_ = RKISP1_CIF_ISP_HIST_BIN_N_MAX_V10; hwGammaOutMaxSamples_ = RKISP1_CIF_ISP_GAMMA_OUT_MAX_SAMPLES_V10; hwHistogramWeightGridsSize_ = RKISP1_CIF_ISP_HISTOGRAM_WEIGHT_GRIDS_SIZE_V10; break; case RKISP1_V12: - hwAeMeanMax_ = RKISP1_CIF_ISP_AE_MEAN_MAX_V12; hwHistBinNMax_ = RKISP1_CIF_ISP_HIST_BIN_N_MAX_V12; hwGammaOutMaxSamples_ = RKISP1_CIF_ISP_GAMMA_OUT_MAX_SAMPLES_V12; hwHistogramWeightGridsSize_ = RKISP1_CIF_ISP_HISTOGRAM_WEIGHT_GRIDS_SIZE_V12; @@ -126,6 +124,9 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision) return -ENODEV; } + /* Construct our Algorithms */ + algorithms_.push_back(std::make_unique()); + return 0; } @@ -167,11 +168,9 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, minExposure_ = itExp->second.min().get(); maxExposure_ = itExp->second.max().get(); - exposure_ = minExposure_; minGain_ = itGain->second.min().get(); maxGain_ = itGain->second.max().get(); - gain_ = minGain_; LOG(IPARkISP1, Info) << "Exposure: " << minExposure_ << "-" << maxExposure_ @@ -183,6 +182,26 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, /* Set the hardware revision for the algorithms. */ context_.configuration.hw.revision = hwRevision_; + context_.configuration.agc.lineDuration = info.lineLength * 1.0s / info.pixelRate; + + /* + * When the AGC computes the new exposure values for a frame, it needs + * to know the limits for shutter speed and analogue gain. + * As it depends on the sensor, update it with the controls. + * + * \todo take VBLANK into account for maximum shutter speed + */ + context_.configuration.agc.minShutterSpeed = minExposure_ * context_.configuration.agc.lineDuration; + context_.configuration.agc.maxShutterSpeed = maxExposure_ * context_.configuration.agc.lineDuration; + context_.configuration.agc.minAnalogueGain = camHelper_->gain(minGain_); + context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain_); + + for (auto const &algo : algorithms_) { + int ret = algo->configure(context_, info); + if (ret) + return ret; + } + return 0; } @@ -227,6 +246,11 @@ void IPARkISP1::processEvent(const RkISP1Event &event) reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); + context_.frameContext.sensor.exposure = + event.sensorControls.get(V4L2_CID_EXPOSURE).get(); + context_.frameContext.sensor.gain = + camHelper_->gain(event.sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + updateStatistics(frame, stats); break; } @@ -271,44 +295,12 @@ void IPARkISP1::queueRequest(unsigned int frame, rkisp1_params_cfg *params, void IPARkISP1::updateStatistics(unsigned int frame, const rkisp1_stat_buffer *stats) { - const rkisp1_cif_isp_stat *params = &stats->params; unsigned int aeState = 0; - if (stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP) { - const rkisp1_cif_isp_ae_stat *ae = ¶ms->ae; - - const unsigned int target = 60; - - unsigned int value = 0; - unsigned int num = 0; - for (unsigned int i = 0; i < hwAeMeanMax_; i++) { - if (ae->exp_mean[i] <= 15) - continue; - - value += ae->exp_mean[i]; - num++; - } - value /= num; + for (auto const &algo : algorithms_) + algo->process(context_, stats); - double factor = (double)target / value; - - if (frame % 3 == 0) { - double exposure; - - exposure = factor * exposure_ * gain_ / minGain_; - exposure_ = std::clamp((uint64_t)exposure, - minExposure_, - maxExposure_); - - exposure = exposure / exposure_ * minGain_; - gain_ = std::clamp((uint64_t)exposure, - minGain_, maxGain_); - - setControls(frame + 1); - } - - aeState = fabs(factor - 1.0f) < 0.05f ? 2 : 1; - } + setControls(frame); metadataReady(frame, aeState); } @@ -318,9 +310,12 @@ void IPARkISP1::setControls(unsigned int frame) RkISP1Action op; op.op = ActionV4L2Set; + uint32_t exposure = context_.frameContext.agc.exposure; + uint32_t gain = camHelper_->gainCode(context_.frameContext.agc.gain); + ControlList ctrls(ctrls_); - ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); - ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); + ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); + ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain)); op.sensorControls = ctrls; queueFrameAction.emit(frame, op); From patchwork Thu Nov 25 05:42:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14770 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3FCBDC3252 for ; Thu, 25 Nov 2021 05:43:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E541B60492; Thu, 25 Nov 2021 06:43:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ANxkv7lL"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 01B9A60228 for ; Thu, 25 Nov 2021 06:43:06 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:d9a5:5e40:3323:d95]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id ABEB4A24; Thu, 25 Nov 2021 06:43:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637818985; bh=4V9e984NDrpDeJPFUtWLq2BuCoOwQmXmXW8XZi+CqZE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ANxkv7lL3wBCMVpNuUxdXIgu0iyj3eb+WNHEDDZGJvnawCkA+l4/h2oNZLgvLShwx BCNj8CCcmMLxkfGmiDwA86jOmme9GT2oMqWrS1pcpLcOIxxFeqQB8la1XxM1BvQZoB 5NWpBEDu/qcjdoDqndODg7R0OVaMrYNSojUaJd+8= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 25 Nov 2021 06:42:59 +0100 Message-Id: <20211125054259.24792-12-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> References: <20211125054259.24792-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 11/11] ipa: rkisp1: agc: Introduce prepare call X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a new parameter buffer needs to be queued, we need to specify which algorithm is activated or not in the ISP. Add a simple prepare function in AGC for that, which may later evolve to take the exposure locking into account. For that function to be called, we also need to add the loop on the algorithms in IPARkISP1::queueRequest. We no longer disable the AE algorithm based on the controls::AeEnable, which will be handled in a different manner later. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/rkisp1/algorithms/agc.cpp | 7 +++++++ src/ipa/rkisp1/algorithms/agc.h | 1 + src/ipa/rkisp1/rkisp1.cpp | 12 +++--------- 3 files changed, 11 insertions(+), 9 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 31a3276f..c1bdb20a 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -270,6 +270,13 @@ void Agc::process(IPAContext &context, const rkisp1_stat_buffer *stats) frameCount_++; } +void Agc::prepare([[maybe_unused]] IPAContext &context, + rkisp1_params_cfg *params) +{ + params->module_ens |= RKISP1_CIF_ISP_MODULE_AEC; + params->module_en_update |= RKISP1_CIF_ISP_MODULE_AEC; +} + } /* namespace ipa::rkisp1::algorithms */ } /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index 8ec172d7..717818f6 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -28,6 +28,7 @@ public: ~Agc() = default; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void prepare(IPAContext &context, rkisp1_params_cfg *params) override; void process(IPAContext &context, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 1710f9a3..949fa818 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -272,19 +272,13 @@ void IPARkISP1::processEvent(const RkISP1Event &event) } void IPARkISP1::queueRequest(unsigned int frame, rkisp1_params_cfg *params, - const ControlList &controls) + [[maybe_unused]] const ControlList &controls) { /* Prepare parameters buffer. */ memset(params, 0, sizeof(*params)); - /* Auto Exposure on/off. */ - if (controls.contains(controls::AeEnable)) { - autoExposure_ = controls.get(controls::AeEnable); - if (autoExposure_) - params->module_ens = RKISP1_CIF_ISP_MODULE_AEC; - - params->module_en_update = RKISP1_CIF_ISP_MODULE_AEC; - } + for (auto const &algo : algorithms_) + algo->prepare(context_, params); RkISP1Action op; op.op = ActionParamFilled;