From patchwork Mon Nov 8 13:13:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14470 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3F35FBDB1C for ; Mon, 8 Nov 2021 13:14:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BDE5B60423; Mon, 8 Nov 2021 14:14:04 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dv+YQdnN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C37DF6034A for ; Mon, 8 Nov 2021 14:13:59 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5FF4E1853; Mon, 8 Nov 2021 14:13:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377239; bh=wHKKLWoTGN1oJjNI1eTwCu0kK/BjSbXOLNQPYejrFbA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dv+YQdnNeqkG0Df0VwjWopXi91n0KnUBTstIZg5JV+xrNk1tuMb8iHJTlfGBBdii1 q+tss/N34VXOQHd0xGYIxTuzdW45kxK8T2WspB6FdmC6bYgdysO6fB5mIV8j1cOt+R wv0iq9HEgkFB0VnWQq2g9MvRdOZ375RVU3h0+BGE= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:29 +0100 Message-Id: <20211108131350.130665-2-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 01/22] ipa: ipu3: Extend ipu3 ipa interface for sensor controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Han-Lin Chen IPU3Event and IPU3Action use single ControlList for both libcamera and V4L2 controls, and it's content could be either one based on the context. Extend IPU3Event and IPU3Action for sensor V4L2 controls, and preserve the original one for only libcamera Controls to make the content of an event more specific. Signed-off-by: Han-Lin Chen [Jean-Michel: remove lensControls from the original patch] Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder --- include/libcamera/ipa/ipu3.mojom | 2 ++ src/ipa/ipu3/ipu3.cpp | 2 +- src/libcamera/pipeline/ipu3/ipu3.cpp | 2 +- 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/include/libcamera/ipa/ipu3.mojom b/include/libcamera/ipa/ipu3.mojom index 2f254ed4..16e3462e 100644 --- a/include/libcamera/ipa/ipu3.mojom +++ b/include/libcamera/ipa/ipu3.mojom @@ -23,11 +23,13 @@ struct IPU3Event { int64 frameTimestamp; uint32 bufferId; libcamera.ControlList controls; + libcamera.ControlList sensorControls; }; struct IPU3Action { IPU3Operations op; libcamera.ControlList controls; + libcamera.ControlList sensorControls; }; struct IPAConfigInfo { diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 93b700bd..bcc3863b 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -653,7 +653,7 @@ void IPAIPU3::setControls(unsigned int frame) ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); - op.controls = ctrls; + op.sensorControls = ctrls; queueFrameAction.emit(frame, op); } diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index eb714aa6..8816efc5 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1248,7 +1248,7 @@ void IPU3CameraData::queueFrameAction(unsigned int id, { switch (action.op) { case ipa::ipu3::ActionSetSensorControls: { - const ControlList &controls = action.controls; + const ControlList &controls = action.sensorControls; delayedCtrls_->push(controls); break; } From patchwork Mon Nov 8 13:13:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14471 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 67961BDB1C for ; Mon, 8 Nov 2021 13:14:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 050336038B; Mon, 8 Nov 2021 14:14:06 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cIHWJto5"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D4A0F6034D for ; Mon, 8 Nov 2021 14:13:59 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8F47B187C; Mon, 8 Nov 2021 14:13:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377239; bh=JTIA8hYd6p2uyv0s7SYTkhlea7VrdT5sCViiG0lNHxA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cIHWJto5+Xc595D/RiTYWzzGP+zLv7GOUaswrzAEbhqVaCtEgnpVqU24bW2chGPyV ysVK0F6n8tdzMmn0Fru2mmTzB8+s+FsRG+w0JErZX2nBC1yofFPDvaZVI61zvYlKFO oCJZoz6eGRi/5BgUsxI57TFvefBPrq3DscwRyE08= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:30 +0100 Message-Id: <20211108131350.130665-3-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 02/22] ipu3: ipa: Report effective sensor controls with statistics to IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Han-Lin Chen The Intel close sourced IPA requires the effective controls applied to the sensor when the statistics are generated. Report effective sensor controls with the stastistics to IPA. Signed-off-by: Han-Lin Chen [Jean-Michel: Reword s/stastistics/statistics] Signed-off-by: Jean-Michel Hautbois --- src/libcamera/pipeline/ipu3/frames.h | 3 +++ src/libcamera/pipeline/ipu3/ipu3.cpp | 5 +++++ 2 files changed, 8 insertions(+) diff --git a/src/libcamera/pipeline/ipu3/frames.h b/src/libcamera/pipeline/ipu3/frames.h index 3ef7e445..a897e307 100644 --- a/src/libcamera/pipeline/ipu3/frames.h +++ b/src/libcamera/pipeline/ipu3/frames.h @@ -12,6 +12,7 @@ #include #include +#include #include namespace libcamera { @@ -34,6 +35,8 @@ public: FrameBuffer *paramBuffer; FrameBuffer *statBuffer; + ControlList effectiveSensorControls; + bool paramDequeued; bool metadataProcessed; }; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 8816efc5..6a7f5b9a 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -667,6 +667,8 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) return ret; } + data->delayedCtrls_->reset(); + return updateControls(data); } @@ -1363,6 +1365,8 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) request->metadata().set(controls::SensorTimestamp, buffer->metadata().timestamp); + info->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence); + if (request->findBuffer(&rawStream_)) pipe()->completeBuffer(request, buffer); @@ -1419,6 +1423,7 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer) ev.frame = info->id; ev.bufferId = info->statBuffer->cookie(); ev.frameTimestamp = request->metadata().get(controls::SensorTimestamp); + ev.sensorControls = info->effectiveSensorControls; ipa_->processEvent(ev); } From patchwork Mon Nov 8 13:13:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14472 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8AE7EBDB1C for ; Mon, 8 Nov 2021 13:14:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 84C63603C4; Mon, 8 Nov 2021 14:14:06 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WPxF2VzG"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 08A3A6035D for ; Mon, 8 Nov 2021 14:14:00 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B8EAA18CE; Mon, 8 Nov 2021 14:13:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377239; bh=HIy4L0Vz8vpJyU8EqvVV7m8w9ft2/jrhgSbi7EG/G+g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WPxF2VzGqiWptQr6e7/uxMdT6WE2defVocxGyGzv2wqRrA3/qf9N2oEpio4U7hx4E I358T04y6q+ULYENASO1JKDtJ9Y3+WtSUKxAsYQwDMWSUFK0VnFhfKNiGBlC66pLcp K4cyaZsUyUDf3et3/0kxdDskds/vgxtC3CjPhhr0= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:31 +0100 Message-Id: <20211108131350.130665-4-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 03/22] ipa: ipu3: Use sensor controls to update frameContext X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The pipeline handler populates the new sensrControls ControlList, to have the effective exposure and gain values for the current frame. This is done when a statistics buffer is received. Make those values the frameContext::agc values for the frame when the EventStatReady event is received. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipu3.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index bcc3863b..b4e0d49f 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -549,6 +549,10 @@ void IPAIPU3::processEvent(const IPU3Event &event) const ipu3_uapi_stats_3a *stats = reinterpret_cast(mem.data()); + /* \todo move those into processControls */ + context_.frameContext.agc.exposure = event.sensorControls.get(V4L2_CID_EXPOSURE).get(); + context_.frameContext.agc.gain = camHelper_->gain(event.sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + parseStatistics(event.frame, event.frameTimestamp, stats); break; } From patchwork Mon Nov 8 13:13:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14474 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8BDA8C324E for ; Mon, 8 Nov 2021 13:14:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3B223603CB; Mon, 8 Nov 2021 14:14:07 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VIH6l66c"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2AA1F6035F for ; Mon, 8 Nov 2021 14:14:00 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E36EE18F6; Mon, 8 Nov 2021 14:13:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377240; bh=m/WSgiABApRZavKvLz+wivWfJPXldf4jOeFzHbxlpmE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VIH6l66c/+qlua0AFwI3KART0jswyuYAXNp/zl9gElRz38CYDZfnPH4t4O/LifzUQ a0IbkXq7ne3Bmz92BYG87TaUgK5WUmrhtkC7NKDn9zrpjOeUxuOg3cuL9BfnLulZPE Re2RvoghbeAxduGWrjUOP+YTiLke3TY0JSQsltrk= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:32 +0100 Message-Id: <20211108131350.130665-5-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 04/22] ipa: ipu3: agc: Limit the number of saturated cells X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the histogram is calculated, we check if a cell is saturated or not before cumulating its green value. This is wrong, and it can lead to an empty histogram in case of a fully saturated frame. Use a constant to limit the amount of pixels within a cell before considering it saturated. If at the end of the loop we still have an empty histogram, then make it a fully saturated one. Bug: https://bugs.libcamera.org/show_bug.cgi?id=84 Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/agc.cpp | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index b5d736c1..2bf68e04 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -63,6 +63,12 @@ static constexpr uint32_t knumHistogramBins = 256; /* Target value to reach for the top 2% of the histogram */ static constexpr double kEvGainTarget = 0.5; +/* + * Maximum ratio of saturated pixels in a cell for the cell to be considered + * non-saturated and counted by the AGC algorithm. + */ +static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 20 / 100; + Agc::Agc() : frameCount_(0), lastFrame_(0), iqMean_(0.0), lineDuration_(0s), minExposureLines_(0), maxExposureLines_(0), filteredExposure_(0s), @@ -124,7 +130,7 @@ void Agc::measureBrightness(const ipu3_uapi_stats_3a *stats, &stats->awb_raw_buffer.meta_data[cellPosition] ); - if (cell->sat_ratio == 0) { + if (cell->sat_ratio <= kMinCellsPerZoneRatio) { uint8_t gr = cell->Gr_avg; uint8_t gb = cell->Gb_avg; /* @@ -137,8 +143,14 @@ void Agc::measureBrightness(const ipu3_uapi_stats_3a *stats, } } + Histogram cumulativeHist = Histogram(Span(hist)); /* Estimate the quantile mean of the top 2% of the histogram */ - iqMean_ = Histogram(Span(hist)).interQuantileMean(0.98, 1.0); + if (cumulativeHist.total() == 0) { + /* Force the value as histogram is empty */ + iqMean_ = knumHistogramBins - 0.5; + } else { + iqMean_ = cumulativeHist.interQuantileMean(0.98, 1.0); + } } /** From patchwork Mon Nov 8 13:13:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14473 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 56556BDB1C for ; Mon, 8 Nov 2021 13:14:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9F96F6043C; Mon, 8 Nov 2021 14:14:07 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fBLvpPxQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 59A9260362 for ; Mon, 8 Nov 2021 14:14:00 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 19DC5190C; Mon, 8 Nov 2021 14:14:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377240; bh=BBISLBkrdBqqRRc3oTJ1P9RNJ/DXAJEGYSFnPNmfG9M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fBLvpPxQqyNxoOC8zRpkBD+SKn451xfqBfvBLq2SaIVEsYccW23oGNkNrmRpKoQPL WnazCHtrX6cJlelMlIY/RkcNLBORZ/6Lcb2FY/qUMna9SO8dIO55tw14kd+PNd7+t5 nmtJRk2YPkBreAMaYcsNGkxsVqxPXEdCMt4l+hO0= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:33 +0100 Message-Id: <20211108131350.130665-6-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 05/22] ipa: ipu3: agc: Compute the gain for each frame X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that we have the real exposure applied at each frame, remove the early return based on a frame counter and compute the gain for each frame. Instead of that, introduce a number of startup frames during which the filter speed is 1.0, meaning we apply instantly the exposure value calculated and not a slower filtered one. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/agc.cpp | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 2bf68e04..133f5931 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -45,11 +45,6 @@ namespace ipa::ipu3::algorithms { LOG_DEFINE_CATEGORY(IPU3Agc) -/* Number of frames to wait before calculating stats on minimum exposure */ -static constexpr uint32_t kInitialFrameMinAECount = 4; -/* Number of frames to wait between new gain/shutter time estimations */ -static constexpr uint32_t kFrameSkipCount = 6; - /* Limits for analogue gain values */ static constexpr double kMinAnalogueGain = 1.0; static constexpr double kMaxAnalogueGain = 8.0; @@ -69,10 +64,13 @@ static constexpr double kEvGainTarget = 0.5; */ static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 20 / 100; +/* Number of frames to wait before calculating stats on minimum exposure */ +static constexpr uint32_t kNumStartupFrames = 10; + Agc::Agc() - : frameCount_(0), lastFrame_(0), iqMean_(0.0), lineDuration_(0s), - minExposureLines_(0), maxExposureLines_(0), filteredExposure_(0s), - currentExposure_(0s), prevExposureValue_(0s) + : frameCount_(0), iqMean_(0.0), lineDuration_(0s), minExposureLines_(0), + maxExposureLines_(0), filteredExposure_(0s), currentExposure_(0s), + prevExposureValue_(0s) { } @@ -159,6 +157,11 @@ void Agc::measureBrightness(const ipu3_uapi_stats_3a *stats, void Agc::filterExposure() { double speed = 0.2; + + /* Adapt instantly if we are in startup phase */ + if (frameCount_ < kNumStartupFrames) + speed = 1.0; + if (filteredExposure_ == 0s) { /* DG stands for digital gain.*/ filteredExposure_ = currentExposure_; @@ -186,13 +189,6 @@ void Agc::filterExposure() */ void Agc::computeExposure(uint32_t &exposure, double &analogueGain) { - /* Algorithm initialization should wait for first valid frames */ - /* \todo - have a number of frames given by DelayedControls ? - * - implement a function for IIR */ - if ((frameCount_ < kInitialFrameMinAECount) || (frameCount_ - lastFrame_ < kFrameSkipCount)) - return; - - lastFrame_ = frameCount_; /* Are we correctly exposed ? */ if (std::abs(iqMean_ - kEvGainTarget * knumHistogramBins) <= 1) { From patchwork Mon Nov 8 13:13:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14475 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EDB6FC324F for ; Mon, 8 Nov 2021 13:14:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 18DB1603D9; Mon, 8 Nov 2021 14:14:08 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="iVKjn6aF"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 84AEE6032C for ; Mon, 8 Nov 2021 14:14:00 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 43ED51853; Mon, 8 Nov 2021 14:14:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377240; bh=xYoyXO2xIpO0T3PlvfATsenTDcmVIxKJiA7AEOKpRI8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iVKjn6aFOfPWOkEyr4IEDEb4LSSLt1pmcE/25qsVMOZ4V46RkCbIN+F5eVXHTRf04 ovsaiu3gBLR/jrOKx16JH7riXWoGr+DbU1Fr9zrBSsOM7iXsBPBKVWleetCMn84n9S L3s2ab4Hb9igTyxlGhPNWOrjFZy9LnJABi9JXmkc= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:34 +0100 Message-Id: <20211108131350.130665-7-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 06/22] ipa: ipu3: agc: Refactor ev gain calculation and testing X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When we compute the new gain, we use the iqMean_ and estimate an exposure value gain to apply. Return early when the gain is less than 1%. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/agc.cpp | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 133f5931..119a7938 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -189,17 +189,15 @@ void Agc::filterExposure() */ void Agc::computeExposure(uint32_t &exposure, double &analogueGain) { + /* Estimate the gain needed to have the proportion wanted */ + double evGain = kEvGainTarget * knumHistogramBins / iqMean_; - /* Are we correctly exposed ? */ - if (std::abs(iqMean_ - kEvGainTarget * knumHistogramBins) <= 1) { + if (std::abs(evGain - 1.0) < 0.01) { LOG(IPU3Agc, Debug) << "We are well exposed (iqMean = " << iqMean_ << ")"; return; } - /* Estimate the gain needed to have the proportion wanted */ - double evGain = kEvGainTarget * knumHistogramBins / iqMean_; - /* extracted from Rpi::Agc::computeTargetExposure */ /* Calculate the shutter time in seconds */ utils::Duration currentShutter = exposure * lineDuration_; From patchwork Mon Nov 8 13:13:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14476 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C9FEDBDB1C for ; Mon, 8 Nov 2021 13:14:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D44A76039C; Mon, 8 Nov 2021 14:14:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="afwnBodz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AF16360368 for ; Mon, 8 Nov 2021 14:14:00 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6EF71110E; Mon, 8 Nov 2021 14:14:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377240; bh=8ssxltMYMTz012AgiKESipgfnNQY25+vOCry7kbaFLQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=afwnBodzIkU8zNr+snHIN2+U3thd4qH8rRCh2Zf6bPYp96m91uOylp56ODpXFfG5z ddf02XjaTk+hj+W6n/ebws+65h2lvDERUjj+NmsTuon5OZY5PqYygNWCW7F1DlpfMK QB7X2JDiuOiKxejBqz9xK/VbzLYVSDNnfL4721s8= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:35 +0100 Message-Id: <20211108131350.130665-8-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 07/22] ipa: ipu3: agc: Improve gain calculation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When an image is partially saturated, its brightness is not increasing linearly when the shutter time or gain increases. It is a big issue with a backlight as the algorithm is fading to darkness right now. Introduce a function to estimate the brightness of the frame, based on the current exposure/gain and loop on it several times to estimate it again and approach the non linear function. Inspired-by: 7de5506c30b3 ("libcamera: src: ipa: raspberrypi: agc: Improve gain update calculation for partly saturated images") Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/agc.cpp | 77 ++++++++++++++++++++++++++++++++- src/ipa/ipu3/algorithms/agc.h | 6 ++- 2 files changed, 80 insertions(+), 3 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 119a7938..e4048d40 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -186,8 +186,9 @@ void Agc::filterExposure() * \brief Estimate the new exposure and gain values * \param[inout] exposure The exposure value reference as a number of lines * \param[inout] gain The gain reference to be updated + * \param[in] currentYGain The gain calculated on the current brightness level */ -void Agc::computeExposure(uint32_t &exposure, double &analogueGain) +void Agc::computeExposure(uint32_t &exposure, double &analogueGain, double currentYGain) { /* Estimate the gain needed to have the proportion wanted */ double evGain = kEvGainTarget * knumHistogramBins / iqMean_; @@ -206,6 +207,14 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain) << " Gain " << analogueGain << " Needed ev gain " << evGain; + if (evGain < currentYGain) + evGain = currentYGain; + + if (std::abs(evGain - 1.0) < 0.01) { + LOG(IPU3Agc, Debug) << "We are well exposed (iqMean = " + << iqMean_ << ")"; + } + /* * Calculate the current exposure value for the scene as the latest * exposure value applied multiplied by the new estimated gain. @@ -253,6 +262,48 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain) prevExposureValue_ = shutterTime * analogueGain; } +/** + * \brief Estimate the average brightness of the frame + * \param[in] context The shared IPA context + * \param[in] grid The grid used to store the statistics in the IPU3 + * \param[in] stats The IPU3 statistics and ISP results + * \param[in] currentYGain The gain calculated on the current brightness level + */ +double Agc::computeInitialY(IPAFrameContext &frameContext, + const ipu3_uapi_grid_config &grid, + const ipu3_uapi_stats_3a *stats, + double currentYGain) +{ + double redSum = 0, greenSum = 0, blueSum = 0; + + for (unsigned int cellY = 0; cellY < grid.height; cellY++) { + for (unsigned int cellX = 0; cellX < grid.width; cellX++) { + uint32_t cellPosition = cellY * stride_ + cellX; + + const ipu3_uapi_awb_set_item *cell = + reinterpret_cast( + &stats->awb_raw_buffer.meta_data[cellPosition] + ); + + redSum += cell->R_avg * currentYGain; + greenSum += (cell->Gr_avg + cell->Gb_avg) / 2 * currentYGain; + blueSum += cell->B_avg * currentYGain; + } + } + + /* + * Estimate the sum of the brightness values, weighted with the gains + * applied on the channels in AWB. + */ + double Y_sum = redSum * frameContext.awb.gains.red * .299 + + greenSum * frameContext.awb.gains.green * .587 + + blueSum * frameContext.awb.gains.blue * .114; + + /* And return the average brightness */ + return Y_sum / (grid.height * grid.width); +} + + /** * \brief Process IPU3 statistics, and run AGC operations * \param[in] context The shared IPA context @@ -267,7 +318,29 @@ void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) uint32_t &exposure = context.frameContext.agc.exposure; double &analogueGain = context.frameContext.agc.gain; measureBrightness(stats, context.configuration.grid.bdsGrid); - computeExposure(exposure, analogueGain); + + double currentYGain = 1.0; + /* \todo: the target Y needs to be grabbed from a configuration */ + double targetY = 60; + /* + * Do this calculation a few times as brightness increase can be + * non-linear when there are saturated regions. + */ + for (int i = 0; i < 8; i++) { + double initialY = computeInitialY(context.frameContext, + context.configuration.grid.bdsGrid, + stats, currentYGain); + double extra_gain = std::min(10.0, targetY / (initialY + .001)); + + currentYGain *= extra_gain; + LOG(IPU3Agc, Debug) << "Initial Y " << initialY + << " target " << targetY + << " gives gain " << currentYGain; + if (extra_gain < 1.01) + break; + } + + computeExposure(exposure, analogueGain, currentYGain); frameCount_++; } diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 69e0b831..0a9152a9 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -34,7 +34,11 @@ private: void measureBrightness(const ipu3_uapi_stats_3a *stats, const ipu3_uapi_grid_config &grid); void filterExposure(); - void computeExposure(uint32_t &exposure, double &gain); + void computeExposure(uint32_t &exposure, double &gain, double currentYGain); + double computeInitialY(IPAFrameContext &frameContext, + const ipu3_uapi_grid_config &grid, + const ipu3_uapi_stats_3a *stats, + double currentYGain); uint64_t frameCount_; uint64_t lastFrame_; From patchwork Mon Nov 8 13:13:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14478 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 35F94C324F for ; Mon, 8 Nov 2021 13:14:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B65A160362; Mon, 8 Nov 2021 14:14:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="S+mVIz0q"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D6AE060369 for ; Mon, 8 Nov 2021 14:14:00 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 98EC8187C; Mon, 8 Nov 2021 14:14:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377240; bh=C/oCPXfue5HN2TLOPr7mAjPhdhZpXtoy5/swaT55odc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S+mVIz0qFE/Vk+P4k6xHvE3j1XNRyylHcYERLeoRsmUrPNTwtvGB3Nmp7xJ/8Fvoz 3yRQ154t2ZaG+Wabk+ZC79yc9PfvoK+z1kI8JMhD2JMtAvq6/vr6+3N5LqdxFd5pbl bfMjqwJv4Ap4uYPYbY781GOC6B7H2IQJ6aSFAbMM= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:36 +0100 Message-Id: <20211108131350.130665-9-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 08/22] ipa: ipu3: agc: Update previous exposure value X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Previous exposure value is calculated based on the estimated shutter time and gain applied. Now that we have the real values for the current frame, use those before estimating the next one. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/agc.cpp | 19 +++---------------- 1 file changed, 3 insertions(+), 16 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index e4048d40..475e715f 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -193,15 +193,12 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain, double curre /* Estimate the gain needed to have the proportion wanted */ double evGain = kEvGainTarget * knumHistogramBins / iqMean_; - if (std::abs(evGain - 1.0) < 0.01) { - LOG(IPU3Agc, Debug) << "We are well exposed (iqMean = " - << iqMean_ << ")"; - return; - } - /* extracted from Rpi::Agc::computeTargetExposure */ /* Calculate the shutter time in seconds */ utils::Duration currentShutter = exposure * lineDuration_; + /* Update the exposure value for the next computation. */ + prevExposureValue_ = currentShutter * analogueGain; + LOG(IPU3Agc, Debug) << "Actual total exposure " << currentShutter * analogueGain << " Shutter speed " << currentShutter << " Gain " << analogueGain @@ -250,16 +247,6 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain, double curre exposure = shutterTime / lineDuration_; analogueGain = stepGain; - - /* - * Update the exposure value for the next process call. - * - * \todo Obtain the values of the exposure time and analog gain - * that were actually used by the sensor, either from embedded - * data when available, or from the delayed controls - * infrastructure in case a slow down caused a mismatch. - */ - prevExposureValue_ = shutterTime * analogueGain; } /** From patchwork Mon Nov 8 13:13:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14477 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A07C3C324E for ; Mon, 8 Nov 2021 13:14:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9F4EB60368; Mon, 8 Nov 2021 14:14:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jZ6vewz6"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0971A6036B for ; Mon, 8 Nov 2021 14:14:01 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C1DB11A38; Mon, 8 Nov 2021 14:14:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377240; bh=wEuvknCICgZ9EnicDevnBfry3lxBAeajNBFAWKpA2NI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jZ6vewz6gMRynI39IM5Gj6ynD98LutQWufFbd9+FcN/aaQRAf7CexMqK6jKpijV60 NosJ3GgnH8ubR77lxe6Y0YyyrClkCoApJFG2arOiMqsrSo2g+nUft1v0+82gqSrW31 NXVgD7OW7R0eyxxaqjA1WwN28XgX/yjrVEQLDm6E= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:37 +0100 Message-Id: <20211108131350.130665-10-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 09/22] ipa: ipu3: agc: Use exposure in time for storage X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The minimum and maximum exposure are stored in lines. Replace it by values in time, which removes a bit of code. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/agc.cpp | 21 +++++++++------------ src/ipa/ipu3/algorithms/agc.h | 4 ++-- 2 files changed, 11 insertions(+), 14 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 475e715f..3a15f5d9 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -68,8 +68,8 @@ static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 20 / 100; static constexpr uint32_t kNumStartupFrames = 10; Agc::Agc() - : frameCount_(0), iqMean_(0.0), lineDuration_(0s), minExposureLines_(0), - maxExposureLines_(0), filteredExposure_(0s), currentExposure_(0s), + : frameCount_(0), iqMean_(0.0), lineDuration_(0s), minShutterSpeed_(0s), + maxShutterSpeed_(0s), filteredExposure_(0s), currentExposure_(0s), prevExposureValue_(0s) { } @@ -89,17 +89,16 @@ int Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) lineDuration_ = configInfo.sensorInfo.lineLength * 1.0s / configInfo.sensorInfo.pixelRate; - /* \todo replace the exposure in lines storage with time based ones. */ - minExposureLines_ = context.configuration.agc.minShutterSpeed / lineDuration_; - maxExposureLines_ = std::min(context.configuration.agc.maxShutterSpeed / lineDuration_, - kMaxShutterSpeed / lineDuration_); + minShutterSpeed_ = context.configuration.agc.minShutterSpeed; + maxShutterSpeed_ = std::min(context.configuration.agc.maxShutterSpeed, + kMaxShutterSpeed); minAnalogueGain_ = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain); maxAnalogueGain_ = std::min(context.configuration.agc.maxAnalogueGain, kMaxAnalogueGain); /* Configure the default exposure and gain. */ context.frameContext.agc.gain = minAnalogueGain_; - context.frameContext.agc.exposure = minExposureLines_; + context.frameContext.agc.exposure = minShutterSpeed_ / lineDuration_; prevExposureValue_ = context.frameContext.agc.gain * context.frameContext.agc.exposure @@ -217,11 +216,9 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain, double curre * exposure value applied multiplied by the new estimated gain. */ currentExposure_ = prevExposureValue_ * evGain; - utils::Duration minShutterSpeed = minExposureLines_ * lineDuration_; - utils::Duration maxShutterSpeed = maxExposureLines_ * lineDuration_; /* Clamp the exposure value to the min and max authorized */ - utils::Duration maxTotalExposure = maxShutterSpeed * maxAnalogueGain_; + utils::Duration maxTotalExposure = maxShutterSpeed_ * maxAnalogueGain_; currentExposure_ = std::min(currentExposure_, maxTotalExposure); LOG(IPU3Agc, Debug) << "Target total exposure " << currentExposure_ << ", maximum is " << maxTotalExposure; @@ -231,14 +228,14 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain, double curre /* Divide the exposure value as new exposure and gain values */ utils::Duration exposureValue = filteredExposure_; - utils::Duration shutterTime = minShutterSpeed; + utils::Duration shutterTime = minShutterSpeed_; /* * Push the shutter time up to the maximum first, and only then * increase the gain. */ shutterTime = std::clamp(exposureValue / minAnalogueGain_, - minShutterSpeed, maxShutterSpeed); + minShutterSpeed_, maxShutterSpeed_); double stepGain = std::clamp(exposureValue / shutterTime, minAnalogueGain_, maxAnalogueGain_); LOG(IPU3Agc, Debug) << "Divided up shutter and gain are " diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 0a9152a9..6f5d71e0 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -46,8 +46,8 @@ private: double iqMean_; utils::Duration lineDuration_; - uint32_t minExposureLines_; - uint32_t maxExposureLines_; + utils::Duration minShutterSpeed_; + utils::Duration maxShutterSpeed_; double minAnalogueGain_; double maxAnalogueGain_; From patchwork Mon Nov 8 13:13:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14479 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A9C13C3250 for ; Mon, 8 Nov 2021 13:14:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3B4EA60451; Mon, 8 Nov 2021 14:14:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="QSFJnQR0"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 327EB6036E for ; Mon, 8 Nov 2021 14:14:01 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EBEE218CE; Mon, 8 Nov 2021 14:14:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377241; bh=Pai8YVjZRK0wB27hmZvREjBbgZ7WIqWptOPn3cGDj4M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QSFJnQR0PsGUMT1ydykjKlcV151bUK7fP6DuWBFLeg+diNFZonYS7y922np7/Ni/Q oyJn6P9zVNkYAFdywuHQDcAfcxPjVhrtlf8TgkZ50XhxfPsLqYG/leDitcOlTZomC4 BhPDhsj96pmlW0c9o6aM3GKLb7SXe6fJ7Mmzfz6E= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:38 +0100 Message-Id: <20211108131350.130665-11-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 10/22] ipu3: Move delayedControls reset after IPA is started X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We can call configure multiple times and we don't need to reset delayedControls for each of the configure calls. Reset it after the IPA is started. Signed-off-by: Jean-Michel Hautbois --- src/libcamera/pipeline/ipu3/ipu3.cpp | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 6a7f5b9a..3fcfa777 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -667,8 +667,6 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) return ret; } - data->delayedCtrls_->reset(); - return updateControls(data); } @@ -769,6 +767,12 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis if (ret) goto error; + /* + * Reset the delayed controls with the gain and exposure values set by + * the IPA. + */ + data->delayedCtrls_->reset(); + /* * Start the ImgU video devices, buffers will be queued to the * ImgU output and viewfinder when requests will be queued. From patchwork Mon Nov 8 13:13:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14480 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3A71FBDB1C for ; Mon, 8 Nov 2021 13:14:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CA26E60497; Mon, 8 Nov 2021 14:14:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jAVx0zpz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6839860361 for ; Mon, 8 Nov 2021 14:14:01 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 21A1618F6; Mon, 8 Nov 2021 14:14:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377241; bh=JWmVe5SdOqLQGnYyPzoipGFVdrcBUx1c0Hd0TiPDt/4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jAVx0zpzHGC4GmSWuSy8TGkA0Ggr236PA4Q7rPkH7Thm19lUcW1TsUnVundp1cOT5 TDTQr1UnqQr3Hr9ZeR+x83dVnQL43bsi39GxjLlwehEPXRTNuhpu5LZxrsEyLi6vjn j+zxCGOZJnfxuukD21Gv76bMfTHToqsOw7xOaSyw= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:39 +0100 Message-Id: <20211108131350.130665-12-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 11/22] ipu3: Populate the EventProcessCall with the sensor controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When EventProcessCall is to be sent, we populate the controls and notify the IPA. Add the sensor controls in the event to notify their values for the given frame. Signed-off-by: Jean-Michel Hautbois --- src/libcamera/pipeline/ipu3/ipu3.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 3fcfa777..5d87f6e5 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -862,6 +862,7 @@ void IPU3CameraData::queuePendingRequests() ev.op = ipa::ipu3::EventProcessControls; ev.frame = info->id; ev.controls = request->controls(); + ev.sensorControls = delayedCtrls_->get(ev.frame); ipa_->processEvent(ev); pendingRequests_.pop(); From patchwork Mon Nov 8 13:13:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14482 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 53FF2C324F for ; Mon, 8 Nov 2021 13:14:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D0C2F6048E; Mon, 8 Nov 2021 14:14:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gZh6d74A"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9540E60378 for ; Mon, 8 Nov 2021 14:14:01 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 568741ACA; Mon, 8 Nov 2021 14:14:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377241; bh=+ft/Zw5mEnoyI6JC+5VXkxbWd8+28J0UfwAtYsExxVs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gZh6d74A4KgMPmsHOFduy+u1XqrqN73hfWQLQaML1dfNPU2eihb3v6GtHk6hlt523 2+6TU8sjV5FuElGO5ZStl8zKYtq/22BeGjt99jFOnqalKPiQDyUF5n1BSa1IyAYSnN W1JxGXRCdlpjCR4qR4fAJSYa1M+6kvzMeMsKVZZc= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:40 +0100 Message-Id: <20211108131350.130665-13-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 12/22] ipa: ipu3: Introduce a frame index for the IPAFrameContext X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As a preliminery to the usage of a queue of IPAFrameContext introduce a frame index. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipa_context.cpp | 5 +++++ src/ipa/ipu3/ipa_context.h | 2 ++ 2 files changed, 7 insertions(+) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 2355a9c7..ea92be77 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -133,4 +133,9 @@ namespace libcamera::ipa::ipu3 { * struct ipu3_uapi_gamma_corr_lut for further details. */ +/** + * \var IPAFrameContext::frameId + * \brief Frame number for this frame context + */ + } /* namespace libcamera::ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 1e46c61a..ee8f7b55 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -51,6 +51,8 @@ struct IPAFrameContext { double gamma; struct ipu3_uapi_gamma_corr_lut gammaCorrection; } toneMapping; + + unsigned int frameId; }; struct IPAContext { From patchwork Mon Nov 8 13:13:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14481 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9F202C324E for ; Mon, 8 Nov 2021 13:14:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 554016038C; Mon, 8 Nov 2021 14:14:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="EO1nxSlj"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BCDE46034A for ; Mon, 8 Nov 2021 14:14:01 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7F7041BBB; Mon, 8 Nov 2021 14:14:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377241; bh=RfWb2qBLypl+lelXAetoohvrf7uE+ckZ1qI+84ozJhY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EO1nxSljFm6l1E2pdy8AVSpv1jbbtCkm+ocjYE9vPYavDImXLchwV8VeQavtMLEFA 5luT8TGaSARSWBJ7vDInc66OItauTsaO1HJGQIXiO+iSn7U/O4P9BT3iH9cudOw51q ZrLqYCvUSt3EE2uxdRYUvAWAcMkxiMGxG88OY+3Y= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:41 +0100 Message-Id: <20211108131350.130665-14-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 13/22] ipa: ipu3: Do not access IPAFrameContext in configure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The IPAContext contains a SessionConfiguration and a FrameContext. The configure call can populate the SessionConfiguration, but there is not yet a frame. Do not use the IPAFrameContext to prepare the next patches which will introduce per-frame controls. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/agc.cpp | 8 -------- src/ipa/ipu3/algorithms/tone_mapping.cpp | 5 +---- 2 files changed, 1 insertion(+), 12 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 3a15f5d9..52cf2753 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -96,14 +96,6 @@ int Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) minAnalogueGain_ = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain); maxAnalogueGain_ = std::min(context.configuration.agc.maxAnalogueGain, kMaxAnalogueGain); - /* Configure the default exposure and gain. */ - context.frameContext.agc.gain = minAnalogueGain_; - context.frameContext.agc.exposure = minShutterSpeed_ / lineDuration_; - - prevExposureValue_ = context.frameContext.agc.gain - * context.frameContext.agc.exposure - * lineDuration_; - return 0; } diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 2040eda5..5d74c552 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -38,12 +38,9 @@ ToneMapping::ToneMapping() * * \return 0 */ -int ToneMapping::configure(IPAContext &context, +int ToneMapping::configure([[maybe_unused]] IPAContext &context, [[maybe_unused]] const IPAConfigInfo &configInfo) { - /* Initialise tone mapping gamma value. */ - context.frameContext.toneMapping.gamma = 0.0; - return 0; } From patchwork Mon Nov 8 13:13:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14483 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 659B9C3250 for ; Mon, 8 Nov 2021 13:14:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ED6BF603B5; Mon, 8 Nov 2021 14:14:15 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="NrPyoC/A"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 35F4F6037A for ; Mon, 8 Nov 2021 14:14:02 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A9D2D1BE8; Mon, 8 Nov 2021 14:14:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377241; bh=la0uzaOGE8NdQMnZyS4sV7mdK16aDXWS2hZF/H+EIyA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NrPyoC/AHvzEAkUlWo9MB1u5Fznm2apoa4ON6wLScjO6amYGvCv0DMzO5Zn7utRWj fT/5pmhHDM/7PgL5MbnsktmRVaBy8evyfQLp3vSggde7Fvxzhn8niEiGLnhv9PaJT7 WpAKmOyCBgpTcUF5SItUQ5j/2jVHZZgMAaC8YzDg= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:42 +0100 Message-Id: <20211108131350.130665-15-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 14/22] ipa: ipu3: Introduce a queue of IPAFrameContext X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to have multiple frame contexts, we will need a queue to store those. Introduce it in IPAIPU3 but do not use it in this commit yet. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipu3.cpp | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index b4e0d49f..62df7819 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -180,7 +181,10 @@ private: /* Maintain the algorithms used by the IPA */ std::list> algorithms_; - /* Local parameter storage */ + /* Queue for storing the per-frame context */ + std::queue frameContextQueue; + + /* Local context storage */ struct IPAContext context_; }; From patchwork Mon Nov 8 13:13:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14484 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0CB25BDB1C for ; Mon, 8 Nov 2021 13:14:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A7C166034A; Mon, 8 Nov 2021 14:14:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XFPLJ0Yg"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 612736036C for ; Mon, 8 Nov 2021 14:14:02 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1AB971853; Mon, 8 Nov 2021 14:14:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377242; bh=47OtOxB8pGy1t3K0IQ656RXKkOy86iLRbjLJf3XaLac=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XFPLJ0YgGo4iCIqC9so58FOQcblOGRuVMERkGBDDVkHFOc0Rf5PrTwBiOMwURdNZF MrjjtbJS8N4bOymp9U96rqj8dVwezGd0SL8sNiA49lQ1nxG8ioIu+vvy3KMnwI0GTe alZXnm1OpSK2fh7KgeRLDFkYihcZqE+L71jsrC7I= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:43 +0100 Message-Id: <20211108131350.130665-16-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 15/22] ipa: ipu3: Mark the beginning and and of a frame X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Introduce the skeleton for two functions which will be used to instantiate a frame context, and do everything needed when a frame is received. Do the same for the other end, once the algorithms have run and updated the frame context to later deallocate the corresponding frame context. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipu3.cpp | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 62df7819..dcf4da65 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -160,6 +160,14 @@ private: void setControls(unsigned int frame); void calculateBdsGrid(const Size &bdsOutputSize); + /* + * Internal events that mark the beginning of processing a new frame + * to the point that it has successfully completed processing its + * statistics. + */ + void frameStarted(unsigned int frame); + void frameCompleted(unsigned int frame); + std::map buffers_; ControlInfoMap ctrls_; @@ -510,6 +518,14 @@ void IPAIPU3::unmapBuffers(const std::vector &ids) } } +void IPAIPU3::frameStarted([[maybe_unused]] unsigned int frame) +{ +} + +void IPAIPU3::frameCompleted([[maybe_unused]] unsigned int frame) +{ +} + /** * \brief Process an event generated by the pipeline handler * \param[in] event The event sent from pipeline handler @@ -525,6 +541,14 @@ void IPAIPU3::processEvent(const IPU3Event &event) { switch (event.op) { case EventProcessControls: { + /* + * To save incurring extra IPC calls, we do not send explicit events + * when a new request is started or completed. + * ProcessControls is the first event handled upon receipt of a new + * request, so we can handle all actions required to start processing + * a new frame. + */ + frameStarted(event.frame); processControls(event.frame, event.controls); break; } @@ -558,6 +582,13 @@ void IPAIPU3::processEvent(const IPU3Event &event) context_.frameContext.agc.gain = camHelper_->gain(event.sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); parseStatistics(event.frame, event.frameTimestamp, stats); + /* + * To save incurring extra IPC calls, we do not send explicit events + * when we have completed all handling of a request. + * Once the statistics are fully processed, we will no longer handle this + * frame. + */ + frameCompleted(event.frame); break; } default: From patchwork Mon Nov 8 13:13:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14485 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9C508C3251 for ; Mon, 8 Nov 2021 13:14:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 494BD603C4; Mon, 8 Nov 2021 14:14:17 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="AzVY9L4M"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 892176038C for ; Mon, 8 Nov 2021 14:14:02 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 44F76190C; Mon, 8 Nov 2021 14:14:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377242; bh=EtwI61Kvv4r1oZ635E9dmb6qP9qEYqj11s+iLZiyn2w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AzVY9L4MBAHbBkFj7ibvBqIwMRyOC7N7ebVVKdcJUhZDRaDN6EHMOdVojUWiATBsF +OeRPVk0+aoT1jMmrnljwPne2FKMDkR7iu7JJJ/fEw6nyFkfbZEIcX+WD4o6G3XpFD LAzU0B8mxJlfu5OXMJHMBCQ/L4PAtRWXcwKDxlBU= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:44 +0100 Message-Id: <20211108131350.130665-17-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 16/22] ipa: ipu3: Introduce sensor controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Pass the sensor controls as well as the libcamera controls to processControl() on an EventProcessControls event received. This will be used to populate the frame context. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipu3.cpp | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index dcf4da65..177c5c2f 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -151,7 +151,9 @@ private: ControlInfoMap *ipaControls); void updateSessionConfiguration(const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls); - void processControls(unsigned int frame, const ControlList &controls); + void processControls(unsigned int frame, + const ControlList &controls, + const ControlList &sensorCtrls); void fillParams(unsigned int frame, ipu3_uapi_params *params); void parseStatistics(unsigned int frame, int64_t frameTimestamp, @@ -549,7 +551,7 @@ void IPAIPU3::processEvent(const IPU3Event &event) * a new frame. */ frameStarted(event.frame); - processControls(event.frame, event.controls); + processControls(event.frame, event.controls, event.sensorControls); break; } case EventFillParams: { @@ -601,12 +603,14 @@ void IPAIPU3::processEvent(const IPU3Event &event) * \brief Process a control list for a request from the application * \param[in] frame The number of the frame which will be processed next * \param[in] controls The controls for the \a frame + * \param[in] controls The sensor controls for the \a frame * * Parse the request to handle any IPA-managed controls that were set from the * application such as manual sensor settings. */ void IPAIPU3::processControls([[maybe_unused]] unsigned int frame, - [[maybe_unused]] const ControlList &controls) + [[maybe_unused]] const ControlList &controls, + [[maybe_unused]] const ControlList &sensorCtrls) { /* \todo Start processing for 'frame' based on 'controls'. */ } From patchwork Mon Nov 8 13:13:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14486 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 270ACC324E for ; Mon, 8 Nov 2021 13:14:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C94A960494; Mon, 8 Nov 2021 14:14:17 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="OmN7yEkt"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B88BF6036F for ; Mon, 8 Nov 2021 14:14:02 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 77C16110E; Mon, 8 Nov 2021 14:14:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377242; bh=Hu5ecbd7XqF8iAIMtPJo8bSofYYRY2Qk2zP2CMlxgF0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OmN7yEktdAm4ADi12RaI7cCD0D/2klt73LRmupkqz75IIibRwiCs6wyXPXguv2si6 QgyZPxSwOeFl42HXJEi34nspkndlLgmmPnhURgxPTGgQEi7Jdhbbj1Bgy2ljs4WCt2 4dryd1sTycA1ab7Eec7h9OH3fqmwipY/aBM8C3hU= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:45 +0100 Message-Id: <20211108131350.130665-18-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 17/22] ipa: ipu3: Use a IPAFrameContext pointer from the per-frame queue X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We have one frame context shared between the algorithms thanks to a local cached context_ variable in IPAIPU3. Now that we can store the frame contexts in a queue, implement all the needed functions for that and convert the frame context to a pointer. The algorithm are now using the values applied on the frame they are processing, and not the latest one. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/agc.cpp | 12 ++++---- src/ipa/ipu3/algorithms/agc.h | 2 +- src/ipa/ipu3/algorithms/awb.cpp | 14 ++++----- src/ipa/ipu3/algorithms/tone_mapping.cpp | 8 ++--- src/ipa/ipu3/ipa_context.h | 2 +- src/ipa/ipu3/ipu3.cpp | 38 +++++++++++++++++++----- 6 files changed, 49 insertions(+), 27 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 52cf2753..2eee5b6b 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -245,7 +245,7 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain, double curre * \param[in] stats The IPU3 statistics and ISP results * \param[in] currentYGain The gain calculated on the current brightness level */ -double Agc::computeInitialY(IPAFrameContext &frameContext, +double Agc::computeInitialY(IPAFrameContext *frameContext, const ipu3_uapi_grid_config &grid, const ipu3_uapi_stats_3a *stats, double currentYGain) @@ -271,9 +271,9 @@ double Agc::computeInitialY(IPAFrameContext &frameContext, * Estimate the sum of the brightness values, weighted with the gains * applied on the channels in AWB. */ - double Y_sum = redSum * frameContext.awb.gains.red * .299 + - greenSum * frameContext.awb.gains.green * .587 + - blueSum * frameContext.awb.gains.blue * .114; + double Y_sum = redSum * frameContext->awb.gains.red * .299 + + greenSum * frameContext->awb.gains.green * .587 + + blueSum * frameContext->awb.gains.blue * .114; /* And return the average brightness */ return Y_sum / (grid.height * grid.width); @@ -291,8 +291,8 @@ double Agc::computeInitialY(IPAFrameContext &frameContext, void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { /* Get the latest exposure and gain applied */ - uint32_t &exposure = context.frameContext.agc.exposure; - double &analogueGain = context.frameContext.agc.gain; + uint32_t &exposure = context.frameContext->agc.exposure; + double &analogueGain = context.frameContext->agc.gain; measureBrightness(stats, context.configuration.grid.bdsGrid); double currentYGain = 1.0; diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 6f5d71e0..5d6bef9d 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -35,7 +35,7 @@ private: const ipu3_uapi_grid_config &grid); void filterExposure(); void computeExposure(uint32_t &exposure, double &gain, double currentYGain); - double computeInitialY(IPAFrameContext &frameContext, + double computeInitialY(IPAFrameContext *frameContext, const ipu3_uapi_grid_config &grid, const ipu3_uapi_stats_3a *stats, double currentYGain); diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index a4114659..bd55d377 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -382,9 +382,9 @@ void Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) * The results are cached, so if no results were calculated, we set the * cached values from asyncResults_ here. */ - context.frameContext.awb.gains.blue = asyncResults_.blueGain; - context.frameContext.awb.gains.green = asyncResults_.greenGain; - context.frameContext.awb.gains.red = asyncResults_.redGain; + context.frameContext->awb.gains.blue = asyncResults_.blueGain; + context.frameContext->awb.gains.green = asyncResults_.greenGain; + context.frameContext->awb.gains.red = asyncResults_.redGain; } constexpr uint16_t Awb::threshold(float value) @@ -432,10 +432,10 @@ void Awb::prepare(IPAContext &context, ipu3_uapi_params *params) params->acc_param.bnr.opt_center_sqr.y_sqr_reset = params->acc_param.bnr.opt_center.y_reset * params->acc_param.bnr.opt_center.y_reset; /* Convert to u3.13 fixed point values */ - params->acc_param.bnr.wb_gains.gr = 8192 * context.frameContext.awb.gains.green; - params->acc_param.bnr.wb_gains.r = 8192 * context.frameContext.awb.gains.red; - params->acc_param.bnr.wb_gains.b = 8192 * context.frameContext.awb.gains.blue; - params->acc_param.bnr.wb_gains.gb = 8192 * context.frameContext.awb.gains.green; + params->acc_param.bnr.wb_gains.gr = 8192 * context.frameContext->awb.gains.green; + params->acc_param.bnr.wb_gains.r = 8192 * context.frameContext->awb.gains.red; + params->acc_param.bnr.wb_gains.b = 8192 * context.frameContext->awb.gains.blue; + params->acc_param.bnr.wb_gains.gb = 8192 * context.frameContext->awb.gains.green; LOG(IPU3Awb, Debug) << "Color temperature estimated: " << asyncResults_.temperatureK; diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 5d74c552..50498f41 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -57,7 +57,7 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, { /* Copy the calculated LUT into the parameters buffer. */ memcpy(params->acc_param.gamma.gc_lut.lut, - context.frameContext.toneMapping.gammaCorrection.lut, + context.frameContext->toneMapping.gammaCorrection.lut, IPU3_UAPI_GAMMA_CORR_LUT_ENTRIES * sizeof(params->acc_param.gamma.gc_lut.lut[0])); @@ -84,11 +84,11 @@ void ToneMapping::process(IPAContext &context, */ gamma_ = 1.1; - if (context.frameContext.toneMapping.gamma == gamma_) + if (context.frameContext->toneMapping.gamma == gamma_) return; struct ipu3_uapi_gamma_corr_lut &lut = - context.frameContext.toneMapping.gammaCorrection; + context.frameContext->toneMapping.gammaCorrection; for (uint32_t i = 0; i < std::size(lut.lut); i++) { double j = static_cast(i) / (std::size(lut.lut) - 1); @@ -98,7 +98,7 @@ void ToneMapping::process(IPAContext &context, lut.lut[i] = gamma * 8191; } - context.frameContext.toneMapping.gamma = gamma_; + context.frameContext->toneMapping.gamma = gamma_; } } /* namespace ipa::ipu3::algorithms */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index ee8f7b55..69780915 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -57,7 +57,7 @@ struct IPAFrameContext { struct IPAContext { IPASessionConfiguration configuration; - IPAFrameContext frameContext; + IPAFrameContext *frameContext; }; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 177c5c2f..b60ab7e7 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -522,10 +522,26 @@ void IPAIPU3::unmapBuffers(const std::vector &ids) void IPAIPU3::frameStarted([[maybe_unused]] unsigned int frame) { + /* + * Instantiate a new IPAFrameContext to push to the queue. The lifetime + * of this pointer is indirectly controlled by the + * ActionSetSensorControls event, as this is where we will delete it. + */ + struct IPAFrameContext *frameContext = new IPAFrameContext; + frameContext->frameId = frame; + frameContextQueue.push(frameContext); } void IPAIPU3::frameCompleted([[maybe_unused]] unsigned int frame) { + /* + * Remove the pointer from the queue, it should not be accessed + * anymore and delete it. + */ + struct IPAFrameContext *frameContext = frameContextQueue.front(); + ASSERT(frameContext->frameId == frame); + frameContextQueue.pop(); + delete frameContext; } /** @@ -579,10 +595,6 @@ void IPAIPU3::processEvent(const IPU3Event &event) const ipu3_uapi_stats_3a *stats = reinterpret_cast(mem.data()); - /* \todo move those into processControls */ - context_.frameContext.agc.exposure = event.sensorControls.get(V4L2_CID_EXPOSURE).get(); - context_.frameContext.agc.gain = camHelper_->gain(event.sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); - parseStatistics(event.frame, event.frameTimestamp, stats); /* * To save incurring extra IPC calls, we do not send explicit events @@ -610,9 +622,12 @@ void IPAIPU3::processEvent(const IPU3Event &event) */ void IPAIPU3::processControls([[maybe_unused]] unsigned int frame, [[maybe_unused]] const ControlList &controls, - [[maybe_unused]] const ControlList &sensorCtrls) + const ControlList &sensorCtrls) { - /* \todo Start processing for 'frame' based on 'controls'. */ + struct IPAFrameContext *frameContext = frameContextQueue.back(); + ASSERT(frameContext->frameId == frame); + frameContext->agc.exposure = sensorCtrls.get(V4L2_CID_EXPOSURE).get(); + frameContext->agc.gain = camHelper_->gain(sensorCtrls.get(V4L2_CID_ANALOGUE_GAIN).get()); } /** @@ -636,6 +651,9 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params) */ params->use = {}; + /* Set the reference to the current frame context */ + context_.frameContext = frameContextQueue.front(); + for (auto const &algo : algorithms_) algo->prepare(context_, params); @@ -661,6 +679,9 @@ void IPAIPU3::parseStatistics(unsigned int frame, { ControlList ctrls(controls::controls); + /* Set the reference to the current frame context */ + context_.frameContext = frameContextQueue.front(); + for (auto const &algo : algorithms_) algo->process(context_, stats); @@ -690,8 +711,9 @@ void IPAIPU3::setControls(unsigned int frame) IPU3Action op; op.op = ActionSetSensorControls; - exposure_ = context_.frameContext.agc.exposure; - gain_ = camHelper_->gainCode(context_.frameContext.agc.gain); + /* Apply the exposure and gain updated values */ + exposure_ = context_.frameContext->agc.exposure; + gain_ = camHelper_->gainCode(context_.frameContext->agc.gain); ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); From patchwork Mon Nov 8 13:13:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14487 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 991BEC3252 for ; Mon, 8 Nov 2021 13:14:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4D7806049B; Mon, 8 Nov 2021 14:14:18 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="O66lnqQr"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E694060398 for ; Mon, 8 Nov 2021 14:14:02 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A187B187C; Mon, 8 Nov 2021 14:14:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377242; bh=rg5fIYEY9UfUrV1R3AkxL5UAWNSg3Nzf/MnmXTE7/P4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=O66lnqQrwkulj98zx8CpJcpLXb/jSetQ2aQdXSgMkv3j6BYP5tM5ORLShAmSG7ig5 Hcbvw8XCyAqJYOrm81SSh9ARLaURWpwGklNfOAWa8VowmUqc84OGSWtLfGEhZdYD7v 8kMQmHgX6tWZZhfS54ys6bn+LI9/bxg8GP7t4DcI= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:46 +0100 Message-Id: <20211108131350.130665-19-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 18/22] ipa: ipu3: Move the sensor update controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We want the setControls() to use the cached exposure_ and gain_ values, so move the frame context update of those variables into frameCompleted() call instead. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipu3.cpp | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index b60ab7e7..9b20e8ab 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -534,6 +534,10 @@ void IPAIPU3::frameStarted([[maybe_unused]] unsigned int frame) void IPAIPU3::frameCompleted([[maybe_unused]] unsigned int frame) { + /* Apply the exposure and gain updated values */ + exposure_ = context_.frameContext->agc.exposure; + gain_ = camHelper_->gainCode(context_.frameContext->agc.gain); + /* * Remove the pointer from the queue, it should not be accessed * anymore and delete it. @@ -711,10 +715,6 @@ void IPAIPU3::setControls(unsigned int frame) IPU3Action op; op.op = ActionSetSensorControls; - /* Apply the exposure and gain updated values */ - exposure_ = context_.frameContext->agc.exposure; - gain_ = camHelper_->gainCode(context_.frameContext->agc.gain); - ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); From patchwork Mon Nov 8 13:13:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14489 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6AF4FC3250 for ; Mon, 8 Nov 2021 13:14:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1C76D60398; Mon, 8 Nov 2021 14:14:20 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cIQYOar3"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 471A36035D for ; Mon, 8 Nov 2021 14:14:03 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CBEB31A38; Mon, 8 Nov 2021 14:14:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377242; bh=cbMbwEVNYJgf7LSFwCuIfXkF0PIusRSEgf4QABqntVM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cIQYOar324kd4z5xCxL0BgLossqQY3d+gf1GlnNd3ty0ouFxfciuEGdw26KI9jHP6 9+Q4S2hyEyStCHBrsYBPk40zR7HRMjgDZKbmsa7mcq5Yr/ZZotOMSvkrtVZpUf8ljF hd6pvfZAv2UwSz5hUnPJvU9cQ0hHldUk0/38N9lI= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:47 +0100 Message-Id: <20211108131350.130665-20-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 19/22] ipa: ipu3: awb: Add support for color temperature X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AWB estimates the color temperature, but it is not used at all. It can be useful for debug purpose at least, but also for lux estimation later, to be able to know the temperature estimated for a given frame. Add a new member to the IPAFrameContext::awb for this purpose, and update the value in AWB. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/awb.cpp | 1 + src/ipa/ipu3/ipa_context.cpp | 3 +++ src/ipa/ipu3/ipa_context.h | 2 ++ 3 files changed, 6 insertions(+) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index bd55d377..a3a5ac66 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -385,6 +385,7 @@ void Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) context.frameContext->awb.gains.blue = asyncResults_.blueGain; context.frameContext->awb.gains.green = asyncResults_.greenGain; context.frameContext->awb.gains.red = asyncResults_.redGain; + context.frameContext->awb.temperatureK = asyncResults_.temperatureK; } constexpr uint16_t Awb::threshold(float value) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index ea92be77..9c0407df 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -117,6 +117,9 @@ namespace libcamera::ipa::ipu3 { * * \var IPAFrameContext::awb.gains.blue * \brief White balance gain for B channel + * + * \var IPAFrameContext::awb.temperatureK + * \brief Color temperature estimated */ /** diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 69780915..3566dea1 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -45,6 +45,8 @@ struct IPAFrameContext { double green; double blue; } gains; + + double temperatureK; } awb; struct { From patchwork Mon Nov 8 13:13:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14490 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DD7B5BDB1C for ; Mon, 8 Nov 2021 13:14:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8D44E6037A; Mon, 8 Nov 2021 14:14:20 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="NF7OF7DU"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 74E8B603B5 for ; Mon, 8 Nov 2021 14:14:03 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 36E891C58; Mon, 8 Nov 2021 14:14:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377243; bh=wCUA4iq0IYhYz3m+M7QrbY5vrLAPpFiP6YEnSdaBTrk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NF7OF7DU0XcONLMdBUo5aaiEQ8n+3UlxiM57V1QjnbRYMvfN1mDBMiszrzHuIAKPE YCwO+MDTCstEk71cz0n2l51cTbY6O3yzUF8siKlNMsxAaiR+1o4liMz1aIty8HVavf E3uYRqwuEPyFJneiQWCvomCeuL1YV7Rb5tmoKFNA= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:48 +0100 Message-Id: <20211108131350.130665-21-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 20/22] ipa: ipu3: Send color temperature in the metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that the color temperature is updated per-frame, use the value and set the corresponding controls::ColourTemperature. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipu3.cpp | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 9b20e8ab..ca3f2417 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -696,6 +696,14 @@ void IPAIPU3::parseStatistics(unsigned int frame, (sensorInfo_.pixelRate / 1e6); ctrls.set(controls::FrameDuration, frameDuration); + ctrls.set(controls::ColourTemperature, context_.frameContext->awb.temperatureK); + + /* + * \todo We should be able to add 'anything' (with a Control) in here to + * get information to say. + * It can be for debug purposes (qcam) or for any other HAL. + */ + IPU3Action op; op.op = ActionMetadataReady; op.controls = ctrls; From patchwork Mon Nov 8 13:13:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14491 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8EB1FC3251 for ; Mon, 8 Nov 2021 13:14:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2C3446055D; Mon, 8 Nov 2021 14:14:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="l0m5iakP"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A468F603C2 for ; Mon, 8 Nov 2021 14:14:04 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 61C9718CE; Mon, 8 Nov 2021 14:14:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377243; bh=sPnE4NLtex+I4cA5tW8E17QMPkawPxLcZABU/uhQoeQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=l0m5iakPqeB3XqlE/7E6zNqFt6PKmFseIKE1AG6yRRBsXSC7Wbz3paPgSIs5tzeKC vVWrfYby5kSIyzS2iwQfkR7z+s3a1DcnSCLtbhhOx2X1GYuKenLQvAJuPFzQp0XCd1 RoKuvQy1hkK/1W+M4TaqmzTC0AFmeL+ontqG4MAQ= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:49 +0100 Message-Id: <20211108131350.130665-22-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 21/22] ipa: ipu3: Move ExposureTime to IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that we have the exposure time calculated, pass it to the controls::ExposureTime and don't use the pipeline handler for it anymore. While at it, use the same line duration value for ExposureTime and FrameDuration. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipu3.cpp | 9 +++++++-- src/libcamera/pipeline/ipu3/ipu3.cpp | 2 -- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index ca3f2417..24c77be8 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -691,12 +691,17 @@ void IPAIPU3::parseStatistics(unsigned int frame, setControls(frame); + double lineDuration = sensorInfo_.lineLength + / (sensorInfo_.pixelRate / 1e6); + /* \todo Use VBlank value calculated from each frame exposure. */ - int64_t frameDuration = sensorInfo_.lineLength * (defVBlank_ + sensorInfo_.outputSize.height) / - (sensorInfo_.pixelRate / 1e6); + int64_t frameDuration = (defVBlank_ + sensorInfo_.outputSize.height) + * lineDuration; ctrls.set(controls::FrameDuration, frameDuration); ctrls.set(controls::ColourTemperature, context_.frameContext->awb.temperatureK); + int32_t exposureTime = context_.frameContext->agc.exposure * lineDuration; + ctrls.set(controls::ExposureTime, exposureTime); /* * \todo We should be able to add 'anything' (with a Control) in here to diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 5d87f6e5..97003681 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1322,8 +1322,6 @@ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) pipe()->completeBuffer(request, buffer); request->metadata().set(controls::draft::PipelineDepth, 3); - /* \todo Move the ExposureTime control to the IPA. */ - request->metadata().set(controls::ExposureTime, exposureTime_); /* \todo Actually apply the scaler crop region to the ImgU. */ if (request->controls().contains(controls::ScalerCrop)) cropRegion_ = request->controls().get(controls::ScalerCrop); From patchwork Mon Nov 8 13:13:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14488 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B2CE8C324F for ; Mon, 8 Nov 2021 13:14:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5E0796055F; Mon, 8 Nov 2021 14:14:19 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bQ5Ca3Oa"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E26CF603BE for ; Mon, 8 Nov 2021 14:14:03 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 89F8E1C5E; Mon, 8 Nov 2021 14:14:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377243; bh=SdvsSzH/x/r4nhOdujm6E0L1+Zuji+dVt+EpKE/JVP4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bQ5Ca3Oaq3ezuhlRGpRWG0IOyfbDs3hhzcShl+P2mc6aCNb5cIHF5U7x733P71Lso u6iLGEH0VMPF+dFBO+1KvaTftpR4ppP0n/+BeKGGeC2vTLWKEVt8SP7TgydqzArQAG knkhGSVsCFNhJIcE7GV6pJxbFvvA0Ge+xlda9mXA= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:50 +0100 Message-Id: <20211108131350.130665-23-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 22/22] ipa: ipu3: Pass the AnalogueGain control X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We can set the controls::AnalogueGain metadata now that AGC is updating it correctly. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipu3.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 24c77be8..46cbc68d 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -702,6 +702,7 @@ void IPAIPU3::parseStatistics(unsigned int frame, ctrls.set(controls::ColourTemperature, context_.frameContext->awb.temperatureK); int32_t exposureTime = context_.frameContext->agc.exposure * lineDuration; ctrls.set(controls::ExposureTime, exposureTime); + ctrls.set(controls::AnalogueGain, context_.frameContext->agc.gain); /* * \todo We should be able to add 'anything' (with a Control) in here to