From patchwork Wed Nov 10 19:58:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14529 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BEFCFBDB1C for ; Wed, 10 Nov 2021 19:59:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5619B60368; Wed, 10 Nov 2021 20:59:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="JJj1PcTX"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 71E9B6034E for ; Wed, 10 Nov 2021 20:59:06 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:7713:3465:89e6:c5cd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F0B148BB; Wed, 10 Nov 2021 20:59:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636574346; bh=N81w8/G3lWtPy4LdH5xSsM77aOlGhlOyxo0vtEPgaEo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JJj1PcTXpLsoEJ4JgrQTaqCdrc8yrF3Mt2cZ6fkiUZVdoWJh0ZGEJZaylerIlaON/ E4EswlUcsAxKcqsHQQVeuOZIOxG892QGoHsaJRGjI9q0S4dNeqxSIHgL0KTxB7kR7q ZN7PzdkrElGJx/tmEbCqkOk21jtpn6JMKfpqKW6c= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 20:58:48 +0100 Message-Id: <20211110195901.85597-2-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> References: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 01/14] ipa: ipu3: Extend ipu3 ipa interface for sensor controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Han-Lin Chen IPU3Event and IPU3Action use single ControlList for both libcamera and V4L2 controls, and it's content could be either one based on the context. Extend IPU3Event and IPU3Action for sensor V4L2 controls, and preserve the original one for only libcamera Controls to make the content of an event more specific. Signed-off-by: Han-Lin Chen [Jean-Michel: remove lensControls from the original patch] Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder --- include/libcamera/ipa/ipu3.mojom | 2 ++ src/ipa/ipu3/ipu3.cpp | 2 +- src/libcamera/pipeline/ipu3/ipu3.cpp | 2 +- 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/include/libcamera/ipa/ipu3.mojom b/include/libcamera/ipa/ipu3.mojom index 2f254ed4..16e3462e 100644 --- a/include/libcamera/ipa/ipu3.mojom +++ b/include/libcamera/ipa/ipu3.mojom @@ -23,11 +23,13 @@ struct IPU3Event { int64 frameTimestamp; uint32 bufferId; libcamera.ControlList controls; + libcamera.ControlList sensorControls; }; struct IPU3Action { IPU3Operations op; libcamera.ControlList controls; + libcamera.ControlList sensorControls; }; struct IPAConfigInfo { diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 93b700bd..bcc3863b 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -653,7 +653,7 @@ void IPAIPU3::setControls(unsigned int frame) ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); - op.controls = ctrls; + op.sensorControls = ctrls; queueFrameAction.emit(frame, op); } diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index eb714aa6..8816efc5 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1248,7 +1248,7 @@ void IPU3CameraData::queueFrameAction(unsigned int id, { switch (action.op) { case ipa::ipu3::ActionSetSensorControls: { - const ControlList &controls = action.controls; + const ControlList &controls = action.sensorControls; delayedCtrls_->push(controls); break; } From patchwork Wed Nov 10 19:58:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14530 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3B332BDB1C for ; Wed, 10 Nov 2021 19:59:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 37AD26038B; Wed, 10 Nov 2021 20:59:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="C+qiAwxg"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7F37E6035A for ; Wed, 10 Nov 2021 20:59:06 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:7713:3465:89e6:c5cd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 34C4E9FF; Wed, 10 Nov 2021 20:59:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636574346; bh=zN1A2JCDmbXNy9LOEdVyYSE6rbn1dmLIfjJAeVVCjPQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=C+qiAwxgZjyjiwaKwt5HHhq1lCl3vhlyqz40M3freuFj+vIsKk5/x0/dwTClo59yi mWSzsxU/fFiVxAKOQnTMF7MLYUa1eY8d9A3ImOOSTF3fM7zing+1ymEPrHLpxgWd78 abHGIH1RF1MY7F9/0BuUMvEWoNeTk0AF0RYCl5B8= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 20:58:49 +0100 Message-Id: <20211110195901.85597-3-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> References: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 02/14] ipu3: ipa: Report effective sensor controls with statistics to IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Han-Lin Chen The Intel close sourced IPA requires the effective controls applied to the sensor when the statistics are generated. Report effective sensor controls with the statistics to IPA. Signed-off-by: Han-Lin Chen [Jean-Michel: Reword s/stastistics/statistics and move reset after IPA start] Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/ipu3/frames.h | 4 ++++ src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++++++++ 2 files changed, 13 insertions(+) diff --git a/src/libcamera/pipeline/ipu3/frames.h b/src/libcamera/pipeline/ipu3/frames.h index 3ef7e445..614701e2 100644 --- a/src/libcamera/pipeline/ipu3/frames.h +++ b/src/libcamera/pipeline/ipu3/frames.h @@ -14,6 +14,8 @@ #include +#include + namespace libcamera { class FrameBuffer; @@ -34,6 +36,8 @@ public: FrameBuffer *paramBuffer; FrameBuffer *statBuffer; + ControlList effectiveSensorControls; + bool paramDequeued; bool metadataProcessed; }; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 8816efc5..3fcfa777 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -767,6 +767,12 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis if (ret) goto error; + /* + * Reset the delayed controls with the gain and exposure values set by + * the IPA. + */ + data->delayedCtrls_->reset(); + /* * Start the ImgU video devices, buffers will be queued to the * ImgU output and viewfinder when requests will be queued. @@ -1363,6 +1369,8 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) request->metadata().set(controls::SensorTimestamp, buffer->metadata().timestamp); + info->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence); + if (request->findBuffer(&rawStream_)) pipe()->completeBuffer(request, buffer); @@ -1419,6 +1427,7 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer) ev.frame = info->id; ev.bufferId = info->statBuffer->cookie(); ev.frameTimestamp = request->metadata().get(controls::SensorTimestamp); + ev.sensorControls = info->effectiveSensorControls; ipa_->processEvent(ev); } From patchwork Wed Nov 10 19:58:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14531 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6573ABDB1C for ; Wed, 10 Nov 2021 19:59:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A2B9C6035F; Wed, 10 Nov 2021 20:59:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="YTOlUJRO"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A4E8E6035D for ; Wed, 10 Nov 2021 20:59:06 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:7713:3465:89e6:c5cd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 617BC119F; Wed, 10 Nov 2021 20:59:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636574346; bh=7IODOKBDi/ZsBc6GXmQAqV6fx53ViiQkcWxUqo0OejU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YTOlUJROh60DEfj2Zh3nHIcspv4xbLPt3d3IkEEilRVV2k1reOLiYh6cOsolXO6n1 lbbDsrzkkUkU2Wo9B56ERuz6ZsXWcl6ww9DoABgM6WTF8ZpVlQr46sVJ6KDFJ9kbg4 5FP4WfpJhpBDzGN3wSRNPxgRTE28Iz6qLt/tpvxM= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 20:58:50 +0100 Message-Id: <20211110195901.85597-4-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> References: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 03/14] ipa: ipu3: Use sensor controls to update frameContext X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The pipeline handler populates the new sensorControls ControlList, to have the effective exposure and gain values for the current frame. This is done when a statistics buffer is received. Make those values the frameContext::agc values for the frame when the EventStatReady event is received. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/ipu3.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index bcc3863b..1111f200 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -549,6 +549,10 @@ void IPAIPU3::processEvent(const IPU3Event &event) const ipu3_uapi_stats_3a *stats = reinterpret_cast(mem.data()); + /* \todo move those into processControls ? */ + context_.frameContext.agc.exposure = event.sensorControls.get(V4L2_CID_EXPOSURE).get(); + context_.frameContext.agc.gain = camHelper_->gain(event.sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + parseStatistics(event.frame, event.frameTimestamp, stats); break; } From patchwork Wed Nov 10 19:58:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14532 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4772EC324E for ; Wed, 10 Nov 2021 19:59:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1159E603EE; Wed, 10 Nov 2021 20:59:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BiYAPtEL"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DB8B66035F for ; Wed, 10 Nov 2021 20:59:06 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:7713:3465:89e6:c5cd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9089B11FB; Wed, 10 Nov 2021 20:59:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636574346; bh=hyHcJcHG38gpe/ZG8HtCcRXH9TFLVLLWAH5nYGm6YGE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BiYAPtELa7Tn3kvN+kwXpRKyvHQikcRGTLpVVEy/k2mwIVzgNkRXQ5XDOzBSSc+o8 MnNp/AjEPI1NedDTvY7m8764tTdeEvcvUmS4ppkGRVrp0k5FFGnc2Bo+QXUSVr6XDW zzj+//LyZ3b2EN0cTu+wA9XFdGWG0J8/maMOZXQA= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 20:58:51 +0100 Message-Id: <20211110195901.85597-5-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> References: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 04/14] ipa: ipu3: agc: Limit the number of saturated cells X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the histogram is calculated, we check if a cell is saturated or not before cumulating its green value. This is wrong, and it can lead to an empty histogram in case of a fully saturated frame. Use a constant to limit the amount of pixels within a cell before considering it saturated. If at the end of the loop we still have an empty histogram, then make it a fully saturated one. Bug: https://bugs.libcamera.org/show_bug.cgi?id=84 Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/agc.cpp | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index b5d736c1..2bf68e04 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -63,6 +63,12 @@ static constexpr uint32_t knumHistogramBins = 256; /* Target value to reach for the top 2% of the histogram */ static constexpr double kEvGainTarget = 0.5; +/* + * Maximum ratio of saturated pixels in a cell for the cell to be considered + * non-saturated and counted by the AGC algorithm. + */ +static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 20 / 100; + Agc::Agc() : frameCount_(0), lastFrame_(0), iqMean_(0.0), lineDuration_(0s), minExposureLines_(0), maxExposureLines_(0), filteredExposure_(0s), @@ -124,7 +130,7 @@ void Agc::measureBrightness(const ipu3_uapi_stats_3a *stats, &stats->awb_raw_buffer.meta_data[cellPosition] ); - if (cell->sat_ratio == 0) { + if (cell->sat_ratio <= kMinCellsPerZoneRatio) { uint8_t gr = cell->Gr_avg; uint8_t gb = cell->Gb_avg; /* @@ -137,8 +143,14 @@ void Agc::measureBrightness(const ipu3_uapi_stats_3a *stats, } } + Histogram cumulativeHist = Histogram(Span(hist)); /* Estimate the quantile mean of the top 2% of the histogram */ - iqMean_ = Histogram(Span(hist)).interQuantileMean(0.98, 1.0); + if (cumulativeHist.total() == 0) { + /* Force the value as histogram is empty */ + iqMean_ = knumHistogramBins - 0.5; + } else { + iqMean_ = cumulativeHist.interQuantileMean(0.98, 1.0); + } } /** From patchwork Wed Nov 10 19:58:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14533 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D9B07BDB1C for ; Wed, 10 Nov 2021 19:59:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C3EC2603D9; Wed, 10 Nov 2021 20:59:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gXVv1HdW"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1434C6033C for ; Wed, 10 Nov 2021 20:59:07 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:7713:3465:89e6:c5cd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C3B3D8B6; Wed, 10 Nov 2021 20:59:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636574346; bh=BBISLBkrdBqqRRc3oTJ1P9RNJ/DXAJEGYSFnPNmfG9M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gXVv1HdW3sLC4cQFBdpNiyO5ob471Oqa/54Nsec/3H102lg0NXOFDEovZ1kzjhAjL 8EHLl1GDNz9D44p2rUi1Iy9doI0cS+x25HvQvkYvOCHnvFwwFnARusyunF+WV+mA0I XV2pUk2XRduNzjnz+Ner42G7Zyc/b5GM0dCDpQlQ= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 20:58:52 +0100 Message-Id: <20211110195901.85597-6-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> References: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 05/14] ipa: ipu3: agc: Compute the gain for each frame X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that we have the real exposure applied at each frame, remove the early return based on a frame counter and compute the gain for each frame. Instead of that, introduce a number of startup frames during which the filter speed is 1.0, meaning we apply instantly the exposure value calculated and not a slower filtered one. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/agc.cpp | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 2bf68e04..133f5931 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -45,11 +45,6 @@ namespace ipa::ipu3::algorithms { LOG_DEFINE_CATEGORY(IPU3Agc) -/* Number of frames to wait before calculating stats on minimum exposure */ -static constexpr uint32_t kInitialFrameMinAECount = 4; -/* Number of frames to wait between new gain/shutter time estimations */ -static constexpr uint32_t kFrameSkipCount = 6; - /* Limits for analogue gain values */ static constexpr double kMinAnalogueGain = 1.0; static constexpr double kMaxAnalogueGain = 8.0; @@ -69,10 +64,13 @@ static constexpr double kEvGainTarget = 0.5; */ static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 20 / 100; +/* Number of frames to wait before calculating stats on minimum exposure */ +static constexpr uint32_t kNumStartupFrames = 10; + Agc::Agc() - : frameCount_(0), lastFrame_(0), iqMean_(0.0), lineDuration_(0s), - minExposureLines_(0), maxExposureLines_(0), filteredExposure_(0s), - currentExposure_(0s), prevExposureValue_(0s) + : frameCount_(0), iqMean_(0.0), lineDuration_(0s), minExposureLines_(0), + maxExposureLines_(0), filteredExposure_(0s), currentExposure_(0s), + prevExposureValue_(0s) { } @@ -159,6 +157,11 @@ void Agc::measureBrightness(const ipu3_uapi_stats_3a *stats, void Agc::filterExposure() { double speed = 0.2; + + /* Adapt instantly if we are in startup phase */ + if (frameCount_ < kNumStartupFrames) + speed = 1.0; + if (filteredExposure_ == 0s) { /* DG stands for digital gain.*/ filteredExposure_ = currentExposure_; @@ -186,13 +189,6 @@ void Agc::filterExposure() */ void Agc::computeExposure(uint32_t &exposure, double &analogueGain) { - /* Algorithm initialization should wait for first valid frames */ - /* \todo - have a number of frames given by DelayedControls ? - * - implement a function for IIR */ - if ((frameCount_ < kInitialFrameMinAECount) || (frameCount_ - lastFrame_ < kFrameSkipCount)) - return; - - lastFrame_ = frameCount_; /* Are we correctly exposed ? */ if (std::abs(iqMean_ - kEvGainTarget * knumHistogramBins) <= 1) { From patchwork Wed Nov 10 19:58:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14534 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1CC4FC324F for ; Wed, 10 Nov 2021 19:59:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B7AAA6038C; Wed, 10 Nov 2021 20:59:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cJDo7MCg"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3CFF460362 for ; Wed, 10 Nov 2021 20:59:07 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:7713:3465:89e6:c5cd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F29CB8BB; Wed, 10 Nov 2021 20:59:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636574347; bh=xYoyXO2xIpO0T3PlvfATsenTDcmVIxKJiA7AEOKpRI8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cJDo7MCgHXXSaR1wrzFVYT/mRopkIUd8971w2Xk1P/WaRFNhWx9rjF7CGmayVQsmt 9pva2VxnobjhfadEmf1ydFkw7bbdMyQ2416xaD3mgdM0yOrvAX4RBXM9ZYsxp10EiD fTXw8BVgMnqTmwIPxoyYZfI4WTKbLjRUlJp4UHm4= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 20:58:53 +0100 Message-Id: <20211110195901.85597-7-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> References: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 06/14] ipa: ipu3: agc: Refactor ev gain calculation and testing X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When we compute the new gain, we use the iqMean_ and estimate an exposure value gain to apply. Return early when the gain is less than 1%. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/agc.cpp | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 133f5931..119a7938 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -189,17 +189,15 @@ void Agc::filterExposure() */ void Agc::computeExposure(uint32_t &exposure, double &analogueGain) { + /* Estimate the gain needed to have the proportion wanted */ + double evGain = kEvGainTarget * knumHistogramBins / iqMean_; - /* Are we correctly exposed ? */ - if (std::abs(iqMean_ - kEvGainTarget * knumHistogramBins) <= 1) { + if (std::abs(evGain - 1.0) < 0.01) { LOG(IPU3Agc, Debug) << "We are well exposed (iqMean = " << iqMean_ << ")"; return; } - /* Estimate the gain needed to have the proportion wanted */ - double evGain = kEvGainTarget * knumHistogramBins / iqMean_; - /* extracted from Rpi::Agc::computeTargetExposure */ /* Calculate the shutter time in seconds */ utils::Duration currentShutter = exposure * lineDuration_; From patchwork Wed Nov 10 19:58:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14536 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4B859C324E for ; Wed, 10 Nov 2021 19:59:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EC8726035D; Wed, 10 Nov 2021 20:59:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CMcJxqQZ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6B16A60361 for ; Wed, 10 Nov 2021 20:59:07 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:7713:3465:89e6:c5cd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2E0961203; Wed, 10 Nov 2021 20:59:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636574347; bh=QNlsYC/OZjS6lvW+7UJez1A3UxSSPFqQiRM/o5r9eSg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CMcJxqQZMG15+G0ceAIimZEL1Spcbmlsh8J3JWcdF+pj+mDQolSvRAfm3FD4KeaQY O7FTTtlyX50RaQoU1QpfBUgI08CxbuiRRyHhR9OsVKwmwnBODarXDEGT6tFzrHW2fU w8FDdX4SbjrN8b83WsZqJZenwaqZKtrcC9HoNr1M= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 20:58:54 +0100 Message-Id: <20211110195901.85597-8-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> References: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 07/14] ipa: ipu3: agc: Improve gain calculation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When an image is partially saturated, its brightness is not increasing linearly when the shutter time or gain increases. It is a big issue with a backlight as the algorithm is fading to darkness right now. Introduce a function to estimate the brightness of the frame, based on the current exposure/gain and loop on it several times to estimate it again and approach the non linear function. Inspired-by: 7de5506c30b3 ("libcamera: src: ipa: raspberrypi: agc: Improve gain update calculation for partly saturated images") Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/agc.cpp | 103 +++++++++++++++++++++++++++++++- src/ipa/ipu3/algorithms/agc.h | 6 +- 2 files changed, 105 insertions(+), 4 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 119a7938..ee37a9d5 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -67,6 +67,9 @@ static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 20 / 100; /* Number of frames to wait before calculating stats on minimum exposure */ static constexpr uint32_t kNumStartupFrames = 10; +/* Maximum luminance used for brightness normalization */ +static constexpr uint32_t kMaxLuminance = 255; + Agc::Agc() : frameCount_(0), iqMean_(0.0), lineDuration_(0s), minExposureLines_(0), maxExposureLines_(0), filteredExposure_(0s), currentExposure_(0s), @@ -186,10 +189,16 @@ void Agc::filterExposure() * \brief Estimate the new exposure and gain values * \param[inout] exposure The exposure value reference as a number of lines * \param[inout] gain The gain reference to be updated + * \param[in] currentYGain The gain calculated on the current brightness level */ -void Agc::computeExposure(uint32_t &exposure, double &analogueGain) +void Agc::computeExposure(uint32_t &exposure, double &analogueGain, double currentYGain) { - /* Estimate the gain needed to have the proportion wanted */ + /* + * Estimate the gain needed to have the proportion of pixels in a given + * range wanted. iqMean_ returns the mean value of the top 2% of the + * cumulative histogram, and we want it to be as close as possible to a + * configured target. + */ double evGain = kEvGainTarget * knumHistogramBins / iqMean_; if (std::abs(evGain - 1.0) < 0.01) { @@ -199,6 +208,7 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain) } /* extracted from Rpi::Agc::computeTargetExposure */ + /* Calculate the shutter time in seconds */ utils::Duration currentShutter = exposure * lineDuration_; LOG(IPU3Agc, Debug) << "Actual total exposure " << currentShutter * analogueGain @@ -206,6 +216,14 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain) << " Gain " << analogueGain << " Needed ev gain " << evGain; + if (evGain < currentYGain) + evGain = currentYGain; + + /* Consider within 1% of the target as correctly exposed */ + if (std::abs(evGain - 1.0) < 0.01) + LOG(IPU3Agc, Debug) << "We are well exposed (iqMean = " + << iqMean_ << ")"; + /* * Calculate the current exposure value for the scene as the latest * exposure value applied multiplied by the new estimated gain. @@ -253,6 +271,57 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain) prevExposureValue_ = shutterTime * analogueGain; } +/** + * \brief Estimate the average brightness of the frame + * \param[in] context The shared IPA context + * \param[in] grid The grid used to store the statistics in the IPU3 + * \param[in] stats The IPU3 statistics and ISP results + * \param[in] currentYGain The gain calculated on the current brightness level + * \return The normalized luma + * + * Luma is the weighted sum of gamma-compressed R′G′B′ components of a color + * video. The luma values are normalized as 0.0 to 1.0, with 1.0 being a + * theoretical perfect reflector of 100% reference white. We use the Rec. 601 + * luma here. + * + * More detailed information can be found in: + * https://en.wikipedia.org/wiki/Luma_(video) + */ +double Agc::computeInitialY(IPAFrameContext &frameContext, + const ipu3_uapi_grid_config &grid, + const ipu3_uapi_stats_3a *stats, + double currentYGain) +{ + double redSum = 0, greenSum = 0, blueSum = 0; + + for (unsigned int cellY = 0; cellY < grid.height; cellY++) { + for (unsigned int cellX = 0; cellX < grid.width; cellX++) { + uint32_t cellPosition = cellY * stride_ + cellX; + + const ipu3_uapi_awb_set_item *cell = + reinterpret_cast( + &stats->awb_raw_buffer.meta_data[cellPosition] + ); + + redSum += cell->R_avg * currentYGain; + greenSum += (cell->Gr_avg + cell->Gb_avg) / 2 * currentYGain; + blueSum += cell->B_avg * currentYGain; + } + } + + /* + * Estimate the sum of the brightness values, weighted with the gains + * applied on the channels in AWB as the Rec. 601 luma. + */ + double Y_sum = redSum * frameContext.awb.gains.red * .299 + + greenSum * frameContext.awb.gains.green * .587 + + blueSum * frameContext.awb.gains.blue * .114; + + /* Return the normalized relative luminance. */ + return Y_sum / (grid.height * grid.width) / kMaxLuminance; +} + + /** * \brief Process IPU3 statistics, and run AGC operations * \param[in] context The shared IPA context @@ -267,7 +336,35 @@ void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) uint32_t &exposure = context.frameContext.agc.exposure; double &analogueGain = context.frameContext.agc.gain; measureBrightness(stats, context.configuration.grid.bdsGrid); - computeExposure(exposure, analogueGain); + + double currentYGain = 1.0; + /* + * Normalized luma value target. + * + * It's a number that's chosen so that, when the camera points at a grey + * target, the resulting image brightness is considered right. + */ + double targetY = 0.16; + + /* + * Do this calculation a few times as brightness increase can be + * non-linear when there are saturated regions. + */ + for (int i = 0; i < 8; i++) { + double initialY = computeInitialY(context.frameContext, + context.configuration.grid.bdsGrid, + stats, currentYGain); + double extra_gain = std::min(10.0, targetY / (initialY + .001)); + + currentYGain *= extra_gain; + LOG(IPU3Agc, Debug) << "Initial Y " << initialY + << " target " << targetY + << " gives gain " << currentYGain; + if (extra_gain < 1.01) + break; + } + + computeExposure(exposure, analogueGain, currentYGain); frameCount_++; } diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 69e0b831..0a9152a9 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -34,7 +34,11 @@ private: void measureBrightness(const ipu3_uapi_stats_3a *stats, const ipu3_uapi_grid_config &grid); void filterExposure(); - void computeExposure(uint32_t &exposure, double &gain); + void computeExposure(uint32_t &exposure, double &gain, double currentYGain); + double computeInitialY(IPAFrameContext &frameContext, + const ipu3_uapi_grid_config &grid, + const ipu3_uapi_stats_3a *stats, + double currentYGain); uint64_t frameCount_; uint64_t lastFrame_; From patchwork Wed Nov 10 19:58:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14535 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 93641C3250 for ; Wed, 10 Nov 2021 19:59:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2347D6036B; Wed, 10 Nov 2021 20:59:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bpxwb1O/"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A11696036C for ; Wed, 10 Nov 2021 20:59:07 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:7713:3465:89e6:c5cd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5B1C59FF; Wed, 10 Nov 2021 20:59:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636574347; bh=35XmsoZ3Aln9poVfgpV5vjAvkAYCJ2IXBJjbUbUxgrg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bpxwb1O/jNwVr6C0V/Y2dachHIGOsGKwQ9Z79cEIKJ1RBLEKaXvK9yMRCltWaMwNa wJrA88TJA4MIYwVGo3XYCNERKjnu5PI1uEkpFlegM9G4syFEpaRV8nMf3nue8aaXQ0 sylEfxa2Tn6UCNiPxRPhCTj0EV1KRsCKpk1AmBVo= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 20:58:55 +0100 Message-Id: <20211110195901.85597-9-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> References: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 08/14] ipa: ipu3: agc: Update previous exposure value X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Previously, the exposure value was calculated based on the estimated shutter time and gain applied. Now that we have the real values for the current frame, use those before estimating the next one and rename the variable accordingly. Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/agc.cpp | 32 +++++++++++--------------------- src/ipa/ipu3/algorithms/agc.h | 2 +- 2 files changed, 12 insertions(+), 22 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index ee37a9d5..38667e61 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -73,7 +73,7 @@ static constexpr uint32_t kMaxLuminance = 255; Agc::Agc() : frameCount_(0), iqMean_(0.0), lineDuration_(0s), minExposureLines_(0), maxExposureLines_(0), filteredExposure_(0s), currentExposure_(0s), - prevExposureValue_(0s) + effectiveExposureValue_(0s) { } @@ -104,9 +104,9 @@ int Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) context.frameContext.agc.gain = minAnalogueGain_; context.frameContext.agc.exposure = minExposureLines_; - prevExposureValue_ = context.frameContext.agc.gain - * context.frameContext.agc.exposure - * lineDuration_; + effectiveExposureValue_ = context.frameContext.agc.gain + * context.frameContext.agc.exposure + * lineDuration_; return 0; } @@ -201,16 +201,16 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain, double curre */ double evGain = kEvGainTarget * knumHistogramBins / iqMean_; - if (std::abs(evGain - 1.0) < 0.01) { - LOG(IPU3Agc, Debug) << "We are well exposed (iqMean = " - << iqMean_ << ")"; - return; - } - /* extracted from Rpi::Agc::computeTargetExposure */ /* Calculate the shutter time in seconds */ utils::Duration currentShutter = exposure * lineDuration_; + /* + * Update the exposure value for the next computation using the values + * of exposure and gain really used by the sensor. + */ + effectiveExposureValue_ = currentShutter * analogueGain; + LOG(IPU3Agc, Debug) << "Actual total exposure " << currentShutter * analogueGain << " Shutter speed " << currentShutter << " Gain " << analogueGain @@ -228,7 +228,7 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain, double curre * Calculate the current exposure value for the scene as the latest * exposure value applied multiplied by the new estimated gain. */ - currentExposure_ = prevExposureValue_ * evGain; + currentExposure_ = effectiveExposureValue_ * evGain; utils::Duration minShutterSpeed = minExposureLines_ * lineDuration_; utils::Duration maxShutterSpeed = maxExposureLines_ * lineDuration_; @@ -259,16 +259,6 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain, double curre exposure = shutterTime / lineDuration_; analogueGain = stepGain; - - /* - * Update the exposure value for the next process call. - * - * \todo Obtain the values of the exposure time and analog gain - * that were actually used by the sensor, either from embedded - * data when available, or from the delayed controls - * infrastructure in case a slow down caused a mismatch. - */ - prevExposureValue_ = shutterTime * analogueGain; } /** diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 0a9152a9..51174639 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -54,7 +54,7 @@ private: utils::Duration filteredExposure_; utils::Duration currentExposure_; - utils::Duration prevExposureValue_; + utils::Duration effectiveExposureValue_; uint32_t stride_; }; From patchwork Wed Nov 10 19:58:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14538 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 786C5C3251 for ; Wed, 10 Nov 2021 19:59:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2E8036039C; Wed, 10 Nov 2021 20:59:18 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DSLTz3ax"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D47E360371 for ; Wed, 10 Nov 2021 20:59:07 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:7713:3465:89e6:c5cd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 891D2119F; Wed, 10 Nov 2021 20:59:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636574347; bh=xd11F5oUsz3o+9adKBvwo2Qr6EGFbowbbsbNqHvt8vQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DSLTz3axqrwRautznzP3Ee4ThQYkrfZ6chtDuYMFG7z+Fdr9mY9H1Nki4MjXRXIBZ 01u7f7Tcb/XgFEXS9UhoFlObXGoQz528ec+kmMLZLgszrzTxUrJ+7vdyf0UqHXRqxy CIjeYmz/Um5nxIsoVuXUBaMcQvjdDH1+XslkUiTM= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 20:58:56 +0100 Message-Id: <20211110195901.85597-10-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> References: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 09/14] ipa: ipu3: agc: Use exposure in time for storage X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The minimum and maximum exposure are stored in lines. Replace it by values in time to simplify the calculations. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/agc.cpp | 21 +++++++++------------ src/ipa/ipu3/algorithms/agc.h | 4 ++-- 2 files changed, 11 insertions(+), 14 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 38667e61..08bf2a5f 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -71,8 +71,8 @@ static constexpr uint32_t kNumStartupFrames = 10; static constexpr uint32_t kMaxLuminance = 255; Agc::Agc() - : frameCount_(0), iqMean_(0.0), lineDuration_(0s), minExposureLines_(0), - maxExposureLines_(0), filteredExposure_(0s), currentExposure_(0s), + : frameCount_(0), iqMean_(0.0), lineDuration_(0s), minShutterSpeed_(0s), + maxShutterSpeed_(0s), filteredExposure_(0s), currentExposure_(0s), effectiveExposureValue_(0s) { } @@ -92,17 +92,16 @@ int Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) lineDuration_ = configInfo.sensorInfo.lineLength * 1.0s / configInfo.sensorInfo.pixelRate; - /* \todo replace the exposure in lines storage with time based ones. */ - minExposureLines_ = context.configuration.agc.minShutterSpeed / lineDuration_; - maxExposureLines_ = std::min(context.configuration.agc.maxShutterSpeed / lineDuration_, - kMaxShutterSpeed / lineDuration_); + minShutterSpeed_ = context.configuration.agc.minShutterSpeed; + maxShutterSpeed_ = std::min(context.configuration.agc.maxShutterSpeed, + kMaxShutterSpeed); minAnalogueGain_ = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain); maxAnalogueGain_ = std::min(context.configuration.agc.maxAnalogueGain, kMaxAnalogueGain); /* Configure the default exposure and gain. */ context.frameContext.agc.gain = minAnalogueGain_; - context.frameContext.agc.exposure = minExposureLines_; + context.frameContext.agc.exposure = minShutterSpeed_ / lineDuration_; effectiveExposureValue_ = context.frameContext.agc.gain * context.frameContext.agc.exposure @@ -229,11 +228,9 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain, double curre * exposure value applied multiplied by the new estimated gain. */ currentExposure_ = effectiveExposureValue_ * evGain; - utils::Duration minShutterSpeed = minExposureLines_ * lineDuration_; - utils::Duration maxShutterSpeed = maxExposureLines_ * lineDuration_; /* Clamp the exposure value to the min and max authorized */ - utils::Duration maxTotalExposure = maxShutterSpeed * maxAnalogueGain_; + utils::Duration maxTotalExposure = maxShutterSpeed_ * maxAnalogueGain_; currentExposure_ = std::min(currentExposure_, maxTotalExposure); LOG(IPU3Agc, Debug) << "Target total exposure " << currentExposure_ << ", maximum is " << maxTotalExposure; @@ -243,14 +240,14 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain, double curre /* Divide the exposure value as new exposure and gain values */ utils::Duration exposureValue = filteredExposure_; - utils::Duration shutterTime = minShutterSpeed; + utils::Duration shutterTime; /* * Push the shutter time up to the maximum first, and only then * increase the gain. */ shutterTime = std::clamp(exposureValue / minAnalogueGain_, - minShutterSpeed, maxShutterSpeed); + minShutterSpeed_, maxShutterSpeed_); double stepGain = std::clamp(exposureValue / shutterTime, minAnalogueGain_, maxAnalogueGain_); LOG(IPU3Agc, Debug) << "Divided up shutter and gain are " diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 51174639..eab0a631 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -46,8 +46,8 @@ private: double iqMean_; utils::Duration lineDuration_; - uint32_t minExposureLines_; - uint32_t maxExposureLines_; + utils::Duration minShutterSpeed_; + utils::Duration maxShutterSpeed_; double minAnalogueGain_; double maxAnalogueGain_; From patchwork Wed Nov 10 19:58:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14537 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D81F7BDB1C for ; Wed, 10 Nov 2021 19:59:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7B3386034E; Wed, 10 Nov 2021 20:59:17 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="e2GLxx2q"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 102EE60378 for ; Wed, 10 Nov 2021 20:59:08 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:7713:3465:89e6:c5cd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BD05B8B6; Wed, 10 Nov 2021 20:59:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636574347; bh=Zfa8T5He8JzrMkOGNJeAvgz2SjuZrJ3s+Kv+0r0+1oM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=e2GLxx2qbXUopjXeMu7ZYsviu45iwvhy9oGUotbSXiAMXWvc+WlKdfQatT2xnZxpr HeuALdY/T3TNk0UcTXIK1UnxbJJYF3/q349zcSpbQNxTQ0N5GQRU8yo/vymAxOoc4F Qgo+ms49Yt6pSQ1qpaiDwAQHQTIdhNm+wlQuYCuE= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 20:58:57 +0100 Message-Id: <20211110195901.85597-11-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> References: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 10/14] ipa: ipu3: awb: Add support for color temperature X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AWB estimates the color temperature, but it is not used at all. It can be useful for debug purpose at least, but also for lux estimation later, to be able to know the temperature estimated for a given frame. Add a new member to the IPAFrameContext::awb for this purpose, and update the value in AWB. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/awb.cpp | 1 + src/ipa/ipu3/ipa_context.cpp | 3 +++ src/ipa/ipu3/ipa_context.h | 2 ++ 3 files changed, 6 insertions(+) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index a4114659..c7bcb20e 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -385,6 +385,7 @@ void Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) context.frameContext.awb.gains.blue = asyncResults_.blueGain; context.frameContext.awb.gains.green = asyncResults_.greenGain; context.frameContext.awb.gains.red = asyncResults_.redGain; + context.frameContext.awb.temperatureK = asyncResults_.temperatureK; } constexpr uint16_t Awb::threshold(float value) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 2355a9c7..0205640e 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -117,6 +117,9 @@ namespace libcamera::ipa::ipu3 { * * \var IPAFrameContext::awb.gains.blue * \brief White balance gain for B channel + * + * \var IPAFrameContext::awb.temperatureK + * \brief Estimated color temperature */ /** diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 1e46c61a..e1087224 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -45,6 +45,8 @@ struct IPAFrameContext { double green; double blue; } gains; + + double temperatureK; } awb; struct { From patchwork Wed Nov 10 19:58:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14540 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 57056C324F for ; Wed, 10 Nov 2021 19:59:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0FCA360378; Wed, 10 Nov 2021 20:59:19 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="u3xd1nU+"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3D31B60369 for ; Wed, 10 Nov 2021 20:59:08 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:7713:3465:89e6:c5cd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EF8DC11FB; Wed, 10 Nov 2021 20:59:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636574348; bh=W2eGcRXAmwOmajAzEGBHvTt2AB2+IkneZAd20cwqRlk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=u3xd1nU+rdJ+JdS8xa+foHYaURwmVtdrsuFREXAIsJ8rFbyShJxVryTOMC2g27AZH 7yAcY39qyquwezzZsJuAm8mjES1Q6Ym7J0Oc7KRurffS8rrPiXGuxzJeyF5UoxG1U1 74/c2nHkyRHWNl1RvXlUZCo+qdsnNQs7dQhnTWck= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 20:58:58 +0100 Message-Id: <20211110195901.85597-12-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> References: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 11/14] ipa: ipu3: Send color temperature in the metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that the color temperature is updated per-frame, use the value and set the corresponding controls::ColourTemperature. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipu3.cpp | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 1111f200..eb8c0dc4 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -632,6 +632,14 @@ void IPAIPU3::parseStatistics(unsigned int frame, (sensorInfo_.pixelRate / 1e6); ctrls.set(controls::FrameDuration, frameDuration); + ctrls.set(controls::ColourTemperature, context_.frameContext.awb.temperatureK); + + /* + * \todo We should be able to add 'anything' (with a Control) in here to + * get information to say. + * It can be for debug purposes (qcam) or for any other HAL. + */ + IPU3Action op; op.op = ActionMetadataReady; op.controls = ctrls; From patchwork Wed Nov 10 19:58:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14539 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EAFE7C3252 for ; Wed, 10 Nov 2021 19:59:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9AFF6603B5; Wed, 10 Nov 2021 20:59:18 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="A4MIGnoS"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 682256035A for ; Wed, 10 Nov 2021 20:59:08 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:7713:3465:89e6:c5cd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 28F198BB; Wed, 10 Nov 2021 20:59:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636574348; bh=Y5+8e0if8PJu2nwjoP6woC2NVmFfYH6j/iRq9KhM31k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A4MIGnoSEoUn9vWdpyqrt0QD/fH2Hb3yaMGkRhVtijdJZvMf0VejpuwwoYrvxrJWL LppAECwORKdePGqGUZnKcUzWhMtnfjP9/EPUZAZtKhg2CRdKs5RqMqjJrVggzjWhk2 +Dp6dVccuzHkrxj4V2rujK2TVKqjQX1RgAtoo2nA= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 20:58:59 +0100 Message-Id: <20211110195901.85597-13-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> References: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 12/14] ipa: ipu3: Cache line duration at configure call X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We use the line duration several times in the IPAIPU3. Instead of recalculating it each time, cache the value as a utils::Duration. When the value is needed in double, call the intern count() function to get it. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipu3.cpp | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index eb8c0dc4..68a97e8e 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -174,6 +174,8 @@ private: uint32_t minGain_; uint32_t maxGain_; + utils::Duration lineDuration_; + /* Interface to the Camera Helper */ std::unique_ptr camHelper_; @@ -195,8 +197,8 @@ void IPAIPU3::updateSessionConfiguration(const IPACameraSensorInfo &sensorInfo, int32_t minExposure = v4l2Exposure.min().get(); int32_t maxExposure = v4l2Exposure.max().get(); - utils::Duration lineDuration = sensorInfo.lineLength * 1.0s - / sensorInfo.pixelRate; + lineDuration_ = sensorInfo.lineLength * 1.0s + / sensorInfo.pixelRate; const ControlInfo &v4l2Gain = sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; int32_t minGain = v4l2Gain.min().get(); @@ -209,8 +211,8 @@ void IPAIPU3::updateSessionConfiguration(const IPACameraSensorInfo &sensorInfo, * * \todo take VBLANK into account for maximum shutter speed */ - context_.configuration.agc.minShutterSpeed = minExposure * lineDuration; - context_.configuration.agc.maxShutterSpeed = maxExposure * lineDuration; + context_.configuration.agc.minShutterSpeed = minExposure * lineDuration_; + context_.configuration.agc.maxShutterSpeed = maxExposure * lineDuration_; context_.configuration.agc.minAnalogueGain = camHelper_->gain(minGain); context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain); } @@ -239,11 +241,10 @@ void IPAIPU3::updateControls(const IPACameraSensorInfo &sensorInfo, * exposure min, max and default and convert it from lines to * microseconds. */ - double lineDuration = sensorInfo.lineLength / (sensorInfo.pixelRate / 1e6); const ControlInfo &v4l2Exposure = sensorControls.find(V4L2_CID_EXPOSURE)->second; - int32_t minExposure = v4l2Exposure.min().get() * lineDuration; - int32_t maxExposure = v4l2Exposure.max().get() * lineDuration; - int32_t defExposure = v4l2Exposure.def().get() * lineDuration; + int32_t minExposure = v4l2Exposure.min().get() * lineDuration_.count(); + int32_t maxExposure = v4l2Exposure.max().get() * lineDuration_.count(); + int32_t defExposure = v4l2Exposure.def().get() * lineDuration_.count(); controls[&controls::ExposureTime] = ControlInfo(minExposure, maxExposure, defExposure); From patchwork Wed Nov 10 19:59:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14541 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7987FC3250 for ; Wed, 10 Nov 2021 19:59:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3321B603D9; Wed, 10 Nov 2021 20:59:20 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ex2DiTAp"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9BC156038A for ; Wed, 10 Nov 2021 20:59:08 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:7713:3465:89e6:c5cd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5654E1203; Wed, 10 Nov 2021 20:59:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636574348; bh=2A3NQGgjUtHDOYwChd++zg1d2BLD0Pz9tz26tsWIFZA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ex2DiTApmjrTnTUROA4F2G38WlmQ0MQ5VkzjNoUVgwscq0vt6ILHVM+ycopCmkfsH M2UbflEJJd/gMCSJs+++0ZPratu/gnNgxRpgls3t0cD/OFW26i3rtLt2aDWlP1f8e5 q9iKTxM08yd+e3mKqhEb1NX7bUfAYci/hhfJR2ks= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 20:59:00 +0100 Message-Id: <20211110195901.85597-14-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> References: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 13/14] ipa: ipu3: Move ExposureTime to IPA X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that we have the exposure time calculated, pass it to the controls::ExposureTime and don't use the pipeline handler for it anymore. While at it, use the same line duration value for ExposureTime and FrameDuration. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipu3.cpp | 6 ++++-- src/libcamera/pipeline/ipu3/ipu3.cpp | 16 +--------------- 2 files changed, 5 insertions(+), 17 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 68a97e8e..9220fea5 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -629,12 +629,14 @@ void IPAIPU3::parseStatistics(unsigned int frame, setControls(frame); /* \todo Use VBlank value calculated from each frame exposure. */ - int64_t frameDuration = sensorInfo_.lineLength * (defVBlank_ + sensorInfo_.outputSize.height) / - (sensorInfo_.pixelRate / 1e6); + int64_t frameDuration = (defVBlank_ + sensorInfo_.outputSize.height) + * lineDuration_.count(); ctrls.set(controls::FrameDuration, frameDuration); ctrls.set(controls::ColourTemperature, context_.frameContext.awb.temperatureK); + ctrls.set(controls::ExposureTime, context_.frameContext.agc.exposure * lineDuration_.count()); + /* * \todo We should be able to add 'anything' (with a Control) in here to * get information to say. diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 3fcfa777..fe37e9c7 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -47,7 +47,7 @@ class IPU3CameraData : public Camera::Private { public: IPU3CameraData(PipelineHandler *pipe) - : Camera::Private(pipe), exposureTime_(0), supportsFlips_(false) + : Camera::Private(pipe), supportsFlips_(false) { } @@ -67,7 +67,6 @@ public: Stream vfStream_; Stream rawStream_; - uint32_t exposureTime_; Rectangle cropRegion_; bool supportsFlips_; Transform rotationTransform_; @@ -1049,17 +1048,6 @@ int PipelineHandlerIPU3::updateControls(IPU3CameraData *data) controls[&controls::ScalerCrop] = ControlInfo(minCrop, maxCrop, maxCrop); - /* - * \todo Report the actual exposure time, use the default for the - * moment. - */ - const auto exposureInfo = data->ipaControls_.find(&controls::ExposureTime); - if (exposureInfo == data->ipaControls_.end()) { - LOG(IPU3, Error) << "Exposure control not initialized by the IPA"; - return -EINVAL; - } - data->exposureTime_ = exposureInfo->second.def().get(); - /* Add the IPA registered controls to list of camera controls. */ for (const auto &ipaControl : data->ipaControls_) controls[ipaControl.first] = ipaControl.second; @@ -1321,8 +1309,6 @@ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) pipe()->completeBuffer(request, buffer); request->metadata().set(controls::draft::PipelineDepth, 3); - /* \todo Move the ExposureTime control to the IPA. */ - request->metadata().set(controls::ExposureTime, exposureTime_); /* \todo Actually apply the scaler crop region to the ImgU. */ if (request->controls().contains(controls::ScalerCrop)) cropRegion_ = request->controls().get(controls::ScalerCrop); From patchwork Wed Nov 10 19:59:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14542 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 29038C324E for ; Wed, 10 Nov 2021 19:59:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9FAEC60362; Wed, 10 Nov 2021 20:59:20 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="f1rqmZHr"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D97046035D for ; Wed, 10 Nov 2021 20:59:08 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:7713:3465:89e6:c5cd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 857A69FF; Wed, 10 Nov 2021 20:59:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636574348; bh=OQNoTUpii98tHfVGldIPhX97z3lGJngwbxxcO0qhcJY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f1rqmZHrhlQESFbHtqhiJ5iBftKMF3+Vmd/j/Tg4XdeIG/eF69Dh9Yqkw0F5XABIn OmUu7XCofsJfgDgIyDYeOFSBeeIsqoGbDc2/EC5yxy1JE2sxJa28sgvfukbPjQ2cvn D2MWmy+IR5kNdh/20GVnEchpNdpj0SOa/ljaoK8g= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Wed, 10 Nov 2021 20:59:01 +0100 Message-Id: <20211110195901.85597-15-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> References: <20211110195901.85597-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 14/14] ipa: ipu3: Pass the AnalogueGain control X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We can set the controls::AnalogueGain metadata now that AGC is updating it correctly. Signed-off-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipu3.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 9220fea5..e69382f9 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -637,6 +637,8 @@ void IPAIPU3::parseStatistics(unsigned int frame, ctrls.set(controls::ExposureTime, context_.frameContext.agc.exposure * lineDuration_.count()); + ctrls.set(controls::AnalogueGain, context_.frameContext.agc.gain); + /* * \todo We should be able to add 'anything' (with a Control) in here to * get information to say.