From patchwork Thu Nov 11 11:05:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14555 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7A7F0BF415 for ; Thu, 11 Nov 2021 11:06:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 34BBA6036E; Thu, 11 Nov 2021 12:06:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nOza/HxN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9D1D2600B5 for ; Thu, 11 Nov 2021 12:06:09 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:e627:8337:a781:d98]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 316134A6; Thu, 11 Nov 2021 12:06:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636628769; bh=uY3GX4EIPcF6+iVnqBcs5f2cuaoknANDyrO2nkvHLp8=; h=From:To:Cc:Subject:Date:From; b=nOza/HxNviWoodJ2VaAGzqydHUgsXYlcOfsZhSa1D6kqKdWchKuuGDFxfapG9MnZx onmww6tKzeWf3uA8VizAKyMonlhC7dqzJo4N0V6yBd2vH7mmPzZU54cBpHf+pjO32+ BzriewvLAlh3nV9pb4q0QM0oCoUyndUo8M0ej8Lo= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Thu, 11 Nov 2021 12:05:51 +0100 Message-Id: <20211111110605.105202-1-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 00/14] IPA: IPU3: Introduce per-frame controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hello, This is v3 of the patch series, but only half of v1 is here, as the second part, which adds the std::queue, is subject to discussion. This patch series is not only introducing per-frame controls, using the sensor controls directly in the EventStatReady, it also improves the AGC in some difficult cases (saturated scenes, backlight, etc.) thanks to the possibility to use the effective exposure and gains values for a given frame. The first two patches 1/14 and 2/14 are Han-Lin Chan series, splitted to keep only the sensor controls part. I have modified the first patch to remove the lens controls references, and another series containing only the lens controls should be sent separately as it requires a bit more review. In v2, I also modified the patch 2/14 to call the reset on DelayedControl at start and not at configure. For the next patches, v3 introduces a sensor structure in the frameContext, to distinguish input and output for the algorithms. From patches 3/14 to 9/14 we use those sensor controls as input when a EventStatReady event is received, by updating the frameContext values. With those values, we can be better at estimating the exposure value, and we introduce a brightness measure to have a better behaviour in saturated cases. Patches 10/14 to 14/14 are here to improve the per-frame control updates by the IPA, introducing the colour temperature control, and updating the ExposureTime and AnalogueGain controls. Han-Lin Chen (2): ipa: ipu3: Extend ipu3 ipa interface for sensor controls ipu3: ipa: Report effective sensor controls with statistics to IPA Jean-Michel Hautbois (12): ipa: ipu3: Use sensor controls to update frameContext ipa: ipu3: agc: Limit the number of saturated cells ipa: ipu3: agc: Compute the gain for each frame ipa: ipu3: agc: Refactor ev gain calculation and testing ipa: ipu3: agc: Improve gain calculation ipa: ipu3: agc: Update previous exposure value ipa: ipu3: agc: Use exposure in time for storage ipa: ipu3: awb: Add support for color temperature ipa: ipu3: Send color temperature in the metadata ipa: ipu3: Cache line duration at configure call ipa: ipu3: Move ExposureTime to IPA ipa: ipu3: Pass the AnalogueGain control include/libcamera/ipa/ipu3.mojom | 2 + src/ipa/ipu3/algorithms/agc.cpp | 200 ++++++++++++++++++++------- src/ipa/ipu3/algorithms/agc.h | 12 +- src/ipa/ipu3/algorithms/awb.cpp | 1 + src/ipa/ipu3/ipa_context.cpp | 14 ++ src/ipa/ipu3/ipa_context.h | 7 + src/ipa/ipu3/ipu3.cpp | 49 ++++--- src/libcamera/pipeline/ipu3/frames.h | 4 + src/libcamera/pipeline/ipu3/ipu3.cpp | 27 ++-- 9 files changed, 227 insertions(+), 89 deletions(-)