From patchwork Mon Nov 8 13:13:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14469 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3E5D3BDB1C for ; Mon, 8 Nov 2021 13:14:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D27136034D; Mon, 8 Nov 2021 14:14:01 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="NMK4dmRp"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8F7736032C for ; Mon, 8 Nov 2021 14:13:59 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 33427110E; Mon, 8 Nov 2021 14:13:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377239; bh=4mz/djZr3QNMMAhU39cE2IIcw0f0kRPijy68FcPO/zY=; h=From:To:Cc:Subject:Date:From; b=NMK4dmRpdrYbh9ZKWMpu8mTpoac5p599KpWyTO7U1j9s4IM07eLVG/PxbW6XvsW/8 wS8nsEX+8oojfRmN10lmoitZc+1FeCl9tgCm/vL84dM0NYR7M05F/mSzt7L3m/zZ3C uE2yjYKEQoqtr+a3m2NNw2lrGadfgjoUfmaoYcHM= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:28 +0100 Message-Id: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 00/22] IPA: IPU3: Introduce per-frame controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hello, This patch series is not only introducing per-frame controls, it also improves the AGC in some difficult cases (saturated scenes, backlight, etc.) thanks to the possibility to use the effective exposure and gains values for a given frame. The first two patches 1/22 and 2/22 are Han-Lin Chan series, splitted to keep only the sensor controls part. I have modified the first patch to remove the lens controls references, and another series containing only the lens controls should be sent separately as it requires a bit more review. Han-Lin, I hope you are ok with this plan :-) ? Next, from patches 3/22 to 9/22 we use those sensor controls as input when a EventStatReady event is received, by updating the frameContext values. With those values, we can be better at estimating the exposure value, and we introduce a brightness measure to have a better behaviour in saturated cases. Patches 10/22 to 18/22 are the per-frame control usage. Instead of using the current input frame value, we populate a frameContext queue which is introduced in patch 14/22 giving the ability to know the values in a request lifetime. In order to not break the previous usage of EventStatReady to pass the sensor controls, we keep those in the pipeline handler. Patches 19/22 to 22/22 are here to improve the per-frame control updates by the IPA, introducing the colour temperature control, and updating the ExposureTime and AnalogueGain controls. Han-Lin Chen (2): ipa: ipu3: Extend ipu3 ipa interface for sensor controls ipu3: ipa: Report effective sensor controls with statistics to IPA Jean-Michel Hautbois (20): ipa: ipu3: Use sensor controls to update frameContext ipa: ipu3: agc: Limit the number of saturated cells ipa: ipu3: agc: Compute the gain for each frame ipa: ipu3: agc: Refactor ev gain calculation and testing ipa: ipu3: agc: Improve gain calculation ipa: ipu3: agc: Update previous exposure value ipa: ipu3: agc: Use exposure in time for storage ipu3: Move delayedControls reset after IPA is started ipu3: Populate the EventProcessCall with the sensor controls ipa: ipu3: Introduce a frame index for the IPAFrameContext ipa: ipu3: Do not access IPAFrameContext in configure ipa: ipu3: Introduce a queue of IPAFrameContext ipa: ipu3: Mark the beginning and and of a frame ipa: ipu3: Introduce sensor controls ipa: ipu3: Use a IPAFrameContext pointer from the per-frame queue ipa: ipu3: Move the sensor update controls ipa: ipu3: awb: Add support for color temperature ipa: ipu3: Send color temperature in the metadata ipa: ipu3: Move ExposureTime to IPA ipa: ipu3: Pass the AnalogueGain control include/libcamera/ipa/ipu3.mojom | 2 + src/ipa/ipu3/algorithms/agc.cpp | 161 +++++++++++++++-------- src/ipa/ipu3/algorithms/agc.h | 10 +- src/ipa/ipu3/algorithms/awb.cpp | 15 ++- src/ipa/ipu3/algorithms/tone_mapping.cpp | 13 +- src/ipa/ipu3/ipa_context.cpp | 8 ++ src/ipa/ipu3/ipa_context.h | 6 +- src/ipa/ipu3/ipu3.cpp | 101 ++++++++++++-- src/libcamera/pipeline/ipu3/frames.h | 3 + src/libcamera/pipeline/ipu3/ipu3.cpp | 14 +- 10 files changed, 247 insertions(+), 86 deletions(-)