From patchwork Fri Sep 19 09:40:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24409 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EC198C328C for ; Fri, 19 Sep 2025 09:40:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B9D336B599; Fri, 19 Sep 2025 11:40:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ghDvdRQa"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CAD2E62C38 for ; Fri, 19 Sep 2025 11:40:53 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:4d54:eab8:98ca:163b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 8E3737E4; Fri, 19 Sep 2025 11:39:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1758274773; bh=9TSFe25nhm9+MEMz4yO9YIRYY0N3IoIEHM8HAz6VVI0=; h=From:To:Cc:Subject:Date:From; b=ghDvdRQa75bAFYSmNm7C4+RAMASTR/C44SFmHEtjZDzK8JMTe/aI92OzBvxmN/CVK 5jRRNnjIOvlQ0GXsU6xZuI1tWW4iMKiaf5FFFdUeJPnvRP9RM0CTCfRTmpIRLMQS2b a7btn1kPTNKJDd8bVhm4tAd/ajqQEWYhNiu3sKUk= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v5 00/19] Implement WDR algorithm Date: Fri, 19 Sep 2025 11:40:15 +0200 Message-ID: <20250919094041.183031-1-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hi all, This series implements a global tone mapping algorithm for rkisp1. The details of the algorithms are explained in patch 15/16, so I won't repeat them here. v1 of the series was here: https://patchwork.libcamera.org/project/libcamera/list/?series=5122 v2 was here: https://patchwork.libcamera.org/project/libcamera/list/?series=5364 v3 was here: https://patchwork.libcamera.org/project/libcamera/list/?series=5382 v4 was here: https://patchwork.libcamera.org/project/libcamera/list/?series=5449 Changes in v5: The Wdr controls were moved from draft to core namespace and a few small fixes in patch 1 and 18. CI is also happy: https://gitlab.freedesktop.org/camera/libcamera/-/pipelines/1513542 So I guess I'll merge that :-) Changes in v4: Some smaller fixes from the review. Please see individual patches for the local changelog. Changes in v3: I hope I addressed all points from the last review. Some feedback got me thinking and that lead to the WDR algorithm being reworked a bit. Basically I replaced the former regulation loop + WDR-exposure-value with a additional constraint in the AGC regulation. That allowed me to ditch the PID controller (for now). I believe the structure is easier now. For that to work I needed a way to add constraints to the AGC at runtime. The current implementation is in patch 16. I'm not entirely happy with that due to dynamic allocations in the scope of the frame context. So maybe we have to come up with a neater way there. Changes in v2: Compared to v1 of the series there were quite some changes. The old series contained mitigation for exposure quantization using the AWB gains. During my work on HDR stitching support it got clear that we need to use the compression curve from the companding block of the imx8mp. That curve can serve as a way better gain than the AWB gains, so I added a compression algorithm (patches 1-2) which then is used for the quantization correction. The quantization correction was generalized to support exposure and gain quantization effects (patches 3-9). Patch 10 spilled over from the WDR preparation series and now has a (possibly) better commit message. The rest is the actual WDR implementation that was mostly there in v1. I fixed a few bugs and included changes from review (thanks Paul). The details are in the individual patches. Best regards, Stefan Stefan Klug (19): ipa: rkisp1: Add basic compression algorithm tuning: rksip1: Add a static Compress entry libipa: camera_sensor_helper: Add quantizeGain() function libipa: exposure_mode_helper: Take exposure/gain quantization into account libipa: exposure_mode_helper: Remove double calculation of lastStageGain libipa: exposure_mode_helper: Remove unnecessary clamp calls libipa: agc_mean_luminance: Fix constraint logging libipa: agc_mean_luminance: Configure the exposure mode helpers libipa: exposure_mode_helper: Calculate quantization gain in splitExposure() ipa: rkisp1: agc: Add correction for exposure quantization pipeline: rkisp1: Add error log when parameter queuing fails include: linux: Partially update linux headers from v6.16-rc1-310-gd968e50b5c26 ipa: rkisp1: Switch histogram to RGB combined mode pipeline: rkisp1: Query kernel for available params blocks libipa: agc_mean_luminance: Introduce effectiveYTarget() accessor libipa: agc_mean_luminance: Add support for additional constraints rkisp1: agc: Agc add yTarget to frame context ipa: rkisp1: Add WDR algorithm tuning: rksip1: Add a static WideDynamicRange entry include/libcamera/ipa/rkisp1.mojom | 2 +- include/linux/rkisp1-config.h | 108 ++++- include/linux/v4l2-controls.h | 6 + src/ipa/ipu3/algorithms/agc.cpp | 6 +- src/ipa/libipa/agc_mean_luminance.cpp | 59 ++- src/ipa/libipa/agc_mean_luminance.h | 8 +- src/ipa/libipa/camera_sensor_helper.cpp | 23 ++ src/ipa/libipa/camera_sensor_helper.h | 1 + src/ipa/libipa/exposure_mode_helper.cpp | 108 +++-- src/ipa/libipa/exposure_mode_helper.h | 12 +- src/ipa/mali-c55/algorithms/agc.cpp | 7 +- src/ipa/rkisp1/algorithms/agc.cpp | 85 +++- src/ipa/rkisp1/algorithms/blc.cpp | 4 +- src/ipa/rkisp1/algorithms/compress.cpp | 103 +++++ src/ipa/rkisp1/algorithms/compress.h | 30 ++ src/ipa/rkisp1/algorithms/goc.cpp | 4 +- src/ipa/rkisp1/algorithms/lux.cpp | 2 +- src/ipa/rkisp1/algorithms/meson.build | 2 + src/ipa/rkisp1/algorithms/wdr.cpp | 493 +++++++++++++++++++++++ src/ipa/rkisp1/algorithms/wdr.h | 58 +++ src/ipa/rkisp1/ipa_context.cpp | 19 + src/ipa/rkisp1/ipa_context.h | 32 +- src/ipa/rkisp1/params.cpp | 1 + src/ipa/rkisp1/params.h | 2 + src/ipa/rkisp1/rkisp1.cpp | 12 +- src/libcamera/control_ids_core.yaml | 62 +++ src/libcamera/pipeline/rkisp1/rkisp1.cpp | 38 +- utils/tuning/rkisp1.py | 8 +- 28 files changed, 1211 insertions(+), 84 deletions(-) create mode 100644 src/ipa/rkisp1/algorithms/compress.cpp create mode 100644 src/ipa/rkisp1/algorithms/compress.h create mode 100644 src/ipa/rkisp1/algorithms/wdr.cpp create mode 100644 src/ipa/rkisp1/algorithms/wdr.h