From patchwork Thu Jan 23 11:40:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22615 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2527ABDE6B for ; Thu, 23 Jan 2025 11:42:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0320A6855B; Thu, 23 Jan 2025 12:42:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nFJf+kqn"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 89CC961878 for ; Thu, 23 Jan 2025 12:42:11 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:c0a:33cd:b453:5d3f]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EE350465; Thu, 23 Jan 2025 12:41:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1737632468; bh=ViB+/2LaWYwnv8VGYJ6/1UvEzYlrpb1y8bJu2/pYC8U=; h=From:To:Cc:Subject:Date:From; b=nFJf+kqnHWq1gLCg9wVXRtctCZdyu5AAai//zOENDksIci+sDa2LlYJk/AafM1L5w bRzyQQeN1+B4YGxHGwJb5mKeiZmx3/pYc7CQp/LrJlMimIPXJNppdUTlktOl0Msm0H w73RTUOMMiu7XP3GTc6jfhndfXvqSsNHxiruoBV0= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 00/17] Add Bayesian AWB algorithm to libipa and rkisp1 Date: Thu, 23 Jan 2025 12:40:50 +0100 Message-ID: <20250123114204.79321-1-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Bayesian AWB algorithm improves the AWB performance by taking prior likelihoods for a colour temperature given a measured lux level into account. The algorithm was first implemented by RaspberryPi and is documented in the RaspberryPi camera guide. This series takes that implementation and adapts it for the use on libipa and the rkisp1 pipeline. It still keeps the grey world algorithm as a fallback or as a simple alternative that does not require any tuning. Patches 1-2 add small utility functions in lipia. Patches 3-7 prepare for the bayes AWB integration. First some base classes are added to be able to implement different AWB algorithms with a common interface. Then the grey world algorithm from rkisp1 is moved into libipa. Patches 8-10 add the bayes AWB algorithm to libtuning and libipa and integrate it into the rkisp1 IPA. Patch 11 is a small fix to start the camera with better default gains. In version 2 the review comments were handled. Additionally the lux support was completed with a tuning module and corresponding plumbing (patches 12-14). On top of that the bayesian AWB algorithm logging was further improved and the scale of the priors was changed to be linear. I kept the last three patches at the end because they show structural changes made compared to the original version (which was already reviewed). I could squash them into the corresponding earlier patches if needed. Please see the individual patches for a detailed changelog. Best regards, Stefan Stefan Klug (17): libipa: interpolator: Add accessor to internal data libipa: pwl: Add clear() function libipa: Add AWB algorithm base class libipa: awb: Add helper functions for AWB mode support libipa: Add grey world AWB algorithm ipa: rkisp1: Move calculation of RGB means into own function ipa: rkisp1: Use grey world algorithm from libipa libtuning: module: awb: Add bayes AWB support libipa: Add bayesian AWB algorithm ipa: rkisp1: Add support for bayes AWB algorithm from libipa ipa: rkisp1: awb: Apply gains based on default colour temperature on start libtuning: Add module for lux calibration utils: tuning: rkisp1: Add lux module libipa: lux: Update for a normalized referenceY to 1 libipa: awb_bayes: Add logging of value limits libipa: awb_bayes: Remove overly verbose log messages libipa: awb_bayes: Change the probabilities from log space into linear space src/ipa/libipa/awb.cpp | 265 ++++++++++ src/ipa/libipa/awb.h | 64 +++ src/ipa/libipa/awb_bayes.cpp | 500 ++++++++++++++++++ src/ipa/libipa/awb_bayes.h | 67 +++ src/ipa/libipa/awb_grey.cpp | 114 ++++ src/ipa/libipa/awb_grey.h | 35 ++ src/ipa/libipa/interpolator.cpp | 10 +- src/ipa/libipa/interpolator.h | 5 + src/ipa/libipa/lux.cpp | 16 +- src/ipa/libipa/lux.h | 3 +- src/ipa/libipa/meson.build | 6 + src/ipa/libipa/pwl.cpp | 5 + src/ipa/libipa/pwl.h | 1 + src/ipa/rkisp1/algorithms/awb.cpp | 166 ++++-- src/ipa/rkisp1/algorithms/awb.h | 7 +- src/ipa/rkisp1/algorithms/lux.cpp | 4 - utils/tuning/config-example.yaml | 44 +- utils/tuning/libtuning/modules/awb/awb.py | 16 +- utils/tuning/libtuning/modules/awb/rkisp1.py | 21 +- .../tuning/libtuning/modules/lux/__init__.py | 6 + utils/tuning/libtuning/modules/lux/lux.py | 70 +++ utils/tuning/libtuning/modules/lux/rkisp1.py | 22 + utils/tuning/rkisp1.py | 14 +- 23 files changed, 1367 insertions(+), 94 deletions(-) create mode 100644 src/ipa/libipa/awb.cpp create mode 100644 src/ipa/libipa/awb.h create mode 100644 src/ipa/libipa/awb_bayes.cpp create mode 100644 src/ipa/libipa/awb_bayes.h create mode 100644 src/ipa/libipa/awb_grey.cpp create mode 100644 src/ipa/libipa/awb_grey.h create mode 100644 utils/tuning/libtuning/modules/lux/__init__.py create mode 100644 utils/tuning/libtuning/modules/lux/lux.py create mode 100644 utils/tuning/libtuning/modules/lux/rkisp1.py