From patchwork Tue Apr 30 17:34:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19971 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 25329C32A2 for ; Tue, 30 Apr 2024 17:35:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C1E7B63424; Tue, 30 Apr 2024 19:35:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="ZaHxMEMX"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A19866341B for ; Tue, 30 Apr 2024 19:35:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1714498499; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=olvBuaQpgD6Wxrktiam3erqiSenpkLpngmJ1trLOqW4=; b=ZaHxMEMXcd3nVGkdZ6tWv4Fyg6EdikW/x3ZG6IV1IYecrtq01ytJCGborEUlgkV92fo0vG B9Bb85fN16CMvrrfIxrTXjOm8xt3dNng7yeCLzXvoy4iA041H2li2JgLUtkIHikFHYF1FV 4nUvKGMxj0IdIatR4lqZ1gbRcUz1q3E= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-296-K9mOjVFvODeV4GkJKO71yQ-1; Tue, 30 Apr 2024 13:34:53 -0400 X-MC-Unique: K9mOjVFvODeV4GkJKO71yQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9D6C81C03149; Tue, 30 Apr 2024 17:34:52 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 90EB9EC687; Tue, 30 Apr 2024 17:34:51 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart Subject: [PATCH v2 1/5] libcamera: software_isp: Use a specific integer type for black level Date: Tue, 30 Apr 2024 19:34:26 +0200 Message-ID: <20240430173430.200392-2-mzamazal@redhat.com> In-Reply-To: <20240430173430.200392-1-mzamazal@redhat.com> References: <20240430173430.200392-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The documented range of values corresponds to uint8_t, so let's use that type. Signed-off-by: Milan Zamazal Reviewed-by: Laurent Pinchart Reviewed-by: Andrei Konovalov --- src/ipa/simple/black_level.cpp | 2 +- src/ipa/simple/black_level.h | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/src/ipa/simple/black_level.cpp b/src/ipa/simple/black_level.cpp index c7e8d8b7..410dc17b 100644 --- a/src/ipa/simple/black_level.cpp +++ b/src/ipa/simple/black_level.cpp @@ -43,7 +43,7 @@ BlackLevel::BlackLevel() * \return The black level, in the range from 0 (minimum) to 255 (maximum). * If the black level couldn't be determined yet, return 0. */ -unsigned int BlackLevel::get() const +uint8_t BlackLevel::get() const { return blackLevelSet_ ? blackLevel_ : 0; } diff --git a/src/ipa/simple/black_level.h b/src/ipa/simple/black_level.h index 7e37757e..e6cfa7ed 100644 --- a/src/ipa/simple/black_level.h +++ b/src/ipa/simple/black_level.h @@ -8,6 +8,7 @@ #pragma once #include +#include #include "libcamera/internal/software_isp/swisp_stats.h" @@ -17,11 +18,11 @@ class BlackLevel { public: BlackLevel(); - unsigned int get() const; + uint8_t get() const; void update(SwIspStats::Histogram &yHistogram); private: - unsigned int blackLevel_; + uint8_t blackLevel_; bool blackLevelSet_; }; From patchwork Tue Apr 30 17:34:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19969 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6C0A3C3220 for ; Tue, 30 Apr 2024 17:34:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 153C16341B; Tue, 30 Apr 2024 19:34:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="bgzswFVK"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E6B1A63415 for ; Tue, 30 Apr 2024 19:34:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1714498496; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=C7GP/0/DF8Lmd5f5yLEVjIMYHES4hilTXexSX60y46o=; b=bgzswFVK2jx8vdmggtDoPU9vAWVCQMMy5t0kHVZhxfT+ggNRZeN+kCJGTUOOdKIG3KkWUe vYm/xeH8B78FqbrU3vkodythvkXqiAG7eL/hLw2E0pfe7VUFZ0XW4ZCaWChUsUZNNzly2M dCryESGazQYD/lgrcVyFyS+gY0lekBI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-501-anaKCHfEPquiJm3a0QuSgw-1; Tue, 30 Apr 2024 13:34:54 -0400 X-MC-Unique: anaKCHfEPquiJm3a0QuSgw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 10ED5182717E; Tue, 30 Apr 2024 17:34:54 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 07EC51BDB2; Tue, 30 Apr 2024 17:34:52 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart Subject: [PATCH v2 2/5] libcamera: software_isp: Honor black level in AWB Date: Tue, 30 Apr 2024 19:34:27 +0200 Message-ID: <20240430173430.200392-3-mzamazal@redhat.com> In-Reply-To: <20240430173430.200392-1-mzamazal@redhat.com> References: <20240430173430.200392-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The white balance computation didn't consider black level. This is wrong because then the computed ratios are off when they are computed from the whole brightness range rather than the sensor range. This patch adjusts white balance computation for the black level. There is no need to change white balance application in debayering as this is already applied after black level correction. Exposure computation already subtracts black level, no changes are needed there. Signed-off-by: Milan Zamazal Reviewed-by: Andrei Konovalov --- src/ipa/simple/soft_simple.cpp | 36 ++++++++++++++++++++++------------ 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index b9fb58b5..f595abc2 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -5,6 +5,8 @@ * soft_simple.cpp - Simple Software Image Processing Algorithm module */ +#include +#include #include #include @@ -240,28 +242,36 @@ void IPASoftSimple::stop() void IPASoftSimple::processStats(const ControlList &sensorControls) { + SwIspStats::Histogram histogram = stats_->yHistogram; + if (ignoreUpdates_ > 0) + blackLevel_.update(histogram); + const uint8_t blackLevel = blackLevel_.get(); + params_->blackLevel = blackLevel; + /* * Calculate red and blue gains for AWB. * Clamp max gain at 4.0, this also avoids 0 division. + * Black level must be subtracted to get the correct AWB ratios, + * from the sensor range. */ - if (stats_->sumR_ <= stats_->sumG_ / 4) - params_->gainR = 1024; - else - params_->gainR = 256 * stats_->sumG_ / stats_->sumR_; - - if (stats_->sumB_ <= stats_->sumG_ / 4) - params_->gainB = 1024; - else - params_->gainB = 256 * stats_->sumG_ / stats_->sumB_; + const uint64_t nPixels = std::accumulate( + histogram.begin(), histogram.end(), 0); + auto subtractBlackLevel = [blackLevel, nPixels]( + uint64_t sum, uint64_t div) + -> uint64_t { + return sum - blackLevel * nPixels / div; + }; + const uint64_t sumR = subtractBlackLevel(stats_->sumR_, 4); + const uint64_t sumG = subtractBlackLevel(stats_->sumG_, 2); + const uint64_t sumB = subtractBlackLevel(stats_->sumB_, 4); + + params_->gainR = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; + params_->gainB = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; /* Green gain and gamma values are fixed */ params_->gainG = 256; params_->gamma = 0.5; - if (ignoreUpdates_ > 0) - blackLevel_.update(stats_->yHistogram); - params_->blackLevel = blackLevel_.get(); - setIspParams.emit(); /* \todo Switch to the libipa/algorithm.h API someday. */ From patchwork Tue Apr 30 17:34:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19972 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A7F95C32C9 for ; Tue, 30 Apr 2024 17:35:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ED4DE6341E; Tue, 30 Apr 2024 19:35:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="ZJDVMrGs"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F1F946341F for ; Tue, 30 Apr 2024 19:35:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1714498501; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+vOAR6YpKScmu4TPdHN59HdLNE9qKv6Lfoio9bNn1SQ=; b=ZJDVMrGsJTZsluBlR6rMklCY9jitx/aO1m39jKPRHutXB0YLKWdZpODS7mXv4dyPPIdUbc WbB8yvl51iq5BHG5uPrT90OrTkHEECarckgGIsYIoNEjoj4dqyR9xRvddWb82Xt/EBd0I/ tfLKzqrgWNAd7qnIoJFh6z+/qqaK97M= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-658-fVd62z06MzertGIWWC0KuQ-1; Tue, 30 Apr 2024 13:34:57 -0400 X-MC-Unique: fVd62z06MzertGIWWC0KuQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6A4E11C02CB1; Tue, 30 Apr 2024 17:34:55 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 707B551BF; Tue, 30 Apr 2024 17:34:54 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart Subject: [PATCH v2 3/5] libcamera: software_isp: Move color mappings out of debayering Date: Tue, 30 Apr 2024 19:34:28 +0200 Message-ID: <20240430173430.200392-4-mzamazal@redhat.com> In-Reply-To: <20240430173430.200392-1-mzamazal@redhat.com> References: <20240430173430.200392-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Constructing the color mapping tables is related to stats rather than debayering, where they are applied. Let's move the corresponding code to stats processing. This is a preliminary step towards building this functionality on top of libipa/algorithm.h, which should follow. Signed-off-by: Milan Zamazal Reviewed-by: Andrei Konovalov --- .../internal/software_isp/debayer_params.h | 18 +++---- src/ipa/simple/soft_simple.cpp | 51 +++++++++++++++---- src/libcamera/software_isp/debayer.cpp | 29 +++++------ src/libcamera/software_isp/debayer_cpu.cpp | 41 +++------------ src/libcamera/software_isp/debayer_cpu.h | 9 ++-- src/libcamera/software_isp/software_isp.cpp | 4 +- 6 files changed, 75 insertions(+), 77 deletions(-) diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index 32cd448a..09f4ff00 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2023, Red Hat Inc. + * Copyright (C) 2023, 2024 Red Hat Inc. * * Authors: * Hans de Goede @@ -10,20 +10,20 @@ #pragma once +#include +#include + namespace libcamera { struct DebayerParams { static constexpr unsigned int kGain10 = 256; + static constexpr unsigned int kRGBLookupSize = 256; - unsigned int gainR; - unsigned int gainG; - unsigned int gainB; + using ColorLookupTable = std::array; - float gamma; - /** - * \brief Level of the black point, 0..255, 0 is no correction. - */ - unsigned int blackLevel; + ColorLookupTable red; + ColorLookupTable green; + ColorLookupTable blue; }; } /* namespace libcamera */ diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index f595abc2..981e1ae0 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -5,6 +5,7 @@ * soft_simple.cpp - Simple Software Image Processing Algorithm module */ +#include #include #include #include @@ -84,6 +85,10 @@ private: ControlInfoMap sensorInfoMap_; BlackLevel blackLevel_; + static constexpr unsigned int kGammaLookupSize = 1024; + std::array gammaTable_; + int lastBlackLevel_ = -1; + int32_t exposureMin_, exposureMax_; int32_t exposure_; double againMin_, againMax_, againMinStep_; @@ -246,7 +251,6 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) if (ignoreUpdates_ > 0) blackLevel_.update(histogram); const uint8_t blackLevel = blackLevel_.get(); - params_->blackLevel = blackLevel; /* * Calculate red and blue gains for AWB. @@ -265,12 +269,41 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) const uint64_t sumG = subtractBlackLevel(stats_->sumG_, 2); const uint64_t sumB = subtractBlackLevel(stats_->sumB_, 4); - params_->gainR = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; - params_->gainB = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; - + /* Gain: 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. */ + const unsigned int gainR = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; + const unsigned int gainB = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; /* Green gain and gamma values are fixed */ - params_->gainG = 256; - params_->gamma = 0.5; + constexpr unsigned int gainG = 256; + /* gamma == 1.0 means no correction */ + constexpr float gamma = 0.5; + + /* Update the gamma table if needed */ + if (blackLevel != lastBlackLevel_) { + const unsigned int blackIndex = blackLevel * kGammaLookupSize / 256; + std::fill(gammaTable_.begin(), gammaTable_.begin() + blackIndex, 0); + const float divisor = kGammaLookupSize - blackIndex - 1.0; + for (unsigned int i = blackIndex; i < kGammaLookupSize; i++) + gammaTable_[i] = UINT8_MAX * powf((i - blackIndex) / divisor, gamma); + + lastBlackLevel_ = blackLevel; + } + + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { + constexpr unsigned int div = + DebayerParams::kRGBLookupSize * DebayerParams::kGain10 / + kGammaLookupSize; + unsigned int idx; + + /* Apply gamma after gain! */ + idx = std::min({ i * gainR / div, (kGammaLookupSize - 1) }); + params_->red[i] = gammaTable_[idx]; + + idx = std::min({ i * gainG / div, (kGammaLookupSize - 1) }); + params_->green[i] = gammaTable_[idx]; + + idx = std::min({ i * gainB / div, (kGammaLookupSize - 1) }); + params_->blue[i] = gammaTable_[idx]; + } setIspParams.emit(); @@ -291,7 +324,7 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) * https://www.araa.asn.au/acra/acra2007/papers/paper84final.pdf */ const unsigned int blackLevelHistIdx = - params_->blackLevel / (256 / SwIspStats::kYHistogramSize); + blackLevel / (256 / SwIspStats::kYHistogramSize); const unsigned int histogramSize = SwIspStats::kYHistogramSize - blackLevelHistIdx; const unsigned int yHistValsPerBin = histogramSize / kExposureBinsCount; @@ -339,8 +372,8 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) LOG(IPASoft, Debug) << "exposureMSV " << exposureMSV << " exp " << exposure_ << " again " << again_ - << " gain R/B " << params_->gainR << "/" << params_->gainB - << " black level " << params_->blackLevel; + << " gain R/B " << gainR << "/" << gainB + << " black level " << blackLevel; } void IPASoftSimple::updateExposure(double exposureMSV) diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index 1c035e9b..ac438a33 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* * Copyright (C) 2023, Linaro Ltd - * Copyright (C) 2023, Red Hat Inc. + * Copyright (C) 2023, 2024 Red Hat Inc. * * Authors: * Hans de Goede @@ -24,29 +24,28 @@ namespace libcamera { */ /** - * \var DebayerParams::gainR - * \brief Red gain - * - * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + * \var DebayerParams::kRGBLookupSize + * \brief Size of a color lookup table */ /** - * \var DebayerParams::gainG - * \brief Green gain - * - * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + * \typedef DebayerParams::ColorLookupTable + * \brief Type of the lookup tables for red, green, blue values */ /** - * \var DebayerParams::gainB - * \brief Blue gain - * - * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + * \var DebayerParams::red + * \brief Lookup table for red color, mapping input values to output values + */ + +/** + * \var DebayerParams::green + * \brief Lookup table for green color, mapping input values to output values */ /** - * \var DebayerParams::gamma - * \brief Gamma correction, 1.0 is no correction + * \var DebayerParams::blue + * \brief Lookup table for blue color, mapping input values to output values */ /** diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 88d6578b..8b2b2f40 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -11,7 +11,6 @@ #include "debayer_cpu.h" -#include #include #include @@ -47,9 +46,9 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats) */ enableInputMemcpy_ = true; - /* Initialize gamma to 1.0 curve */ - for (unsigned int i = 0; i < kGammaLookupSize; i++) - gamma_[i] = i / (kGammaLookupSize / kRGBLookupSize); + /* Initialize color lookup tables */ + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) + red_[i] = green_[i] = blue_[i] = i; for (unsigned int i = 0; i < kMaxLineBuffers; i++) lineBuffers_[i] = nullptr; @@ -698,37 +697,9 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams clock_gettime(CLOCK_MONOTONIC_RAW, &frameStartTime); } - /* Apply DebayerParams */ - if (params.gamma != gammaCorrection_ || params.blackLevel != blackLevel_) { - const unsigned int blackIndex = - params.blackLevel * kGammaLookupSize / 256; - std::fill(gamma_.begin(), gamma_.begin() + blackIndex, 0); - const float divisor = kGammaLookupSize - blackIndex - 1.0; - for (unsigned int i = blackIndex; i < kGammaLookupSize; i++) - gamma_[i] = UINT8_MAX * powf((i - blackIndex) / divisor, params.gamma); - - gammaCorrection_ = params.gamma; - blackLevel_ = params.blackLevel; - } - - if (swapRedBlueGains_) - std::swap(params.gainR, params.gainB); - - for (unsigned int i = 0; i < kRGBLookupSize; i++) { - constexpr unsigned int div = - kRGBLookupSize * DebayerParams::kGain10 / kGammaLookupSize; - unsigned int idx; - - /* Apply gamma after gain! */ - idx = std::min({ i * params.gainR / div, (kGammaLookupSize - 1) }); - red_[i] = gamma_[idx]; - - idx = std::min({ i * params.gainG / div, (kGammaLookupSize - 1) }); - green_[i] = gamma_[idx]; - - idx = std::min({ i * params.gainB / div, (kGammaLookupSize - 1) }); - blue_[i] = gamma_[idx]; - } + green_ = params.green; + red_ = swapRedBlueGains_ ? params.blue : params.red; + blue_ = swapRedBlueGains_ ? params.red : params.blue; /* Copy metadata from the input buffer */ FrameMetadata &metadata = output->_d()->metadata(); diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 689c1075..47373426 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -122,15 +122,12 @@ private: void process2(const uint8_t *src, uint8_t *dst); void process4(const uint8_t *src, uint8_t *dst); - static constexpr unsigned int kGammaLookupSize = 1024; - static constexpr unsigned int kRGBLookupSize = 256; /* Max. supported Bayer pattern height is 4, debayering this requires 5 lines */ static constexpr unsigned int kMaxLineBuffers = 5; - std::array gamma_; - std::array red_; - std::array green_; - std::array blue_; + DebayerParams::ColorLookupTable red_; + DebayerParams::ColorLookupTable green_; + DebayerParams::ColorLookupTable blue_; debayerFn debayer0_; debayerFn debayer1_; debayerFn debayer2_; diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index e4e56086..3e07453d 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -63,9 +63,7 @@ LOG_DEFINE_CATEGORY(SoftwareIsp) * handler */ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor) - : debayerParams_{ DebayerParams::kGain10, DebayerParams::kGain10, - DebayerParams::kGain10, 0.5f, 0 }, - dmaHeap_(DmaHeap::DmaHeapFlag::Cma | DmaHeap::DmaHeapFlag::System) + : dmaHeap_(DmaHeap::DmaHeapFlag::Cma | DmaHeap::DmaHeapFlag::System) { if (!dmaHeap_.isValid()) { LOG(SoftwareIsp, Error) << "Failed to create DmaHeap object"; From patchwork Tue Apr 30 17:34:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19970 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2F52CC3220 for ; Tue, 30 Apr 2024 17:35:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CE8686341B; Tue, 30 Apr 2024 19:35:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="S34g3DBp"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E642E63418 for ; Tue, 30 Apr 2024 19:35:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1714498500; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ljbHLZZsSzg15d6L7pje8jb7SlEffnNyoibm7/huId4=; b=S34g3DBpFJJgSG+KJmx32twlKBpSbSXpzlvEXmoYB0qE9MuhtHIqNklBmudImN5+tAzQF/ s8CcwRTgVwbBFca+e6xmwp/gxHga20atboj2+DtpUy3VMbOmJq/ya770XKFebYvXam0hu0 VdkLqbJbRgJ5/kHd4D61mEX7Gb5ZXjM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-596-_ZpdexPhNZSeIYBu4ddwuA-1; Tue, 30 Apr 2024 13:34:58 -0400 X-MC-Unique: _ZpdexPhNZSeIYBu4ddwuA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EEC86104D982; Tue, 30 Apr 2024 17:34:56 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id C16F31BDB2; Tue, 30 Apr 2024 17:34:55 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart Subject: [PATCH v2 4/5] libcamera: software_isp: Remove DebayerParams::kGain10 Date: Tue, 30 Apr 2024 19:34:29 +0200 Message-ID: <20240430173430.200392-5-mzamazal@redhat.com> In-Reply-To: <20240430173430.200392-1-mzamazal@redhat.com> References: <20240430173430.200392-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The constant is used in a single place internally and doesn't belong to DebayerParams anymore. Let's use 256 directly. Signed-off-by: Milan Zamazal Reviewed-by: Andrei Konovalov --- include/libcamera/internal/software_isp/debayer_params.h | 1 - src/ipa/simple/soft_simple.cpp | 3 +-- src/libcamera/software_isp/debayer.cpp | 5 ----- 3 files changed, 1 insertion(+), 8 deletions(-) diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index 09f4ff00..6aaa7d4c 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -16,7 +16,6 @@ namespace libcamera { struct DebayerParams { - static constexpr unsigned int kGain10 = 256; static constexpr unsigned int kRGBLookupSize = 256; using ColorLookupTable = std::array; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 981e1ae0..0af23ee7 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -290,8 +290,7 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { constexpr unsigned int div = - DebayerParams::kRGBLookupSize * DebayerParams::kGain10 / - kGammaLookupSize; + DebayerParams::kRGBLookupSize * 256 / kGammaLookupSize; unsigned int idx; /* Apply gamma after gain! */ diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index ac438a33..548c2ccd 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -18,11 +18,6 @@ namespace libcamera { * \brief Struct to hold the debayer parameters. */ -/** - * \var DebayerParams::kGain10 - * \brief const value for 1.0 gain - */ - /** * \var DebayerParams::kRGBLookupSize * \brief Size of a color lookup table From patchwork Tue Apr 30 17:34:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19973 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 386FDC32CA for ; Tue, 30 Apr 2024 17:35:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9322F63420; Tue, 30 Apr 2024 19:35:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="aMgSV3oc"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DE5EA6341E for ; Tue, 30 Apr 2024 19:35:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1714498500; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9vOTcsvb0XG7fZp/r6Z5kg0EbCgL2arKuk5L0aAvEys=; b=aMgSV3ocqXGpwCPv8YCnmAv/G1vd9VuZ79dUJ6n5qWbi0VxQh77D9F83lviLDwC4xr9gio FscN1uro0UZoZJ0f7OZ4b8l20srQA00uHAaD/xezF2WsIxMPmDtqtW2hVqIu/O6er0lXX+ tMx5pywEsALkiZMBckO62aiJm3jHfSU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-444-6cVfNg0LPFi9wtTvrMF9Jw-1; Tue, 30 Apr 2024 13:34:58 -0400 X-MC-Unique: 6cVfNg0LPFi9wtTvrMF9Jw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6450D80017B; Tue, 30 Apr 2024 17:34:58 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 506901BDB7; Tue, 30 Apr 2024 17:34:57 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart Subject: [PATCH v2 5/5] libcamera: software_isp: Pass color lookup tables as floats Date: Tue, 30 Apr 2024 19:34:30 +0200 Message-ID: <20240430173430.200392-6-mzamazal@redhat.com> In-Reply-To: <20240430173430.200392-1-mzamazal@redhat.com> References: <20240430173430.200392-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The color lookup tables are passed from stats processing to debayering for direct use as 8-bit color output values. Let's pass the values as 0.0..1.0 floats to make the gains color bit width independent. Completes software ISP TODO #4. Signed-off-by: Milan Zamazal Reviewed-by: Andrei Konovalov --- .../internal/software_isp/debayer_params.h | 2 +- src/ipa/simple/soft_simple.cpp | 4 ++-- src/libcamera/software_isp/TODO | 13 ------------- src/libcamera/software_isp/debayer.cpp | 6 +++--- src/libcamera/software_isp/debayer_cpu.cpp | 18 +++++++++++++++--- src/libcamera/software_isp/debayer_cpu.h | 10 +++++++--- 6 files changed, 28 insertions(+), 25 deletions(-) diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index 6aaa7d4c..a29eb37a 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -18,7 +18,7 @@ namespace libcamera { struct DebayerParams { static constexpr unsigned int kRGBLookupSize = 256; - using ColorLookupTable = std::array; + using ColorLookupTable = std::array; ColorLookupTable red; ColorLookupTable green; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 0af23ee7..017fd15a 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -86,7 +86,7 @@ private: BlackLevel blackLevel_; static constexpr unsigned int kGammaLookupSize = 1024; - std::array gammaTable_; + std::array gammaTable_; int lastBlackLevel_ = -1; int32_t exposureMin_, exposureMax_; @@ -283,7 +283,7 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) std::fill(gammaTable_.begin(), gammaTable_.begin() + blackIndex, 0); const float divisor = kGammaLookupSize - blackIndex - 1.0; for (unsigned int i = blackIndex; i < kGammaLookupSize; i++) - gammaTable_[i] = UINT8_MAX * powf((i - blackIndex) / divisor, gamma); + gammaTable_[i] = powf((i - blackIndex) / divisor, gamma); lastBlackLevel_ = blackLevel; } diff --git a/src/libcamera/software_isp/TODO b/src/libcamera/software_isp/TODO index 4fcee39b..6bdc5905 100644 --- a/src/libcamera/software_isp/TODO +++ b/src/libcamera/software_isp/TODO @@ -72,19 +72,6 @@ stats in hardware, such as the i.MX7), but please keep it on your radar. --- -4. Hide internal representation of gains from callers - -> struct DebayerParams { -> static constexpr unsigned int kGain10 = 256; - -Forcing the caller to deal with the internal representation of gains -isn't nice, especially given that it precludes implementing gains of -different precisions in different backend. Wouldn't it be better to pass -the values as floating point numbers, and convert them to the internal -representation in the implementation of process() before using them ? - ---- - 5. Store ISP parameters in per-frame buffers > /** diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index 548c2ccd..acc9cd21 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -30,17 +30,17 @@ namespace libcamera { /** * \var DebayerParams::red - * \brief Lookup table for red color, mapping input values to output values + * \brief Lookup table for red color, mapping input values to 0.0..1.0 */ /** * \var DebayerParams::green - * \brief Lookup table for green color, mapping input values to output values + * \brief Lookup table for green color, mapping input values to 0.0..1.0 */ /** * \var DebayerParams::blue - * \brief Lookup table for blue color, mapping input values to output values + * \brief Lookup table for blue color, mapping input values to 0.0..1.0 */ /** diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 8b2b2f40..df4c6e88 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -11,6 +11,8 @@ #include "debayer_cpu.h" +#include +#include #include #include @@ -688,6 +690,14 @@ static inline int64_t timeDiff(timespec &after, timespec &before) (int64_t)after.tv_nsec - (int64_t)before.tv_nsec; } +void DebayerCpu::updateColorLookupTable( + const DebayerParams::ColorLookupTable &src, + ColorLookupTable &dst) +{ + for (std::size_t i = 0; i < src.size(); i++) + dst[i] = src[i] * UINT8_MAX; +} + void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams params) { timespec frameStartTime; @@ -697,9 +707,11 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams clock_gettime(CLOCK_MONOTONIC_RAW, &frameStartTime); } - green_ = params.green; - red_ = swapRedBlueGains_ ? params.blue : params.red; - blue_ = swapRedBlueGains_ ? params.red : params.blue; + updateColorLookupTable(params.green, green_); + updateColorLookupTable(swapRedBlueGains_ ? params.blue : params.red, + red_); + updateColorLookupTable(swapRedBlueGains_ ? params.red : params.blue, + blue_); /* Copy metadata from the input buffer */ FrameMetadata &metadata = output->_d()->metadata(); diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 47373426..18808076 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -18,6 +18,7 @@ #include #include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/software_isp/debayer_params.h" #include "debayer.h" #include "swstats_cpu.h" @@ -125,9 +126,12 @@ private: /* Max. supported Bayer pattern height is 4, debayering this requires 5 lines */ static constexpr unsigned int kMaxLineBuffers = 5; - DebayerParams::ColorLookupTable red_; - DebayerParams::ColorLookupTable green_; - DebayerParams::ColorLookupTable blue_; + using ColorLookupTable = std::array; + void updateColorLookupTable(const DebayerParams::ColorLookupTable &src, + ColorLookupTable &dst); + ColorLookupTable red_; + ColorLookupTable green_; + ColorLookupTable blue_; debayerFn debayer0_; debayerFn debayer1_; debayerFn debayer2_;