From patchwork Thu May 30 21:09:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20153 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 468B4BD87C for ; Thu, 30 May 2024 21:10:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DF073634BA; Thu, 30 May 2024 23:10:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="AFdcwuLO"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F1D40634B5 for ; Thu, 30 May 2024 23:10:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1717103420; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WQa/iD8tOxjW06QizxUm8aX0OapgFoAkAtI5JQNluMg=; b=AFdcwuLOq2vFqce60tdzakWTRUuh7ZsTBZLCx6AC0blV+c8K19f3t6AyG3n0jYU/3RHl5q JNsuZYzvl0iL0sm7g+K3PcY4a2qGEHujNCvDnMBTR2S3jIHSXsOL4mzBXvhduXfk30gX+C 6QbmDXopObZ5B7ZtkQ6FjDSUcbeFQMk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-377-3MnYAzw8PrKiNgHgIMfblw-1; Thu, 30 May 2024 17:10:16 -0400 X-MC-Unique: 3MnYAzw8PrKiNgHgIMfblw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EDDD28058D1; Thu, 30 May 2024 21:10:15 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0129940C6EB7; Thu, 30 May 2024 21:10:13 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v5 1/5] libcamera: software_isp: Use a specific integer type for black level Date: Thu, 30 May 2024 23:09:53 +0200 Message-ID: <20240530210957.2211897-2-mzamazal@redhat.com> In-Reply-To: <20240530210957.2211897-1-mzamazal@redhat.com> References: <20240530210957.2211897-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The documented range of values corresponds to uint8_t, so let's use that type. Signed-off-by: Milan Zamazal Reviewed-by: Laurent Pinchart Reviewed-by: Andrei Konovalov --- src/ipa/simple/black_level.cpp | 2 +- src/ipa/simple/black_level.h | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/src/ipa/simple/black_level.cpp b/src/ipa/simple/black_level.cpp index 53b4c040..cc490eb5 100644 --- a/src/ipa/simple/black_level.cpp +++ b/src/ipa/simple/black_level.cpp @@ -43,7 +43,7 @@ BlackLevel::BlackLevel() * \return The black level, in the range from 0 (minimum) to 255 (maximum). * If the black level couldn't be determined yet, return 0. */ -unsigned int BlackLevel::get() const +uint8_t BlackLevel::get() const { return blackLevelSet_ ? blackLevel_ : 0; } diff --git a/src/ipa/simple/black_level.h b/src/ipa/simple/black_level.h index 25094217..5e032f9f 100644 --- a/src/ipa/simple/black_level.h +++ b/src/ipa/simple/black_level.h @@ -8,6 +8,7 @@ #pragma once #include +#include #include "libcamera/internal/software_isp/swisp_stats.h" @@ -17,11 +18,11 @@ class BlackLevel { public: BlackLevel(); - unsigned int get() const; + uint8_t get() const; void update(SwIspStats::Histogram &yHistogram); private: - unsigned int blackLevel_; + uint8_t blackLevel_; bool blackLevelSet_; }; From patchwork Thu May 30 21:09:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20154 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D0A02BD87C for ; Thu, 30 May 2024 21:10:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2D968634BE; Thu, 30 May 2024 23:10:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="CJEitlsk"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B439A634B5 for ; Thu, 30 May 2024 23:10:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1717103421; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SJgPpn2HtHOhkI0g0uQ7mR/RPptwwtRtM4m9nkZBjVc=; b=CJEitlskfXdUbpdg0ANdI9ZGarNnvmGKgD1R6gJoEYHP/x/Tt3M/TkWWtL9RfP5My4Vzit XeOJ758/yM1TOmVWUqpvkM6yjSDfqbYwCpJ/Q+PiYLBnmIa0gMepa2ZCbYq6aX6qPbMC8H DoHePWu0yOzBYTd7jouy6YxQxTdGoP0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-687-3klSFe_2PlqrjiUk8entng-1; Thu, 30 May 2024 17:10:18 -0400 X-MC-Unique: 3klSFe_2PlqrjiUk8entng-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 982D2101A525; Thu, 30 May 2024 21:10:17 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3E99440C6EB7; Thu, 30 May 2024 21:10:16 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v5 2/5] libcamera: software_isp: Honor black level in AWB Date: Thu, 30 May 2024 23:09:54 +0200 Message-ID: <20240530210957.2211897-3-mzamazal@redhat.com> In-Reply-To: <20240530210957.2211897-1-mzamazal@redhat.com> References: <20240530210957.2211897-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The white balance computation didn't consider black level. This is wrong because then the computed ratios are off when they are computed from the whole brightness range rather than the sensor range. This patch adjusts white balance computation for the black level. There is no need to change white balance application in debayering as this is already applied after black level correction. Exposure computation already subtracts black level, no changes are needed there. Signed-off-by: Milan Zamazal Reviewed-by: Andrei Konovalov Reviewed-by: Laurent Pinchart --- src/ipa/simple/soft_simple.cpp | 35 +++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-) diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index a5bb2bbf..f383f994 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -5,6 +5,8 @@ * Simple Software Image Processing Algorithm module */ +#include +#include #include #include @@ -240,28 +242,35 @@ void IPASoftSimple::stop() void IPASoftSimple::processStats(const ControlList &sensorControls) { + SwIspStats::Histogram histogram = stats_->yHistogram; + if (ignoreUpdates_ > 0) + blackLevel_.update(histogram); + const uint8_t blackLevel = blackLevel_.get(); + params_->blackLevel = blackLevel; + + /* + * Black level must be subtracted to get the correct AWB ratios, + * they would be off if they were computed from the whole brightness + * range rather than from the sensor range. + */ + const uint64_t nPixels = std::accumulate( + histogram.begin(), histogram.end(), 0); + const uint64_t offset = blackLevel * nPixels; + const uint64_t sumR = stats_->sumR_ - offset / 4; + const uint64_t sumG = stats_->sumG_ - offset / 2; + const uint64_t sumB = stats_->sumB_ - offset / 4; + /* * Calculate red and blue gains for AWB. * Clamp max gain at 4.0, this also avoids 0 division. */ - if (stats_->sumR_ <= stats_->sumG_ / 4) - params_->gainR = 1024; - else - params_->gainR = 256 * stats_->sumG_ / stats_->sumR_; - - if (stats_->sumB_ <= stats_->sumG_ / 4) - params_->gainB = 1024; - else - params_->gainB = 256 * stats_->sumG_ / stats_->sumB_; + params_->gainR = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; + params_->gainB = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; /* Green gain and gamma values are fixed */ params_->gainG = 256; params_->gamma = 0.5; - if (ignoreUpdates_ > 0) - blackLevel_.update(stats_->yHistogram); - params_->blackLevel = blackLevel_.get(); - setIspParams.emit(); /* \todo Switch to the libipa/algorithm.h API someday. */ From patchwork Thu May 30 21:09:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20155 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6E9B0C32C8 for ; Thu, 30 May 2024 21:10:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D0233634C0; Thu, 30 May 2024 23:10:27 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="KD3KaCVz"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2538D634BC for ; Thu, 30 May 2024 23:10:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1717103423; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2KIeslmbtQb94ed+BGAptnB0kbr3TMxKmCBdv3+0szo=; b=KD3KaCVzazpfCE8PkeZGaHWiQwwTzJ3p2J59hoEXw5/xIoRJivgAcfLLOW7JU1J7wqhNPz pkExGf212VDBGOo8IoLv13Qnfc3yYYySjmU5pQEIocVuXpVoxRUQxxyEFlBZC64goc3sDC 1SrAF7JIPLNGMRnQCdt0WzBD/GByNpM= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-41-Vdut7ObSP86I-Ay17_m7zA-1; Thu, 30 May 2024 17:10:19 -0400 X-MC-Unique: Vdut7ObSP86I-Ay17_m7zA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 563D13806700; Thu, 30 May 2024 21:10:19 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id F407840C6EB7; Thu, 30 May 2024 21:10:17 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v5 3/5] libcamera: software_isp: Move color mappings out of debayering Date: Thu, 30 May 2024 23:09:55 +0200 Message-ID: <20240530210957.2211897-4-mzamazal@redhat.com> In-Reply-To: <20240530210957.2211897-1-mzamazal@redhat.com> References: <20240530210957.2211897-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Constructing the color mapping tables is related to stats rather than debayering, where they are applied. Let's move the corresponding code to stats processing. The same applies to the auxiliary gamma table. As the gamma value is currently fixed and used in a single place, with the temporary exception mentioned below, there is no need to share it anywhere anymore. It's necessary to initialize SoftwareIsp::debayerParams_ to default values. These initial values are used for the first two frames, before they are changed based on determined stats. To avoid sharing the gamma value constant in artificial ways, we use 0.5 directly in the initialization. This all is not a particularly elegant thing to do, such a code belongs conceptually to the similar code in stats processing, but doing better is left for larger refactoring. This is a preliminary step towards building this functionality on top of libipa/algorithm.h, which should follow. Signed-off-by: Milan Zamazal Reviewed-by: Andrei Konovalov --- .../internal/software_isp/debayer_params.h | 18 +++---- src/ipa/simple/soft_simple.cpp | 51 +++++++++++++++---- src/libcamera/software_isp/debayer.cpp | 29 +++++------ src/libcamera/software_isp/debayer_cpu.cpp | 43 +++------------- src/libcamera/software_isp/debayer_cpu.h | 11 ++-- src/libcamera/software_isp/software_isp.cpp | 16 ++++-- 6 files changed, 88 insertions(+), 80 deletions(-) diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index ce1b5945..463d24b3 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2023, Red Hat Inc. + * Copyright (C) 2023, 2024 Red Hat Inc. * * Authors: * Hans de Goede @@ -10,20 +10,20 @@ #pragma once +#include +#include + namespace libcamera { struct DebayerParams { static constexpr unsigned int kGain10 = 256; + static constexpr unsigned int kRGBLookupSize = 256; - unsigned int gainR; - unsigned int gainG; - unsigned int gainB; + using ColorLookupTable = std::array; - float gamma; - /** - * \brief Level of the black point, 0..255, 0 is no correction. - */ - unsigned int blackLevel; + ColorLookupTable red; + ColorLookupTable green; + ColorLookupTable blue; }; } /* namespace libcamera */ diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index f383f994..3388686f 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -5,6 +5,7 @@ * Simple Software Image Processing Algorithm module */ +#include #include #include #include @@ -84,6 +85,10 @@ private: ControlInfoMap sensorInfoMap_; BlackLevel blackLevel_; + static constexpr unsigned int kGammaLookupSize = 1024; + std::array gammaTable_; + int lastBlackLevel_ = -1; + int32_t exposureMin_, exposureMax_; int32_t exposure_; double againMin_, againMax_, againMinStep_; @@ -246,7 +251,6 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) if (ignoreUpdates_ > 0) blackLevel_.update(histogram); const uint8_t blackLevel = blackLevel_.get(); - params_->blackLevel = blackLevel; /* * Black level must be subtracted to get the correct AWB ratios, @@ -263,13 +267,42 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) /* * Calculate red and blue gains for AWB. * Clamp max gain at 4.0, this also avoids 0 division. + * Gain: 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. */ - params_->gainR = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; - params_->gainB = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; - + const unsigned int gainR = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; + const unsigned int gainB = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; /* Green gain and gamma values are fixed */ - params_->gainG = 256; - params_->gamma = 0.5; + constexpr unsigned int gainG = 256; + + /* Update the gamma table if needed */ + if (blackLevel != lastBlackLevel_) { + constexpr float gamma = 0.5; + const unsigned int blackIndex = blackLevel * kGammaLookupSize / 256; + std::fill(gammaTable_.begin(), gammaTable_.begin() + blackIndex, 0); + const float divisor = kGammaLookupSize - blackIndex - 1.0; + for (unsigned int i = blackIndex; i < kGammaLookupSize; i++) + gammaTable_[i] = UINT8_MAX * + std::pow((i - blackIndex) / divisor, gamma); + + lastBlackLevel_ = blackLevel; + } + + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { + constexpr unsigned int div = + DebayerParams::kRGBLookupSize * DebayerParams::kGain10 / + kGammaLookupSize; + unsigned int idx; + + /* Apply gamma after gain! */ + idx = std::min({ i * gainR / div, (kGammaLookupSize - 1) }); + params_->red[i] = gammaTable_[idx]; + + idx = std::min({ i * gainG / div, (kGammaLookupSize - 1) }); + params_->green[i] = gammaTable_[idx]; + + idx = std::min({ i * gainB / div, (kGammaLookupSize - 1) }); + params_->blue[i] = gammaTable_[idx]; + } setIspParams.emit(); @@ -290,7 +323,7 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) * https://www.araa.asn.au/acra/acra2007/papers/paper84final.pdf */ const unsigned int blackLevelHistIdx = - params_->blackLevel / (256 / SwIspStats::kYHistogramSize); + blackLevel / (256 / SwIspStats::kYHistogramSize); const unsigned int histogramSize = SwIspStats::kYHistogramSize - blackLevelHistIdx; const unsigned int yHistValsPerBin = histogramSize / kExposureBinsCount; @@ -338,8 +371,8 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) LOG(IPASoft, Debug) << "exposureMSV " << exposureMSV << " exp " << exposure_ << " again " << again_ - << " gain R/B " << params_->gainR << "/" << params_->gainB - << " black level " << params_->blackLevel; + << " gain R/B " << gainR << "/" << gainB + << " black level " << static_cast(blackLevel); } void IPASoftSimple::updateExposure(double exposureMSV) diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index efe75ea8..3f3969f7 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* * Copyright (C) 2023, Linaro Ltd - * Copyright (C) 2023, Red Hat Inc. + * Copyright (C) 2023, 2024 Red Hat Inc. * * Authors: * Hans de Goede @@ -24,29 +24,28 @@ namespace libcamera { */ /** - * \var DebayerParams::gainR - * \brief Red gain - * - * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + * \var DebayerParams::kRGBLookupSize + * \brief Size of a color lookup table */ /** - * \var DebayerParams::gainG - * \brief Green gain - * - * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + * \typedef DebayerParams::ColorLookupTable + * \brief Type of the lookup tables for red, green, blue values */ /** - * \var DebayerParams::gainB - * \brief Blue gain - * - * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + * \var DebayerParams::red + * \brief Lookup table for red color, mapping input values to output values + */ + +/** + * \var DebayerParams::green + * \brief Lookup table for green color, mapping input values to output values */ /** - * \var DebayerParams::gamma - * \brief Gamma correction, 1.0 is no correction + * \var DebayerParams::blue + * \brief Lookup table for blue color, mapping input values to output values */ /** diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 8254bbe9..c038eed4 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -11,7 +11,6 @@ #include "debayer_cpu.h" -#include #include #include @@ -35,7 +34,7 @@ namespace libcamera { * \param[in] stats Pointer to the stats object to use */ DebayerCpu::DebayerCpu(std::unique_ptr stats) - : stats_(std::move(stats)), gammaCorrection_(1.0), blackLevel_(0) + : stats_(std::move(stats)) { /* * Reading from uncached buffers may be very slow. @@ -47,9 +46,9 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats) */ enableInputMemcpy_ = true; - /* Initialize gamma to 1.0 curve */ - for (unsigned int i = 0; i < kGammaLookupSize; i++) - gamma_[i] = i / (kGammaLookupSize / kRGBLookupSize); + /* Initialize color lookup tables */ + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) + red_[i] = green_[i] = blue_[i] = i; for (unsigned int i = 0; i < kMaxLineBuffers; i++) lineBuffers_[i] = nullptr; @@ -698,37 +697,9 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams clock_gettime(CLOCK_MONOTONIC_RAW, &frameStartTime); } - /* Apply DebayerParams */ - if (params.gamma != gammaCorrection_ || params.blackLevel != blackLevel_) { - const unsigned int blackIndex = - params.blackLevel * kGammaLookupSize / 256; - std::fill(gamma_.begin(), gamma_.begin() + blackIndex, 0); - const float divisor = kGammaLookupSize - blackIndex - 1.0; - for (unsigned int i = blackIndex; i < kGammaLookupSize; i++) - gamma_[i] = UINT8_MAX * powf((i - blackIndex) / divisor, params.gamma); - - gammaCorrection_ = params.gamma; - blackLevel_ = params.blackLevel; - } - - if (swapRedBlueGains_) - std::swap(params.gainR, params.gainB); - - for (unsigned int i = 0; i < kRGBLookupSize; i++) { - constexpr unsigned int div = - kRGBLookupSize * DebayerParams::kGain10 / kGammaLookupSize; - unsigned int idx; - - /* Apply gamma after gain! */ - idx = std::min({ i * params.gainR / div, (kGammaLookupSize - 1) }); - red_[i] = gamma_[idx]; - - idx = std::min({ i * params.gainG / div, (kGammaLookupSize - 1) }); - green_[i] = gamma_[idx]; - - idx = std::min({ i * params.gainB / div, (kGammaLookupSize - 1) }); - blue_[i] = gamma_[idx]; - } + green_ = params.green; + red_ = swapRedBlueGains_ ? params.blue : params.red; + blue_ = swapRedBlueGains_ ? params.red : params.blue; /* Copy metadata from the input buffer */ FrameMetadata &metadata = output->_d()->metadata(); diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index de216fe3..be7dcdca 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -122,15 +122,12 @@ private: void process2(const uint8_t *src, uint8_t *dst); void process4(const uint8_t *src, uint8_t *dst); - static constexpr unsigned int kGammaLookupSize = 1024; - static constexpr unsigned int kRGBLookupSize = 256; /* Max. supported Bayer pattern height is 4, debayering this requires 5 lines */ static constexpr unsigned int kMaxLineBuffers = 5; - std::array gamma_; - std::array red_; - std::array green_; - std::array blue_; + DebayerParams::ColorLookupTable red_; + DebayerParams::ColorLookupTable green_; + DebayerParams::ColorLookupTable blue_; debayerFn debayer0_; debayerFn debayer1_; debayerFn debayer2_; @@ -146,8 +143,6 @@ private: unsigned int xShift_; /* Offset of 0/1 applied to window_.x */ bool enableInputMemcpy_; bool swapRedBlueGains_; - float gammaCorrection_; - unsigned int blackLevel_; unsigned int measuredFrames_; int64_t frameProcessTime_; /* Skip 30 frames for things to stabilize then measure 30 frames */ diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index c9b6be56..2cde4f28 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -7,6 +7,8 @@ #include "libcamera/internal/software_isp/software_isp.h" +#include +#include #include #include #include @@ -18,6 +20,7 @@ #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/mapped_framebuffer.h" +#include "libcamera/internal/software_isp/debayer_params.h" #include "debayer_cpu.h" @@ -63,10 +66,17 @@ LOG_DEFINE_CATEGORY(SoftwareIsp) * handler */ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor) - : debayerParams_{ DebayerParams::kGain10, DebayerParams::kGain10, - DebayerParams::kGain10, 0.5f, 0 }, - dmaHeap_(DmaHeap::DmaHeapFlag::Cma | DmaHeap::DmaHeapFlag::System) + : dmaHeap_(DmaHeap::DmaHeapFlag::Cma | DmaHeap::DmaHeapFlag::System) { + std::array gammaTable; + for (unsigned int i = 0; i < 256; i++) + gammaTable[i] = UINT8_MAX * std::pow(i / 256.0, 0.5); + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { + debayerParams_.red[i] = gammaTable[i]; + debayerParams_.green[i] = gammaTable[i]; + debayerParams_.blue[i] = gammaTable[i]; + } + if (!dmaHeap_.isValid()) { LOG(SoftwareIsp, Error) << "Failed to create DmaHeap object"; return; From patchwork Thu May 30 21:09:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20156 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 80520BD87C for ; Thu, 30 May 2024 21:10:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1A441634C0; Thu, 30 May 2024 23:10:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="ShVuKnEX"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 398ED634BA for ; Thu, 30 May 2024 23:10:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1717103424; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hMOYFPNh1mwKw7Jp42jxDGKyIob1XorqW1o8GBmD+IU=; b=ShVuKnEX8Du5F9Trn7fzS5C8gqlbvgnAsHv5Df1+LwRlnc7KT4E8hrJD8MiVzwh9wHY2hJ cPDADFkBhOHo4HXyjVvHkI31fAYKOmvHP3gFLpZBLw0wXf9EPWW594yCJouTC1h6CxKXCh 9Qprx01mxFdeNmwRPviyRKnEJTaNdmw= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-481-MDQTl5zRNWOV7_xSmkwK1Q-1; Thu, 30 May 2024 17:10:21 -0400 X-MC-Unique: MDQTl5zRNWOV7_xSmkwK1Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2EB021C05122; Thu, 30 May 2024 21:10:21 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id B456E40C6EB7; Thu, 30 May 2024 21:10:19 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v5 4/5] libcamera: software_isp: Remove DebayerParams::kGain10 Date: Thu, 30 May 2024 23:09:56 +0200 Message-ID: <20240530210957.2211897-5-mzamazal@redhat.com> In-Reply-To: <20240530210957.2211897-1-mzamazal@redhat.com> References: <20240530210957.2211897-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The constant is used in a single place internally and doesn't belong to DebayerParams anymore. Let's use 256 directly. Signed-off-by: Milan Zamazal Reviewed-by: Andrei Konovalov Reviewed-by: Laurent Pinchart --- include/libcamera/internal/software_isp/debayer_params.h | 1 - src/ipa/simple/soft_simple.cpp | 3 +-- src/libcamera/software_isp/debayer.cpp | 5 ----- 3 files changed, 1 insertion(+), 8 deletions(-) diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index 463d24b3..7d8fdd48 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -16,7 +16,6 @@ namespace libcamera { struct DebayerParams { - static constexpr unsigned int kGain10 = 256; static constexpr unsigned int kRGBLookupSize = 256; using ColorLookupTable = std::array; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 3388686f..fb974bc3 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -289,8 +289,7 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { constexpr unsigned int div = - DebayerParams::kRGBLookupSize * DebayerParams::kGain10 / - kGammaLookupSize; + DebayerParams::kRGBLookupSize * 256 / kGammaLookupSize; unsigned int idx; /* Apply gamma after gain! */ diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index 3f3969f7..f4a299d5 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -18,11 +18,6 @@ namespace libcamera { * \brief Struct to hold the debayer parameters. */ -/** - * \var DebayerParams::kGain10 - * \brief const value for 1.0 gain - */ - /** * \var DebayerParams::kRGBLookupSize * \brief Size of a color lookup table From patchwork Thu May 30 21:09:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20157 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F31AFBD87C for ; Thu, 30 May 2024 21:10:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 92A59634BE; Thu, 30 May 2024 23:10:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="iejFhypz"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7EB64634C2 for ; Thu, 30 May 2024 23:10:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1717103427; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=j0kjzg3Hv4mvw/YY3V7nth4nT2OkL8VqbvovO+kaRrI=; b=iejFhypz3y7Gbpzsvxcdoa5Vv5AHxSDdHhaXW1DJx2zZu886Obg0n9oqTyjDZIV70DimFb qPWHHQpqPV8GypwJF6YE0nMVVnMsiLvw89MHD75GjBaHTk4uqNYGzgsv1QKz8E8BRRITqf 6ok0XKMKAmngGGJ/Ut2WWnyqzeKsFv8= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-355-SlxO33mhMXmwAPebSkc7JQ-1; Thu, 30 May 2024 17:10:23 -0400 X-MC-Unique: SlxO33mhMXmwAPebSkc7JQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 212E11C05122; Thu, 30 May 2024 21:10:23 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8E1F440C6EB7; Thu, 30 May 2024 21:10:21 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v5 5/5] libcamera: software_isp: Remove TODO about internal representation Date: Thu, 30 May 2024 23:09:57 +0200 Message-ID: <20240530210957.2211897-6-mzamazal@redhat.com> In-Reply-To: <20240530210957.2211897-1-mzamazal@redhat.com> References: <20240530210957.2211897-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" TODO #4 was recorded at a time where the IPA module computed gain values and the ISP computed the look up tables. The gains were higher-level parameters. Now that the look up tables are computed in the IPA module, the IPA and ISP are more tightly coupled and the TODO item is less relevant. Let's drop the TODO item. We may or may not need to switch to a different representation in future but there is currently no good need for this and the conversion of the values would be just waste of CPU cycles. Signed-off-by: Milan Zamazal --- src/libcamera/software_isp/TODO | 13 ------------- 1 file changed, 13 deletions(-) diff --git a/src/libcamera/software_isp/TODO b/src/libcamera/software_isp/TODO index 4fcee39b..6bdc5905 100644 --- a/src/libcamera/software_isp/TODO +++ b/src/libcamera/software_isp/TODO @@ -72,19 +72,6 @@ stats in hardware, such as the i.MX7), but please keep it on your radar. --- -4. Hide internal representation of gains from callers - -> struct DebayerParams { -> static constexpr unsigned int kGain10 = 256; - -Forcing the caller to deal with the internal representation of gains -isn't nice, especially given that it precludes implementing gains of -different precisions in different backend. Wouldn't it be better to pass -the values as floating point numbers, and convert them to the internal -representation in the implementation of process() before using them ? - ---- - 5. Store ISP parameters in per-frame buffers > /**