From patchwork Thu May 23 11:26:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20092 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 929D8BD87C for ; Thu, 23 May 2024 11:27:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A29FB634A0; Thu, 23 May 2024 13:27:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="USWO481c"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3D56463497 for ; Thu, 23 May 2024 13:27:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1716463644; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WQa/iD8tOxjW06QizxUm8aX0OapgFoAkAtI5JQNluMg=; b=USWO481crWpJ6lmPl8Fb+YZFlYX6Gs4mK5ID2ZdCIkAvBCr5MjxS+RE/a/PFAaw2ErHZFB c3UhdV3jh7A1IIfr83r4qrDNxbWxWMt4EO1Bdj6CjEApEteRG/Z5GM21vBU6v6SkroyRhF icKtLQKb9llGv52bg/Quy23SDOoe8u4= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-76-EclhlbGXNc-EDOqEZlVqxg-1; Thu, 23 May 2024 07:27:21 -0400 X-MC-Unique: EclhlbGXNc-EDOqEZlVqxg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D95EF29AB3F9; Thu, 23 May 2024 11:27:20 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.226.101]) by smtp.corp.redhat.com (Postfix) with ESMTP id 983772026D68; Thu, 23 May 2024 11:27:19 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v3 1/5] libcamera: software_isp: Use a specific integer type for black level Date: Thu, 23 May 2024 13:26:52 +0200 Message-ID: <20240523112656.559150-2-mzamazal@redhat.com> In-Reply-To: <20240523112656.559150-1-mzamazal@redhat.com> References: <20240523112656.559150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The documented range of values corresponds to uint8_t, so let's use that type. Signed-off-by: Milan Zamazal Reviewed-by: Laurent Pinchart Reviewed-by: Andrei Konovalov --- src/ipa/simple/black_level.cpp | 2 +- src/ipa/simple/black_level.h | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/src/ipa/simple/black_level.cpp b/src/ipa/simple/black_level.cpp index 53b4c040..cc490eb5 100644 --- a/src/ipa/simple/black_level.cpp +++ b/src/ipa/simple/black_level.cpp @@ -43,7 +43,7 @@ BlackLevel::BlackLevel() * \return The black level, in the range from 0 (minimum) to 255 (maximum). * If the black level couldn't be determined yet, return 0. */ -unsigned int BlackLevel::get() const +uint8_t BlackLevel::get() const { return blackLevelSet_ ? blackLevel_ : 0; } diff --git a/src/ipa/simple/black_level.h b/src/ipa/simple/black_level.h index 25094217..5e032f9f 100644 --- a/src/ipa/simple/black_level.h +++ b/src/ipa/simple/black_level.h @@ -8,6 +8,7 @@ #pragma once #include +#include #include "libcamera/internal/software_isp/swisp_stats.h" @@ -17,11 +18,11 @@ class BlackLevel { public: BlackLevel(); - unsigned int get() const; + uint8_t get() const; void update(SwIspStats::Histogram &yHistogram); private: - unsigned int blackLevel_; + uint8_t blackLevel_; bool blackLevelSet_; }; From patchwork Thu May 23 11:26:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20091 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2EEB8BD87C for ; Thu, 23 May 2024 11:27:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A37066349E; Thu, 23 May 2024 13:27:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="hX8zvZCI"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9F88061A49 for ; Thu, 23 May 2024 13:27:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1716463644; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WoKBh+fsa/QbsCJq8amuQ7y/TEGKKNbJAwRW+PCWVxc=; b=hX8zvZCIKfGTUcbg4LRbidloIVoJaIPiGro/gmLSDIKRK3Vn6c1NCXA8dTVPTooTVJsCLD lrJg9I+IolIz/yWommPiNlt/qjyZL7xfPxqazefD5B/5NDjj/tUn1FM33ao7X4aoRKZs72 8jBGtErsfxQjuJUf8l2tIuyiG9gd3jg= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-563-olLa5im0MpKxmoPHL5T4Zw-1; Thu, 23 May 2024 07:27:22 -0400 X-MC-Unique: olLa5im0MpKxmoPHL5T4Zw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 845511C031A0; Thu, 23 May 2024 11:27:22 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.226.101]) by smtp.corp.redhat.com (Postfix) with ESMTP id 406752026D33; Thu, 23 May 2024 11:27:21 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v3 2/5] libcamera: software_isp: Honor black level in AWB Date: Thu, 23 May 2024 13:26:53 +0200 Message-ID: <20240523112656.559150-3-mzamazal@redhat.com> In-Reply-To: <20240523112656.559150-1-mzamazal@redhat.com> References: <20240523112656.559150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The white balance computation didn't consider black level. This is wrong because then the computed ratios are off when they are computed from the whole brightness range rather than the sensor range. This patch adjusts white balance computation for the black level. There is no need to change white balance application in debayering as this is already applied after black level correction. Exposure computation already subtracts black level, no changes are needed there. Signed-off-by: Milan Zamazal Reviewed-by: Andrei Konovalov --- src/ipa/simple/soft_simple.cpp | 36 ++++++++++++++++++++++------------ 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index a5bb2bbf..722aac83 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -5,6 +5,8 @@ * Simple Software Image Processing Algorithm module */ +#include +#include #include #include @@ -240,28 +242,36 @@ void IPASoftSimple::stop() void IPASoftSimple::processStats(const ControlList &sensorControls) { + SwIspStats::Histogram histogram = stats_->yHistogram; + if (ignoreUpdates_ > 0) + blackLevel_.update(histogram); + const uint8_t blackLevel = blackLevel_.get(); + params_->blackLevel = blackLevel; + /* * Calculate red and blue gains for AWB. * Clamp max gain at 4.0, this also avoids 0 division. + * Black level must be subtracted to get the correct AWB ratios, + * from the sensor range. */ - if (stats_->sumR_ <= stats_->sumG_ / 4) - params_->gainR = 1024; - else - params_->gainR = 256 * stats_->sumG_ / stats_->sumR_; - - if (stats_->sumB_ <= stats_->sumG_ / 4) - params_->gainB = 1024; - else - params_->gainB = 256 * stats_->sumG_ / stats_->sumB_; + const uint64_t nPixels = std::accumulate( + histogram.begin(), histogram.end(), 0); + auto subtractBlackLevel = [blackLevel, nPixels]( + uint64_t sum, uint64_t div) + -> uint64_t { + return sum - blackLevel * nPixels / div; + }; + const uint64_t sumR = subtractBlackLevel(stats_->sumR_, 4); + const uint64_t sumG = subtractBlackLevel(stats_->sumG_, 2); + const uint64_t sumB = subtractBlackLevel(stats_->sumB_, 4); + + params_->gainR = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; + params_->gainB = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; /* Green gain and gamma values are fixed */ params_->gainG = 256; params_->gamma = 0.5; - if (ignoreUpdates_ > 0) - blackLevel_.update(stats_->yHistogram); - params_->blackLevel = blackLevel_.get(); - setIspParams.emit(); /* \todo Switch to the libipa/algorithm.h API someday. */ From patchwork Thu May 23 11:26:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20093 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9CAD1BD87C for ; Thu, 23 May 2024 11:27:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3BBF4634A8; Thu, 23 May 2024 13:27:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="iie8KklX"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 95F8F63497 for ; Thu, 23 May 2024 13:27:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1716463647; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yvj/yVo8tAK4iWzIE07/7lv0bpsxDFql1hya9+QDAK8=; b=iie8KklXOJbzsTGZ3G+SwC7a3k52it4HZVGaC9Iz6K9QN3J9V8HFiLwgRqmM+Ew2UXfCIz jcyKOP3RaMvI9Gj5G135Mczf9Xxz1pbFHahcNc4eDgsrKBKoyIj99JSfzM8zsX0le99blm z2Xl+izc9YCFAuVSuns5EBpm7BjCarI= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-556-SNwZ9_1WObGZJiqnwFCkQA-1; Thu, 23 May 2024 07:27:24 -0400 X-MC-Unique: SNwZ9_1WObGZJiqnwFCkQA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E7EEE29AB3F9; Thu, 23 May 2024 11:27:23 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.226.101]) by smtp.corp.redhat.com (Postfix) with ESMTP id DC07E2026D68; Thu, 23 May 2024 11:27:22 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v3 3/5] libcamera: software_isp: Move color mappings out of debayering Date: Thu, 23 May 2024 13:26:54 +0200 Message-ID: <20240523112656.559150-4-mzamazal@redhat.com> In-Reply-To: <20240523112656.559150-1-mzamazal@redhat.com> References: <20240523112656.559150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Constructing the color mapping tables is related to stats rather than debayering, where they are applied. Let's move the corresponding code to stats processing. This is a preliminary step towards building this functionality on top of libipa/algorithm.h, which should follow. Signed-off-by: Milan Zamazal Reviewed-by: Andrei Konovalov --- .../internal/software_isp/debayer_params.h | 18 +++---- src/ipa/simple/soft_simple.cpp | 51 +++++++++++++++---- src/libcamera/software_isp/debayer.cpp | 29 +++++------ src/libcamera/software_isp/debayer_cpu.cpp | 41 +++------------ src/libcamera/software_isp/debayer_cpu.h | 9 ++-- src/libcamera/software_isp/software_isp.cpp | 4 +- 6 files changed, 75 insertions(+), 77 deletions(-) diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index ce1b5945..463d24b3 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2023, Red Hat Inc. + * Copyright (C) 2023, 2024 Red Hat Inc. * * Authors: * Hans de Goede @@ -10,20 +10,20 @@ #pragma once +#include +#include + namespace libcamera { struct DebayerParams { static constexpr unsigned int kGain10 = 256; + static constexpr unsigned int kRGBLookupSize = 256; - unsigned int gainR; - unsigned int gainG; - unsigned int gainB; + using ColorLookupTable = std::array; - float gamma; - /** - * \brief Level of the black point, 0..255, 0 is no correction. - */ - unsigned int blackLevel; + ColorLookupTable red; + ColorLookupTable green; + ColorLookupTable blue; }; } /* namespace libcamera */ diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 722aac83..1b0655fe 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -5,6 +5,7 @@ * Simple Software Image Processing Algorithm module */ +#include #include #include #include @@ -84,6 +85,10 @@ private: ControlInfoMap sensorInfoMap_; BlackLevel blackLevel_; + static constexpr unsigned int kGammaLookupSize = 1024; + std::array gammaTable_; + int lastBlackLevel_ = -1; + int32_t exposureMin_, exposureMax_; int32_t exposure_; double againMin_, againMax_, againMinStep_; @@ -246,7 +251,6 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) if (ignoreUpdates_ > 0) blackLevel_.update(histogram); const uint8_t blackLevel = blackLevel_.get(); - params_->blackLevel = blackLevel; /* * Calculate red and blue gains for AWB. @@ -265,12 +269,41 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) const uint64_t sumG = subtractBlackLevel(stats_->sumG_, 2); const uint64_t sumB = subtractBlackLevel(stats_->sumB_, 4); - params_->gainR = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; - params_->gainB = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; - + /* Gain: 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. */ + const unsigned int gainR = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; + const unsigned int gainB = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; /* Green gain and gamma values are fixed */ - params_->gainG = 256; - params_->gamma = 0.5; + constexpr unsigned int gainG = 256; + /* gamma == 1.0 means no correction */ + constexpr float gamma = 0.5; + + /* Update the gamma table if needed */ + if (blackLevel != lastBlackLevel_) { + const unsigned int blackIndex = blackLevel * kGammaLookupSize / 256; + std::fill(gammaTable_.begin(), gammaTable_.begin() + blackIndex, 0); + const float divisor = kGammaLookupSize - blackIndex - 1.0; + for (unsigned int i = blackIndex; i < kGammaLookupSize; i++) + gammaTable_[i] = UINT8_MAX * powf((i - blackIndex) / divisor, gamma); + + lastBlackLevel_ = blackLevel; + } + + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { + constexpr unsigned int div = + DebayerParams::kRGBLookupSize * DebayerParams::kGain10 / + kGammaLookupSize; + unsigned int idx; + + /* Apply gamma after gain! */ + idx = std::min({ i * gainR / div, (kGammaLookupSize - 1) }); + params_->red[i] = gammaTable_[idx]; + + idx = std::min({ i * gainG / div, (kGammaLookupSize - 1) }); + params_->green[i] = gammaTable_[idx]; + + idx = std::min({ i * gainB / div, (kGammaLookupSize - 1) }); + params_->blue[i] = gammaTable_[idx]; + } setIspParams.emit(); @@ -291,7 +324,7 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) * https://www.araa.asn.au/acra/acra2007/papers/paper84final.pdf */ const unsigned int blackLevelHistIdx = - params_->blackLevel / (256 / SwIspStats::kYHistogramSize); + blackLevel / (256 / SwIspStats::kYHistogramSize); const unsigned int histogramSize = SwIspStats::kYHistogramSize - blackLevelHistIdx; const unsigned int yHistValsPerBin = histogramSize / kExposureBinsCount; @@ -339,8 +372,8 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) LOG(IPASoft, Debug) << "exposureMSV " << exposureMSV << " exp " << exposure_ << " again " << again_ - << " gain R/B " << params_->gainR << "/" << params_->gainB - << " black level " << params_->blackLevel; + << " gain R/B " << gainR << "/" << gainB + << " black level " << static_cast(blackLevel); } void IPASoftSimple::updateExposure(double exposureMSV) diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index efe75ea8..3f3969f7 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* * Copyright (C) 2023, Linaro Ltd - * Copyright (C) 2023, Red Hat Inc. + * Copyright (C) 2023, 2024 Red Hat Inc. * * Authors: * Hans de Goede @@ -24,29 +24,28 @@ namespace libcamera { */ /** - * \var DebayerParams::gainR - * \brief Red gain - * - * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + * \var DebayerParams::kRGBLookupSize + * \brief Size of a color lookup table */ /** - * \var DebayerParams::gainG - * \brief Green gain - * - * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + * \typedef DebayerParams::ColorLookupTable + * \brief Type of the lookup tables for red, green, blue values */ /** - * \var DebayerParams::gainB - * \brief Blue gain - * - * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + * \var DebayerParams::red + * \brief Lookup table for red color, mapping input values to output values + */ + +/** + * \var DebayerParams::green + * \brief Lookup table for green color, mapping input values to output values */ /** - * \var DebayerParams::gamma - * \brief Gamma correction, 1.0 is no correction + * \var DebayerParams::blue + * \brief Lookup table for blue color, mapping input values to output values */ /** diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 8254bbe9..f3099a62 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -11,7 +11,6 @@ #include "debayer_cpu.h" -#include #include #include @@ -47,9 +46,9 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats) */ enableInputMemcpy_ = true; - /* Initialize gamma to 1.0 curve */ - for (unsigned int i = 0; i < kGammaLookupSize; i++) - gamma_[i] = i / (kGammaLookupSize / kRGBLookupSize); + /* Initialize color lookup tables */ + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) + red_[i] = green_[i] = blue_[i] = i; for (unsigned int i = 0; i < kMaxLineBuffers; i++) lineBuffers_[i] = nullptr; @@ -698,37 +697,9 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams clock_gettime(CLOCK_MONOTONIC_RAW, &frameStartTime); } - /* Apply DebayerParams */ - if (params.gamma != gammaCorrection_ || params.blackLevel != blackLevel_) { - const unsigned int blackIndex = - params.blackLevel * kGammaLookupSize / 256; - std::fill(gamma_.begin(), gamma_.begin() + blackIndex, 0); - const float divisor = kGammaLookupSize - blackIndex - 1.0; - for (unsigned int i = blackIndex; i < kGammaLookupSize; i++) - gamma_[i] = UINT8_MAX * powf((i - blackIndex) / divisor, params.gamma); - - gammaCorrection_ = params.gamma; - blackLevel_ = params.blackLevel; - } - - if (swapRedBlueGains_) - std::swap(params.gainR, params.gainB); - - for (unsigned int i = 0; i < kRGBLookupSize; i++) { - constexpr unsigned int div = - kRGBLookupSize * DebayerParams::kGain10 / kGammaLookupSize; - unsigned int idx; - - /* Apply gamma after gain! */ - idx = std::min({ i * params.gainR / div, (kGammaLookupSize - 1) }); - red_[i] = gamma_[idx]; - - idx = std::min({ i * params.gainG / div, (kGammaLookupSize - 1) }); - green_[i] = gamma_[idx]; - - idx = std::min({ i * params.gainB / div, (kGammaLookupSize - 1) }); - blue_[i] = gamma_[idx]; - } + green_ = params.green; + red_ = swapRedBlueGains_ ? params.blue : params.red; + blue_ = swapRedBlueGains_ ? params.red : params.blue; /* Copy metadata from the input buffer */ FrameMetadata &metadata = output->_d()->metadata(); diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index de216fe3..768a4643 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -122,15 +122,12 @@ private: void process2(const uint8_t *src, uint8_t *dst); void process4(const uint8_t *src, uint8_t *dst); - static constexpr unsigned int kGammaLookupSize = 1024; - static constexpr unsigned int kRGBLookupSize = 256; /* Max. supported Bayer pattern height is 4, debayering this requires 5 lines */ static constexpr unsigned int kMaxLineBuffers = 5; - std::array gamma_; - std::array red_; - std::array green_; - std::array blue_; + DebayerParams::ColorLookupTable red_; + DebayerParams::ColorLookupTable green_; + DebayerParams::ColorLookupTable blue_; debayerFn debayer0_; debayerFn debayer1_; debayerFn debayer2_; diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index c9b6be56..1003d835 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -63,9 +63,7 @@ LOG_DEFINE_CATEGORY(SoftwareIsp) * handler */ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor) - : debayerParams_{ DebayerParams::kGain10, DebayerParams::kGain10, - DebayerParams::kGain10, 0.5f, 0 }, - dmaHeap_(DmaHeap::DmaHeapFlag::Cma | DmaHeap::DmaHeapFlag::System) + : dmaHeap_(DmaHeap::DmaHeapFlag::Cma | DmaHeap::DmaHeapFlag::System) { if (!dmaHeap_.isValid()) { LOG(SoftwareIsp, Error) << "Failed to create DmaHeap object"; From patchwork Thu May 23 11:26:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20094 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E6FD3BD87C for ; Thu, 23 May 2024 11:27:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CDD1A634AA; Thu, 23 May 2024 13:27:35 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="TWSZpAl3"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 97720634A2 for ; Thu, 23 May 2024 13:27:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1716463649; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GFE8obJIxpEiOdx2OVXRa2I77214va2tRKqE7Ik7ptw=; b=TWSZpAl3+Gq6RRyZXQ3o//1V2Y9mSItHuWqcb9eEEpUmeTNJ1SeAEcqO8Qum/iDkoBNoBP FaoYtIT+WbC3OmpwxxBg6QSDueLbVn4C6HwNU+iYjAiZlQ63IZDu6DqobVTH4G14VU3/RK 74noWZPXdOREoKMyGCamvUIJzzmi2cg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-57-yCq1eVefPOmEpqaBlVUw3Q-1; Thu, 23 May 2024 07:27:25 -0400 X-MC-Unique: yCq1eVefPOmEpqaBlVUw3Q-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8DE0585A58C; Thu, 23 May 2024 11:27:25 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.226.101]) by smtp.corp.redhat.com (Postfix) with ESMTP id 516CB2026D68; Thu, 23 May 2024 11:27:24 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v3 4/5] libcamera: software_isp: Remove DebayerParams::kGain10 Date: Thu, 23 May 2024 13:26:55 +0200 Message-ID: <20240523112656.559150-5-mzamazal@redhat.com> In-Reply-To: <20240523112656.559150-1-mzamazal@redhat.com> References: <20240523112656.559150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The constant is used in a single place internally and doesn't belong to DebayerParams anymore. Let's use 256 directly. Signed-off-by: Milan Zamazal Reviewed-by: Andrei Konovalov --- include/libcamera/internal/software_isp/debayer_params.h | 1 - src/ipa/simple/soft_simple.cpp | 3 +-- src/libcamera/software_isp/debayer.cpp | 5 ----- 3 files changed, 1 insertion(+), 8 deletions(-) diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index 463d24b3..7d8fdd48 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -16,7 +16,6 @@ namespace libcamera { struct DebayerParams { - static constexpr unsigned int kGain10 = 256; static constexpr unsigned int kRGBLookupSize = 256; using ColorLookupTable = std::array; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 1b0655fe..cd89e673 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -290,8 +290,7 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { constexpr unsigned int div = - DebayerParams::kRGBLookupSize * DebayerParams::kGain10 / - kGammaLookupSize; + DebayerParams::kRGBLookupSize * 256 / kGammaLookupSize; unsigned int idx; /* Apply gamma after gain! */ diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index 3f3969f7..f4a299d5 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -18,11 +18,6 @@ namespace libcamera { * \brief Struct to hold the debayer parameters. */ -/** - * \var DebayerParams::kGain10 - * \brief const value for 1.0 gain - */ - /** * \var DebayerParams::kRGBLookupSize * \brief Size of a color lookup table From patchwork Thu May 23 11:26:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20095 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E464EBD87C for ; Thu, 23 May 2024 11:27:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 85CC8634A5; Thu, 23 May 2024 13:27:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="IjewvqLn"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 297BF634A6 for ; Thu, 23 May 2024 13:27:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1716463651; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=skUQ1Y2KuLbcKlSWlti06cq5C2+zC5GBBds10V4jwd0=; b=IjewvqLnOJXiWmOKmGtyj+RM2iaQj9oM40QnaOcF2ilt7o3UoTR+VqrA+WoX2d9wVcgbIp 2vobuilc8lgOLa492Nh8se/ksMLyVVu/cdd1mKLmu9m1KzrTxnMqxKUekafywoQ8hD++5m tu+/4Q8EdwKKa0pbPsAlLESM8G0l47U= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-453-MN-a_gWBP_iz2VCa9EKkMw-1; Thu, 23 May 2024 07:27:27 -0400 X-MC-Unique: MN-a_gWBP_iz2VCa9EKkMw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 37B393C025A1; Thu, 23 May 2024 11:27:27 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.226.101]) by smtp.corp.redhat.com (Postfix) with ESMTP id ED9C22026D68; Thu, 23 May 2024 11:27:25 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v3 5/5] libcamera: software_isp: Pass color lookup tables as floats Date: Thu, 23 May 2024 13:26:56 +0200 Message-ID: <20240523112656.559150-6-mzamazal@redhat.com> In-Reply-To: <20240523112656.559150-1-mzamazal@redhat.com> References: <20240523112656.559150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The color lookup tables are passed from stats processing to debayering for direct use as 8-bit color output values. Let's pass the values as 0.0..1.0 floats to make the gains color bit width independent. Completes software ISP TODO #4. Signed-off-by: Milan Zamazal Reviewed-by: Andrei Konovalov --- .../internal/software_isp/debayer_params.h | 2 +- src/ipa/simple/soft_simple.cpp | 4 ++-- src/libcamera/software_isp/TODO | 13 ------------- src/libcamera/software_isp/debayer.cpp | 6 +++--- src/libcamera/software_isp/debayer_cpu.cpp | 18 +++++++++++++++--- src/libcamera/software_isp/debayer_cpu.h | 10 +++++++--- 6 files changed, 28 insertions(+), 25 deletions(-) diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index 7d8fdd48..af959e1a 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -18,7 +18,7 @@ namespace libcamera { struct DebayerParams { static constexpr unsigned int kRGBLookupSize = 256; - using ColorLookupTable = std::array; + using ColorLookupTable = std::array; ColorLookupTable red; ColorLookupTable green; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index cd89e673..11678393 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -86,7 +86,7 @@ private: BlackLevel blackLevel_; static constexpr unsigned int kGammaLookupSize = 1024; - std::array gammaTable_; + std::array gammaTable_; int lastBlackLevel_ = -1; int32_t exposureMin_, exposureMax_; @@ -283,7 +283,7 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) std::fill(gammaTable_.begin(), gammaTable_.begin() + blackIndex, 0); const float divisor = kGammaLookupSize - blackIndex - 1.0; for (unsigned int i = blackIndex; i < kGammaLookupSize; i++) - gammaTable_[i] = UINT8_MAX * powf((i - blackIndex) / divisor, gamma); + gammaTable_[i] = powf((i - blackIndex) / divisor, gamma); lastBlackLevel_ = blackLevel; } diff --git a/src/libcamera/software_isp/TODO b/src/libcamera/software_isp/TODO index 4fcee39b..6bdc5905 100644 --- a/src/libcamera/software_isp/TODO +++ b/src/libcamera/software_isp/TODO @@ -72,19 +72,6 @@ stats in hardware, such as the i.MX7), but please keep it on your radar. --- -4. Hide internal representation of gains from callers - -> struct DebayerParams { -> static constexpr unsigned int kGain10 = 256; - -Forcing the caller to deal with the internal representation of gains -isn't nice, especially given that it precludes implementing gains of -different precisions in different backend. Wouldn't it be better to pass -the values as floating point numbers, and convert them to the internal -representation in the implementation of process() before using them ? - ---- - 5. Store ISP parameters in per-frame buffers > /** diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index f4a299d5..1c93ac44 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -30,17 +30,17 @@ namespace libcamera { /** * \var DebayerParams::red - * \brief Lookup table for red color, mapping input values to output values + * \brief Lookup table for red color, mapping input values to 0.0..1.0 */ /** * \var DebayerParams::green - * \brief Lookup table for green color, mapping input values to output values + * \brief Lookup table for green color, mapping input values to 0.0..1.0 */ /** * \var DebayerParams::blue - * \brief Lookup table for blue color, mapping input values to output values + * \brief Lookup table for blue color, mapping input values to 0.0..1.0 */ /** diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index f3099a62..e71b5057 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -11,6 +11,8 @@ #include "debayer_cpu.h" +#include +#include #include #include @@ -688,6 +690,14 @@ static inline int64_t timeDiff(timespec &after, timespec &before) (int64_t)after.tv_nsec - (int64_t)before.tv_nsec; } +void DebayerCpu::updateColorLookupTable( + const DebayerParams::ColorLookupTable &src, + ColorLookupTable &dst) +{ + for (std::size_t i = 0; i < src.size(); i++) + dst[i] = src[i] * UINT8_MAX; +} + void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams params) { timespec frameStartTime; @@ -697,9 +707,11 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams clock_gettime(CLOCK_MONOTONIC_RAW, &frameStartTime); } - green_ = params.green; - red_ = swapRedBlueGains_ ? params.blue : params.red; - blue_ = swapRedBlueGains_ ? params.red : params.blue; + updateColorLookupTable(params.green, green_); + updateColorLookupTable(swapRedBlueGains_ ? params.blue : params.red, + red_); + updateColorLookupTable(swapRedBlueGains_ ? params.red : params.blue, + blue_); /* Copy metadata from the input buffer */ FrameMetadata &metadata = output->_d()->metadata(); diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 768a4643..ed1003a9 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -18,6 +18,7 @@ #include #include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/software_isp/debayer_params.h" #include "debayer.h" #include "swstats_cpu.h" @@ -125,9 +126,12 @@ private: /* Max. supported Bayer pattern height is 4, debayering this requires 5 lines */ static constexpr unsigned int kMaxLineBuffers = 5; - DebayerParams::ColorLookupTable red_; - DebayerParams::ColorLookupTable green_; - DebayerParams::ColorLookupTable blue_; + using ColorLookupTable = std::array; + void updateColorLookupTable(const DebayerParams::ColorLookupTable &src, + ColorLookupTable &dst); + ColorLookupTable red_; + ColorLookupTable green_; + ColorLookupTable blue_; debayerFn debayer0_; debayerFn debayer1_; debayerFn debayer2_;