From patchwork Fri May 31 12:38:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20172 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9A6AEC32C8 for ; Fri, 31 May 2024 12:39:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C42B5634BA; Fri, 31 May 2024 14:39:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="UE0gxRPW"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D4AD8634B2 for ; Fri, 31 May 2024 14:39:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1717159140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7mVUQu4RbkXAbmj4aVpbY+oRVzHlovFGPVLQMG01k5I=; b=UE0gxRPWVt05A3ZkA9k1BlrfXllKNFqbwisl5PKf1p8pyDxzQHxgYZLlMyzk1ucYu6swWe g7WRx3T9JqBrjXFc3fdHa86K2tCOGq8I+MlNwBPkgAPgkad4z1rGM0dor9H8VrFknZ5A71 1G8tpeUpnZMyaCEv5t39TSsofqbgeDg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-111-xxKA6qFiPqqUIGzzRtbZvg-1; Fri, 31 May 2024 08:38:57 -0400 X-MC-Unique: xxKA6qFiPqqUIGzzRtbZvg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2FA9A85A58C; Fri, 31 May 2024 12:38:57 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id B50A4C15BB1; Fri, 31 May 2024 12:38:55 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v6 1/5] libcamera: software_isp: Use a specific integer type for black level Date: Fri, 31 May 2024 14:38:36 +0200 Message-ID: <20240531123840.713364-2-mzamazal@redhat.com> In-Reply-To: <20240531123840.713364-1-mzamazal@redhat.com> References: <20240531123840.713364-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The documented range of values corresponds to uint8_t, so let's use that type. Signed-off-by: Milan Zamazal Reviewed-by: Laurent Pinchart Reviewed-by: Andrei Konovalov --- src/ipa/simple/black_level.cpp | 2 +- src/ipa/simple/black_level.h | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/src/ipa/simple/black_level.cpp b/src/ipa/simple/black_level.cpp index 53b4c040..cc490eb5 100644 --- a/src/ipa/simple/black_level.cpp +++ b/src/ipa/simple/black_level.cpp @@ -43,7 +43,7 @@ BlackLevel::BlackLevel() * \return The black level, in the range from 0 (minimum) to 255 (maximum). * If the black level couldn't be determined yet, return 0. */ -unsigned int BlackLevel::get() const +uint8_t BlackLevel::get() const { return blackLevelSet_ ? blackLevel_ : 0; } diff --git a/src/ipa/simple/black_level.h b/src/ipa/simple/black_level.h index 25094217..5e032f9f 100644 --- a/src/ipa/simple/black_level.h +++ b/src/ipa/simple/black_level.h @@ -8,6 +8,7 @@ #pragma once #include +#include #include "libcamera/internal/software_isp/swisp_stats.h" @@ -17,11 +18,11 @@ class BlackLevel { public: BlackLevel(); - unsigned int get() const; + uint8_t get() const; void update(SwIspStats::Histogram &yHistogram); private: - unsigned int blackLevel_; + uint8_t blackLevel_; bool blackLevelSet_; }; From patchwork Fri May 31 12:38:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20171 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8C8C9BDE6B for ; Fri, 31 May 2024 12:39:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 34DBC634BC; Fri, 31 May 2024 14:39:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="bQcxwxWO"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B4A59634AF for ; Fri, 31 May 2024 14:39:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1717159140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tmp8x2JElBY+1aZMMPEis5CdFwhlbaspITWXBXWrkXk=; b=bQcxwxWOpibSk9ho2xvyAniPhRXMcV6XottGva6D+ms4FYfAKHipPLcndd4lN19O+h7wKb P/Gqo43UepeMrdn9CrLsgoRvBDmBhqMH8ctSZaxqt6IDRFnPplmNu02famDeIen4xCq24u j7YgEifk6fEZyQ47qH9GaUpnnlCCC/Q= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-206-URmQWt2HPBeAVp_lux9RWA-1; Fri, 31 May 2024 08:38:59 -0400 X-MC-Unique: URmQWt2HPBeAVp_lux9RWA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E970C185A780; Fri, 31 May 2024 12:38:58 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8F5BAC15BB1; Fri, 31 May 2024 12:38:57 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v6 2/5] libcamera: software_isp: Honor black level in AWB Date: Fri, 31 May 2024 14:38:37 +0200 Message-ID: <20240531123840.713364-3-mzamazal@redhat.com> In-Reply-To: <20240531123840.713364-1-mzamazal@redhat.com> References: <20240531123840.713364-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The white balance computation didn't consider black level. This is wrong because then the computed ratios are off when they are computed from the whole brightness range rather than the sensor range. This patch adjusts white balance computation for the black level. There is no need to change white balance application in debayering as this is already applied after black level correction. Exposure computation already subtracts black level, no changes are needed there. Signed-off-by: Milan Zamazal Reviewed-by: Andrei Konovalov Reviewed-by: Laurent Pinchart --- src/ipa/simple/soft_simple.cpp | 35 +++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-) diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index a5bb2bbf..f383f994 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -5,6 +5,8 @@ * Simple Software Image Processing Algorithm module */ +#include +#include #include #include @@ -240,28 +242,35 @@ void IPASoftSimple::stop() void IPASoftSimple::processStats(const ControlList &sensorControls) { + SwIspStats::Histogram histogram = stats_->yHistogram; + if (ignoreUpdates_ > 0) + blackLevel_.update(histogram); + const uint8_t blackLevel = blackLevel_.get(); + params_->blackLevel = blackLevel; + + /* + * Black level must be subtracted to get the correct AWB ratios, + * they would be off if they were computed from the whole brightness + * range rather than from the sensor range. + */ + const uint64_t nPixels = std::accumulate( + histogram.begin(), histogram.end(), 0); + const uint64_t offset = blackLevel * nPixels; + const uint64_t sumR = stats_->sumR_ - offset / 4; + const uint64_t sumG = stats_->sumG_ - offset / 2; + const uint64_t sumB = stats_->sumB_ - offset / 4; + /* * Calculate red and blue gains for AWB. * Clamp max gain at 4.0, this also avoids 0 division. */ - if (stats_->sumR_ <= stats_->sumG_ / 4) - params_->gainR = 1024; - else - params_->gainR = 256 * stats_->sumG_ / stats_->sumR_; - - if (stats_->sumB_ <= stats_->sumG_ / 4) - params_->gainB = 1024; - else - params_->gainB = 256 * stats_->sumG_ / stats_->sumB_; + params_->gainR = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; + params_->gainB = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; /* Green gain and gamma values are fixed */ params_->gainG = 256; params_->gamma = 0.5; - if (ignoreUpdates_ > 0) - blackLevel_.update(stats_->yHistogram); - params_->blackLevel = blackLevel_.get(); - setIspParams.emit(); /* \todo Switch to the libipa/algorithm.h API someday. */ From patchwork Fri May 31 12:38:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20173 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A9F81BDE6B for ; Fri, 31 May 2024 12:39:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5BD46634C2; Fri, 31 May 2024 14:39:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="KRniua2W"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2FCC5634B2 for ; Fri, 31 May 2024 14:39:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1717159143; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5b4brVMuOT33ENBQ8qwMM/3NY2uaUNe+iJFfoclFH1s=; b=KRniua2WhIYpMSCyCcQWc15pih3/0FhzWxGxDPo1RRbzpxOZtuNQ/pOtNyNNAvksCkWroo mdY4wJgmJvffJ5QAGZH1/iBqeKG75mTAXuqnhgqz6s8lZK4+je5eWPbKuf1DfSOETOfFO4 GEvjkw1lExD0w90t9tAfA9aNV4AKVJE= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-691-KYbTRGihOV2cz05mdBgsTg-1; Fri, 31 May 2024 08:39:01 -0400 X-MC-Unique: KYbTRGihOV2cz05mdBgsTg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F1A2C1C0512B; Fri, 31 May 2024 12:39:00 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6870AC15BB1; Fri, 31 May 2024 12:38:59 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v6 3/5] libcamera: software_isp: Move color mappings out of debayering Date: Fri, 31 May 2024 14:38:38 +0200 Message-ID: <20240531123840.713364-4-mzamazal@redhat.com> In-Reply-To: <20240531123840.713364-1-mzamazal@redhat.com> References: <20240531123840.713364-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Constructing the color mapping tables is related to stats rather than debayering, where they are applied. Let's move the corresponding code to stats processing. The same applies to the auxiliary gamma table. As the gamma value is currently fixed and used in a single place, with the temporary exception mentioned below, there is no need to share it anywhere anymore. It's necessary to initialize SoftwareIsp::debayerParams_ to default values. These initial values are used for the first two frames, before they are changed based on determined stats. To avoid sharing the gamma value constant in artificial ways, we use 0.5 directly in the initialization. This all is not a particularly elegant thing to do, such a code belongs conceptually to the similar code in stats processing, but doing better is left for larger refactoring. This is a preliminary step towards building this functionality on top of libipa/algorithm.h, which should follow. Signed-off-by: Milan Zamazal Reviewed-by: Andrei Konovalov Reviewed-by: Laurent Pinchart --- .../internal/software_isp/debayer_params.h | 18 +++---- src/ipa/simple/soft_simple.cpp | 51 +++++++++++++++---- src/libcamera/software_isp/debayer.cpp | 29 +++++------ src/libcamera/software_isp/debayer_cpu.cpp | 43 +++------------- src/libcamera/software_isp/debayer_cpu.h | 11 ++-- src/libcamera/software_isp/software_isp.cpp | 23 +++++++-- 6 files changed, 95 insertions(+), 80 deletions(-) diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index ce1b5945..463d24b3 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2023, Red Hat Inc. + * Copyright (C) 2023, 2024 Red Hat Inc. * * Authors: * Hans de Goede @@ -10,20 +10,20 @@ #pragma once +#include +#include + namespace libcamera { struct DebayerParams { static constexpr unsigned int kGain10 = 256; + static constexpr unsigned int kRGBLookupSize = 256; - unsigned int gainR; - unsigned int gainG; - unsigned int gainB; + using ColorLookupTable = std::array; - float gamma; - /** - * \brief Level of the black point, 0..255, 0 is no correction. - */ - unsigned int blackLevel; + ColorLookupTable red; + ColorLookupTable green; + ColorLookupTable blue; }; } /* namespace libcamera */ diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index f383f994..3388686f 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -5,6 +5,7 @@ * Simple Software Image Processing Algorithm module */ +#include #include #include #include @@ -84,6 +85,10 @@ private: ControlInfoMap sensorInfoMap_; BlackLevel blackLevel_; + static constexpr unsigned int kGammaLookupSize = 1024; + std::array gammaTable_; + int lastBlackLevel_ = -1; + int32_t exposureMin_, exposureMax_; int32_t exposure_; double againMin_, againMax_, againMinStep_; @@ -246,7 +251,6 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) if (ignoreUpdates_ > 0) blackLevel_.update(histogram); const uint8_t blackLevel = blackLevel_.get(); - params_->blackLevel = blackLevel; /* * Black level must be subtracted to get the correct AWB ratios, @@ -263,13 +267,42 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) /* * Calculate red and blue gains for AWB. * Clamp max gain at 4.0, this also avoids 0 division. + * Gain: 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. */ - params_->gainR = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; - params_->gainB = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; - + const unsigned int gainR = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; + const unsigned int gainB = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; /* Green gain and gamma values are fixed */ - params_->gainG = 256; - params_->gamma = 0.5; + constexpr unsigned int gainG = 256; + + /* Update the gamma table if needed */ + if (blackLevel != lastBlackLevel_) { + constexpr float gamma = 0.5; + const unsigned int blackIndex = blackLevel * kGammaLookupSize / 256; + std::fill(gammaTable_.begin(), gammaTable_.begin() + blackIndex, 0); + const float divisor = kGammaLookupSize - blackIndex - 1.0; + for (unsigned int i = blackIndex; i < kGammaLookupSize; i++) + gammaTable_[i] = UINT8_MAX * + std::pow((i - blackIndex) / divisor, gamma); + + lastBlackLevel_ = blackLevel; + } + + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { + constexpr unsigned int div = + DebayerParams::kRGBLookupSize * DebayerParams::kGain10 / + kGammaLookupSize; + unsigned int idx; + + /* Apply gamma after gain! */ + idx = std::min({ i * gainR / div, (kGammaLookupSize - 1) }); + params_->red[i] = gammaTable_[idx]; + + idx = std::min({ i * gainG / div, (kGammaLookupSize - 1) }); + params_->green[i] = gammaTable_[idx]; + + idx = std::min({ i * gainB / div, (kGammaLookupSize - 1) }); + params_->blue[i] = gammaTable_[idx]; + } setIspParams.emit(); @@ -290,7 +323,7 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) * https://www.araa.asn.au/acra/acra2007/papers/paper84final.pdf */ const unsigned int blackLevelHistIdx = - params_->blackLevel / (256 / SwIspStats::kYHistogramSize); + blackLevel / (256 / SwIspStats::kYHistogramSize); const unsigned int histogramSize = SwIspStats::kYHistogramSize - blackLevelHistIdx; const unsigned int yHistValsPerBin = histogramSize / kExposureBinsCount; @@ -338,8 +371,8 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) LOG(IPASoft, Debug) << "exposureMSV " << exposureMSV << " exp " << exposure_ << " again " << again_ - << " gain R/B " << params_->gainR << "/" << params_->gainB - << " black level " << params_->blackLevel; + << " gain R/B " << gainR << "/" << gainB + << " black level " << static_cast(blackLevel); } void IPASoftSimple::updateExposure(double exposureMSV) diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index efe75ea8..3f3969f7 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* * Copyright (C) 2023, Linaro Ltd - * Copyright (C) 2023, Red Hat Inc. + * Copyright (C) 2023, 2024 Red Hat Inc. * * Authors: * Hans de Goede @@ -24,29 +24,28 @@ namespace libcamera { */ /** - * \var DebayerParams::gainR - * \brief Red gain - * - * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + * \var DebayerParams::kRGBLookupSize + * \brief Size of a color lookup table */ /** - * \var DebayerParams::gainG - * \brief Green gain - * - * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + * \typedef DebayerParams::ColorLookupTable + * \brief Type of the lookup tables for red, green, blue values */ /** - * \var DebayerParams::gainB - * \brief Blue gain - * - * 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + * \var DebayerParams::red + * \brief Lookup table for red color, mapping input values to output values + */ + +/** + * \var DebayerParams::green + * \brief Lookup table for green color, mapping input values to output values */ /** - * \var DebayerParams::gamma - * \brief Gamma correction, 1.0 is no correction + * \var DebayerParams::blue + * \brief Lookup table for blue color, mapping input values to output values */ /** diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 8254bbe9..c038eed4 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -11,7 +11,6 @@ #include "debayer_cpu.h" -#include #include #include @@ -35,7 +34,7 @@ namespace libcamera { * \param[in] stats Pointer to the stats object to use */ DebayerCpu::DebayerCpu(std::unique_ptr stats) - : stats_(std::move(stats)), gammaCorrection_(1.0), blackLevel_(0) + : stats_(std::move(stats)) { /* * Reading from uncached buffers may be very slow. @@ -47,9 +46,9 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats) */ enableInputMemcpy_ = true; - /* Initialize gamma to 1.0 curve */ - for (unsigned int i = 0; i < kGammaLookupSize; i++) - gamma_[i] = i / (kGammaLookupSize / kRGBLookupSize); + /* Initialize color lookup tables */ + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) + red_[i] = green_[i] = blue_[i] = i; for (unsigned int i = 0; i < kMaxLineBuffers; i++) lineBuffers_[i] = nullptr; @@ -698,37 +697,9 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams clock_gettime(CLOCK_MONOTONIC_RAW, &frameStartTime); } - /* Apply DebayerParams */ - if (params.gamma != gammaCorrection_ || params.blackLevel != blackLevel_) { - const unsigned int blackIndex = - params.blackLevel * kGammaLookupSize / 256; - std::fill(gamma_.begin(), gamma_.begin() + blackIndex, 0); - const float divisor = kGammaLookupSize - blackIndex - 1.0; - for (unsigned int i = blackIndex; i < kGammaLookupSize; i++) - gamma_[i] = UINT8_MAX * powf((i - blackIndex) / divisor, params.gamma); - - gammaCorrection_ = params.gamma; - blackLevel_ = params.blackLevel; - } - - if (swapRedBlueGains_) - std::swap(params.gainR, params.gainB); - - for (unsigned int i = 0; i < kRGBLookupSize; i++) { - constexpr unsigned int div = - kRGBLookupSize * DebayerParams::kGain10 / kGammaLookupSize; - unsigned int idx; - - /* Apply gamma after gain! */ - idx = std::min({ i * params.gainR / div, (kGammaLookupSize - 1) }); - red_[i] = gamma_[idx]; - - idx = std::min({ i * params.gainG / div, (kGammaLookupSize - 1) }); - green_[i] = gamma_[idx]; - - idx = std::min({ i * params.gainB / div, (kGammaLookupSize - 1) }); - blue_[i] = gamma_[idx]; - } + green_ = params.green; + red_ = swapRedBlueGains_ ? params.blue : params.red; + blue_ = swapRedBlueGains_ ? params.red : params.blue; /* Copy metadata from the input buffer */ FrameMetadata &metadata = output->_d()->metadata(); diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index de216fe3..be7dcdca 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -122,15 +122,12 @@ private: void process2(const uint8_t *src, uint8_t *dst); void process4(const uint8_t *src, uint8_t *dst); - static constexpr unsigned int kGammaLookupSize = 1024; - static constexpr unsigned int kRGBLookupSize = 256; /* Max. supported Bayer pattern height is 4, debayering this requires 5 lines */ static constexpr unsigned int kMaxLineBuffers = 5; - std::array gamma_; - std::array red_; - std::array green_; - std::array blue_; + DebayerParams::ColorLookupTable red_; + DebayerParams::ColorLookupTable green_; + DebayerParams::ColorLookupTable blue_; debayerFn debayer0_; debayerFn debayer1_; debayerFn debayer2_; @@ -146,8 +143,6 @@ private: unsigned int xShift_; /* Offset of 0/1 applied to window_.x */ bool enableInputMemcpy_; bool swapRedBlueGains_; - float gammaCorrection_; - unsigned int blackLevel_; unsigned int measuredFrames_; int64_t frameProcessTime_; /* Skip 30 frames for things to stabilize then measure 30 frames */ diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index c9b6be56..ccd703bb 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -7,6 +7,8 @@ #include "libcamera/internal/software_isp/software_isp.h" +#include +#include #include #include #include @@ -18,6 +20,7 @@ #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/mapped_framebuffer.h" +#include "libcamera/internal/software_isp/debayer_params.h" #include "debayer_cpu.h" @@ -63,10 +66,24 @@ LOG_DEFINE_CATEGORY(SoftwareIsp) * handler */ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor) - : debayerParams_{ DebayerParams::kGain10, DebayerParams::kGain10, - DebayerParams::kGain10, 0.5f, 0 }, - dmaHeap_(DmaHeap::DmaHeapFlag::Cma | DmaHeap::DmaHeapFlag::System) + : dmaHeap_(DmaHeap::DmaHeapFlag::Cma | DmaHeap::DmaHeapFlag::System) { + /* + * debayerParams_ must be initialized because the initial value is used for + * the first two frames, i.e. until stats processing starts providing its + * own parameters. + * + * \todo This should be handled in the same place as the related operations. + */ + std::array gammaTable; + for (unsigned int i = 0; i < 256; i++) + gammaTable[i] = UINT8_MAX * std::pow(i / 256.0, 0.5); + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { + debayerParams_.red[i] = gammaTable[i]; + debayerParams_.green[i] = gammaTable[i]; + debayerParams_.blue[i] = gammaTable[i]; + } + if (!dmaHeap_.isValid()) { LOG(SoftwareIsp, Error) << "Failed to create DmaHeap object"; return; From patchwork Fri May 31 12:38:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20174 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 91239C32CF for ; Fri, 31 May 2024 12:39:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 04DF1634BB; Fri, 31 May 2024 14:39:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="HJBzDnat"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C7268634AF for ; Fri, 31 May 2024 14:39:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1717159144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/fVxUNuN4Dbu/i/awUriJfka2P8NIY37h2BlWfSWK00=; b=HJBzDnatLeFMbtagO0jCVRRsELhA/T63+p/OAXr3cVTnEZcxIwDyRSx77OmPN0VUSa2Wlu aJ6SLIyeOFfcdwBdjElToKORiV8dXEebpKbk3G9ripronwN+6F+15quSw55Me3yHAxdBH6 +NwRTlehlsV3ITEpLlxH4EDViTqxLMA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-451-8x4kCdHoOkOwebuODblCXQ-1; Fri, 31 May 2024 08:39:03 -0400 X-MC-Unique: 8x4kCdHoOkOwebuODblCXQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E2D9D8008A4; Fri, 31 May 2024 12:39:02 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id 73DDEC15BB1; Fri, 31 May 2024 12:39:01 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v6 4/5] libcamera: software_isp: Remove DebayerParams::kGain10 Date: Fri, 31 May 2024 14:38:39 +0200 Message-ID: <20240531123840.713364-5-mzamazal@redhat.com> In-Reply-To: <20240531123840.713364-1-mzamazal@redhat.com> References: <20240531123840.713364-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The constant is used in a single place internally and doesn't belong to DebayerParams anymore. Let's use 256 directly. Signed-off-by: Milan Zamazal Reviewed-by: Andrei Konovalov Reviewed-by: Laurent Pinchart --- include/libcamera/internal/software_isp/debayer_params.h | 1 - src/ipa/simple/soft_simple.cpp | 3 +-- src/libcamera/software_isp/debayer.cpp | 5 ----- 3 files changed, 1 insertion(+), 8 deletions(-) diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index 463d24b3..7d8fdd48 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -16,7 +16,6 @@ namespace libcamera { struct DebayerParams { - static constexpr unsigned int kGain10 = 256; static constexpr unsigned int kRGBLookupSize = 256; using ColorLookupTable = std::array; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 3388686f..fb974bc3 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -289,8 +289,7 @@ void IPASoftSimple::processStats(const ControlList &sensorControls) for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { constexpr unsigned int div = - DebayerParams::kRGBLookupSize * DebayerParams::kGain10 / - kGammaLookupSize; + DebayerParams::kRGBLookupSize * 256 / kGammaLookupSize; unsigned int idx; /* Apply gamma after gain! */ diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index 3f3969f7..f4a299d5 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -18,11 +18,6 @@ namespace libcamera { * \brief Struct to hold the debayer parameters. */ -/** - * \var DebayerParams::kGain10 - * \brief const value for 1.0 gain - */ - /** * \var DebayerParams::kRGBLookupSize * \brief Size of a color lookup table From patchwork Fri May 31 12:38:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20175 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 814F4BDE6B for ; Fri, 31 May 2024 12:39:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 35FE8634BB; Fri, 31 May 2024 14:39:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="IXPgQLYO"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2DFD6634C1 for ; Fri, 31 May 2024 14:39:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1717159148; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nAJJ0CwBMeWZALbSvYF5n1d2870rlVjZNSEQuNL68U4=; b=IXPgQLYO4SEqMwnYJRsjxj6heilo9AcWwQDZ2kyNjDrzOiBqivZfBpJ+zgi8AAMEJHl5WW 8nVeIgTs1lauFOkxGBcr4P6P//zP/XYlAziHUmlh+zwBYxXLhqKAL0MAYwwbBoejhq7q41 MFi7F3TG3RzgHF2VJpUzR1ot4JV1ohs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-333-NNT-RhFjNQmhAaw33yadNw-1; Fri, 31 May 2024 08:39:04 -0400 X-MC-Unique: NNT-RhFjNQmhAaw33yadNw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5A99885A58C; Fri, 31 May 2024 12:39:04 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4DCEAC15BB1; Fri, 31 May 2024 12:39:03 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Andrei Konovalov , Laurent Pinchart Subject: [PATCH v6 5/5] libcamera: software_isp: Remove TODO about internal representation Date: Fri, 31 May 2024 14:38:40 +0200 Message-ID: <20240531123840.713364-6-mzamazal@redhat.com> In-Reply-To: <20240531123840.713364-1-mzamazal@redhat.com> References: <20240531123840.713364-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" TODO #4 was recorded at a time where the IPA module computed gain values and the ISP computed the look up tables. The gains were higher-level parameters. Now that the look up tables are computed in the IPA module, the IPA and ISP are more tightly coupled and the TODO item is less relevant. Let's drop the TODO item. We may or may not need to switch to a different representation in future but there is currently no good need for this and the conversion of the values would be just waste of CPU cycles. Signed-off-by: Milan Zamazal Reviewed-by: Laurent Pinchart --- src/libcamera/software_isp/TODO | 13 ------------- 1 file changed, 13 deletions(-) diff --git a/src/libcamera/software_isp/TODO b/src/libcamera/software_isp/TODO index 4fcee39b..6bdc5905 100644 --- a/src/libcamera/software_isp/TODO +++ b/src/libcamera/software_isp/TODO @@ -72,19 +72,6 @@ stats in hardware, such as the i.MX7), but please keep it on your radar. --- -4. Hide internal representation of gains from callers - -> struct DebayerParams { -> static constexpr unsigned int kGain10 = 256; - -Forcing the caller to deal with the internal representation of gains -isn't nice, especially given that it precludes implementing gains of -different precisions in different backend. Wouldn't it be better to pass -the values as floating point numbers, and convert them to the internal -representation in the implementation of process() before using them ? - ---- - 5. Store ISP parameters in per-frame buffers > /**