From patchwork Wed Jul 3 13:49:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20508 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8C0EEBD87C for ; Wed, 3 Jul 2024 13:50:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0319362E25; Wed, 3 Jul 2024 15:50:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="D38LDlSF"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D98EE62C95 for ; Wed, 3 Jul 2024 15:50:08 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BFE713D6; Wed, 3 Jul 2024 15:49:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720014580; bh=HX3uAUPRMm66rx8wnqck16xTrciPzeJCvjFVfNnc+Ls=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=D38LDlSF970FKEbn5zqrMmGQYaytb8NOD0+vRAJCGb+WD/Z/3Qte2dowxbVeqjaiW aFlp7ZTjLTtCw/ytDqUObN23Sel9CoGkRfK+Jon2i9nG61xl7mP322fDaJPrQdxJfR Q9KhxhPRh5dqaE59X3ZmLbKt8YJ+A/uv4OADpk1A= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Kieran Bingham , Laurent Pinchart Subject: [PATCH v3 1/6] ipa: libipa: Add black levels to camera sensor helper Date: Wed, 3 Jul 2024 15:49:49 +0200 Message-ID: <20240703135000.242227-2-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703135000.242227-1-stefan.klug@ideasonboard.com> References: <20240703135000.242227-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For a proper tuning process we need to know the sensor black levels. In most cases these are fixed and not reported by the kernel driver. Store them inside the sensor helpers for later retrieval by the algorithms. Add black level value corresponding to the data pedestal for three initial sensors as documented in the datasheets. More should be added, eventually filling the gaps for all supported sensors. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/libipa/camera_sensor_helper.cpp | 38 +++++++++++++++++++++++++ src/ipa/libipa/camera_sensor_helper.h | 3 ++ 2 files changed, 41 insertions(+) diff --git a/src/ipa/libipa/camera_sensor_helper.cpp b/src/ipa/libipa/camera_sensor_helper.cpp index 782ff9904e81..82c795eac71b 100644 --- a/src/ipa/libipa/camera_sensor_helper.cpp +++ b/src/ipa/libipa/camera_sensor_helper.cpp @@ -47,6 +47,33 @@ namespace ipa { * function. */ +/** + * \fn CameraSensorHelper::blackLevel() + * \brief Fetch the black level of the sensor + * + * This function returns the black level of the sensor scaled to a 16bit pixel + * width. If it is unknown an empty optional is returned. + * + * \todo Fill the blanks and add pedestal values for all supported sensors. Once + * done, drop the std::optional<>. + * + * Black levels are typically the result of the following phenomena: + * - Pedestal added by the sensor to pixel values. They are typically fixed, + * sometimes programmable and should be reported in datasheets (but + * documentation is not always available). + * - Dark currents and other physical effects that add charge to pixels in the + * absence of light. Those can depend on the integration time and the sensor + * die temperature, and their contribution to pixel values depend on the + * sensor gains. + * + * The pedestal is usually the value with the biggest contribution to the + * overall black level. In most cases it is either known before or in rare cases + * (there is not a single driver with such a control in the linux kernel) can be + * queried from the sensor. This function provides that fixed, known value. + * + * \return The black level of the sensor, or std::nullopt if not known + */ + /** * \brief Compute gain code from the analogue gain absolute value * \param[in] gain The real gain to pass @@ -204,6 +231,11 @@ double CameraSensorHelper::gain(uint32_t gainCode) const * \brief Constants for the exponential gain model */ +/** + * \var CameraSensorHelper::blackLevel_ + * \brief The black level of the sensor + */ + /** * \var CameraSensorHelper::gainType_ * \brief The analogue gain model type @@ -396,6 +428,8 @@ class CameraSensorHelperImx219 : public CameraSensorHelper public: CameraSensorHelperImx219() { + /* From datasheet: 64 at 10bits. */ + blackLevel_ = 4096; gainType_ = AnalogueGainLinear; gainConstants_.linear = { 0, 256, -1, 256 }; } @@ -407,6 +441,8 @@ class CameraSensorHelperImx258 : public CameraSensorHelper public: CameraSensorHelperImx258() { + /* From datasheet: 0x40 at 10bits. */ + blackLevel_ = 4096; gainType_ = AnalogueGainLinear; gainConstants_.linear = { 0, 512, -1, 512 }; } @@ -456,6 +492,8 @@ class CameraSensorHelperImx335 : public CameraSensorHelper public: CameraSensorHelperImx335() { + /* From datasheet: 0x32 at 10bits. */ + blackLevel_ = 3200; gainType_ = AnalogueGainExponential; gainConstants_.exp = { 1.0, expGainDb(0.3) }; } diff --git a/src/ipa/libipa/camera_sensor_helper.h b/src/ipa/libipa/camera_sensor_helper.h index 0d99073bea82..ac276e27f523 100644 --- a/src/ipa/libipa/camera_sensor_helper.h +++ b/src/ipa/libipa/camera_sensor_helper.h @@ -10,6 +10,7 @@ #include #include +#include #include #include @@ -25,6 +26,7 @@ public: CameraSensorHelper() = default; virtual ~CameraSensorHelper() = default; + std::optional blackLevel() const { return blackLevel_; } virtual uint32_t gainCode(double gain) const; virtual double gain(uint32_t gainCode) const; @@ -51,6 +53,7 @@ protected: AnalogueGainExpConstants exp; }; + std::optional blackLevel_; AnalogueGainType gainType_; AnalogueGainConstants gainConstants_; From patchwork Wed Jul 3 13:49:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20509 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B100CBD87C for ; Wed, 3 Jul 2024 13:50:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5E3F5632EA; Wed, 3 Jul 2024 15:50:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="e2DM5XC9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 155A462E27 for ; Wed, 3 Jul 2024 15:50:12 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8A1848A9; Wed, 3 Jul 2024 15:49:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720014583; bh=G383aHbw8E7pTJIMKzLg+UXFUSpS+FMnr4iJpel8N8U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=e2DM5XC9qq2O1WVpqJ59yOx2XoSG+m5lcd2p3aXQiiQ9UH33jvaaYqHUKzz3S5s+H sWgeAxe5Znk7WRTWE9nWj9ECdt1AZ4UYkJ91Cg2UHSR2AjdQZNFPqVZojncWHlykkY MOMiMjU2ZeRh7bO2iLWxC7rxoRJ09jmysepal9Hs= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Jacopo Mondi , Laurent Pinchart , Paul Elder Subject: [PATCH v3 2/6] ipa: rkisp1: Move camHelper into IPAContext Date: Wed, 3 Jul 2024 15:49:50 +0200 Message-ID: <20240703135000.242227-3-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703135000.242227-1-stefan.klug@ideasonboard.com> References: <20240703135000.242227-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To be able to query the black levels, the black level correction algorithm needs access to the camera sensor helper. Allow this by moving the camHelper_ member from IPARkISP1 into IPAContext. Signed-off-by: Stefan Klug Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart Reviewed-by: Paul Elder --- src/ipa/rkisp1/ipa_context.h | 6 ++++++ src/ipa/rkisp1/rkisp1.cpp | 26 ++++++++++++-------------- 2 files changed, 18 insertions(+), 14 deletions(-) diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 8602b408870e..1d0e9030af1c 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -8,6 +8,8 @@ #pragma once +#include + #include #include @@ -16,6 +18,7 @@ #include #include +#include #include #include @@ -178,6 +181,9 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; + + /* Interface to the Camera Helper */ + std::unique_ptr camHelper; }; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index d31cdbab020b..23e0826cc335 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -29,7 +29,6 @@ #include "libcamera/internal/yaml_parser.h" #include "algorithms/algorithm.h" -#include "libipa/camera_sensor_helper.h" #include "ipa_context.h" @@ -81,9 +80,6 @@ private: ControlInfoMap sensorControls_; - /* Interface to the Camera Helper */ - std::unique_ptr camHelper_; - /* Local parameter storage */ struct IPAContext context_; }; @@ -115,7 +111,7 @@ const ControlInfoMap::Map rkisp1Controls{ } /* namespace */ IPARkISP1::IPARkISP1() - : context_({ {}, {}, {}, { kMaxFrameContexts }, {} }) + : context_({ {}, {}, {}, { kMaxFrameContexts }, {}, {} }) { } @@ -147,8 +143,8 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision, LOG(IPARkISP1, Debug) << "Hardware revision is " << hwRevision; - camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); - if (!camHelper_) { + context_.camHelper = CameraSensorHelperFactoryBase::create(settings.sensorModel); + if (!context_.camHelper) { LOG(IPARkISP1, Error) << "Failed to create camera sensor helper for " << settings.sensorModel; @@ -250,8 +246,10 @@ int IPARkISP1::configure(const IPAConfigInfo &ipaConfig, minExposure * context_.configuration.sensor.lineDuration; context_.configuration.sensor.maxShutterSpeed = maxExposure * context_.configuration.sensor.lineDuration; - context_.configuration.sensor.minAnalogueGain = camHelper_->gain(minGain); - context_.configuration.sensor.maxAnalogueGain = camHelper_->gain(maxGain); + context_.configuration.sensor.minAnalogueGain = + context_.camHelper->gain(minGain); + context_.configuration.sensor.maxAnalogueGain = + context_.camHelper->gain(maxGain); context_.configuration.raw = std::any_of(streamConfig.begin(), streamConfig.end(), [](auto &cfg) -> bool { @@ -352,7 +350,7 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); frameContext.sensor.gain = - camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + context_.camHelper->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); ControlList metadata(controls::controls); @@ -389,9 +387,9 @@ void IPARkISP1::updateControls(const IPACameraSensorInfo &sensorInfo, /* Compute the analogue gain limits. */ const ControlInfo &v4l2Gain = sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; - float minGain = camHelper_->gain(v4l2Gain.min().get()); - float maxGain = camHelper_->gain(v4l2Gain.max().get()); - float defGain = camHelper_->gain(v4l2Gain.def().get()); + float minGain = context_.camHelper->gain(v4l2Gain.min().get()); + float maxGain = context_.camHelper->gain(v4l2Gain.max().get()); + float defGain = context_.camHelper->gain(v4l2Gain.def().get()); ctrlMap.emplace(std::piecewise_construct, std::forward_as_tuple(&controls::AnalogueGain), std::forward_as_tuple(minGain, maxGain, defGain)); @@ -436,7 +434,7 @@ void IPARkISP1::setControls(unsigned int frame) IPAFrameContext &frameContext = context_.frameContexts.get(frame); uint32_t exposure = frameContext.agc.exposure; - uint32_t gain = camHelper_->gainCode(frameContext.agc.gain); + uint32_t gain = context_.camHelper->gainCode(frameContext.agc.gain); ControlList ctrls(sensorControls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); From patchwork Wed Jul 3 13:49:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20510 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 014E1BD87C for ; Wed, 3 Jul 2024 13:50:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AB46963339; Wed, 3 Jul 2024 15:50:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="F90E28fe"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ED11962E25 for ; Wed, 3 Jul 2024 15:50:14 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BFFA43D6; Wed, 3 Jul 2024 15:49:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720014586; bh=14LbDAN6T5tIXqacl03ylzHMz+akW+S3DqGC85tHzXQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F90E28fe1/nM8RDu++sENsY3aIBKrIyD0cMe0NRP2RqOKD5FfuF4busGnNkNGRDO0 NfGYiaNq5dLF+ox0NFTdFjVcsSdiYSFwGB1SY5AVO3Cqa5HU45RP+qinCCW2pk0VK1 Dw2/XVX/TlEY/UEkYTxpk7La9x4tP16XAWzfr62I= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham , Laurent Pinchart Subject: [PATCH v3 3/6] ipa: rkisp1: blc: Query black levels from camera sensor helper Date: Wed, 3 Jul 2024 15:49:51 +0200 Message-ID: <20240703135000.242227-4-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703135000.242227-1-stefan.klug@ideasonboard.com> References: <20240703135000.242227-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As the camera sensor helper now has the ability to provide the black level, use it. Black levels can still be overwritten by the tuning file, but the direction is to remove them from the tuning files and move them into the sensor helpers. Additionally interpret all values based on 16bits. The conversion to the scale required by the hardware is done in process(). It ensures all the values inside libcamera are the same scale and is in preparation for the i.MX8MP where black levels are based on a 20bit scale. Note that this breaks existing tuning files. The tuning files distributed with libcamera will be fixed in a later patch. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Paul Elder --- src/ipa/rkisp1/algorithms/blc.cpp | 54 ++++++++++++++++++++++++++----- 1 file changed, 46 insertions(+), 8 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index d2e743541c99..b17b549c55df 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -46,10 +46,47 @@ BlackLevelCorrection::BlackLevelCorrection() int BlackLevelCorrection::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) { - blackLevelRed_ = tuningData["R"].get(256); - blackLevelGreenR_ = tuningData["Gr"].get(256); - blackLevelGreenB_ = tuningData["Gb"].get(256); - blackLevelBlue_ = tuningData["B"].get(256); + std::optional levelRed = tuningData["R"].get(); + std::optional levelGreenR = tuningData["Gr"].get(); + std::optional levelGreenB = tuningData["Gb"].get(); + std::optional levelBlue = tuningData["B"].get(); + bool tuningHasLevels = levelRed && levelGreenR && levelGreenB && levelBlue; + + auto blackLevel = context.camHelper->blackLevel(); + if (!blackLevel) { + /* + * Not all camera sensor helpers have been updated with black + * levels. Print a warning and fall back to the levels from the + * tuning data to preserve backward compatibility. This should + * be removed once all helpers provide the data. + */ + LOG(RkISP1Blc, Warning) + << "No black levels provided by camera sensor helper" + << ", please fix"; + + blackLevelRed_ = levelRed.value_or(4096); + blackLevelGreenR_ = levelGreenR.value_or(4096); + blackLevelGreenB_ = levelGreenB.value_or(4096); + blackLevelBlue_ = levelBlue.value_or(4096); + } else if (tuningHasLevels) { + /* + * If black levels are provided in the tuning file, use them to + * avoid breaking existing camera tuning. This is deprecated and + * will be removed. + */ + LOG(RkISP1Blc, Warning) + << "Deprecated: black levels overwritten by tuning file"; + + blackLevelRed_ = *levelRed; + blackLevelGreenR_ = *levelGreenR; + blackLevelGreenB_ = *levelGreenB; + blackLevelBlue_ = *levelBlue; + } else { + blackLevelRed_ = *blackLevel; + blackLevelGreenR_ = *blackLevel; + blackLevelGreenB_ = *blackLevel; + blackLevelBlue_ = *blackLevel; + } tuningParameters_ = true; @@ -77,10 +114,11 @@ void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context, return; params->others.bls_config.enable_auto = 0; - params->others.bls_config.fixed_val.r = blackLevelRed_; - params->others.bls_config.fixed_val.gr = blackLevelGreenR_; - params->others.bls_config.fixed_val.gb = blackLevelGreenB_; - params->others.bls_config.fixed_val.b = blackLevelBlue_; + /* The rkisp1 uses 12bit based black levels. Scale down accordingly. */ + params->others.bls_config.fixed_val.r = blackLevelRed_ >> 4; + params->others.bls_config.fixed_val.gr = blackLevelGreenR_ >> 4; + params->others.bls_config.fixed_val.gb = blackLevelGreenB_ >> 4; + params->others.bls_config.fixed_val.b = blackLevelBlue_ >> 4; params->module_en_update |= RKISP1_CIF_ISP_MODULE_BLS; params->module_ens |= RKISP1_CIF_ISP_MODULE_BLS; From patchwork Wed Jul 3 13:49:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20511 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1484EBD87C for ; Wed, 3 Jul 2024 13:50:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A24FB6333F; Wed, 3 Jul 2024 15:50:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="b4Z7+6A5"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C08E062E27 for ; Wed, 3 Jul 2024 15:50:17 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A0CC63D6; Wed, 3 Jul 2024 15:49:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720014589; bh=ofJCdRARy3bVq6ylNyfVEotk9tOJ0bSugQ9wzbHXLJk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b4Z7+6A5InGQDuS4ogX/KeDwH7CEPBZUiLxwgSZUWu+FVq9H1x+c5+3SstkIEcrOA XnPpewANuhFX8YGnHXSeayxLfGd22nFEbx1r0CkvNij9WTu5n1yPlgQuPhyCjbIltg urI4/kIDYEC5Nq7aSw9Dh6rby/z6P8HyRdNvTraU= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Kieran Bingham , Laurent Pinchart Subject: [PATCH v3 4/6] ipa: rkisp1: blc: Report sensor black levels in metadata Date: Wed, 3 Jul 2024 15:49:52 +0200 Message-ID: <20240703135000.242227-5-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703135000.242227-1-stefan.klug@ideasonboard.com> References: <20240703135000.242227-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add sensor black levels to the metadata of the rkisp1 pipeline. Additionally enable raw support for this algorithm and add it to uncalibrated.yaml, so that black levels get reported when capturing tuning images. This is a bit of a hack, because no actual black level correction is taking place in raw mode, but it is the easiest way to get blacklevel reported for raw streams. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/algorithms/blc.cpp | 28 +++++++++++++++++++++++++++ src/ipa/rkisp1/algorithms/blc.h | 5 ++++- src/ipa/rkisp1/data/uncalibrated.yaml | 1 + 3 files changed, 33 insertions(+), 1 deletion(-) diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index b17b549c55df..03ec57f24be0 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -9,6 +9,8 @@ #include +#include + #include "libcamera/internal/yaml_parser.h" /** @@ -38,6 +40,13 @@ LOG_DEFINE_CATEGORY(RkISP1Blc) BlackLevelCorrection::BlackLevelCorrection() : tuningParameters_(false) { + /* + * This is a bit of a hack. In raw mode no black level correction + * happens. This flag is used to ensure the metadata gets populated with + * the black level which is needed to capture proper raw images for + * tuning. + */ + supportsRaw_ = true; } /** @@ -107,6 +116,9 @@ void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { + if (context.configuration.raw) + return; + if (frame > 0) return; @@ -125,6 +137,22 @@ void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context, params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_BLS; } +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void BlackLevelCorrection::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] const rkisp1_stat_buffer *stats, + ControlList &metadata) +{ + metadata.set(controls::SensorBlackLevels, + { static_cast(blackLevelRed_), + static_cast(blackLevelGreenR_), + static_cast(blackLevelGreenB_), + static_cast(blackLevelBlue_) }); +} + REGISTER_IPA_ALGORITHM(BlackLevelCorrection, "BlackLevelCorrection") } /* namespace ipa::rkisp1::algorithms */ diff --git a/src/ipa/rkisp1/algorithms/blc.h b/src/ipa/rkisp1/algorithms/blc.h index 460ebcc15739..4ecac233f88b 100644 --- a/src/ipa/rkisp1/algorithms/blc.h +++ b/src/ipa/rkisp1/algorithms/blc.h @@ -23,7 +23,10 @@ public: void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, rkisp1_params_cfg *params) override; - + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const rkisp1_stat_buffer *stats, + ControlList &metadata) override; private: bool tuningParameters_; int16_t blackLevelRed_; diff --git a/src/ipa/rkisp1/data/uncalibrated.yaml b/src/ipa/rkisp1/data/uncalibrated.yaml index a7bbd8d84263..609012967e02 100644 --- a/src/ipa/rkisp1/data/uncalibrated.yaml +++ b/src/ipa/rkisp1/data/uncalibrated.yaml @@ -5,4 +5,5 @@ version: 1 algorithms: - Agc: - Awb: + - BlackLevelCorrection: ... From patchwork Wed Jul 3 13:49:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20512 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5F158BD87C for ; Wed, 3 Jul 2024 13:50:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0E100632EA; Wed, 3 Jul 2024 15:50:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="wAY5EJCf"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6934562E27 for ; Wed, 3 Jul 2024 15:50:20 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3F5146D6; Wed, 3 Jul 2024 15:49:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720014592; bh=IQ3s1rqGzGWvS5f5TZ5PlU7yx5z5cuqCRYfPcfMk2Wg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wAY5EJCfZlgefMq0qJrlep5/aPZsXjCvRkA8XT/aVngvWDxhZj9UsW62wk+iCOhS3 sLI4XkQSLajksOCp8Ylk+R1z5Ga/v57/Y8DizlZmx7qdbBEKe6citrd5EOjRNvPy1g osk/kbxVeNCepd8zkC+TlpozsG7IFq6/f7zmW4SU= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham , Laurent Pinchart Subject: [PATCH v3 5/6] ipa: rkisp1: data: Update tuning files for imx219 and imx258 Date: Wed, 3 Jul 2024 15:49:53 +0200 Message-ID: <20240703135000.242227-6-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703135000.242227-1-stefan.klug@ideasonboard.com> References: <20240703135000.242227-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The black levels for imx219 and imx258 are now contained in the camera sensor helpers. Remove them from the tuning file for the imx219. Add a BLC entry to the imx258 tuning file. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Reviewed-by: Paul Elder --- src/ipa/rkisp1/data/imx219.yaml | 4 ---- src/ipa/rkisp1/data/imx258.yaml | 1 + 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/src/ipa/rkisp1/data/imx219.yaml b/src/ipa/rkisp1/data/imx219.yaml index cbcc43b84ac7..0d99cb529392 100644 --- a/src/ipa/rkisp1/data/imx219.yaml +++ b/src/ipa/rkisp1/data/imx219.yaml @@ -6,10 +6,6 @@ algorithms: - Agc: - Awb: - BlackLevelCorrection: - R: 256 - Gr: 256 - Gb: 256 - B: 256 - LensShadingCorrection: x-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ] y-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ] diff --git a/src/ipa/rkisp1/data/imx258.yaml b/src/ipa/rkisp1/data/imx258.yaml index 43dddf20dcd2..202af36afbee 100644 --- a/src/ipa/rkisp1/data/imx258.yaml +++ b/src/ipa/rkisp1/data/imx258.yaml @@ -5,6 +5,7 @@ version: 1 algorithms: - Agc: - Awb: + - BlackLevelCorrection: - LensShadingCorrection: x-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ] y-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ] From patchwork Wed Jul 3 13:49:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20513 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A3B90BD87C for ; Wed, 3 Jul 2024 13:50:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EFC6963334; Wed, 3 Jul 2024 15:50:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cHrmybGD"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E808D62C96 for ; Wed, 3 Jul 2024 15:50:22 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CA3C56D6; Wed, 3 Jul 2024 15:49:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720014594; bh=Cr0An1Q//TfybVUxkoTtx3xp4rKuO3uPiB8KbIFCzY4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cHrmybGDGzkSy6uH8+G7VGUPbHNjtnZ8YLQNvT88wpbp96Et53v1Y28cTLQrXdV3e ezbbYLovgxtSieiCfdnt53KiFpr9twiYkEIpw+ZfpqK7ZEuTkP7twcmlIXyatNu9e1 GaM2xoYzWCko08pJs0zvAgVVf1qgqt5pw3WY7teQ= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham , Laurent Pinchart Subject: [PATCH v3 6/6] ipa: rkisp1: Move ov4689 and ov5640 black levels into sensor helpers Date: Wed, 3 Jul 2024 15:49:54 +0200 Message-ID: <20240703135000.242227-7-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703135000.242227-1-stefan.klug@ideasonboard.com> References: <20240703135000.242227-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Move black levels for tuning files that contained a BLC block into the camera sensor helpers. ov4689.yaml had 66@12bit while the datasheet states 64@12bit. Use the value from the datasheet (scaled to 16bit). ov5640.yaml had 256@12bit while the datasheet states 16@10bit. Looking at the commit message the 256 most likely stems from the imx219 tuning file and 16@10bit is the same as the 64@12bit from the ov4689. This seems more likely and is therefore used. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/libipa/camera_sensor_helper.cpp | 4 ++++ src/ipa/rkisp1/data/ov4689.yaml | 4 ---- src/ipa/rkisp1/data/ov5640.yaml | 4 ---- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/src/ipa/libipa/camera_sensor_helper.cpp b/src/ipa/libipa/camera_sensor_helper.cpp index 82c795eac71b..a1339c83c4cb 100644 --- a/src/ipa/libipa/camera_sensor_helper.cpp +++ b/src/ipa/libipa/camera_sensor_helper.cpp @@ -553,6 +553,8 @@ class CameraSensorHelperOv4689 : public CameraSensorHelper public: CameraSensorHelperOv4689() { + /* From datasheet: 0x40 at 12bits. */ + blackLevel_ = 1024; gainType_ = AnalogueGainLinear; gainConstants_.linear = { 1, 0, 0, 128 }; } @@ -564,6 +566,8 @@ class CameraSensorHelperOv5640 : public CameraSensorHelper public: CameraSensorHelperOv5640() { + /* From datasheet: 0x10 at 10bits. */ + blackLevel_ = 1024; gainType_ = AnalogueGainLinear; gainConstants_.linear = { 1, 0, 0, 16 }; } diff --git a/src/ipa/rkisp1/data/ov4689.yaml b/src/ipa/rkisp1/data/ov4689.yaml index 2068684cafcd..609012967e02 100644 --- a/src/ipa/rkisp1/data/ov4689.yaml +++ b/src/ipa/rkisp1/data/ov4689.yaml @@ -6,8 +6,4 @@ algorithms: - Agc: - Awb: - BlackLevelCorrection: - R: 66 - Gr: 66 - Gb: 66 - B: 66 ... diff --git a/src/ipa/rkisp1/data/ov5640.yaml b/src/ipa/rkisp1/data/ov5640.yaml index 897b83cb435b..4b21d412e44e 100644 --- a/src/ipa/rkisp1/data/ov5640.yaml +++ b/src/ipa/rkisp1/data/ov5640.yaml @@ -6,10 +6,6 @@ algorithms: - Agc: - Awb: - BlackLevelCorrection: - R: 256 - Gr: 256 - Gb: 256 - B: 256 - ColorProcessing: - GammaSensorLinearization: x-intervals: [ 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2 ]