From patchwork Wed Jul 3 10:39:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20501 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3F83ABEFBE for ; Wed, 3 Jul 2024 10:40:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DE92862E25; Wed, 3 Jul 2024 12:40:16 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GMH3gIv1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2246C62C95 for ; Wed, 3 Jul 2024 12:40:15 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 125046D6; Wed, 3 Jul 2024 12:39:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720003187; bh=dBTzKx/UVMGyiAOzeu1YD+O1cwsF+xeLRK6elxGwmXo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GMH3gIv13mGr9tTo4qNV8n0Y1uhTKB5tyr3YqJDLQDCAOqJcOJe/vaXMiSKL1dma4 2o3FlTq2u7ZTDy6TbVuB3Eshou38jfSKO7t//p0cA1+cK1PS52Z/LlGDTtp+9FUOrx DHSB7DSWjTuA5Sn5Mr7DjV3PgI/IZgd1iGNzlUxY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 1/6] ipa: libipa: Add black levels to camera sensor helper Date: Wed, 3 Jul 2024 12:39:48 +0200 Message-ID: <20240703104004.184783-2-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703104004.184783-1-stefan.klug@ideasonboard.com> References: <20240703104004.184783-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For a proper tuning process we need to know the sensor black levels. In most cases these are fixed and not reported by the kernel driver. Store them inside the sensor helpers for later retrieval by the algorithms. Add black level value corresponding to the data pedestal for three initial sensors as documented in the datasheets. More should be added, eventually filling the gaps for all supported sensors. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/libipa/camera_sensor_helper.cpp | 27 +++++++++++++++++++++++++ src/ipa/libipa/camera_sensor_helper.h | 3 +++ 2 files changed, 30 insertions(+) diff --git a/src/ipa/libipa/camera_sensor_helper.cpp b/src/ipa/libipa/camera_sensor_helper.cpp index 782ff9904e81..3d0e756927f0 100644 --- a/src/ipa/libipa/camera_sensor_helper.cpp +++ b/src/ipa/libipa/camera_sensor_helper.cpp @@ -47,6 +47,30 @@ namespace ipa { * function. */ +/** + * \fn CameraSensorHelper::blackLevel() + * \brief Fetch the black level of the sensor + * + * This function returns the black level of the sensor scaled to a 16bit pixel + * width. If it is unknown an empty optional is returned. + * + * Black levels are typically the result of the following phenomena: + * - Pedestal added by the sensor to pixel values. They are typically fixed, + * sometimes programmable and should be reported in datasheets (but + * documentation is not always available). + * - Dark currents and other physical effects that add charge to pixels in the + * absence of light. Those can depend on the integration time and the sensor + * die temperature, and their contribution to pixel values depend on the + * sensor gains. + * + * The pedestal is usually the value with the biggest contribution to the + * overall black level. In most cases it is either known before or in rare cases + * (There is not a single driver with such a control in the linux kernel) can be + * queried from the sensor. This function shall provide that fixed, known value. + * + * \return The black level of the sensor, or std::nullopt if not known + */ + /** * \brief Compute gain code from the analogue gain absolute value * \param[in] gain The real gain to pass @@ -396,6 +420,7 @@ class CameraSensorHelperImx219 : public CameraSensorHelper public: CameraSensorHelperImx219() { + blackLevel_ = 4096; gainType_ = AnalogueGainLinear; gainConstants_.linear = { 0, 256, -1, 256 }; } @@ -407,6 +432,7 @@ class CameraSensorHelperImx258 : public CameraSensorHelper public: CameraSensorHelperImx258() { + blackLevel_ = 4096; gainType_ = AnalogueGainLinear; gainConstants_.linear = { 0, 512, -1, 512 }; } @@ -456,6 +482,7 @@ class CameraSensorHelperImx335 : public CameraSensorHelper public: CameraSensorHelperImx335() { + blackLevel_ = 3200; gainType_ = AnalogueGainExponential; gainConstants_.exp = { 1.0, expGainDb(0.3) }; } diff --git a/src/ipa/libipa/camera_sensor_helper.h b/src/ipa/libipa/camera_sensor_helper.h index 0d99073bea82..ac276e27f523 100644 --- a/src/ipa/libipa/camera_sensor_helper.h +++ b/src/ipa/libipa/camera_sensor_helper.h @@ -10,6 +10,7 @@ #include #include +#include #include #include @@ -25,6 +26,7 @@ public: CameraSensorHelper() = default; virtual ~CameraSensorHelper() = default; + std::optional blackLevel() const { return blackLevel_; } virtual uint32_t gainCode(double gain) const; virtual double gain(uint32_t gainCode) const; @@ -51,6 +53,7 @@ protected: AnalogueGainExpConstants exp; }; + std::optional blackLevel_; AnalogueGainType gainType_; AnalogueGainConstants gainConstants_; From patchwork Wed Jul 3 10:39:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20502 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7A3CBBEFBE for ; Wed, 3 Jul 2024 10:40:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0FCBA632EA; Wed, 3 Jul 2024 12:40:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uRIMMdae"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 27F8562C99 for ; Wed, 3 Jul 2024 12:40:18 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F0A383E6; Wed, 3 Jul 2024 12:39:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720003190; bh=q/YSilUgnwsOWh4ve41Ybz0Qx+TGqQGZFZiMqo3YbF0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uRIMMdae5YeboN9RXl897cAU3RIazrULDV2KkeYXJdsPLH8WSKIKgfGQWVAyCQgc+ gc/yJ4+WGN/ihMUEs9MaAWP1DvUiYlnkR4alQHlzpwCJsrvkJlfU+YEIXh5vDUq7M2 dR8+4WtUyg54vTW0QyoWdjlrXb6wDje2XOJ1M5Ag= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Jacopo Mondi , Laurent Pinchart Subject: [PATCH v2 2/6] ipa: rkisp1: Move camHelper into IPAContext Date: Wed, 3 Jul 2024 12:39:49 +0200 Message-ID: <20240703104004.184783-3-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703104004.184783-1-stefan.klug@ideasonboard.com> References: <20240703104004.184783-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To be able to query the black levels, the black level correction algorithm needs access to the camera sensor helper. Allow this by moving the camHelper_ member from IPARkISP1 into IPAContext. Signed-off-by: Stefan Klug Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart Reviewed-by: Paul Elder --- src/ipa/rkisp1/ipa_context.h | 6 ++++++ src/ipa/rkisp1/rkisp1.cpp | 26 ++++++++++++-------------- 2 files changed, 18 insertions(+), 14 deletions(-) diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 8602b408870e..1d0e9030af1c 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -8,6 +8,8 @@ #pragma once +#include + #include #include @@ -16,6 +18,7 @@ #include #include +#include #include #include @@ -178,6 +181,9 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; + + /* Interface to the Camera Helper */ + std::unique_ptr camHelper; }; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index d31cdbab020b..23e0826cc335 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -29,7 +29,6 @@ #include "libcamera/internal/yaml_parser.h" #include "algorithms/algorithm.h" -#include "libipa/camera_sensor_helper.h" #include "ipa_context.h" @@ -81,9 +80,6 @@ private: ControlInfoMap sensorControls_; - /* Interface to the Camera Helper */ - std::unique_ptr camHelper_; - /* Local parameter storage */ struct IPAContext context_; }; @@ -115,7 +111,7 @@ const ControlInfoMap::Map rkisp1Controls{ } /* namespace */ IPARkISP1::IPARkISP1() - : context_({ {}, {}, {}, { kMaxFrameContexts }, {} }) + : context_({ {}, {}, {}, { kMaxFrameContexts }, {}, {} }) { } @@ -147,8 +143,8 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision, LOG(IPARkISP1, Debug) << "Hardware revision is " << hwRevision; - camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); - if (!camHelper_) { + context_.camHelper = CameraSensorHelperFactoryBase::create(settings.sensorModel); + if (!context_.camHelper) { LOG(IPARkISP1, Error) << "Failed to create camera sensor helper for " << settings.sensorModel; @@ -250,8 +246,10 @@ int IPARkISP1::configure(const IPAConfigInfo &ipaConfig, minExposure * context_.configuration.sensor.lineDuration; context_.configuration.sensor.maxShutterSpeed = maxExposure * context_.configuration.sensor.lineDuration; - context_.configuration.sensor.minAnalogueGain = camHelper_->gain(minGain); - context_.configuration.sensor.maxAnalogueGain = camHelper_->gain(maxGain); + context_.configuration.sensor.minAnalogueGain = + context_.camHelper->gain(minGain); + context_.configuration.sensor.maxAnalogueGain = + context_.camHelper->gain(maxGain); context_.configuration.raw = std::any_of(streamConfig.begin(), streamConfig.end(), [](auto &cfg) -> bool { @@ -352,7 +350,7 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); frameContext.sensor.gain = - camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + context_.camHelper->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); ControlList metadata(controls::controls); @@ -389,9 +387,9 @@ void IPARkISP1::updateControls(const IPACameraSensorInfo &sensorInfo, /* Compute the analogue gain limits. */ const ControlInfo &v4l2Gain = sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; - float minGain = camHelper_->gain(v4l2Gain.min().get()); - float maxGain = camHelper_->gain(v4l2Gain.max().get()); - float defGain = camHelper_->gain(v4l2Gain.def().get()); + float minGain = context_.camHelper->gain(v4l2Gain.min().get()); + float maxGain = context_.camHelper->gain(v4l2Gain.max().get()); + float defGain = context_.camHelper->gain(v4l2Gain.def().get()); ctrlMap.emplace(std::piecewise_construct, std::forward_as_tuple(&controls::AnalogueGain), std::forward_as_tuple(minGain, maxGain, defGain)); @@ -436,7 +434,7 @@ void IPARkISP1::setControls(unsigned int frame) IPAFrameContext &frameContext = context_.frameContexts.get(frame); uint32_t exposure = frameContext.agc.exposure; - uint32_t gain = camHelper_->gainCode(frameContext.agc.gain); + uint32_t gain = context_.camHelper->gainCode(frameContext.agc.gain); ControlList ctrls(sensorControls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); From patchwork Wed Jul 3 10:39:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20503 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0FBEBBEFBE for ; Wed, 3 Jul 2024 10:40:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B614262E25; Wed, 3 Jul 2024 12:40:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="KXwVq3Xp"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CC86E62E27 for ; Wed, 3 Jul 2024 12:40:20 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B9BD53E6; Wed, 3 Jul 2024 12:39:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720003192; bh=bEemI1oYKmHfWod+RJxLxtC1lGGLWuKyrT+kIYXOPoo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KXwVq3Xp2O/ZHhcVZe+AxBsc7J5Cs9+kQSkYyTHgIqm00VazcFe+EUYPly09QGqNF BU+gWYVzwZOCSac766p5WT3xlrTwlY8LyAVPRJdAgIzzCRUbE6JmRqyeuewlz140nx 5WUiEXvrR+px37ILp1y/RUmHV8MKa+kbxLFjZqu0= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 3/6] ipa: rkisp1: blc: Query black levels from camera sensor helper Date: Wed, 3 Jul 2024 12:39:50 +0200 Message-ID: <20240703104004.184783-4-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703104004.184783-1-stefan.klug@ideasonboard.com> References: <20240703104004.184783-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As the camera sensor helper now has the ability to provide the black level, use them. Black levels can still be overwritten by the tuning file. But the direction is to remove them from the tuning files and move them into the sensor helpers. Additionally interpret all values based on 16bits. The conversion to the scale required by the hardware is done in process(). It ensures all the values inside libcamera are the same scale and is in preparation for the i.MX8MP where black levels are based on a 20bit scale. Note that this breaks existing tuning files. The tuning files distributed with libcamera will be fixed in a later patch. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/algorithms/blc.cpp | 54 ++++++++++++++++++++++++++----- 1 file changed, 46 insertions(+), 8 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index d2e743541c99..87025e4f8c72 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -46,10 +46,47 @@ BlackLevelCorrection::BlackLevelCorrection() int BlackLevelCorrection::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) { - blackLevelRed_ = tuningData["R"].get(256); - blackLevelGreenR_ = tuningData["Gr"].get(256); - blackLevelGreenB_ = tuningData["Gb"].get(256); - blackLevelBlue_ = tuningData["B"].get(256); + std::optional levelRed = tuningData["R"].get(); + std::optional levelGreenR = tuningData["Gr"].get(); + std::optional levelGreenB = tuningData["Gb"].get(); + std::optional levelBlue = tuningData["B"].get(); + bool tuningHasLevels = levelRed && levelGreenR && levelGreenB && levelBlue; + + auto blackLevel = context.camHelper->blackLevel(); + if (!blackLevel) { + /* + * Not all camera sensor helpers have been updated with black + * levels. Print a warning and fall back to the levels from the + * tuning data to preserve backward compatibility. This should + * be removed once all helpers provide the data. + */ + LOG(RkISP1Blc, Warning) + << "No black levels provided by camera sensor helper" + << ", please fix"; + + blackLevelRed_ = levelRed.value_or(4096); + blackLevelGreenR_ = levelGreenR.value_or(4096); + blackLevelGreenB_ = levelGreenB.value_or(4096); + blackLevelBlue_ = levelBlue.value_or(4096); + } else if (tuningHasLevels) { + /* + * If black levels are provided in the tuning file, use them to + * avoid breaking existing camera tuning. This is deprecated and + * will be removed. + */ + LOG(RkISP1Blc, Warning) + << "Deprecated: black levels overwritten by tuning file"; + + blackLevelRed_ = *levelRed; + blackLevelGreenR_ = *levelGreenR; + blackLevelGreenB_ = *levelGreenB; + blackLevelBlue_ = *levelBlue; + } else { + blackLevelRed_ = *blackLevel; + blackLevelGreenR_ = *blackLevel; + blackLevelGreenB_ = *blackLevel; + blackLevelBlue_ = *blackLevel; + } tuningParameters_ = true; @@ -77,10 +114,11 @@ void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context, return; params->others.bls_config.enable_auto = 0; - params->others.bls_config.fixed_val.r = blackLevelRed_; - params->others.bls_config.fixed_val.gr = blackLevelGreenR_; - params->others.bls_config.fixed_val.gb = blackLevelGreenB_; - params->others.bls_config.fixed_val.b = blackLevelBlue_; + /* The rkisp1 uses 12bit based black levels. Scale down accordingly. */ + params->others.bls_config.fixed_val.r = blackLevelRed_ << 4; + params->others.bls_config.fixed_val.gr = blackLevelGreenR_ << 4; + params->others.bls_config.fixed_val.gb = blackLevelGreenB_ << 4; + params->others.bls_config.fixed_val.b = blackLevelBlue_ << 4; params->module_en_update |= RKISP1_CIF_ISP_MODULE_BLS; params->module_ens |= RKISP1_CIF_ISP_MODULE_BLS; From patchwork Wed Jul 3 10:39:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20504 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 215A2BEFBE for ; Wed, 3 Jul 2024 10:40:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 711DE63334; Wed, 3 Jul 2024 12:40:27 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DhBqgwJe"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 36E8462E25 for ; Wed, 3 Jul 2024 12:40:23 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 27C116D6; Wed, 3 Jul 2024 12:39:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720003195; bh=XVpE0gqzyfuvdVCeycPDg3ZtgWWWa1emzt9Ylz3TJzI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DhBqgwJetlX9noUaUW5P/xPySApepe/PZjK9USJUif0JS+8RmE3GY381PqhdIiBN1 8GXsbIqCO/20tNYZY9CHJOBf2sDjr9vDCBQndgFrcl/i3Xhq7OJmkJQa/IbI/bkJJq Or75p8nUOEWjW73JkF9yAui459tklgKJAV+SVnWI= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 4/6] ipa: rkisp1: blc: Report sensor black levels in metadata Date: Wed, 3 Jul 2024 12:39:51 +0200 Message-ID: <20240703104004.184783-5-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703104004.184783-1-stefan.klug@ideasonboard.com> References: <20240703104004.184783-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add sensor black levels to the metadata of the rkisp1 pipeline. Additionally enable raw support for this algorithm and add it to uncalibrated.yaml, so that black levels get reported when capturing tuning images. This is a bit of a hack, because no actual black level correction is taking place in raw mode, but it is the easiest way to get blacklevel reported for raw streams. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/algorithms/blc.cpp | 28 +++++++++++++++++++++++++++ src/ipa/rkisp1/algorithms/blc.h | 5 ++++- src/ipa/rkisp1/data/uncalibrated.yaml | 1 + 3 files changed, 33 insertions(+), 1 deletion(-) diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index 87025e4f8c72..71c62b009707 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -9,6 +9,8 @@ #include +#include + #include "libcamera/internal/yaml_parser.h" /** @@ -38,6 +40,13 @@ LOG_DEFINE_CATEGORY(RkISP1Blc) BlackLevelCorrection::BlackLevelCorrection() : tuningParameters_(false) { + /* + * This is a bit of a hack. In raw mode no black level correction + * happens. This flag is used to ensure the metadata gets populated with + * the black level which is needed to capture proper raw images for + * tuning. + */ + supportsRaw_ = true; } /** @@ -107,6 +116,9 @@ void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { + if (context.configuration.raw) + return; + if (frame > 0) return; @@ -125,6 +137,22 @@ void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context, params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_BLS; } +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void BlackLevelCorrection::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] const rkisp1_stat_buffer *stats, + [[maybe_unused]] ControlList &metadata) +{ + metadata.set(controls::SensorBlackLevels, + { static_cast(blackLevelRed_), + static_cast(blackLevelGreenR_), + static_cast(blackLevelGreenB_), + static_cast(blackLevelBlue_) }); +} + REGISTER_IPA_ALGORITHM(BlackLevelCorrection, "BlackLevelCorrection") } /* namespace ipa::rkisp1::algorithms */ diff --git a/src/ipa/rkisp1/algorithms/blc.h b/src/ipa/rkisp1/algorithms/blc.h index 460ebcc15739..4ecac233f88b 100644 --- a/src/ipa/rkisp1/algorithms/blc.h +++ b/src/ipa/rkisp1/algorithms/blc.h @@ -23,7 +23,10 @@ public: void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, rkisp1_params_cfg *params) override; - + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const rkisp1_stat_buffer *stats, + ControlList &metadata) override; private: bool tuningParameters_; int16_t blackLevelRed_; diff --git a/src/ipa/rkisp1/data/uncalibrated.yaml b/src/ipa/rkisp1/data/uncalibrated.yaml index a7bbd8d84263..609012967e02 100644 --- a/src/ipa/rkisp1/data/uncalibrated.yaml +++ b/src/ipa/rkisp1/data/uncalibrated.yaml @@ -5,4 +5,5 @@ version: 1 algorithms: - Agc: - Awb: + - BlackLevelCorrection: ... From patchwork Wed Jul 3 10:39:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20505 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0DDF5BEFBE for ; Wed, 3 Jul 2024 10:40:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BB45F63352; Wed, 3 Jul 2024 12:40:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="lPyIH7SE"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7B4E962E27 for ; Wed, 3 Jul 2024 12:40:26 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6F25B3E6; Wed, 3 Jul 2024 12:39:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720003198; bh=X4rClZPn9FouOcVqNKttYH6S+sumwQKUduxEXHoiyd0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lPyIH7SEcdFAWXsCMFTshsN5wk+pGSWEXSZc4hbeMl7Zg8Fshdp6h1YlDbeTVl1Xn TPGUrmlxNyJo7A3/X6fWHgXD8B5HNP/UcAO5lYdP4pLVYyqg/lGx83MrOgfLVKYaZC HaZUjFmkgtW55/SzDZ1azevpzucGbPzufZK3aDEs= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 5/6] ipa: rkisp1: data: Update tuning files for imx219 and imx258 Date: Wed, 3 Jul 2024 12:39:52 +0200 Message-ID: <20240703104004.184783-6-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703104004.184783-1-stefan.klug@ideasonboard.com> References: <20240703104004.184783-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The black levels for imx219 and imx258 are now contained in the camera sensor helpers. Remove them from the tuning file for the imx219. Add a BLC entry to the imx258 tuning file. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/data/imx219.yaml | 4 ---- src/ipa/rkisp1/data/imx258.yaml | 1 + 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/src/ipa/rkisp1/data/imx219.yaml b/src/ipa/rkisp1/data/imx219.yaml index cbcc43b84ac7..0d99cb529392 100644 --- a/src/ipa/rkisp1/data/imx219.yaml +++ b/src/ipa/rkisp1/data/imx219.yaml @@ -6,10 +6,6 @@ algorithms: - Agc: - Awb: - BlackLevelCorrection: - R: 256 - Gr: 256 - Gb: 256 - B: 256 - LensShadingCorrection: x-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ] y-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ] diff --git a/src/ipa/rkisp1/data/imx258.yaml b/src/ipa/rkisp1/data/imx258.yaml index 43dddf20dcd2..202af36afbee 100644 --- a/src/ipa/rkisp1/data/imx258.yaml +++ b/src/ipa/rkisp1/data/imx258.yaml @@ -5,6 +5,7 @@ version: 1 algorithms: - Agc: - Awb: + - BlackLevelCorrection: - LensShadingCorrection: x-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ] y-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ] From patchwork Wed Jul 3 10:39:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20506 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 81FB9BEFBE for ; Wed, 3 Jul 2024 10:40:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EA326632EA; Wed, 3 Jul 2024 12:40:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="qtOyON4B"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8074E6333B for ; Wed, 3 Jul 2024 12:40:29 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5CA856D6; Wed, 3 Jul 2024 12:40:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720003201; bh=8mvjzSJjIYUSejtAaHxegbvpSTD5fdZSRvy1+AwL1l4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qtOyON4BArj81p2Aoi05y7yX5UncRIgCf0hS6sKMPJc9GM4J2wnqzxfZC/lmFQa8L E2KSGucpL8Pfkd4VK54NymM4GuDMlhAJA+X8SMmtA7w1LdduEZXz7g+qnhBLlYeim6 cgBfwZZRmFsLvGXhiVUfYMyzNpKppZjIos3NvTLU= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 6/6] ipa: rkisp1: Move ov4689 and ov5640 black levels into sensor helpers Date: Wed, 3 Jul 2024 12:39:53 +0200 Message-ID: <20240703104004.184783-7-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703104004.184783-1-stefan.klug@ideasonboard.com> References: <20240703104004.184783-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Move black levels for tuning files that contained a BLC block into the camera sensor helpers. ov4689.yaml had 66@12bit while the datasheet states 64@12bit. Use the value from the datasheet (scaled to 16bit). ov5640.yaml had 256@12bit while the datasheet states 16@10bit. Looking at the commit message the 256 most likely stems from the imx219 tuning file and 16@10bit is the same as the 64@12bit from the ov4689. This seems more likely and is therefore used. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Acked-by: Mikhail Rudenko --- src/ipa/libipa/camera_sensor_helper.cpp | 2 ++ src/ipa/rkisp1/data/ov4689.yaml | 4 ---- src/ipa/rkisp1/data/ov5640.yaml | 4 ---- 3 files changed, 2 insertions(+), 8 deletions(-) diff --git a/src/ipa/libipa/camera_sensor_helper.cpp b/src/ipa/libipa/camera_sensor_helper.cpp index 3d0e756927f0..982e35beaa90 100644 --- a/src/ipa/libipa/camera_sensor_helper.cpp +++ b/src/ipa/libipa/camera_sensor_helper.cpp @@ -542,6 +542,7 @@ class CameraSensorHelperOv4689 : public CameraSensorHelper public: CameraSensorHelperOv4689() { + blackLevel_ = 1024; gainType_ = AnalogueGainLinear; gainConstants_.linear = { 1, 0, 0, 128 }; } @@ -553,6 +554,7 @@ class CameraSensorHelperOv5640 : public CameraSensorHelper public: CameraSensorHelperOv5640() { + blackLevel_ = 1024; gainType_ = AnalogueGainLinear; gainConstants_.linear = { 1, 0, 0, 16 }; } diff --git a/src/ipa/rkisp1/data/ov4689.yaml b/src/ipa/rkisp1/data/ov4689.yaml index 2068684cafcd..609012967e02 100644 --- a/src/ipa/rkisp1/data/ov4689.yaml +++ b/src/ipa/rkisp1/data/ov4689.yaml @@ -6,8 +6,4 @@ algorithms: - Agc: - Awb: - BlackLevelCorrection: - R: 66 - Gr: 66 - Gb: 66 - B: 66 ... diff --git a/src/ipa/rkisp1/data/ov5640.yaml b/src/ipa/rkisp1/data/ov5640.yaml index 897b83cb435b..4b21d412e44e 100644 --- a/src/ipa/rkisp1/data/ov5640.yaml +++ b/src/ipa/rkisp1/data/ov5640.yaml @@ -6,10 +6,6 @@ algorithms: - Agc: - Awb: - BlackLevelCorrection: - R: 256 - Gr: 256 - Gb: 256 - B: 256 - ColorProcessing: - GammaSensorLinearization: x-intervals: [ 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2 ]