| Message ID | 20260909070146.18825-3-dmanresa@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series |
|
| Related | show |
Hi 2026. 09. 09. 9:01 keltezéssel, D. Manresa írta: > The Image Enhancement Filter Directed (IEFD) block of the ImgU combines > denoising with directional and unsharp-mask sharpening. It is the > sharpening block that the ImgU firmware binaries actually run: none of > them enables the Y_EE_NR edge enhancement accelerator, so programming that > block has no effect. When IEFD is not programmed the driver enables it with > a moderate default sharpening, and there is currently no way to change it. > > Add a Sharpness algorithm that programs the IEFD block with the driver > defaults and scales the sharpening parameters (unsharp mask amount, > directional sharpening weight and overshoot limits) by a sharpness > factor: 1.0 keeps the driver behaviour, 0.0 disables the sharpening while > keeping the denoising, and values up to 9.0 (where the unsharp mask amount > saturates) sharpen more. The default comes from the optional 'sharpness' > tuning parameter and can be changed at runtime with the Sharpness > control, which is also reported in the metadata. > > Enable the algorithm in the uncalibrated tuning file; with the default > value the output is unchanged. > > Measured on a Dell Latitude 7275 (OV5670, 1280x720 NV12, same scene): the > standard deviation of the Laplacian of Y is 2.24, 2.47, 2.81 and 3.43 for > sharpness 0.0, 1.0, 4.0 and 9.0, with the mean luminance and chroma > unchanged. > > Signed-off-by: D. Manresa <dmanresa@gmail.com> > --- > src/ipa/ipu3/algorithms/meson.build | 1 + > src/ipa/ipu3/algorithms/sharpness.cpp | 208 ++++++++++++++++++++++++++ > src/ipa/ipu3/algorithms/sharpness.h | 43 ++++++ > src/ipa/ipu3/data/uncalibrated.yaml | 1 + > src/ipa/ipu3/ipa_context.h | 9 ++ > 5 files changed, 262 insertions(+) > create mode 100644 src/ipa/ipu3/algorithms/sharpness.cpp > create mode 100644 src/ipa/ipu3/algorithms/sharpness.h > > diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build > index 833583e..fd97330 100644 > --- a/src/ipa/ipu3/algorithms/meson.build > +++ b/src/ipa/ipu3/algorithms/meson.build > @@ -8,5 +8,6 @@ ipu3_ipa_algorithms = files([ > 'ccm.cpp', > 'lsc.cpp', > 'saturation.cpp', > + 'sharpness.cpp', > 'tone_mapping.cpp', > ]) > diff --git a/src/ipa/ipu3/algorithms/sharpness.cpp b/src/ipa/ipu3/algorithms/sharpness.cpp > new file mode 100644 > index 0000000..da26351 > --- /dev/null > +++ b/src/ipa/ipu3/algorithms/sharpness.cpp > @@ -0,0 +1,208 @@ > +/* SPDX-License-Identifier: LGPL-2.1-or-later */ > +/* > + * Copyright (C) 2026, D. Manresa > + * > + * IPU3 Sharpness control > + */ > + > +#include "sharpness.h" > + > +#include <algorithm> > +#include <cmath> > + > +#include <libcamera/base/log.h> > + > +#include <libcamera/control_ids.h> > + > +#include "libcamera/internal/value_node.h" > + > +/** > + * \file sharpness.h > + */ > + > +namespace libcamera { > + > +namespace ipa::ipu3::algorithms { > + > +/** > + * \class Sharpness > + * \brief Control of the sharpening through the ImgU IEFD block > + * > + * The Image Enhancement Filter Directed (IEFD) block of the ImgU combines > + * denoising with a directional and an unsharp-mask sharpening. It is the > + * sharpening block that the ImgU firmware binaries actually run: the Y_EE_NR > + * edge enhancement block is never enabled by any of them, so programming it > + * has no effect. > + * > + * When the IEFD block is not programmed the ImgU driver enables it with a > + * moderate default sharpening. This algorithm programs the block with the > + * same default configuration and scales the sharpening strength, that is the > + * unsharp mask amount, the directional sharpening weight and the overshoot > + * limits, by the requested sharpness. A value of 1.0 reproduces the driver > + * defaults, 0.0 disables the sharpening while keeping the denoising active, > + * and larger values sharpen more. > + * > + * The default sharpness is taken from the optional \a sharpness tuning > + * parameter (1.0 if absent) and can be changed at runtime with the > + * controls::Sharpness control. > + */ > + > +LOG_DEFINE_CATEGORY(IPU3Sharpness) > + > +namespace { > + > +constexpr double kMinSharpness = 0.0; > +/* The unsharp mask amount saturates at 511, 9 times the default of 56. */ > +constexpr double kMaxSharpness = 9.0; I think it might be better to switch to `float` since the `ControlValue` itself can only deal with `float`, and I am also not sure that the extra precision of `double` is needed here. So it's probably simpler to keep everything `float`. > + > +/* clang-format off */ > +/* imgu_css_iefd_defaults from the ImgU driver (ipu3-tables.c), reserved fields as 0 */ > +static const struct ipu3_uapi_yuvp1_iefd_config kIefdDefaults = { Would it not be better to use designated initializers instead of commenting the names of fields? (Just like it was already in the kernel) You can also drop the `static` because it's in an anonymous namespace. > + /* units */ { > + /* cu_1 */ { 0, 150, 7, 0 }, > + /* cu_ed */ { 7, 110, 244, 0, 307, 409, 511, 0, > + 184, 255, 255, 0, 0, 0, 0, > + 7, 81, 255, 0, 255, 255, 0 }, > + /* cu_3 */ { 148, 251, 10, 0 }, > + /* cu_5 */ { 25, 70, 501, 0, 32, 0 }, > + /* cu_6 */ { 32, 63, 183, 0, 397, > + 33, 0, 0, 0, > + 0, 64, 0, 64, 0 }, > + /* cu_7 */ { 200, 303, 10, 0 }, > + /* cu_unsharp */ { 10, 64, 110, 0, 511, > + 66, 12, 0, 0, > + 0, 56, 0, 64, 0 }, > + /* cu_radial */ { 6, 203, 255, 255, 255, 255, 0, > + 84, 444, 397, 288, 300, 0, > + 4, 69, 207, 0, 369, 448, 0 }, > + /* cu_vssnlm */ { 61, 100, 25, 0 } > + }, > + /* config */ { 45, 0, 0, 0, 16, 0, 45, 0 }, > + /* control */ { 1, 1, 1, 1, 1, 0 }, > + /* sharp */ { { 50, 0, 511, 0, 50, 0, 50, 0 }, > + { 64, 0, 0, 0, 0, 0 }, > + { 56, 0, 56, 0 } }, > + /* unsharp */ { { 36, 17, 8, 0 }, > + { 13, 7, 3, 0 } }, > + /* rad */ { { -2104, 0, -1559, 0 }, > + { 4426816, 0 }, > + { 2430481, 0 }, > + { 6, 0, 79, 0 }, > + { 64, 0, 0, 0 }, > + { 1, 0, 2, 0, 0, 0, 0, 0 }, > + { 40, 0, 62, 0 } }, > + /* vsslnm */ { { 16, 32, 64, 0 }, > + { 1, 0, 2, 0, 8, 0 } }, > +}; > +/* clang-format on */ > + > +uint32_t scaleU32(uint32_t value, double factor, uint32_t max) > +{ > + long scaled = std::lround(value * factor); > + return std::clamp<long>(scaled, 0, max); > +} > + > +} /* namespace */ > + > +/** > + * \copydoc libcamera::ipa::Algorithm::init > + */ > +int Sharpness::init(IPAContext &context, const ValueNode &tuningData) > +{ > + defaultSharpness_ = std::clamp(tuningData["sharpness"].get<double>(1.0), > + kMinSharpness, kMaxSharpness); > + > + context.ctrlMap[&controls::Sharpness] = > + ControlInfo(static_cast<float>(kMinSharpness), > + static_cast<float>(kMaxSharpness), > + static_cast<float>(defaultSharpness_)); > + > + return 0; > +} > + > +/** > + * \copydoc libcamera::ipa::Algorithm::configure > + */ > +int Sharpness::configure(IPAContext &context, > + [[maybe_unused]] const IPAConfigInfo &configInfo) > +{ > + context.activeState.sharpness.value = defaultSharpness_; > + > + return 0; > +} > + > +/** > + * \copydoc libcamera::ipa::Algorithm::queueRequest > + */ > +void Sharpness::queueRequest(IPAContext &context, const uint32_t frame, > + IPAFrameContext &frameContext, > + const ControlList &controls) > +{ > + auto &sharpness = context.activeState.sharpness; > + > + frameContext.sharpness.update = frame == 0; > + > + const auto &value = controls.get(controls::Sharpness); > + if (value) { > + sharpness.value = std::clamp<double>(*value, kMinSharpness, > + kMaxSharpness); > + frameContext.sharpness.update = true; > + LOG(IPU3Sharpness, Debug) << "Set sharpness to " << sharpness.value; > + } > + > + frameContext.sharpness.value = sharpness.value; > +} > + > +/** > + * \copydoc libcamera::ipa::Algorithm::prepare > + * > + * Program the IEFD block with the ImgU driver defaults, scaling the > + * sharpening parameters by the sharpness of the frame. > + */ > +void Sharpness::prepare([[maybe_unused]] IPAContext &context, > + [[maybe_unused]] const uint32_t frame, > + IPAFrameContext &frameContext, > + ipu3_uapi_params *params) > +{ > + if (!frameContext.sharpness.update) > + return; > + > + const double sharpness = frameContext.sharpness.value; > + struct ipu3_uapi_yuvp1_iefd_config &iefd = params->acc_param.iefd; > + > + iefd = kIefdDefaults; > + > + struct ipu3_uapi_yuvp1_iefd_shrp_cfg &sharp = iefd.sharp; > + sharp.unshrp_cfg.unsharp_amount = > + scaleU32(kIefdDefaults.sharp.unshrp_cfg.unsharp_amount, sharpness, 511); > + sharp.far_w.dir_shrp = > + scaleU32(kIefdDefaults.sharp.far_w.dir_shrp, sharpness, 64); > + sharp.cfg.nega_lmt_txt = > + scaleU32(kIefdDefaults.sharp.cfg.nega_lmt_txt, sharpness, 8191); > + sharp.cfg.posi_lmt_txt = > + scaleU32(kIefdDefaults.sharp.cfg.posi_lmt_txt, sharpness, 8191); > + sharp.cfg.nega_lmt_dir = > + scaleU32(kIefdDefaults.sharp.cfg.nega_lmt_dir, sharpness, 8191); > + sharp.cfg.posi_lmt_dir = > + scaleU32(kIefdDefaults.sharp.cfg.posi_lmt_dir, sharpness, 8191); Have you considered using the `UQ` type (alias) from `libipa/quantized.h`? E.g. sharp.cfg.posi_lmt_dir = UQ<13, 0>(sharpness * kIefdDefaults.sharp.cfg.posi_lmt_dir).quantized(); > + > + params->use.acc_iefd = 1; > +} > + > +/** > + * \copydoc libcamera::ipa::Algorithm::process > + */ > +void Sharpness::process([[maybe_unused]] IPAContext &context, > + [[maybe_unused]] const uint32_t frame, > + IPAFrameContext &frameContext, > + [[maybe_unused]] const ipu3_uapi_stats_3a *stats, > + ControlList &metadata) > +{ > + metadata.set(controls::Sharpness, frameContext.sharpness.value); > +} > + > +REGISTER_IPA_ALGORITHM(Sharpness, "Sharpness") > + > +} /* namespace ipa::ipu3::algorithms */ > + > +} /* namespace libcamera */ > [...]
diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build index 833583e..fd97330 100644 --- a/src/ipa/ipu3/algorithms/meson.build +++ b/src/ipa/ipu3/algorithms/meson.build @@ -8,5 +8,6 @@ ipu3_ipa_algorithms = files([ 'ccm.cpp', 'lsc.cpp', 'saturation.cpp', + 'sharpness.cpp', 'tone_mapping.cpp', ]) diff --git a/src/ipa/ipu3/algorithms/sharpness.cpp b/src/ipa/ipu3/algorithms/sharpness.cpp new file mode 100644 index 0000000..da26351 --- /dev/null +++ b/src/ipa/ipu3/algorithms/sharpness.cpp @@ -0,0 +1,208 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, D. Manresa + * + * IPU3 Sharpness control + */ + +#include "sharpness.h" + +#include <algorithm> +#include <cmath> + +#include <libcamera/base/log.h> + +#include <libcamera/control_ids.h> + +#include "libcamera/internal/value_node.h" + +/** + * \file sharpness.h + */ + +namespace libcamera { + +namespace ipa::ipu3::algorithms { + +/** + * \class Sharpness + * \brief Control of the sharpening through the ImgU IEFD block + * + * The Image Enhancement Filter Directed (IEFD) block of the ImgU combines + * denoising with a directional and an unsharp-mask sharpening. It is the + * sharpening block that the ImgU firmware binaries actually run: the Y_EE_NR + * edge enhancement block is never enabled by any of them, so programming it + * has no effect. + * + * When the IEFD block is not programmed the ImgU driver enables it with a + * moderate default sharpening. This algorithm programs the block with the + * same default configuration and scales the sharpening strength, that is the + * unsharp mask amount, the directional sharpening weight and the overshoot + * limits, by the requested sharpness. A value of 1.0 reproduces the driver + * defaults, 0.0 disables the sharpening while keeping the denoising active, + * and larger values sharpen more. + * + * The default sharpness is taken from the optional \a sharpness tuning + * parameter (1.0 if absent) and can be changed at runtime with the + * controls::Sharpness control. + */ + +LOG_DEFINE_CATEGORY(IPU3Sharpness) + +namespace { + +constexpr double kMinSharpness = 0.0; +/* The unsharp mask amount saturates at 511, 9 times the default of 56. */ +constexpr double kMaxSharpness = 9.0; + +/* clang-format off */ +/* imgu_css_iefd_defaults from the ImgU driver (ipu3-tables.c), reserved fields as 0 */ +static const struct ipu3_uapi_yuvp1_iefd_config kIefdDefaults = { + /* units */ { + /* cu_1 */ { 0, 150, 7, 0 }, + /* cu_ed */ { 7, 110, 244, 0, 307, 409, 511, 0, + 184, 255, 255, 0, 0, 0, 0, + 7, 81, 255, 0, 255, 255, 0 }, + /* cu_3 */ { 148, 251, 10, 0 }, + /* cu_5 */ { 25, 70, 501, 0, 32, 0 }, + /* cu_6 */ { 32, 63, 183, 0, 397, + 33, 0, 0, 0, + 0, 64, 0, 64, 0 }, + /* cu_7 */ { 200, 303, 10, 0 }, + /* cu_unsharp */ { 10, 64, 110, 0, 511, + 66, 12, 0, 0, + 0, 56, 0, 64, 0 }, + /* cu_radial */ { 6, 203, 255, 255, 255, 255, 0, + 84, 444, 397, 288, 300, 0, + 4, 69, 207, 0, 369, 448, 0 }, + /* cu_vssnlm */ { 61, 100, 25, 0 } + }, + /* config */ { 45, 0, 0, 0, 16, 0, 45, 0 }, + /* control */ { 1, 1, 1, 1, 1, 0 }, + /* sharp */ { { 50, 0, 511, 0, 50, 0, 50, 0 }, + { 64, 0, 0, 0, 0, 0 }, + { 56, 0, 56, 0 } }, + /* unsharp */ { { 36, 17, 8, 0 }, + { 13, 7, 3, 0 } }, + /* rad */ { { -2104, 0, -1559, 0 }, + { 4426816, 0 }, + { 2430481, 0 }, + { 6, 0, 79, 0 }, + { 64, 0, 0, 0 }, + { 1, 0, 2, 0, 0, 0, 0, 0 }, + { 40, 0, 62, 0 } }, + /* vsslnm */ { { 16, 32, 64, 0 }, + { 1, 0, 2, 0, 8, 0 } }, +}; +/* clang-format on */ + +uint32_t scaleU32(uint32_t value, double factor, uint32_t max) +{ + long scaled = std::lround(value * factor); + return std::clamp<long>(scaled, 0, max); +} + +} /* namespace */ + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Sharpness::init(IPAContext &context, const ValueNode &tuningData) +{ + defaultSharpness_ = std::clamp(tuningData["sharpness"].get<double>(1.0), + kMinSharpness, kMaxSharpness); + + context.ctrlMap[&controls::Sharpness] = + ControlInfo(static_cast<float>(kMinSharpness), + static_cast<float>(kMaxSharpness), + static_cast<float>(defaultSharpness_)); + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Sharpness::configure(IPAContext &context, + [[maybe_unused]] const IPAConfigInfo &configInfo) +{ + context.activeState.sharpness.value = defaultSharpness_; + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Sharpness::queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + auto &sharpness = context.activeState.sharpness; + + frameContext.sharpness.update = frame == 0; + + const auto &value = controls.get(controls::Sharpness); + if (value) { + sharpness.value = std::clamp<double>(*value, kMinSharpness, + kMaxSharpness); + frameContext.sharpness.update = true; + LOG(IPU3Sharpness, Debug) << "Set sharpness to " << sharpness.value; + } + + frameContext.sharpness.value = sharpness.value; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + * + * Program the IEFD block with the ImgU driver defaults, scaling the + * sharpening parameters by the sharpness of the frame. + */ +void Sharpness::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + ipu3_uapi_params *params) +{ + if (!frameContext.sharpness.update) + return; + + const double sharpness = frameContext.sharpness.value; + struct ipu3_uapi_yuvp1_iefd_config &iefd = params->acc_param.iefd; + + iefd = kIefdDefaults; + + struct ipu3_uapi_yuvp1_iefd_shrp_cfg &sharp = iefd.sharp; + sharp.unshrp_cfg.unsharp_amount = + scaleU32(kIefdDefaults.sharp.unshrp_cfg.unsharp_amount, sharpness, 511); + sharp.far_w.dir_shrp = + scaleU32(kIefdDefaults.sharp.far_w.dir_shrp, sharpness, 64); + sharp.cfg.nega_lmt_txt = + scaleU32(kIefdDefaults.sharp.cfg.nega_lmt_txt, sharpness, 8191); + sharp.cfg.posi_lmt_txt = + scaleU32(kIefdDefaults.sharp.cfg.posi_lmt_txt, sharpness, 8191); + sharp.cfg.nega_lmt_dir = + scaleU32(kIefdDefaults.sharp.cfg.nega_lmt_dir, sharpness, 8191); + sharp.cfg.posi_lmt_dir = + scaleU32(kIefdDefaults.sharp.cfg.posi_lmt_dir, sharpness, 8191); + + params->use.acc_iefd = 1; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Sharpness::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const ipu3_uapi_stats_3a *stats, + ControlList &metadata) +{ + metadata.set(controls::Sharpness, frameContext.sharpness.value); +} + +REGISTER_IPA_ALGORITHM(Sharpness, "Sharpness") + +} /* namespace ipa::ipu3::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/ipu3/algorithms/sharpness.h b/src/ipa/ipu3/algorithms/sharpness.h new file mode 100644 index 0000000..900fbf7 --- /dev/null +++ b/src/ipa/ipu3/algorithms/sharpness.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, D. Manresa + * + * IPU3 Sharpness control + */ + +#pragma once + +#include <linux/intel-ipu3.h> + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::ipu3::algorithms { + +class Sharpness : public Algorithm +{ +public: + Sharpness() = default; + ~Sharpness() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + ipu3_uapi_params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ipu3_uapi_stats_3a *stats, + ControlList &metadata) override; + +private: + double defaultSharpness_ = 1.0; +}; + +} /* namespace ipa::ipu3::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/ipu3/data/uncalibrated.yaml b/src/ipa/ipu3/data/uncalibrated.yaml index b7ab0d7..930d4ed 100644 --- a/src/ipa/ipu3/data/uncalibrated.yaml +++ b/src/ipa/ipu3/data/uncalibrated.yaml @@ -10,4 +10,5 @@ algorithms: - Ccm: - ToneMapping: - Saturation: + - Sharpness: ... diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 76147e3..860655e 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -59,6 +59,10 @@ struct IPAActiveState { struct { double value; } saturation; + + struct { + double value; + } sharpness; }; struct IPAFrameContext : public FrameContext { @@ -77,6 +81,11 @@ struct IPAFrameContext : public FrameContext { double value; bool update; } saturation; + + struct { + double value; + bool update; + } sharpness; }; struct IPAContext {
The Image Enhancement Filter Directed (IEFD) block of the ImgU combines denoising with directional and unsharp-mask sharpening. It is the sharpening block that the ImgU firmware binaries actually run: none of them enables the Y_EE_NR edge enhancement accelerator, so programming that block has no effect. When IEFD is not programmed the driver enables it with a moderate default sharpening, and there is currently no way to change it. Add a Sharpness algorithm that programs the IEFD block with the driver defaults and scales the sharpening parameters (unsharp mask amount, directional sharpening weight and overshoot limits) by a sharpness factor: 1.0 keeps the driver behaviour, 0.0 disables the sharpening while keeping the denoising, and values up to 9.0 (where the unsharp mask amount saturates) sharpen more. The default comes from the optional 'sharpness' tuning parameter and can be changed at runtime with the Sharpness control, which is also reported in the metadata. Enable the algorithm in the uncalibrated tuning file; with the default value the output is unchanged. Measured on a Dell Latitude 7275 (OV5670, 1280x720 NV12, same scene): the standard deviation of the Laplacian of Y is 2.24, 2.47, 2.81 and 3.43 for sharpness 0.0, 1.0, 4.0 and 9.0, with the mean luminance and chroma unchanged. Signed-off-by: D. Manresa <dmanresa@gmail.com> --- src/ipa/ipu3/algorithms/meson.build | 1 + src/ipa/ipu3/algorithms/sharpness.cpp | 208 ++++++++++++++++++++++++++ src/ipa/ipu3/algorithms/sharpness.h | 43 ++++++ src/ipa/ipu3/data/uncalibrated.yaml | 1 + src/ipa/ipu3/ipa_context.h | 9 ++ 5 files changed, 262 insertions(+) create mode 100644 src/ipa/ipu3/algorithms/sharpness.cpp create mode 100644 src/ipa/ipu3/algorithms/sharpness.h