From patchwork Tue Jul 26 14:32:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florian Sylvestre X-Patchwork-Id: 16793 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A7911BE173 for ; Tue, 26 Jul 2022 14:32:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EA8AA63316; Tue, 26 Jul 2022 16:32:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658845941; bh=B15tR4exQ5ad+DVRVzVKa2zOUP8utzDvdEnXK3P6wXs=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=BQRiefDl0sdP8CEWIcP2qSYaAzG2n7UiE3heu7i3am8WiAJXp8Ycuz76ZhCV0dm2M UTSSdDLUbiC289bqYHswgK1/SHdZzQjXy9dYjxWUW9v+C3492zd+pJsVR8hDpGpbUa sqvVI1u9xY6GJxlXHXuAFm6WYvncGeN/nziecm3nrw+q//x6OFwGMjkEJ+pc+fG4q/ ls0IZVI72z3wiVsNi8K9VpoYoa1YE4PfDssItq4ollK0krBtBt2bUiJi3EPQVgfQcP JCYs0vJ6YQpna62JZSwxy+uRoEsbY8GpJcUA9L3u9TZenad3+a5E92tEk5avuKvQeH 16PEa/vh7gSeA== Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 776086330E for ; Tue, 26 Jul 2022 16:32:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20210112.gappssmtp.com header.i=@baylibre-com.20210112.gappssmtp.com header.b="XyAYUawy"; dkim-atps=neutral Received: by mail-wr1-x42b.google.com with SMTP id q18so10000807wrx.8 for ; Tue, 26 Jul 2022 07:32:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uM93IPim+okWklce49IUeMeEqvQD4hnz2VfGxJMf+jM=; b=XyAYUawyszE0UWWirnM8Nhn1ksWEe7wviKrLL3ntAQzbUA64NpReomarxio9eNWoFr Lw/soZjbyiZ6CD5+dE6p+8usZFDak0o5QENeI77Y00XsFFmxxLZAnbXhbHdgarwGhKE2 LueiKUAHA/Oyvu1IZpLSxkonSG4ngKB7PthhnQGUMkykPPb5sXBewmlq5wIzzk5GUzH6 yOQW1KYpXNeGaXCm2IiQ/E4LbCXzb2OVKAXj/lO96B12MUqNbK6lEnkRyCNtmCqxc2iD I0kHiHwR6TiMi4J0qTFpWZfjaJuAv0MQs5X9FR0B65VjvDzniZyn42jwFTAykP9Rly+s xkTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uM93IPim+okWklce49IUeMeEqvQD4hnz2VfGxJMf+jM=; b=LGj548gJP5SmVBbgQ2OthVLpZ0tiJQwuv+2eCOt+Xrj4jThkwhTNDTraHg8MwK5nXp zcOnhI426lt6W9Bh7+OhMz1rsiosOtarBRnruukqTwFbbTKf2VjmzvzVu1Qhh5KjVlAX eS+MC3al02jJptoVL7wxhkzuBiv2+aE5opFqfVRf/ykAIyH8xw0/796N9L8QnA7v7zPF mDNTL81vzZ6+WF9XDyQ7nJ5jFuOYjvwXmnyFX14hwR6dIr6UkRMmP0N9hARTvS97kM4W 4tES6cMh2LsZ5vpa0Ff3PmLkqv6pOcV2jPaZ8BRuAZ1SvC/potQKFGdnhNXiCG+gWGm6 u+Tw== X-Gm-Message-State: AJIora+jUitW4Siqq/FihfkD6kFIm/QxxrYstdZa1AlPFcPfLNKzkMck 3SlPWNKMcfpYf4rMKpxkzRH/3xFb3piWLFP0 X-Google-Smtp-Source: AGRyM1s2AJRfT67CNlf+ZyhrFpqg6ecoRfp/v4po4bRTbiifq+NNIAuA/aX0ZR4eJ0SBF5U0d69Mmg== X-Received: by 2002:a05:6000:1c7:b0:21e:68b9:c047 with SMTP id t7-20020a05600001c700b0021e68b9c047mr11729122wrx.597.1658845937685; Tue, 26 Jul 2022 07:32:17 -0700 (PDT) Received: from BL087.. ([2a01:e34:eea9:e630:c7b2:2183:8a06:b4dc]) by smtp.gmail.com with ESMTPSA id k1-20020adff5c1000000b0020fff0ea0a3sm14377177wrp.116.2022.07.26.07.32.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jul 2022 07:32:17 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Jul 2022 16:32:10 +0200 Message-Id: <20220726143211.517231-2-fsylvestre@baylibre.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220726143211.517231-1-fsylvestre@baylibre.com> References: <20220726143211.517231-1-fsylvestre@baylibre.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/2] ipa: rkisp1: Add support of Filter control X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Florian Sylvestre via libcamera-devel From: Florian Sylvestre Reply-To: Florian Sylvestre Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Denoise and Sharpness filters will be applied by RKISP1 during the demosaicing step. The denoise filter is responsible to remove noise from the image, while the sharpness filter will enhance its acutance. Add filter algorithm with denoise and sharpness values based on user controls. Signed-off-by: Florian Sylvestre Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/algorithms/filter.cpp | 201 +++++++++++++++++++++++ src/ipa/rkisp1/algorithms/filter.h | 30 ++++ src/ipa/rkisp1/algorithms/meson.build | 1 + src/ipa/rkisp1/data/ov5640.yaml | 1 + src/ipa/rkisp1/ipa_context.cpp | 14 ++ src/ipa/rkisp1/ipa_context.h | 6 + src/libcamera/pipeline/rkisp1/rkisp1.cpp | 8 + 7 files changed, 261 insertions(+) create mode 100644 src/ipa/rkisp1/algorithms/filter.cpp create mode 100644 src/ipa/rkisp1/algorithms/filter.h diff --git a/src/ipa/rkisp1/algorithms/filter.cpp b/src/ipa/rkisp1/algorithms/filter.cpp new file mode 100644 index 00000000..3a9f9be5 --- /dev/null +++ b/src/ipa/rkisp1/algorithms/filter.cpp @@ -0,0 +1,201 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2022, Ideas On Board + * + * filter.cpp - RkISP1 Filter control + */ + +#include "filter.h" + +#include + +#include + +#include + +/** + * \file filter.h + */ + +namespace libcamera { + +namespace ipa::rkisp1::algorithms { + +/** + * \class Filter + * \brief RkISP1 Filter control + * + * Denoise and Sharpness filters will be applied by RKISP1 during the + * demosaicing step. The denoise filter is responsible to remove noise from the + * image, while the sharpness filter will enhance its acutance. + * + * \todo In current version the denoise and sharpness control is based on user + * controls. In a future version it should be controlled automatically by the + * algorithm. + */ + +LOG_DEFINE_CATEGORY(RkISP1Filter) + +static constexpr uint32_t kFiltLumWeightDefault = 0x00022040; +static constexpr uint32_t kFiltModeDefault = 0x000004f2; + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Filter::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + const ControlList &controls) +{ + auto &filter = context.frameContext.filter; + + const auto &sharpness = controls.get(controls::Sharpness); + if (sharpness) { + filter.sharpness = std::round(std::clamp(*sharpness, 0.0f, 10.0f)); + filter.updateParams = true; + + LOG(RkISP1Filter, Debug) << "Set sharpness to " << *sharpness; + } + + const auto &denoise = controls.get(controls::draft::NoiseReductionMode); + if (denoise) { + LOG(RkISP1Filter, Debug) << "Set denoise to " << *denoise; + + switch (*denoise) { + case controls::draft::NoiseReductionModeOff: + filter.denoise = 0; + filter.updateParams = true; + break; + case controls::draft::NoiseReductionModeMinimal: + filter.denoise = 1; + filter.updateParams = true; + break; + case controls::draft::NoiseReductionModeHighQuality: + case controls::draft::NoiseReductionModeFast: + filter.denoise = 3; + filter.updateParams = true; + break; + default: + LOG(RkISP1Filter, Error) + << "Unsupported denoise value " + << *denoise; + } + } +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Filter::prepare(IPAContext &context, + rkisp1_params_cfg *params) +{ + /* Check if the algorithm configuration has been updated. */ + if (!context.frameContext.filter.updateParams) + return; + + context.frameContext.filter.updateParams = false; + + static constexpr uint16_t filt_fac_sh0[] = { + 0x04, 0x07, 0x0a, 0x0c, 0x10, 0x14, 0x1a, 0x1e, 0x24, 0x2a, 0x30 + }; + + static constexpr uint16_t filt_fac_sh1[] = { + 0x04, 0x08, 0x0c, 0x10, 0x16, 0x1b, 0x20, 0x26, 0x2c, 0x30, 0x3f + }; + + static constexpr uint16_t filt_fac_mid[] = { + 0x04, 0x06, 0x08, 0x0a, 0x0c, 0x10, 0x13, 0x17, 0x1d, 0x22, 0x28 + }; + + static constexpr uint16_t filt_fac_bl0[] = { + 0x02, 0x02, 0x04, 0x06, 0x08, 0x0a, 0x0c, 0x10, 0x15, 0x1a, 0x24 + }; + + static constexpr uint16_t filt_fac_bl1[] = { + 0x00, 0x00, 0x00, 0x02, 0x04, 0x04, 0x06, 0x08, 0x0d, 0x14, 0x20 + }; + + static constexpr uint16_t filt_thresh_sh0[] = { + 0, 18, 26, 36, 41, 75, 90, 120, 170, 250, 1023 + }; + + static constexpr uint16_t filt_thresh_sh1[] = { + 0, 33, 44, 51, 67, 100, 120, 150, 200, 300, 1023 + }; + + static constexpr uint16_t filt_thresh_bl0[] = { + 0, 8, 13, 23, 26, 50, 60, 80, 140, 180, 1023 + }; + + static constexpr uint16_t filt_thresh_bl1[] = { + 0, 2, 5, 10, 15, 20, 26, 51, 100, 150, 1023 + }; + + static constexpr uint16_t stage1_select[] = { + 6, 6, 4, 4, 3, 3, 2, 2, 2, 1, 0 + }; + + static constexpr uint16_t filt_chr_v_mode[] = { + 1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3 + }; + + static constexpr uint16_t filt_chr_h_mode[] = { + 0, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3 + }; + + uint8_t denoise = context.frameContext.filter.denoise; + uint8_t sharpness = context.frameContext.filter.sharpness; + auto &flt_config = params->others.flt_config; + + flt_config.fac_sh0 = filt_fac_sh0[sharpness]; + flt_config.fac_sh1 = filt_fac_sh1[sharpness]; + flt_config.fac_mid = filt_fac_mid[sharpness]; + flt_config.fac_bl0 = filt_fac_bl0[sharpness]; + flt_config.fac_bl1 = filt_fac_bl1[sharpness]; + + flt_config.lum_weight = kFiltLumWeightDefault; + flt_config.mode = kFiltModeDefault; + flt_config.thresh_sh0 = filt_thresh_sh0[denoise]; + flt_config.thresh_sh1 = filt_thresh_sh1[denoise]; + flt_config.thresh_bl0 = filt_thresh_bl0[denoise]; + flt_config.thresh_bl1 = filt_thresh_bl1[denoise]; + flt_config.grn_stage1 = stage1_select[denoise]; + flt_config.chr_v_mode = filt_chr_v_mode[denoise]; + flt_config.chr_h_mode = filt_chr_h_mode[denoise]; + + /* + * Combined high denoising and high sharpening requires some + * adjustments to the configuration of the filters. A first stage + * filter with a lower strength must be selected, and the blur factors + * must be decreased. + */ + if (denoise == 9) { + if (sharpness > 3) + flt_config.grn_stage1 = 2; + } else if (denoise == 10) { + if (sharpness > 5) + flt_config.grn_stage1 = 2; + else if (sharpness > 3) + flt_config.grn_stage1 = 1; + } + + if (denoise > 7) { + if (sharpness > 7) { + flt_config.fac_bl0 /= 2; + flt_config.fac_bl1 /= 4; + } else if (sharpness > 4) { + flt_config.fac_bl0 = + flt_config.fac_bl0 * 3 / 4; + flt_config.fac_bl1 /= 2; + } + } + + params->module_en_update |= RKISP1_CIF_ISP_MODULE_FLT; + params->module_ens |= RKISP1_CIF_ISP_MODULE_FLT; + params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_FLT; +} + +REGISTER_IPA_ALGORITHM(Filter, "Filter") + +} /* namespace ipa::rkisp1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/filter.h b/src/ipa/rkisp1/algorithms/filter.h new file mode 100644 index 00000000..9eb170eb --- /dev/null +++ b/src/ipa/rkisp1/algorithms/filter.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2022, Ideas On Board + * + * filter.h - RkISP1 Filter control + */ + +#pragma once + +#include + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp1::algorithms { + +class Filter : public Algorithm +{ +public: + Filter() = default; + ~Filter() = default; + + void queueRequest(IPAContext &context, const uint32_t frame, + const ControlList &controls) override; + void prepare(IPAContext &context, rkisp1_params_cfg *params) override; +}; + +} /* namespace ipa::rkisp1::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/meson.build b/src/ipa/rkisp1/algorithms/meson.build index 87007493..dcd24fe0 100644 --- a/src/ipa/rkisp1/algorithms/meson.build +++ b/src/ipa/rkisp1/algorithms/meson.build @@ -5,6 +5,7 @@ rkisp1_ipa_algorithms = files([ 'awb.cpp', 'blc.cpp', 'dpcc.cpp', + 'filter.cpp', 'gsl.cpp', 'lsc.cpp', ]) diff --git a/src/ipa/rkisp1/data/ov5640.yaml b/src/ipa/rkisp1/data/ov5640.yaml index 2315ec43..99529481 100644 --- a/src/ipa/rkisp1/data/ov5640.yaml +++ b/src/ipa/rkisp1/data/ov5640.yaml @@ -155,4 +155,5 @@ algorithms: rnd-offsets: green: 2 red-blue: 2 + - Filter: ... diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 30bb87a9..4b117186 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -136,6 +136,20 @@ namespace libcamera::ipa::rkisp1 { * \brief Estimated color temperature */ +/** + * \var IPAFrameContext::filter + * \brief Context for the Filter algorithm + * + * \struct IPAFrameContext::filter.denoise + * \brief Denoising level + * + * \var IPAFrameContext::filter.sharpness + * \brief Sharpness level + * + * \var IPAFrameContext::filter.updateParams + * \brief Indicates if ISP parameters need to be updated + */ + /** * \var IPAFrameContext::sensor * \brief Effective sensor values diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 3bfb262c..3b2f6af1 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -57,6 +57,12 @@ struct IPAFrameContext { double temperatureK; } awb; + struct { + uint8_t denoise; + uint8_t sharpness; + bool updateParams; + } filter; + struct { uint32_t exposure; double gain; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 99eecd44..4e000d31 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -968,6 +968,14 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) hasSelfPath_ ? &selfPath_ : nullptr); ControlInfoMap::Map ctrls; + ctrls.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::Sharpness), + std::forward_as_tuple(0.0f, 10.0f, 1.0f)); + + ctrls.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::draft::NoiseReductionMode), + std::forward_as_tuple(controls::draft::NoiseReductionModeValues)); + ctrls.emplace(std::piecewise_construct, std::forward_as_tuple(&controls::AeEnable), std::forward_as_tuple(false, true)); From patchwork Tue Jul 26 14:32:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florian Sylvestre X-Patchwork-Id: 16794 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6BD98BE173 for ; Tue, 26 Jul 2022 14:32:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2978763312; Tue, 26 Jul 2022 16:32:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658845944; bh=/IDrj4FnlSkQA+RiWoZF4++na77Ygvs8pRcPMkacvDA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=wQ4YE6QEVtn55u5F9KT6AqytsuI4gLJHvuvdmS49Fwwd7nfKau3F76gUvz+YkQk9c wB2MsNTlCWXWqRv7ZWtxXVQHxV/Rc/5BbBQoEEyx0T4OoW8OXmdgADyssUvBdcg/6y afvL6M6HewFFZlsiUBLtzyzkyqJgmd5Yacf53nZC1IlkmoRvmF2r3NCPMectbIUQrf I/otC0K/vbqgv9bT98knxsHXSKbagPCmCV0eoqbOSAE9k0XFHVLJ9qQ9xUzMDPgicL Sq/tIi/AygEnlb4TVydY75WYInJTY1IxC5+udM3NJZoVe72/PqxjlcBrlGydRcme31 02k9DPZlla02A== Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C6E9263312 for ; Tue, 26 Jul 2022 16:32:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20210112.gappssmtp.com header.i=@baylibre-com.20210112.gappssmtp.com header.b="4JxytryS"; dkim-atps=neutral Received: by mail-wr1-x429.google.com with SMTP id k12so2695992wrm.13 for ; Tue, 26 Jul 2022 07:32:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=84D5xlQPO5UtzupAmm+41xsn6Ktd/RM5CJ/0P4SstzU=; b=4JxytrySKFRKoLHpkH4BIAkpQtM9v9SyFDu+Pqb3dWiOX1BjX0p6Snujoy7maD7+lM 8fwtm4LERKlJZwRi1yRCSO354pP1lST10cJVYl/T398l/yC952jJtKCsJUQHyRzngBga jFsuYZ+S+x1ApAsu8H2A74yaZpHCQ5sZUgAAmVWn52kFOup7FPdF0v3i8a7oI4xegaRL SjcLN4Lz6st0tM5L9CPuhTPNXD2shg0QK1hFAWliEyYfraGxjEFYfciIu5V2PeM4/Kkq A9V85Z7O/ZzQEJT3VG4rtUn0+ebtTZes1GfDMXELJftj6Ms/5tLJclaSH3zBV1WdljMQ GFSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=84D5xlQPO5UtzupAmm+41xsn6Ktd/RM5CJ/0P4SstzU=; b=Vf7Fl2vH6P4eVAPFaYK7HT3GTxHsDUrt0xNjPoQuBg/9/3QfiElHucPvdoZoZaG67p sqkzevgO6SdIb4Mz5IV6gUmCRZ3bRjDCblbwL+Gl/5fXp34KtxqKL8kL7J1Ln27/sGOd /TXNrgapxIMVIwnDNdgSMltXpuzwrJDtQ44S5RTsJ6HXRDFWcEaj0cc8lZzQtb+V3SDN 70JUhKA7YQfp1UvJ3FAP8QZ7onHzeThNOW9p6BFqBCgGLK20WXfuddr4s8D4ag2CNNGj KCST+Mk2hlEHYJZD4NWKQDFzPmJgDlm6+ylJ2hL9xLBOHJbzVDIBry56hkCyySZw7vSm iIrQ== X-Gm-Message-State: AJIora9kafWODT3XfEDSukHugvJ+R2yljJznL4yXzxDNE1SpKrCrZqp5 ivUlDfMQIqwNC3/CK8yHepzmkIC3315BSXmQ X-Google-Smtp-Source: AGRyM1sEWApbsAwD+NfoG4e+rFAGA29C+/CCCWg3f8X/ZY+N2/X2svo43H6L+2ZCLOif5G6i6L4A4Q== X-Received: by 2002:adf:e282:0:b0:21e:5546:5a26 with SMTP id v2-20020adfe282000000b0021e55465a26mr11432363wri.77.1658845938896; Tue, 26 Jul 2022 07:32:18 -0700 (PDT) Received: from BL087.. ([2a01:e34:eea9:e630:c7b2:2183:8a06:b4dc]) by smtp.gmail.com with ESMTPSA id k1-20020adff5c1000000b0020fff0ea0a3sm14377177wrp.116.2022.07.26.07.32.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jul 2022 07:32:18 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Jul 2022 16:32:11 +0200 Message-Id: <20220726143211.517231-3-fsylvestre@baylibre.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220726143211.517231-1-fsylvestre@baylibre.com> References: <20220726143211.517231-1-fsylvestre@baylibre.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/2] ipa: rkisp1: Add support of ColorProcessing control X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Florian Sylvestre via libcamera-devel From: Florian Sylvestre Reply-To: Florian Sylvestre Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add ColorProcessing algorithm that is in charge to manage brightness, contrast and saturation controls. These controls are currently based on user controls. Signed-off-by: Florian Sylvestre Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/algorithms/cproc.cpp | 97 ++++++++++++++++++++++++ src/ipa/rkisp1/algorithms/cproc.h | 30 ++++++++ src/ipa/rkisp1/algorithms/meson.build | 1 + src/ipa/rkisp1/data/ov5640.yaml | 1 + src/ipa/rkisp1/ipa_context.cpp | 17 +++++ src/ipa/rkisp1/ipa_context.h | 7 ++ src/libcamera/pipeline/rkisp1/rkisp1.cpp | 12 +++ 7 files changed, 165 insertions(+) create mode 100644 src/ipa/rkisp1/algorithms/cproc.cpp create mode 100644 src/ipa/rkisp1/algorithms/cproc.h diff --git a/src/ipa/rkisp1/algorithms/cproc.cpp b/src/ipa/rkisp1/algorithms/cproc.cpp new file mode 100644 index 00000000..fa989497 --- /dev/null +++ b/src/ipa/rkisp1/algorithms/cproc.cpp @@ -0,0 +1,97 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2022, Ideas On Board + * + * cproc.cpp - RkISP1 Color Processing control + */ + +#include "cproc.h" + +#include +#include + +#include + +#include + +/** + * \file cproc.h + */ + +namespace libcamera { + +namespace ipa::rkisp1::algorithms { + +/** + * \class ColorProcessing + * \brief RkISP1 Color Processing control + * + * The ColorProcessing algorithm is responsible for applying brightness, + * contrast and saturation corrections. The values are directly provided + * through requests by the corresponding controls. + */ + +LOG_DEFINE_CATEGORY(RkISP1CProc) + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void ColorProcessing::queueRequest([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + const ControlList &controls) +{ + auto &cproc = context.frameContext.cproc; + + const auto &brightness = controls.get(controls::Brightness); + if (brightness) { + cproc.brightness = std::round(std::clamp(*brightness * 128.0f, -128.0f, 127.0f)); + cproc.updateParams = true; + + LOG(RkISP1CProc, Debug) << "Set brightness to " << *brightness; + } + + const auto &contrast = controls.get(controls::Contrast); + if (contrast) { + cproc.contrast = std::round(std::clamp(*contrast * 128.0f, 0.0f, 255.0f)); + cproc.updateParams = true; + + LOG(RkISP1CProc, Debug) << "Set contrast to " << *contrast; + } + + const auto saturation = controls.get(controls::Saturation); + if (saturation) { + cproc.saturation = std::round(std::clamp(*saturation * 128.0f, 0.0f, 255.0f)); + cproc.updateParams = true; + + LOG(RkISP1CProc, Debug) << "Set saturation to " << *saturation; + } +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void ColorProcessing::prepare([[maybe_unused]] IPAContext &context, + rkisp1_params_cfg *params) +{ + auto &cproc = context.frameContext.cproc; + + /* Check if the algorithm configuration has been updated. */ + if (!cproc.updateParams) + return; + + cproc.updateParams = false; + + params->others.cproc_config.brightness = cproc.brightness; + params->others.cproc_config.contrast = cproc.contrast; + params->others.cproc_config.sat = cproc.saturation; + + params->module_en_update |= RKISP1_CIF_ISP_MODULE_CPROC; + params->module_ens |= RKISP1_CIF_ISP_MODULE_CPROC; + params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_CPROC; +} + +REGISTER_IPA_ALGORITHM(ColorProcessing, "ColorProcessing") + +} /* namespace ipa::rkisp1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/cproc.h b/src/ipa/rkisp1/algorithms/cproc.h new file mode 100644 index 00000000..4b7e4064 --- /dev/null +++ b/src/ipa/rkisp1/algorithms/cproc.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2022, Ideas On Board + * + * cproc.h - RkISP1 Color Processing control + */ + +#pragma once + +#include + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp1::algorithms { + +class ColorProcessing : public Algorithm +{ +public: + ColorProcessing() = default; + ~ColorProcessing() = default; + + void queueRequest(IPAContext &context, const uint32_t frame, + const ControlList &controls) override; + void prepare(IPAContext &context, rkisp1_params_cfg *params) override; +}; + +} /* namespace ipa::rkisp1::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/meson.build b/src/ipa/rkisp1/algorithms/meson.build index dcd24fe0..e48974b4 100644 --- a/src/ipa/rkisp1/algorithms/meson.build +++ b/src/ipa/rkisp1/algorithms/meson.build @@ -4,6 +4,7 @@ rkisp1_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', 'blc.cpp', + 'cproc.cpp', 'dpcc.cpp', 'filter.cpp', 'gsl.cpp', diff --git a/src/ipa/rkisp1/data/ov5640.yaml b/src/ipa/rkisp1/data/ov5640.yaml index 99529481..93d7d1e7 100644 --- a/src/ipa/rkisp1/data/ov5640.yaml +++ b/src/ipa/rkisp1/data/ov5640.yaml @@ -10,6 +10,7 @@ algorithms: Gr: 256 Gb: 256 B: 256 + - ColorProcessing: - GammaSensorLinearization: x-intervals: [ 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2 ] y: diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 4b117186..ef8bb8e9 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -136,6 +136,23 @@ namespace libcamera::ipa::rkisp1 { * \brief Estimated color temperature */ +/** + * \var IPAFrameContext::cproc + * \brief Context for the Color Processing algorithm + * + * \struct IPAFrameContext::cproc.brightness + * \brief Brightness level + * + * \var IPAFrameContext::cproc.contrast + * \brief Contrast level + * + * \var IPAFrameContext::cproc.saturation + * \brief Saturation level + * + * \var IPAFrameContext::cproc.updateParams + * \brief Indicates if ISP parameters need to be updated + */ + /** * \var IPAFrameContext::filter * \brief Context for the Filter algorithm diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 3b2f6af1..2bdb6a81 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -57,6 +57,13 @@ struct IPAFrameContext { double temperatureK; } awb; + struct { + int8_t brightness; + uint8_t contrast; + uint8_t saturation; + bool updateParams; + } cproc; + struct { uint8_t denoise; uint8_t sharpness; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 4e000d31..de687f4d 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -972,6 +972,18 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) std::forward_as_tuple(&controls::Sharpness), std::forward_as_tuple(0.0f, 10.0f, 1.0f)); + ctrls.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::Brightness), + std::forward_as_tuple(-1.0f, 0.993f)); + + ctrls.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::Contrast), + std::forward_as_tuple(0.0f, 1.993f)); + + ctrls.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::Saturation), + std::forward_as_tuple(0.0f, 1.993f)); + ctrls.emplace(std::piecewise_construct, std::forward_as_tuple(&controls::draft::NoiseReductionMode), std::forward_as_tuple(controls::draft::NoiseReductionModeValues));