From patchwork Wed Jul 20 15:42:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florian Sylvestre X-Patchwork-Id: 16708 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6E163BD1F1 for ; Wed, 20 Jul 2022 15:42:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 295BA6331E; Wed, 20 Jul 2022 17:42:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658331757; bh=N6m86BYNr3brvqxQw+DsuRHS9vY8zBPseaNyrwh0rDI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=LeM7gN6OETFkizADm7cg7U+jGU3xwV6VHpsfpktBMFkhUIi5nriJ6Fanvv2IdMRgh XWG5UA9Eu7XLP/nz+cEdba7jgvnDcPoCVbXBkA0RXUBmYZIlNRloB+/2hMTLDdt7xC Gw8VfijvuHveYJvWqBRw10B0wk316C6nQ0KNOOInoswlVGy1IpBAFfn4Jo4gY9L963 x7QdJXvt1O+On/k1ifu3wFYkTZo1HFtUory8gD5d0YhnlfN4Lx1a3ylBOGvyorMiEC QQHW9igWorBFEu/pdLVxWR9XYUP+pIGzfiX4joAOLE9FL3jtK9Xt7gSkHmCDi3NwfP Xj2V6wGm4A+SQ== Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5B23E6331A for ; Wed, 20 Jul 2022 17:42:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20210112.gappssmtp.com header.i=@baylibre-com.20210112.gappssmtp.com header.b="2v2QEDqo"; dkim-atps=neutral Received: by mail-wm1-x335.google.com with SMTP id h14-20020a1ccc0e000000b0039eff745c53so1523355wmb.5 for ; Wed, 20 Jul 2022 08:42:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Cuvyk6OqGESFgQnoygLrvmtHcgBgxWfAz3NtCXSLi5M=; b=2v2QEDqovw3EWg5hoZ/UKx55Db4dcvTAE0JCfDeOdA/uLl7XvSrOIjurlT5YS02aih +5F9JAOvnb0AGLDCGYHiGHSz02zol/c42BnCU0a0ROFOx0smQXKXKXYYcBMcAAhr3Coe 2b2h6b8/j2OApXsAQ0heqOr8jZLC43DF6qCdP1VMpp+F/XPabDA0NJphuLVceX+rfUIy zAmZDse2nK2qJfH3SAcTKiMo2irYvc8/cSDhayvruKzQVK0HO6h/7hamwe1ynAFQsaWH abrDW/VbknnBvCCQn0My4tGDj0yPLZurB9yo1k1RbvNDB/sEZC1nbcW0CuTOvQHr1OVo SvjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Cuvyk6OqGESFgQnoygLrvmtHcgBgxWfAz3NtCXSLi5M=; b=EqaYpOWk1TmkIJY+eUB5VcbMk12G98ZOEj0IU1A3+hBw1Ter0fKM/u7YWPq/sKAAMl aaBcbT5DVQrMRAmTZtYFIVBWa4o4+mBjiDN+PuVtHQ6CoYhz4Jk+QVIVbWYtbKcrZQ0c OS4p8q74ppv40iZHAonb5O43YaXgbg6FxB77q37afLPEI0LRZRQpxGdiem71wgPYmazn Jy1G11vykSq+jTrRyy2D17aeTvuLUSFkfEzqv46Up+qo03tOOToEj4AiByPLigzvEZKc /J8yWJf4LRxenar8hdP6Zzp5zAsSLidzIqZGP3MPd/kDoNUXInM7dH2JnXiW6Bv2BnoV ziMw== X-Gm-Message-State: AJIora/UoiFI4uj/Y7a1WHQ8TDnRbVRfbfFbLiUpe7YqBleaiIbslud4 a67+TYynSkA4eOSggvoBm3I5twNS1kP1pUp1 X-Google-Smtp-Source: AGRyM1v78lIsbkP/XfqlfhJGvpAjtDmbx9uiWq6QP+7z+9ADY31nb+utLCyQd0i7RO2Rx3zTBX8A5w== X-Received: by 2002:a05:600c:4e88:b0:3a3:1bdc:cb72 with SMTP id f8-20020a05600c4e8800b003a31bdccb72mr4502989wmq.59.1658331753459; Wed, 20 Jul 2022 08:42:33 -0700 (PDT) Received: from BL087.. ([2a01:e34:eea9:e630:516d:f303:6345:b768]) by smtp.gmail.com with ESMTPSA id d2-20020adffbc2000000b0021e489ec78bsm2159294wrs.21.2022.07.20.08.42.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Jul 2022 08:42:33 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 20 Jul 2022 17:42:20 +0200 Message-Id: <20220720154221.50937-4-fsylvestre@baylibre.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220720154221.50937-1-fsylvestre@baylibre.com> References: <20220720154221.50937-1-fsylvestre@baylibre.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/4] ipa: rkisp1: Add support of Demosaicing control X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Florian Sylvestre via libcamera-devel From: Florian Sylvestre Reply-To: Florian Sylvestre Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" During the demosaicing step, rkisp1 ISP is processing denoising and sharpness control. Add demosaicing algorithm with denoise and sharpness values based on user controls. Signed-off-by: Florian Sylvestre --- src/ipa/rkisp1/algorithms/demosaicing.cpp | 197 ++++++++++++++++++++++ src/ipa/rkisp1/algorithms/demosaicing.h | 30 ++++ src/ipa/rkisp1/algorithms/meson.build | 1 + src/ipa/rkisp1/data/ov5640.yaml | 1 + src/ipa/rkisp1/ipa_context.h | 6 + src/ipa/rkisp1/rkisp1.cpp | 1 + src/libcamera/pipeline/rkisp1/rkisp1.cpp | 8 + 7 files changed, 244 insertions(+) create mode 100644 src/ipa/rkisp1/algorithms/demosaicing.cpp create mode 100644 src/ipa/rkisp1/algorithms/demosaicing.h diff --git a/src/ipa/rkisp1/algorithms/demosaicing.cpp b/src/ipa/rkisp1/algorithms/demosaicing.cpp new file mode 100644 index 00000000..7d55eaab --- /dev/null +++ b/src/ipa/rkisp1/algorithms/demosaicing.cpp @@ -0,0 +1,197 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2022, Ideas On Board + * + * demosaicing.cpp - RkISP1 Demosaicing control + */ + +#include "demosaicing.h" + +#include + +#include + +#include "libcamera/internal/yaml_parser.h" + +/** + * \file demosaicing.h + */ + +static constexpr uint32_t kFiltLumWeightDefault = 0x00022040; +static constexpr uint32_t kFiltModeDefault = 0x000004f2; + +namespace libcamera { + +namespace ipa::rkisp1::algorithms { + +/** + * \class Demosaicing + * \brief RkISP1 Demosaicing control + * + * The Demosaicing algorithm is responsible for reconstructing a full color + * image from the sensor raw data. During the demosaicing step, The RKISP1 + * will additionally process denoise and sharpness controls. + * + * /todo In current version the denoise and sharpness control is based on user + * controls. In a future version it should be controlled automatically by the + * algorithm. + */ + +LOG_DEFINE_CATEGORY(RkISP1Demosaicing) + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Demosaicing::queueRequest([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + const ControlList &controls) +{ + const auto &sharpness = controls.get(controls::Sharpness); + if (sharpness) { + context.frameContext.demosaicing.sharpness = std::clamp(int(*sharpness), 0, 10); + context.frameContext.demosaicing.updateParams = true; + + LOG(RkISP1Demosaicing, Debug) << "Set sharpness to " << *sharpness; + } + + const auto &denoise = controls.get(controls::draft::NoiseReductionMode); + if (denoise) { + LOG(RkISP1Demosaicing, Debug) << "Set denoise to " << *denoise; + + switch (*denoise) { + case controls::draft::NoiseReductionModeOff: + context.frameContext.demosaicing.denoise = 0; + context.frameContext.demosaicing.updateParams = true; + break; + case controls::draft::NoiseReductionModeMinimal: + context.frameContext.demosaicing.denoise = 1; + context.frameContext.demosaicing.updateParams = true; + break; + case controls::draft::NoiseReductionModeHighQuality: + case controls::draft::NoiseReductionModeFast: + context.frameContext.demosaicing.denoise = 3; + context.frameContext.demosaicing.updateParams = true; + break; + default: + LOG(RkISP1Demosaicing, Error) + << "Unsupported denoise value " + << *denoise; + } + } +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Demosaicing::prepare([[maybe_unused]] IPAContext &context, + rkisp1_params_cfg *params) +{ + /* Check if the algorithm configuration has been updated. */ + if (!context.frameContext.demosaicing.updateParams) + return; + + context.frameContext.demosaicing.updateParams = false; + + static constexpr uint16_t filt_fac_sh0[] = { + 0x04, 0x07, 0x0A, 0x0C, 0x10, 0x14, 0x1A, 0x1E, 0x24, 0x2A, 0x30 + }; + + static constexpr uint16_t filt_fac_sh1[] = { + 0x04, 0x08, 0x0C, 0x10, 0x16, 0x1B, 0x20, 0x26, 0x2C, 0x30, 0x3F + }; + + static constexpr uint16_t filt_fac_mid[] = { + 0x04, 0x06, 0x08, 0x0A, 0x0C, 0x10, 0x13, 0x17, 0x1D, 0x22, 0x28 + }; + + static constexpr uint16_t filt_fac_bl0[] = { + 0x02, 0x02, 0x04, 0x06, 0x08, 0x0A, 0x0C, 0x10, 0x15, 0x1A, 0x24 + }; + + static constexpr uint16_t filt_fac_bl1[] = { + 0x00, 0x00, 0x00, 0x02, 0x04, 0x04, 0x06, 0x08, 0x0D, 0x14, 0x20 + }; + + static constexpr uint16_t filt_thresh_sh0[] = { + 0, 18, 26, 36, 41, 75, 90, 120, 170, 250, 1023 + }; + + static constexpr uint16_t filt_thresh_sh1[] = { + 0, 33, 44, 51, 67, 100, 120, 150, 200, 300, 1023 + }; + + static constexpr uint16_t filt_thresh_bl0[] = { + 0, 8, 13, 23, 26, 50, 60, 80, 140, 180, 1023 + }; + + static constexpr uint16_t filt_thresh_bl1[] = { + 0, 2, 5, 10, 15, 20, 26, 51, 100, 150, 1023 + }; + + static constexpr uint16_t stage1_select[] = { + 6, 6, 4, 4, 3, 3, 2, 2, 2, 2, 2 + }; + + static constexpr uint16_t filt_chr_v_mode[] = { + 1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3 + }; + + static constexpr uint16_t filt_chr_h_mode[] = { + 0, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3 + }; + + uint8_t denoise = context.frameContext.demosaicing.denoise; + uint8_t sharpness = context.frameContext.demosaicing.sharpness; + + params->others.flt_config.fac_sh0 = filt_fac_sh0[sharpness]; + params->others.flt_config.fac_sh1 = filt_fac_sh1[sharpness]; + params->others.flt_config.fac_mid = filt_fac_mid[sharpness]; + params->others.flt_config.fac_bl0 = filt_fac_bl0[sharpness]; + params->others.flt_config.fac_bl1 = filt_fac_bl1[sharpness]; + + params->others.flt_config.lum_weight = kFiltLumWeightDefault; + params->others.flt_config.mode = kFiltModeDefault; + params->others.flt_config.thresh_sh0 = filt_thresh_sh0[denoise]; + params->others.flt_config.thresh_sh1 = filt_thresh_sh1[denoise]; + params->others.flt_config.thresh_bl0 = filt_thresh_bl0[denoise]; + params->others.flt_config.thresh_bl1 = filt_thresh_bl1[denoise]; + params->others.flt_config.grn_stage1 = stage1_select[denoise]; + params->others.flt_config.chr_v_mode = filt_chr_v_mode[denoise]; + params->others.flt_config.chr_h_mode = filt_chr_h_mode[denoise]; + + if (denoise == 9) { + if (sharpness > 3) + params->others.flt_config.grn_stage1 = 2; + else + params->others.flt_config.grn_stage1 = 1; + } else if (denoise == 10) { + if (sharpness > 5) + params->others.flt_config.grn_stage1 = 2; + else if (sharpness > 3) + params->others.flt_config.grn_stage1 = 1; + else + params->others.flt_config.grn_stage1 = 0; + } else if (denoise > 7) { + if (sharpness > 7) { + params->others.flt_config.fac_bl0 = + params->others.flt_config.fac_bl0 / 2; + params->others.flt_config.fac_bl1 = + params->others.flt_config.fac_bl1 / 4; + } else if (sharpness > 4) { + params->others.flt_config.fac_bl0 = + params->others.flt_config.fac_bl0 * 3 / 4; + params->others.flt_config.fac_bl1 = + params->others.flt_config.fac_bl1 / 2; + } + } + + params->module_en_update |= RKISP1_CIF_ISP_MODULE_FLT; + params->module_ens |= RKISP1_CIF_ISP_MODULE_FLT; + params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_FLT; +} + +REGISTER_IPA_ALGORITHM(Demosaicing, "Demosaicing") + +} /* namespace ipa::rkisp1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/demosaicing.h b/src/ipa/rkisp1/algorithms/demosaicing.h new file mode 100644 index 00000000..0d0f778f --- /dev/null +++ b/src/ipa/rkisp1/algorithms/demosaicing.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2022, Ideas On Board + * + * demosaicing.h - RkISP1 Demosaicing control + */ + +#pragma once + +#include + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp1::algorithms { + +class Demosaicing : public Algorithm +{ +public: + Demosaicing() = default; + ~Demosaicing() = default; + + void queueRequest(IPAContext &context, const uint32_t frame, + const ControlList &controls) override; + void prepare(IPAContext &context, rkisp1_params_cfg *params) override; +}; + +} /* namespace ipa::rkisp1::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/meson.build b/src/ipa/rkisp1/algorithms/meson.build index 87007493..7e078b0d 100644 --- a/src/ipa/rkisp1/algorithms/meson.build +++ b/src/ipa/rkisp1/algorithms/meson.build @@ -4,6 +4,7 @@ rkisp1_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', 'blc.cpp', + 'demosaicing.cpp', 'dpcc.cpp', 'gsl.cpp', 'lsc.cpp', diff --git a/src/ipa/rkisp1/data/ov5640.yaml b/src/ipa/rkisp1/data/ov5640.yaml index 51228218..4ae0ffc0 100644 --- a/src/ipa/rkisp1/data/ov5640.yaml +++ b/src/ipa/rkisp1/data/ov5640.yaml @@ -157,4 +157,5 @@ algorithms: rnd-offsets: green: 2 red-blue: 2 + - Demosaicing: ... diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index f387cace..241a4d04 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -56,6 +56,12 @@ struct IPAFrameContext { double temperatureK; } awb; + struct { + uint8_t denoise; + uint8_t sharpness; + bool updateParams; + } demosaicing; + struct { uint32_t exposure; double gain; diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 34034526..aeec8f50 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -31,6 +31,7 @@ #include "algorithms/algorithm.h" #include "algorithms/awb.h" #include "algorithms/blc.h" +#include "algorithms/demosaicing.h" #include "algorithms/dpcc.h" #include "algorithms/gsl.h" #include "algorithms/lsc.h" diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 99eecd44..4e000d31 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -968,6 +968,14 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) hasSelfPath_ ? &selfPath_ : nullptr); ControlInfoMap::Map ctrls; + ctrls.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::Sharpness), + std::forward_as_tuple(0.0f, 10.0f, 1.0f)); + + ctrls.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::draft::NoiseReductionMode), + std::forward_as_tuple(controls::draft::NoiseReductionModeValues)); + ctrls.emplace(std::piecewise_construct, std::forward_as_tuple(&controls::AeEnable), std::forward_as_tuple(false, true));