From patchwork Wed Jul 27 08:38:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florian Sylvestre X-Patchwork-Id: 16829 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 56E7AC3275 for ; Wed, 27 Jul 2022 08:38:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0F68C63319; Wed, 27 Jul 2022 10:38:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658911108; bh=Jw/X/1xSbD9xYjfpC+uoj+s3NsRD51la7Mlj+MeOOtI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=szTSwyh2J+1zYJoDlhpsXTrNY0CydMaQI9GeiM5fl2V7ESpx0SWWEkgoVhhFt9elm H38Nar3YbojB+xJj7LYgkaWUJCGF9lNNmhNYONA5PVZzcqcT5UgtxVGAvqtkJ1ZNsF HjKR3cTaALp2bQsAIex+XDXKsQzgrHF/mWxhCr3Ni7/ghfz1nZTpc61D6mbzJ2xMce vGlztk+AxDTDbU3YdYQnX/d8r2JeCUknkrr5wUrtu1gmZeIP3hEfCa2kWq8VX4ccAD E0kP9qdghp1HUy041idILojNrvc1PjuYvflEAV2OD1MQ7bvSfLK9ghLkU2rMNg+LNJ GG0hn/B78hFHA== Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E18186330E for ; Wed, 27 Jul 2022 10:38:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20210112.gappssmtp.com header.i=@baylibre-com.20210112.gappssmtp.com header.b="X3ogbDjt"; dkim-atps=neutral Received: by mail-wm1-x32a.google.com with SMTP id b6so10088129wmq.5 for ; Wed, 27 Jul 2022 01:38:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bLtF4fq9+3aONrH1M7FuchSGGH4uA7dDAozCTZFU6Ew=; b=X3ogbDjtUz9gwH3qCmMxwIagD7kBZi1SL5bcDUV7E9HlzLAriW4odHeE43OH6aB47X VXRnEMeOHLaUIJS+tv1qyL6jQr/nGyZPy1m5gMvACIbAxB3DER6CUbmYXbMpW5CK0Y1s wKjnyor4SRUSYNik1EW4VIfmaWCrQFFRAyXidyF3IdZbAsScwpJvUe1Vlb4nx8deMLtB ygj8h2/8TEO91KnfqV3RZsgJe2ZbEgX8Ev4kAmEtYx79klyLjMt3kKF5iYDRlIp+b4oK m4DjyI3+whLW80RAu1KBnWXAzcaBoxfDPQicvaLh0LgyyORD49Wz4S1sB4yZDNV8uUyG 71Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bLtF4fq9+3aONrH1M7FuchSGGH4uA7dDAozCTZFU6Ew=; b=4o/wsZLotPJArWaPVEq9twvnPsA4M7nUlpLsVtuAgL83oe8LlD8mszR8UkaRF58JfM JdFNmbt/ESSGhjEzIGDLPuXKekqa4K3i/BNXdYVk4yun1PNgB9hCyjSvo4DkuJ0UgH0Y OiRCINK79pcjjqiZN9WxvrHk7fC/W9F8t7mpZQTHWPVXxyrHe6BQB4JiowqK9uoMXDZ2 RQBKAYsAaqkqsBG0OVwDhn4qM9w4WyubT0q0VICtWXhkaqt1BqTWcDjJ976jF4+tSKTW SENC7eNnNCoNJUFJZBTpzjrbVctRunJPWKdQkK4GDeqvvW6eawmHXk0igQusOgLRShsY 62HA== X-Gm-Message-State: AJIora/UPUeezhF/jn4vX73AAin/PwkrCbVhrOnUEXWlasSAOCgY2ma/ rfidqYJm1w1z4ut61uuW1/t+Tz1FrK3SvExD X-Google-Smtp-Source: AGRyM1uu8hSTRNaEtVPAkoz0E0CeRdjprCwiuwWUQeA2g5IEynk+mHCf360gCsK9n10/FdiQHm1VXA== X-Received: by 2002:a05:600c:1d92:b0:3a3:2167:b8e5 with SMTP id p18-20020a05600c1d9200b003a32167b8e5mr2181258wms.24.1658911104829; Wed, 27 Jul 2022 01:38:24 -0700 (PDT) Received: from BL087.. ([2a01:e34:eea9:e630:5810:316e:87c:fc17]) by smtp.gmail.com with ESMTPSA id l18-20020a05600c1d1200b003a2fb1224d9sm1745040wms.19.2022.07.27.01.38.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jul 2022 01:38:23 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 27 Jul 2022 10:38:18 +0200 Message-Id: <20220727083819.589595-2-fsylvestre@baylibre.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220727083819.589595-1-fsylvestre@baylibre.com> References: <20220727083819.589595-1-fsylvestre@baylibre.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 1/2] ipa: rkisp1: Add support of Filter control X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Florian Sylvestre via libcamera-devel From: Florian Sylvestre Reply-To: Florian Sylvestre Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Denoise and Sharpness filters will be applied by RkISP1 during the demosaicing step. The denoise filter is responsible for removing noise from the image, while the sharpness filter will enhance its acutance. Add filter algorithm with denoise and sharpness values based on user controls. Signed-off-by: Florian Sylvestre Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/algorithms/filter.cpp | 201 +++++++++++++++++++++++ src/ipa/rkisp1/algorithms/filter.h | 30 ++++ src/ipa/rkisp1/algorithms/meson.build | 1 + src/ipa/rkisp1/data/ov5640.yaml | 1 + src/ipa/rkisp1/ipa_context.cpp | 14 ++ src/ipa/rkisp1/ipa_context.h | 6 + src/libcamera/pipeline/rkisp1/rkisp1.cpp | 8 + 7 files changed, 261 insertions(+) create mode 100644 src/ipa/rkisp1/algorithms/filter.cpp create mode 100644 src/ipa/rkisp1/algorithms/filter.h diff --git a/src/ipa/rkisp1/algorithms/filter.cpp b/src/ipa/rkisp1/algorithms/filter.cpp new file mode 100644 index 00000000..8ca10fd1 --- /dev/null +++ b/src/ipa/rkisp1/algorithms/filter.cpp @@ -0,0 +1,201 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2022, Ideas On Board + * + * filter.cpp - RkISP1 Filter control + */ + +#include "filter.h" + +#include + +#include + +#include + +/** + * \file filter.h + */ + +namespace libcamera { + +namespace ipa::rkisp1::algorithms { + +/** + * \class Filter + * \brief RkISP1 Filter control + * + * Denoise and Sharpness filters will be applied by RkISP1 during the + * demosaicing step. The denoise filter is responsible for removing noise from + * the image, while the sharpness filter will enhance its acutance. + * + * \todo In current version the denoise and sharpness control is based on user + * controls. In a future version it should be controlled automatically by the + * algorithm. + */ + +LOG_DEFINE_CATEGORY(RkISP1Filter) + +static constexpr uint32_t kFiltLumWeightDefault = 0x00022040; +static constexpr uint32_t kFiltModeDefault = 0x000004f2; + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Filter::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + const ControlList &controls) +{ + auto &filter = context.frameContext.filter; + + const auto &sharpness = controls.get(controls::Sharpness); + if (sharpness) { + filter.sharpness = std::round(std::clamp(*sharpness, 0.0f, 10.0f)); + filter.updateParams = true; + + LOG(RkISP1Filter, Debug) << "Set sharpness to " << *sharpness; + } + + const auto &denoise = controls.get(controls::draft::NoiseReductionMode); + if (denoise) { + LOG(RkISP1Filter, Debug) << "Set denoise to " << *denoise; + + switch (*denoise) { + case controls::draft::NoiseReductionModeOff: + filter.denoise = 0; + filter.updateParams = true; + break; + case controls::draft::NoiseReductionModeMinimal: + filter.denoise = 1; + filter.updateParams = true; + break; + case controls::draft::NoiseReductionModeHighQuality: + case controls::draft::NoiseReductionModeFast: + filter.denoise = 3; + filter.updateParams = true; + break; + default: + LOG(RkISP1Filter, Error) + << "Unsupported denoise value " + << *denoise; + } + } +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Filter::prepare(IPAContext &context, rkisp1_params_cfg *params) +{ + auto &filter = context.frameContext.filter; + + /* Check if the algorithm configuration has been updated. */ + if (!filter.updateParams) + return; + + filter.updateParams = false; + + static constexpr uint16_t filt_fac_sh0[] = { + 0x04, 0x07, 0x0a, 0x0c, 0x10, 0x14, 0x1a, 0x1e, 0x24, 0x2a, 0x30 + }; + + static constexpr uint16_t filt_fac_sh1[] = { + 0x04, 0x08, 0x0c, 0x10, 0x16, 0x1b, 0x20, 0x26, 0x2c, 0x30, 0x3f + }; + + static constexpr uint16_t filt_fac_mid[] = { + 0x04, 0x06, 0x08, 0x0a, 0x0c, 0x10, 0x13, 0x17, 0x1d, 0x22, 0x28 + }; + + static constexpr uint16_t filt_fac_bl0[] = { + 0x02, 0x02, 0x04, 0x06, 0x08, 0x0a, 0x0c, 0x10, 0x15, 0x1a, 0x24 + }; + + static constexpr uint16_t filt_fac_bl1[] = { + 0x00, 0x00, 0x00, 0x02, 0x04, 0x04, 0x06, 0x08, 0x0d, 0x14, 0x20 + }; + + static constexpr uint16_t filt_thresh_sh0[] = { + 0, 18, 26, 36, 41, 75, 90, 120, 170, 250, 1023 + }; + + static constexpr uint16_t filt_thresh_sh1[] = { + 0, 33, 44, 51, 67, 100, 120, 150, 200, 300, 1023 + }; + + static constexpr uint16_t filt_thresh_bl0[] = { + 0, 8, 13, 23, 26, 50, 60, 80, 140, 180, 1023 + }; + + static constexpr uint16_t filt_thresh_bl1[] = { + 0, 2, 5, 10, 15, 20, 26, 51, 100, 150, 1023 + }; + + static constexpr uint16_t stage1_select[] = { + 6, 6, 4, 4, 3, 3, 2, 2, 2, 1, 0 + }; + + static constexpr uint16_t filt_chr_v_mode[] = { + 1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3 + }; + + static constexpr uint16_t filt_chr_h_mode[] = { + 0, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3 + }; + + uint8_t denoise = filter.denoise; + uint8_t sharpness = filter.sharpness; + auto &flt_config = params->others.flt_config; + + flt_config.fac_sh0 = filt_fac_sh0[sharpness]; + flt_config.fac_sh1 = filt_fac_sh1[sharpness]; + flt_config.fac_mid = filt_fac_mid[sharpness]; + flt_config.fac_bl0 = filt_fac_bl0[sharpness]; + flt_config.fac_bl1 = filt_fac_bl1[sharpness]; + + flt_config.lum_weight = kFiltLumWeightDefault; + flt_config.mode = kFiltModeDefault; + flt_config.thresh_sh0 = filt_thresh_sh0[denoise]; + flt_config.thresh_sh1 = filt_thresh_sh1[denoise]; + flt_config.thresh_bl0 = filt_thresh_bl0[denoise]; + flt_config.thresh_bl1 = filt_thresh_bl1[denoise]; + flt_config.grn_stage1 = stage1_select[denoise]; + flt_config.chr_v_mode = filt_chr_v_mode[denoise]; + flt_config.chr_h_mode = filt_chr_h_mode[denoise]; + + /* + * Combined high denoising and high sharpening requires some + * adjustments to the configuration of the filters. A first stage + * filter with a lower strength must be selected, and the blur factors + * must be decreased. + */ + if (denoise == 9) { + if (sharpness > 3) + flt_config.grn_stage1 = 2; + } else if (denoise == 10) { + if (sharpness > 5) + flt_config.grn_stage1 = 2; + else if (sharpness > 3) + flt_config.grn_stage1 = 1; + } + + if (denoise > 7) { + if (sharpness > 7) { + flt_config.fac_bl0 /= 2; + flt_config.fac_bl1 /= 4; + } else if (sharpness > 4) { + flt_config.fac_bl0 = flt_config.fac_bl0 * 3 / 4; + flt_config.fac_bl1 /= 2; + } + } + + params->module_en_update |= RKISP1_CIF_ISP_MODULE_FLT; + params->module_ens |= RKISP1_CIF_ISP_MODULE_FLT; + params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_FLT; +} + +REGISTER_IPA_ALGORITHM(Filter, "Filter") + +} /* namespace ipa::rkisp1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/filter.h b/src/ipa/rkisp1/algorithms/filter.h new file mode 100644 index 00000000..9eb170eb --- /dev/null +++ b/src/ipa/rkisp1/algorithms/filter.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2022, Ideas On Board + * + * filter.h - RkISP1 Filter control + */ + +#pragma once + +#include + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp1::algorithms { + +class Filter : public Algorithm +{ +public: + Filter() = default; + ~Filter() = default; + + void queueRequest(IPAContext &context, const uint32_t frame, + const ControlList &controls) override; + void prepare(IPAContext &context, rkisp1_params_cfg *params) override; +}; + +} /* namespace ipa::rkisp1::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/meson.build b/src/ipa/rkisp1/algorithms/meson.build index 87007493..dcd24fe0 100644 --- a/src/ipa/rkisp1/algorithms/meson.build +++ b/src/ipa/rkisp1/algorithms/meson.build @@ -5,6 +5,7 @@ rkisp1_ipa_algorithms = files([ 'awb.cpp', 'blc.cpp', 'dpcc.cpp', + 'filter.cpp', 'gsl.cpp', 'lsc.cpp', ]) diff --git a/src/ipa/rkisp1/data/ov5640.yaml b/src/ipa/rkisp1/data/ov5640.yaml index 2315ec43..99529481 100644 --- a/src/ipa/rkisp1/data/ov5640.yaml +++ b/src/ipa/rkisp1/data/ov5640.yaml @@ -155,4 +155,5 @@ algorithms: rnd-offsets: green: 2 red-blue: 2 + - Filter: ... diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 30bb87a9..4b117186 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -136,6 +136,20 @@ namespace libcamera::ipa::rkisp1 { * \brief Estimated color temperature */ +/** + * \var IPAFrameContext::filter + * \brief Context for the Filter algorithm + * + * \struct IPAFrameContext::filter.denoise + * \brief Denoising level + * + * \var IPAFrameContext::filter.sharpness + * \brief Sharpness level + * + * \var IPAFrameContext::filter.updateParams + * \brief Indicates if ISP parameters need to be updated + */ + /** * \var IPAFrameContext::sensor * \brief Effective sensor values diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 3bfb262c..3b2f6af1 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -57,6 +57,12 @@ struct IPAFrameContext { double temperatureK; } awb; + struct { + uint8_t denoise; + uint8_t sharpness; + bool updateParams; + } filter; + struct { uint32_t exposure; double gain; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 99eecd44..4e000d31 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -968,6 +968,14 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) hasSelfPath_ ? &selfPath_ : nullptr); ControlInfoMap::Map ctrls; + ctrls.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::Sharpness), + std::forward_as_tuple(0.0f, 10.0f, 1.0f)); + + ctrls.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::draft::NoiseReductionMode), + std::forward_as_tuple(controls::draft::NoiseReductionModeValues)); + ctrls.emplace(std::piecewise_construct, std::forward_as_tuple(&controls::AeEnable), std::forward_as_tuple(false, true)); From patchwork Wed Jul 27 08:38:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florian Sylvestre X-Patchwork-Id: 16830 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3F447C3275 for ; Wed, 27 Jul 2022 08:38:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 79DF263315; Wed, 27 Jul 2022 10:38:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658911108; bh=fRxgl9QBv6vKlU2gboxZOFP+Rzorp44DWRIDo1zaTFQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=t3AlIzkwv+cuDNR2HucKFuML7Zlzp24QRVWGl0bq9FZxbNw9B9Q5wqb1+bCCG1rwn 5wCHHayRttIYpJn8d+ZlNFv+INX1bTFiwqTX1yikzhwvzEpKS49TzKMkpBESLbOEsf sgRwUdqiiEr4Eo2g/CQDGE23vLB6+xx9JBRSe9YO90orMzPSquda0Lbd8N3tlIo3TK a33ad2Ih6fQ0qubwl19E4t0r20xjDmSosqPiogzyj8nG+xV907OCPm1xljv+57YU4g e9GtzUncKJoiRPk2wNh2tfHELQ1uWnKce2QbCijMCtD6nWowmpeBYpf3mI5npPtXeF iBA1RT1YeUsKg== Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CEDFB603EC for ; Wed, 27 Jul 2022 10:38:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20210112.gappssmtp.com header.i=@baylibre-com.20210112.gappssmtp.com header.b="gebSbnJi"; dkim-atps=neutral Received: by mail-wm1-x32c.google.com with SMTP id f24-20020a1cc918000000b003a30178c022so766591wmb.3 for ; Wed, 27 Jul 2022 01:38:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4rlHmoqkDjkuduxxctlpRVN5rs7z0KBK+iTZ5lY3Gho=; b=gebSbnJissE7iG4FNapQaj7WXYolDW9PHM4W0amftQyQljRH6Lo0WkskV0DPdyAttE LvfVz4giM8FZ5scvCX1RkUzPvEndPpGwcB+3VTG7tWz4PoooF/GWIczAvt0jz1lU0TrU FmsV1jGK3+cN2p4Hd3FS6jezaMi58uVUMw2SDoW9GFGxhVDkqgXTM5imQZCxAFwVxeQY phb1LocIehWQ5HOCUnrgDhMboPQRAioAgrywQJhiZxKRKs1GBxWUCvY7448YR9NFRe+2 jgkdFk8TugzQEpa68aEClBQLpPp2pZzLQODa8WCdpEKo8Kjfb6alPLnfaHO4wbj05O+n YsGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4rlHmoqkDjkuduxxctlpRVN5rs7z0KBK+iTZ5lY3Gho=; b=0NFfBhxuM7QvlJnWIjxA5hFAbCrQRK7mP4LC4ugILDpSATSZmgI29/0FzMjrYpwpod 1cXEX0oUWGV7E/0Xi1JNyNUeZv071HKnHqy2QFUo5gbJhOITbeHjo5lPFGfP9hDFUOq4 USucImeoeP9oFGf93psw/kxQB+M7uruJFmR62cK50QRzZxinypgN4TMBz8xES6gEyjLx x1c1fOXjK6SqFrYFoOpcYcnW84WHDZu04Zz/NFB9aUk9c2dM21nopcV42r7/rZTfDUEJ WkCOUvyfCw6MEDfx0auWce7CCipa0Zi8ubDW7pohIeKwq8MQw7Gw+MP0Q1EmUVRWdl9d g/0Q== X-Gm-Message-State: AJIora+cT2JOZed8gBh1YPcymKextsbsRWf4HdkyE0IcqWlIkAsgItZX 8T6KJRbKSCQd4JJdZQLIkpYs4qv+aWD90eVq X-Google-Smtp-Source: AGRyM1vdb50CsV9fQJLvjD5QHOsWeDtEfITNO0scHpktdLM8qRSjy1WFJcdEiOxyhOeATDPVFnBtPQ== X-Received: by 2002:a1c:f716:0:b0:3a1:8f0e:66b4 with SMTP id v22-20020a1cf716000000b003a18f0e66b4mr2187517wmh.140.1658911106116; Wed, 27 Jul 2022 01:38:26 -0700 (PDT) Received: from BL087.. ([2a01:e34:eea9:e630:5810:316e:87c:fc17]) by smtp.gmail.com with ESMTPSA id l18-20020a05600c1d1200b003a2fb1224d9sm1745040wms.19.2022.07.27.01.38.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jul 2022 01:38:25 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 27 Jul 2022 10:38:19 +0200 Message-Id: <20220727083819.589595-3-fsylvestre@baylibre.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220727083819.589595-1-fsylvestre@baylibre.com> References: <20220727083819.589595-1-fsylvestre@baylibre.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 2/2] ipa: rkisp1: Add support of ColorProcessing control X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Florian Sylvestre via libcamera-devel From: Florian Sylvestre Reply-To: Florian Sylvestre Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add ColorProcessing algorithm that is in charge to manage brightness, contrast and saturation controls. These controls are currently based on user controls. Signed-off-by: Florian Sylvestre Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/algorithms/cproc.cpp | 97 ++++++++++++++++++++++++ src/ipa/rkisp1/algorithms/cproc.h | 30 ++++++++ src/ipa/rkisp1/algorithms/meson.build | 1 + src/ipa/rkisp1/data/ov5640.yaml | 1 + src/ipa/rkisp1/ipa_context.cpp | 17 +++++ src/ipa/rkisp1/ipa_context.h | 7 ++ src/libcamera/pipeline/rkisp1/rkisp1.cpp | 12 +++ 7 files changed, 165 insertions(+) create mode 100644 src/ipa/rkisp1/algorithms/cproc.cpp create mode 100644 src/ipa/rkisp1/algorithms/cproc.h diff --git a/src/ipa/rkisp1/algorithms/cproc.cpp b/src/ipa/rkisp1/algorithms/cproc.cpp new file mode 100644 index 00000000..74c08215 --- /dev/null +++ b/src/ipa/rkisp1/algorithms/cproc.cpp @@ -0,0 +1,97 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2022, Ideas On Board + * + * cproc.cpp - RkISP1 Color Processing control + */ + +#include "cproc.h" + +#include +#include + +#include + +#include + +/** + * \file cproc.h + */ + +namespace libcamera { + +namespace ipa::rkisp1::algorithms { + +/** + * \class ColorProcessing + * \brief RkISP1 Color Processing control + * + * The ColorProcessing algorithm is responsible for applying brightness, + * contrast and saturation corrections. The values are directly provided + * through requests by the corresponding controls. + */ + +LOG_DEFINE_CATEGORY(RkISP1CProc) + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void ColorProcessing::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + const ControlList &controls) +{ + auto &cproc = context.frameContext.cproc; + + const auto &brightness = controls.get(controls::Brightness); + if (brightness) { + cproc.brightness = std::clamp(std::lround(*brightness * 128.0f), -128, 127); + cproc.updateParams = true; + + LOG(RkISP1CProc, Debug) << "Set brightness to " << *brightness; + } + + const auto &contrast = controls.get(controls::Contrast); + if (contrast) { + cproc.contrast = std::clamp(std::lround(*contrast * 128.0f), 0.0f, 255.0f); + cproc.updateParams = true; + + LOG(RkISP1CProc, Debug) << "Set contrast to " << *contrast; + } + + const auto saturation = controls.get(controls::Saturation); + if (saturation) { + cproc.saturation = std::clamp(std::lround(*saturation * 128.0f), 0.0f, 255.0f); + cproc.updateParams = true; + + LOG(RkISP1CProc, Debug) << "Set saturation to " << *saturation; + } +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void ColorProcessing::prepare(IPAContext &context, + rkisp1_params_cfg *params) +{ + auto &cproc = context.frameContext.cproc; + + /* Check if the algorithm configuration has been updated. */ + if (!cproc.updateParams) + return; + + cproc.updateParams = false; + + params->others.cproc_config.brightness = cproc.brightness; + params->others.cproc_config.contrast = cproc.contrast; + params->others.cproc_config.sat = cproc.saturation; + + params->module_en_update |= RKISP1_CIF_ISP_MODULE_CPROC; + params->module_ens |= RKISP1_CIF_ISP_MODULE_CPROC; + params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_CPROC; +} + +REGISTER_IPA_ALGORITHM(ColorProcessing, "ColorProcessing") + +} /* namespace ipa::rkisp1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/cproc.h b/src/ipa/rkisp1/algorithms/cproc.h new file mode 100644 index 00000000..4b7e4064 --- /dev/null +++ b/src/ipa/rkisp1/algorithms/cproc.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021-2022, Ideas On Board + * + * cproc.h - RkISP1 Color Processing control + */ + +#pragma once + +#include + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp1::algorithms { + +class ColorProcessing : public Algorithm +{ +public: + ColorProcessing() = default; + ~ColorProcessing() = default; + + void queueRequest(IPAContext &context, const uint32_t frame, + const ControlList &controls) override; + void prepare(IPAContext &context, rkisp1_params_cfg *params) override; +}; + +} /* namespace ipa::rkisp1::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/meson.build b/src/ipa/rkisp1/algorithms/meson.build index dcd24fe0..e48974b4 100644 --- a/src/ipa/rkisp1/algorithms/meson.build +++ b/src/ipa/rkisp1/algorithms/meson.build @@ -4,6 +4,7 @@ rkisp1_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', 'blc.cpp', + 'cproc.cpp', 'dpcc.cpp', 'filter.cpp', 'gsl.cpp', diff --git a/src/ipa/rkisp1/data/ov5640.yaml b/src/ipa/rkisp1/data/ov5640.yaml index 99529481..93d7d1e7 100644 --- a/src/ipa/rkisp1/data/ov5640.yaml +++ b/src/ipa/rkisp1/data/ov5640.yaml @@ -10,6 +10,7 @@ algorithms: Gr: 256 Gb: 256 B: 256 + - ColorProcessing: - GammaSensorLinearization: x-intervals: [ 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2 ] y: diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 4b117186..ef8bb8e9 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -136,6 +136,23 @@ namespace libcamera::ipa::rkisp1 { * \brief Estimated color temperature */ +/** + * \var IPAFrameContext::cproc + * \brief Context for the Color Processing algorithm + * + * \struct IPAFrameContext::cproc.brightness + * \brief Brightness level + * + * \var IPAFrameContext::cproc.contrast + * \brief Contrast level + * + * \var IPAFrameContext::cproc.saturation + * \brief Saturation level + * + * \var IPAFrameContext::cproc.updateParams + * \brief Indicates if ISP parameters need to be updated + */ + /** * \var IPAFrameContext::filter * \brief Context for the Filter algorithm diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 3b2f6af1..2bdb6a81 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -57,6 +57,13 @@ struct IPAFrameContext { double temperatureK; } awb; + struct { + int8_t brightness; + uint8_t contrast; + uint8_t saturation; + bool updateParams; + } cproc; + struct { uint8_t denoise; uint8_t sharpness; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 4e000d31..de687f4d 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -972,6 +972,18 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) std::forward_as_tuple(&controls::Sharpness), std::forward_as_tuple(0.0f, 10.0f, 1.0f)); + ctrls.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::Brightness), + std::forward_as_tuple(-1.0f, 0.993f)); + + ctrls.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::Contrast), + std::forward_as_tuple(0.0f, 1.993f)); + + ctrls.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::Saturation), + std::forward_as_tuple(0.0f, 1.993f)); + ctrls.emplace(std::piecewise_construct, std::forward_as_tuple(&controls::draft::NoiseReductionMode), std::forward_as_tuple(controls::draft::NoiseReductionModeValues));