From patchwork Tue Mar 14 14:48:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Semkowicz X-Patchwork-Id: 18397 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 641CCC329D for ; Tue, 14 Mar 2023 14:50:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 77B2E62716; Tue, 14 Mar 2023 15:50:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1678805404; bh=AnuirllEo1Cb2hAfdXm0mYzwNlQEAJgYBUji5i6UfEE=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=WV8S8MSSRHivk/s/IBc4lmvIHGXaUZbUChln2e+9uiNxfQkxODDttKXXZnd0MkrLT du+oPE5X7n2lt32jh/woxUMtsf4N+9iDmde0vEYY70t+6U0vUZBG/oVW5r2UGXQKkL vIsfqqh/5dGCO7/NCyc2ox88sTcPIQgXNZkfEYvu1UEHBm/1kl8x8QX+kDAzI2hDLF Q7UhsQznsM9s1leyPqmna/HYqBDl0SJbf2JPXHKFSOS04ep84cxuoGUZfaM/vzp4Sn OKudmjQKhekTp2JSfW13JqgFxsriVWSrD9XZFskOcGs/ysZfenDG54CXDqAqfUshgf BwGeMNaPigeRw== Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com [IPv6:2a00:1450:4864:20::132]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0609062709 for ; Tue, 14 Mar 2023 15:49:54 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=thaumatec-com.20210112.gappssmtp.com header.i=@thaumatec-com.20210112.gappssmtp.com header.b="H2M0l2Pq"; dkim-atps=neutral Received: by mail-lf1-x132.google.com with SMTP id d36so20348501lfv.8 for ; Tue, 14 Mar 2023 07:49:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=thaumatec-com.20210112.gappssmtp.com; s=20210112; t=1678805394; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=P60WbR8KYzaoiKJz8hyfEKZ8GtJHFrfGVTuX5vXZzgo=; b=H2M0l2PqSgDQG37X+eCJkhdGRNE+4DbUbSwk8GuogOata95czoBv7AxCifm0IWqizV sdWt/2/sACHEHRj6Y/FHAODqPporu9UPMg1fZGK5gcDA/YEDXrViowVRN3RfqigJlo1+ aJtLW7DoeFrWSSnuz2QNCFtBsB/31LJeoz+aT4XoVkUWOuYQJdwrxv/0TMI6Lgpot3tP Bizia1clt/h+ofJpP1R+YCHfAM7u25hocTG/vzZWLVOu5kpaPGBcxA6zXa+4aWx2xfUG +PWaibssun+cLlT0EnvUoqCB5HgewUhQ5TxI3rLafejwjd74itNqF6sRzYkcfOqeqWfn RH6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678805394; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P60WbR8KYzaoiKJz8hyfEKZ8GtJHFrfGVTuX5vXZzgo=; b=o/tKRYWB6Pi819XNtbmRd5dUSdnhZ6DftZtCyXEzhWNh/g8IfH7olzRbdP0LCqBkSO SGJjHFVHHZ7pt+qDr3eywZjXUCnXqPeHKFXBH/iJ5b5ZDQz/CB5X1wwWL73sHGNLt+kx GsnLIff1rKxK0LPgKY9xKGhZtFHrOoUjpbHycVLZRp+iM1BCqwZf4T9hRucGzUa0whzH IVXW7WcRfFPgyJfQIJAuXAht5B2boW+yMGzgkaVvVMxG/Cf0LxqziARJ02KOqfhHiks3 Dv4IczNQt+Uzliq3vypwT5VC/f6uxCQIHOscSdmBuXkEPKlNixkdXVG52xTCT+DLdUPR gsvw== X-Gm-Message-State: AO0yUKVoGGyRBZNiS9QVlXfxCENve7qPMDRJIhm4eFO5BbRwkU0S0KrP 4I5ypPh3Itk0SHB7F/d6j8WA8R9l4m1vLk6vlw4= X-Google-Smtp-Source: AK7set93uzwMz0v9NgBRQrjUyW+Wia5VWSd22o/ZvrtUhoh30GDQv+Mk2oeChvnBXFugA/EEJYytXQ== X-Received: by 2002:ac2:514b:0:b0:4d8:8ad1:a05f with SMTP id q11-20020ac2514b000000b004d88ad1a05fmr913313lfd.48.1678805394128; Tue, 14 Mar 2023 07:49:54 -0700 (PDT) Received: from localhost.localdomain ([91.90.166.178]) by smtp.gmail.com with ESMTPSA id q25-20020a2eb4b9000000b00295a35f30e1sm470961ljm.115.2023.03.14.07.49.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Mar 2023 07:49:53 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Tue, 14 Mar 2023 15:48:30 +0100 Message-Id: <20230314144834.85193-7-dse@thaumatec.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230314144834.85193-1-dse@thaumatec.com> References: <20230314144834.85193-1-dse@thaumatec.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 06/10] ipa: rkisp1: Add AF algorithm basing on AfHillClimbing X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Daniel Semkowicz via libcamera-devel From: Daniel Semkowicz Reply-To: Daniel Semkowicz Cc: jacopo.mondi@ideasonboard.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add contrast based auto focus implementation for rkisp1 platform using the common AfHillClimbing algorithm. Rockchip ISP AF block allows calculation of sharpness and luminance in up to three user defined windows. If no windows are set, there are some default settings applied for the first window and exposed through the driver. For each frame, use the sharpness value calculated for this default window and feed the hill climbing algorithm with them. Then set the lens position to value calculated by the algorithm. Signed-off-by: Daniel Semkowicz --- src/ipa/rkisp1/algorithms/af.cpp | 107 ++++++++++++++++++++++++++ src/ipa/rkisp1/algorithms/af.h | 39 ++++++++++ src/ipa/rkisp1/algorithms/meson.build | 1 + src/ipa/rkisp1/ipa_context.cpp | 13 ++++ src/ipa/rkisp1/ipa_context.h | 5 ++ 5 files changed, 165 insertions(+) create mode 100644 src/ipa/rkisp1/algorithms/af.cpp create mode 100644 src/ipa/rkisp1/algorithms/af.h diff --git a/src/ipa/rkisp1/algorithms/af.cpp b/src/ipa/rkisp1/algorithms/af.cpp new file mode 100644 index 00000000..abf5d91f --- /dev/null +++ b/src/ipa/rkisp1/algorithms/af.cpp @@ -0,0 +1,107 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Theobroma Systems + * + * af.cpp - RkISP1 AF hill climbing based control algorithm + */ + +#include "af.h" + +/** + * \file af.h + */ + +namespace libcamera::ipa::rkisp1::algorithms { + +/** + * \class Af + * \brief AF contrast based hill climbing control algorithm for RkISP platforms + * + * Auto focus algorithm for RkISP platforms, basing on the common contrast based + * hill climbing auto focus implementation + * (libcamera::ipa::common::algorithms::AfHillClimbing). + * + * This is the platform layer of the algorithm. + * + * Tuning file parameters: + * - **wait-frames-lens:** Number of frames that should be skipped when lens + * position is changed. Lens movement takes some time and statistics measured + * during the lens movement are unstable. Currently there is no way to know + * when lens movement finished and this is a workaround for this. Wait a fixed + * amount of time on each movement. This parameter should be set according + * to the worst case - the number of frames it takes to move lens between + * limit positions. + */ + +LOG_DEFINE_CATEGORY(RkISP1Af) + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Af::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) +{ + waitFramesLens_ = tuningData["wait-frames-lens"].get(1); + + LOG(RkISP1Af, Debug) << "waitFramesLens_: " << waitFramesLens_; + + return af.init(tuningData); +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Af::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) +{ + const auto &lensConfig = context.configuration.lens; + + af.configure(lensConfig.minFocusPosition, lensConfig.maxFocusPosition, + configInfo.outputSize); + + /* + * Lens position is unknown at the startup, so initialize + * the current position to something out of range. + */ + context.activeState.af.lensPosition = lensConfig.maxFocusPosition + 1; + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Af::queueRequest([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + const ControlList &controls) +{ + af.queueRequest(controls); +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Af::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + const rkisp1_stat_buffer *stats, + [[maybe_unused]] ControlList &metadata) +{ + uint32_t sharpness = stats->params.af.window[0].sum; + uint32_t luminance = stats->params.af.window[0].lum; + + LOG(RkISP1Af, Debug) + << "lensPosition: " << context.activeState.af.lensPosition + << ", Sharpness: " << sharpness + << ", Luminance: " << luminance; + + int32_t newLensPosition = af.process(sharpness); + + if (newLensPosition != context.activeState.af.lensPosition) { + context.activeState.af.lensPosition = newLensPosition; + context.activeState.af.applyLensCtrls = true; + af.setFramesToSkip(waitFramesLens_); + } +} + +REGISTER_IPA_ALGORITHM(Af, "Af") + +} /* namespace libcamera::ipa::rkisp1::algorithms */ diff --git a/src/ipa/rkisp1/algorithms/af.h b/src/ipa/rkisp1/algorithms/af.h new file mode 100644 index 00000000..85de0a64 --- /dev/null +++ b/src/ipa/rkisp1/algorithms/af.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2023, Theobroma Systems + * + * af.h - RkISP1 AF hill climbing based control algorithm + */ + +#pragma once + +#include + +#include "libipa/algorithms/af_hill_climbing.h" + +#include "algorithm.h" + +namespace libcamera::ipa::rkisp1::algorithms { + +class Af : public Algorithm +{ +public: + int init(IPAContext &context, const YamlObject &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const rkisp1_stat_buffer *stats, + ControlList &metadata) override; + +private: + ipa::common::algorithms::AfHillClimbing af; + + /* Wait number of frames after changing lens position */ + uint32_t waitFramesLens_; +}; + +} /* namespace libcamera::ipa::rkisp1::algorithms */ diff --git a/src/ipa/rkisp1/algorithms/meson.build b/src/ipa/rkisp1/algorithms/meson.build index 93a48329..ab7e44f3 100644 --- a/src/ipa/rkisp1/algorithms/meson.build +++ b/src/ipa/rkisp1/algorithms/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 rkisp1_ipa_algorithms = files([ + 'af.cpp', 'agc.cpp', 'awb.cpp', 'blc.cpp', diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 401c098f..5c5f80c7 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -134,6 +134,19 @@ namespace libcamera::ipa::rkisp1 { * member may be read by any algorithm, but shall only be written by its owner. */ +/** + * \var IPAActiveState::af + * \brief State for the Automatic Focus Control algorithm + * + * \var IPAActiveState::af.lensPosition + * \brief Lens position calculated by the AF algorithm + * + * \var IPAActiveState::af.applyLensCtrls + * \brief Whether the lens position should be applied + * + * If true, IPA should send new controls to the PH to set new lens position. + */ + /** * \var IPAActiveState::agc * \brief State for the Automatic Gain Control algorithm diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index bfb6e1b7..2c2eec3b 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -58,6 +58,11 @@ struct IPASessionConfiguration { }; struct IPAActiveState { + struct { + int32_t lensPosition; + bool applyLensCtrls; + } af; + struct { struct { uint32_t exposure;