From patchwork Fri Aug 30 07:25:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21057 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0B6F3C324C for ; Fri, 30 Aug 2024 07:26:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A484A634BA; Fri, 30 Aug 2024 09:26:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="KDdDDXqa"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DCC8C633CD for ; Fri, 30 Aug 2024 09:26:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002768; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rKWQHrO/S9qgxIaMcZCbyPXr9JvRVCJjfvb8264XDuI=; b=KDdDDXqa/kgHXlLAWPsvpkAkmSj22pclPzzkETH+5Pfyp2WdMr9033JRAmU3Ijg6G2BEvw WIF7QgigdQrYPZl0tsUb5u6HqrMJnSqkGGoHtSgKr6hFrJaW2jstrmZadahz0NhP7i2tF7 /QJRytdjU+o7/Ex2wcfxb1h+FaegNgA= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-207-Vc3sFP-wNGyM89k8TIFDQw-1; Fri, 30 Aug 2024 03:26:06 -0400 X-MC-Unique: Vc3sFP-wNGyM89k8TIFDQw-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C15FE1955D4F; Fri, 30 Aug 2024 07:26:05 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9BFC219560AA; Fri, 30 Aug 2024 07:26:03 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 01/18] libcamera: software_isp: Remove superfluous includes Date: Fri, 30 Aug 2024 09:25:37 +0200 Message-ID: <20240830072554.180672-2-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Remove unused libcamera internal headers bayer_format.h, framebuffer.h and mapped_frameBuffer.h. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Daniel Scally --- src/libcamera/software_isp/software_isp.cpp | 3 --- 1 file changed, 3 deletions(-) diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 1140372c..a3ae3e22 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -16,10 +16,7 @@ #include #include -#include "libcamera/internal/bayer_format.h" -#include "libcamera/internal/framebuffer.h" #include "libcamera/internal/ipa_manager.h" -#include "libcamera/internal/mapped_framebuffer.h" #include "libcamera/internal/software_isp/debayer_params.h" #include "debayer_cpu.h" From patchwork Fri Aug 30 07:25:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21058 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 54BD1C32C2 for ; Fri, 30 Aug 2024 07:26:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DDF69634CB; Fri, 30 Aug 2024 09:26:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="L2n/CeYD"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2DFFC63458 for ; Fri, 30 Aug 2024 09:26:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002771; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SxcxrWdqckPYJJM31vXC2zzG/WyUmEsjDhhR7dm2j24=; b=L2n/CeYDahV8r62JVWU6crdd/jf95iUK9YZDI293kqVySpwPlJpC7WvZ5vR/045UGR9vB5 eYYWIMkRBZDNQ/+747B2QMGCZ1pRFt6lXuoVBZvXZipIyJv2nsxVLc6bldtvgusupT1Uft oVTaa3hXCgRO7kj/aqS7KSEVVnRof94= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-691-RPYJ8WfxOQe-NGSra4w2cg-1; Fri, 30 Aug 2024 03:26:09 -0400 X-MC-Unique: RPYJ8WfxOQe-NGSra4w2cg-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 742F31955D4F; Fri, 30 Aug 2024 07:26:08 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6281419560A3; Fri, 30 Aug 2024 07:26:06 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 02/18] libcamera: software_isp: Move BlackLevel to libcamera::ipa::soft Date: Fri, 30 Aug 2024 09:25:38 +0200 Message-ID: <20240830072554.180672-3-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" IPA modules use custom namespaces for all their internal components to avoid namespace clashes. The simple IPA module for the software ISP uses libcamera::ipa::soft for this purpose. It however defines an internal class named BlackLevel in the root of the libcamera namespace, making it prone to clashes. Move it to the ipa::soft namespace along with the rest of the code. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Daniel Scally --- src/ipa/simple/black_level.cpp | 5 +++++ src/ipa/simple/black_level.h | 4 ++++ 2 files changed, 9 insertions(+) diff --git a/src/ipa/simple/black_level.cpp b/src/ipa/simple/black_level.cpp index cc490eb5..37e0109c 100644 --- a/src/ipa/simple/black_level.cpp +++ b/src/ipa/simple/black_level.cpp @@ -15,6 +15,8 @@ namespace libcamera { LOG_DEFINE_CATEGORY(IPASoftBL) +namespace ipa::soft { + /** * \class BlackLevel * \brief Object providing black point level for software ISP @@ -85,4 +87,7 @@ void BlackLevel::update(SwIspStats::Histogram &yHistogram) } }; } + +} /* namespace ipa::soft */ + } /* namespace libcamera */ diff --git a/src/ipa/simple/black_level.h b/src/ipa/simple/black_level.h index 5e032f9f..a04230c9 100644 --- a/src/ipa/simple/black_level.h +++ b/src/ipa/simple/black_level.h @@ -14,6 +14,8 @@ namespace libcamera { +namespace ipa::soft { + class BlackLevel { public: @@ -26,4 +28,6 @@ private: bool blackLevelSet_; }; +} /* namespace ipa::soft */ + } /* namespace libcamera */ From patchwork Fri Aug 30 07:25:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21060 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2274AC32D5 for ; Fri, 30 Aug 2024 07:26:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 54BD7634D2; Fri, 30 Aug 2024 09:26:22 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="QEQM5foc"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DD245634A6 for ; Fri, 30 Aug 2024 09:26:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002777; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K5sz67NkMkKrwfn/yYMG0bC9SxrqWTZ2/kfYntqLcX8=; b=QEQM5focbh9LWs8ttQJwK3NcFXnyCMvTZyxIC05/Nkwxx6fFEVr9XHhx8a/ksH4WUha34a yEqE6G+PYPly84IoRpI89f76cGhYfVN+wypkmzP4zN2fGG+yLCXZzRy+ytgOLnuwa8LTno k5x0hOIZRyuttoL4z2TL1E4TxD+Dn3M= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-88-OgVaui_6MpqyAy2kKkYwOQ-1; Fri, 30 Aug 2024 03:26:12 -0400 X-MC-Unique: OgVaui_6MpqyAy2kKkYwOQ-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5159219560AD; Fri, 30 Aug 2024 07:26:11 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0861D19560A3; Fri, 30 Aug 2024 07:26:08 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 03/18] libcamera: software_isp: Define skeletons for IPA refactoring Date: Fri, 30 Aug 2024 09:25:39 +0200 Message-ID: <20240830072554.180672-4-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Software ISP image processing algorithms are currently defined in a simplified way, different from other libcamera pipelines. This is not good for several reasons: - It makes the software ISP code harder to understand due to its different structuring. - Adding more algorithms may make the code harder to understand generally. - Mass libcamera code changes may not be easily applicable to software ISP. - Algorithm sharing with other pipelines is not easily possible. This patch introduces basic software ISP IPA skeletons structured similarly to the other pipelines. The newly added files are currently not used or compiled and the general skeleton structures don't contain anything particular. It is just a preparation step for a larger refactoring and the code will be actually used and extended as needed in followup patches. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Reviewed-by: Daniel Scally Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/soft.mojom | 4 ++ src/ipa/simple/algorithms/algorithm.h | 22 +++++++++++ src/ipa/simple/ipa_context.cpp | 53 +++++++++++++++++++++++++++ src/ipa/simple/ipa_context.h | 33 +++++++++++++++++ src/ipa/simple/meson.build | 1 + src/ipa/simple/module.h | 29 +++++++++++++++ 6 files changed, 142 insertions(+) create mode 100644 src/ipa/simple/algorithms/algorithm.h create mode 100644 src/ipa/simple/ipa_context.cpp create mode 100644 src/ipa/simple/ipa_context.h create mode 100644 src/ipa/simple/module.h diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index 3aa2066e..0fd47bb0 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -8,6 +8,10 @@ module ipa.soft; import "include/libcamera/ipa/core.mojom"; +struct IPAConfigInfo { + libcamera.ControlInfoMap sensorControls; +}; + interface IPASoftInterface { init(libcamera.IPASettings settings, libcamera.SharedFD fdStats, diff --git a/src/ipa/simple/algorithms/algorithm.h b/src/ipa/simple/algorithms/algorithm.h new file mode 100644 index 00000000..41f63170 --- /dev/null +++ b/src/ipa/simple/algorithms/algorithm.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024 Red Hat, Inc. + * + * Software ISP control algorithm interface + */ + +#pragma once + +#include + +#include "module.h" + +namespace libcamera { + +namespace ipa::soft { + +using Algorithm = libcamera::ipa::Algorithm; + +} /* namespace ipa::soft */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/ipa_context.cpp b/src/ipa/simple/ipa_context.cpp new file mode 100644 index 00000000..3c1c7262 --- /dev/null +++ b/src/ipa/simple/ipa_context.cpp @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2021, Google Inc. + * Copyright (C) 2024 Red Hat Inc. + * + * Software ISP IPA Context + */ + +#include "ipa_context.h" + +/** + * \file ipa_context.h + * \brief Context and state information shared between the algorithms + */ + +namespace libcamera::ipa::soft { + +/** + * \struct IPASessionConfiguration + * \brief Session configuration for the IPA module + * + * The session configuration contains all IPA configuration parameters that + * remain constant during the capture session, from IPA module start to stop. + * It is typically set during the configure() operation of the IPA module, but + * may also be updated in the start() operation. + */ + +/** + * \struct IPAActiveState + * \brief The active state of the IPA algorithms + * + * The IPA is fed with the statistics generated from the latest frame processed. + * The statistics are then processed by the IPA algorithms to compute parameters + * required for the next frame capture and processing. The current state of the + * algorithms is reflected through the IPAActiveState to store the values most + * recently computed by the IPA algorithms. + */ + +/** + * \struct IPAContext + * \brief Global IPA context data shared between all algorithms + * + * \var IPAContext::configuration + * \brief The IPA session configuration, immutable during the session + * + * \var IPAContext::frameContexts + * \brief Ring buffer of the IPAFrameContext(s) + * + * \var IPAContext::activeState + * \brief The current state of IPA algorithms + */ + +} /* namespace libcamera::ipa::soft */ diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h new file mode 100644 index 00000000..e7d8b8df --- /dev/null +++ b/src/ipa/simple/ipa_context.h @@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024 Red Hat, Inc. + * + * Simple pipeline IPA Context + */ + +#pragma once + +#include + +namespace libcamera { + +namespace ipa::soft { + +struct IPASessionConfiguration { +}; + +struct IPAActiveState { +}; + +struct IPAFrameContext : public FrameContext { +}; + +struct IPAContext { + IPASessionConfiguration configuration; + IPAActiveState activeState; + FCQueue frameContexts; +}; + +} /* namespace ipa::soft */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/meson.build b/src/ipa/simple/meson.build index b297e1d2..7757f99b 100644 --- a/src/ipa/simple/meson.build +++ b/src/ipa/simple/meson.build @@ -3,6 +3,7 @@ ipa_name = 'ipa_soft_simple' soft_simple_sources = files([ + 'ipa_context.cpp', 'soft_simple.cpp', 'black_level.cpp', ]) diff --git a/src/ipa/simple/module.h b/src/ipa/simple/module.h new file mode 100644 index 00000000..5aaf8eb0 --- /dev/null +++ b/src/ipa/simple/module.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024 Red Hat, Inc. + * + * Software ISP IPA Module + */ + +#pragma once + +#include + +#include "libcamera/internal/software_isp/debayer_params.h" +#include "libcamera/internal/software_isp/swisp_stats.h" + +#include + +#include "ipa_context.h" +#include "soft_ipa_interface.h" + +namespace libcamera { + +namespace ipa::soft { + +using Module = ipa::Module; + +} /* namespace ipa::soft */ + +} /* namespace libcamera */ From patchwork Fri Aug 30 07:25:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21059 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CA4AEC324C for ; Fri, 30 Aug 2024 07:26:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 55FF3634CD; Fri, 30 Aug 2024 09:26:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="aFFhFBXN"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 84E3A63458 for ; Fri, 30 Aug 2024 09:26:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002776; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MX9NCOwcE1ykHIPz/zU/TVZmkBNqq1TkDXFuf3lnGdo=; b=aFFhFBXN8Go+3YZAlkvgWfMPSJwFDt105s/bknhU2qsKCQmT3oqWpsAW1vyT/K673w5+Y0 ho5o2z3G4ElYJW/rJEzJ9q55X//1h9RgXgQzrYiGdjA9F1ZnrOU8RKS19cjr0QUnt1pt6C XBJhHLdUlp+7QK1yf+tldZ6IEnCzOtI= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-54-ram6AAMEO_m9ULK0p2eSHQ-1; Fri, 30 Aug 2024 03:26:15 -0400 X-MC-Unique: ram6AAMEO_m9ULK0p2eSHQ-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D9D1A19560A3; Fri, 30 Aug 2024 07:26:13 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B1C6A19560A3; Fri, 30 Aug 2024 07:26:11 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 04/18] libcamera: software_isp: Let IPASoftSimple inherit Module Date: Fri, 30 Aug 2024 09:25:40 +0200 Message-ID: <20240830072554.180672-5-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Module class is a base class for all IPA modules. In addition, implement logPrefix() of the module for the softIPA. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Reviewed-by: Daniel Scally Reviewed-by: Laurent Pinchart --- src/ipa/simple/soft_simple.cpp | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index b7746ce0..72321f44 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -30,6 +30,7 @@ #include "libipa/camera_sensor_helper.h" #include "black_level.h" +#include "module.h" namespace libcamera { LOG_DEFINE_CATEGORY(IPASoft) @@ -54,7 +55,7 @@ static constexpr float kExposureOptimal = kExposureBinsCount / 2.0; */ static constexpr float kExposureSatisfactory = 0.2; -class IPASoftSimple : public ipa::soft::IPASoftInterface +class IPASoftSimple : public ipa::soft::IPASoftInterface, public Module { public: IPASoftSimple() @@ -76,6 +77,9 @@ public: void processStats(const ControlList &sensorControls) override; +protected: + std::string logPrefix() const override; + private: void updateExposure(double exposureMSV); @@ -421,6 +425,11 @@ void IPASoftSimple::updateExposure(double exposureMSV) again_ = std::clamp(again_, againMin_, againMax_); } +std::string IPASoftSimple::logPrefix() const +{ + return "IPASoft"; +} + } /* namespace ipa::soft */ /* From patchwork Fri Aug 30 07:25:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21061 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0AD5DC32C2 for ; Fri, 30 Aug 2024 07:26:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 90C3B634CB; Fri, 30 Aug 2024 09:26:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="CiLFGxYq"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CF53A634BA for ; Fri, 30 Aug 2024 09:26:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002780; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q2v30WFtc+QLkbchyPhtbVVN0nHMzx8KfR3ygJ58HBw=; b=CiLFGxYqDbU66Z5pkBLL+wE6ZqT7G8UHGFNTI0fXt3QRjmtOp6dsNedyizGsP3Cz0F5Yvd LtT1y0JXW21tVJ7TP2FHQshsQk30mALRKLPSJMgo4WECZGGRAHtkW3T2k/JRcz83KytqP+ VTPoLjRjntyIxiRtdRjUPiyHD4fSBhY= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-523-Io6m27VJML6RX3GpbSi-6g-1; Fri, 30 Aug 2024 03:26:17 -0400 X-MC-Unique: Io6m27VJML6RX3GpbSi-6g-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8FCE819560B7; Fri, 30 Aug 2024 07:26:16 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7072519560A3; Fri, 30 Aug 2024 07:26:14 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 05/18] libcamera: software_isp: Make stats frame and buffer aware Date: Fri, 30 Aug 2024 09:25:41 +0200 Message-ID: <20240830072554.180672-6-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch adds frame and bufferId arguments to stats related calls. Although the parameters are currently unused, because frame ids are not tracked and used and the stats buffer is passed around directly rather than being referred by its id, they bring the internal APIs closer to their counterparts in hardware pipelines. It serves as a preparation for followup patches that will introduce: - Frame number tracking in order to switch to DelayedControls (software ISP TODO #11 + #12). - A ring buffer for stats in order to improve passing the stats (software ISP TODO #2). Frame and buffer ids are unrelated for the given purposes but since they are passed together at the same places, the change is implemented as a single patch rather than two, basically the same, patches. Signed-off-by: Milan Zamazal Reviewed-by: Laurent Pinchart --- .../libcamera/internal/software_isp/software_isp.h | 8 +++++--- include/libcamera/ipa/soft.mojom | 4 +++- src/ipa/simple/soft_simple.cpp | 7 +++++-- src/libcamera/pipeline/simple/simple.cpp | 8 +++++--- src/libcamera/software_isp/debayer_cpu.cpp | 8 +++++++- src/libcamera/software_isp/software_isp.cpp | 11 +++++++---- src/libcamera/software_isp/swstats_cpu.cpp | 6 ++++-- src/libcamera/software_isp/swstats_cpu.h | 4 ++-- 8 files changed, 38 insertions(+), 18 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index f8e00003..3602bce8 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -66,7 +67,8 @@ public: int exportBuffers(const Stream *stream, unsigned int count, std::vector> *buffers); - void processStats(const ControlList &sensorControls); + void processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls); int start(); void stop(); @@ -78,13 +80,13 @@ public: Signal inputBufferReady; Signal outputBufferReady; - Signal<> ispStatsReady; + Signal ispStatsReady; Signal setSensorControls; private: void saveIspParams(); void setSensorCtrls(const ControlList &sensorControls); - void statsReady(); + void statsReady(uint32_t frame, uint32_t bufferId); void inputReady(FrameBuffer *input); void outputReady(FrameBuffer *output); diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index 0fd47bb0..cc5c37b2 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -23,7 +23,9 @@ interface IPASoftInterface { configure(libcamera.ControlInfoMap sensorCtrlInfoMap) => (int32 ret); - [async] processStats(libcamera.ControlList sensorControls); + [async] processStats(uint32 frame, + uint32 bufferId, + libcamera.ControlList sensorControls); }; interface IPASoftEventInterface { diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 72321f44..12b5245e 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -75,7 +75,8 @@ public: int start() override; void stop() override; - void processStats(const ControlList &sensorControls) override; + void processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls) override; protected: std::string logPrefix() const override; @@ -249,7 +250,9 @@ void IPASoftSimple::stop() { } -void IPASoftSimple::processStats(const ControlList &sensorControls) +void IPASoftSimple::processStats([[maybe_unused]] const uint32_t frame, + [[maybe_unused]] const uint32_t bufferId, + const ControlList &sensorControls) { SwIspStats::Histogram histogram = stats_->yHistogram; if (ignoreUpdates_ > 0) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 1e7ec7d9..48a568da 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -291,7 +292,7 @@ private: void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); - void ispStatsReady(); + void ispStatsReady(uint32_t frame, uint32_t bufferId); void setSensorControls(const ControlList &sensorControls); }; @@ -891,10 +892,11 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) pipe->completeRequest(request); } -void SimpleCameraData::ispStatsReady() +void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) { /* \todo Use the DelayedControls class */ - swIsp_->processStats(sensor_->getControls({ V4L2_CID_ANALOGUE_GAIN, + swIsp_->processStats(frame, bufferId, + sensor_->getControls({ V4L2_CID_ANALOGUE_GAIN, V4L2_CID_EXPOSURE })); } diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 077f7f4b..2a2e7edb 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -777,7 +777,13 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams } } - stats_->finishFrame(); + /* + * Frame and buffer ids are currently not used, so pass zeros as parameters. + * + * \todo Pass real values once frame is passed here and stats buffer passing + * is changed. + */ + stats_->finishFrame(0, 0); outputBufferReady.emit(output); inputBufferReady.emit(input); } diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index a3ae3e22..a3855568 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -155,15 +155,18 @@ SoftwareIsp::~SoftwareIsp() /** * \brief Process the statistics gathered + * \param[in] frame The frame number + * \param[in] bufferId ID of the statistics buffer * \param[in] sensorControls The sensor controls * * Requests the IPA to calculate new parameters for ISP and new control * values for the sensor. */ -void SoftwareIsp::processStats(const ControlList &sensorControls) +void SoftwareIsp::processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls) { ASSERT(ipa_); - ipa_->processStats(sensorControls); + ipa_->processStats(frame, bufferId, sensorControls); } /** @@ -349,9 +352,9 @@ void SoftwareIsp::setSensorCtrls(const ControlList &sensorControls) setSensorControls.emit(sensorControls); } -void SoftwareIsp::statsReady() +void SoftwareIsp::statsReady(uint32_t frame, uint32_t bufferId) { - ispStatsReady.emit(); + ispStatsReady.emit(frame, bufferId); } void SoftwareIsp::inputReady(FrameBuffer *input) diff --git a/src/libcamera/software_isp/swstats_cpu.cpp b/src/libcamera/software_isp/swstats_cpu.cpp index 815c4d4f..c520c806 100644 --- a/src/libcamera/software_isp/swstats_cpu.cpp +++ b/src/libcamera/software_isp/swstats_cpu.cpp @@ -311,13 +311,15 @@ void SwStatsCpu::startFrame(void) /** * \brief Finish statistics calculation for the current frame + * \param[in] frame The frame number + * \param[in] bufferId ID of the statistics buffer * * This may only be called after a successful setWindow() call. */ -void SwStatsCpu::finishFrame(void) +void SwStatsCpu::finishFrame(uint32_t frame, uint32_t bufferId) { *sharedStats_ = stats_; - statsReady.emit(); + statsReady.emit(frame, bufferId); } /** diff --git a/src/libcamera/software_isp/swstats_cpu.h b/src/libcamera/software_isp/swstats_cpu.h index 363e326f..26a2f462 100644 --- a/src/libcamera/software_isp/swstats_cpu.h +++ b/src/libcamera/software_isp/swstats_cpu.h @@ -41,7 +41,7 @@ public: int configure(const StreamConfiguration &inputCfg); void setWindow(const Rectangle &window); void startFrame(); - void finishFrame(); + void finishFrame(uint32_t frame, uint32_t bufferId); void processLine0(unsigned int y, const uint8_t *src[]) { @@ -61,7 +61,7 @@ public: (this->*stats2_)(src); } - Signal<> statsReady; + Signal statsReady; private: using statsProcessFn = void (SwStatsCpu::*)(const uint8_t *src[]); From patchwork Fri Aug 30 07:25:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21062 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BF4DEC32D6 for ; Fri, 30 Aug 2024 07:26:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 38D94634D5; Fri, 30 Aug 2024 09:26:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="hO8dMZlD"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0832A634D3 for ; Fri, 30 Aug 2024 09:26:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002782; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1KZmM50ODUWNoDjYHlqfTIxDIdoCNLWj3qYt2cCvtr4=; b=hO8dMZlDXvtPKzgxnuS4OaMDLmst2kcCKC/RNI2zjwKhIBPK2OKdaFka8nMOdKVghRT6Rv kLOO3SYpprPUsUs2lWEGx6uO1IkMQJBtqB0KhY6E+xjjTNegx6w7Zp9vkUxHl3NnkSQ07P XO6NAgGG5SZnYlo2ArWTdoq/jHO6r2o= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-441-YTEREavPOaeUDVOJrstbmQ-1; Fri, 30 Aug 2024 03:26:20 -0400 X-MC-Unique: YTEREavPOaeUDVOJrstbmQ-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 31EB91955D4C; Fri, 30 Aug 2024 07:26:19 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 04EA519560A3; Fri, 30 Aug 2024 07:26:16 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 06/18] libcamera: software_isp: Remove final dots in debayer.cpp docstrings Date: Fri, 30 Aug 2024 09:25:42 +0200 Message-ID: <20240830072554.180672-7-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The policy and the style checker require that \brief, \param and \return texts don't finish with a dot. This needs to be fixed in debayer.cpp. Also leading spaces in a \return statement are removed from there. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Reviewed-by: Daniel Scally Reviewed-by: Laurent Pinchart --- src/libcamera/software_isp/debayer.cpp | 48 +++++++++++++------------- 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index f4a299d5..db26c380 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -58,47 +58,47 @@ Debayer::~Debayer() /** * \fn int Debayer::configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfgs) - * \brief Configure the debayer object according to the passed in parameters. - * \param[in] inputCfg The input configuration. - * \param[in] outputCfgs The output configurations. + * \brief Configure the debayer object according to the passed in parameters + * \param[in] inputCfg The input configuration + * \param[in] outputCfgs The output configurations * - * \return 0 on success, a negative errno on failure. + * \return 0 on success, a negative errno on failure */ /** * \fn Size Debayer::patternSize(PixelFormat inputFormat) - * \brief Get the width and height at which the bayer pattern repeats. - * \param[in] inputFormat The input format. + * \brief Get the width and height at which the bayer pattern repeats + * \param[in] inputFormat The input format * * Valid sizes are: 2x2, 4x2 or 4x4. * - * \return Pattern size or an empty size for unsupported inputFormats. + * \return Pattern size or an empty size for unsupported inputFormats */ /** * \fn std::vector Debayer::formats(PixelFormat inputFormat) - * \brief Get the supported output formats. - * \param[in] inputFormat The input format. + * \brief Get the supported output formats + * \param[in] inputFormat The input format * - * \return All supported output formats or an empty vector if there are none. + * \return All supported output formats or an empty vector if there are none */ /** * \fn std::tuple Debayer::strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) - * \brief Get the stride and the frame size. - * \param[in] outputFormat The output format. - * \param[in] size The output size. + * \brief Get the stride and the frame size + * \param[in] outputFormat The output format + * \param[in] size The output size * * \return A tuple of the stride and the frame size, or a tuple with 0,0 if - * there is no valid output config. + * there is no valid output config */ /** * \fn void Debayer::process(FrameBuffer *input, FrameBuffer *output, DebayerParams params) - * \brief Process the bayer data into the requested format. - * \param[in] input The input buffer. - * \param[in] output The output buffer. - * \param[in] params The parameters to be used in debayering. + * \brief Process the bayer data into the requested format + * \param[in] input The input buffer + * \param[in] output The output buffer + * \param[in] params The parameters to be used in debayering * * \note DebayerParams is passed by value deliberately so that a copy is passed * when this is run in another thread by invokeMethod(). @@ -106,21 +106,21 @@ Debayer::~Debayer() /** * \fn virtual SizeRange Debayer::sizes(PixelFormat inputFormat, const Size &inputSize) - * \brief Get the supported output sizes for the given input format and size. - * \param[in] inputFormat The input format. - * \param[in] inputSize The input size. + * \brief Get the supported output sizes for the given input format and size + * \param[in] inputFormat The input format + * \param[in] inputSize The input size * - * \return The valid size ranges or an empty range if there are none. + * \return The valid size ranges or an empty range if there are none */ /** * \var Signal Debayer::inputBufferReady - * \brief Signals when the input buffer is ready. + * \brief Signals when the input buffer is ready */ /** * \var Signal Debayer::outputBufferReady - * \brief Signals when the output buffer is ready. + * \brief Signals when the output buffer is ready */ } /* namespace libcamera */ From patchwork Fri Aug 30 07:25:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21063 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D08FAC324C for ; Fri, 30 Aug 2024 07:26:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 66C20634D5; Fri, 30 Aug 2024 09:26:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="NiWdBtaO"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E9365634A6 for ; Fri, 30 Aug 2024 09:26:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002786; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=laVMS9tWTJVwDwJX0wk8aN8Lf7oY7XY3hIONyTbhkQI=; b=NiWdBtaOjAK5EzLuCJNAfjW3aNC2yM1SoM1Chw7ybr9uQ8HIslkN5beBdwMNiPQWMv9dhF rZlREcJh+cbR2ZSxgTQ8b9KyjaRqMjT4J8pUeRad6uxZht5NLkH5gUxGWDdCn8vLP6+RRQ WLudfj8IUnpPXvR2kQWh9as18eDnYzc= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-655-9T2v7GoMPVewzLDojHu0MQ-1; Fri, 30 Aug 2024 03:26:22 -0400 X-MC-Unique: 9T2v7GoMPVewzLDojHu0MQ-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D1F511955D4D; Fri, 30 Aug 2024 07:26:21 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id C078819560A3; Fri, 30 Aug 2024 07:26:19 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 07/18] libcamera: software_isp: Track and pass frame ids Date: Fri, 30 Aug 2024 09:25:43 +0200 Message-ID: <20240830072554.180672-8-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" A previous preparation patch implemented passing frame ids to stats processing but without actual meaningful frame id value passed there. This patch extends that by actually providing the frame id and passing it through to the stats processor. The frame id is taken from the request sequence number, the same as in hardware pipelines. Signed-off-by: Milan Zamazal --- .../libcamera/internal/software_isp/software_isp.h | 4 ++-- src/libcamera/pipeline/simple/simple.cpp | 8 +++++++- src/libcamera/software_isp/debayer.cpp | 3 ++- src/libcamera/software_isp/debayer.h | 2 +- src/libcamera/software_isp/debayer_cpu.cpp | 9 ++++----- src/libcamera/software_isp/debayer_cpu.h | 2 +- src/libcamera/software_isp/software_isp.cpp | 11 +++++++---- 7 files changed, 24 insertions(+), 15 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index 3602bce8..3a84418e 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -73,10 +73,10 @@ public: int start(); void stop(); - int queueBuffers(FrameBuffer *input, + int queueBuffers(uint32_t frame, FrameBuffer *input, const std::map &outputs); - void process(FrameBuffer *input, FrameBuffer *output); + void process(uint32_t frame, FrameBuffer *input, FrameBuffer *output); Signal inputBufferReady; Signal outputBufferReady; diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 48a568da..ebec592a 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -865,7 +865,13 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) if (converter_) converter_->queueBuffers(buffer, conversionQueue_.front()); else - swIsp_->queueBuffers(buffer, conversionQueue_.front()); + /* + * request->sequence() cannot be retrieved from `buffer' inside + * queueBuffers because unique_ptr's make buffer->request() invalid + * already here. + */ + swIsp_->queueBuffers(request->sequence(), buffer, + conversionQueue_.front()); conversionQueue_.pop(); return; diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index db26c380..f0b83261 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -94,8 +94,9 @@ Debayer::~Debayer() */ /** - * \fn void Debayer::process(FrameBuffer *input, FrameBuffer *output, DebayerParams params) + * \fn void Debayer::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, DebayerParams params) * \brief Process the bayer data into the requested format + * \param[in] frame The frame number * \param[in] input The input buffer * \param[in] output The output buffer * \param[in] params The parameters to be used in debayering diff --git a/src/libcamera/software_isp/debayer.h b/src/libcamera/software_isp/debayer.h index c151fe5d..d7ca060d 100644 --- a/src/libcamera/software_isp/debayer.h +++ b/src/libcamera/software_isp/debayer.h @@ -40,7 +40,7 @@ public: virtual std::tuple strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) = 0; - virtual void process(FrameBuffer *input, FrameBuffer *output, DebayerParams params) = 0; + virtual void process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, DebayerParams params) = 0; virtual SizeRange sizes(PixelFormat inputFormat, const Size &inputSize) = 0; diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 2a2e7edb..f7b3a7d1 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -724,7 +724,7 @@ static inline int64_t timeDiff(timespec &after, timespec &before) (int64_t)after.tv_nsec - (int64_t)before.tv_nsec; } -void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams params) +void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, DebayerParams params) { timespec frameStartTime; @@ -778,12 +778,11 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams } /* - * Frame and buffer ids are currently not used, so pass zeros as parameters. + * Buffer ids are currently not used, so pass zeros as its parameter. * - * \todo Pass real values once frame is passed here and stats buffer passing - * is changed. + * \todo Pass real bufferId once stats buffer passing is changed. */ - stats_->finishFrame(0, 0); + stats_->finishFrame(frame, 0); outputBufferReady.emit(output); inputBufferReady.emit(input); } diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 8237a64b..2c47e7c6 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -36,7 +36,7 @@ public: std::vector formats(PixelFormat input); std::tuple strideAndFrameSize(const PixelFormat &outputFormat, const Size &size); - void process(FrameBuffer *input, FrameBuffer *output, DebayerParams params); + void process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, DebayerParams params); SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); /** diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index a3855568..c5db45ae 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -14,6 +14,7 @@ #include #include +#include #include #include "libcamera/internal/ipa_manager.h" @@ -278,12 +279,13 @@ int SoftwareIsp::exportBuffers(const Stream *stream, unsigned int count, /** * \brief Queue buffers to Software ISP + * \param[in] frame The frame number * \param[in] input The input framebuffer * \param[in] outputs The container holding the output stream pointers and * their respective frame buffer outputs * \return 0 on success, a negative errno on failure */ -int SoftwareIsp::queueBuffers(FrameBuffer *input, +int SoftwareIsp::queueBuffers(uint32_t frame, FrameBuffer *input, const std::map &outputs) { /* @@ -301,7 +303,7 @@ int SoftwareIsp::queueBuffers(FrameBuffer *input, } for (auto iter = outputs.begin(); iter != outputs.end(); iter++) - process(input, iter->second); + process(frame, input, iter->second); return 0; } @@ -333,13 +335,14 @@ void SoftwareIsp::stop() /** * \brief Passes the input framebuffer to the ISP worker to process + * \param[in] frame The frame number * \param[in] input The input framebuffer * \param[out] output The framebuffer to write the processed frame to */ -void SoftwareIsp::process(FrameBuffer *input, FrameBuffer *output) +void SoftwareIsp::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output) { debayer_->invokeMethod(&DebayerCpu::process, - ConnectionTypeQueued, input, output, debayerParams_); + ConnectionTypeQueued, frame, input, output, debayerParams_); } void SoftwareIsp::saveIspParams() From patchwork Fri Aug 30 07:25:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21064 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8F15DC32D7 for ; Fri, 30 Aug 2024 07:26:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CBCC5634DF; Fri, 30 Aug 2024 09:26:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="heEmtB/v"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 10609634C8 for ; Fri, 30 Aug 2024 09:26:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002789; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4OojRvR7hV5VSh6rZzOTFwKTXnvVsUvy9KHsSwPlpII=; b=heEmtB/vkiR24evP4VMsLx2zgq/8niqNzwHtldEm0dk4Gy2hSqT/t5qmwiV/ax5349d/mk RyWmDQPxpz7jkPUV3gVqfQmzb9PrtMKQF9ftIgGurarF80bPgfVLtBKD4DfvSqGhp+m1sW 5LgKbvgLTsM6u4v1GJ0ldS+mVUxBj7w= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-551-HIk5TId5OVeBNXrrwLQmBw-1; Fri, 30 Aug 2024 03:26:25 -0400 X-MC-Unique: HIk5TId5OVeBNXrrwLQmBw-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A73071955D47; Fri, 30 Aug 2024 07:26:24 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 74B5819560A3; Fri, 30 Aug 2024 07:26:22 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 08/18] libcamera: software_isp: Create algorithms Date: Fri, 30 Aug 2024 09:25:44 +0200 Message-ID: <20240830072554.180672-9-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We are ready to introduce algorithms now. First, let's create algorithms. The algorithms are not called yet, calls to them will be added in followup patches. The maximum number of contexts is set to the same value as in hardware pipelines. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Reviewed-by: Daniel Scally Reviewed-by: Laurent Pinchart --- src/ipa/simple/algorithms/meson.build | 4 ++++ src/ipa/simple/data/uncalibrated.yaml | 1 + src/ipa/simple/meson.build | 7 +++++-- src/ipa/simple/soft_simple.cpp | 14 ++++++++++++++ 4 files changed, 24 insertions(+), 2 deletions(-) create mode 100644 src/ipa/simple/algorithms/meson.build diff --git a/src/ipa/simple/algorithms/meson.build b/src/ipa/simple/algorithms/meson.build new file mode 100644 index 00000000..31d26e43 --- /dev/null +++ b/src/ipa/simple/algorithms/meson.build @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: CC0-1.0 + +soft_simple_ipa_algorithms = files([ +]) diff --git a/src/ipa/simple/data/uncalibrated.yaml b/src/ipa/simple/data/uncalibrated.yaml index ff981a1a..2cdc39a8 100644 --- a/src/ipa/simple/data/uncalibrated.yaml +++ b/src/ipa/simple/data/uncalibrated.yaml @@ -2,4 +2,5 @@ %YAML 1.1 --- version: 1 +algorithms: ... diff --git a/src/ipa/simple/meson.build b/src/ipa/simple/meson.build index 7757f99b..dcd7c70a 100644 --- a/src/ipa/simple/meson.build +++ b/src/ipa/simple/meson.build @@ -1,5 +1,8 @@ # SPDX-License-Identifier: CC0-1.0 +subdir('algorithms') +subdir('data') + ipa_name = 'ipa_soft_simple' soft_simple_sources = files([ @@ -8,6 +11,8 @@ soft_simple_sources = files([ 'black_level.cpp', ]) +soft_simple_sources += soft_simple_ipa_algorithms + mod = shared_module(ipa_name, soft_simple_sources, name_prefix : '', include_directories : [ipa_includes], @@ -24,6 +29,4 @@ if ipa_sign_module build_by_default : true) endif -subdir('data') - ipa_names += ipa_name diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 12b5245e..79ed4891 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -54,12 +54,15 @@ static constexpr float kExposureOptimal = kExposureBinsCount / 2.0; * enough to prevent the exposure from wobbling around the optimal value. */ static constexpr float kExposureSatisfactory = 0.2; +/* Maximum number of frame contexts to be held */ +static constexpr uint32_t kMaxFrameContexts = 16; class IPASoftSimple : public ipa::soft::IPASoftInterface, public Module { public: IPASoftSimple() : params_(nullptr), stats_(nullptr), blackLevel_(BlackLevel()), + context_({ {}, {}, { kMaxFrameContexts } }), ignoreUpdates_(0) { } @@ -93,6 +96,8 @@ private: static constexpr unsigned int kGammaLookupSize = 1024; std::array gammaTable_; int lastBlackLevel_ = -1; + /* Local parameter storage */ + struct IPAContext context_; int32_t exposureMin_, exposureMax_; int32_t exposure_; @@ -139,6 +144,15 @@ int IPASoftSimple::init(const IPASettings &settings, unsigned int version = (*data)["version"].get(0); LOG(IPASoft, Debug) << "Tuning file version " << version; + if (!data->contains("algorithms")) { + LOG(IPASoft, Error) << "Tuning file doesn't contain algorithms"; + return -EINVAL; + } + + int ret = createAlgorithms(context_, (*data)["algorithms"]); + if (ret) + return ret; + params_ = nullptr; stats_ = nullptr; From patchwork Fri Aug 30 07:25:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21065 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 118F6C32D5 for ; Fri, 30 Aug 2024 07:26:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 922CE634DE; Fri, 30 Aug 2024 09:26:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Ky+LENSD"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A29E0634CD for ; Fri, 30 Aug 2024 09:26:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002791; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0BO/w5QMYz6bbvwUpiY8EPkIgmGoWmsNkoodaDvNTIU=; b=Ky+LENSDrFjQVkbNVFlkl2V2yjfPzVuq3uXtVkpvBzVkm5N9vLOvcE93G6yToojbNSSLiU 3pWL5sN5FxVrEx3i0lkmor+aKlgKSU5BYgHgyhaTAhM3k+mbKt+XF95pciSy792s3mRxfw UQX8/vz/YkwQadlfwQCWi+0AN5f3/+k= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-290-GibWb7dcOcuQb4stE9zJXw-1; Fri, 30 Aug 2024 03:26:28 -0400 X-MC-Unique: GibWb7dcOcuQb4stE9zJXw-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1546A1956095; Fri, 30 Aug 2024 07:26:27 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 01B1D19560A3; Fri, 30 Aug 2024 07:26:24 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 09/18] libcamera: software_isp: Call Algorithm::configure Date: Fri, 30 Aug 2024 09:25:45 +0200 Message-ID: <20240830072554.180672-10-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch adds Algorithm::configure call for the defined algorithms. This is preparation only since there are currently no Algorithm based algorithms defined. A part of this change is passing IPAConfigInfo instead of ControlInfoMap to configure() calls as this is what Algorithm::configure expects. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- .../libcamera/internal/software_isp/software_isp.h | 2 +- include/libcamera/ipa/soft.mojom | 2 +- src/ipa/simple/module.h | 2 ++ src/ipa/simple/soft_simple.cpp | 12 +++++++++--- src/libcamera/pipeline/simple/simple.cpp | 11 +++++++---- src/libcamera/software_isp/software_isp.cpp | 7 ++++--- 6 files changed, 24 insertions(+), 12 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index 3a84418e..c5d5a46f 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -62,7 +62,7 @@ public: int configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfgs, - const ControlInfoMap &sensorControls); + const ipa::soft::IPAConfigInfo &configInfo); int exportBuffers(const Stream *stream, unsigned int count, std::vector> *buffers); diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index cc5c37b2..6b049d20 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -20,7 +20,7 @@ interface IPASoftInterface { => (int32 ret); start() => (int32 ret); stop(); - configure(libcamera.ControlInfoMap sensorCtrlInfoMap) + configure(IPAConfigInfo configInfo) => (int32 ret); [async] processStats(uint32 frame, diff --git a/src/ipa/simple/module.h b/src/ipa/simple/module.h index 5aaf8eb0..d1321f59 100644 --- a/src/ipa/simple/module.h +++ b/src/ipa/simple/module.h @@ -9,6 +9,8 @@ #include +#include + #include "libcamera/internal/software_isp/debayer_params.h" #include "libcamera/internal/software_isp/swisp_stats.h" diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 79ed4891..0ea62266 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -73,7 +73,7 @@ public: const SharedFD &fdStats, const SharedFD &fdParams, const ControlInfoMap &sensorInfoMap) override; - int configure(const ControlInfoMap &sensorInfoMap) override; + int configure(const IPAConfigInfo &configInfo) override; int start() override; void stop() override; @@ -207,9 +207,9 @@ int IPASoftSimple::init(const IPASettings &settings, return 0; } -int IPASoftSimple::configure(const ControlInfoMap &sensorInfoMap) +int IPASoftSimple::configure(const IPAConfigInfo &configInfo) { - sensorInfoMap_ = sensorInfoMap; + sensorInfoMap_ = configInfo.sensorControls; const ControlInfo &exposureInfo = sensorInfoMap_.find(V4L2_CID_EXPOSURE)->second; const ControlInfo &gainInfo = sensorInfoMap_.find(V4L2_CID_ANALOGUE_GAIN)->second; @@ -248,6 +248,12 @@ int IPASoftSimple::configure(const ControlInfoMap &sensorInfoMap) againMinStep_ = 1.0; } + for (auto const &algo : algorithms()) { + int ret = algo->configure(context_, configInfo); + if (ret) + return ret; + } + LOG(IPASoft, Info) << "Exposure " << exposureMin_ << "-" << exposureMax_ << ", gain " << againMin_ << "-" << againMax_ << " (" << againMinStep_ << ")"; diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index ebec592a..834b33d9 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1294,10 +1294,13 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) inputCfg.stride = captureFormat.planes[0].bpl; inputCfg.bufferCount = kNumInternalBuffers; - return data->converter_ - ? data->converter_->configure(inputCfg, outputCfgs) - : data->swIsp_->configure(inputCfg, outputCfgs, - data->sensor_->controls()); + if (data->converter_) { + return data->converter_->configure(inputCfg, outputCfgs); + } else { + ipa::soft::IPAConfigInfo configInfo; + configInfo.sensorControls = data->sensor_->controls(); + return data->swIsp_->configure(inputCfg, outputCfgs, configInfo); + } } int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index c5db45ae..8f1c0f65 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -223,16 +223,17 @@ SoftwareIsp::strideAndFrameSize(const PixelFormat &outputFormat, const Size &siz * \brief Configure the SoftwareIsp object according to the passed in parameters * \param[in] inputCfg The input configuration * \param[in] outputCfgs The output configurations - * \param[in] sensorControls ControlInfoMap of the controls supported by the sensor + * \param[in] configInfo The IPA configuration data, received from the pipeline + * handler * \return 0 on success, a negative errno on failure */ int SoftwareIsp::configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfgs, - const ControlInfoMap &sensorControls) + const ipa::soft::IPAConfigInfo &configInfo) { ASSERT(ipa_ && debayer_); - int ret = ipa_->configure(sensorControls); + int ret = ipa_->configure(configInfo); if (ret < 0) return ret; From patchwork Fri Aug 30 07:25:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21066 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 178B2C32D8 for ; Fri, 30 Aug 2024 07:26:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 48339634DF; Fri, 30 Aug 2024 09:26:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="eLPy9df3"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EDE9F634D5 for ; Fri, 30 Aug 2024 09:26:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002793; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QhxqDGmaspBPYM04rM9qZqhO81940aSHi9UFpNeLeDM=; b=eLPy9df3wB6D1h2SebVLN/iqT/3hoTm2R7T7TFTNimWmG0eZ4qYDvVb3asw/50MjRZkhI+ gcM3wsOrr06mYTst2koGLRaIoFOq0pdNo4JDG3jiZv4pHINaKcl2kTWKQfjJttswOI0VKv R4nTblvE3A2NwV+I+JldcU7C14uOClQ= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-478-Z5XhZ6AOO2K7T8GTfbSY9A-1; Fri, 30 Aug 2024 03:26:30 -0400 X-MC-Unique: Z5XhZ6AOO2K7T8GTfbSY9A-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D00061955D4F; Fri, 30 Aug 2024 07:26:29 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A424F19560A3; Fri, 30 Aug 2024 07:26:27 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 10/18] libcamera: software_isp: Call Algorithm::queueRequest Date: Fri, 30 Aug 2024 09:25:46 +0200 Message-ID: <20240830072554.180672-11-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch adds Algorithm::queueRequest call for the defined algorithms. As there are currently no control knobs in software ISP nor the corresponding queueRequest call chain, the patch also introduces the queueRequest methods and calls from the pipeline to the IPA. This is preparation only since there are currently no Algorithm based algorithms defined and no current software ISP algorithms support control knobs. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Reviewed-by: Daniel Scally Reviewed-by: Laurent Pinchart --- include/libcamera/internal/software_isp/software_isp.h | 1 + include/libcamera/ipa/soft.mojom | 1 + src/ipa/simple/soft_simple.cpp | 9 +++++++++ src/libcamera/pipeline/simple/simple.cpp | 2 ++ src/libcamera/software_isp/software_isp.cpp | 10 ++++++++++ 5 files changed, 23 insertions(+) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index c5d5a46f..a3e3a9da 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -73,6 +73,7 @@ public: int start(); void stop(); + void queueRequest(const uint32_t frame, const ControlList &controls); int queueBuffers(uint32_t frame, FrameBuffer *input, const std::map &outputs); diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index 6b049d20..0ca2869c 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -23,6 +23,7 @@ interface IPASoftInterface { configure(IPAConfigInfo configInfo) => (int32 ret); + [async] queueRequest(uint32 frame, libcamera.ControlList sensorControls); [async] processStats(uint32 frame, uint32 bufferId, libcamera.ControlList sensorControls); diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 0ea62266..eb3bbd92 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -78,6 +78,7 @@ public: int start() override; void stop() override; + void queueRequest(const uint32_t frame, const ControlList &controls) override; void processStats(const uint32_t frame, const uint32_t bufferId, const ControlList &sensorControls) override; @@ -270,6 +271,14 @@ void IPASoftSimple::stop() { } +void IPASoftSimple::queueRequest(const uint32_t frame, const ControlList &controls) +{ + IPAFrameContext &frameContext = context_.frameContexts.alloc(frame); + + for (auto const &algo : algorithms()) + algo->queueRequest(context_, frame, frameContext, controls); +} + void IPASoftSimple::processStats([[maybe_unused]] const uint32_t frame, [[maybe_unused]] const uint32_t bufferId, const ControlList &sensorControls) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 834b33d9..ed50ab52 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1428,6 +1428,8 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) if (data->useConversion_) data->conversionQueue_.push(std::move(buffers)); + data->swIsp_->queueRequest(request->sequence(), request->controls()); + return 0; } diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 8f1c0f65..eda18947 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -278,6 +278,16 @@ int SoftwareIsp::exportBuffers(const Stream *stream, unsigned int count, return count; } +/** + * \brief Queue a request and process the control list from the application + * \param[in] frame The number of the frame which will be processed next + * \param[in] controls The controls for the \a frame + */ +void SoftwareIsp::queueRequest(const uint32_t frame, const ControlList &controls) +{ + ipa_->queueRequest(frame, controls); +} + /** * \brief Queue buffers to Software ISP * \param[in] frame The frame number From patchwork Fri Aug 30 07:25:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21067 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A9529C32C2 for ; Fri, 30 Aug 2024 07:26:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 32546634D3; Fri, 30 Aug 2024 09:26:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="GJpKQxMm"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B6865634C8 for ; Fri, 30 Aug 2024 09:26:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002797; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LzExjFmOMWaQshd+wHdnRR6XuJZ3FZ5MPMv6UWL7yfs=; b=GJpKQxMmYR4+nwxtHBns7Aulc9JlJ7Iq8dT7opmlkrq6oRgIPnjyv+ARN5u4PM9xlcyjH0 kMSWfrZyUTlZzz0q6ZF60ots7B6JsZnA/kTiGxVa/mIUWOdD221XmDHC87gQ+9sTPc3oUt faVjsBVrkKUFOGQ3hvvDAu2fKRiLaqc= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-18-sorwhQFyOSmVtIi5Kd1KUw-1; Fri, 30 Aug 2024 03:26:33 -0400 X-MC-Unique: sorwhQFyOSmVtIi5Kd1KUw-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8106C1955D4D; Fri, 30 Aug 2024 07:26:32 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6A38219560A3; Fri, 30 Aug 2024 07:26:30 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 11/18] libcamera: software_isp: Call Algorithm::prepare Date: Fri, 30 Aug 2024 09:25:47 +0200 Message-ID: <20240830072554.180672-12-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch adds Algorithm::prepare call for the defined algorithms. This is preparation only since there are currently no Algorithm based algorithms defined. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/soft.mojom | 1 + src/ipa/simple/soft_simple.cpp | 8 ++++++++ src/libcamera/software_isp/software_isp.cpp | 1 + 3 files changed, 10 insertions(+) diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index 0ca2869c..bc70d4e4 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -24,6 +24,7 @@ interface IPASoftInterface { => (int32 ret); [async] queueRequest(uint32 frame, libcamera.ControlList sensorControls); + [async] prepare(uint32 frame); [async] processStats(uint32 frame, uint32 bufferId, libcamera.ControlList sensorControls); diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index eb3bbd92..d52242be 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -79,6 +79,7 @@ public: void stop() override; void queueRequest(const uint32_t frame, const ControlList &controls) override; + void prepare(const uint32_t frame) override; void processStats(const uint32_t frame, const uint32_t bufferId, const ControlList &sensorControls) override; @@ -279,6 +280,13 @@ void IPASoftSimple::queueRequest(const uint32_t frame, const ControlList &contro algo->queueRequest(context_, frame, frameContext, controls); } +void IPASoftSimple::prepare(const uint32_t frame) +{ + IPAFrameContext &frameContext = context_.frameContexts.get(frame); + for (auto const &algo : algorithms()) + algo->prepare(context_, frame, frameContext, params_); +} + void IPASoftSimple::processStats([[maybe_unused]] const uint32_t frame, [[maybe_unused]] const uint32_t bufferId, const ControlList &sensorControls) diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index eda18947..dbf27f31 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -352,6 +352,7 @@ void SoftwareIsp::stop() */ void SoftwareIsp::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output) { + ipa_->prepare(frame); debayer_->invokeMethod(&DebayerCpu::process, ConnectionTypeQueued, frame, input, output, debayerParams_); } From patchwork Fri Aug 30 07:25:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21068 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0DFD8C32D6 for ; Fri, 30 Aug 2024 07:26:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7A668634CD; Fri, 30 Aug 2024 09:26:44 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="AR6ykITJ"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 328E8634CC for ; Fri, 30 Aug 2024 09:26:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002798; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R6pipysZ/qYOzyonkafacIN3VlyHa05TRmcXWXBfMSU=; b=AR6ykITJjaKZfnaBHd3ayHB5+EcQnjVUDa3x6Pgo/L+rL3VNpcqsy3ne3OX1qFDJsNoVlQ xDCzHL1WM61gaIn91zL5OzsaSJoM6X0A84UCiYP3LyBFxA/jTbCnC2+7s6/QjXj/UIGWPa ap79CdyfXsno3XDV1dFEKLH5xM1N43Y= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-631-ggVS8wYlPniccyXOwSd_BQ-1; Fri, 30 Aug 2024 03:26:36 -0400 X-MC-Unique: ggVS8wYlPniccyXOwSd_BQ-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 462AC19560AD; Fri, 30 Aug 2024 07:26:35 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1790219560A3; Fri, 30 Aug 2024 07:26:32 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 12/18] libcamera: software_isp: Call Algorithm::process Date: Fri, 30 Aug 2024 09:25:48 +0200 Message-ID: <20240830072554.180672-13-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch adds Algorithm::process call for the defined algorithms. This is preparation only since there are currently no Algorithm based algorithms defined. As software ISP currently doesn't produce any metadata, a dummy and unused metadata instance is created to satisfy Algorithm::process API. This should be changed in future. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Reviewed-by: Daniel Scally Reviewed-by: Laurent Pinchart --- src/ipa/simple/soft_simple.cpp | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index d52242be..4b9201dc 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -287,10 +287,21 @@ void IPASoftSimple::prepare(const uint32_t frame) algo->prepare(context_, frame, frameContext, params_); } -void IPASoftSimple::processStats([[maybe_unused]] const uint32_t frame, +void IPASoftSimple::processStats(const uint32_t frame, [[maybe_unused]] const uint32_t bufferId, const ControlList &sensorControls) { + IPAFrameContext &frameContext = context_.frameContexts.get(frame); + /* + * Software ISP currently does not produce any metadata. Use an empty + * ControlList for now. + * + * \todo Implement proper metadata handling + */ + ControlList metadata(controls::controls); + for (auto const &algo : algorithms()) + algo->process(context_, frame, frameContext, stats_, metadata); + SwIspStats::Histogram histogram = stats_->yHistogram; if (ignoreUpdates_ > 0) blackLevel_.update(histogram); From patchwork Fri Aug 30 07:25:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21069 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 508F6C32D9 for ; Fri, 30 Aug 2024 07:26:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 91668634E7; Fri, 30 Aug 2024 09:26:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="bgoC3md2"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4CAAD634E1 for ; Fri, 30 Aug 2024 09:26:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002802; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VqIsveihM9RNHJG1JC7Ty0xirratIuyEXSrqRU2uvVw=; b=bgoC3md2K734+m7HdVVD4dy9g115lKwLIjOiSSPER1+lG3KZsfqo7uIyPcgkM/fTseb/K2 pTmVlS+uFJElCqQ9JfB5oif83+dNQ9Ak/zoEH/plNMer6Ri4ZWXtoV8pDhoo2WcRXsvAi7 21lPUyqlP+iEigZrWqUP/9TxbFAkFiU= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-568-7SFpcPbeMJeb4BIbHwTL2w-1; Fri, 30 Aug 2024 03:26:39 -0400 X-MC-Unique: 7SFpcPbeMJeb4BIbHwTL2w-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D96EE1955BEE; Fri, 30 Aug 2024 07:26:37 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 87A5219560A3; Fri, 30 Aug 2024 07:26:35 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 13/18] libcamera: software_isp: Move black level to an algorithm module Date: Fri, 30 Aug 2024 09:25:49 +0200 Message-ID: <20240830072554.180672-14-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The black level determination, already present as a separate class, can be moved to the prepared Algorithm processing structure. It is the first of the current software ISP algorithms that is called in the stats processing sequence, which means it is also the first one that should be moved to the new structure. Stats processing starts with calling Algorithm-based processing so the call order of the algorithms is retained. Movement of this algorithm is relatively straightforward because all it does is processing stats. The comment about getting black level from the tuning files is dropped. The black level will be taken from CameraSensorHelper if available and that will be implemented in one of the followup patches. Black level is now recomputed on each stats processing. In a future patch, after DelayedControls are used, this will be changed to recompute the black level only after exposure/gain changes. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/ipa/simple/algorithms/blc.cpp | 71 ++++++++++++++++++++ src/ipa/simple/algorithms/blc.h | 37 +++++++++++ src/ipa/simple/algorithms/meson.build | 1 + src/ipa/simple/black_level.cpp | 93 --------------------------- src/ipa/simple/black_level.h | 33 ---------- src/ipa/simple/data/uncalibrated.yaml | 1 + src/ipa/simple/ipa_context.cpp | 8 +++ src/ipa/simple/ipa_context.h | 5 ++ src/ipa/simple/meson.build | 1 - src/ipa/simple/soft_simple.cpp | 8 +-- 10 files changed, 125 insertions(+), 133 deletions(-) create mode 100644 src/ipa/simple/algorithms/blc.cpp create mode 100644 src/ipa/simple/algorithms/blc.h delete mode 100644 src/ipa/simple/black_level.cpp delete mode 100644 src/ipa/simple/black_level.h diff --git a/src/ipa/simple/algorithms/blc.cpp b/src/ipa/simple/algorithms/blc.cpp new file mode 100644 index 00000000..3b73d830 --- /dev/null +++ b/src/ipa/simple/algorithms/blc.cpp @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Red Hat Inc. + * + * Black level handling + */ + +#include "blc.h" + +#include + +#include + +namespace libcamera { + +namespace ipa::soft::algorithms { + +LOG_DEFINE_CATEGORY(IPASoftBL) + +BlackLevel::BlackLevel() +{ +} + +int BlackLevel::init(IPAContext &context, + [[maybe_unused]] const YamlObject &tuningData) +{ + context.activeState.black.level = 255; + return 0; +} + +void BlackLevel::process(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + const SwIspStats *stats, + [[maybe_unused]] ControlList &metadata) +{ + const SwIspStats::Histogram &histogram = stats->yHistogram; + + /* + * The constant is selected to be "good enough", not overly conservative or + * aggressive. There is no magic about the given value. + */ + constexpr float ignoredPercentage_ = 0.02; + const unsigned int total = + std::accumulate(begin(histogram), end(histogram), 0); + const unsigned int pixelThreshold = ignoredPercentage_ * total; + const unsigned int histogramRatio = 256 / SwIspStats::kYHistogramSize; + const unsigned int currentBlackIdx = + context.activeState.black.level / histogramRatio; + + for (unsigned int i = 0, seen = 0; + i < currentBlackIdx && i < SwIspStats::kYHistogramSize; + i++) { + seen += histogram[i]; + if (seen >= pixelThreshold) { + context.activeState.black.level = i * histogramRatio; + LOG(IPASoftBL, Debug) + << "Auto-set black level: " + << i << "/" << SwIspStats::kYHistogramSize + << " (" << 100 * (seen - histogram[i]) / total << "% below, " + << 100 * seen / total << "% at or below)"; + break; + } + }; +} + +REGISTER_IPA_ALGORITHM(BlackLevel, "BlackLevel") + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/blc.h b/src/ipa/simple/algorithms/blc.h new file mode 100644 index 00000000..2f73db52 --- /dev/null +++ b/src/ipa/simple/algorithms/blc.h @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Red Hat Inc. + * + * Black level handling + */ + +#pragma once + +#include + +#include "libcamera/internal/software_isp/swisp_stats.h" + +#include "algorithm.h" +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::soft::algorithms { + +class BlackLevel : public Algorithm +{ +public: + BlackLevel(); + ~BlackLevel() = default; + + int init(IPAContext &context, const YamlObject &tuningData) + override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const SwIspStats *stats, + ControlList &metadata) override; +}; + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/meson.build b/src/ipa/simple/algorithms/meson.build index 31d26e43..1f63c220 100644 --- a/src/ipa/simple/algorithms/meson.build +++ b/src/ipa/simple/algorithms/meson.build @@ -1,4 +1,5 @@ # SPDX-License-Identifier: CC0-1.0 soft_simple_ipa_algorithms = files([ + 'blc.cpp', ]) diff --git a/src/ipa/simple/black_level.cpp b/src/ipa/simple/black_level.cpp deleted file mode 100644 index 37e0109c..00000000 --- a/src/ipa/simple/black_level.cpp +++ /dev/null @@ -1,93 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2024, Red Hat Inc. - * - * black level handling - */ - -#include "black_level.h" - -#include - -#include - -namespace libcamera { - -LOG_DEFINE_CATEGORY(IPASoftBL) - -namespace ipa::soft { - -/** - * \class BlackLevel - * \brief Object providing black point level for software ISP - * - * Black level can be provided in hardware tuning files or, if no tuning file is - * available for the given hardware, guessed automatically, with less accuracy. - * As tuning files are not yet implemented for software ISP, BlackLevel - * currently provides only guessed black levels. - * - * This class serves for tracking black level as a property of the underlying - * hardware, not as means of enhancing a particular scene or image. - * - * The class is supposed to be instantiated for the given camera stream. - * The black level can be retrieved using BlackLevel::get() method. It is - * initially 0 and may change when updated using BlackLevel::update() method. - */ - -BlackLevel::BlackLevel() - : blackLevel_(255), blackLevelSet_(false) -{ -} - -/** - * \brief Return the current black level - * - * \return The black level, in the range from 0 (minimum) to 255 (maximum). - * If the black level couldn't be determined yet, return 0. - */ -uint8_t BlackLevel::get() const -{ - return blackLevelSet_ ? blackLevel_ : 0; -} - -/** - * \brief Update black level from the provided histogram - * \param[in] yHistogram The histogram to be used for updating black level - * - * The black level is property of the given hardware, not image. It is updated - * only if it has not been yet set or if it is lower than the lowest value seen - * so far. - */ -void BlackLevel::update(SwIspStats::Histogram &yHistogram) -{ - /* - * The constant is selected to be "good enough", not overly conservative or - * aggressive. There is no magic about the given value. - */ - constexpr float ignoredPercentage_ = 0.02; - const unsigned int total = - std::accumulate(begin(yHistogram), end(yHistogram), 0); - const unsigned int pixelThreshold = ignoredPercentage_ * total; - const unsigned int histogramRatio = 256 / SwIspStats::kYHistogramSize; - const unsigned int currentBlackIdx = blackLevel_ / histogramRatio; - - for (unsigned int i = 0, seen = 0; - i < currentBlackIdx && i < SwIspStats::kYHistogramSize; - i++) { - seen += yHistogram[i]; - if (seen >= pixelThreshold) { - blackLevel_ = i * histogramRatio; - blackLevelSet_ = true; - LOG(IPASoftBL, Debug) - << "Auto-set black level: " - << i << "/" << SwIspStats::kYHistogramSize - << " (" << 100 * (seen - yHistogram[i]) / total << "% below, " - << 100 * seen / total << "% at or below)"; - break; - } - }; -} - -} /* namespace ipa::soft */ - -} /* namespace libcamera */ diff --git a/src/ipa/simple/black_level.h b/src/ipa/simple/black_level.h deleted file mode 100644 index a04230c9..00000000 --- a/src/ipa/simple/black_level.h +++ /dev/null @@ -1,33 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2024, Red Hat Inc. - * - * black level handling - */ - -#pragma once - -#include -#include - -#include "libcamera/internal/software_isp/swisp_stats.h" - -namespace libcamera { - -namespace ipa::soft { - -class BlackLevel -{ -public: - BlackLevel(); - uint8_t get() const; - void update(SwIspStats::Histogram &yHistogram); - -private: - uint8_t blackLevel_; - bool blackLevelSet_; -}; - -} /* namespace ipa::soft */ - -} /* namespace libcamera */ diff --git a/src/ipa/simple/data/uncalibrated.yaml b/src/ipa/simple/data/uncalibrated.yaml index 2cdc39a8..e0d03d96 100644 --- a/src/ipa/simple/data/uncalibrated.yaml +++ b/src/ipa/simple/data/uncalibrated.yaml @@ -3,4 +3,5 @@ --- version: 1 algorithms: + - BlackLevel: ... diff --git a/src/ipa/simple/ipa_context.cpp b/src/ipa/simple/ipa_context.cpp index 3c1c7262..268cff32 100644 --- a/src/ipa/simple/ipa_context.cpp +++ b/src/ipa/simple/ipa_context.cpp @@ -50,4 +50,12 @@ namespace libcamera::ipa::soft { * \brief The current state of IPA algorithms */ +/** + * \var IPAActiveState::black + * \brief Context for the Black Level algorithm + * + * \var IPAActiveState::black.level + * \brief Current determined black level + */ + } /* namespace libcamera::ipa::soft */ diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index e7d8b8df..22156569 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -7,6 +7,8 @@ #pragma once +#include + #include namespace libcamera { @@ -17,6 +19,9 @@ struct IPASessionConfiguration { }; struct IPAActiveState { + struct { + uint8_t level; + } black; }; struct IPAFrameContext : public FrameContext { diff --git a/src/ipa/simple/meson.build b/src/ipa/simple/meson.build index dcd7c70a..2f9f15f4 100644 --- a/src/ipa/simple/meson.build +++ b/src/ipa/simple/meson.build @@ -8,7 +8,6 @@ ipa_name = 'ipa_soft_simple' soft_simple_sources = files([ 'ipa_context.cpp', 'soft_simple.cpp', - 'black_level.cpp', ]) soft_simple_sources += soft_simple_ipa_algorithms diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 4b9201dc..a1bd2f0c 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -29,7 +29,6 @@ #include "libipa/camera_sensor_helper.h" -#include "black_level.h" #include "module.h" namespace libcamera { @@ -61,7 +60,7 @@ class IPASoftSimple : public ipa::soft::IPASoftInterface, public Module { public: IPASoftSimple() - : params_(nullptr), stats_(nullptr), blackLevel_(BlackLevel()), + : params_(nullptr), stats_(nullptr), context_({ {}, {}, { kMaxFrameContexts } }), ignoreUpdates_(0) { @@ -93,7 +92,6 @@ private: SwIspStats *stats_; std::unique_ptr camHelper_; ControlInfoMap sensorInfoMap_; - BlackLevel blackLevel_; static constexpr unsigned int kGammaLookupSize = 1024; std::array gammaTable_; @@ -303,9 +301,7 @@ void IPASoftSimple::processStats(const uint32_t frame, algo->process(context_, frame, frameContext, stats_, metadata); SwIspStats::Histogram histogram = stats_->yHistogram; - if (ignoreUpdates_ > 0) - blackLevel_.update(histogram); - const uint8_t blackLevel = blackLevel_.get(); + const uint8_t blackLevel = context_.activeState.black.level; /* * Black level must be subtracted to get the correct AWB ratios, they From patchwork Fri Aug 30 07:25:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21070 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 42BE2C324C for ; Fri, 30 Aug 2024 07:26:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9FD1E634E7; Fri, 30 Aug 2024 09:26:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="jPnqVhLU"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C4B716341E for ; Fri, 30 Aug 2024 09:26:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002806; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8B202D3YmiI9zxvPcauUXitgE2tZBKhGaVNdjiB2mS8=; b=jPnqVhLU1lbKiI6PVoPQGWHzjT8HMCDXvQWfMzQsYyXYnW9VtNifJ7jzDhpL/+Kg/+4Tkq 2XmeD1/QTjc/dHfQFG+K31FCC0Et64GxbBKcYRpgtsvkRUFxJWWoOTjY6VC1jlnKPzhq8e z7j77V7apQmWED9egxmEIHqxGBIEBEs= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-534-ccVu-sqPMGydhGAcqYjwEw-1; Fri, 30 Aug 2024 03:26:42 -0400 X-MC-Unique: ccVu-sqPMGydhGAcqYjwEw-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E38231955D48; Fri, 30 Aug 2024 07:26:40 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4ECCF19560AA; Fri, 30 Aug 2024 07:26:38 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 14/18] libcamera: software_isp: Move color handling to an algorithm module Date: Fri, 30 Aug 2024 09:25:50 +0200 Message-ID: <20240830072554.180672-15-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" After black level handling has been moved to an algorithm module, white balance and the construction of color tables can be moved to algorithm modules too. This time, the moved code is split between stats processing and parameter construction methods. It is also split to three algorithm modules: - Gamma table computation. This is actually independent of color handling. - White balance computation. - Color lookup tables construction. While this applies the color gains computed by the white balance algorithm, it is not directly related to white balance. We may want to modify the color lookup tables in future according to other parameters than just gamma and white balance gains. This is the only part of the software ISP algorithms that sets the parameters so emitting setIspParams can be moved to prepare() method. A more natural representation of the gains (and black level) would be floating point numbers. This is not done here in order to minimize changes in code movements. It will be addressed in a followup patch. Signed-off-by: Milan Zamazal --- src/ipa/simple/algorithms/awb.cpp | 70 +++++++++++++++++++++++++++ src/ipa/simple/algorithms/awb.h | 38 +++++++++++++++ src/ipa/simple/algorithms/gamma.cpp | 64 ++++++++++++++++++++++++ src/ipa/simple/algorithms/gamma.h | 42 ++++++++++++++++ src/ipa/simple/algorithms/lut.cpp | 57 ++++++++++++++++++++++ src/ipa/simple/algorithms/lut.h | 37 ++++++++++++++ src/ipa/simple/algorithms/meson.build | 3 ++ src/ipa/simple/data/uncalibrated.yaml | 3 ++ src/ipa/simple/ipa_context.cpp | 30 ++++++++++++ src/ipa/simple/ipa_context.h | 12 +++++ src/ipa/simple/soft_simple.cpp | 66 ++----------------------- 11 files changed, 360 insertions(+), 62 deletions(-) create mode 100644 src/ipa/simple/algorithms/awb.cpp create mode 100644 src/ipa/simple/algorithms/awb.h create mode 100644 src/ipa/simple/algorithms/gamma.cpp create mode 100644 src/ipa/simple/algorithms/gamma.h create mode 100644 src/ipa/simple/algorithms/lut.cpp create mode 100644 src/ipa/simple/algorithms/lut.h diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp new file mode 100644 index 00000000..4b49996a --- /dev/null +++ b/src/ipa/simple/algorithms/awb.cpp @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Red Hat Inc. + * + * Auto white balance + */ + +#include "awb.h" + +#include +#include + +#include + +#include "simple/ipa_context.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(IPASoftAwb) + +namespace ipa::soft::algorithms { + +int Awb::init(IPAContext &context, + [[maybe_unused]] const YamlObject &tuningData) +{ + auto &gains = context.activeState.gains; + gains.red = gains.green = gains.blue = 256; + + return 0; +} + +void Awb::process(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + const SwIspStats *stats, + [[maybe_unused]] ControlList &metadata) +{ + const SwIspStats::Histogram &histogram = stats->yHistogram; + const uint8_t blackLevel = context.activeState.black.level; + + /* + * Black level must be subtracted to get the correct AWB ratios, they + * would be off if they were computed from the whole brightness range + * rather than from the sensor range. + */ + const uint64_t nPixels = std::accumulate( + histogram.begin(), histogram.end(), 0); + const uint64_t offset = blackLevel * nPixels; + const uint64_t sumR = stats->sumR_ - offset / 4; + const uint64_t sumG = stats->sumG_ - offset / 2; + const uint64_t sumB = stats->sumB_ - offset / 4; + + /* + * Calculate red and blue gains for AWB. + * Clamp max gain at 4.0, this also avoids 0 division. + * Gain: 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. + */ + auto &gains = context.activeState.gains; + gains.red = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; + gains.blue = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; + /* Green gain is fixed to 256 */ + + LOG(IPASoftAwb, Debug) << "gain R/B " << gains.red << "/" << gains.blue; +} + +REGISTER_IPA_ALGORITHM(Awb, "Awb") + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/awb.h b/src/ipa/simple/algorithms/awb.h new file mode 100644 index 00000000..c0b88bec --- /dev/null +++ b/src/ipa/simple/algorithms/awb.h @@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Red Hat Inc. + * + * Auto white balance + */ + +#pragma once + +#include + +#include "libcamera/internal/software_isp/swisp_stats.h" + +#include "algorithm.h" +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::soft::algorithms { + +class Awb : public Algorithm +{ +public: + Awb() = default; + ~Awb() = default; + + int init(IPAContext &context, const YamlObject &tuningData) + override; + void process(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const SwIspStats *stats, + ControlList &metadata) override; +}; + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/gamma.cpp b/src/ipa/simple/algorithms/gamma.cpp new file mode 100644 index 00000000..0b8ec5ee --- /dev/null +++ b/src/ipa/simple/algorithms/gamma.cpp @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Red Hat Inc. + * + * Gamma table construction + */ + +#include "gamma.h" + +#include +#include +#include + +#include + +#include "simple/ipa_context.h" + +namespace libcamera { + +namespace ipa::soft::algorithms { + +int Gamma::init(IPAContext &context, + [[maybe_unused]] const YamlObject &tuningData) +{ + /* Gamma value is fixed */ + context.configuration.gamma = 0.5; + updateGammaTable(context); + + return 0; +} + +void Gamma::updateGammaTable(IPAContext &context) +{ + auto &gammaTable = context.activeState.gamma.gammaTable; + auto blackLevel = context.activeState.black.level; + const unsigned int blackIndex = + blackLevel * IPAActiveState::kGammaLookupSize / 256; + std::fill(gammaTable.begin(), gammaTable.begin() + blackIndex, 0); + const float divisor = kGammaLookupSize - blackIndex - 1.0; + for (unsigned int i = blackIndex; i < kGammaLookupSize; i++) + gammaTable[i] = UINT8_MAX * powf((i - blackIndex) / divisor, + context.configuration.gamma); + context.activeState.gamma.blackLevel = blackLevel; +} + +void Gamma::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] DebayerParams *params) +{ + /* + * Update the gamma table if needed. This means if black level changes and + * since the black level gets updated only if a lower value is observed, + * it's not permanently prone to minor fluctuations or rounding errors. + */ + if (context.activeState.gamma.blackLevel != context.activeState.black.level) + updateGammaTable(context); +} + +REGISTER_IPA_ALGORITHM(Gamma, "Gamma") + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/gamma.h b/src/ipa/simple/algorithms/gamma.h new file mode 100644 index 00000000..43e88e87 --- /dev/null +++ b/src/ipa/simple/algorithms/gamma.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Red Hat Inc. + * + * Gamma table construction + */ + +#pragma once + +#include + +#include "libcamera/internal/software_isp/debayer_params.h" + +#include "algorithm.h" +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::soft::algorithms { + +static constexpr unsigned int kGammaLookupSize = 1024; + +class Gamma : public Algorithm +{ +public: + Gamma() = default; + ~Gamma() = default; + + int init(IPAContext &context, const YamlObject &tuningData) + override; + void prepare(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + DebayerParams *params) override; + +private: + void updateGammaTable(IPAContext &context); +}; + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp new file mode 100644 index 00000000..748f10aa --- /dev/null +++ b/src/ipa/simple/algorithms/lut.cpp @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Red Hat Inc. + * + * Lookup table construction + */ + +#include "lut.h" + +#include +#include + +#include + +#include "simple/ipa_context.h" + +#include "gamma.h" + +namespace libcamera { + +namespace ipa::soft::algorithms { + +int Lut::init(IPAContext &context, + [[maybe_unused]] const YamlObject &tuningData) +{ + auto &gains = context.activeState.gains; + gains.red = gains.green = gains.blue = 1.0; + + return 0; +} + +void Lut::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + DebayerParams *params) +{ + auto &gains = context.activeState.gains; + auto &gammaTable = context.activeState.gamma.gammaTable; + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { + constexpr unsigned int div = + static_cast(DebayerParams::kRGBLookupSize) * 256 / kGammaLookupSize; + /* Apply gamma after gain! */ + unsigned int idx; + idx = std::min({ i * gains.red / div, kGammaLookupSize - 1 }); + params->red[i] = gammaTable[idx]; + idx = std::min({ i * gains.green / div, kGammaLookupSize - 1 }); + params->green[i] = gammaTable[idx]; + idx = std::min({ i * gains.blue / div, kGammaLookupSize - 1 }); + params->blue[i] = gammaTable[idx]; + } +} + +REGISTER_IPA_ALGORITHM(Lut, "Lut") + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/lut.h b/src/ipa/simple/algorithms/lut.h new file mode 100644 index 00000000..5b5e554e --- /dev/null +++ b/src/ipa/simple/algorithms/lut.h @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Red Hat Inc. + * + * Lookup table construction + */ + +#pragma once + +#include + +#include "libcamera/internal/software_isp/debayer_params.h" + +#include "algorithm.h" +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::soft::algorithms { + +class Lut : public Algorithm +{ +public: + Lut() = default; + ~Lut() = default; + + int init(IPAContext &context, const YamlObject &tuningData) + override; + void prepare(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + DebayerParams *params) override; +}; + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/meson.build b/src/ipa/simple/algorithms/meson.build index 1f63c220..68e5aa58 100644 --- a/src/ipa/simple/algorithms/meson.build +++ b/src/ipa/simple/algorithms/meson.build @@ -1,5 +1,8 @@ # SPDX-License-Identifier: CC0-1.0 soft_simple_ipa_algorithms = files([ + 'awb.cpp', 'blc.cpp', + 'gamma.cpp', + 'lut.cpp', ]) diff --git a/src/ipa/simple/data/uncalibrated.yaml b/src/ipa/simple/data/uncalibrated.yaml index e0d03d96..deaea6b1 100644 --- a/src/ipa/simple/data/uncalibrated.yaml +++ b/src/ipa/simple/data/uncalibrated.yaml @@ -4,4 +4,7 @@ version: 1 algorithms: - BlackLevel: + - Gamma: + - Awb: + - Lut: ... diff --git a/src/ipa/simple/ipa_context.cpp b/src/ipa/simple/ipa_context.cpp index 268cff32..5fa492cb 100644 --- a/src/ipa/simple/ipa_context.cpp +++ b/src/ipa/simple/ipa_context.cpp @@ -50,6 +50,11 @@ namespace libcamera::ipa::soft { * \brief The current state of IPA algorithms */ +/** + * \var IPASessionConfiguration::gamma + * \brief Gamma value to be used in the raw image processing + */ + /** * \var IPAActiveState::black * \brief Context for the Black Level algorithm @@ -58,4 +63,29 @@ namespace libcamera::ipa::soft { * \brief Current determined black level */ +/** + * \var IPAActiveState::gains + * \brief Context for gains in the Colors algorithm + * + * \var IPAActiveState::gains.red + * \brief Gain of red color + * + * \var IPAActiveState::gains.green + * \brief Gain of green color + * + * \var IPAActiveState::gains.blue + * \brief Gain of blue color + */ + +/** + * \var IPAActiveState::gamma + * \brief Context for gamma in the Colors algorithm + * + * \var IPAActiveState::gamma.gammaTable + * \brief Computed gamma table + * + * \var IPAActiveState::gamma.blackLevel + * \brief Black level used for the gamma table computation + */ + } /* namespace libcamera::ipa::soft */ diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 22156569..f8b8834d 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -7,6 +7,7 @@ #pragma once +#include #include #include @@ -16,12 +17,23 @@ namespace libcamera { namespace ipa::soft { struct IPASessionConfiguration { + float gamma; }; struct IPAActiveState { struct { uint8_t level; } black; + struct { + unsigned int red; + unsigned int green; + unsigned int blue; + } gains; + static constexpr unsigned int kGammaLookupSize = 1024; + struct { + std::array gammaTable; + uint8_t blackLevel; + } gamma; }; struct IPAFrameContext : public FrameContext { diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index a1bd2f0c..8f01bf7d 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -5,8 +5,6 @@ * Simple Software Image Processing Algorithm module */ -#include -#include #include #include @@ -93,9 +91,6 @@ private: std::unique_ptr camHelper_; ControlInfoMap sensorInfoMap_; - static constexpr unsigned int kGammaLookupSize = 1024; - std::array gammaTable_; - int lastBlackLevel_ = -1; /* Local parameter storage */ struct IPAContext context_; @@ -283,6 +278,7 @@ void IPASoftSimple::prepare(const uint32_t frame) IPAFrameContext &frameContext = context_.frameContexts.get(frame); for (auto const &algo : algorithms()) algo->prepare(context_, frame, frameContext, params_); + setIspParams.emit(); } void IPASoftSimple::processStats(const uint32_t frame, @@ -300,62 +296,6 @@ void IPASoftSimple::processStats(const uint32_t frame, for (auto const &algo : algorithms()) algo->process(context_, frame, frameContext, stats_, metadata); - SwIspStats::Histogram histogram = stats_->yHistogram; - const uint8_t blackLevel = context_.activeState.black.level; - - /* - * Black level must be subtracted to get the correct AWB ratios, they - * would be off if they were computed from the whole brightness range - * rather than from the sensor range. - */ - const uint64_t nPixels = std::accumulate( - histogram.begin(), histogram.end(), 0); - const uint64_t offset = blackLevel * nPixels; - const uint64_t sumR = stats_->sumR_ - offset / 4; - const uint64_t sumG = stats_->sumG_ - offset / 2; - const uint64_t sumB = stats_->sumB_ - offset / 4; - - /* - * Calculate red and blue gains for AWB. - * Clamp max gain at 4.0, this also avoids 0 division. - * Gain: 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. - */ - const unsigned int gainR = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; - const unsigned int gainB = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; - /* Green gain and gamma values are fixed */ - constexpr unsigned int gainG = 256; - - /* Update the gamma table if needed */ - if (blackLevel != lastBlackLevel_) { - constexpr float gamma = 0.5; - const unsigned int blackIndex = blackLevel * kGammaLookupSize / 256; - std::fill(gammaTable_.begin(), gammaTable_.begin() + blackIndex, 0); - const float divisor = kGammaLookupSize - blackIndex - 1.0; - for (unsigned int i = blackIndex; i < kGammaLookupSize; i++) - gammaTable_[i] = UINT8_MAX * - std::pow((i - blackIndex) / divisor, gamma); - - lastBlackLevel_ = blackLevel; - } - - for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { - constexpr unsigned int div = - DebayerParams::kRGBLookupSize * 256 / kGammaLookupSize; - unsigned int idx; - - /* Apply gamma after gain! */ - idx = std::min({ i * gainR / div, (kGammaLookupSize - 1) }); - params_->red[i] = gammaTable_[idx]; - - idx = std::min({ i * gainG / div, (kGammaLookupSize - 1) }); - params_->green[i] = gammaTable_[idx]; - - idx = std::min({ i * gainB / div, (kGammaLookupSize - 1) }); - params_->blue[i] = gammaTable_[idx]; - } - - setIspParams.emit(); - /* \todo Switch to the libipa/algorithm.h API someday. */ /* @@ -372,6 +312,7 @@ void IPASoftSimple::processStats(const uint32_t frame, * Calculate Mean Sample Value (MSV) according to formula from: * https://www.araa.asn.au/acra/acra2007/papers/paper84final.pdf */ + const uint8_t blackLevel = context_.activeState.black.level; const unsigned int blackLevelHistIdx = blackLevel / (256 / SwIspStats::kYHistogramSize); const unsigned int histogramSize = @@ -421,7 +362,8 @@ void IPASoftSimple::processStats(const uint32_t frame, LOG(IPASoft, Debug) << "exposureMSV " << exposureMSV << " exp " << exposure_ << " again " << again_ - << " gain R/B " << gainR << "/" << gainB + << " gain R/B " << context_.activeState.gains.red + << "/" << context_.activeState.gains.blue << " black level " << static_cast(blackLevel); } From patchwork Fri Aug 30 07:25:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21071 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4B856C32D5 for ; Fri, 30 Aug 2024 07:26:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AF8B0634EB; Fri, 30 Aug 2024 09:26:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Qzg0HYxD"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 59D0C6341E for ; Fri, 30 Aug 2024 09:26:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002810; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JTLOpHrUNalfC38eZPdCfc/vm6uc9RnUCdBkVqwy9H8=; b=Qzg0HYxD8AX6YF8ACsRr48qfKxSdiTjymV4s+HjM8bX65haJhT3Gf8gMDjqlhrAtxL2RDC u2Ej29IReEu/ycfJCfC2HIccMfyNHANZnOR81poJ/OIEXcq0bRjdJbErufRc+ZomPO16bY kYp5x05vms6cNK3iip2T100Dw5YVxhY= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-616-ZH8UeedSOt2ayfoET0nNdw-1; Fri, 30 Aug 2024 03:26:44 -0400 X-MC-Unique: ZH8UeedSOt2ayfoET0nNdw-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B350919560B1; Fri, 30 Aug 2024 07:26:43 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5F62C19560A3; Fri, 30 Aug 2024 07:26:41 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 15/18] libcamera: software_isp: Use floating point for color parameters Date: Fri, 30 Aug 2024 09:25:51 +0200 Message-ID: <20240830072554.180672-16-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" It's more natural to represent color gains and black level as floating point numbers rather than using a particular pixel-related representation. double is used rather than float because it's a more common floating point type in libcamera algorithms. Otherwise there is no obvious reason to select one over the other here. The constructed color tables still use integer representation for efficiency. Signed-off-by: Milan Zamazal --- src/ipa/simple/algorithms/awb.cpp | 11 +++++------ src/ipa/simple/algorithms/blc.cpp | 8 ++++---- src/ipa/simple/algorithms/gamma.cpp | 2 +- src/ipa/simple/algorithms/lut.cpp | 13 ++++++++----- src/ipa/simple/ipa_context.h | 11 +++++------ 5 files changed, 23 insertions(+), 22 deletions(-) diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp index 4b49996a..16a0fa61 100644 --- a/src/ipa/simple/algorithms/awb.cpp +++ b/src/ipa/simple/algorithms/awb.cpp @@ -24,7 +24,7 @@ int Awb::init(IPAContext &context, [[maybe_unused]] const YamlObject &tuningData) { auto &gains = context.activeState.gains; - gains.red = gains.green = gains.blue = 256; + gains.red = gains.green = gains.blue = 1.0; return 0; } @@ -36,7 +36,7 @@ void Awb::process(IPAContext &context, [[maybe_unused]] ControlList &metadata) { const SwIspStats::Histogram &histogram = stats->yHistogram; - const uint8_t blackLevel = context.activeState.black.level; + const double blackLevel = context.activeState.black.level; /* * Black level must be subtracted to get the correct AWB ratios, they @@ -53,12 +53,11 @@ void Awb::process(IPAContext &context, /* * Calculate red and blue gains for AWB. * Clamp max gain at 4.0, this also avoids 0 division. - * Gain: 128 = 0.5, 256 = 1.0, 512 = 2.0, etc. */ auto &gains = context.activeState.gains; - gains.red = sumR <= sumG / 4 ? 1024 : 256 * sumG / sumR; - gains.blue = sumB <= sumG / 4 ? 1024 : 256 * sumG / sumB; - /* Green gain is fixed to 256 */ + gains.red = sumR <= sumG / 4 ? 4.0 : static_cast(sumG) / sumR; + gains.blue = sumB <= sumG / 4 ? 4.0 : static_cast(sumG) / sumB; + /* Green gain is fixed to 1.0 */ LOG(IPASoftAwb, Debug) << "gain R/B " << gains.red << "/" << gains.blue; } diff --git a/src/ipa/simple/algorithms/blc.cpp b/src/ipa/simple/algorithms/blc.cpp index 3b73d830..ab7e7dd9 100644 --- a/src/ipa/simple/algorithms/blc.cpp +++ b/src/ipa/simple/algorithms/blc.cpp @@ -24,7 +24,7 @@ BlackLevel::BlackLevel() int BlackLevel::init(IPAContext &context, [[maybe_unused]] const YamlObject &tuningData) { - context.activeState.black.level = 255; + context.activeState.black.level = 1.0; return 0; } @@ -44,16 +44,16 @@ void BlackLevel::process(IPAContext &context, const unsigned int total = std::accumulate(begin(histogram), end(histogram), 0); const unsigned int pixelThreshold = ignoredPercentage_ * total; - const unsigned int histogramRatio = 256 / SwIspStats::kYHistogramSize; const unsigned int currentBlackIdx = - context.activeState.black.level / histogramRatio; + context.activeState.black.level * SwIspStats::kYHistogramSize; for (unsigned int i = 0, seen = 0; i < currentBlackIdx && i < SwIspStats::kYHistogramSize; i++) { seen += histogram[i]; if (seen >= pixelThreshold) { - context.activeState.black.level = i * histogramRatio; + context.activeState.black.level = + static_cast(i) / SwIspStats::kYHistogramSize; LOG(IPASoftBL, Debug) << "Auto-set black level: " << i << "/" << SwIspStats::kYHistogramSize diff --git a/src/ipa/simple/algorithms/gamma.cpp b/src/ipa/simple/algorithms/gamma.cpp index 0b8ec5ee..b03dff25 100644 --- a/src/ipa/simple/algorithms/gamma.cpp +++ b/src/ipa/simple/algorithms/gamma.cpp @@ -34,7 +34,7 @@ void Gamma::updateGammaTable(IPAContext &context) auto &gammaTable = context.activeState.gamma.gammaTable; auto blackLevel = context.activeState.black.level; const unsigned int blackIndex = - blackLevel * IPAActiveState::kGammaLookupSize / 256; + blackLevel * IPAActiveState::kGammaLookupSize; std::fill(gammaTable.begin(), gammaTable.begin() + blackIndex, 0); const float divisor = kGammaLookupSize - blackIndex - 1.0; for (unsigned int i = blackIndex; i < kGammaLookupSize; i++) diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp index 748f10aa..a0e67080 100644 --- a/src/ipa/simple/algorithms/lut.cpp +++ b/src/ipa/simple/algorithms/lut.cpp @@ -37,15 +37,18 @@ void Lut::prepare(IPAContext &context, auto &gains = context.activeState.gains; auto &gammaTable = context.activeState.gamma.gammaTable; for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { - constexpr unsigned int div = - static_cast(DebayerParams::kRGBLookupSize) * 256 / kGammaLookupSize; + constexpr double div = + static_cast(DebayerParams::kRGBLookupSize) / kGammaLookupSize; /* Apply gamma after gain! */ unsigned int idx; - idx = std::min({ i * gains.red / div, kGammaLookupSize - 1 }); + idx = std::min({ static_cast(i * gains.red / div), + kGammaLookupSize - 1 }); params->red[i] = gammaTable[idx]; - idx = std::min({ i * gains.green / div, kGammaLookupSize - 1 }); + idx = std::min({ static_cast(i * gains.green / div), + kGammaLookupSize - 1 }); params->green[i] = gammaTable[idx]; - idx = std::min({ i * gains.blue / div, kGammaLookupSize - 1 }); + idx = std::min({ static_cast(i * gains.blue / div), + kGammaLookupSize - 1 }); params->blue[i] = gammaTable[idx]; } } diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index f8b8834d..28a02475 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -8,7 +8,6 @@ #pragma once #include -#include #include @@ -22,17 +21,17 @@ struct IPASessionConfiguration { struct IPAActiveState { struct { - uint8_t level; + double level; } black; struct { - unsigned int red; - unsigned int green; - unsigned int blue; + double red; + double green; + double blue; } gains; static constexpr unsigned int kGammaLookupSize = 1024; struct { std::array gammaTable; - uint8_t blackLevel; + double blackLevel; } gamma; }; From patchwork Fri Aug 30 07:25:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21072 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EBABCC32D7 for ; Fri, 30 Aug 2024 07:26:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4300F634EA; Fri, 30 Aug 2024 09:26:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="hqnilink"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 760D4634C8 for ; Fri, 30 Aug 2024 09:26:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002810; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zqi62KONqOYAkH9I2+EWHCyIrENNhFJ60R7QakIJ3pA=; b=hqnilink0p3J8aOH17jwMzSzo4lOA3Vx69ermPbpHPhcP0jd4sJvA6GCGEpRzkAWXLyIRG zSF36ivCWkBTDa6FuZta+Z4ENm27+QHqyMIL+Lah0UiAMS0QxNHn6qT39LpTGzsxlTtF+8 DAdNnffx0oSc6VuH6h2wEzj86owhqf0= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-644-JcISMMQzPFWGmEGws7071Q-1; Fri, 30 Aug 2024 03:26:47 -0400 X-MC-Unique: JcISMMQzPFWGmEGws7071Q-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 207191956048; Fri, 30 Aug 2024 07:26:46 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E8A6D19560A3; Fri, 30 Aug 2024 07:26:43 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 16/18] libcamera: software_isp: Use DelayedControls Date: Fri, 30 Aug 2024 09:25:52 +0200 Message-ID: <20240830072554.180672-17-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the standard libcamera mechanism to report the "current" controls rather than delaying updates by counting from the last update. MY SPECULATION -- valid or not?: A problem is that with software ISP we cannot be sure about the sensor delay. The original implementation simply skips exposure updates for 2 frames, which should be enough in most cases. After this change, we assume the delay being exactly 2 frames, which may or may not be correct and may work with outdated values if the delay is shorter. This patch also prepares moving exposure+gain to an algorithm module where the original delay mechanism would be a (possibly unnecessary) complication. Resolves software ISP TODO #11 + #12. Signed-off-by: Milan Zamazal --- src/ipa/simple/soft_simple.cpp | 16 +------------ src/libcamera/pipeline/simple/simple.cpp | 18 ++++++++++++--- src/libcamera/software_isp/TODO | 29 ------------------------ 3 files changed, 16 insertions(+), 47 deletions(-) diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 8f01bf7d..0e47b9c5 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -59,8 +59,7 @@ class IPASoftSimple : public ipa::soft::IPASoftInterface, public Module public: IPASoftSimple() : params_(nullptr), stats_(nullptr), - context_({ {}, {}, { kMaxFrameContexts } }), - ignoreUpdates_(0) + context_({ {}, {}, { kMaxFrameContexts } }) { } @@ -98,7 +97,6 @@ private: int32_t exposure_; double againMin_, againMax_, againMinStep_; double again_; - unsigned int ignoreUpdates_; }; IPASoftSimple::~IPASoftSimple() @@ -298,16 +296,6 @@ void IPASoftSimple::processStats(const uint32_t frame, /* \todo Switch to the libipa/algorithm.h API someday. */ - /* - * AE / AGC, use 2 frames delay to make sure that the exposure and - * the gain set have applied to the camera sensor. - * \todo This could be handled better with DelayedControls. - */ - if (ignoreUpdates_ > 0) { - --ignoreUpdates_; - return; - } - /* * Calculate Mean Sample Value (MSV) according to formula from: * https://www.araa.asn.au/acra/acra2007/papers/paper84final.pdf @@ -356,8 +344,6 @@ void IPASoftSimple::processStats(const uint32_t frame, ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(camHelper_ ? camHelper_->gainCode(again_) : again_)); - ignoreUpdates_ = 2; - setSensorControls.emit(ctrls); LOG(IPASoft, Debug) << "exposureMSV " << exposureMSV diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index ed50ab52..9a778994 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -32,6 +32,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/converter.h" +#include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" @@ -278,6 +279,8 @@ public: std::vector configs_; std::map> formats_; + std::unique_ptr delayedCtrls_; + std::vector> conversionBuffers_; std::queue> conversionQueue_; bool useConversion_; @@ -900,14 +903,13 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) { - /* \todo Use the DelayedControls class */ swIsp_->processStats(frame, bufferId, - sensor_->getControls({ V4L2_CID_ANALOGUE_GAIN, - V4L2_CID_EXPOSURE })); + delayedCtrls_->get(frame)); } void SimpleCameraData::setSensorControls(const ControlList &sensorControls) { + delayedCtrls_->push(sensorControls); ControlList ctrls(sensorControls); sensor_->setControls(&ctrls); } @@ -1288,6 +1290,16 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) if (outputCfgs.empty()) return 0; + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { 2, false } }, + { V4L2_CID_EXPOSURE, { 2, false } }, + }; + data->delayedCtrls_ = + std::make_unique(data->sensor_->device(), + params); + data->video_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + StreamConfiguration inputCfg; inputCfg.pixelFormat = pipeConfig->captureFormat; inputCfg.size = pipeConfig->captureSize; diff --git a/src/libcamera/software_isp/TODO b/src/libcamera/software_isp/TODO index 9978afc0..8eeede46 100644 --- a/src/libcamera/software_isp/TODO +++ b/src/libcamera/software_isp/TODO @@ -209,35 +209,6 @@ At some point, yes. --- -11. Improve handling the sensor controls which take effect with a delay - -> void IPASoftSimple::processStats(const ControlList &sensorControls) -> { -> ... -> /* -> * AE / AGC, use 2 frames delay to make sure that the exposure and -> * the gain set have applied to the camera sensor. -> */ -> if (ignore_updates_ > 0) { -> --ignore_updates_; -> return; -> } - -This could be handled better with DelayedControls. - ---- - -12. Use DelayedControls class in ispStatsReady() - -> void SimpleCameraData::ispStatsReady() -> { -> swIsp_->processStats(sensor_->getControls({ V4L2_CID_ANALOGUE_GAIN, -> V4L2_CID_EXPOSURE })); - -You should use the DelayedControls class. - ---- - 13. Improve black level and colour gains application I think the black level should eventually be moved before debayering, and From patchwork Fri Aug 30 07:25:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21073 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0E039C32DA for ; Fri, 30 Aug 2024 07:27:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3AB7F634E9; Fri, 30 Aug 2024 09:26:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="iQm7hxcf"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 78A99634DE for ; Fri, 30 Aug 2024 09:26:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002811; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mU0ahwN3yNF0hktTXAQJziNQSb/JJLkF4B9X9GsMCXQ=; b=iQm7hxcfpqHItz0VZ5zwkjjSR6FgR4GOWOvZRjs6vgkc09uaSE1C9LIl+hRNaWM9GPPQTX JuS3CvGqjGP0JZlUlrQJrAqJN2KHXU6fn6ObKKa0DhpzwkDci/LiPbFqFqQ1i9LMfRollB yqHfK81n+tKu7LgneGrWIAha0OwBDWU= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-680-v5716Q0aOXqdp9gNIyZfHg-1; Fri, 30 Aug 2024 03:26:50 -0400 X-MC-Unique: v5716Q0aOXqdp9gNIyZfHg-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E539E1955F3D; Fri, 30 Aug 2024 07:26:48 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id AEBE719560A3; Fri, 30 Aug 2024 07:26:46 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 17/18] libcamera: software_isp: Move exposure+gain to an algorithm module Date: Fri, 30 Aug 2024 09:25:53 +0200 Message-ID: <20240830072554.180672-18-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is the last step to fully convert software ISP to Algorithm-based processing. The newly introduced frameContext.sensor properties are set, and the updated code moved, before calling Algorithm::process() to have the values up-to-date in stats processing. Resolves software ISP TODO #10. Signed-off-by: Milan Zamazal --- src/ipa/simple/algorithms/agc.cpp | 139 ++++++++++++++++++++++ src/ipa/simple/algorithms/agc.h | 36 ++++++ src/ipa/simple/algorithms/meson.build | 1 + src/ipa/simple/data/uncalibrated.yaml | 1 + src/ipa/simple/ipa_context.cpp | 11 ++ src/ipa/simple/ipa_context.h | 17 +++ src/ipa/simple/soft_simple.cpp | 159 +++++--------------------- src/libcamera/software_isp/TODO | 10 -- 8 files changed, 235 insertions(+), 139 deletions(-) create mode 100644 src/ipa/simple/algorithms/agc.cpp create mode 100644 src/ipa/simple/algorithms/agc.h diff --git a/src/ipa/simple/algorithms/agc.cpp b/src/ipa/simple/algorithms/agc.cpp new file mode 100644 index 00000000..821b2971 --- /dev/null +++ b/src/ipa/simple/algorithms/agc.cpp @@ -0,0 +1,139 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Red Hat Inc. + * + * Exposure and gain + */ + +#include "agc.h" + +#include + +#include + +namespace libcamera { + +LOG_DEFINE_CATEGORY(IPASoftExposure) + +namespace ipa::soft::algorithms { + +/* + * The number of bins to use for the optimal exposure calculations. + */ +static constexpr unsigned int kExposureBinsCount = 5; + +/* + * The exposure is optimal when the mean sample value of the histogram is + * in the middle of the range. + */ +static constexpr float kExposureOptimal = kExposureBinsCount / 2.0; + +/* + * The below value implements the hysteresis for the exposure adjustment. + * It is small enough to have the exposure close to the optimal, and is big + * enough to prevent the exposure from wobbling around the optimal value. + */ +static constexpr float kExposureSatisfactory = 0.2; + +Agc::Agc() +{ +} + +void Agc::updateExposure(IPAContext &context, double exposureMSV) +{ + /* + * kExpDenominator of 10 gives ~10% increment/decrement; + * kExpDenominator of 5 - about ~20% + */ + static constexpr uint8_t kExpDenominator = 10; + static constexpr uint8_t kExpNumeratorUp = kExpDenominator + 1; + static constexpr uint8_t kExpNumeratorDown = kExpDenominator - 1; + + double next; + int32_t &exposure = context.activeState.agc.exposure; + double &again = context.activeState.agc.again; + + if (exposureMSV < kExposureOptimal - kExposureSatisfactory) { + next = exposure * kExpNumeratorUp / kExpDenominator; + if (next - exposure < 1) + exposure += 1; + else + exposure = next; + if (exposure >= context.configuration.agc.exposureMax) { + next = again * kExpNumeratorUp / kExpDenominator; + if (next - again < context.configuration.agc.againMinStep) + again += context.configuration.agc.againMinStep; + else + again = next; + } + } + + if (exposureMSV > kExposureOptimal + kExposureSatisfactory) { + if (exposure == context.configuration.agc.exposureMax && + again > context.configuration.agc.againMin) { + next = again * kExpNumeratorDown / kExpDenominator; + if (again - next < context.configuration.agc.againMinStep) + again -= context.configuration.agc.againMinStep; + else + again = next; + } else { + next = exposure * kExpNumeratorDown / kExpDenominator; + if (exposure - next < 1) + exposure -= 1; + else + exposure = next; + } + } + + exposure = std::clamp(exposure, context.configuration.agc.exposureMin, + context.configuration.agc.exposureMax); + again = std::clamp(again, context.configuration.agc.againMin, + context.configuration.agc.againMax); + + LOG(IPASoftExposure, Debug) + << "exposureMSV " << exposureMSV + << " exp " << exposure << " again " << again; +} + +void Agc::process(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + const SwIspStats *stats, + [[maybe_unused]] ControlList &metadata) +{ + /* + * Calculate Mean Sample Value (MSV) according to formula from: + * https://www.araa.asn.au/acra/acra2007/papers/paper84final.pdf + */ + const auto &histogram = stats->yHistogram; + const unsigned int blackLevelHistIdx = + context.activeState.black.level * SwIspStats::kYHistogramSize; + const unsigned int histogramSize = + SwIspStats::kYHistogramSize - blackLevelHistIdx; + const unsigned int yHistValsPerBin = histogramSize / kExposureBinsCount; + const unsigned int yHistValsPerBinMod = + histogramSize / (histogramSize % kExposureBinsCount + 1); + int exposureBins[kExposureBinsCount] = {}; + unsigned int denom = 0; + unsigned int num = 0; + + for (unsigned int i = 0; i < histogramSize; i++) { + unsigned int idx = (i - (i / yHistValsPerBinMod)) / yHistValsPerBin; + exposureBins[idx] += histogram[blackLevelHistIdx + i]; + } + + for (unsigned int i = 0; i < kExposureBinsCount; i++) { + LOG(IPASoftExposure, Debug) << i << ": " << exposureBins[i]; + denom += exposureBins[i]; + num += exposureBins[i] * (i + 1); + } + + float exposureMSV = (denom == 0 ? 0 : static_cast(num) / denom); + updateExposure(context, exposureMSV); +} + +REGISTER_IPA_ALGORITHM(Agc, "Agc") + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/agc.h b/src/ipa/simple/algorithms/agc.h new file mode 100644 index 00000000..a4aa656c --- /dev/null +++ b/src/ipa/simple/algorithms/agc.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Red Hat Inc. + * + * Exposure and gain + */ + +#pragma once + +#include "libcamera/internal/software_isp/swisp_stats.h" + +#include "algorithm.h" +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::soft::algorithms { + +class Agc : public Algorithm +{ +public: + Agc(); + ~Agc() = default; + + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const SwIspStats *stats, + ControlList &metadata) override; + +private: + void updateExposure(IPAContext &context, double exposureMSV); +}; + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/meson.build b/src/ipa/simple/algorithms/meson.build index 68e5aa58..ec2638c2 100644 --- a/src/ipa/simple/algorithms/meson.build +++ b/src/ipa/simple/algorithms/meson.build @@ -2,6 +2,7 @@ soft_simple_ipa_algorithms = files([ 'awb.cpp', + 'agc.cpp', 'blc.cpp', 'gamma.cpp', 'lut.cpp', diff --git a/src/ipa/simple/data/uncalibrated.yaml b/src/ipa/simple/data/uncalibrated.yaml index deaea6b1..324f98f4 100644 --- a/src/ipa/simple/data/uncalibrated.yaml +++ b/src/ipa/simple/data/uncalibrated.yaml @@ -7,4 +7,5 @@ algorithms: - Gamma: - Awb: - Lut: + - Agc: ... diff --git a/src/ipa/simple/ipa_context.cpp b/src/ipa/simple/ipa_context.cpp index 5fa492cb..3f94bbeb 100644 --- a/src/ipa/simple/ipa_context.cpp +++ b/src/ipa/simple/ipa_context.cpp @@ -77,6 +77,17 @@ namespace libcamera::ipa::soft { * \brief Gain of blue color */ +/** + * \var IPAActiveState::agc + * \brief Context for the AGC algorithm + * + * \var IPAActiveState::agc.exposure + * \brief Current exposure value + * + * \var IPAActiveState::agc.again + * \brief Current analog gain value + */ + /** * \var IPAActiveState::gamma * \brief Context for gamma in the Colors algorithm diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 28a02475..0e2096f8 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -9,6 +9,8 @@ #include +#include + #include namespace libcamera { @@ -17,6 +19,13 @@ namespace ipa::soft { struct IPASessionConfiguration { float gamma; + struct { + int32_t exposureMin, exposureMax; + double againMin, againMax, againMinStep; + } agc; + struct { + double level; + } black; }; struct IPAActiveState { @@ -28,6 +37,10 @@ struct IPAActiveState { double green; double blue; } gains; + struct { + int32_t exposure; + double again; + } agc; static constexpr unsigned int kGammaLookupSize = 1024; struct { std::array gammaTable; @@ -36,6 +49,10 @@ struct IPAActiveState { }; struct IPAFrameContext : public FrameContext { + struct { + uint32_t exposure; + double gain; + } sensor; }; struct IPAContext { diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 0e47b9c5..aef5f6e5 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -34,23 +34,6 @@ LOG_DEFINE_CATEGORY(IPASoft) namespace ipa::soft { -/* - * The number of bins to use for the optimal exposure calculations. - */ -static constexpr unsigned int kExposureBinsCount = 5; - -/* - * The exposure is optimal when the mean sample value of the histogram is - * in the middle of the range. - */ -static constexpr float kExposureOptimal = kExposureBinsCount / 2.0; - -/* - * The below value implements the hysteresis for the exposure adjustment. - * It is small enough to have the exposure close to the optimal, and is big - * enough to prevent the exposure from wobbling around the optimal value. - */ -static constexpr float kExposureSatisfactory = 0.2; /* Maximum number of frame contexts to be held */ static constexpr uint32_t kMaxFrameContexts = 16; @@ -58,8 +41,7 @@ class IPASoftSimple : public ipa::soft::IPASoftInterface, public Module { public: IPASoftSimple() - : params_(nullptr), stats_(nullptr), - context_({ {}, {}, { kMaxFrameContexts } }) + : context_({ {}, {}, { kMaxFrameContexts } }) { } @@ -92,11 +74,6 @@ private: /* Local parameter storage */ struct IPAContext context_; - - int32_t exposureMin_, exposureMax_; - int32_t exposure_; - double againMin_, againMax_, againMinStep_; - double again_; }; IPASoftSimple::~IPASoftSimple() @@ -207,20 +184,23 @@ int IPASoftSimple::configure(const IPAConfigInfo &configInfo) const ControlInfo &exposureInfo = sensorInfoMap_.find(V4L2_CID_EXPOSURE)->second; const ControlInfo &gainInfo = sensorInfoMap_.find(V4L2_CID_ANALOGUE_GAIN)->second; - exposureMin_ = exposureInfo.min().get(); - exposureMax_ = exposureInfo.max().get(); - if (!exposureMin_) { + context_.configuration.agc.exposureMin = exposureInfo.min().get(); + context_.configuration.agc.exposureMax = exposureInfo.max().get(); + if (!context_.configuration.agc.exposureMin) { LOG(IPASoft, Warning) << "Minimum exposure is zero, that can't be linear"; - exposureMin_ = 1; + context_.configuration.agc.exposureMin = 1; } int32_t againMin = gainInfo.min().get(); int32_t againMax = gainInfo.max().get(); if (camHelper_) { - againMin_ = camHelper_->gain(againMin); - againMax_ = camHelper_->gain(againMax); - againMinStep_ = (againMax_ - againMin_) / 100.0; + context_.configuration.agc.againMin = camHelper_->gain(againMin); + context_.configuration.agc.againMax = camHelper_->gain(againMax); + context_.configuration.agc.againMinStep = + (context_.configuration.agc.againMax - + context_.configuration.agc.againMin) / + 100.0; } else { /* * The camera sensor gain (g) is usually not equal to the value written @@ -232,13 +212,14 @@ int IPASoftSimple::configure(const IPAConfigInfo &configInfo) * the AGC algorithm (abrupt near one edge, and very small near the * other) we limit the range of the gain values used. */ - againMax_ = againMax; + context_.configuration.agc.againMax = againMax; if (!againMin) { LOG(IPASoft, Warning) << "Minimum gain is zero, that can't be linear"; - againMin_ = std::min(100, againMin / 2 + againMax / 2); + context_.configuration.agc.againMin = + std::min(100, againMin / 2 + againMax / 2); } - againMinStep_ = 1.0; + context_.configuration.agc.againMinStep = 1.0; } for (auto const &algo : algorithms()) { @@ -247,9 +228,12 @@ int IPASoftSimple::configure(const IPAConfigInfo &configInfo) return ret; } - LOG(IPASoft, Info) << "Exposure " << exposureMin_ << "-" << exposureMax_ - << ", gain " << againMin_ << "-" << againMax_ - << " (" << againMinStep_ << ")"; + LOG(IPASoft, Info) + << "Exposure " << context_.configuration.agc.exposureMin << "-" + << context_.configuration.agc.exposureMax + << ", gain " << context_.configuration.agc.againMin << "-" + << context_.configuration.agc.againMax + << " (" << context_.configuration.agc.againMinStep << ")"; return 0; } @@ -284,6 +268,12 @@ void IPASoftSimple::processStats(const uint32_t frame, const ControlList &sensorControls) { IPAFrameContext &frameContext = context_.frameContexts.get(frame); + + frameContext.sensor.exposure = + sensorControls.get(V4L2_CID_EXPOSURE).get(); + int32_t again = sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get(); + frameContext.sensor.gain = camHelper_ ? camHelper_->gain(again) : again; + /* * Software ISP currently does not produce any metadata. Use an empty * ControlList for now. @@ -294,37 +284,6 @@ void IPASoftSimple::processStats(const uint32_t frame, for (auto const &algo : algorithms()) algo->process(context_, frame, frameContext, stats_, metadata); - /* \todo Switch to the libipa/algorithm.h API someday. */ - - /* - * Calculate Mean Sample Value (MSV) according to formula from: - * https://www.araa.asn.au/acra/acra2007/papers/paper84final.pdf - */ - const uint8_t blackLevel = context_.activeState.black.level; - const unsigned int blackLevelHistIdx = - blackLevel / (256 / SwIspStats::kYHistogramSize); - const unsigned int histogramSize = - SwIspStats::kYHistogramSize - blackLevelHistIdx; - const unsigned int yHistValsPerBin = histogramSize / kExposureBinsCount; - const unsigned int yHistValsPerBinMod = - histogramSize / (histogramSize % kExposureBinsCount + 1); - int exposureBins[kExposureBinsCount] = {}; - unsigned int denom = 0; - unsigned int num = 0; - - for (unsigned int i = 0; i < histogramSize; i++) { - unsigned int idx = (i - (i / yHistValsPerBinMod)) / yHistValsPerBin; - exposureBins[idx] += stats_->yHistogram[blackLevelHistIdx + i]; - } - - for (unsigned int i = 0; i < kExposureBinsCount; i++) { - LOG(IPASoft, Debug) << i << ": " << exposureBins[i]; - denom += exposureBins[i]; - num += exposureBins[i] * (i + 1); - } - - float exposureMSV = static_cast(num) / denom; - /* Sanity check */ if (!sensorControls.contains(V4L2_CID_EXPOSURE) || !sensorControls.contains(V4L2_CID_ANALOGUE_GAIN)) { @@ -332,72 +291,14 @@ void IPASoftSimple::processStats(const uint32_t frame, return; } - exposure_ = sensorControls.get(V4L2_CID_EXPOSURE).get(); - int32_t again = sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get(); - again_ = camHelper_ ? camHelper_->gain(again) : again; - - updateExposure(exposureMSV); - ControlList ctrls(sensorInfoMap_); - ctrls.set(V4L2_CID_EXPOSURE, exposure_); + auto &againNew = context_.activeState.agc.again; + ctrls.set(V4L2_CID_EXPOSURE, context_.activeState.agc.exposure); ctrls.set(V4L2_CID_ANALOGUE_GAIN, - static_cast(camHelper_ ? camHelper_->gainCode(again_) : again_)); + static_cast(camHelper_ ? camHelper_->gainCode(againNew) : againNew)); setSensorControls.emit(ctrls); - - LOG(IPASoft, Debug) << "exposureMSV " << exposureMSV - << " exp " << exposure_ << " again " << again_ - << " gain R/B " << context_.activeState.gains.red - << "/" << context_.activeState.gains.blue - << " black level " << static_cast(blackLevel); -} - -void IPASoftSimple::updateExposure(double exposureMSV) -{ - /* - * kExpDenominator of 10 gives ~10% increment/decrement; - * kExpDenominator of 5 - about ~20% - */ - static constexpr uint8_t kExpDenominator = 10; - static constexpr uint8_t kExpNumeratorUp = kExpDenominator + 1; - static constexpr uint8_t kExpNumeratorDown = kExpDenominator - 1; - - double next; - - if (exposureMSV < kExposureOptimal - kExposureSatisfactory) { - next = exposure_ * kExpNumeratorUp / kExpDenominator; - if (next - exposure_ < 1) - exposure_ += 1; - else - exposure_ = next; - if (exposure_ >= exposureMax_) { - next = again_ * kExpNumeratorUp / kExpDenominator; - if (next - again_ < againMinStep_) - again_ += againMinStep_; - else - again_ = next; - } - } - - if (exposureMSV > kExposureOptimal + kExposureSatisfactory) { - if (exposure_ == exposureMax_ && again_ > againMin_) { - next = again_ * kExpNumeratorDown / kExpDenominator; - if (again_ - next < againMinStep_) - again_ -= againMinStep_; - else - again_ = next; - } else { - next = exposure_ * kExpNumeratorDown / kExpDenominator; - if (exposure_ - next < 1) - exposure_ -= 1; - else - exposure_ = next; - } - } - - exposure_ = std::clamp(exposure_, exposureMin_, exposureMax_); - again_ = std::clamp(again_, againMin_, againMax_); } std::string IPASoftSimple::logPrefix() const diff --git a/src/libcamera/software_isp/TODO b/src/libcamera/software_isp/TODO index 8eeede46..a50db668 100644 --- a/src/libcamera/software_isp/TODO +++ b/src/libcamera/software_isp/TODO @@ -199,16 +199,6 @@ Yes, because, well... all the other IPAs were doing that... --- -10. Switch to libipa/algorithm.h API in processStats - ->> void IPASoftSimple::processStats(const ControlList &sensorControls) ->> -> Do you envision switching to the libipa/algorithm.h API at some point ? - -At some point, yes. - ---- - 13. Improve black level and colour gains application I think the black level should eventually be moved before debayering, and From patchwork Fri Aug 30 07:25:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21074 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 138C1C32DB for ; Fri, 30 Aug 2024 07:27:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9F1BB634E9; Fri, 30 Aug 2024 09:27:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="dVvyKtmZ"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A4E126341E for ; Fri, 30 Aug 2024 09:26:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725002815; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=G6Rb7ZNQHHJHzus5nxcQH64kx4ElvfE8MPFWA9uBZ+E=; b=dVvyKtmZiw3rDB18vYMR5XPmTsUYtWJ1NmBnwlCsC4PqiK63UMzp08XERaKdCJE00GBCTM ZmjP82VYKFC4PWQ5tmhJ1kC8YmjuNc6jvix/itolpVY+0Nl+zs4IZBdSbu18lAb4PoTZSG ah0ekKw8bBICTWw43JmOspmBmVSH0Ho= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-678-o-iSRPTeOR64iG27gIXTLA-1; Fri, 30 Aug 2024 03:26:52 -0400 X-MC-Unique: o-iSRPTeOR64iG27gIXTLA-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4DBBA19560A2; Fri, 30 Aug 2024 07:26:51 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.65]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 436F119560A3; Fri, 30 Aug 2024 07:26:48 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally Subject: [PATCH v5 18/18] libcamera: software_isp: Update black level only on exposure changes Date: Fri, 30 Aug 2024 09:25:54 +0200 Message-ID: <20240830072554.180672-19-mzamazal@redhat.com> In-Reply-To: <20240830072554.180672-1-mzamazal@redhat.com> References: <20240830072554.180672-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The black level is likely to get updated, if ever, only after exposure or gain changes. Don't compute its possible updates if exposure and gain are unchanged. It's probably not worth trying to implement something more sophisticated. Better to spend the effort on supporting tuning files. Signed-off-by: Milan Zamazal --- src/ipa/simple/algorithms/blc.cpp | 9 ++++++++- src/ipa/simple/algorithms/blc.h | 4 ++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/src/ipa/simple/algorithms/blc.cpp b/src/ipa/simple/algorithms/blc.cpp index ab7e7dd9..d01c2c14 100644 --- a/src/ipa/simple/algorithms/blc.cpp +++ b/src/ipa/simple/algorithms/blc.cpp @@ -30,10 +30,15 @@ int BlackLevel::init(IPAContext &context, void BlackLevel::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, const SwIspStats *stats, [[maybe_unused]] ControlList &metadata) { + if (frameContext.sensor.exposure == exposure_ && + frameContext.sensor.gain == gain_) { + return; + } + const SwIspStats::Histogram &histogram = stats->yHistogram; /* @@ -54,6 +59,8 @@ void BlackLevel::process(IPAContext &context, if (seen >= pixelThreshold) { context.activeState.black.level = static_cast(i) / SwIspStats::kYHistogramSize; + exposure_ = frameContext.sensor.exposure; + gain_ = frameContext.sensor.gain; LOG(IPASoftBL, Debug) << "Auto-set black level: " << i << "/" << SwIspStats::kYHistogramSize diff --git a/src/ipa/simple/algorithms/blc.h b/src/ipa/simple/algorithms/blc.h index 2f73db52..9b9daab1 100644 --- a/src/ipa/simple/algorithms/blc.h +++ b/src/ipa/simple/algorithms/blc.h @@ -30,6 +30,10 @@ public: IPAFrameContext &frameContext, const SwIspStats *stats, ControlList &metadata) override; + +private: + uint32_t exposure_; + double gain_; }; } /* namespace ipa::soft::algorithms */