From patchwork Thu Oct 22 09:40:43 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28551 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 33036C3261 for ; Thu, 22 Oct 2026 09:41:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DF85868F63; Thu, 22 Oct 2026 11:41:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="ZnggIWCw"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 67D5168ECB for ; Thu, 22 Oct 2026 11:41:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1792662084; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UDuu4GgQJXJhm9n3X7TiwI4xDsNPVDtlY/SGrOUiaz0=; b=ZnggIWCwooOktteVPzprRrL6x886KWzt9b8Xsn2EPAWBBtQIVmFqsOTKcHtjow8kCOs8w2 H8dU08fi50bq8/D3OnH1b6Et8It/ceDApynuvm4NVaKb3tmawN5HM8h/dmsILhoPL3QTW4 Upd4sn6mf7OtwbK+Ub082c3ngDJF4X0= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-618-qQ7d5m2ZPnmhGv5BHF0qIw-1; Thu, 22 Oct 2026 09:41:21 +0000 X-MC-Unique: qQ7d5m2ZPnmhGv5BHF0qIw-1 X-Mimecast-MFC-AGG-ID: qQ7d5m2ZPnmhGv5BHF0qIw_1792662080 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 561891800597; Thu, 22 Oct 2026 09:41:19 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.redhat.corp (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D565518004D2; Thu, 22 Oct 2026 09:41:16 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Xander Pronk , Kieran Bingham , Bryan O'Donoghue , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Rick ten Wolde , Milan Zamazal Subject: [PATCH v16 1/6] libcamera: software_isp: egl: Add filter parameter to createTexture2D() Date: Thu, 22 Oct 2026 11:40:43 +0200 Message-ID: <20261022094048.20277-2-mzamazal@redhat.com> In-Reply-To: <20261022094048.20277-1-mzamazal@redhat.com> References: <20261022094048.20277-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: qf2ZytZ1EcvjrjEvM6a9ms_E1Y7LHVVS9lzi1i0ZWdY_1792662080 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Xander Pronk Add `filter' parameter to createTexture2D() to allow overriding the currently hardcoded GL_NEAREST value. This is needed for grid-based lens shading interpolation, which is implemented in followup patches. Reviewed-by: Bryan O'Donoghue Reviewed-by: Barnabás Pőcze Co-developed-by: Rick ten Wolde Signed-off-by: Rick ten Wolde Signed-off-by: Xander Pronk Signed-off-by: Milan Zamazal --- include/libcamera/internal/egl.h | 2 +- src/libcamera/egl.cpp | 11 ++++++----- src/libcamera/software_isp/debayer_egl.cpp | 2 +- 3 files changed, 8 insertions(+), 7 deletions(-) diff --git a/include/libcamera/internal/egl.h b/include/libcamera/internal/egl.h index 9d2c9b5bc..dc4c6fdb6 100644 --- a/include/libcamera/internal/egl.h +++ b/include/libcamera/internal/egl.h @@ -111,7 +111,7 @@ public: int createInputDMABufTexture2D(eGLImage &eglImage, int fd); int createOutputDMABufTexture2D(eGLImage &eglImage, int fd); - void createTexture2D(eGLImage &eglImage, void *data); + void createTexture2D(eGLImage &eglImage, void *data, GLint filter); void updateTexture2D(eGLImage &eglImage, void *data); void createOutputTexture2D(eGLImage &eglImage); diff --git a/src/libcamera/egl.cpp b/src/libcamera/egl.cpp index 3cde9040a..4c89eea05 100644 --- a/src/libcamera/egl.cpp +++ b/src/libcamera/egl.cpp @@ -279,13 +279,14 @@ int eGL::createOutputDMABufTexture2D(eGLImage &eglImage, int fd) * \brief Create a 2D texture from a memory buffer * \param[in,out] eglImage EGL image to associate with the texture * \param[in] data Pointer to pixel data, or nullptr for uninitialised texture + * \param[in] filter GL texture filter setting * * Creates a 2D texture from a CPU-accessible memory buffer. The texture - * is configured with nearest filtering and clamp-to-edge wrapping. This + * is configured the specified filtering and clamp-to-edge wrapping. This * is useful for uploading static data like lookup tables or uniform color * matrices to the GPU. */ -void eGL::createTexture2D(eGLImage &eglImage, void *data) +void eGL::createTexture2D(eGLImage &eglImage, void *data, GLint filter) { assertThread(); @@ -295,8 +296,8 @@ void eGL::createTexture2D(eGLImage &eglImage, void *data) glTexImage2D(GL_TEXTURE_2D, 0, eglImage.format_, eglImage.width_, eglImage.height_, 0, eglImage.format_, GL_UNSIGNED_BYTE, data); // Nearest filtering - glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); - glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, filter); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, filter); // Wrap to edge to avoid edge artifacts glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); @@ -357,7 +358,7 @@ void eGL::updateTexture2D(eGLImage &eglImage, void *data) */ void eGL::createOutputTexture2D(eGLImage &eglImage) { - createTexture2D(eglImage, NULL); + createTexture2D(eglImage, NULL, GL_NEAREST); attachTextureToFBO(eglImage); } diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index 300822f9d..23dee13e7 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -538,7 +538,7 @@ eGLImage *DebayerEGL::getCachedInputFrameBuffer(FrameBuffer *input, std::optiona return nullptr; } if (cache_miss) - egl_.createTexture2D(*eglImageIn, inMapped->value().planes()[0].data()); + egl_.createTexture2D(*eglImageIn, inMapped->value().planes()[0].data(), GL_NEAREST); else egl_.updateTexture2D(*eglImageIn, inMapped->value().planes()[0].data()); From patchwork Thu Oct 22 09:40:44 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28552 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2F8EDC3261 for ; Thu, 22 Oct 2026 09:41:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D067F68F6E; Thu, 22 Oct 2026 11:41:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="WZbvKI15"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 67CD868ECB for ; Thu, 22 Oct 2026 11:41:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1792662087; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cHYVu7Jv+0IjviZNqa8pM7okZyXPN4OnxGgo2IY3CBA=; b=WZbvKI15n8xheqadIu3OF8mrMp1Vmsxx/atQW9cjtvRoDAwZQi8c2P7cS3RYx0h5sG+b0V ZZUlNYe8G41AfAnUcZFZTxEu1A5GV7j/jLkXL7/GFauBdnbkXO86tO0Do3yrYSf2b/JGbd HqmDbHpd/5cdpZhDkBqizHU/zSUdxfY= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-107-JHsmh3yDOja2DHT6xTcucQ-1; Thu, 22 Oct 2026 05:41:23 -0400 X-MC-Unique: JHsmh3yDOja2DHT6xTcucQ-1 X-Mimecast-MFC-AGG-ID: JHsmh3yDOja2DHT6xTcucQ_1792662082 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 06319195605A; Thu, 22 Oct 2026 09:41:22 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.redhat.corp (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id DF67118004D2; Thu, 22 Oct 2026 09:41:19 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Xander Pronk , Kieran Bingham , Rick ten Wolde , Milan Zamazal Subject: [PATCH v16 2/6] libcamera: software_isp: Add LSC data to DebayerParams Date: Thu, 22 Oct 2026 11:40:44 +0200 Message-ID: <20261022094048.20277-3-mzamazal@redhat.com> In-Reply-To: <20261022094048.20277-1-mzamazal@redhat.com> References: <20261022094048.20277-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: jRIEYYnZJbKFMNCXhlsl4VI03thACC6OAQzWgtWwTOs_1792662082 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Xander Pronk Add a lookup table for grid-based lens shading correction to debayer params. The lookup table is in the format of an RGBA texture to be passed to debayering. The alpha value serves only for padding and can be arbitrary; OpenGL wants by default rows aligned to 4 bytes and using RGBA format is the simplest way to achieve it. The parameter is filled in by Lsc algorithm implemented in a follow-up patch. If the algorithm is not enabled, the table is filled in with values corresponding to identity multiplication (1.0). In order the image processing would be able to update its internal data (e.g. the GPU texture) on and only on the lookup table changes, lscLutVersion parameter is added, which is incremented on each lookup table change. The parameters are currently unused, their handling is implemented in follow-up patches. The grid size is selected to be the same as in the tuning files already present in rkisp1. Another popular grid size in other pipelines is 32. Co-developed-by: Rick ten Wolde Signed-off-by: Rick ten Wolde Signed-off-by: Xander Pronk Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham --- .../internal/software_isp/debayer_params.h | 13 +++++++++ src/libcamera/software_isp/debayer.cpp | 28 +++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index 1074720d7..0d9196874 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -10,6 +10,7 @@ #pragma once +#include #include #include "libcamera/internal/matrix.h" @@ -25,6 +26,18 @@ struct DebayerParams { float gamma = 1.0; float contrastExp = 1.0; RGB gains = RGB({ 1.0, 1.0, 1.0 }); + + /** + * To prevent OpenGL alignment issues, the number of bytes in each row + * must be a multiple of 4 (the initial value of GL_UNPACK_ALIGNMENT) + * and lscLut is also aligned to 4 bytes. + **/ + static constexpr unsigned int kLscGridSize = 17; + static constexpr unsigned int kLscValuesPerCell = 4; + using LscLookupTable = + std::array; + alignas(4) LscLookupTable lscLut{}; + uint64_t lscLutVersion = 0; }; } /* namespace libcamera */ diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index a4854e51b..82cdf1dc2 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -43,6 +43,34 @@ namespace libcamera { * \brief Contrast value to be used as an exponent */ +/** + * \var DebayerParams::kLscGridSize + * \brief Number of lens shading grid areas in one direction + */ + +/** + * \var DebayerParams::kLscValuesPerCell + * \brief Number of pixel values per each of the lens shading grid areas + */ + +/** + * \typedef DebayerParams::LscLookupTable + * \brief Lookup table for lens shading correction + * + * It's an array of values to be later used as a texture. + * The values are in row - column - RGB order. + */ + +/** + * \var DebayerParams::lscLut + * \brief Lens shading lookup table + */ + +/** + * \var DebayerParams::lscLutVersion + * \brief Incremented on each \a lscLut change + */ + /** * \class Debayer * \brief Base debayering class From patchwork Thu Oct 22 09:40:45 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28553 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1FD76C3261 for ; Thu, 22 Oct 2026 09:41:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 98A9C68F76; Thu, 22 Oct 2026 11:41:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="bXrPmHWQ"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D43F368F6F for ; Thu, 22 Oct 2026 11:41:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1792662089; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YTvD5kvSrXSJ0ekJegsKH/JLo76GyYot4K3y17M3gfQ=; b=bXrPmHWQIELBjZDXJc5DoY2CKkW3MwyDb27N3nldNnV0phVIe80ixSdLS6KwtU64yCJTbx Cc8fTC1Oi6lgAdVmqw6hA5+LuOvzv+KXnmxSTRYrfEzffMkQuazvZ4nnt1hoXabyOm1LnO +89PLn+mosNbTUH19dNS2ahtDrWGt/0= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-649-kCO_-43DPH27sYkI3y9NOQ-1; Thu, 22 Oct 2026 09:41:25 +0000 X-MC-Unique: kCO_-43DPH27sYkI3y9NOQ-1 X-Mimecast-MFC-AGG-ID: kCO_-43DPH27sYkI3y9NOQ_1792662084 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 259491800576; Thu, 22 Oct 2026 09:41:24 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.redhat.corp (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4CED71800643; Thu, 22 Oct 2026 09:41:22 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Xander Pronk , Kieran Bingham , Rick ten Wolde , Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= Subject: [PATCH v16 3/6] ipa: simple: Add LSC algorithm Date: Thu, 22 Oct 2026 11:40:45 +0200 Message-ID: <20261022094048.20277-4-mzamazal@redhat.com> In-Reply-To: <20261022094048.20277-1-mzamazal@redhat.com> References: <20261022094048.20277-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 4TQZCBm68deQZR4LX1aKDwwvXP3qkFMfHXJUNVFz7dM_1792662084 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Xander Pronk The algorithm is based on the common libipa lens shading correction implementation. The grid values obtained from the libipa algorithm are passed to the debayer algorithm as an array and used as an an RGB texture. Notes on the implementation: - The overall idea is to keep things simple, to not make the LSC computation unnecessarily expensive. - LscAlgorithm accepts only quantised types. UQ<2,6> is used, to be converted to float in debayering. - The limit of 100 degrees to consider a temperature change noticeable is arbitrary. Co-developed-by: Rick ten Wolde Signed-off-by: Rick ten Wolde Signed-off-by: Xander Pronk Signed-off-by: Milan Zamazal Tested-by: Barnabás Pőcze Reviewed-by: Kieran Bingham --- .../internal/software_isp/debayer_params.h | 12 +- src/ipa/softisp/algorithms/lsc.cpp | 104 ++++++++++++++++++ src/ipa/softisp/algorithms/lsc.h | 51 +++++++++ src/ipa/softisp/algorithms/meson.build | 1 + src/ipa/softisp/ipa_context.h | 5 + src/ipa/softisp/softisp.cpp | 2 + src/libcamera/software_isp/debayer.cpp | 8 ++ 7 files changed, 182 insertions(+), 1 deletion(-) create mode 100644 src/ipa/softisp/algorithms/lsc.cpp create mode 100644 src/ipa/softisp/algorithms/lsc.h diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index 0d9196874..2ae1614b3 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -36,7 +36,17 @@ struct DebayerParams { static constexpr unsigned int kLscValuesPerCell = 4; using LscLookupTable = std::array; - alignas(4) LscLookupTable lscLut{}; + static constexpr auto identityLscLut = [] { + LscLookupTable lut = {}; + /* lut.fill(64) could be used, but it fails with older gcc versions */ + for (size_t i = 0; kLscValuesPerCell * i < lut.size(); i++) { + lut[i * kLscValuesPerCell + 0] = 64; /* == UQ<2, 6>(1.0f).quantized() */ + lut[i * kLscValuesPerCell + 1] = 64; + lut[i * kLscValuesPerCell + 2] = 64; + } + return lut; + }(); + alignas(4) LscLookupTable lscLut = identityLscLut; uint64_t lscLutVersion = 0; }; diff --git a/src/ipa/softisp/algorithms/lsc.cpp b/src/ipa/softisp/algorithms/lsc.cpp new file mode 100644 index 000000000..8316c7e9b --- /dev/null +++ b/src/ipa/softisp/algorithms/lsc.cpp @@ -0,0 +1,104 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Lens shading correction + */ + +#include "lsc.h" + +#include + +namespace libcamera { + +namespace ipa::softisp::algorithms { + +LOG_DEFINE_CATEGORY(IPASoftIspLsc) + +int Lsc::init(IPAContext &context, const ValueNode &tuningData) +{ + static constexpr unsigned int kGridSize = DebayerParams::kLscGridSize; + + for (unsigned int i = 0; i < kGridSize; i++) + gridPos_.push_back(static_cast(i) / (kGridSize - 1)); + + return lscAlgo_.init(tuningData, context.ctrlMap, + { .keys = { "r", "g", "b" }, + .numHSamples = kGridSize, + .numVSamples = kGridSize, + .sensorSize = context.sensorInfo.activeArea.size() }); +} + +int Lsc::configure(IPAContext &context, + [[maybe_unused]] const IPAConfigInfo &configInfo) +{ + return lscAlgo_.configure(context.activeState.lsc, + context.sensorInfo.analogCrop, + gridPos_, gridPos_); +} + +void Lsc::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + DebayerParams *params) +{ + unsigned int ct = frameContext.awb.colourTemperature; + constexpr unsigned int minTemperatureChange = 100; + + if (!frameContext.lsc.enabled) { + if (lastAppliedCt_ != 0) { + params->lscLut = DebayerParams::identityLscLut; + params->lscLutVersion++; + lastAppliedCt_ = 0; + + LOG(IPASoftIspLsc, Debug) + << "LSC disabled (version " << params->lscLutVersion << ")"; + } + return; + } + + if (utils::abs_diff(ct, lastAppliedCt_) < minTemperatureChange) + return; + + const auto &set = lscAlgo_.interpolateComponents(ct); + + const auto &red = set.at("r"); + const auto &green = set.at("g"); + const auto &blue = set.at("b"); + + DebayerParams::LscLookupTable &lut = params->lscLut; + constexpr unsigned int gridSize = DebayerParams::kLscGridSize; + for (unsigned int i = 0, j = 0; i < gridSize * gridSize; i++) { + lut[j++] = red[i]; + lut[j++] = green[i]; + lut[j++] = blue[i]; + lut[j++] = 0; /* padding */ + } + params->lscLutVersion++; + + LOG(IPASoftIspLsc, Debug) + << "LSC updated to version " << params->lscLutVersion + << " and temperature " << ct << " (was " << lastAppliedCt_ << ")"; + + lastAppliedCt_ = ct; +} + +void Lsc::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, const ControlList &controls) +{ + lscAlgo_.queueRequest(context.activeState.lsc, frameContext.lsc, + controls); +} + +void Lsc::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const SwIspStats *stats, + ControlList &metadata) +{ + lscAlgo_.process(frameContext.lsc, metadata); +} + +REGISTER_IPA_ALGORITHM(Lsc, "Lsc") + +} /* namespace ipa::softisp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/softisp/algorithms/lsc.h b/src/ipa/softisp/algorithms/lsc.h new file mode 100644 index 000000000..83ec5aec2 --- /dev/null +++ b/src/ipa/softisp/algorithms/lsc.h @@ -0,0 +1,51 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Lens shading correction + */ + +#pragma once + +#include + +#include "libipa/fixedpoint.h" +#include "libipa/lsc.h" + +#include "algorithm.h" +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::softisp::algorithms { + +class Lsc : public Algorithm +{ +public: + Lsc() = default; + ~Lsc() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPAConfigInfo &configInfo) override; + void queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, const ControlList &controls) override; + void prepare(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + DebayerParams *params) override; + void process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const SwIspStats *stats, + ControlList &metadata) override; + +private: + LscAlgorithm> lscAlgo_; + + std::vector gridPos_; + + unsigned int lastAppliedCt_ = 0; +}; + +} /* namespace ipa::softisp::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/softisp/algorithms/meson.build b/src/ipa/softisp/algorithms/meson.build index d240409e5..d98f4b1f4 100644 --- a/src/ipa/softisp/algorithms/meson.build +++ b/src/ipa/softisp/algorithms/meson.build @@ -6,4 +6,5 @@ softisp_ipa_algorithms = files([ 'agc.cpp', 'blc.cpp', 'ccm.cpp', + 'lsc.cpp', ]) diff --git a/src/ipa/softisp/ipa_context.h b/src/ipa/softisp/ipa_context.h index 397e6c99b..00326e6c7 100644 --- a/src/ipa/softisp/ipa_context.h +++ b/src/ipa/softisp/ipa_context.h @@ -22,6 +22,7 @@ #include #include #include +#include "libipa/lsc.h" #include "core_ipa_interface.h" @@ -55,6 +56,8 @@ struct IPAActiveState { std::optional contrast; std::optional saturation; } knobs; + + ipa::lsc::ActiveState lsc; }; struct IPAFrameContext : public FrameContext { @@ -70,6 +73,7 @@ struct IPAFrameContext : public FrameContext { float gamma; std::optional contrast; std::optional saturation; + ipa::lsc::FrameContext lsc; }; struct IPAContext { @@ -86,6 +90,7 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; bool ccmEnabled = false; + ipa::lsc::ActiveState lsc; }; } /* namespace ipa::softisp */ diff --git a/src/ipa/softisp/softisp.cpp b/src/ipa/softisp/softisp.cpp index 8acfcd1a0..420fee745 100644 --- a/src/ipa/softisp/softisp.cpp +++ b/src/ipa/softisp/softisp.cpp @@ -164,6 +164,8 @@ int IPASoftIsp::init(const IPASettings &settings, params_->gamma = 1.0 / algorithms::kDefaultGamma; params_->contrastExp = 1.0; params_->gains = { { 1.0, 1.0, 1.0 } }; + params_->lscLutVersion = 0; + params_->lscLut = DebayerParams::identityLscLut; /* combinedMatrix is reset for each frame. */ } diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index 82cdf1dc2..6ebf7ed1a 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -66,6 +66,14 @@ namespace libcamera { * \brief Lens shading lookup table */ +/** + * \var DebayerParams::identityLscLut + * \brief Lens shading lookup table of identity mapping + * + * This table can be used to initialise the lens shading lookup table or to + * not apply any real lens shading correction. + */ + /** * \var DebayerParams::lscLutVersion * \brief Incremented on each \a lscLut change From patchwork Thu Oct 22 09:40:46 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28554 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 38641C3261 for ; Thu, 22 Oct 2026 09:41:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C591B68F78; Thu, 22 Oct 2026 11:41:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="cVrsBVWR"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8D2E768F5E for ; Thu, 22 Oct 2026 11:41:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1792662089; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CM8FHC7s6yx6pU4x75EC1tOcxwQsxNTgnSiGYjluz5o=; b=cVrsBVWRF+1hHvU6ZG/35TteSrnMCdny46Hq7WpKXZfAq9qBmpU8WHXD6+e86rYGbRsv2S 8+7H9pNFPgOQo+HixbGzdJACxW4Ms4FXHbaxORQbPsN3EOARaaKx07e0M0bUtCiWOyoMY8 gv+EOnozUInUMBM8qvPYxYBMq5P6s1U= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-643-jlMnygmROhusIF0Ga9bRCw-1; Thu, 22 Oct 2026 09:41:28 +0000 X-MC-Unique: jlMnygmROhusIF0Ga9bRCw-1 X-Mimecast-MFC-AGG-ID: jlMnygmROhusIF0Ga9bRCw_1792662087 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 69117180058A; Thu, 22 Oct 2026 09:41:26 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.redhat.corp (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 825541800642; Thu, 22 Oct 2026 09:41:24 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Xander Pronk , Kieran Bingham , Rick ten Wolde , Milan Zamazal , Bryan O'Donoghue , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= Subject: [PATCH v16 4/6] libcamera: shaders: Add LSC support Date: Thu, 22 Oct 2026 11:40:46 +0200 Message-ID: <20261022094048.20277-5-mzamazal@redhat.com> In-Reply-To: <20261022094048.20277-1-mzamazal@redhat.com> References: <20261022094048.20277-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: zpXCodhTqZ2LhhUSUJE9xb3LSU_g_ndTtxj9ZBZ7hVk_1792662087 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Xander Pronk Lens shading correction should be applied after black level subtraction (in order to make the computations with meaningful values) and before white balance (especially before white balance stats are computed). Note that lens shading correction depends on temperature, which is computed from the preceding, rather than current, frame (this is due to how white balance is currently computed). The shaders are compiled on initialisation, while lens shading correction can be enabled or disabled dynamically, using the corresponding control. The whole correction in the shader is wrapped by a conditional macro to be able to disable it and all the related overheads if Lsc algorithm is not enabled at all. If Lsc algorithm is enabled, the correction is always applied, whether the given control is enabled or not. If it is not, 1.0 multipliers are expected to be provided. This is because lens shading correction is supposed to be rarely switched off using the runtime control when Lsc algorithm is enabled, e.g. only for demoing or debugging purposes. We prefer accepting the performance penalty of applying a no-op correction in such cases rather than disturbing GPU computation in the regular flow by introducing a runtime `if'. lsc_tex texture contains 8-bit UQ<2, 6> quantized data as provided by the IPA algorithm. It is converted to float in the shader by multiplying it by 4.0. Co-developed-by: Rick ten Wolde Signed-off-by: Rick ten Wolde Signed-off-by: Xander Pronk Signed-off-by: Milan Zamazal Reviewed-by: Bryan O'Donoghue Tested-by: Barnabás Pőcze --- src/libcamera/shaders/bayer_1x_packed.frag | 12 ++++++++++++ src/libcamera/shaders/bayer_unpacked.frag | 9 +++++++++ 2 files changed, 21 insertions(+) diff --git a/src/libcamera/shaders/bayer_1x_packed.frag b/src/libcamera/shaders/bayer_1x_packed.frag index 0b641a5f3..e61334070 100644 --- a/src/libcamera/shaders/bayer_1x_packed.frag +++ b/src/libcamera/shaders/bayer_1x_packed.frag @@ -71,6 +71,10 @@ uniform vec3 blacklevel; uniform float gamma; uniform float contrastExp; +#if defined(APPLY_LSC) +uniform sampler2D lsc_tex; +#endif + float apply_contrast(float value) { // Apply simple S-curve @@ -232,6 +236,14 @@ void main(void) */ rgb = (rgb - blacklevel) / (1.0 - blacklevel); +#if defined(APPLY_LSC) + /* Multiple by 4.0 for UQ<2, 6> -> float conversion: + * texture2D(...) = original_float * 2^6 (quantization) * 1/2^8 (texture byte to float) + * => original_float = texture2D(...) / 2^6 * 2^8 = texture2D(...) * 2^2 + */ + rgb = rgb * (texture2D(lsc_tex, textureOut).rgb * 4.0); +#endif + /* Apply AWB gains, and saturate each channel at sensor range */ rgb = clamp(rgb * awb, vec3(0.0), vec3(1.0)); diff --git a/src/libcamera/shaders/bayer_unpacked.frag b/src/libcamera/shaders/bayer_unpacked.frag index 10c5e941b..be2f89dd0 100644 --- a/src/libcamera/shaders/bayer_unpacked.frag +++ b/src/libcamera/shaders/bayer_unpacked.frag @@ -30,6 +30,10 @@ uniform vec3 blacklevel; uniform float gamma; uniform float contrastExp; +#if defined(APPLY_LSC) +uniform sampler2D lsc_tex; +#endif + float apply_contrast(float value) { // Apply simple S-curve @@ -135,6 +139,11 @@ void main(void) { */ rgb = (rgb - blacklevel) / (1.0 - blacklevel); +#if defined(APPLY_LSC) + /* Multiple by 4.0 for UQ<2, 6> -> float conversion */ + rgb = rgb * (texture2D(lsc_tex, center.xy).rgb * 4.0); +#endif + /* Apply AWB gains, and saturate each channel at sensor range */ rgb = clamp(rgb * awb, vec3(0.0), vec3(1.0)); From patchwork Thu Oct 22 09:40:47 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28555 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A5632C3261 for ; Thu, 22 Oct 2026 09:41:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7F2F468F72; Thu, 22 Oct 2026 11:41:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="fBrgB0ZF"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1C7DD68F67 for ; Thu, 22 Oct 2026 11:41:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1792662091; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HTMGjAESVZ+YoZMeZb8l6w2s7T/dLSIoz7iWB/Ax8gQ=; b=fBrgB0ZFMYDrkSbKiBvge2BRnAAb9i/HjDruZ5Cr3UpcRmjRfY8/6Gy5k0EapiLvDLBqke E6leQj1d8CRLwYM9CiM3gykCnOxr8CubWxqGuJwdYbc2LaAE4ZyggQ6m6rwgsaXpvzUhc0 boHepW52/NctnUEfOCvsRrodg69jUYI= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-515-_XNzziErMgCA3_vBtB6csA-1; Thu, 22 Oct 2026 09:41:30 +0000 X-MC-Unique: _XNzziErMgCA3_vBtB6csA-1 X-Mimecast-MFC-AGG-ID: _XNzziErMgCA3_vBtB6csA_1792662089 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 734481955F13; Thu, 22 Oct 2026 09:41:28 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.redhat.corp (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id C635918001CA; Thu, 22 Oct 2026 09:41:26 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Bryan O'Donoghue , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= Subject: [PATCH v16 5/6] libcamera: software_isp: Pass LSC availability to debayering Date: Thu, 22 Oct 2026 11:40:47 +0200 Message-ID: <20261022094048.20277-6-mzamazal@redhat.com> In-Reply-To: <20261022094048.20277-1-mzamazal@redhat.com> References: <20261022094048.20277-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: pqLS9_Wwqy-fN227FU96wuBtn2ZZZpNj40AFfIl4XrY_1792662089 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The lens shading algorithm may or may not be enabled in the tuning file. Debayering configuration needs this information to arrange things accordingly, e.g. to set up shader parameters. Let's add a corresponding flag that is set in the LSC algorithm (if present) and passed to Debayer::configure. This is similar to what we already do with CCM availability. The flag is ignored by CPU ISP, where LSC is not going to be implemented for now. Signed-off-by: Milan Zamazal Reviewed-by: Bryan O'Donoghue Reviewed-by: Barnabás Pőcze --- include/libcamera/internal/software_isp/software_isp.h | 1 + include/libcamera/ipa/softisp.mojom | 4 +++- src/ipa/softisp/algorithms/lsc.cpp | 8 +++++++- src/ipa/softisp/ipa_context.h | 1 + src/ipa/softisp/softisp.cpp | 7 +++++-- src/libcamera/software_isp/debayer.cpp | 1 + src/libcamera/software_isp/debayer.h | 3 ++- src/libcamera/software_isp/debayer_cpu.cpp | 3 ++- src/libcamera/software_isp/debayer_cpu.h | 3 ++- src/libcamera/software_isp/debayer_egl.cpp | 5 ++++- src/libcamera/software_isp/debayer_egl.h | 4 +++- src/libcamera/software_isp/software_isp.cpp | 5 +++-- 12 files changed, 34 insertions(+), 11 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index b8b24bb1b..b64910637 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -101,6 +101,7 @@ private: DebayerParams debayerParams_; DmaBufAllocator dmaHeap_; bool ccmEnabled_; + bool lscEnabled_; std::unique_ptr ipa_; std::deque queuedInputBuffers_; diff --git a/include/libcamera/ipa/softisp.mojom b/include/libcamera/ipa/softisp.mojom index 209af1fde..e42742ed7 100644 --- a/include/libcamera/ipa/softisp.mojom +++ b/include/libcamera/ipa/softisp.mojom @@ -18,7 +18,9 @@ interface IPASoftIspInterface { libcamera.SharedFD fdParams, libcamera.IPACameraSensorInfo sensorInfo, libcamera.ControlInfoMap sensorControls) - => (int32 ret, libcamera.ControlInfoMap ipaControls, bool ccmEnabled); + => (int32 ret, + libcamera.ControlInfoMap ipaControls, + bool ccmEnabled, bool lscEnabled); start() => (int32 ret); stop(); configure(IPAConfigInfo configInfo) diff --git a/src/ipa/softisp/algorithms/lsc.cpp b/src/ipa/softisp/algorithms/lsc.cpp index 8316c7e9b..308939f19 100644 --- a/src/ipa/softisp/algorithms/lsc.cpp +++ b/src/ipa/softisp/algorithms/lsc.cpp @@ -20,11 +20,17 @@ int Lsc::init(IPAContext &context, const ValueNode &tuningData) for (unsigned int i = 0; i < kGridSize; i++) gridPos_.push_back(static_cast(i) / (kGridSize - 1)); - return lscAlgo_.init(tuningData, context.ctrlMap, + int ret = lscAlgo_.init(tuningData, context.ctrlMap, { .keys = { "r", "g", "b" }, .numHSamples = kGridSize, .numVSamples = kGridSize, .sensorSize = context.sensorInfo.activeArea.size() }); + if (ret) + return ret; + + context.lscEnabled = true; + + return 0; } int Lsc::configure(IPAContext &context, diff --git a/src/ipa/softisp/ipa_context.h b/src/ipa/softisp/ipa_context.h index 00326e6c7..1c5db17e3 100644 --- a/src/ipa/softisp/ipa_context.h +++ b/src/ipa/softisp/ipa_context.h @@ -90,6 +90,7 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; bool ccmEnabled = false; + bool lscEnabled = false; ipa::lsc::ActiveState lsc; }; diff --git a/src/ipa/softisp/softisp.cpp b/src/ipa/softisp/softisp.cpp index 420fee745..00249a300 100644 --- a/src/ipa/softisp/softisp.cpp +++ b/src/ipa/softisp/softisp.cpp @@ -58,7 +58,8 @@ public: const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls, - bool *ccmEnabled) override; + bool *ccmEnabled, + bool *lscEnabled) override; int configure(const IPAConfigInfo &configInfo, ControlInfoMap *ipaControls) override; @@ -97,7 +98,8 @@ int IPASoftIsp::init(const IPASettings &settings, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls, - bool *ccmEnabled) + bool *ccmEnabled, + bool *lscEnabled) { context_.camHelper = CameraSensorHelperFactoryBase::create(settings.sensorModel); if (!context_.camHelper) { @@ -137,6 +139,7 @@ int IPASoftIsp::init(const IPASettings &settings, return ret; *ccmEnabled = context_.ccmEnabled; + *lscEnabled = context_.lscEnabled; params_ = nullptr; stats_ = nullptr; diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index 6ebf7ed1a..8295b8fc2 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -107,6 +107,7 @@ Debayer::~Debayer() * \param[in] inputCfg The input configuration * \param[in] outputCfgs The output configurations * \param[in] ccmEnabled Whether a color correction matrix is applied + * \param[in] lscEnabled Whether lens shading correction grid is provided * * \return 0 on success, a negative errno on failure */ diff --git a/src/libcamera/software_isp/debayer.h b/src/libcamera/software_isp/debayer.h index 556852260..e30a05279 100644 --- a/src/libcamera/software_isp/debayer.h +++ b/src/libcamera/software_isp/debayer.h @@ -40,7 +40,8 @@ public: virtual int configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfgs, - bool ccmEnabled) = 0; + bool ccmEnabled, + bool lscEnabled) = 0; virtual std::vector formats(PixelFormat inputFormat) = 0; diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index ce8b3c647..91867baa6 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -624,7 +624,8 @@ int DebayerCpu::setDebayerFunctions(PixelFormat inputFormat, int DebayerCpu::configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfgs, - bool ccmEnabled) + bool ccmEnabled, + [[maybe_unused]] bool lscEnabled) { if (getInputConfig(inputCfg.pixelFormat, inputConfig_) != 0) { LOG(Debayer, Error) diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 2c88c9e1a..ae570893b 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -37,7 +37,8 @@ public: int configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfgs, - bool ccmEnabled) override; + bool ccmEnabled, + bool lscEnabled) override; Size patternSize(PixelFormat inputFormat) override; std::vector formats(PixelFormat input) override; std::tuple diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index 23dee13e7..bae63e076 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -273,7 +273,8 @@ int DebayerEGL::initBayerShaders(PixelFormat inputFormat, PixelFormat outputForm int DebayerEGL::configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfgs, - [[maybe_unused]] bool ccmEnabled) + [[maybe_unused]] bool ccmEnabled, + bool lscEnabled) { if (getInputConfig(inputCfg.pixelFormat, inputConfig_) != 0) { LOG(Debayer, Error) @@ -293,6 +294,8 @@ int DebayerEGL::configure(const StreamConfiguration &inputCfg, return -EINVAL; } + lscEnabled_ = lscEnabled; + inputConfig_.stride = inputCfg.stride; inputPixelFormat_ = inputCfg.pixelFormat; width_ = inputCfg.size.width; diff --git a/src/libcamera/software_isp/debayer_egl.h b/src/libcamera/software_isp/debayer_egl.h index 30e51a477..a64dc925a 100644 --- a/src/libcamera/software_isp/debayer_egl.h +++ b/src/libcamera/software_isp/debayer_egl.h @@ -45,7 +45,8 @@ public: int configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfgs, - bool ccmEnabled) override; + bool ccmEnabled, + bool lscEnabled) override; Size patternSize(PixelFormat inputFormat) override; @@ -82,6 +83,7 @@ private: unsigned int inputBufferCount_; unsigned int outputBufferCount_; + bool lscEnabled_; /* Shader parameters */ float firstRed_x_; float firstRed_y_; diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index a40976055..627625b4f 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -163,7 +163,8 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, sensorInfo, sensor->controls(), ipaControls, - &ccmEnabled_); + &ccmEnabled_, + &lscEnabled_); if (ret) { LOG(SoftwareIsp, Error) << "IPA init failed"; debayer_.reset(); @@ -293,7 +294,7 @@ int SoftwareIsp::configure(const StreamConfiguration &inputCfg, if (ret < 0) return ret; - ret = debayer_->configure(inputCfg, outputCfgs, ccmEnabled_); + ret = debayer_->configure(inputCfg, outputCfgs, ccmEnabled_, lscEnabled_); if (ret < 0) return ret; From patchwork Thu Oct 22 09:40:48 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28556 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8AEE3C3261 for ; Thu, 22 Oct 2026 09:41:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 556DA68F6F; Thu, 22 Oct 2026 11:41:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="CoXsRug8"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 04D9E68F68 for ; Thu, 22 Oct 2026 11:41:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1792662095; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=X5n3s1B4EcaAH0HhDTi4I/S2JM6kttTgvDkdk+uzfOI=; b=CoXsRug8jZv6AI4OwOECzlZUs4gUMTMSbZCKaofSNgs91Va1fIAYV1GDsmkFV0QKfLaQvm 5yUrx+XdAxNP8ymZHuoc8nTlQVqX7TuotcipUzUReyVFDWfJLP3Gwtevn8x95fHd8YJ8rf cbLYMhALX3rSTDcE0W4F5bJq9EtKA0E= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-519-qed2dPxHPQW8bVf3W-baEg-1; Thu, 22 Oct 2026 09:41:31 +0000 X-MC-Unique: qed2dPxHPQW8bVf3W-baEg-1 X-Mimecast-MFC-AGG-ID: qed2dPxHPQW8bVf3W-baEg_1792662090 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7A733195608C; Thu, 22 Oct 2026 09:41:30 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.redhat.corp (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D1AB81800642; Thu, 22 Oct 2026 09:41:28 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Xander Pronk , Kieran Bingham , Rick ten Wolde , Milan Zamazal , =?utf-8?b?QmFybmFiw6FzIFDFkWN6?= =?utf-8?q?e?= Subject: [PATCH v16 6/6] libcamera: software_isp: debayer_egl: Add LSC support Date: Thu, 22 Oct 2026 11:40:48 +0200 Message-ID: <20261022094048.20277-7-mzamazal@redhat.com> In-Reply-To: <20261022094048.20277-1-mzamazal@redhat.com> References: <20261022094048.20277-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 5a6b0VGP7kXyoFAo0WkVE0i6rjvLy0AWrl2SsRABu5o_1792662090 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Xander Pronk Add support for passing the LSC table from debayerParams to the shaders. We use 8-bit texture with the original UQ<2, 6> values to avoid trouble with floats. 32-bit floats don't work in my environment and 16-bit floats may not be supported in all OpenGL environments. When LSC is disabled by the runtime control, we provide a constant 1.0 correction table, i.e. a no-op table. This is preferred to having a runtime flag, to not disturb GPU computation with runtime switches, assuming that disabling LSC in runtime is not a common use case. The LSC texture is created once and then updated. The texture creation cannot be done in `configure', because of a different thread. Co-developed-by: Rick ten Wolde Signed-off-by: Rick ten Wolde Signed-off-by: Xander Pronk Signed-off-by: Milan Zamazal Tested-by: Barnabás Pőcze Reviewed-by: Kieran Bingham --- include/libcamera/internal/egl.h | 4 +-- .../internal/software_isp/debayer_params.h | 2 ++ src/libcamera/egl.cpp | 4 +-- src/libcamera/software_isp/debayer.cpp | 5 ++++ src/libcamera/software_isp/debayer_egl.cpp | 30 ++++++++++++++++++- src/libcamera/software_isp/debayer_egl.h | 7 +++++ 6 files changed, 47 insertions(+), 5 deletions(-) diff --git a/include/libcamera/internal/egl.h b/include/libcamera/internal/egl.h index dc4c6fdb6..d6fb6fe06 100644 --- a/include/libcamera/internal/egl.h +++ b/include/libcamera/internal/egl.h @@ -111,8 +111,8 @@ public: int createInputDMABufTexture2D(eGLImage &eglImage, int fd); int createOutputDMABufTexture2D(eGLImage &eglImage, int fd); - void createTexture2D(eGLImage &eglImage, void *data, GLint filter); - void updateTexture2D(eGLImage &eglImage, void *data); + void createTexture2D(eGLImage &eglImage, const void *data, GLint filter); + void updateTexture2D(eGLImage &eglImage, const void *data); void createOutputTexture2D(eGLImage &eglImage); int attachTextureToFBO(eGLImage &eglImage); diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index 2ae1614b3..300c3f2d6 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -34,6 +34,8 @@ struct DebayerParams { **/ static constexpr unsigned int kLscGridSize = 17; static constexpr unsigned int kLscValuesPerCell = 4; + static constexpr unsigned int kLscBytesPerCell = + kLscValuesPerCell * sizeof(uint8_t); using LscLookupTable = std::array; static constexpr auto identityLscLut = [] { diff --git a/src/libcamera/egl.cpp b/src/libcamera/egl.cpp index 4c89eea05..b3d3b9abb 100644 --- a/src/libcamera/egl.cpp +++ b/src/libcamera/egl.cpp @@ -286,7 +286,7 @@ int eGL::createOutputDMABufTexture2D(eGLImage &eglImage, int fd) * is useful for uploading static data like lookup tables or uniform color * matrices to the GPU. */ -void eGL::createTexture2D(eGLImage &eglImage, void *data, GLint filter) +void eGL::createTexture2D(eGLImage &eglImage, const void *data, GLint filter) { assertThread(); @@ -339,7 +339,7 @@ EGLDisplay eGL::probeDisplay() * * Updates a 2D texture in VRAM. */ -void eGL::updateTexture2D(eGLImage &eglImage, void *data) +void eGL::updateTexture2D(eGLImage &eglImage, const void *data) { assertThread(); diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index 8295b8fc2..4f6659000 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -53,6 +53,11 @@ namespace libcamera { * \brief Number of pixel values per each of the lens shading grid areas */ +/** + * \var DebayerParams::kLscBytesPerCell + * \brief Number of bytes per each of the lens shading grid areas + */ + /** * \typedef DebayerParams::LscLookupTable * \brief Lookup table for lens shading correction diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index bae63e076..33933b11d 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -115,6 +115,8 @@ int DebayerEGL::getShaderVariableLocations(void) textureUniformBayerFirstRed_ = glGetUniformLocation(programId_, "tex_bayer_first_red"); textureUniformProjMatrix_ = glGetUniformLocation(programId_, "proj_matrix"); + textureUniformLsc_ = glGetUniformLocation(programId_, "lsc_tex"); + LOG(Debayer, Debug) << "vertexIn " << attributeVertex_ << " textureIn " << attributeTexture_ << " tex_y " << textureUniformBayerDataIn_ << " awb " << awbUniformDataIn_ @@ -126,7 +128,8 @@ int DebayerEGL::getShaderVariableLocations(void) << " tex_size " << textureUniformSize_ << " stride_factor " << textureUniformStrideFactor_ << " tex_bayer_first_red " << textureUniformBayerFirstRed_ - << " proj_matrix " << textureUniformProjMatrix_; + << " proj_matrix " << textureUniformProjMatrix_ + << " tex_lsc " << textureUniformLsc_; return 0; } @@ -143,6 +146,9 @@ int DebayerEGL::initBayerShaders(PixelFormat inputFormat, PixelFormat outputForm /* Specify GL_OES_EGL_image_external */ egl_.pushEnv(shaderEnv, "#extension GL_OES_EGL_image_external: enable"); + if (lscEnabled_) + egl_.pushEnv(shaderEnv, "#define APPLY_LSC"); + /* * Tell shaders how to re-order output taking account of how the pixels * are actually stored by EGL. @@ -472,6 +478,13 @@ void DebayerEGL::setShaderVariableValues(eGLImage &eglImageIn, const DebayerPara glUniformMatrix3fv(ccmUniformDataIn_, 1, GL_FALSE, ccm.data().data()); LOG(Debayer, Debug) << " ccmUniformDataIn_ " << ccmUniformDataIn_ << " data " << params.combinedMatrix; + if (lscEnabled_ && params.lscLutVersion != lscLutVersion_) { + egl_.updateTexture2D(*eglImageLscLookup_, params.lscLut.data()); + lscLutVersion_ = params.lscLutVersion; + glUniform1i(textureUniformLsc_, eglImageLscLookup_->texture_unit_uniform_id_); + LOG(Debayer, Debug) << "LSC texture updated to version " << lscLutVersion_.value_or(-1); + } + /* * 0 = Red, 1 = Green, 2 = Blue */ @@ -678,6 +691,18 @@ int DebayerEGL::start() if (initBayerShaders(inputPixelFormat_, outputPixelFormat_)) return -EINVAL; + if (lscEnabled_) { + constexpr unsigned int gridSize = DebayerParams::kLscGridSize; + constexpr unsigned int stride = gridSize * DebayerParams::kLscBytesPerCell; + eglImageLscLookup_.emplace(GL_RGBA, + gridSize, + gridSize, + stride, + GL_TEXTURE2, + 2); + egl_.createTexture2D(*eglImageLscLookup_, nullptr, GL_LINEAR); + } + return 0; } @@ -686,6 +711,9 @@ void DebayerEGL::stop() eglImageOutCache_.clear(); eglImageInCache_.clear(); + eglImageLscLookup_.reset(); + lscLutVersion_.reset(); + if (programId_) glDeleteProgram(programId_); diff --git a/src/libcamera/software_isp/debayer_egl.h b/src/libcamera/software_isp/debayer_egl.h index a64dc925a..cbbc6a450 100644 --- a/src/libcamera/software_isp/debayer_egl.h +++ b/src/libcamera/software_isp/debayer_egl.h @@ -83,7 +83,11 @@ private: unsigned int inputBufferCount_; unsigned int outputBufferCount_; + /* LSC lookup table */ bool lscEnabled_; + std::optional eglImageLscLookup_; + std::optional lscLutVersion_; + /* Shader parameters */ float firstRed_x_; float firstRed_y_; @@ -100,6 +104,9 @@ private: /* Per-frame AWB gains */ GLint awbUniformDataIn_; + /* Lens shading correction */ + GLint textureUniformLsc_; + /* Represent per-frame CCM as a uniform vector of floats 3 x 3 */ GLint ccmUniformDataIn_;