From patchwork Wed Feb 5 21:27:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22729 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 78B97C32F2 for ; Wed, 5 Feb 2025 21:27:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4694B685D4; Wed, 5 Feb 2025 22:27:43 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="WmTntr2L"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 56EE8685C7 for ; Wed, 5 Feb 2025 22:27:41 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738790860; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e39xbB09xFZ79lqfzDk33yX3B8VodGckeLJ1adgFWeI=; b=WmTntr2LwFsNPPOEeyhikpEYaKgm06DPW7ZatN69PP4D8p+rpr3yUDqJdCnUh7HG9JAFDC OczF3W3F1VVqi3Xyv13oGosAugsxR1xAa+y4mdgJW3FkNUjEHUL5YRwWAOZ/4BZIFvY5rn Y+JWnohmBWmHjcOIR1Dus18JYE/STzc= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-660-lnIhztc3PhaDoJ-WQnL4Bw-1; Wed, 05 Feb 2025 16:27:37 -0500 X-MC-Unique: lnIhztc3PhaDoJ-WQnL4Bw-1 X-Mimecast-MFC-AGG-ID: lnIhztc3PhaDoJ-WQnL4Bw Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B6EAC1800875; Wed, 5 Feb 2025 21:27:35 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.28]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2F1421800871; Wed, 5 Feb 2025 21:27:32 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Robert Mader , Hans de Goede , Laurent Pinchart , Kieran Bingham , Jacopo Mondi Subject: [PATCH v6 1/1] libcamera: pipeline: virtual: Set `FrameError` on error Date: Wed, 5 Feb 2025 22:27:13 +0100 Message-ID: <20250205212725.138055-2-mzamazal@redhat.com> In-Reply-To: <20250205212725.138055-1-mzamazal@redhat.com> References: <20250205212725.138055-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Vxhtqnzsw0Dk7edG9OQghv4wkQUa9MNX0s7gsLxYNZk_1738790855 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Barnabás Pőcze Do not cancel, simply set the buffer's status to `FrameError` to notify the user about the error condition. Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/libcamera/pipeline/virtual/virtual.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index 82894270..049ebcba 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -322,7 +322,7 @@ int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera, if (streamConfig.frameGenerator->generateFrame( stream->configuration().size, buffer)) - buffer->_d()->cancel(); + fmd.status = FrameMetadata::Status::FrameError; completeBuffer(request, buffer); break; From patchwork Wed Feb 5 21:27:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22731 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BE869C32F4 for ; Wed, 5 Feb 2025 21:27:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 473A4685E1; Wed, 5 Feb 2025 22:27:50 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="X5fk1NIy"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 456A1685D7 for ; Wed, 5 Feb 2025 22:27:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738790867; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FSgPOHOIEXUaZq+OrL5DpDNYD0XSQoynmU/AG+qRxa4=; b=X5fk1NIyN+tguqZhQdNccbNK1tl8eGF9r3uZucvKb21BndxfMDvxJUz4acSnGUoIIWmEZr k/c7mG8PoukVYN5/yx6ZUDZ8xq52Wy5+00vzEXvJhIloj7Xs1w6bjKCD47JtZbvjPyQ4rp T1Pm/T3OtCsETs7HNPnnMvh/8w1N7SE= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-164-f5acmzp-MbmLwV4nt-cK4Q-1; Wed, 05 Feb 2025 16:27:45 -0500 X-MC-Unique: f5acmzp-MbmLwV4nt-cK4Q-1 X-Mimecast-MFC-AGG-ID: f5acmzp-MbmLwV4nt-cK4Q Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1F75A1800874; Wed, 5 Feb 2025 21:27:41 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.28]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0DDBD1800265; Wed, 5 Feb 2025 21:27:38 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Robert Mader , Hans de Goede , Laurent Pinchart , Kieran Bingham Subject: [PATCH v6 02/10] libcamera: software_isp: Use RGB type to represent gains Date: Wed, 5 Feb 2025 22:27:15 +0100 Message-ID: <20250205212725.138055-4-mzamazal@redhat.com> In-Reply-To: <20250205212725.138055-1-mzamazal@redhat.com> References: <20250205212725.138055-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: iD1vGQFkNVWoNPFS7QfgP7VF8nFMs_PXONYhCw3DOh4_1738790864 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rather than using a custom struct to represent RGB values, let's use the corresponding type and its facilities. Signed-off-by: Milan Zamazal Reviewed-by: Laurent Pinchart --- src/ipa/simple/algorithms/awb.cpp | 16 +++++++++------- src/ipa/simple/algorithms/lut.cpp | 14 ++++---------- src/ipa/simple/ipa_context.h | 7 ++----- 3 files changed, 15 insertions(+), 22 deletions(-) diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp index 1efc7090..310be6a7 100644 --- a/src/ipa/simple/algorithms/awb.cpp +++ b/src/ipa/simple/algorithms/awb.cpp @@ -25,7 +25,7 @@ int Awb::configure(IPAContext &context, [[maybe_unused]] const IPAConfigInfo &configInfo) { auto &gains = context.activeState.awb.gains; - gains.red = gains.green = gains.blue = 1.0; + gains = { { 1.0, 1.0, 1.0 } }; return 0; } @@ -56,16 +56,18 @@ void Awb::process(IPAContext &context, * Clamp max gain at 4.0, this also avoids 0 division. */ auto &gains = context.activeState.awb.gains; - gains.red = sumR <= sumG / 4 ? 4.0 : static_cast(sumG) / sumR; - gains.blue = sumB <= sumG / 4 ? 4.0 : static_cast(sumG) / sumB; - /* Green gain is fixed to 1.0 */ + gains = { { + sumR <= sumG / 4 ? 4.0 : static_cast(sumG) / sumR, + 1.0, + sumB <= sumG / 4 ? 4.0 : static_cast(sumG) / sumB, + } }; - RGB rgbGains{ { 1 / gains.red, 1 / gains.green, 1 / gains.blue } }; + RGB rgbGains{ { 1 / gains.r(), 1 / gains.g(), 1 / gains.b() } }; context.activeState.awb.temperatureK = estimateCCT(rgbGains); LOG(IPASoftAwb, Debug) - << "gain R/B: " << gains.red << "/" << gains.blue - << "; temperature: " << context.activeState.awb.temperatureK; + << "gain R/B: " << gains << "; temperature: " + << context.activeState.awb.temperatureK; } REGISTER_IPA_ALGORITHM(Awb, "Awb") diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp index d75ff710..1d392737 100644 --- a/src/ipa/simple/algorithms/lut.cpp +++ b/src/ipa/simple/algorithms/lut.cpp @@ -103,16 +103,10 @@ void Lut::prepare(IPAContext &context, const double div = static_cast(DebayerParams::kRGBLookupSize) / gammaTableSize; /* Apply gamma after gain! */ - unsigned int idx; - idx = std::min({ static_cast(i * gains.red / div), - gammaTableSize - 1 }); - params->red[i] = gammaTable[idx]; - idx = std::min({ static_cast(i * gains.green / div), - gammaTableSize - 1 }); - params->green[i] = gammaTable[idx]; - idx = std::min({ static_cast(i * gains.blue / div), - gammaTableSize - 1 }); - params->blue[i] = gammaTable[idx]; + const RGB lutGains = (gains * i / div).min(gammaTableSize - 1); + params->red[i] = gammaTable[static_cast(lutGains.r())]; + params->green[i] = gammaTable[static_cast(lutGains.g())]; + params->blue[i] = gammaTable[static_cast(lutGains.b())]; } } diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 607af45a..df0552db 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -14,6 +14,7 @@ #include #include +#include namespace libcamera { @@ -36,11 +37,7 @@ struct IPAActiveState { } blc; struct { - struct { - double red; - double green; - double blue; - } gains; + RGB gains; unsigned int temperatureK; } awb; From patchwork Wed Feb 5 21:27:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22732 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2FE17C32F2 for ; Wed, 5 Feb 2025 21:27:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DC5EC685E5; Wed, 5 Feb 2025 22:27:52 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="G1sAH5Kx"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5C37F685D8 for ; Wed, 5 Feb 2025 22:27:50 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738790869; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7WasbHSFXCV5sy4Y5cLOqvWD35Ya4BxdbgR96cMVhi4=; b=G1sAH5Kx5Spl41fi8H2IxGzeHmVSLTY9o+tVqVoXY9QMkMO0vT5XbHUcETj/fBuSAXsB7S /F/OmtTAUhn6sT3UlK1r4hTPLSz8yKzCw2037R51vtEFZRaMf5Kj65scoXjhiJipvm7NXH 9Hgs+CdcUmiv5uuhhSmdxB8vLvRMzgA= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-299-gYDTPAYEMLymSX0dGCW58Q-1; Wed, 05 Feb 2025 16:27:45 -0500 X-MC-Unique: gYDTPAYEMLymSX0dGCW58Q-1 X-Mimecast-MFC-AGG-ID: gYDTPAYEMLymSX0dGCW58Q Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 12AD91956096; Wed, 5 Feb 2025 21:27:44 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.28]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B19CF1800265; Wed, 5 Feb 2025 21:27:41 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Robert Mader , Hans de Goede , Laurent Pinchart , Kieran Bingham Subject: [PATCH v6 03/10] libcamera: software_isp: Store color temperature to metadata Date: Wed, 5 Feb 2025 22:27:16 +0100 Message-ID: <20250205212725.138055-5-mzamazal@redhat.com> In-Reply-To: <20250205212725.138055-1-mzamazal@redhat.com> References: <20250205212725.138055-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: CE5CjHfjrg8KGQHrwUwXsha4PdYrTX14ltlWRwCHlWg_1738790864 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Image color temperature is a piece of information that should be reported in metadata, let's put it there. Metadata is currently not reported in simple pipeline but we should make at least newly added information ready to be reported. Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/simple/algorithms/awb.cpp | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp index 310be6a7..4e31e386 100644 --- a/src/ipa/simple/algorithms/awb.cpp +++ b/src/ipa/simple/algorithms/awb.cpp @@ -12,6 +12,8 @@ #include +#include + #include "libipa/colours.h" #include "simple/ipa_context.h" @@ -34,7 +36,7 @@ void Awb::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameContext, const SwIspStats *stats, - [[maybe_unused]] ControlList &metadata) + ControlList &metadata) { const SwIspStats::Histogram &histogram = stats->yHistogram; const uint8_t blackLevel = context.activeState.blc.level; @@ -64,6 +66,7 @@ void Awb::process(IPAContext &context, RGB rgbGains{ { 1 / gains.r(), 1 / gains.g(), 1 / gains.b() } }; context.activeState.awb.temperatureK = estimateCCT(rgbGains); + metadata.set(controls::ColourTemperature, context.activeState.awb.temperatureK); LOG(IPASoftAwb, Debug) << "gain R/B: " << gains << "; temperature: " From patchwork Wed Feb 5 21:27:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22733 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CDD99C32F4 for ; Wed, 5 Feb 2025 21:27:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 88807685DD; Wed, 5 Feb 2025 22:27:53 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="cec3L5Hd"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4C979685D8 for ; Wed, 5 Feb 2025 22:27:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738790871; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o96CmCq3PVlOnAb3Qa+uYCHAcz5mRCDlEMGfw+67XTE=; b=cec3L5Hdto8uOiMgXTClv5phNVxXtYF/W9uSFuZc31hJJQpwuPuPl5NyJZKDkEIgY1XBqf yLvuCiKQZmqOSp4T07GGaQIJVjr4VhwL+pWEnFVzLrlD+EjqnHrtXnQ+GSqddLy81eDusD WTyPH5SsZyfglRJEnFjj5yfFWGsdYR8= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-49-zdRIxT3RNGCbwbh-0XTGcQ-1; Wed, 05 Feb 2025 16:27:48 -0500 X-MC-Unique: zdRIxT3RNGCbwbh-0XTGcQ-1 X-Mimecast-MFC-AGG-ID: zdRIxT3RNGCbwbh-0XTGcQ Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0E7051801A15; Wed, 5 Feb 2025 21:27:47 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.28]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A94DE1800265; Wed, 5 Feb 2025 21:27:44 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Robert Mader , Hans de Goede , Laurent Pinchart , Kieran Bingham Subject: [PATCH v6 04/10] libcamera: software_isp: lut: Remove maybe_unused on a used argument Date: Wed, 5 Feb 2025 22:27:17 +0100 Message-ID: <20250205212725.138055-6-mzamazal@redhat.com> In-Reply-To: <20250205212725.138055-1-mzamazal@redhat.com> References: <20250205212725.138055-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: SkcVSA0pZ2dsAeOSD_kDAtMffUrioz7gQyCa87Vhkm0_1738790867 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" `params' argument of Lut::prepare is actually used, let's remove maybe_unused from it. Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/simple/algorithms/lut.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp index 1d392737..60dd0624 100644 --- a/src/ipa/simple/algorithms/lut.cpp +++ b/src/ipa/simple/algorithms/lut.cpp @@ -83,7 +83,7 @@ void Lut::updateGammaTable(IPAContext &context) void Lut::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameContext, - [[maybe_unused]] DebayerParams *params) + DebayerParams *params) { /* * Update the gamma table if needed. This means if black level changes From patchwork Wed Feb 5 21:27:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22734 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6EF8CC32F2 for ; Wed, 5 Feb 2025 21:27:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 280CD685E5; Wed, 5 Feb 2025 22:27:57 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="gAk+Sori"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 42440685D8 for ; Wed, 5 Feb 2025 22:27:55 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738790874; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h3TLxkLfx75gkBcwBzzl3bglAMArwb7yA9H1laXF9pA=; b=gAk+SoriyHJu5trlB0uGAn/0w4KbqDtQ/laTKKblwg5od1XjukTsDndU2ehL63IjXKVosf cr5girSYmHr06CvxjeRr8zaEbpebx0A1EHsYLuybwtKQaiNj3/8NSEEjsG4TB7V777x2np sLUBFq75EdKIUQNAEdQax71zhXj6KNs= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-587-Mgbcr1-ZM3-6n_QOyh3nZA-1; Wed, 05 Feb 2025 16:27:51 -0500 X-MC-Unique: Mgbcr1-ZM3-6n_QOyh3nZA-1 X-Mimecast-MFC-AGG-ID: Mgbcr1-ZM3-6n_QOyh3nZA Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 029EF1956046; Wed, 5 Feb 2025 21:27:50 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.28]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9A1D91800265; Wed, 5 Feb 2025 21:27:47 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Robert Mader , Hans de Goede , Laurent Pinchart , Kieran Bingham Subject: [PATCH v6 05/10] libcamera: software_isp: Use common code to store debayered pixels Date: Wed, 5 Feb 2025 22:27:18 +0100 Message-ID: <20250205212725.138055-7-mzamazal@redhat.com> In-Reply-To: <20250205212725.138055-1-mzamazal@redhat.com> References: <20250205212725.138055-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: -HOoE_G1t-AWZWP4AxkPAGK6QwiBQz_n2jGm7CJaTVE_1738790870 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The debayering macros use the same pattern, let's extract it to a common macro. This reduces code duplication a bit now and it'll make changes of debayering easier when color correction matrix is introduced. Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/libcamera/software_isp/debayer_cpu.cpp | 56 +++++++++++----------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 31ab96ab..0eabced2 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -62,57 +62,57 @@ DebayerCpu::~DebayerCpu() = default; const pixel_t *curr = (const pixel_t *)src[1] + xShift_; \ const pixel_t *next = (const pixel_t *)src[2] + xShift_; +#define STORE_PIXEL(b, g, r) \ + *dst++ = blue_[b]; \ + *dst++ = green_[g]; \ + *dst++ = red_[r]; \ + if constexpr (addAlphaByte) \ + *dst++ = 255; \ + x++; + /* * RGR * GBG * RGR */ -#define BGGR_BGR888(p, n, div) \ - *dst++ = blue_[curr[x] / (div)]; \ - *dst++ = green_[(prev[x] + curr[x - p] + curr[x + n] + next[x]) / (4 * (div))]; \ - *dst++ = red_[(prev[x - p] + prev[x + n] + next[x - p] + next[x + n]) / (4 * (div))]; \ - if constexpr (addAlphaByte) \ - *dst++ = 255; \ - x++; +#define BGGR_BGR888(p, n, div) \ + STORE_PIXEL( \ + curr[x] / (div), \ + (prev[x] + curr[x - p] + curr[x + n] + next[x]) / (4 * (div)), \ + (prev[x - p] + prev[x + n] + next[x - p] + next[x + n]) / (4 * (div))) /* * GBG * RGR * GBG */ -#define GRBG_BGR888(p, n, div) \ - *dst++ = blue_[(prev[x] + next[x]) / (2 * (div))]; \ - *dst++ = green_[curr[x] / (div)]; \ - *dst++ = red_[(curr[x - p] + curr[x + n]) / (2 * (div))]; \ - if constexpr (addAlphaByte) \ - *dst++ = 255; \ - x++; +#define GRBG_BGR888(p, n, div) \ + STORE_PIXEL( \ + (prev[x] + next[x]) / (2 * (div)), \ + curr[x] / (div), \ + (curr[x - p] + curr[x + n]) / (2 * (div))) /* * GRG * BGB * GRG */ -#define GBRG_BGR888(p, n, div) \ - *dst++ = blue_[(curr[x - p] + curr[x + n]) / (2 * (div))]; \ - *dst++ = green_[curr[x] / (div)]; \ - *dst++ = red_[(prev[x] + next[x]) / (2 * (div))]; \ - if constexpr (addAlphaByte) \ - *dst++ = 255; \ - x++; +#define GBRG_BGR888(p, n, div) \ + STORE_PIXEL( \ + (curr[x - p] + curr[x + n]) / (2 * (div)), \ + curr[x] / (div), \ + (prev[x] + next[x]) / (2 * (div))) /* * BGB * GRG * BGB */ -#define RGGB_BGR888(p, n, div) \ - *dst++ = blue_[(prev[x - p] + prev[x + n] + next[x - p] + next[x + n]) / (4 * (div))]; \ - *dst++ = green_[(prev[x] + curr[x - p] + curr[x + n] + next[x]) / (4 * (div))]; \ - *dst++ = red_[curr[x] / (div)]; \ - if constexpr (addAlphaByte) \ - *dst++ = 255; \ - x++; +#define RGGB_BGR888(p, n, div) \ + STORE_PIXEL( \ + (prev[x - p] + prev[x + n] + next[x - p] + next[x + n]) / (4 * (div)), \ + (prev[x] + curr[x - p] + curr[x + n] + next[x]) / (4 * (div)), \ + curr[x] / (div)) template void DebayerCpu::debayer8_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]) From patchwork Wed Feb 5 21:27:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22735 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 40358C32F2 for ; Wed, 5 Feb 2025 21:28:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E41DC685ED; Wed, 5 Feb 2025 22:27:59 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="MWWkABGq"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F114C685DD for ; Wed, 5 Feb 2025 22:27:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738790877; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9n7bJsn0/PE4cketqm77uO5Dv7ouRSYpHuiOJOzrbEE=; b=MWWkABGqqKU2mBk4Bjm5TRItHo4PmOzpBcwp68voCPQXHJ2p6Vuw+n67ByAHy3RwfthGnB BNA13uWlboQb7X1wpFQqG/6lrm/8b/XkfEYU/dOS/pPuZbsLYMJtCheVP4Mo6ePOAiTko9 Dwz2My5k24YYcKNmZl0D9xyKEMTPZsw= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-451-zhercmJdNSuqqptw9O_YbQ-1; Wed, 05 Feb 2025 16:27:53 -0500 X-MC-Unique: zhercmJdNSuqqptw9O_YbQ-1 X-Mimecast-MFC-AGG-ID: zhercmJdNSuqqptw9O_YbQ Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BE8B1195608B; Wed, 5 Feb 2025 21:27:52 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.28]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9341B1800360; Wed, 5 Feb 2025 21:27:50 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Robert Mader , Hans de Goede , Laurent Pinchart , Kieran Bingham Subject: [PATCH v6 06/10] libcamera: software_isp: Use a macro to assign debayering methods Date: Wed, 5 Feb 2025 22:27:19 +0100 Message-ID: <20250205212725.138055-8-mzamazal@redhat.com> In-Reply-To: <20250205212725.138055-1-mzamazal@redhat.com> References: <20250205212725.138055-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Zd66CGf0OOcnQvWRkwuDJyc9hk43Tkh9fEPDDZ62gCU_1738790872 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Assignments of the debayering methods to be used is a repetitive pattern that can be (arguably) better expressed by using a macro. This removes some duplication and also makes easier to introduce more complex assignment patterns. This will be useful once color correction matrix support is added. Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/libcamera/software_isp/debayer_cpu.cpp | 25 ++++++++++------------ 1 file changed, 11 insertions(+), 14 deletions(-) diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 0eabced2..01cfb36b 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -368,6 +368,10 @@ int DebayerCpu::setupStandardBayerOrder(BayerFormat::Order order) return 0; } +#define SET_DEBAYER_METHODS(method0, method1) \ + debayer0_ = addAlphaByte ? &DebayerCpu::method0 : &DebayerCpu::method0; \ + debayer1_ = addAlphaByte ? &DebayerCpu::method1 : &DebayerCpu::method1; + int DebayerCpu::setDebayerFunctions(PixelFormat inputFormat, PixelFormat outputFormat) { BayerFormat bayerFormat = @@ -423,16 +427,13 @@ int DebayerCpu::setDebayerFunctions(PixelFormat inputFormat, PixelFormat outputF isStandardBayerOrder(bayerFormat.order)) { switch (bayerFormat.bitDepth) { case 8: - debayer0_ = addAlphaByte ? &DebayerCpu::debayer8_BGBG_BGR888 : &DebayerCpu::debayer8_BGBG_BGR888; - debayer1_ = addAlphaByte ? &DebayerCpu::debayer8_GRGR_BGR888 : &DebayerCpu::debayer8_GRGR_BGR888; + SET_DEBAYER_METHODS(debayer8_BGBG_BGR888, debayer8_GRGR_BGR888) break; case 10: - debayer0_ = addAlphaByte ? &DebayerCpu::debayer10_BGBG_BGR888 : &DebayerCpu::debayer10_BGBG_BGR888; - debayer1_ = addAlphaByte ? &DebayerCpu::debayer10_GRGR_BGR888 : &DebayerCpu::debayer10_GRGR_BGR888; + SET_DEBAYER_METHODS(debayer10_BGBG_BGR888, debayer10_GRGR_BGR888) break; case 12: - debayer0_ = addAlphaByte ? &DebayerCpu::debayer12_BGBG_BGR888 : &DebayerCpu::debayer12_BGBG_BGR888; - debayer1_ = addAlphaByte ? &DebayerCpu::debayer12_GRGR_BGR888 : &DebayerCpu::debayer12_GRGR_BGR888; + SET_DEBAYER_METHODS(debayer12_BGBG_BGR888, debayer12_GRGR_BGR888) break; } setupStandardBayerOrder(bayerFormat.order); @@ -443,20 +444,16 @@ int DebayerCpu::setDebayerFunctions(PixelFormat inputFormat, PixelFormat outputF bayerFormat.packing == BayerFormat::Packing::CSI2) { switch (bayerFormat.order) { case BayerFormat::BGGR: - debayer0_ = addAlphaByte ? &DebayerCpu::debayer10P_BGBG_BGR888 : &DebayerCpu::debayer10P_BGBG_BGR888; - debayer1_ = addAlphaByte ? &DebayerCpu::debayer10P_GRGR_BGR888 : &DebayerCpu::debayer10P_GRGR_BGR888; + SET_DEBAYER_METHODS(debayer10P_BGBG_BGR888, debayer10P_GRGR_BGR888) return 0; case BayerFormat::GBRG: - debayer0_ = addAlphaByte ? &DebayerCpu::debayer10P_GBGB_BGR888 : &DebayerCpu::debayer10P_GBGB_BGR888; - debayer1_ = addAlphaByte ? &DebayerCpu::debayer10P_RGRG_BGR888 : &DebayerCpu::debayer10P_RGRG_BGR888; + SET_DEBAYER_METHODS(debayer10P_GBGB_BGR888, debayer10P_RGRG_BGR888) return 0; case BayerFormat::GRBG: - debayer0_ = addAlphaByte ? &DebayerCpu::debayer10P_GRGR_BGR888 : &DebayerCpu::debayer10P_GRGR_BGR888; - debayer1_ = addAlphaByte ? &DebayerCpu::debayer10P_BGBG_BGR888 : &DebayerCpu::debayer10P_BGBG_BGR888; + SET_DEBAYER_METHODS(debayer10P_GRGR_BGR888, debayer10P_BGBG_BGR888) return 0; case BayerFormat::RGGB: - debayer0_ = addAlphaByte ? &DebayerCpu::debayer10P_RGRG_BGR888 : &DebayerCpu::debayer10P_RGRG_BGR888; - debayer1_ = addAlphaByte ? &DebayerCpu::debayer10P_GBGB_BGR888 : &DebayerCpu::debayer10P_GBGB_BGR888; + SET_DEBAYER_METHODS(debayer10P_RGRG_BGR888, debayer10P_GBGB_BGR888) return 0; default: break; From patchwork Wed Feb 5 21:27:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22736 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CEF22C32F4 for ; Wed, 5 Feb 2025 21:28:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7E325685E5; Wed, 5 Feb 2025 22:28:00 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="FFQDFFPg"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C77F5685DD for ; Wed, 5 Feb 2025 22:27:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738790877; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gu9W2L/K/kPTCkTbmOYYJlWjwmYzDXzV5tME2ZuVYrw=; b=FFQDFFPgstetESL+sVmsDtdqwenX1D79IE3ARpHQI/uVYqQNcdnsetMf2+05jd8o2E3dO1 bpfHeP+R5S3y7OaVj4eRkult8kUoQLEtjHhbPmJKKcG5grJjaIOrreA008lfwWe60cxvTL UPgEbAD626nc79SHJFQolRPKu3qY0MQ= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-617-YhmlzwdCPYy7-weSSpMlfA-1; Wed, 05 Feb 2025 16:27:56 -0500 X-MC-Unique: YhmlzwdCPYy7-weSSpMlfA-1 X-Mimecast-MFC-AGG-ID: YhmlzwdCPYy7-weSSpMlfA Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6AFD91800878; Wed, 5 Feb 2025 21:27:55 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.28]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 412221800872; Wed, 5 Feb 2025 21:27:53 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Robert Mader , Hans de Goede , Laurent Pinchart , Kieran Bingham Subject: [PATCH v6 07/10] libcamera: software_isp: Add CCM algorithm Date: Wed, 5 Feb 2025 22:27:20 +0100 Message-ID: <20250205212725.138055-9-mzamazal@redhat.com> In-Reply-To: <20250205212725.138055-1-mzamazal@redhat.com> References: <20250205212725.138055-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Q7sGjOiKb-3E-TiN5yuwCUXijBU0HuDf7OGLuQsaGZY_1738790875 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch adds color correction matrix (CCM) algorithm to software ISP. It is based on the corresponding algorithm in rkisp1. The primary difference against hardware pipelines is that applying the CCM is optional. Applying CCM causes a significant slowdown, time needed to process a frame raises by 40-90% on tested platforms. If CCM is really needed, it can be applied, if not, it's better to stick without it. This can be configured by presence or omission of Ccm algorithm in the tuning file. CCM is changed only if the determined temperature changes by at least 100 K (an arbitrarily selected value), to avoid recomputing the matrices and lookup tables all the time. Since the CCM is float, rather than double, to use the same type as in the rkisp1 pipeline, the type of color gains is changed from double to float. The outputs of the algorithm are not used yet, they will be enabled in followup patches. Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/simple/algorithms/awb.cpp | 4 +- src/ipa/simple/algorithms/ccm.cpp | 74 +++++++++++++++++++++++++++ src/ipa/simple/algorithms/ccm.h | 43 ++++++++++++++++ src/ipa/simple/algorithms/lut.cpp | 2 +- src/ipa/simple/algorithms/meson.build | 1 + src/ipa/simple/ipa_context.h | 14 ++++- 6 files changed, 134 insertions(+), 4 deletions(-) create mode 100644 src/ipa/simple/algorithms/ccm.cpp create mode 100644 src/ipa/simple/algorithms/ccm.h diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp index 4e31e386..ec77c6e5 100644 --- a/src/ipa/simple/algorithms/awb.cpp +++ b/src/ipa/simple/algorithms/awb.cpp @@ -59,9 +59,9 @@ void Awb::process(IPAContext &context, */ auto &gains = context.activeState.awb.gains; gains = { { - sumR <= sumG / 4 ? 4.0 : static_cast(sumG) / sumR, + sumR <= sumG / 4 ? 4.0f : static_cast(sumG) / sumR, 1.0, - sumB <= sumG / 4 ? 4.0 : static_cast(sumG) / sumB, + sumB <= sumG / 4 ? 4.0f : static_cast(sumG) / sumB, } }; RGB rgbGains{ { 1 / gains.r(), 1 / gains.g(), 1 / gains.b() } }; diff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp new file mode 100644 index 00000000..86e0395c --- /dev/null +++ b/src/ipa/simple/algorithms/ccm.cpp @@ -0,0 +1,74 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas On Board + * Copyright (C) 2024-2025, Red Hat Inc. + * + * Color correction matrix + */ + +#include "ccm.h" + +#include +#include + +#include + +namespace { + +constexpr unsigned int kTemperatureThreshold = 100; + +} + +namespace libcamera { + +namespace ipa::soft::algorithms { + +LOG_DEFINE_CATEGORY(IPASoftCcm) + +int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) +{ + int ret = ccm_.readYaml(tuningData["ccms"], "ct", "ccm"); + if (ret < 0) { + LOG(IPASoftCcm, Error) + << "Failed to parse 'ccm' parameter from tuning file."; + return ret; + } + + return 0; +} + +void Ccm::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, [[maybe_unused]] DebayerParams *params) +{ + const unsigned int ct = context.activeState.awb.temperatureK; + + /* Change CCM only on bigger temperature changes. */ + if (frame > 0 && + utils::abs_diff(ct, lastCt_) < kTemperatureThreshold) { + frameContext.ccm.ccm = context.activeState.ccm.ccm; + context.activeState.ccm.changed = false; + return; + } + + lastCt_ = ct; + Matrix ccm = ccm_.getInterpolated(ct); + + context.activeState.ccm.ccm = ccm; + frameContext.ccm.ccm = ccm; + context.activeState.ccm.changed = true; +} + +void Ccm::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const SwIspStats *stats, + ControlList &metadata) +{ + metadata.set(controls::ColourCorrectionMatrix, frameContext.ccm.ccm.data()); +} + +REGISTER_IPA_ALGORITHM(Ccm, "Ccm") + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/ccm.h b/src/ipa/simple/algorithms/ccm.h new file mode 100644 index 00000000..f4e2b85b --- /dev/null +++ b/src/ipa/simple/algorithms/ccm.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024-2025, Red Hat Inc. + * + * Color correction matrix + */ + +#pragma once + +#include "libcamera/internal/matrix.h" + +#include + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::soft::algorithms { + +class Ccm : public Algorithm +{ +public: + Ccm() = default; + ~Ccm() = default; + + int init(IPAContext &context, const YamlObject &tuningData) override; + void prepare(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + DebayerParams *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const SwIspStats *stats, + ControlList &metadata) override; + +private: + unsigned int lastCt_; + Interpolator> ccm_; +}; + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp index 60dd0624..352bbf57 100644 --- a/src/ipa/simple/algorithms/lut.cpp +++ b/src/ipa/simple/algorithms/lut.cpp @@ -103,7 +103,7 @@ void Lut::prepare(IPAContext &context, const double div = static_cast(DebayerParams::kRGBLookupSize) / gammaTableSize; /* Apply gamma after gain! */ - const RGB lutGains = (gains * i / div).min(gammaTableSize - 1); + const RGB lutGains = (gains * i / div).min(gammaTableSize - 1); params->red[i] = gammaTable[static_cast(lutGains.r())]; params->green[i] = gammaTable[static_cast(lutGains.g())]; params->blue[i] = gammaTable[static_cast(lutGains.b())]; diff --git a/src/ipa/simple/algorithms/meson.build b/src/ipa/simple/algorithms/meson.build index 37a2eb53..2d0adb05 100644 --- a/src/ipa/simple/algorithms/meson.build +++ b/src/ipa/simple/algorithms/meson.build @@ -4,5 +4,6 @@ soft_simple_ipa_algorithms = files([ 'awb.cpp', 'agc.cpp', 'blc.cpp', + 'ccm.cpp', 'lut.cpp', ]) diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index df0552db..57883218 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -13,6 +13,8 @@ #include +#include "libcamera/internal/matrix.h" + #include #include @@ -37,7 +39,7 @@ struct IPAActiveState { } blc; struct { - RGB gains; + RGB gains; unsigned int temperatureK; } awb; @@ -47,6 +49,12 @@ struct IPAActiveState { uint8_t blackLevel; double contrast; } gamma; + + struct { + Matrix ccm; + bool changed; + } ccm; + struct { /* 0..2 range, 1.0 = normal */ std::optional contrast; @@ -54,6 +62,10 @@ struct IPAActiveState { }; struct IPAFrameContext : public FrameContext { + struct { + Matrix ccm; + } ccm; + struct { int32_t exposure; double gain; From patchwork Wed Feb 5 21:27:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22737 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7DB72C32F2 for ; Wed, 5 Feb 2025 21:28:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 369D8685EC; Wed, 5 Feb 2025 22:28:06 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Pg0QYCQ2"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C543B685E5 for ; Wed, 5 Feb 2025 22:28:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738790882; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ayjQ16AKVVeKMm/QChUsk+0M85GFICyQUxjzHnGuJts=; b=Pg0QYCQ2CFNzirC+7qaAqu0JtCjfiNhAh4o/8BYykRjh8OGpp/EEYwjp+hh2qpAFDMHJrN Lv7wIBY2JnhtSmQzueTIWBtZ126hRqLuKVrzdFwLwyT/oUgJzLYJIHs/vaKPJp3EthvkGI aSneb8mGP/QwrhneaGjFdGMadAbsLl0= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-247-wi7v3xW-NJCArzGZfViEZA-1; Wed, 05 Feb 2025 16:27:59 -0500 X-MC-Unique: wi7v3xW-NJCArzGZfViEZA-1 X-Mimecast-MFC-AGG-ID: wi7v3xW-NJCArzGZfViEZA Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 646C219560A3; Wed, 5 Feb 2025 21:27:58 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.28]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 06E401800265; Wed, 5 Feb 2025 21:27:55 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Robert Mader , Hans de Goede , Laurent Pinchart , Kieran Bingham Subject: [PATCH v6 08/10] libcamera: software_isp: Add an example CCM to uncalibrated.yaml Date: Wed, 5 Feb 2025 22:27:21 +0100 Message-ID: <20250205212725.138055-10-mzamazal@redhat.com> In-Reply-To: <20250205212725.138055-1-mzamazal@redhat.com> References: <20250205212725.138055-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: ds1PyGBLw4oRIRQa5tdKqUuQ4cYMx7iAeCXjbPAZCTg_1738790878 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For performance reasons, color correction matrix (CCM) is not applied by default in software ISP. But let's add a commented out example how to define it to the default tuning file. Signed-off-by: Milan Zamazal Acked-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/simple/data/uncalibrated.yaml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/ipa/simple/data/uncalibrated.yaml b/src/ipa/simple/data/uncalibrated.yaml index 3f147112..5508e668 100644 --- a/src/ipa/simple/data/uncalibrated.yaml +++ b/src/ipa/simple/data/uncalibrated.yaml @@ -5,6 +5,15 @@ version: 1 algorithms: - BlackLevel: - Awb: + # Color correction matrices can be defined here. The CCM algorithm + # has a significant performance impact, and should only be enabled + # if tuned. + # - Ccm: + # ccms: + # - ct: 6500 + # ccm: [ 1, 0, 0, + # 0, 1, 0, + # 0, 0, 1] - Lut: - Agc: ... From patchwork Wed Feb 5 21:27:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22738 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 114A5C32F4 for ; Wed, 5 Feb 2025 21:28:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C2013685ED; Wed, 5 Feb 2025 22:28:06 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="CLzoQ1MY"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C39B3685E5 for ; Wed, 5 Feb 2025 22:28:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738790883; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oRYpEViOgcz2ko14P9mS5nNd4l3hlhJI8ocwVH0ATLY=; b=CLzoQ1MYSTVSmy9t+jhvf4aBXy64w7I4l0n/y2Jo56sbK0v1LDjR/lWhfG6itoyQWFVN1+ MQXARyPhXO4VyGsf7plCLt99azauodglRFmWpNG8TkeSH4qdUX0L5T26A9F3JCvDCcLgjb JjWr15CndZOSEmrC/emoyHgyOiw+1GM= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-648-SQ-_R2L2NbStusxOVH9JvQ-1; Wed, 05 Feb 2025 16:28:02 -0500 X-MC-Unique: SQ-_R2L2NbStusxOVH9JvQ-1 X-Mimecast-MFC-AGG-ID: SQ-_R2L2NbStusxOVH9JvQ Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2A7741800373; Wed, 5 Feb 2025 21:28:01 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.28]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id DAAFA1800360; Wed, 5 Feb 2025 21:27:58 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Robert Mader , Hans de Goede , Laurent Pinchart , Kieran Bingham Subject: [PATCH v6 09/10] libcamera: software_isp: Track whether CCM is enabled Date: Wed, 5 Feb 2025 22:27:22 +0100 Message-ID: <20250205212725.138055-11-mzamazal@redhat.com> In-Reply-To: <20250205212725.138055-1-mzamazal@redhat.com> References: <20250205212725.138055-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: MWK9WfmbwjTOCaqcRc7LsI0Vgk_0JgsiNdS5IFBKjgM_1738790881 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Applying color correction matrix (CCM) in software ISP is optional due to performance reasons. CCM is applied if and only if `Ccm' algorithm is present in the tuning file. Software ISP debayering is a performance critical piece of code and we do not want to use dynamic conditionals there. Therefore we pass information about CCM application to debayering configuration and let it select the right versions of debayering functions using templates. This is a trick similar to the previously used one for adding or not adding an alpha channel to the output. Debayering gets this information but it ignores it in this patch. Actual processing with CCM is added in the followup patch. Signed-off-by: Milan Zamazal Reviewed-by: Laurent Pinchart --- .../internal/software_isp/software_isp.h | 1 + include/libcamera/ipa/soft.mojom | 2 +- src/ipa/simple/algorithms/ccm.cpp | 2 + src/ipa/simple/ipa_context.h | 1 + src/ipa/simple/soft_simple.cpp | 8 +++- src/libcamera/software_isp/debayer.cpp | 3 +- src/libcamera/software_isp/debayer.h | 3 +- src/libcamera/software_isp/debayer_cpu.cpp | 44 ++++++++++++------- src/libcamera/software_isp/debayer_cpu.h | 29 ++++++------ src/libcamera/software_isp/software_isp.cpp | 5 ++- 10 files changed, 61 insertions(+), 37 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index 440a296d..0412315b 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -98,6 +98,7 @@ private: SharedMemObject sharedParams_; DebayerParams debayerParams_; DmaBufAllocator dmaHeap_; + bool ccmEnabled_; std::unique_ptr ipa_; }; diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index d52e6f1a..ede74413 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -17,7 +17,7 @@ interface IPASoftInterface { libcamera.SharedFD fdStats, libcamera.SharedFD fdParams, libcamera.ControlInfoMap sensorCtrlInfoMap) - => (int32 ret, libcamera.ControlInfoMap ipaControls); + => (int32 ret, libcamera.ControlInfoMap ipaControls, bool ccmEnabled); start() => (int32 ret); stop(); configure(IPAConfigInfo configInfo) diff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp index 86e0395c..d5ba928d 100644 --- a/src/ipa/simple/algorithms/ccm.cpp +++ b/src/ipa/simple/algorithms/ccm.cpp @@ -34,6 +34,8 @@ int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData return ret; } + context.ccmEnabled = true; + return 0; } diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 57883218..f0d52d6b 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -82,6 +82,7 @@ struct IPAContext { IPAActiveState activeState; FCQueue frameContexts; ControlInfoMap::Map ctrlMap; + bool ccmEnabled; }; } /* namespace ipa::soft */ diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index b26e4e15..a87c6cdd 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -51,7 +51,8 @@ public: const SharedFD &fdStats, const SharedFD &fdParams, const ControlInfoMap &sensorInfoMap, - ControlInfoMap *ipaControls) override; + ControlInfoMap *ipaControls, + bool *ccmEnabled) override; int configure(const IPAConfigInfo &configInfo) override; int start() override; @@ -89,7 +90,8 @@ int IPASoftSimple::init(const IPASettings &settings, const SharedFD &fdStats, const SharedFD &fdParams, const ControlInfoMap &sensorInfoMap, - ControlInfoMap *ipaControls) + ControlInfoMap *ipaControls, + bool *ccmEnabled) { camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); if (!camHelper_) { @@ -125,6 +127,8 @@ int IPASoftSimple::init(const IPASettings &settings, if (ret) return ret; + *ccmEnabled = context_.ccmEnabled; + params_ = nullptr; stats_ = nullptr; diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index f0b83261..34e42201 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -57,10 +57,11 @@ Debayer::~Debayer() } /** - * \fn int Debayer::configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfgs) + * \fn int Debayer::configure() * \brief Configure the debayer object according to the passed in parameters * \param[in] inputCfg The input configuration * \param[in] outputCfgs The output configurations + * \param[in] ccmEnabled Whether a color correction matrix is applied * * \return 0 on success, a negative errno on failure */ diff --git a/src/libcamera/software_isp/debayer.h b/src/libcamera/software_isp/debayer.h index d7ca060d..ba033d44 100644 --- a/src/libcamera/software_isp/debayer.h +++ b/src/libcamera/software_isp/debayer.h @@ -33,7 +33,8 @@ public: virtual ~Debayer() = 0; virtual int configure(const StreamConfiguration &inputCfg, - const std::vector> &outputCfgs) = 0; + const std::vector> &outputCfgs, + bool ccmEnabled) = 0; virtual std::vector formats(PixelFormat inputFormat) = 0; diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 01cfb36b..0cd03a8f 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -114,7 +114,7 @@ DebayerCpu::~DebayerCpu() = default; (prev[x] + curr[x - p] + curr[x + n] + next[x]) / (4 * (div)), \ curr[x] / (div)) -template +template void DebayerCpu::debayer8_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]) { DECLARE_SRC_POINTERS(uint8_t) @@ -125,7 +125,7 @@ void DebayerCpu::debayer8_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]) } } -template +template void DebayerCpu::debayer8_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]) { DECLARE_SRC_POINTERS(uint8_t) @@ -136,7 +136,7 @@ void DebayerCpu::debayer8_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]) } } -template +template void DebayerCpu::debayer10_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]) { DECLARE_SRC_POINTERS(uint16_t) @@ -148,7 +148,7 @@ void DebayerCpu::debayer10_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]) } } -template +template void DebayerCpu::debayer10_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]) { DECLARE_SRC_POINTERS(uint16_t) @@ -160,7 +160,7 @@ void DebayerCpu::debayer10_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]) } } -template +template void DebayerCpu::debayer12_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]) { DECLARE_SRC_POINTERS(uint16_t) @@ -172,7 +172,7 @@ void DebayerCpu::debayer12_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]) } } -template +template void DebayerCpu::debayer12_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]) { DECLARE_SRC_POINTERS(uint16_t) @@ -184,7 +184,7 @@ void DebayerCpu::debayer12_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]) } } -template +template void DebayerCpu::debayer10P_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]) { const int widthInBytes = window_.width * 5 / 4; @@ -210,7 +210,7 @@ void DebayerCpu::debayer10P_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]) } } -template +template void DebayerCpu::debayer10P_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]) { const int widthInBytes = window_.width * 5 / 4; @@ -231,7 +231,7 @@ void DebayerCpu::debayer10P_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]) } } -template +template void DebayerCpu::debayer10P_GBGB_BGR888(uint8_t *dst, const uint8_t *src[]) { const int widthInBytes = window_.width * 5 / 4; @@ -252,7 +252,7 @@ void DebayerCpu::debayer10P_GBGB_BGR888(uint8_t *dst, const uint8_t *src[]) } } -template +template void DebayerCpu::debayer10P_RGRG_BGR888(uint8_t *dst, const uint8_t *src[]) { const int widthInBytes = window_.width * 5 / 4; @@ -368,11 +368,17 @@ int DebayerCpu::setupStandardBayerOrder(BayerFormat::Order order) return 0; } -#define SET_DEBAYER_METHODS(method0, method1) \ - debayer0_ = addAlphaByte ? &DebayerCpu::method0 : &DebayerCpu::method0; \ - debayer1_ = addAlphaByte ? &DebayerCpu::method1 : &DebayerCpu::method1; - -int DebayerCpu::setDebayerFunctions(PixelFormat inputFormat, PixelFormat outputFormat) +#define SET_DEBAYER_METHODS(method0, method1) \ + debayer0_ = addAlphaByte \ + ? (ccmEnabled ? &DebayerCpu::method0 : &DebayerCpu::method0) \ + : (ccmEnabled ? &DebayerCpu::method0 : &DebayerCpu::method0); \ + debayer1_ = addAlphaByte \ + ? (ccmEnabled ? &DebayerCpu::method1 : &DebayerCpu::method1) \ + : (ccmEnabled ? &DebayerCpu::method1 : &DebayerCpu::method1); + +int DebayerCpu::setDebayerFunctions(PixelFormat inputFormat, + PixelFormat outputFormat, + bool ccmEnabled) { BayerFormat bayerFormat = BayerFormat::fromPixelFormat(inputFormat); @@ -464,7 +470,8 @@ int DebayerCpu::setDebayerFunctions(PixelFormat inputFormat, PixelFormat outputF } int DebayerCpu::configure(const StreamConfiguration &inputCfg, - const std::vector> &outputCfgs) + const std::vector> &outputCfgs, + bool ccmEnabled) { if (getInputConfig(inputCfg.pixelFormat, inputConfig_) != 0) return -EINVAL; @@ -503,7 +510,10 @@ int DebayerCpu::configure(const StreamConfiguration &inputCfg, return -EINVAL; } - if (setDebayerFunctions(inputCfg.pixelFormat, outputCfg.pixelFormat) != 0) + int ret = setDebayerFunctions(inputCfg.pixelFormat, + outputCfg.pixelFormat, + ccmEnabled); + if (ret != 0) return -EINVAL; window_.x = ((inputCfg.size.width - outputCfg.size.width) / 2) & diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 2c47e7c6..21c08a2d 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* * Copyright (C) 2023, Linaro Ltd - * Copyright (C) 2023, Red Hat Inc. + * Copyright (C) 2023-2025 Red Hat Inc. * * Authors: * Hans de Goede @@ -31,7 +31,8 @@ public: ~DebayerCpu(); int configure(const StreamConfiguration &inputCfg, - const std::vector> &outputCfgs); + const std::vector> &outputCfgs, + bool ccmEnabled); Size patternSize(PixelFormat inputFormat); std::vector formats(PixelFormat input); std::tuple @@ -85,28 +86,28 @@ private: using debayerFn = void (DebayerCpu::*)(uint8_t *dst, const uint8_t *src[]); /* 8-bit raw bayer format */ - template + template void debayer8_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]); - template + template void debayer8_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]); /* unpacked 10-bit raw bayer format */ - template + template void debayer10_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]); - template + template void debayer10_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]); /* unpacked 12-bit raw bayer format */ - template + template void debayer12_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]); - template + template void debayer12_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]); /* CSI-2 packed 10-bit raw bayer format (all the 4 orders) */ - template + template void debayer10P_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]); - template + template void debayer10P_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]); - template + template void debayer10P_GBGB_BGR888(uint8_t *dst, const uint8_t *src[]); - template + template void debayer10P_RGRG_BGR888(uint8_t *dst, const uint8_t *src[]); struct DebayerInputConfig { @@ -125,7 +126,9 @@ private: int getInputConfig(PixelFormat inputFormat, DebayerInputConfig &config); int getOutputConfig(PixelFormat outputFormat, DebayerOutputConfig &config); int setupStandardBayerOrder(BayerFormat::Order order); - int setDebayerFunctions(PixelFormat inputFormat, PixelFormat outputFormat); + int setDebayerFunctions(PixelFormat inputFormat, + PixelFormat outputFormat, + bool ccmEnabled); void setupInputMemcpy(const uint8_t *linePointers[]); void shiftLinePointers(const uint8_t *linePointers[], const uint8_t *src); void memcpyNextLine(const uint8_t *linePointers[]); diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 44baf200..b03c4baa 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -128,7 +128,8 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, debayer_->getStatsFD(), sharedParams_.fd(), sensor->controls(), - ipaControls); + ipaControls, + &ccmEnabled_); if (ret) { LOG(SoftwareIsp, Error) << "IPA init failed"; debayer_.reset(); @@ -240,7 +241,7 @@ int SoftwareIsp::configure(const StreamConfiguration &inputCfg, if (ret < 0) return ret; - return debayer_->configure(inputCfg, outputCfgs); + return debayer_->configure(inputCfg, outputCfgs, ccmEnabled_); } /** From patchwork Wed Feb 5 21:27:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22739 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EA173C32F2 for ; Wed, 5 Feb 2025 21:28:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A1213685ED; Wed, 5 Feb 2025 22:28:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="BrWUklZK"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E1DC5685ED for ; Wed, 5 Feb 2025 22:28:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738790888; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bcCT4NewJWApKSa+WX63OZhbCWAOBtERwanzk1wfg8o=; b=BrWUklZKiqOBqvPWsAyLonYcJSiDf3wc0Wc1j9bkVChVJ3lhu6qgiS9BFULdZlCKTJhqHs 5pXzLeoDkzmXFyd9/viBfWKjZF71Iw+oHCTjqhrPH1rEEGP+2FzaYL/2qcGB+lVv41uDwc n6RbnM21Huc8s5457+rZXliUO9+mEiA= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-682-I2-zD0K2NtmlneWHgO-ijQ-1; Wed, 05 Feb 2025 16:28:05 -0500 X-MC-Unique: I2-zD0K2NtmlneWHgO-ijQ-1 X-Mimecast-MFC-AGG-ID: I2-zD0K2NtmlneWHgO-ijQ Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 11AA1180087C; Wed, 5 Feb 2025 21:28:04 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.28]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A13FF1800360; Wed, 5 Feb 2025 21:28:01 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Robert Mader , Hans de Goede , Laurent Pinchart , Kieran Bingham Subject: [PATCH v6 10/10] libcamera: software_isp: Apply CCM in debayering Date: Wed, 5 Feb 2025 22:27:23 +0100 Message-ID: <20250205212725.138055-12-mzamazal@redhat.com> In-Reply-To: <20250205212725.138055-1-mzamazal@redhat.com> References: <20250205212725.138055-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: pGqHap_UweuE4LOGT1-r6WrBOmaVZx5HHC3OdPekR3o_1738790884 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch applies color correction matrix (CCM) in debayering if the CCM is specified. Not using CCM must still be supported for performance reasons. The CCM is applied as follows: [r1 g1 b1] [r] [r2 g2 b2] * [g] [r3 g3 b3] [b] The CCM matrix (the left side of the multiplication) is constant during single frame processing, while the input pixel (the right side) changes. Because each of the color channels is only 8-bit in software ISP, we can make 9 lookup tables with 256 input values for multiplications of each of the r_i, g_i, b_i values. This way we don't have to multiply each pixel, we can use table lookups and additions instead. Gamma (which is non-linear and thus cannot be a part of the 9 lookup tables values) is applied on the final values rounded to integers using another lookup table. Because the changing part is the pixel value with three color elements, only three dynamic table lookups are needed. We use three lookup tables to represent the multiplied matrix values, each of the tables corresponding to the given matrix column and pixel color. We use int16_t to store the precomputed multiplications. This seems to be noticeably (>10%) faster than `float' for the price of slightly less accuracy and it covers the range of values that sane CCMs produce. The selection and structure of data is performance critical, for example using bytes would add significant (>10%) speedup but would be too short to cover the value range. The color lookup tables can be represented either as unions, accommodating tables for both the CCM and non-CCM cases, or as separate tables for each of the cases, leaving the tables for the other case unused. The latter is selected as a matter of preference. The tables are copied (as before), which is not elegant but also not a big problem. There are patches posted that use shared buffers for parameters passing in software ISP (see software ISP TODO #5) and they can be adjusted for the new parameter format. Color gains from white balance are supposed not to be a part of the specified CCM. They are applied on it using matrix multiplication, which is simple and in correspondence with future additions in the form of matrix multiplication, like saturation adjustment. With this patch, the reported per-frame slowdown when applying CCM is about 45% on Debix Model A and about 75% on TI AM69 SK. Using std::clamp in debayering adds some performance penalty (a few percent). The clamping is necessary to eliminate out of range values possibly produced by the CCM. If it could be avoided by adjusting the precomputed tables some way then performance could be improved a bit. Signed-off-by: Milan Zamazal --- .../internal/software_isp/debayer_params.h | 34 ++++++++++-- src/ipa/simple/algorithms/lut.cpp | 53 ++++++++++++++----- src/ipa/simple/algorithms/lut.h | 1 + src/libcamera/software_isp/debayer.cpp | 52 +++++++++++++++++- src/libcamera/software_isp/debayer_cpu.cpp | 53 +++++++++++++++---- src/libcamera/software_isp/debayer_cpu.h | 10 ++-- 6 files changed, 172 insertions(+), 31 deletions(-) diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index 7d8fdd48..e85b5c60 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2023, 2024 Red Hat Inc. + * Copyright (C) 2023-2025 Red Hat Inc. * * Authors: * Hans de Goede @@ -18,11 +18,35 @@ namespace libcamera { struct DebayerParams { static constexpr unsigned int kRGBLookupSize = 256; - using ColorLookupTable = std::array; + struct CcmColumn { + int16_t r; + int16_t g; + int16_t b; + }; - ColorLookupTable red; - ColorLookupTable green; - ColorLookupTable blue; + using LookupTable = std::array; + using CcmLookupTable = std::array; + + /* + * Color lookup tables when CCM is not used. + * Each color of a debayered pixel is amended by the corresponding + * value in the given table. + */ + LookupTable red; + LookupTable green; + LookupTable blue; + + /* + * Color and gamma lookup tables when CCM is used. + * Each of the CcmLookupTable's corresponds to a CCM column; together they + * make a complete 3x3 CCM lookup table. The CCM is applied on debayered + * pixels and then the gamma lookup table is used to set the resulting + * values of all the three colors. + */ + CcmLookupTable redCcm; + CcmLookupTable greenCcm; + CcmLookupTable blueCcm; + LookupTable gammaLut; }; } /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp index 352bbf57..1eaa2b5e 100644 --- a/src/ipa/simple/algorithms/lut.cpp +++ b/src/ipa/simple/algorithms/lut.cpp @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2024, Red Hat Inc. + * Copyright (C) 2024-2025, Red Hat Inc. * * Color lookup tables construction */ @@ -80,6 +80,11 @@ void Lut::updateGammaTable(IPAContext &context) context.activeState.gamma.contrast = contrast; } +int16_t Lut::ccmValue(unsigned int i, float ccm) const +{ + return std::round(i * ccm); +} + void Lut::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameContext, @@ -91,22 +96,46 @@ void Lut::prepare(IPAContext &context, * observed, it's not permanently prone to minor fluctuations or * rounding errors. */ - if (context.activeState.gamma.blackLevel != context.activeState.blc.level || - context.activeState.gamma.contrast != context.activeState.knobs.contrast) + const bool gammaUpdateNeeded = + context.activeState.gamma.blackLevel != context.activeState.blc.level || + context.activeState.gamma.contrast != context.activeState.knobs.contrast; + if (gammaUpdateNeeded) updateGammaTable(context); auto &gains = context.activeState.awb.gains; auto &gammaTable = context.activeState.gamma.gammaTable; const unsigned int gammaTableSize = gammaTable.size(); - - for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { - const double div = static_cast(DebayerParams::kRGBLookupSize) / - gammaTableSize; - /* Apply gamma after gain! */ - const RGB lutGains = (gains * i / div).min(gammaTableSize - 1); - params->red[i] = gammaTable[static_cast(lutGains.r())]; - params->green[i] = gammaTable[static_cast(lutGains.g())]; - params->blue[i] = gammaTable[static_cast(lutGains.b())]; + const double div = static_cast(DebayerParams::kRGBLookupSize) / + gammaTableSize; + + if (!context.ccmEnabled) { + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { + /* Apply gamma after gain! */ + const RGB lutGains = (gains * i / div).min(gammaTableSize - 1); + params->red[i] = gammaTable[static_cast(lutGains.r())]; + params->green[i] = gammaTable[static_cast(lutGains.g())]; + params->blue[i] = gammaTable[static_cast(lutGains.b())]; + } + } else if (context.activeState.ccm.changed || gammaUpdateNeeded) { + Matrix gainCcm = { { gains.r(), 0, 0, + 0, gains.g(), 0, + 0, 0, gains.b() } }; + auto ccm = gainCcm * context.activeState.ccm.ccm; + auto &red = params->redCcm; + auto &green = params->greenCcm; + auto &blue = params->blueCcm; + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { + red[i].r = ccmValue(i, ccm[0][0]); + red[i].g = ccmValue(i, ccm[1][0]); + red[i].b = ccmValue(i, ccm[2][0]); + green[i].r = ccmValue(i, ccm[0][1]); + green[i].g = ccmValue(i, ccm[1][1]); + green[i].b = ccmValue(i, ccm[2][1]); + blue[i].r = ccmValue(i, ccm[0][2]); + blue[i].g = ccmValue(i, ccm[1][2]); + blue[i].b = ccmValue(i, ccm[2][2]); + params->gammaLut[i] = gammaTable[i / div]; + } } } diff --git a/src/ipa/simple/algorithms/lut.h b/src/ipa/simple/algorithms/lut.h index 889f864b..77324800 100644 --- a/src/ipa/simple/algorithms/lut.h +++ b/src/ipa/simple/algorithms/lut.h @@ -33,6 +33,7 @@ public: private: void updateGammaTable(IPAContext &context); + int16_t ccmValue(unsigned int i, float ccm) const; }; } /* namespace ipa::soft::algorithms */ diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index 34e42201..2cae8ef0 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -23,9 +23,39 @@ namespace libcamera { * \brief Size of a color lookup table */ +/** + * \struct DebayerParams::CcmColumn + * \brief Type of a single column of a color correction matrix (CCM) + */ + +/** + * \var DebayerParams::CcmColumn::r + * \brief Red (first) component of a CCM column + */ + +/** + * \var DebayerParams::CcmColumn::g + * \brief Green (second) component of a CCM column + */ + +/** + * \var DebayerParams::CcmColumn::b + * \brief Blue (third) component of a CCM column + */ + /** * \typedef DebayerParams::ColorLookupTable - * \brief Type of the lookup tables for red, green, blue values + * \brief Type of the simple lookup tables for red, green, blue values + */ + +/** + * \typedef DebayerParams::CcmLookupTable + * \brief Type of the CCM lookup tables for red, green, blue values + */ + +/** + * \typedef DebayerParams::GammaLookupTable + * \brief Type of the gamma lookup tables for CCM */ /** @@ -43,6 +73,26 @@ namespace libcamera { * \brief Lookup table for blue color, mapping input values to output values */ +/** + * \var DebayerParams::redCcm + * \brief CCM lookup table for red color, mapping input values to output values + */ + +/** + * \var DebayerParams::greenCcm + * \brief CCM lookup table for green color, mapping input values to output values + */ + +/** + * \var DebayerParams::blueCcm + * \brief CCM lookup table for blue color, mapping input values to output values + */ + +/** + * \var DebayerParams::gammaLut + * \brief Gamma lookup table used with color correction matrix + */ + /** * \class Debayer * \brief Base debayering class diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 0cd03a8f..83a171d8 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* * Copyright (C) 2023, Linaro Ltd - * Copyright (C) 2023, Red Hat Inc. + * Copyright (C) 2023-2025 Red Hat Inc. * * Authors: * Hans de Goede @@ -11,9 +11,11 @@ #include "debayer_cpu.h" +#include #include #include #include +#include #include @@ -51,8 +53,12 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats) enableInputMemcpy_ = true; /* Initialize color lookup tables */ - for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) + for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { red_[i] = green_[i] = blue_[i] = i; + redCcm_[i] = { 1, 0, 0 }; + greenCcm_[i] = { 0, 1, 0 }; + blueCcm_[i] = { 0, 0, 1 }; + } } DebayerCpu::~DebayerCpu() = default; @@ -62,12 +68,24 @@ DebayerCpu::~DebayerCpu() = default; const pixel_t *curr = (const pixel_t *)src[1] + xShift_; \ const pixel_t *next = (const pixel_t *)src[2] + xShift_; -#define STORE_PIXEL(b, g, r) \ - *dst++ = blue_[b]; \ - *dst++ = green_[g]; \ - *dst++ = red_[r]; \ - if constexpr (addAlphaByte) \ - *dst++ = 255; \ +#define GAMMA(value) \ + *dst++ = gammaLut_[std::clamp(value, 0, static_cast(gammaLut_.size()) - 1)] + +#define STORE_PIXEL(b_, g_, r_) \ + if constexpr (ccmEnabled) { \ + const DebayerParams::CcmColumn &blue = blueCcm_[b_]; \ + const DebayerParams::CcmColumn &green = greenCcm_[g_]; \ + const DebayerParams::CcmColumn &red = redCcm_[r_]; \ + GAMMA(blue.b + green.b + red.b); \ + GAMMA(blue.g + green.g + red.g); \ + GAMMA(blue.r + green.r + red.r); \ + } else { \ + *dst++ = blue_[b_]; \ + *dst++ = green_[g_]; \ + *dst++ = red_[r_]; \ + } \ + if constexpr (addAlphaByte) \ + *dst++ = 255; \ x++; /* @@ -755,8 +773,23 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output dmaSyncers.emplace_back(plane.fd, DmaSyncer::SyncType::Write); green_ = params.green; - red_ = swapRedBlueGains_ ? params.blue : params.red; - blue_ = swapRedBlueGains_ ? params.red : params.blue; + greenCcm_ = params.greenCcm; + if (swapRedBlueGains_) { + red_ = params.blue; + blue_ = params.red; + redCcm_ = params.blueCcm; + blueCcm_ = params.redCcm; + for (unsigned int i = 0; i < 256; i++) { + std::swap(redCcm_[i].r, redCcm_[i].b); + std::swap(blueCcm_[i].r, blueCcm_[i].b); + } + } else { + red_ = params.red; + blue_ = params.blue; + redCcm_ = params.redCcm; + blueCcm_ = params.blueCcm; + } + gammaLut_ = params.gammaLut; /* Copy metadata from the input buffer */ FrameMetadata &metadata = output->_d()->metadata(); diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 21c08a2d..926195e9 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -138,9 +138,13 @@ private: /* Max. supported Bayer pattern height is 4, debayering this requires 5 lines */ static constexpr unsigned int kMaxLineBuffers = 5; - DebayerParams::ColorLookupTable red_; - DebayerParams::ColorLookupTable green_; - DebayerParams::ColorLookupTable blue_; + DebayerParams::LookupTable red_; + DebayerParams::LookupTable green_; + DebayerParams::LookupTable blue_; + DebayerParams::CcmLookupTable redCcm_; + DebayerParams::CcmLookupTable greenCcm_; + DebayerParams::CcmLookupTable blueCcm_; + DebayerParams::LookupTable gammaLut_; debayerFn debayer0_; debayerFn debayer1_; debayerFn debayer2_;