[v4,1/9] libcamera: software_isp: Determine color temperature
diff mbox series

Message ID 20250113135108.13924-2-mzamazal@redhat.com
State Superseded
Headers show
Series
  • Software ISP support for CCM
Related show

Commit Message

Milan Zamazal Jan. 13, 2025, 1:50 p.m. UTC 
The AWB algorithm has data to determine color temperature of the image.
Let's compute the temperature from it and store it into the context.
This piece of information is currently unused but it will be needed in a
followup patch introducing support for color correction matrix.

Let's store the white balance related information under `awb' subsection
of the active state, as the hardware pipelines do.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
---
 src/ipa/simple/algorithms/awb.cpp | 13 ++++++++++---
 src/ipa/simple/algorithms/lut.cpp |  2 +-
 src/ipa/simple/ipa_context.h      | 11 +++++++----
 3 files changed, 18 insertions(+), 8 deletions(-)

Comments

Laurent Pinchart Jan. 26, 2025, 10:50 p.m. UTC | #1 
Hi Milan,

Thank you for the patch.

On Mon, Jan 13, 2025 at 02:50:58PM +0100, Milan Zamazal wrote:
> The AWB algorithm has data to determine color temperature of the image.
> Let's compute the temperature from it and store it into the context.
> This piece of information is currently unused but it will be needed in a
> followup patch introducing support for color correction matrix.
> 
> Let's store the white balance related information under `awb' subsection
> of the active state, as the hardware pipelines do.
> 
> Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
> ---
>  src/ipa/simple/algorithms/awb.cpp | 13 ++++++++++---
>  src/ipa/simple/algorithms/lut.cpp |  2 +-
>  src/ipa/simple/ipa_context.h      | 11 +++++++----
>  3 files changed, 18 insertions(+), 8 deletions(-)
> 
> diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp
> index 195de41d..f4fe1af6 100644
> --- a/src/ipa/simple/algorithms/awb.cpp
> +++ b/src/ipa/simple/algorithms/awb.cpp
> @@ -12,6 +12,7 @@
>  
>  #include <libcamera/base/log.h>
>  
> +#include "libipa/colours.h"
>  #include "simple/ipa_context.h"
>  
>  namespace libcamera {
> @@ -23,7 +24,7 @@ namespace ipa::soft::algorithms {
>  int Awb::configure(IPAContext &context,
>  		   [[maybe_unused]] const IPAConfigInfo &configInfo)
>  {
> -	auto &gains = context.activeState.gains;
> +	auto &gains = context.activeState.awb.gains;
>  	gains.red = gains.green = gains.blue = 1.0;
>  
>  	return 0;
> @@ -54,12 +55,18 @@ void Awb::process(IPAContext &context,
>  	 * Calculate red and blue gains for AWB.
>  	 * Clamp max gain at 4.0, this also avoids 0 division.
>  	 */
> -	auto &gains = context.activeState.gains;
> +	auto &gains = context.activeState.awb.gains;
>  	gains.red = sumR <= sumG / 4 ? 4.0 : static_cast<double>(sumG) / sumR;
>  	gains.blue = sumB <= sumG / 4 ? 4.0 : static_cast<double>(sumG) / sumB;
>  	/* Green gain is fixed to 1.0 */
>  
> -	LOG(IPASoftAwb, Debug) << "gain R/B " << gains.red << "/" << gains.blue;
> +	RGB<double> rgbGains{ { 1 / gains.red, 1 / gains.green, 1 / gains.blue } };
> +	uint32_t temperature = estimateCCT(rgbGains);
> +	context.activeState.awb.temperatureK = temperature;

	context.activeState.awb.temperatureK = estimateCCT(rgbGains);

> +
> +	LOG(IPASoftAwb, Debug)
> +		<< "gain R/B: " << gains.red << "/" << gains.blue
> +		<< "; temperature: " << context.activeState.awb.temperatureK;
>  }
>  
>  REGISTER_IPA_ALGORITHM(Awb, "Awb")
> diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp
> index 0ba2391f..d75ff710 100644
> --- a/src/ipa/simple/algorithms/lut.cpp
> +++ b/src/ipa/simple/algorithms/lut.cpp
> @@ -95,7 +95,7 @@ void Lut::prepare(IPAContext &context,
>  	    context.activeState.gamma.contrast != context.activeState.knobs.contrast)
>  		updateGammaTable(context);
>  
> -	auto &gains = context.activeState.gains;
> +	auto &gains = context.activeState.awb.gains;
>  	auto &gammaTable = context.activeState.gamma.gammaTable;
>  	const unsigned int gammaTableSize = gammaTable.size();
>  
> diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h
> index 4af51306..607af45a 100644
> --- a/src/ipa/simple/ipa_context.h
> +++ b/src/ipa/simple/ipa_context.h
> @@ -36,10 +36,13 @@ struct IPAActiveState {
>  	} blc;
>  
>  	struct {
> -		double red;
> -		double green;
> -		double blue;
> -	} gains;
> +		struct {
> +			double red;
> +			double green;
> +			double blue;
> +		} gains;

Please add a patch on top (or at the bottom, up to you) to convert this
to

		RGB<double> gains;

(don't forget to include vector.h)

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +		unsigned int temperatureK;
> +	} awb;
>  
>  	static constexpr unsigned int kGammaLookupSize = 1024;
>  	struct {
Milan Zamazal Jan. 28, 2025, 9:26 a.m. UTC | #2 
Hi Laurent,

thank you for review.

Laurent Pinchart <laurent.pinchart@ideasonboard.com> writes:

> Hi Milan,
>
> Thank you for the patch.
>
> On Mon, Jan 13, 2025 at 02:50:58PM +0100, Milan Zamazal wrote:
>> The AWB algorithm has data to determine color temperature of the image.
>> Let's compute the temperature from it and store it into the context.
>> This piece of information is currently unused but it will be needed in a
>> followup patch introducing support for color correction matrix.
>> 
>> Let's store the white balance related information under `awb' subsection
>> of the active state, as the hardware pipelines do.
>> 
>> Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
>> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
>> ---
>>  src/ipa/simple/algorithms/awb.cpp | 13 ++++++++++---
>>  src/ipa/simple/algorithms/lut.cpp |  2 +-
>>  src/ipa/simple/ipa_context.h      | 11 +++++++----
>>  3 files changed, 18 insertions(+), 8 deletions(-)
>> 
>> diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp
>> index 195de41d..f4fe1af6 100644
>> --- a/src/ipa/simple/algorithms/awb.cpp
>> +++ b/src/ipa/simple/algorithms/awb.cpp
>> @@ -12,6 +12,7 @@
>>  
>>  #include <libcamera/base/log.h>
>>  
>> +#include "libipa/colours.h"
>>  #include "simple/ipa_context.h"
>>  
>>  namespace libcamera {
>> @@ -23,7 +24,7 @@ namespace ipa::soft::algorithms {
>>  int Awb::configure(IPAContext &context,
>>  		   [[maybe_unused]] const IPAConfigInfo &configInfo)
>>  {
>> -	auto &gains = context.activeState.gains;
>> +	auto &gains = context.activeState.awb.gains;
>>  	gains.red = gains.green = gains.blue = 1.0;
>>  
>>  	return 0;
>> @@ -54,12 +55,18 @@ void Awb::process(IPAContext &context,
>>  	 * Calculate red and blue gains for AWB.
>>  	 * Clamp max gain at 4.0, this also avoids 0 division.
>>  	 */
>> -	auto &gains = context.activeState.gains;
>> +	auto &gains = context.activeState.awb.gains;
>>  	gains.red = sumR <= sumG / 4 ? 4.0 : static_cast<double>(sumG) / sumR;
>>  	gains.blue = sumB <= sumG / 4 ? 4.0 : static_cast<double>(sumG) / sumB;
>>  	/* Green gain is fixed to 1.0 */
>>  
>> -	LOG(IPASoftAwb, Debug) << "gain R/B " << gains.red << "/" << gains.blue;
>> +	RGB<double> rgbGains{ { 1 / gains.red, 1 / gains.green, 1 / gains.blue } };
>> +	uint32_t temperature = estimateCCT(rgbGains);
>> +	context.activeState.awb.temperatureK = temperature;
>
> 	context.activeState.awb.temperatureK = estimateCCT(rgbGains);

Ack.

>> +
>> +	LOG(IPASoftAwb, Debug)
>> +		<< "gain R/B: " << gains.red << "/" << gains.blue
>> +		<< "; temperature: " << context.activeState.awb.temperatureK;
>>  }
>>  
>>  REGISTER_IPA_ALGORITHM(Awb, "Awb")
>> diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp
>> index 0ba2391f..d75ff710 100644
>> --- a/src/ipa/simple/algorithms/lut.cpp
>> +++ b/src/ipa/simple/algorithms/lut.cpp
>> @@ -95,7 +95,7 @@ void Lut::prepare(IPAContext &context,
>>  	    context.activeState.gamma.contrast != context.activeState.knobs.contrast)
>>  		updateGammaTable(context);
>>  
>> -	auto &gains = context.activeState.gains;
>> +	auto &gains = context.activeState.awb.gains;
>>  	auto &gammaTable = context.activeState.gamma.gammaTable;
>>  	const unsigned int gammaTableSize = gammaTable.size();
>>  
>> diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h
>> index 4af51306..607af45a 100644
>> --- a/src/ipa/simple/ipa_context.h
>> +++ b/src/ipa/simple/ipa_context.h
>> @@ -36,10 +36,13 @@ struct IPAActiveState {
>>  	} blc;
>>  
>>  	struct {
>> -		double red;
>> -		double green;
>> -		double blue;
>> -	} gains;
>> +		struct {
>> +			double red;
>> +			double green;
>> +			double blue;
>> +		} gains;
>
> Please add a patch on top (or at the bottom, up to you) to convert this
> to
>
> 		RGB<double> gains;
>
> (don't forget to include vector.h)

I'll add it on top in v5.

> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>
>> +		unsigned int temperatureK;
>> +	} awb;
>>  
>>  	static constexpr unsigned int kGammaLookupSize = 1024;
>>  	struct {

Patch
diff mbox series 

diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp
index 195de41d..f4fe1af6 100644
--- a/src/ipa/simple/algorithms/awb.cpp
+++ b/src/ipa/simple/algorithms/awb.cpp
@@ -12,6 +12,7 @@ 
 
 #include <libcamera/base/log.h>
 
+#include "libipa/colours.h"
 #include "simple/ipa_context.h"
 
 namespace libcamera {
@@ -23,7 +24,7 @@  namespace ipa::soft::algorithms {
 int Awb::configure(IPAContext &context,
 		   [[maybe_unused]] const IPAConfigInfo &configInfo)
 {
-	auto &gains = context.activeState.gains;
+	auto &gains = context.activeState.awb.gains;
 	gains.red = gains.green = gains.blue = 1.0;
 
 	return 0;
@@ -54,12 +55,18 @@  void Awb::process(IPAContext &context,
 	 * Calculate red and blue gains for AWB.
 	 * Clamp max gain at 4.0, this also avoids 0 division.
 	 */
-	auto &gains = context.activeState.gains;
+	auto &gains = context.activeState.awb.gains;
 	gains.red = sumR <= sumG / 4 ? 4.0 : static_cast<double>(sumG) / sumR;
 	gains.blue = sumB <= sumG / 4 ? 4.0 : static_cast<double>(sumG) / sumB;
 	/* Green gain is fixed to 1.0 */
 
-	LOG(IPASoftAwb, Debug) << "gain R/B " << gains.red << "/" << gains.blue;
+	RGB<double> rgbGains{ { 1 / gains.red, 1 / gains.green, 1 / gains.blue } };
+	uint32_t temperature = estimateCCT(rgbGains);
+	context.activeState.awb.temperatureK = temperature;
+
+	LOG(IPASoftAwb, Debug)
+		<< "gain R/B: " << gains.red << "/" << gains.blue
+		<< "; temperature: " << context.activeState.awb.temperatureK;
 }
 
 REGISTER_IPA_ALGORITHM(Awb, "Awb")
diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp
index 0ba2391f..d75ff710 100644
--- a/src/ipa/simple/algorithms/lut.cpp
+++ b/src/ipa/simple/algorithms/lut.cpp
@@ -95,7 +95,7 @@  void Lut::prepare(IPAContext &context,
 	    context.activeState.gamma.contrast != context.activeState.knobs.contrast)
 		updateGammaTable(context);
 
-	auto &gains = context.activeState.gains;
+	auto &gains = context.activeState.awb.gains;
 	auto &gammaTable = context.activeState.gamma.gammaTable;
 	const unsigned int gammaTableSize = gammaTable.size();
 
diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h
index 4af51306..607af45a 100644
--- a/src/ipa/simple/ipa_context.h
+++ b/src/ipa/simple/ipa_context.h
@@ -36,10 +36,13 @@  struct IPAActiveState {
 	} blc;
 
 	struct {
-		double red;
-		double green;
-		double blue;
-	} gains;
+		struct {
+			double red;
+			double green;
+			double blue;
+		} gains;
+		unsigned int temperatureK;
+	} awb;
 
 	static constexpr unsigned int kGammaLookupSize = 1024;
 	struct {