[v3] libcamera: pipeline: simple: Prefer CSI2 packed bayer formats over unpacket formats
diff mbox series

Message ID 20261028214307.158415-1-johannes.goede@oss.qualcomm.com
State Accepted
Headers show
Series
  • [v3] libcamera: pipeline: simple: Prefer CSI2 packed bayer formats over unpacket formats
Related show

Commit Message

Hans de Goede Oct. 28, 2026, 9:43 p.m. UTC
Some CSI2 receivers like the IPU6 isys can store e.g. 10 bpp Bayer formats
in either CSI2 packed (5 bytes / 4 pixels) or unpacked (2 bytes / pixel)
formats.

Before this change the simple-pipeline handler would pick whatever format
was enumerated first as capture format in this case. Which on IPU6 boards
comes down to picking unpacked.

Packed formats use less memory and typically perform better due to reduced
RAM bandwidth requirements. E.g. on a Lenovo ThinkPad X1 Yoga Gen 8 with
i7-1370P CPU and a ov2740 sensor running at full sensor (FHD) resolution,
3 consecutive benchmark runs using libcamera's software ISP's builtin
Debayer benchmark gives the following results:

Unpacked  EGL us/frame:  6398 / 7393 / 7524
CSI2 pack EGL us/frame:  5682 / 5536 / 5634

Unpacked  CPU us/frame:  7653 / 8147 / 8614
CSI2 pack CPU us/frame:  7183 / 7063 / 6318

Note the EGL comparison is not entirely fair, the unpacked debayer shader
uses the "McGuire" Debayer algorithm where as the packed one uses simpler
linear filtering.

Modify the simple pipeline handler to prefer the more performant CSI2
packed format when both formats are available.

This also fixes a problem with unpacked formats on IPU6 where sometimes at
the start of a stream the IPU6 isys CSI2 receiver may return a corrupt
unpacked frame with high bits set which causes libcamera to crash with
an out of bound array access as the software ISP code expects the high bits
of unpacked formats to always be 0.

Closes: https://gitlab.freedesktop.org/camera/libcamera/-/work_items/328
Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
---
Changes in v3:
- s/support/supported/ in the comment

Changes in v2:
- Add note to the commit message about the EGL benchmark comparison not
  being fair due to Debayer algorithm diferences
- Extend the comment to mention that this works around a bug on ipu6/7
  so that if the decision is made in the future to prefer unpacked it
  will be clear to the person making such a change this might be a problem
---
 src/libcamera/pipeline/simple/simple.cpp | 24 +++++++++++++++++++++++-
 1 file changed, 23 insertions(+), 1 deletion(-)

Comments

Milan Zamazal Oct. 29, 2026, 9:42 a.m. UTC | #1
Hans de Goede <johannes.goede@oss.qualcomm.com> writes:

> Some CSI2 receivers like the IPU6 isys can store e.g. 10 bpp Bayer formats
> in either CSI2 packed (5 bytes / 4 pixels) or unpacked (2 bytes / pixel)
> formats.
>
> Before this change the simple-pipeline handler would pick whatever format
> was enumerated first as capture format in this case. Which on IPU6 boards
> comes down to picking unpacked.
>
> Packed formats use less memory and typically perform better due to reduced
> RAM bandwidth requirements. E.g. on a Lenovo ThinkPad X1 Yoga Gen 8 with
> i7-1370P CPU and a ov2740 sensor running at full sensor (FHD) resolution,
> 3 consecutive benchmark runs using libcamera's software ISP's builtin
> Debayer benchmark gives the following results:
>
> Unpacked  EGL us/frame:  6398 / 7393 / 7524
> CSI2 pack EGL us/frame:  5682 / 5536 / 5634
>
> Unpacked  CPU us/frame:  7653 / 8147 / 8614
> CSI2 pack CPU us/frame:  7183 / 7063 / 6318
>
> Note the EGL comparison is not entirely fair, the unpacked debayer shader
> uses the "McGuire" Debayer algorithm where as the packed one uses simpler
> linear filtering.
>
> Modify the simple pipeline handler to prefer the more performant CSI2
> packed format when both formats are available.
>
> This also fixes a problem with unpacked formats on IPU6 where sometimes at
> the start of a stream the IPU6 isys CSI2 receiver may return a corrupt
> unpacked frame with high bits set which causes libcamera to crash with
> an out of bound array access as the software ISP code expects the high bits
> of unpacked formats to always be 0.
>
> Closes: https://gitlab.freedesktop.org/camera/libcamera/-/work_items/328
> Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>

> ---
> Changes in v3:
> - s/support/supported/ in the comment
>
> Changes in v2:
> - Add note to the commit message about the EGL benchmark comparison not
>   being fair due to Debayer algorithm diferences
> - Extend the comment to mention that this works around a bug on ipu6/7
>   so that if the decision is made in the future to prefer unpacked it
>   will be clear to the person making such a change this might be a problem
> ---
>  src/libcamera/pipeline/simple/simple.cpp | 24 +++++++++++++++++++++++-
>  1 file changed, 23 insertions(+), 1 deletion(-)
>
> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp
> index 35c29ceca..5f7e21463 100644
> --- a/src/libcamera/pipeline/simple/simple.cpp
> +++ b/src/libcamera/pipeline/simple/simple.cpp
> @@ -1216,6 +1216,15 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
>  	 * Then pick, among the possible configuration for the pixel format,
>  	 * the smallest sensor resolution that can accommodate all streams
>  	 * without upscaling.
> +	 *
> +	 * If both packed and unpacked Bayer capture formats are supported
> +	 * prefer packed for lower memory usage and better software ISP
> +	 * performance.
> +	 *
> +	 * Note preferring packed also works around a bug on ipu6/7 setups where
> +	 * the CSI receives sometimes hands over corrupt frames. With unpacked
> +	 * formats this causes OOB array accesses in the software ISP, due to
> +	 * some of the unused high bits being 1 in these corrupt frames.
>  	 */
>  	const SimpleCameraData::Configuration *maxPipeConfig = nullptr;
>  	pipeConfig_ = nullptr;
> @@ -1223,12 +1232,25 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
>  	for (const SimpleCameraData::Configuration *pipeConfig : *configs) {
>  		const Size &captureSize = pipeConfig->captureSize;
>  		const Size &maxOutputSize = pipeConfig->outputSizes.max;
> +		bool curCfgPacked = false;
> +		bool newCfgPacked = false;
> +
> +		if (pipeConfig_) {
> +			BayerFormat curBayerFmt = BayerFormat::fromPixelFormat(pipeConfig_->captureFormat);
> +			BayerFormat newBayerFmt = BayerFormat::fromPixelFormat(pipeConfig->captureFormat);
> +			/* Only compare packed vs unpacked if both formats are Bayer */
> +			if (curBayerFmt.isValid() && newBayerFmt.isValid()) {
> +				curCfgPacked = curBayerFmt.packing == BayerFormat::Packing::CSI2;
> +				newCfgPacked = newBayerFmt.packing == BayerFormat::Packing::CSI2;
> +			}
> +		}
>  
>  		if (maxOutputSize.width >= maxProcessedStreamSize.width &&
>  		    maxOutputSize.height >= maxProcessedStreamSize.height &&
>  		    captureSize.width >= maxRawStreamSize.width &&
>  		    captureSize.height >= maxRawStreamSize.height) {
> -			if (!pipeConfig_ || captureSize < pipeConfig_->captureSize)
> +			if (!pipeConfig_ || captureSize < pipeConfig_->captureSize ||
> +			    (captureSize <= pipeConfig_->captureSize && !curCfgPacked && newCfgPacked))
>  				pipeConfig_ = pipeConfig;
>  		}
Kieran Bingham Oct. 29, 2026, 10:18 a.m. UTC | #2
Quoting Hans de Goede (2026-10-28 21:43:07)
> Some CSI2 receivers like the IPU6 isys can store e.g. 10 bpp Bayer formats
> in either CSI2 packed (5 bytes / 4 pixels) or unpacked (2 bytes / pixel)
> formats.
> 
> Before this change the simple-pipeline handler would pick whatever format
> was enumerated first as capture format in this case. Which on IPU6 boards
> comes down to picking unpacked.
> 
> Packed formats use less memory and typically perform better due to reduced
> RAM bandwidth requirements. E.g. on a Lenovo ThinkPad X1 Yoga Gen 8 with
> i7-1370P CPU and a ov2740 sensor running at full sensor (FHD) resolution,
> 3 consecutive benchmark runs using libcamera's software ISP's builtin
> Debayer benchmark gives the following results:
> 
> Unpacked  EGL us/frame:  6398 / 7393 / 7524
> CSI2 pack EGL us/frame:  5682 / 5536 / 5634
> 
> Unpacked  CPU us/frame:  7653 / 8147 / 8614
> CSI2 pack CPU us/frame:  7183 / 7063 / 6318
> 
> Note the EGL comparison is not entirely fair, the unpacked debayer shader
> uses the "McGuire" Debayer algorithm where as the packed one uses simpler
> linear filtering.

For the moment, this looks like a much better solution than 'crashing'.

Perhaps later someone might want to consider if it's worth porting the
McGuire debayer algorithm into the packed debayer shader if anyone
notices issues.

> Modify the simple pipeline handler to prefer the more performant CSI2
> packed format when both formats are available.
> 
> This also fixes a problem with unpacked formats on IPU6 where sometimes at
> the start of a stream the IPU6 isys CSI2 receiver may return a corrupt
> unpacked frame with high bits set which causes libcamera to crash with
> an out of bound array access as the software ISP code expects the high bits
> of unpacked formats to always be 0.
> 
> Closes: https://gitlab.freedesktop.org/camera/libcamera/-/work_items/328
> Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
> ---
> Changes in v3:
> - s/support/supported/ in the comment
> 
> Changes in v2:
> - Add note to the commit message about the EGL benchmark comparison not
>   being fair due to Debayer algorithm diferences
> - Extend the comment to mention that this works around a bug on ipu6/7
>   so that if the decision is made in the future to prefer unpacked it
>   will be clear to the person making such a change this might be a problem
> ---
>  src/libcamera/pipeline/simple/simple.cpp | 24 +++++++++++++++++++++++-
>  1 file changed, 23 insertions(+), 1 deletion(-)
> 
> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp
> index 35c29ceca..5f7e21463 100644
> --- a/src/libcamera/pipeline/simple/simple.cpp
> +++ b/src/libcamera/pipeline/simple/simple.cpp
> @@ -1216,6 +1216,15 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
>          * Then pick, among the possible configuration for the pixel format,
>          * the smallest sensor resolution that can accommodate all streams
>          * without upscaling.
> +        *
> +        * If both packed and unpacked Bayer capture formats are supported
> +        * prefer packed for lower memory usage and better software ISP
> +        * performance.
> +        *
> +        * Note preferring packed also works around a bug on ipu6/7 setups where
> +        * the CSI receives sometimes hands over corrupt frames. With unpacked
> +        * formats this causes OOB array accesses in the software ISP, due to
> +        * some of the unused high bits being 1 in these corrupt frames.

Ack.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
>          */
>         const SimpleCameraData::Configuration *maxPipeConfig = nullptr;
>         pipeConfig_ = nullptr;
> @@ -1223,12 +1232,25 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
>         for (const SimpleCameraData::Configuration *pipeConfig : *configs) {
>                 const Size &captureSize = pipeConfig->captureSize;
>                 const Size &maxOutputSize = pipeConfig->outputSizes.max;
> +               bool curCfgPacked = false;
> +               bool newCfgPacked = false;
> +
> +               if (pipeConfig_) {
> +                       BayerFormat curBayerFmt = BayerFormat::fromPixelFormat(pipeConfig_->captureFormat);
> +                       BayerFormat newBayerFmt = BayerFormat::fromPixelFormat(pipeConfig->captureFormat);
> +                       /* Only compare packed vs unpacked if both formats are Bayer */
> +                       if (curBayerFmt.isValid() && newBayerFmt.isValid()) {
> +                               curCfgPacked = curBayerFmt.packing == BayerFormat::Packing::CSI2;
> +                               newCfgPacked = newBayerFmt.packing == BayerFormat::Packing::CSI2;
> +                       }
> +               }
>  
>                 if (maxOutputSize.width >= maxProcessedStreamSize.width &&
>                     maxOutputSize.height >= maxProcessedStreamSize.height &&
>                     captureSize.width >= maxRawStreamSize.width &&
>                     captureSize.height >= maxRawStreamSize.height) {
> -                       if (!pipeConfig_ || captureSize < pipeConfig_->captureSize)
> +                       if (!pipeConfig_ || captureSize < pipeConfig_->captureSize ||
> +                           (captureSize <= pipeConfig_->captureSize && !curCfgPacked && newCfgPacked))
>                                 pipeConfig_ = pipeConfig;
>                 }
>  
> -- 
> 2.56.0
>
Milan Zamazal Oct. 29, 2026, 11:20 a.m. UTC | #3
Kieran Bingham <kieran.bingham@ideasonboard.com> writes:

> Quoting Hans de Goede (2026-10-28 21:43:07)
>> Some CSI2 receivers like the IPU6 isys can store e.g. 10 bpp Bayer formats
>> in either CSI2 packed (5 bytes / 4 pixels) or unpacked (2 bytes / pixel)
>
>> formats.
>> 
>> Before this change the simple-pipeline handler would pick whatever format
>> was enumerated first as capture format in this case. Which on IPU6 boards
>> comes down to picking unpacked.
>> 
>> Packed formats use less memory and typically perform better due to reduced
>> RAM bandwidth requirements. E.g. on a Lenovo ThinkPad X1 Yoga Gen 8 with
>> i7-1370P CPU and a ov2740 sensor running at full sensor (FHD) resolution,
>> 3 consecutive benchmark runs using libcamera's software ISP's builtin
>> Debayer benchmark gives the following results:
>> 
>> Unpacked  EGL us/frame:  6398 / 7393 / 7524
>> CSI2 pack EGL us/frame:  5682 / 5536 / 5634
>> 
>> Unpacked  CPU us/frame:  7653 / 8147 / 8614
>> CSI2 pack CPU us/frame:  7183 / 7063 / 6318
>> 
>> Note the EGL comparison is not entirely fair, the unpacked debayer shader
>> uses the "McGuire" Debayer algorithm where as the packed one uses simpler
>> linear filtering.
>
> For the moment, this looks like a much better solution than 'crashing'.
>
> Perhaps later someone might want to consider if it's worth porting the
> McGuire debayer algorithm into the packed debayer shader if anyone
> notices issues.

Bryan's multi-pass uses a common shader / algorithm for both (currently
bilinear IIRC); hopefully it resolves that difference sooner or later.

>
>> Modify the simple pipeline handler to prefer the more performant CSI2
>> packed format when both formats are available.
>> 
>> This also fixes a problem with unpacked formats on IPU6 where sometimes at
>> the start of a stream the IPU6 isys CSI2 receiver may return a corrupt
>> unpacked frame with high bits set which causes libcamera to crash with
>> an out of bound array access as the software ISP code expects the high bits
>> of unpacked formats to always be 0.
>> 
>> Closes: https://gitlab.freedesktop.org/camera/libcamera/-/work_items/328
>> Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
>> ---
>> Changes in v3:
>> - s/support/supported/ in the comment
>> 
>> Changes in v2:
>> - Add note to the commit message about the EGL benchmark comparison not
>>   being fair due to Debayer algorithm diferences
>> - Extend the comment to mention that this works around a bug on ipu6/7
>>   so that if the decision is made in the future to prefer unpacked it
>>   will be clear to the person making such a change this might be a problem
>> ---
>>  src/libcamera/pipeline/simple/simple.cpp | 24 +++++++++++++++++++++++-
>>  1 file changed, 23 insertions(+), 1 deletion(-)
>> 
>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp
>> index 35c29ceca..5f7e21463 100644
>> --- a/src/libcamera/pipeline/simple/simple.cpp
>> +++ b/src/libcamera/pipeline/simple/simple.cpp
>> @@ -1216,6 +1216,15 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
>>          * Then pick, among the possible configuration for the pixel format,
>>          * the smallest sensor resolution that can accommodate all streams
>>          * without upscaling.
>> +        *
>> +        * If both packed and unpacked Bayer capture formats are supported
>> +        * prefer packed for lower memory usage and better software ISP
>> +        * performance.
>> +        *
>> +        * Note preferring packed also works around a bug on ipu6/7 setups where
>> +        * the CSI receives sometimes hands over corrupt frames. With unpacked
>> +        * formats this causes OOB array accesses in the software ISP, due to
>> +        * some of the unused high bits being 1 in these corrupt frames.
>
> Ack.
>
> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
>>          */
>>         const SimpleCameraData::Configuration *maxPipeConfig = nullptr;
>>         pipeConfig_ = nullptr;
>> @@ -1223,12 +1232,25 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
>>         for (const SimpleCameraData::Configuration *pipeConfig : *configs) {
>>                 const Size &captureSize = pipeConfig->captureSize;
>>                 const Size &maxOutputSize = pipeConfig->outputSizes.max;
>> +               bool curCfgPacked = false;
>> +               bool newCfgPacked = false;
>> +
>> +               if (pipeConfig_) {
>> +                       BayerFormat curBayerFmt = BayerFormat::fromPixelFormat(pipeConfig_->captureFormat);
>> +                       BayerFormat newBayerFmt = BayerFormat::fromPixelFormat(pipeConfig->captureFormat);
>> +                       /* Only compare packed vs unpacked if both formats are Bayer */
>> +                       if (curBayerFmt.isValid() && newBayerFmt.isValid()) {
>> +                               curCfgPacked = curBayerFmt.packing == BayerFormat::Packing::CSI2;
>> +                               newCfgPacked = newBayerFmt.packing == BayerFormat::Packing::CSI2;
>> +                       }
>> +               }
>>  
>>                 if (maxOutputSize.width >= maxProcessedStreamSize.width &&
>>                     maxOutputSize.height >= maxProcessedStreamSize.height &&
>>                     captureSize.width >= maxRawStreamSize.width &&
>>                     captureSize.height >= maxRawStreamSize.height) {
>> -                       if (!pipeConfig_ || captureSize < pipeConfig_->captureSize)
>> +                       if (!pipeConfig_ || captureSize < pipeConfig_->captureSize ||
>> +                           (captureSize <= pipeConfig_->captureSize && !curCfgPacked && newCfgPacked))
>>                                 pipeConfig_ = pipeConfig;
>>                 }
>>  
>> -- 
>> 2.56.0
>>
Barnabás Pőcze Nov. 2, 2026, 9:44 a.m. UTC | #4
It is with great sadness that I have to report that this makes the camera unusable
on a thinkpad x1 yoga gen 7 with ipu6 and ov2740. It is as if the camera is
oscillating between present and past frames all the time. It is a cool effect,
but I'm afraid people will complain.

This happens in cam (sdl renderer), qcam, and firefox (via pipewire) as well.

Kernel log: continuous stream of

   ipu6_isys_csi2_error: 235 callbacks suppressed
   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded


2026. 10. 28. 22:43 keltezéssel, Hans de Goede írta:
> Some CSI2 receivers like the IPU6 isys can store e.g. 10 bpp Bayer formats
> in either CSI2 packed (5 bytes / 4 pixels) or unpacked (2 bytes / pixel)
> formats.
> 
> Before this change the simple-pipeline handler would pick whatever format
> was enumerated first as capture format in this case. Which on IPU6 boards
> comes down to picking unpacked.
> 
> Packed formats use less memory and typically perform better due to reduced
> RAM bandwidth requirements. E.g. on a Lenovo ThinkPad X1 Yoga Gen 8 with
> i7-1370P CPU and a ov2740 sensor running at full sensor (FHD) resolution,
> 3 consecutive benchmark runs using libcamera's software ISP's builtin
> Debayer benchmark gives the following results:
> 
> Unpacked  EGL us/frame:  6398 / 7393 / 7524
> CSI2 pack EGL us/frame:  5682 / 5536 / 5634
> 
> Unpacked  CPU us/frame:  7653 / 8147 / 8614
> CSI2 pack CPU us/frame:  7183 / 7063 / 6318
> 
> Note the EGL comparison is not entirely fair, the unpacked debayer shader
> uses the "McGuire" Debayer algorithm where as the packed one uses simpler
> linear filtering.
> 
> Modify the simple pipeline handler to prefer the more performant CSI2
> packed format when both formats are available.
> 
> This also fixes a problem with unpacked formats on IPU6 where sometimes at
> the start of a stream the IPU6 isys CSI2 receiver may return a corrupt
> unpacked frame with high bits set which causes libcamera to crash with
> an out of bound array access as the software ISP code expects the high bits
> of unpacked formats to always be 0.
> 
> Closes: https://gitlab.freedesktop.org/camera/libcamera/-/work_items/328
> Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
> ---
> Changes in v3:
> - s/support/supported/ in the comment
> 
> Changes in v2:
> - Add note to the commit message about the EGL benchmark comparison not
>    being fair due to Debayer algorithm diferences
> - Extend the comment to mention that this works around a bug on ipu6/7
>    so that if the decision is made in the future to prefer unpacked it
>    will be clear to the person making such a change this might be a problem
> ---
>   src/libcamera/pipeline/simple/simple.cpp | 24 +++++++++++++++++++++++-
>   1 file changed, 23 insertions(+), 1 deletion(-)
> 
> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp
> index 35c29ceca..5f7e21463 100644
> --- a/src/libcamera/pipeline/simple/simple.cpp
> +++ b/src/libcamera/pipeline/simple/simple.cpp
> @@ -1216,6 +1216,15 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
>   	 * Then pick, among the possible configuration for the pixel format,
>   	 * the smallest sensor resolution that can accommodate all streams
>   	 * without upscaling.
> +	 *
> +	 * If both packed and unpacked Bayer capture formats are supported
> +	 * prefer packed for lower memory usage and better software ISP
> +	 * performance.
> +	 *
> +	 * Note preferring packed also works around a bug on ipu6/7 setups where
> +	 * the CSI receives sometimes hands over corrupt frames. With unpacked
> +	 * formats this causes OOB array accesses in the software ISP, due to
> +	 * some of the unused high bits being 1 in these corrupt frames.
>   	 */
>   	const SimpleCameraData::Configuration *maxPipeConfig = nullptr;
>   	pipeConfig_ = nullptr;
> @@ -1223,12 +1232,25 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
>   	for (const SimpleCameraData::Configuration *pipeConfig : *configs) {
>   		const Size &captureSize = pipeConfig->captureSize;
>   		const Size &maxOutputSize = pipeConfig->outputSizes.max;
> +		bool curCfgPacked = false;
> +		bool newCfgPacked = false;
> +
> +		if (pipeConfig_) {
> +			BayerFormat curBayerFmt = BayerFormat::fromPixelFormat(pipeConfig_->captureFormat);
> +			BayerFormat newBayerFmt = BayerFormat::fromPixelFormat(pipeConfig->captureFormat);
> +			/* Only compare packed vs unpacked if both formats are Bayer */
> +			if (curBayerFmt.isValid() && newBayerFmt.isValid()) {
> +				curCfgPacked = curBayerFmt.packing == BayerFormat::Packing::CSI2;
> +				newCfgPacked = newBayerFmt.packing == BayerFormat::Packing::CSI2;
> +			}
> +		}
>   
>   		if (maxOutputSize.width >= maxProcessedStreamSize.width &&
>   		    maxOutputSize.height >= maxProcessedStreamSize.height &&
>   		    captureSize.width >= maxRawStreamSize.width &&
>   		    captureSize.height >= maxRawStreamSize.height) {
> -			if (!pipeConfig_ || captureSize < pipeConfig_->captureSize)
> +			if (!pipeConfig_ || captureSize < pipeConfig_->captureSize ||
> +			    (captureSize <= pipeConfig_->captureSize && !curCfgPacked && newCfgPacked))
>   				pipeConfig_ = pipeConfig;
>   		}
>
Hans de Goede Nov. 2, 2026, 1:44 p.m. UTC | #5
Hi,

On 2-Nov-26 10:44 AM, Barnabás Pőcze wrote:
> It is with great sadness that I have to report that this makes the camera unusable
> on a thinkpad x1 yoga gen 7 with ipu6 and ov2740. It is as if the camera is
> oscillating between present and past frames all the time. It is a cool effect,
> but I'm afraid people will complain.
> 
> This happens in cam (sdl renderer), qcam, and firefox (via pipewire) as well.
> 
> Kernel log: continuous stream of
> 
>   ipu6_isys_csi2_error: 235 callbacks suppressed
>   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
>   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
>   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
>   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
>   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
>   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
>   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
>   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
>   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
>   intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded

I think this might be causes by the ringbus of the CPU being clocked too low.

What happens if you do:

md5sum /dev/zero&

say 4 or 8 times to cause some load and cause the CPU (and ringbus) to clock up?

I've seen these errors before and then it was a too slow ringbus, but we never
got around to fixing this. The IPU6/7 driver will need to make some Qos request
to the PMU or something to tell it the CSI2 receiver needs more bandwidth on
the ringbus.  Assuming my guess is that this only happens on an idle system is
correct ...

Regards,

Hans



> 
> 
> 2026. 10. 28. 22:43 keltezéssel, Hans de Goede írta:
>> Some CSI2 receivers like the IPU6 isys can store e.g. 10 bpp Bayer formats
>> in either CSI2 packed (5 bytes / 4 pixels) or unpacked (2 bytes / pixel)
>> formats.
>>
>> Before this change the simple-pipeline handler would pick whatever format
>> was enumerated first as capture format in this case. Which on IPU6 boards
>> comes down to picking unpacked.
>>
>> Packed formats use less memory and typically perform better due to reduced
>> RAM bandwidth requirements. E.g. on a Lenovo ThinkPad X1 Yoga Gen 8 with
>> i7-1370P CPU and a ov2740 sensor running at full sensor (FHD) resolution,
>> 3 consecutive benchmark runs using libcamera's software ISP's builtin
>> Debayer benchmark gives the following results:
>>
>> Unpacked  EGL us/frame:  6398 / 7393 / 7524
>> CSI2 pack EGL us/frame:  5682 / 5536 / 5634
>>
>> Unpacked  CPU us/frame:  7653 / 8147 / 8614
>> CSI2 pack CPU us/frame:  7183 / 7063 / 6318
>>
>> Note the EGL comparison is not entirely fair, the unpacked debayer shader
>> uses the "McGuire" Debayer algorithm where as the packed one uses simpler
>> linear filtering.
>>
>> Modify the simple pipeline handler to prefer the more performant CSI2
>> packed format when both formats are available.
>>
>> This also fixes a problem with unpacked formats on IPU6 where sometimes at
>> the start of a stream the IPU6 isys CSI2 receiver may return a corrupt
>> unpacked frame with high bits set which causes libcamera to crash with
>> an out of bound array access as the software ISP code expects the high bits
>> of unpacked formats to always be 0.
>>
>> Closes: https://gitlab.freedesktop.org/camera/libcamera/-/work_items/328
>> Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
>> ---
>> Changes in v3:
>> - s/support/supported/ in the comment
>>
>> Changes in v2:
>> - Add note to the commit message about the EGL benchmark comparison not
>>    being fair due to Debayer algorithm diferences
>> - Extend the comment to mention that this works around a bug on ipu6/7
>>    so that if the decision is made in the future to prefer unpacked it
>>    will be clear to the person making such a change this might be a problem
>> ---
>>   src/libcamera/pipeline/simple/simple.cpp | 24 +++++++++++++++++++++++-
>>   1 file changed, 23 insertions(+), 1 deletion(-)
>>
>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp
>> index 35c29ceca..5f7e21463 100644
>> --- a/src/libcamera/pipeline/simple/simple.cpp
>> +++ b/src/libcamera/pipeline/simple/simple.cpp
>> @@ -1216,6 +1216,15 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
>>        * Then pick, among the possible configuration for the pixel format,
>>        * the smallest sensor resolution that can accommodate all streams
>>        * without upscaling.
>> +     *
>> +     * If both packed and unpacked Bayer capture formats are supported
>> +     * prefer packed for lower memory usage and better software ISP
>> +     * performance.
>> +     *
>> +     * Note preferring packed also works around a bug on ipu6/7 setups where
>> +     * the CSI receives sometimes hands over corrupt frames. With unpacked
>> +     * formats this causes OOB array accesses in the software ISP, due to
>> +     * some of the unused high bits being 1 in these corrupt frames.
>>        */
>>       const SimpleCameraData::Configuration *maxPipeConfig = nullptr;
>>       pipeConfig_ = nullptr;
>> @@ -1223,12 +1232,25 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
>>       for (const SimpleCameraData::Configuration *pipeConfig : *configs) {
>>           const Size &captureSize = pipeConfig->captureSize;
>>           const Size &maxOutputSize = pipeConfig->outputSizes.max;
>> +        bool curCfgPacked = false;
>> +        bool newCfgPacked = false;
>> +
>> +        if (pipeConfig_) {
>> +            BayerFormat curBayerFmt = BayerFormat::fromPixelFormat(pipeConfig_->captureFormat);
>> +            BayerFormat newBayerFmt = BayerFormat::fromPixelFormat(pipeConfig->captureFormat);
>> +            /* Only compare packed vs unpacked if both formats are Bayer */
>> +            if (curBayerFmt.isValid() && newBayerFmt.isValid()) {
>> +                curCfgPacked = curBayerFmt.packing == BayerFormat::Packing::CSI2;
>> +                newCfgPacked = newBayerFmt.packing == BayerFormat::Packing::CSI2;
>> +            }
>> +        }
>>             if (maxOutputSize.width >= maxProcessedStreamSize.width &&
>>               maxOutputSize.height >= maxProcessedStreamSize.height &&
>>               captureSize.width >= maxRawStreamSize.width &&
>>               captureSize.height >= maxRawStreamSize.height) {
>> -            if (!pipeConfig_ || captureSize < pipeConfig_->captureSize)
>> +            if (!pipeConfig_ || captureSize < pipeConfig_->captureSize ||
>> +                (captureSize <= pipeConfig_->captureSize && !curCfgPacked && newCfgPacked))
>>                   pipeConfig_ = pipeConfig;
>>           }
>>   
>
Barnabás Pőcze Nov. 2, 2026, 1:52 p.m. UTC | #6
2026. 11. 02. 14:44 keltezéssel, Hans de Goede írta:
> Hi,
> 
> On 2-Nov-26 10:44 AM, Barnabás Pőcze wrote:
>> It is with great sadness that I have to report that this makes the camera unusable
>> on a thinkpad x1 yoga gen 7 with ipu6 and ov2740. It is as if the camera is
>> oscillating between present and past frames all the time. It is a cool effect,
>> but I'm afraid people will complain.
>>
>> This happens in cam (sdl renderer), qcam, and firefox (via pipewire) as well.
>>
>> Kernel log: continuous stream of
>>
>>    ipu6_isys_csi2_error: 235 callbacks suppressed
>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
> 
> I think this might be causes by the ringbus of the CPU being clocked too low.
> 
> What happens if you do:
> 
> md5sum /dev/zero&
> 
> say 4 or 8 times to cause some load and cause the CPU (and ringbus) to clock up?
> 
> I've seen these errors before and then it was a too slow ringbus, but we never
> got around to fixing this. The IPU6/7 driver will need to make some Qos request
> to the PMU or something to tell it the CSI2 receiver needs more bandwidth on
> the ringbus.  Assuming my guess is that this only happens on an idle system is
> correct ...

I started `stress -c ...` in parallel to the streaming, and it appears fix the issue.


> 
> Regards,
> 
> Hans
> 
> 
> 
>>
>>
>> 2026. 10. 28. 22:43 keltezéssel, Hans de Goede írta:
>>> Some CSI2 receivers like the IPU6 isys can store e.g. 10 bpp Bayer formats
>>> in either CSI2 packed (5 bytes / 4 pixels) or unpacked (2 bytes / pixel)
>>> formats.
>>>
>>> Before this change the simple-pipeline handler would pick whatever format
>>> was enumerated first as capture format in this case. Which on IPU6 boards
>>> comes down to picking unpacked.
>>>
>>> Packed formats use less memory and typically perform better due to reduced
>>> RAM bandwidth requirements. E.g. on a Lenovo ThinkPad X1 Yoga Gen 8 with
>>> i7-1370P CPU and a ov2740 sensor running at full sensor (FHD) resolution,
>>> 3 consecutive benchmark runs using libcamera's software ISP's builtin
>>> Debayer benchmark gives the following results:
>>>
>>> Unpacked  EGL us/frame:  6398 / 7393 / 7524
>>> CSI2 pack EGL us/frame:  5682 / 5536 / 5634
>>>
>>> Unpacked  CPU us/frame:  7653 / 8147 / 8614
>>> CSI2 pack CPU us/frame:  7183 / 7063 / 6318
>>>
>>> Note the EGL comparison is not entirely fair, the unpacked debayer shader
>>> uses the "McGuire" Debayer algorithm where as the packed one uses simpler
>>> linear filtering.
>>>
>>> Modify the simple pipeline handler to prefer the more performant CSI2
>>> packed format when both formats are available.
>>>
>>> This also fixes a problem with unpacked formats on IPU6 where sometimes at
>>> the start of a stream the IPU6 isys CSI2 receiver may return a corrupt
>>> unpacked frame with high bits set which causes libcamera to crash with
>>> an out of bound array access as the software ISP code expects the high bits
>>> of unpacked formats to always be 0.
>>>
>>> Closes: https://gitlab.freedesktop.org/camera/libcamera/-/work_items/328
>>> Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
>>> ---
>>> Changes in v3:
>>> - s/support/supported/ in the comment
>>>
>>> Changes in v2:
>>> - Add note to the commit message about the EGL benchmark comparison not
>>>     being fair due to Debayer algorithm diferences
>>> - Extend the comment to mention that this works around a bug on ipu6/7
>>>     so that if the decision is made in the future to prefer unpacked it
>>>     will be clear to the person making such a change this might be a problem
>>> ---
>>>    src/libcamera/pipeline/simple/simple.cpp | 24 +++++++++++++++++++++++-
>>>    1 file changed, 23 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp
>>> index 35c29ceca..5f7e21463 100644
>>> --- a/src/libcamera/pipeline/simple/simple.cpp
>>> +++ b/src/libcamera/pipeline/simple/simple.cpp
>>> @@ -1216,6 +1216,15 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
>>>         * Then pick, among the possible configuration for the pixel format,
>>>         * the smallest sensor resolution that can accommodate all streams
>>>         * without upscaling.
>>> +     *
>>> +     * If both packed and unpacked Bayer capture formats are supported
>>> +     * prefer packed for lower memory usage and better software ISP
>>> +     * performance.
>>> +     *
>>> +     * Note preferring packed also works around a bug on ipu6/7 setups where
>>> +     * the CSI receives sometimes hands over corrupt frames. With unpacked
>>> +     * formats this causes OOB array accesses in the software ISP, due to
>>> +     * some of the unused high bits being 1 in these corrupt frames.
>>>         */
>>>        const SimpleCameraData::Configuration *maxPipeConfig = nullptr;
>>>        pipeConfig_ = nullptr;
>>> @@ -1223,12 +1232,25 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
>>>        for (const SimpleCameraData::Configuration *pipeConfig : *configs) {
>>>            const Size &captureSize = pipeConfig->captureSize;
>>>            const Size &maxOutputSize = pipeConfig->outputSizes.max;
>>> +        bool curCfgPacked = false;
>>> +        bool newCfgPacked = false;
>>> +
>>> +        if (pipeConfig_) {
>>> +            BayerFormat curBayerFmt = BayerFormat::fromPixelFormat(pipeConfig_->captureFormat);
>>> +            BayerFormat newBayerFmt = BayerFormat::fromPixelFormat(pipeConfig->captureFormat);
>>> +            /* Only compare packed vs unpacked if both formats are Bayer */
>>> +            if (curBayerFmt.isValid() && newBayerFmt.isValid()) {
>>> +                curCfgPacked = curBayerFmt.packing == BayerFormat::Packing::CSI2;
>>> +                newCfgPacked = newBayerFmt.packing == BayerFormat::Packing::CSI2;
>>> +            }
>>> +        }
>>>              if (maxOutputSize.width >= maxProcessedStreamSize.width &&
>>>                maxOutputSize.height >= maxProcessedStreamSize.height &&
>>>                captureSize.width >= maxRawStreamSize.width &&
>>>                captureSize.height >= maxRawStreamSize.height) {
>>> -            if (!pipeConfig_ || captureSize < pipeConfig_->captureSize)
>>> +            if (!pipeConfig_ || captureSize < pipeConfig_->captureSize ||
>>> +                (captureSize <= pipeConfig_->captureSize && !curCfgPacked && newCfgPacked))
>>>                    pipeConfig_ = pipeConfig;
>>>            }
>>>    
>>
>
Hans de Goede Nov. 2, 2026, 3:08 p.m. UTC | #7
Hi,

On 2-Nov-26 2:52 PM, Barnabás Pőcze wrote:
> 2026. 11. 02. 14:44 keltezéssel, Hans de Goede írta:
>> Hi,
>>
>> On 2-Nov-26 10:44 AM, Barnabás Pőcze wrote:
>>> It is with great sadness that I have to report that this makes the camera unusable
>>> on a thinkpad x1 yoga gen 7 with ipu6 and ov2740. It is as if the camera is
>>> oscillating between present and past frames all the time. It is a cool effect,
>>> but I'm afraid people will complain.
>>>
>>> This happens in cam (sdl renderer), qcam, and firefox (via pipewire) as well.
>>>
>>> Kernel log: continuous stream of
>>>
>>>    ipu6_isys_csi2_error: 235 callbacks suppressed
>>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
>>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
>>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
>>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
>>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
>>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
>>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
>>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
>>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
>>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
>>
>> I think this might be causes by the ringbus of the CPU being clocked too low.
>>
>> What happens if you do:
>>
>> md5sum /dev/zero&
>>
>> say 4 or 8 times to cause some load and cause the CPU (and ringbus) to clock up?
>>
>> I've seen these errors before and then it was a too slow ringbus, but we never
>> got around to fixing this. The IPU6/7 driver will need to make some Qos request
>> to the PMU or something to tell it the CSI2 receiver needs more bandwidth on
>> the ringbus.  Assuming my guess is that this only happens on an idle system is
>> correct ...
> 
> I started `stress -c ...` in parallel to the streaming, and it appears fix the issue.

Yes, so this is the ringbus clocking too low to handle the raw sensor bandwidth
and the IPU6 isys driver not doing whatever magic it needs to do to request more
bandwidth on the ringbus.

Regards,

Hans



>>
>>
>>
>>>
>>>
>>> 2026. 10. 28. 22:43 keltezéssel, Hans de Goede írta:
>>>> Some CSI2 receivers like the IPU6 isys can store e.g. 10 bpp Bayer formats
>>>> in either CSI2 packed (5 bytes / 4 pixels) or unpacked (2 bytes / pixel)
>>>> formats.
>>>>
>>>> Before this change the simple-pipeline handler would pick whatever format
>>>> was enumerated first as capture format in this case. Which on IPU6 boards
>>>> comes down to picking unpacked.
>>>>
>>>> Packed formats use less memory and typically perform better due to reduced
>>>> RAM bandwidth requirements. E.g. on a Lenovo ThinkPad X1 Yoga Gen 8 with
>>>> i7-1370P CPU and a ov2740 sensor running at full sensor (FHD) resolution,
>>>> 3 consecutive benchmark runs using libcamera's software ISP's builtin
>>>> Debayer benchmark gives the following results:
>>>>
>>>> Unpacked  EGL us/frame:  6398 / 7393 / 7524
>>>> CSI2 pack EGL us/frame:  5682 / 5536 / 5634
>>>>
>>>> Unpacked  CPU us/frame:  7653 / 8147 / 8614
>>>> CSI2 pack CPU us/frame:  7183 / 7063 / 6318
>>>>
>>>> Note the EGL comparison is not entirely fair, the unpacked debayer shader
>>>> uses the "McGuire" Debayer algorithm where as the packed one uses simpler
>>>> linear filtering.
>>>>
>>>> Modify the simple pipeline handler to prefer the more performant CSI2
>>>> packed format when both formats are available.
>>>>
>>>> This also fixes a problem with unpacked formats on IPU6 where sometimes at
>>>> the start of a stream the IPU6 isys CSI2 receiver may return a corrupt
>>>> unpacked frame with high bits set which causes libcamera to crash with
>>>> an out of bound array access as the software ISP code expects the high bits
>>>> of unpacked formats to always be 0.
>>>>
>>>> Closes: https://gitlab.freedesktop.org/camera/libcamera/-/work_items/328
>>>> Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
>>>> ---
>>>> Changes in v3:
>>>> - s/support/supported/ in the comment
>>>>
>>>> Changes in v2:
>>>> - Add note to the commit message about the EGL benchmark comparison not
>>>>     being fair due to Debayer algorithm diferences
>>>> - Extend the comment to mention that this works around a bug on ipu6/7
>>>>     so that if the decision is made in the future to prefer unpacked it
>>>>     will be clear to the person making such a change this might be a problem
>>>> ---
>>>>    src/libcamera/pipeline/simple/simple.cpp | 24 +++++++++++++++++++++++-
>>>>    1 file changed, 23 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp
>>>> index 35c29ceca..5f7e21463 100644
>>>> --- a/src/libcamera/pipeline/simple/simple.cpp
>>>> +++ b/src/libcamera/pipeline/simple/simple.cpp
>>>> @@ -1216,6 +1216,15 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
>>>>         * Then pick, among the possible configuration for the pixel format,
>>>>         * the smallest sensor resolution that can accommodate all streams
>>>>         * without upscaling.
>>>> +     *
>>>> +     * If both packed and unpacked Bayer capture formats are supported
>>>> +     * prefer packed for lower memory usage and better software ISP
>>>> +     * performance.
>>>> +     *
>>>> +     * Note preferring packed also works around a bug on ipu6/7 setups where
>>>> +     * the CSI receives sometimes hands over corrupt frames. With unpacked
>>>> +     * formats this causes OOB array accesses in the software ISP, due to
>>>> +     * some of the unused high bits being 1 in these corrupt frames.
>>>>         */
>>>>        const SimpleCameraData::Configuration *maxPipeConfig = nullptr;
>>>>        pipeConfig_ = nullptr;
>>>> @@ -1223,12 +1232,25 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
>>>>        for (const SimpleCameraData::Configuration *pipeConfig : *configs) {
>>>>            const Size &captureSize = pipeConfig->captureSize;
>>>>            const Size &maxOutputSize = pipeConfig->outputSizes.max;
>>>> +        bool curCfgPacked = false;
>>>> +        bool newCfgPacked = false;
>>>> +
>>>> +        if (pipeConfig_) {
>>>> +            BayerFormat curBayerFmt = BayerFormat::fromPixelFormat(pipeConfig_->captureFormat);
>>>> +            BayerFormat newBayerFmt = BayerFormat::fromPixelFormat(pipeConfig->captureFormat);
>>>> +            /* Only compare packed vs unpacked if both formats are Bayer */
>>>> +            if (curBayerFmt.isValid() && newBayerFmt.isValid()) {
>>>> +                curCfgPacked = curBayerFmt.packing == BayerFormat::Packing::CSI2;
>>>> +                newCfgPacked = newBayerFmt.packing == BayerFormat::Packing::CSI2;
>>>> +            }
>>>> +        }
>>>>              if (maxOutputSize.width >= maxProcessedStreamSize.width &&
>>>>                maxOutputSize.height >= maxProcessedStreamSize.height &&
>>>>                captureSize.width >= maxRawStreamSize.width &&
>>>>                captureSize.height >= maxRawStreamSize.height) {
>>>> -            if (!pipeConfig_ || captureSize < pipeConfig_->captureSize)
>>>> +            if (!pipeConfig_ || captureSize < pipeConfig_->captureSize ||
>>>> +                (captureSize <= pipeConfig_->captureSize && !curCfgPacked && newCfgPacked))
>>>>                    pipeConfig_ = pipeConfig;
>>>>            }
>>>>    
>>>
>>
>
Kieran Bingham Nov. 2, 2026, 3:16 p.m. UTC | #8
Quoting Hans de Goede (2026-11-02 15:08:47)
> Hi,
> 
> On 2-Nov-26 2:52 PM, Barnabás Pőcze wrote:
> > 2026. 11. 02. 14:44 keltezéssel, Hans de Goede írta:
> >> Hi,
> >>
> >> On 2-Nov-26 10:44 AM, Barnabás Pőcze wrote:
> >>> It is with great sadness that I have to report that this makes the camera unusable
> >>> on a thinkpad x1 yoga gen 7 with ipu6 and ov2740. It is as if the camera is
> >>> oscillating between present and past frames all the time. It is a cool effect,
> >>> but I'm afraid people will complain.
> >>>
> >>> This happens in cam (sdl renderer), qcam, and firefox (via pipewire) as well.
> >>>
> >>> Kernel log: continuous stream of
> >>>
> >>>    ipu6_isys_csi2_error: 235 callbacks suppressed
> >>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
> >>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
> >>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
> >>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
> >>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
> >>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Transfer FIFO overflow
> >>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
> >>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
> >>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame short packet discarded
> >>>    intel_ipu6_isys.isys intel_ipu6.isys.40: csi2-1 error: Inter-frame long packet discarded
> >>
> >> I think this might be causes by the ringbus of the CPU being clocked too low.
> >>
> >> What happens if you do:
> >>
> >> md5sum /dev/zero&
> >>
> >> say 4 or 8 times to cause some load and cause the CPU (and ringbus) to clock up?
> >>
> >> I've seen these errors before and then it was a too slow ringbus, but we never
> >> got around to fixing this. The IPU6/7 driver will need to make some Qos request
> >> to the PMU or something to tell it the CSI2 receiver needs more bandwidth on
> >> the ringbus.  Assuming my guess is that this only happens on an idle system is
> >> correct ...
> > 
> > I started `stress -c ...` in parallel to the streaming, and it appears fix the issue.
> 
> Yes, so this is the ringbus clocking too low to handle the raw sensor bandwidth
> and the IPU6 isys driver not doing whatever magic it needs to do to request more
> bandwidth on the ringbus.

Can I infer we call this a kernel bug and I don't need to consider a
libcamera-0.8.1 urgent release in the next couple of days ?

What's the next step on this issue ? Are there other devices impacted by
this ?

--
Kieran

> 
> Regards,
> 
> Hans
> 
> 
> 
> >>
> >>
> >>
> >>>
> >>>
> >>> 2026. 10. 28. 22:43 keltezéssel, Hans de Goede írta:
> >>>> Some CSI2 receivers like the IPU6 isys can store e.g. 10 bpp Bayer formats
> >>>> in either CSI2 packed (5 bytes / 4 pixels) or unpacked (2 bytes / pixel)
> >>>> formats.
> >>>>
> >>>> Before this change the simple-pipeline handler would pick whatever format
> >>>> was enumerated first as capture format in this case. Which on IPU6 boards
> >>>> comes down to picking unpacked.
> >>>>
> >>>> Packed formats use less memory and typically perform better due to reduced
> >>>> RAM bandwidth requirements. E.g. on a Lenovo ThinkPad X1 Yoga Gen 8 with
> >>>> i7-1370P CPU and a ov2740 sensor running at full sensor (FHD) resolution,
> >>>> 3 consecutive benchmark runs using libcamera's software ISP's builtin
> >>>> Debayer benchmark gives the following results:
> >>>>
> >>>> Unpacked  EGL us/frame:  6398 / 7393 / 7524
> >>>> CSI2 pack EGL us/frame:  5682 / 5536 / 5634
> >>>>
> >>>> Unpacked  CPU us/frame:  7653 / 8147 / 8614
> >>>> CSI2 pack CPU us/frame:  7183 / 7063 / 6318
> >>>>
> >>>> Note the EGL comparison is not entirely fair, the unpacked debayer shader
> >>>> uses the "McGuire" Debayer algorithm where as the packed one uses simpler
> >>>> linear filtering.
> >>>>
> >>>> Modify the simple pipeline handler to prefer the more performant CSI2
> >>>> packed format when both formats are available.
> >>>>
> >>>> This also fixes a problem with unpacked formats on IPU6 where sometimes at
> >>>> the start of a stream the IPU6 isys CSI2 receiver may return a corrupt
> >>>> unpacked frame with high bits set which causes libcamera to crash with
> >>>> an out of bound array access as the software ISP code expects the high bits
> >>>> of unpacked formats to always be 0.
> >>>>
> >>>> Closes: https://gitlab.freedesktop.org/camera/libcamera/-/work_items/328
> >>>> Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
> >>>> ---
> >>>> Changes in v3:
> >>>> - s/support/supported/ in the comment
> >>>>
> >>>> Changes in v2:
> >>>> - Add note to the commit message about the EGL benchmark comparison not
> >>>>     being fair due to Debayer algorithm diferences
> >>>> - Extend the comment to mention that this works around a bug on ipu6/7
> >>>>     so that if the decision is made in the future to prefer unpacked it
> >>>>     will be clear to the person making such a change this might be a problem
> >>>> ---
> >>>>    src/libcamera/pipeline/simple/simple.cpp | 24 +++++++++++++++++++++++-
> >>>>    1 file changed, 23 insertions(+), 1 deletion(-)
> >>>>
> >>>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp
> >>>> index 35c29ceca..5f7e21463 100644
> >>>> --- a/src/libcamera/pipeline/simple/simple.cpp
> >>>> +++ b/src/libcamera/pipeline/simple/simple.cpp
> >>>> @@ -1216,6 +1216,15 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
> >>>>         * Then pick, among the possible configuration for the pixel format,
> >>>>         * the smallest sensor resolution that can accommodate all streams
> >>>>         * without upscaling.
> >>>> +     *
> >>>> +     * If both packed and unpacked Bayer capture formats are supported
> >>>> +     * prefer packed for lower memory usage and better software ISP
> >>>> +     * performance.
> >>>> +     *
> >>>> +     * Note preferring packed also works around a bug on ipu6/7 setups where
> >>>> +     * the CSI receives sometimes hands over corrupt frames. With unpacked
> >>>> +     * formats this causes OOB array accesses in the software ISP, due to
> >>>> +     * some of the unused high bits being 1 in these corrupt frames.
> >>>>         */
> >>>>        const SimpleCameraData::Configuration *maxPipeConfig = nullptr;
> >>>>        pipeConfig_ = nullptr;
> >>>> @@ -1223,12 +1232,25 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()
> >>>>        for (const SimpleCameraData::Configuration *pipeConfig : *configs) {
> >>>>            const Size &captureSize = pipeConfig->captureSize;
> >>>>            const Size &maxOutputSize = pipeConfig->outputSizes.max;
> >>>> +        bool curCfgPacked = false;
> >>>> +        bool newCfgPacked = false;
> >>>> +
> >>>> +        if (pipeConfig_) {
> >>>> +            BayerFormat curBayerFmt = BayerFormat::fromPixelFormat(pipeConfig_->captureFormat);
> >>>> +            BayerFormat newBayerFmt = BayerFormat::fromPixelFormat(pipeConfig->captureFormat);
> >>>> +            /* Only compare packed vs unpacked if both formats are Bayer */
> >>>> +            if (curBayerFmt.isValid() && newBayerFmt.isValid()) {
> >>>> +                curCfgPacked = curBayerFmt.packing == BayerFormat::Packing::CSI2;
> >>>> +                newCfgPacked = newBayerFmt.packing == BayerFormat::Packing::CSI2;
> >>>> +            }
> >>>> +        }
> >>>>              if (maxOutputSize.width >= maxProcessedStreamSize.width &&
> >>>>                maxOutputSize.height >= maxProcessedStreamSize.height &&
> >>>>                captureSize.width >= maxRawStreamSize.width &&
> >>>>                captureSize.height >= maxRawStreamSize.height) {
> >>>> -            if (!pipeConfig_ || captureSize < pipeConfig_->captureSize)
> >>>> +            if (!pipeConfig_ || captureSize < pipeConfig_->captureSize ||
> >>>> +                (captureSize <= pipeConfig_->captureSize && !curCfgPacked && newCfgPacked))
> >>>>                    pipeConfig_ = pipeConfig;
> >>>>            }
> >>>>    
> >>>
> >>
> > 
>

Patch
diff mbox series

diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp
index 35c29ceca..5f7e21463 100644
--- a/src/libcamera/pipeline/simple/simple.cpp
+++ b/src/libcamera/pipeline/simple/simple.cpp
@@ -1216,6 +1216,15 @@  CameraConfiguration::Status SimpleCameraConfiguration::validate()
 	 * Then pick, among the possible configuration for the pixel format,
 	 * the smallest sensor resolution that can accommodate all streams
 	 * without upscaling.
+	 *
+	 * If both packed and unpacked Bayer capture formats are supported
+	 * prefer packed for lower memory usage and better software ISP
+	 * performance.
+	 *
+	 * Note preferring packed also works around a bug on ipu6/7 setups where
+	 * the CSI receives sometimes hands over corrupt frames. With unpacked
+	 * formats this causes OOB array accesses in the software ISP, due to
+	 * some of the unused high bits being 1 in these corrupt frames.
 	 */
 	const SimpleCameraData::Configuration *maxPipeConfig = nullptr;
 	pipeConfig_ = nullptr;
@@ -1223,12 +1232,25 @@  CameraConfiguration::Status SimpleCameraConfiguration::validate()
 	for (const SimpleCameraData::Configuration *pipeConfig : *configs) {
 		const Size &captureSize = pipeConfig->captureSize;
 		const Size &maxOutputSize = pipeConfig->outputSizes.max;
+		bool curCfgPacked = false;
+		bool newCfgPacked = false;
+
+		if (pipeConfig_) {
+			BayerFormat curBayerFmt = BayerFormat::fromPixelFormat(pipeConfig_->captureFormat);
+			BayerFormat newBayerFmt = BayerFormat::fromPixelFormat(pipeConfig->captureFormat);
+			/* Only compare packed vs unpacked if both formats are Bayer */
+			if (curBayerFmt.isValid() && newBayerFmt.isValid()) {
+				curCfgPacked = curBayerFmt.packing == BayerFormat::Packing::CSI2;
+				newCfgPacked = newBayerFmt.packing == BayerFormat::Packing::CSI2;
+			}
+		}
 
 		if (maxOutputSize.width >= maxProcessedStreamSize.width &&
 		    maxOutputSize.height >= maxProcessedStreamSize.height &&
 		    captureSize.width >= maxRawStreamSize.width &&
 		    captureSize.height >= maxRawStreamSize.height) {
-			if (!pipeConfig_ || captureSize < pipeConfig_->captureSize)
+			if (!pipeConfig_ || captureSize < pipeConfig_->captureSize ||
+			    (captureSize <= pipeConfig_->captureSize && !curCfgPacked && newCfgPacked))
 				pipeConfig_ = pipeConfig;
 		}