libcamera: shaders: bayer_unpacked: Sample at texel centres
diff mbox series

Message ID 20261115175436.8673-1-firassharary3@gmail.com
State New
Headers show
Series
  • libcamera: shaders: bayer_unpacked: Sample at texel centres
Related show

Commit Message

Firas Sharary Nov. 15, 2026, 5:54 p.m. UTC 
When the software ISP scales with the GPU debayer, the fragment shader
samples the Bayer texture at the interpolated position and derives the
Bayer phase of that position from floor(center.zw). The two can disagree:
the sampler resolves nearest filtering in fixed point, so a position just
below a texel boundary may return the next texel, while floor() still
selects the previous one. The pixel is then demosaiced with the phase of
one texel and the data of its neighbour.

With a colour correction matrix loaded this shows up as isolated magenta
columns and rows, one pixel wide, whose position depends on the scaling
ratio. On an OV5693 read at 2592x1944 through an Intel IPU6 (Tiger Lake,
Iris Xe, Mesa 25.0.7) a 1920x1080 stream has them at x=731, x=1211 and
y=731, and a 2560x1440 stream at y=1051. In those lines the green channel
is zero.

Compute the texel from floor(center.zw) once and sample at its centre, so
the phase and the data always come from the same texel.

While at it, handle the neighbours that fall outside the image. They were
clamped to the edge texel, which has a different Bayer colour than the
pixel asked for and tinted the first row and column. Use the neighbour at
the same distance on the other side instead.

Signed-off-by: Firas Sharary <firassharary3@gmail.com>
---

Tested with v0.8.0 on a Surface Pro 7+; bayer_unpacked.frag is unchanged
between v0.8.0 and master. Captures at 1280x720, 1920x1080 and 2560x1440
from the 2592x1944 mode no longer show the lines or the tinted border.

bayer_1x_packed.frag derives its coordinates differently and I could not
test it: this machine's IPU6 does not deliver packed frames at all (see
"media: ipu6: packed raw capture never completes a frame on Tiger Lake"
on linux-media).
 src/libcamera/shaders/bayer_unpacked.frag | 60 +++++++++++++++++------
 1 file changed, 46 insertions(+), 14 deletions(-)

Comments

Kieran Bingham Nov. 15, 2026, 6:03 p.m. UTC | #1 
+Cc Ray.

Quoting Firas Sharary (2026-11-15 17:54:36)
> When the software ISP scales with the GPU debayer, the fragment shader
> samples the Bayer texture at the interpolated position and derives the
> Bayer phase of that position from floor(center.zw). The two can disagree:
> the sampler resolves nearest filtering in fixed point, so a position just
> below a texel boundary may return the next texel, while floor() still
> selects the previous one. The pixel is then demosaiced with the phase of
> one texel and the data of its neighbour.
> 
> With a colour correction matrix loaded this shows up as isolated magenta
> columns and rows, one pixel wide, whose position depends on the scaling
> ratio. On an OV5693 read at 2592x1944 through an Intel IPU6 (Tiger Lake,
> Iris Xe, Mesa 25.0.7) a 1920x1080 stream has them at x=731, x=1211 and
> y=731, and a 2560x1440 stream at y=1051. In those lines the green channel
> is zero.
> 
> Compute the texel from floor(center.zw) once and sample at its centre, so
> the phase and the data always come from the same texel.
> 
> While at it, handle the neighbours that fall outside the image. They were
> clamped to the edge texel, which has a different Bayer colour than the
> pixel asked for and tinted the first row and column. Use the neighbour at
> the same distance on the other side instead.
> 

Is this the same as / related to:

libcamera: software_isp: debayer_egl: Fix texture sampling steps
https://patchwork.libcamera.org/patch/28702/ ?

--
Kieran


> Signed-off-by: Firas Sharary <firassharary3@gmail.com>
> ---
> 
> Tested with v0.8.0 on a Surface Pro 7+; bayer_unpacked.frag is unchanged
> between v0.8.0 and master. Captures at 1280x720, 1920x1080 and 2560x1440
> from the 2592x1944 mode no longer show the lines or the tinted border.
> 
> bayer_1x_packed.frag derives its coordinates differently and I could not
> test it: this machine's IPU6 does not deliver packed frames at all (see
> "media: ipu6: packed raw capture never completes a frame on Tiger Lake"
> on linux-media).
>  src/libcamera/shaders/bayer_unpacked.frag | 60 +++++++++++++++++------
>  1 file changed, 46 insertions(+), 14 deletions(-)
> 
> diff --git a/src/libcamera/shaders/bayer_unpacked.frag b/src/libcamera/shaders/bayer_unpacked.frag
> index be2f89d..f06022b 100644
> --- a/src/libcamera/shaders/bayer_unpacked.frag
> +++ b/src/libcamera/shaders/bayer_unpacked.frag
> @@ -24,6 +24,10 @@ uniform sampler2D       tex_y;
>  varying vec4            center;
>  varying vec4            yCoord;
>  varying vec4            xCoord;
> +uniform vec2            tex_size;
> +uniform vec2            tex_step;
> +uniform vec2            tex_bayer_first_red;
> +uniform float           stride_factor;
>  uniform vec3            awb;
>  uniform mat3            ccm;
>  uniform vec3            blacklevel;
> @@ -56,17 +60,45 @@ void main(void) {
>      #define fetch(x, y) texture2D(tex_y, vec2(x, y)).r
>      #endif
>  
> -    float C = fetch(center.x, center.y); // ( 0, 0)
> +    /*
> +     * Sample at the centre of the source pixel that floor(center.zw) selects,
> +     * not at the interpolated position. When the output is scaled the
> +     * interpolated position can lie so close to a texel boundary that the
> +     * sampler, which resolves nearest filtering in fixed point, returns the
> +     * neighbouring texel. The pixel is then demosaiced with the Bayer phase
> +     * of one texel and the data of another, which shows up as isolated
> +     * magenta rows and columns in the output.
> +     */
> +    vec2 pos = floor(center.zw);
> +    vec2 snapped = (pos - tex_bayer_first_red + 0.5) / tex_size;
> +    vec2 c = vec2(snapped.x * stride_factor, snapped.y);
> +    vec4 dx = vec4(-2.0, -1.0, 1.0, 2.0) * tex_step.x;
> +    vec4 dy = vec4(-2.0, -1.0, 1.0, 2.0) * tex_step.y;
> +    vec4 xC = c.x + dx;
> +    vec4 yC = c.y + dy;
> +
> +    /*
> +     * Neighbours beyond the edge of the image would be clamped to the edge
> +     * texel, which has a different Bayer colour than the pixel asked for and
> +     * tints the outermost row and column. Use the neighbour at the same
> +     * distance on the other side instead, which has the right colour.
> +     */
> +    xC = mix(xC, c.x - dx, vec4(lessThan(xC, vec4(0.0))) +
> +                           vec4(greaterThan(xC, vec4(stride_factor))));
> +    yC = mix(yC, c.y - dy, vec4(lessThan(yC, vec4(0.0))) +
> +                           vec4(greaterThan(yC, vec4(1.0))));
> +
> +    float C = fetch(c.x, c.y); // ( 0, 0)
>      const vec4 kC = vec4( 4.0,  6.0,  5.0,  5.0) / 8.0;
>  
>      // Determine which of four types of pixels we are on.
> -    vec2 alternate = mod(floor(center.zw), 2.0);
> +    vec2 alternate = mod(pos, 2.0);
>  
>      vec4 Dvec = vec4(
> -        fetch(xCoord[1], yCoord[1]),  // (-1,-1)
> -        fetch(xCoord[1], yCoord[2]),  // (-1, 1)
> -        fetch(xCoord[2], yCoord[1]),  // ( 1,-1)
> -        fetch(xCoord[2], yCoord[2])); // ( 1, 1)
> +        fetch(xC[1], yC[1]),  // (-1,-1)
> +        fetch(xC[1], yC[2]),  // (-1, 1)
> +        fetch(xC[2], yC[1]),  // ( 1,-1)
> +        fetch(xC[2], yC[2])); // ( 1, 1)
>  
>      vec4 PATTERN = (kC.xyz * C).xyzz;
>  
> @@ -77,16 +109,16 @@ void main(void) {
>      Dvec.x  += Dvec.y;
>  
>      vec4 value = vec4(
> -        fetch(center.x, yCoord[0]),   // ( 0,-2)
> -        fetch(center.x, yCoord[1]),   // ( 0,-1)
> -        fetch(xCoord[0], center.y),   // (-2, 0)
> -        fetch(xCoord[1], center.y));  // (-1, 0)
> +        fetch(c.x, yC[0]),   // ( 0,-2)
> +        fetch(c.x, yC[1]),   // ( 0,-1)
> +        fetch(xC[0], c.y),   // (-2, 0)
> +        fetch(xC[1], c.y));  // (-1, 0)
>  
>      vec4 temp = vec4(
> -        fetch(center.x, yCoord[3]),   // ( 0, 2)
> -        fetch(center.x, yCoord[2]),   // ( 0, 1)
> -        fetch(xCoord[3], center.y),   // ( 2, 0)
> -        fetch(xCoord[2], center.y));  // ( 1, 0)
> +        fetch(c.x, yC[3]),   // ( 0, 2)
> +        fetch(c.x, yC[2]),   // ( 0, 1)
> +        fetch(xC[3], c.y),   // ( 2, 0)
> +        fetch(xC[2], c.y));  // ( 1, 0)
>  
>      // Even the simplest compilers should be able to constant-fold these to
>      // avoid the division.
> -- 
> 2.47.3

Patch
diff mbox series 

diff --git a/src/libcamera/shaders/bayer_unpacked.frag b/src/libcamera/shaders/bayer_unpacked.frag
index be2f89d..f06022b 100644
--- a/src/libcamera/shaders/bayer_unpacked.frag
+++ b/src/libcamera/shaders/bayer_unpacked.frag
@@ -24,6 +24,10 @@  uniform sampler2D       tex_y;
 varying vec4            center;
 varying vec4            yCoord;
 varying vec4            xCoord;
+uniform vec2            tex_size;
+uniform vec2            tex_step;
+uniform vec2            tex_bayer_first_red;
+uniform float           stride_factor;
 uniform vec3            awb;
 uniform mat3            ccm;
 uniform vec3            blacklevel;
@@ -56,17 +60,45 @@  void main(void) {
     #define fetch(x, y) texture2D(tex_y, vec2(x, y)).r
     #endif
 
-    float C = fetch(center.x, center.y); // ( 0, 0)
+    /*
+     * Sample at the centre of the source pixel that floor(center.zw) selects,
+     * not at the interpolated position. When the output is scaled the
+     * interpolated position can lie so close to a texel boundary that the
+     * sampler, which resolves nearest filtering in fixed point, returns the
+     * neighbouring texel. The pixel is then demosaiced with the Bayer phase
+     * of one texel and the data of another, which shows up as isolated
+     * magenta rows and columns in the output.
+     */
+    vec2 pos = floor(center.zw);
+    vec2 snapped = (pos - tex_bayer_first_red + 0.5) / tex_size;
+    vec2 c = vec2(snapped.x * stride_factor, snapped.y);
+    vec4 dx = vec4(-2.0, -1.0, 1.0, 2.0) * tex_step.x;
+    vec4 dy = vec4(-2.0, -1.0, 1.0, 2.0) * tex_step.y;
+    vec4 xC = c.x + dx;
+    vec4 yC = c.y + dy;
+
+    /*
+     * Neighbours beyond the edge of the image would be clamped to the edge
+     * texel, which has a different Bayer colour than the pixel asked for and
+     * tints the outermost row and column. Use the neighbour at the same
+     * distance on the other side instead, which has the right colour.
+     */
+    xC = mix(xC, c.x - dx, vec4(lessThan(xC, vec4(0.0))) +
+                           vec4(greaterThan(xC, vec4(stride_factor))));
+    yC = mix(yC, c.y - dy, vec4(lessThan(yC, vec4(0.0))) +
+                           vec4(greaterThan(yC, vec4(1.0))));
+
+    float C = fetch(c.x, c.y); // ( 0, 0)
     const vec4 kC = vec4( 4.0,  6.0,  5.0,  5.0) / 8.0;
 
     // Determine which of four types of pixels we are on.
-    vec2 alternate = mod(floor(center.zw), 2.0);
+    vec2 alternate = mod(pos, 2.0);
 
     vec4 Dvec = vec4(
-        fetch(xCoord[1], yCoord[1]),  // (-1,-1)
-        fetch(xCoord[1], yCoord[2]),  // (-1, 1)
-        fetch(xCoord[2], yCoord[1]),  // ( 1,-1)
-        fetch(xCoord[2], yCoord[2])); // ( 1, 1)
+        fetch(xC[1], yC[1]),  // (-1,-1)
+        fetch(xC[1], yC[2]),  // (-1, 1)
+        fetch(xC[2], yC[1]),  // ( 1,-1)
+        fetch(xC[2], yC[2])); // ( 1, 1)
 
     vec4 PATTERN = (kC.xyz * C).xyzz;
 
@@ -77,16 +109,16 @@  void main(void) {
     Dvec.x  += Dvec.y;
 
     vec4 value = vec4(
-        fetch(center.x, yCoord[0]),   // ( 0,-2)
-        fetch(center.x, yCoord[1]),   // ( 0,-1)
-        fetch(xCoord[0], center.y),   // (-2, 0)
-        fetch(xCoord[1], center.y));  // (-1, 0)
+        fetch(c.x, yC[0]),   // ( 0,-2)
+        fetch(c.x, yC[1]),   // ( 0,-1)
+        fetch(xC[0], c.y),   // (-2, 0)
+        fetch(xC[1], c.y));  // (-1, 0)
 
     vec4 temp = vec4(
-        fetch(center.x, yCoord[3]),   // ( 0, 2)
-        fetch(center.x, yCoord[2]),   // ( 0, 1)
-        fetch(xCoord[3], center.y),   // ( 2, 0)
-        fetch(xCoord[2], center.y));  // ( 1, 0)
+        fetch(c.x, yC[3]),   // ( 0, 2)
+        fetch(c.x, yC[2]),   // ( 0, 1)
+        fetch(xC[3], c.y),   // ( 2, 0)
+        fetch(xC[2], c.y));  // ( 1, 0)
 
     // Even the simplest compilers should be able to constant-fold these to
     // avoid the division.