libcamera: software_isp: debayer_egl: Fix texture sampling steps
diff mbox series

Message ID 20261029142649.388275-1-halfline@gmail.com
State New
Headers show
Series
  • libcamera: software_isp: debayer_egl: Fix texture sampling steps
Related show

Commit Message

Ray Strode Oct. 29, 2026, 2:26 p.m. UTC
From: Ray Strode <rstrode@redhat.com>

This commit removes thin horizontal color artifacts caused by the GPU
debayer sampling the wrong Bayer rows.

The shader samples neighboring Bayer pixels using offsets in normalized
texture coordinates. The step denominators currently subtract one from the
input byte stride and image height, making the offsets larger than one
texel. Scaling can then place a lookup on the wrong side of a texel
boundary. A uniform synthetic Bayer image reproduces the lines seen in live
GPU captures.

The offsets now match the reciprocal texture dimensions: bytes per pixel
divided by byte stride horizontally, and one divided by image height
vertically.

Fixes: f520b29fe9e6 ("libcamera: software_isp: debayer_egl: Add an eGL Debayer class")
Signed-off-by: Ray Strode <rstrode@redhat.com>
---
 src/libcamera/software_isp/debayer_egl.cpp | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)


base-commit: 5670220c89cf772e7302d4e0ef6c716c31c00b46

Comments

Hans de Goede Nov. 12, 2026, 3:05 p.m. UTC | #1
Hi,

On 29-Oct-26 15:26, Ray Strode wrote:
> From: Ray Strode <rstrode@redhat.com>
> 
> This commit removes thin horizontal color artifacts caused by the GPU
> debayer sampling the wrong Bayer rows.
> 
> The shader samples neighboring Bayer pixels using offsets in normalized
> texture coordinates. The step denominators currently subtract one from the
> input byte stride and image height, making the offsets larger than one
> texel. Scaling can then place a lookup on the wrong side of a texel
> boundary. A uniform synthetic Bayer image reproduces the lines seen in live
> GPU captures.
> 
> The offsets now match the reciprocal texture dimensions: bytes per pixel
> divided by byte stride horizontally, and one divided by image height
> vertically.
> 
> Fixes: f520b29fe9e6 ("libcamera: software_isp: debayer_egl: Add an eGL Debayer class")
> Signed-off-by: Ray Strode <rstrode@redhat.com>

Thanks, patch looks good to me:

Reviewed-by: Hans de Goede <johannes.goede@oss.qualcomm.com>

Regards,

Hans




> ---
>  src/libcamera/software_isp/debayer_egl.cpp | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp
> index 33933b11..e6df04da 100644
> --- a/src/libcamera/software_isp/debayer_egl.cpp
> +++ b/src/libcamera/software_isp/debayer_egl.cpp
> @@ -394,8 +394,8 @@ void DebayerEGL::setShaderVariableValues(eGLImage &eglImageIn, const DebayerPara
>  	GLfloat firstRed[] = { firstRed_x_, firstRed_y_ };
>  	GLfloat imgSize[] = { (GLfloat)width_,
>  			      (GLfloat)height_ };
> -	GLfloat Step[] = { static_cast<float>(bytesPerPixel_) / (inputConfig_.stride - 1),
> -			   1.0f / (height_ - 1) };
> +	GLfloat Step[] = { static_cast<float>(bytesPerPixel_) / inputConfig_.stride,
> +			   1.0f / height_ };
>  	GLfloat Stride = (GLfloat)width_ / (shaderStridePixels_ / bytesPerPixel_);
>  	/*
>  	 * Scale the output size from the native size the algorithm produces for
> 
> base-commit: 5670220c89cf772e7302d4e0ef6c716c31c00b46
Kieran Bingham Nov. 13, 2026, 1:27 p.m. UTC | #2
Quoting johannes.goede@oss.qualcomm.com (2026-11-12 15:05:20)
> Hi,
> 
> On 29-Oct-26 15:26, Ray Strode wrote:
> > From: Ray Strode <rstrode@redhat.com>
> > 
> > This commit removes thin horizontal color artifacts caused by the GPU
> > debayer sampling the wrong Bayer rows.
> > 
> > The shader samples neighboring Bayer pixels using offsets in normalized
> > texture coordinates. The step denominators currently subtract one from the
> > input byte stride and image height, making the offsets larger than one
> > texel. Scaling can then place a lookup on the wrong side of a texel
> > boundary. A uniform synthetic Bayer image reproduces the lines seen in live
> > GPU captures.
> > 
> > The offsets now match the reciprocal texture dimensions: bytes per pixel
> > divided by byte stride horizontally, and one divided by image height
> > vertically.
> > 
> > Fixes: f520b29fe9e6 ("libcamera: software_isp: debayer_egl: Add an eGL Debayer class")
> > Signed-off-by: Ray Strode <rstrode@redhat.com>
> 
> Thanks, patch looks good to me:
> 
> Reviewed-by: Hans de Goede <johannes.goede@oss.qualcomm.com>

Seems sane to get in:


Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>

> 
> Regards,
> 
> Hans
> 
> 
> 
> 
> > ---
> >  src/libcamera/software_isp/debayer_egl.cpp | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp
> > index 33933b11..e6df04da 100644
> > --- a/src/libcamera/software_isp/debayer_egl.cpp
> > +++ b/src/libcamera/software_isp/debayer_egl.cpp
> > @@ -394,8 +394,8 @@ void DebayerEGL::setShaderVariableValues(eGLImage &eglImageIn, const DebayerPara
> >       GLfloat firstRed[] = { firstRed_x_, firstRed_y_ };
> >       GLfloat imgSize[] = { (GLfloat)width_,
> >                             (GLfloat)height_ };
> > -     GLfloat Step[] = { static_cast<float>(bytesPerPixel_) / (inputConfig_.stride - 1),
> > -                        1.0f / (height_ - 1) };
> > +     GLfloat Step[] = { static_cast<float>(bytesPerPixel_) / inputConfig_.stride,
> > +                        1.0f / height_ };
> >       GLfloat Stride = (GLfloat)width_ / (shaderStridePixels_ / bytesPerPixel_);
> >       /*
> >        * Scale the output size from the native size the algorithm produces for
> > 
> > base-commit: 5670220c89cf772e7302d4e0ef6c716c31c00b46
>
Ray Strode Nov. 13, 2026, 3:26 p.m. UTC | #3
Hi,

On Fri, Nov 13, 2026 at 8:27 AM Kieran Bingham
<kieran.bingham@ideasonboard.com> wrote:
>
> Quoting johannes.goede@oss.qualcomm.com (2026-11-12 15:05:20)
> > Hi,
> >
> > On 29-Oct-26 15:26, Ray Strode wrote:
> > > From: Ray Strode <rstrode@redhat.com>
> > >
> > > This commit removes thin horizontal color artifacts caused by the GPU
> > > debayer sampling the wrong Bayer rows.
> > >
> > > The shader samples neighboring Bayer pixels using offsets in normalized
> > > texture coordinates. The step denominators currently subtract one from the
> > > input byte stride and image height, making the offsets larger than one
> > > texel. Scaling can then place a lookup on the wrong side of a texel
> > > boundary. A uniform synthetic Bayer image reproduces the lines seen in live
> > > GPU captures.
> > >
> > > The offsets now match the reciprocal texture dimensions: bytes per pixel
> > > divided by byte stride horizontally, and one divided by image height
> > > vertically.
> > >
> > > Fixes: f520b29fe9e6 ("libcamera: software_isp: debayer_egl: Add an eGL Debayer class")
> > > Signed-off-by: Ray Strode <rstrode@redhat.com>
> > Thanks, patch looks good to me:
> > Reviewed-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
> Seems sane to get in:
> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>

Thanks for the Review-by's guys! If this is going to go in let's fix
up the commit message to
match other commits in the project a little better (imperative mood, etc).
Maybe something like:

---
libcamera: software_isp: debayer_egl: Fix horizontal line artifacts

The GPU debayer reads samples from the Bayer input using a fixed vertical
step size based on the height of the texture.

The result is supposed to be a normalized step that advances by one row.

Due to an off-by-one error in the code, however, the step is slightly
larger than it's supposed to be, resulting in a lookup that can select the
wrong Bayer row, which can lead to faded thin horizontal line artifacts
in the output.

Correct the off-by-one error in the code, so the debayer uses the correct
vertical step size to eliminate the artifact in the output. Correct the
same off-by-one error in the horizontal step as well.

Fixes: f520b29fe9e6 ("libcamera: software_isp: debayer_egl: Add an eGL
Debayer class")
Reviewed-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
---

Ray
Bryan O'Donoghue Nov. 15, 2026, 11:11 p.m. UTC | #4
On 29/10/2026 14:26, Ray Strode wrote:
> From: Ray Strode <rstrode@redhat.com>
> 
> This commit removes thin horizontal color artifacts caused by the GPU
> debayer sampling the wrong Bayer rows.
> 
> The shader samples neighboring Bayer pixels using offsets in normalized
> texture coordinates. The step denominators currently subtract one from the
> input byte stride and image height, making the offsets larger than one
> texel. Scaling can then place a lookup on the wrong side of a texel
> boundary. A uniform synthetic Bayer image reproduces the lines seen in live
> GPU captures.
> 
> The offsets now match the reciprocal texture dimensions: bytes per pixel
> divided by byte stride horizontally, and one divided by image height
> vertically.
> 
> Fixes: f520b29fe9e6 ("libcamera: software_isp: debayer_egl: Add an eGL Debayer class")
> Signed-off-by: Ray Strode <rstrode@redhat.com>
> ---
>   src/libcamera/software_isp/debayer_egl.cpp | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp
> index 33933b11..e6df04da 100644
> --- a/src/libcamera/software_isp/debayer_egl.cpp
> +++ b/src/libcamera/software_isp/debayer_egl.cpp
> @@ -394,8 +394,8 @@ void DebayerEGL::setShaderVariableValues(eGLImage &eglImageIn, const DebayerPara
>   	GLfloat firstRed[] = { firstRed_x_, firstRed_y_ };
>   	GLfloat imgSize[] = { (GLfloat)width_,
>   			      (GLfloat)height_ };
> -	GLfloat Step[] = { static_cast<float>(bytesPerPixel_) / (inputConfig_.stride - 1),
> -			   1.0f / (height_ - 1) };
> +	GLfloat Step[] = { static_cast<float>(bytesPerPixel_) / inputConfig_.stride,
> +			   1.0f / height_ };
>   	GLfloat Stride = (GLfloat)width_ / (shaderStridePixels_ / bytesPerPixel_);
>   	/*
>   	 * Scale the output size from the native size the algorithm produces for
> 
> base-commit: 5670220c89cf772e7302d4e0ef6c716c31c00b46
> --
> 2.56.0

On its own this patch will cause a bug on the packed shader.

I have something in my multi-pass branch which should fix this as well 
as bring us to _one_ fragment shader.

I believe both changes should go in the same patch or adjacent.

So with your consent I'll absorb this into my series.

---
bod
Ray Strode Nov. 16, 2026, 1:17 a.m. UTC | #5
Hi,

On Sun, Nov 15, 2026 at 6:11 PM Bryan O'Donoghue <bod.linux@nxsw.ie> wrote:
> On its own this patch will cause a bug on the packed shader.

Indeed, looking now, the packed shader rounds down the texture coordinate using
floor() to find the index of the Bayer sample, but the subsequent byte
lookup addresses
the very start of the texel, and floating point imprecision could lead
to it inadvertently
picking the neighboring texel. The `- 1` in the denominator mitigated
that problem
to some extent by giving a slight bias after rounding. This could
still be a problem
even without my patch for big enough textures (since as the texture
size increases,
the margin to the texel boundary can fall below the floating point imprecision).
The right fix, I guess,would be to add .5 to the byte coordinates after rounding
so we sample from the center of the texel instead of the boundary.

> I have something in my multi-pass branch which should fix this as well
> as bring us to _one_ fragment shader.
>
> I believe both changes should go in the same patch or adjacent.
>
> So with your consent I'll absorb this into my series.

Yea that sounds like the best approach all around. I'm not territorial
at all, I don't need
credit or forcing the changes to be separate but adjacent when they
would work better
together, or anything like that, I just care about the purple line
getting fixed :-)

If you have a rewrite in the works that will land soon and fix this,
let's just land it instead.

--Ray

Patch
diff mbox series

diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp
index 33933b11..e6df04da 100644
--- a/src/libcamera/software_isp/debayer_egl.cpp
+++ b/src/libcamera/software_isp/debayer_egl.cpp
@@ -394,8 +394,8 @@  void DebayerEGL::setShaderVariableValues(eGLImage &eglImageIn, const DebayerPara
 	GLfloat firstRed[] = { firstRed_x_, firstRed_y_ };
 	GLfloat imgSize[] = { (GLfloat)width_,
 			      (GLfloat)height_ };
-	GLfloat Step[] = { static_cast<float>(bytesPerPixel_) / (inputConfig_.stride - 1),
-			   1.0f / (height_ - 1) };
+	GLfloat Step[] = { static_cast<float>(bytesPerPixel_) / inputConfig_.stride,
+			   1.0f / height_ };
 	GLfloat Stride = (GLfloat)width_ / (shaderStridePixels_ / bytesPerPixel_);
 	/*
 	 * Scale the output size from the native size the algorithm produces for