[libcamera-devel,RFC,v2,0/2] qcam: viewfinder_gl: add RAW12P format support

10799 mbox series

This patchset adds support for OpenGL accelerated conversion from 12-bit
packed raw Bayer format (debyering).

All the 4 pixel layouts are supported, but only the RGGB one was tested.

The lower 4 bits of the 12-bit colour values are discarded as the target RGBA
format has only 8 bits per colour.

Changes in v2 vs v1 [1]:
* Most of the review comments are addressed. In particular:
  * comments explaining the calculations in the shader are added
  * textureUniformY_ is reused in the same way as RGB formats do
  * uniforms are renamed to follow the common pattern
  * the existing float tex_stepx is extended to vec2 tex_step and is
    used in both YUV_packed and bayer_12_packed shaders
  * firstRed_ type changed to QPointF
  * NOT CHANGED: the center is still vec4, and [xy]coords are still vec2.
    The reason is that I'd like to keep them in the same form as in the
    original code [2] - I still consider moving from the current bilinear
    to the MHC filter implemented in [2] someday.
* The shader code is reworked to address the issue with the garbage due
  to scaling the image from the captured size to the viewfinder window size.
  With v1 one could see something like [3], and in v2 this is fixed [4].
  For this all the processing is moved to the fragment shader, so that the
  hardware interpolators don't have the effect (the price is increased GPU
  load). [PATCH][RFC 1/2] "qcam: viewfinder_gl: set the vertex shader file
  name in selectFormat()" is dropped as the common identity vertex shader is
  used in v2.
* The header in the fragment shader is updated to refer to the work [2] this
  shader is based on. The license is adjusted accordingly.

[1] https://lists.libcamera.org/pipermail/libcamera-devel/2020-November/014893.html
[2] https://casual-effects.com/research/McGuire2009Bayer/index.html
[3] https://people.linaro.org/~andrey.konovalov/qcam-vf_gl-raw12p/v1_maximized.bmp
[4] https://people.linaro.org/~andrey.konovalov/qcam-vf_gl-raw12p/v2-1_maximized.bmp

I'd like to add that I would probably hold off on merging this patchset until I
post the RAW10P one. The latter is in progress, and when it is completed, it will
be more clear if reading the frame into GL_RGB texture (this is what the current
patchset does) vs GL_RED is the right choice.

Thanks,
Andrey

On 01.01.2021 20:15, Andrey Konovalov wrote:
> This patchset adds support for OpenGL accelerated conversion from 12-bit
> packed raw Bayer format (debyering).
> 
> All the 4 pixel layouts are supported, but only the RGGB one was tested.
> 
> The lower 4 bits of the 12-bit colour values are discarded as the target RGBA
> format has only 8 bits per colour.
> 
> Changes in v2 vs v1 [1]:
> * Most of the review comments are addressed. In particular:
>    * comments explaining the calculations in the shader are added
>    * textureUniformY_ is reused in the same way as RGB formats do
>    * uniforms are renamed to follow the common pattern
>    * the existing float tex_stepx is extended to vec2 tex_step and is
>      used in both YUV_packed and bayer_12_packed shaders
>    * firstRed_ type changed to QPointF
>    * NOT CHANGED: the center is still vec4, and [xy]coords are still vec2.
>      The reason is that I'd like to keep them in the same form as in the
>      original code [2] - I still consider moving from the current bilinear
>      to the MHC filter implemented in [2] someday.
> * The shader code is reworked to address the issue with the garbage due
>    to scaling the image from the captured size to the viewfinder window size.
>    With v1 one could see something like [3], and in v2 this is fixed [4].
>    For this all the processing is moved to the fragment shader, so that the
>    hardware interpolators don't have the effect (the price is increased GPU
>    load). [PATCH][RFC 1/2] "qcam: viewfinder_gl: set the vertex shader file
>    name in selectFormat()" is dropped as the common identity vertex shader is
>    used in v2.
> * The header in the fragment shader is updated to refer to the work [2] this
>    shader is based on. The license is adjusted accordingly.
> 
> [1] https://lists.libcamera.org/pipermail/libcamera-devel/2020-November/014893.html
> [2] https://casual-effects.com/research/McGuire2009Bayer/index.html
> [3] https://people.linaro.org/~andrey.konovalov/qcam-vf_gl-raw12p/v1_maximized.bmp
> [4] https://people.linaro.org/~andrey.konovalov/qcam-vf_gl-raw12p/v2-1_maximized.bmp
> 
>