[{"id":12100,"web_url":"https://patchwork.libcamera.org/comment/12100/","msgid":"<20200824000455.GQ6002@pendragon.ideasonboard.com>","date":"2020-08-24T00:04:55","subject":"Re: [libcamera-devel] [PATCH v4 1/3] qcam: add OpenGL shader code\n\tas QT resource","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Shaw,\n\nThank you for the patch.\n\nOn Sat, Aug 22, 2020 at 12:16:00AM +0800, Show Liu wrote:\n> qcam: add OpenGL shader code as QT resource\n\ns/QT/Qt/\n\n> Signed-off-by: Show Liu \n> ---\n> src/qcam/assets/shader/NV_2_planes_UV_f.glsl | 32 +++++++++++++++++++\n> src/qcam/assets/shader/NV_2_planes_VU_f.glsl | 32 +++++++++++++++++++\n> src/qcam/assets/shader/NV_3_planes_UV_f.glsl | 33 ++++++++++++++++++++\n> src/qcam/assets/shader/NV_3_planes_VU_f.glsl | 33 ++++++++++++++++++++\n> src/qcam/assets/shader/NV_vertex_shader.glsl | 16 ++++++++++\n> src/qcam/assets/shader/shaders.qrc | 10 ++++++\n> src/qcam/meson.build | 1 +\n> 7 files changed, 157 insertions(+)\n> create mode 100644 src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> create mode 100644 src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> create mode 100644 src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> create mode 100644 src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> create mode 100644 src/qcam/assets/shader/NV_vertex_shader.glsl\n> create mode 100644 src/qcam/assets/shader/shaders.qrc\n> \n> diff --git a/src/qcam/assets/shader/NV_2_planes_UV_f.glsl b/src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> new file mode 100644\n> index 0000000..32c6e90\n> --- /dev/null\n> +++ b/src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> @@ -0,0 +1,32 @@\n> +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> +/*\n> + * Copyright (C) 2020, Linaro\n> + *\n> + * NV_2_planes_UV_f.glsl - Fragment shader code for NV12, NV16 and NV24 formats\n> + */\n> +\n> +#ifdef GL_ES\n> +precision highp float;\n> +#endif\n> +\n> +varying vec2 textureOut;\n> +uniform sampler2D tex_y;\n> +uniform sampler2D tex_u;\n> +\n> +void main(void)\n> +{\n> +\tvec3 yuv;\n> +\tvec3 rgb;\n> +\tmat3 convert_mat = mat3(\n\nMaybe yuv2rgb or yuv2rgb_mat instead of convert_mat ? Or yuv2rgb_bt601\nor yuv2rgb_bt601_mat ?\n\n> +\t\t\t\t\t\t\tvec3(1.1640625, 1.1640625, 1.1640625),\n> +\t\t\t\t\t\t\tvec3(0.0, -0.390625, 2.015625),\n> +\t\t\t\t\t\t\tvec3(1.5975625, -0.8125, 0.0)\n\nWhere are the coefficients from ? https://en.wikipedia.org/wiki/YCbCr\nlists values very similar but slightly different. Are the values here\nrounded to a multiple of 1/256 on purpose (except for 1.5975625) ?\n\n> +\t\t\t\t\t\t );\n\nThe indentation is a bit weird.\n\n> +\n> +\tyuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n\nIs 0.0625 (16/256) the right value, or should it be 16/255 ? The Y range\non 8-bit is 0-255, which is mapped to 0.0-1.0, right ?\n\n> +\tyuv.y = texture2D(tex_u, textureOut).r - 0.5;\n> +\tyuv.z = texture2D(tex_u, textureOut).g - 0.5;\n> +\n> +\trgb = convert_mat * yuv;\n> +\tgl_FragColor = vec4(rgb, 1.0);\n> +}\n> diff --git a/src/qcam/assets/shader/NV_2_planes_VU_f.glsl b/src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> new file mode 100644\n> index 0000000..aae12de\n> --- /dev/null\n> +++ b/src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> @@ -0,0 +1,32 @@\n> +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> +/*\n> + * Copyright (C) 2020, Linaro\n> + *\n> + * NV_2_planes_VU_f.glsl - Fragment shader code for NV21, NV61 and NV42 formats\n> + */\n> +\n> +#ifdef GL_ES\n> +precision highp float;\n> +#endif\n> +\n> +varying vec2 textureOut;\n> +uniform sampler2D tex_y;\n> +uniform sampler2D tex_u;\n> +\n> +void main(void)\n> +{\n> +\tvec3 yuv;\n> +\tvec3 rgb;\n> +\tmat3 convert_mat = mat3(\n> +\t\t\t\t\t\t\tvec3(1.1640625, 1.1640625, 1.1640625),\n> +\t\t\t\t\t\t\tvec3(0.0, -0.390625, 2.015625),\n> +\t\t\t\t\t\t\tvec3(1.5975625, -0.8125, 0.0)\n> +\t\t\t\t\t\t );\n\nHere too.\n\n> +\n> +\tyuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> +\tyuv.y = texture2D(tex_u, textureOut).g - 0.5;\n> +\tyuv.z = texture2D(tex_u, textureOut).r - 0.5;\n> +\t\n\nAnd there are some white spaces at the end of lines (below too).\n\nWe'll have to make the coefficients configurable to support different\ncolorspaces, but that's for later.\n\n> +\trgb = convert_mat * yuv;\n> +\tgl_FragColor = vec4(rgb, 1.0);\n> +}\n> diff --git a/src/qcam/assets/shader/NV_3_planes_UV_f.glsl b/src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> new file mode 100644\n> index 0000000..21fff3a\n> --- /dev/null\n> +++ b/src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> @@ -0,0 +1,33 @@\n> +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> +/*\n> + * Copyright (C) 2020, Linaro\n> + *\n> + * NV_3_planes_UV_f.glsl - Fragment shader code for YUV420 format\n> + */\n> +\n> +#ifdef GL_ES\n> +precision highp float;\n> +#endif\n> +\n> +varying vec2 textureOut;\n> +uniform sampler2D tex_y;\n> +uniform sampler2D tex_u;\n> +uniform sampler2D tex_v;\n> +\n> +void main(void)\n> +{\n> +\tvec3 yuv;\n> +\tvec3 rgb;\n> +\tmat3 convert_mat = mat3(\n> +\t\t\t\t\t\t\tvec3(1.1640625, 1.1640625, 1.1640625),\n> +\t\t\t\t\t\t\tvec3(0.0, -0.390625, 2.015625),\n> +\t\t\t\t\t\t\tvec3(1.5975625, -0.8125, 0.0)\n> +\t\t\t\t\t\t );\n> +\n> +\tyuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> +\tyuv.y = texture2D(tex_u, textureOut).r - 0.5;\n> +\tyuv.z = texture2D(tex_v, textureOut).g - 0.5;\n\nI'm not sure to understand the .g here, with YUV420 having three planes,\nshouldn't it be .r ? I could be wrong, my GLSL knowledge is very very\nrudimentary :-)\n\n> +\t\n> +\trgb = convert_mat * yuv;\n> +\tgl_FragColor = vec4(rgb, 1.0);\n> +}\n> diff --git a/src/qcam/assets/shader/NV_3_planes_VU_f.glsl b/src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> new file mode 100644\n> index 0000000..df00170\n> --- /dev/null\n> +++ b/src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> @@ -0,0 +1,33 @@\n> +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> +/*\n> + * Copyright (C) 2020, Linaro\n> + *\n> + * NV_3_planes_VU_f.glsl - Fragment shader code for YVU420 format\n> + */\n> +\n> +#ifdef GL_ES\n> +precision highp float;\n> +#endif\n> +\n> +varying vec2 textureOut;\n> +uniform sampler2D tex_y;\n> +uniform sampler2D tex_u;\n> +uniform sampler2D tex_v;\n> +\n> +void main(void)\n> +{\n> +\tvec3 yuv;\n> +\tvec3 rgb;\n> +\tmat3 convert_mat = mat3(\n> +\t\t\t\t\t\t\tvec3(1.1640625, 1.1640625, 1.1640625),\n> +\t\t\t\t\t\t\tvec3(0.0, -0.390625, 2.015625),\n> +\t\t\t\t\t\t\tvec3(1.5975625, -0.8125, 0.0)\n> +\t\t\t\t\t\t );\n> +\n> +\tyuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> +\tyuv.y = texture2D(tex_u, textureOut).g - 0.5;\n> +\tyuv.z = texture2D(tex_v, textureOut).r - 0.5;\n\nSame here, I was expecting\n\n\tyuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n\tyuv.y = texture2D(tex_v, textureOut).r - 0.5;\n\tyuv.z = texture2D(tex_u, textureOut).r - 0.5;\n\n> +\t\n> +\trgb = convert_mat * yuv;\n> +\tgl_FragColor = vec4(rgb, 1.0);\n> +}\n> diff --git a/src/qcam/assets/shader/NV_vertex_shader.glsl b/src/qcam/assets/shader/NV_vertex_shader.glsl\n> new file mode 100644\n> index 0000000..403b791\n> --- /dev/null\n> +++ b/src/qcam/assets/shader/NV_vertex_shader.glsl\n> @@ -0,0 +1,16 @@\n> +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> +/*\n> + * Copyright (C) 2020, Linaro\n> + *\n> + * NV_vertex_shader.glsl - Vertex shader code for NV family\n> + */\n> +\n> +attribute vec4 vertexIn;\n> +attribute vec2 textureIn;\n> +varying vec2 textureOut;\n> +\n> +void main(void)\n> +{\n> +\tgl_Position = vertexIn;\n> +\ttextureOut = textureIn;\n> +}\n> diff --git a/src/qcam/assets/shader/shaders.qrc b/src/qcam/assets/shader/shaders.qrc\n> new file mode 100644\n> index 0000000..6fe4c7f\n> --- /dev/null\n> +++ b/src/qcam/assets/shader/shaders.qrc\n> @@ -0,0 +1,10 @@\n> +\n> +\n> +\n> +./NV_vertex_shader.glsl\n> +./NV_2_planes_UV_f.glsl\n> +./NV_2_planes_VU_f.glsl\n> +./NV_3_planes_UV_f.glsl\n> +./NV_3_planes_VU_f.glsl\n> +\n> +\n> diff --git a/src/qcam/meson.build b/src/qcam/meson.build\n> index 6ea886a..e0c6f26 100644\n> --- a/src/qcam/meson.build\n> +++ b/src/qcam/meson.build\n> @@ -16,6 +16,7 @@ qcam_moc_headers = files([\n> \n> qcam_resources = files([\n> 'assets/feathericons/feathericons.qrc',\n> + 'assets/shader/shaders.qrc'\n> ])\n> \n> qt5 = import('qt5')","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 7D737BD87C\n\tfor ;\n\tMon, 24 Aug 2020 00:05:16 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id CE47F627B5;\n\tMon, 24 Aug 2020 02:05:15 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 50C9C60381\n\tfor ;\n\tMon, 24 Aug 2020 02:05:14 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C3CF5279;\n\tMon, 24 Aug 2020 02:05:13 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"vKaVaHi3\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1598227514;\n\tbh=zksIG8ssIi2ui957/2ZI4o6w35eUWgrC6NoutGRY+2Q=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=vKaVaHi3+guADpXV5MmsaUyIOFYnQ5Ddjvwf52SLAg/V12/unqVzBYgVa72l2fQ5j\n\t4DQhdfD3EtoG5TrOecqTwk+nQu9hrcjSMT5s+mIj5aN0YH554Pic8cnhcAU7I6gE+U\n\tCeJsILzrjOpFkXj0j4vHVhIu8aERZVNjPFJLGOds=","Date":"Mon, 24 Aug 2020 03:04:55 +0300","From":"Laurent Pinchart ","To":"Show Liu ","Message-ID":"<20200824000455.GQ6002@pendragon.ideasonboard.com>","References":"<20200821161602.5093-1-show.liu@linaro.org>\n\t<20200821161602.5093-2-show.liu@linaro.org>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200821161602.5093-2-show.liu@linaro.org>","Subject":"Re: [libcamera-devel] [PATCH v4 1/3] qcam: add OpenGL shader code\n\tas QT resource","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12101,"web_url":"https://patchwork.libcamera.org/comment/12101/","msgid":"<20200824000722.GA11768@pendragon.ideasonboard.com>","date":"2020-08-24T00:07:22","subject":"Re: [libcamera-devel] [PATCH v4 1/3] qcam: add OpenGL shader code\n\tas QT resource","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Mon, Aug 24, 2020 at 03:04:56AM +0300, Laurent Pinchart wrote:\n> Hi Shaw,\n\nApologies for the typo.\n\n> Thank you for the patch.\n> \n> On Sat, Aug 22, 2020 at 12:16:00AM +0800, Show Liu wrote:\n> > qcam: add OpenGL shader code as QT resource\n> \n> s/QT/Qt/\n> \n> > Signed-off-by: Show Liu \n> > ---\n> > src/qcam/assets/shader/NV_2_planes_UV_f.glsl | 32 +++++++++++++++++++\n> > src/qcam/assets/shader/NV_2_planes_VU_f.glsl | 32 +++++++++++++++++++\n> > src/qcam/assets/shader/NV_3_planes_UV_f.glsl | 33 ++++++++++++++++++++\n> > src/qcam/assets/shader/NV_3_planes_VU_f.glsl | 33 ++++++++++++++++++++\n> > src/qcam/assets/shader/NV_vertex_shader.glsl | 16 ++++++++++\n> > src/qcam/assets/shader/shaders.qrc | 10 ++++++\n> > src/qcam/meson.build | 1 +\n> > 7 files changed, 157 insertions(+)\n> > create mode 100644 src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> > create mode 100644 src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> > create mode 100644 src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> > create mode 100644 src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> > create mode 100644 src/qcam/assets/shader/NV_vertex_shader.glsl\n> > create mode 100644 src/qcam/assets/shader/shaders.qrc\n> > \n> > diff --git a/src/qcam/assets/shader/NV_2_planes_UV_f.glsl b/src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> > new file mode 100644\n> > index 0000000..32c6e90\n> > --- /dev/null\n> > +++ b/src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> > @@ -0,0 +1,32 @@\n> > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > +/*\n> > + * Copyright (C) 2020, Linaro\n> > + *\n> > + * NV_2_planes_UV_f.glsl - Fragment shader code for NV12, NV16 and NV24 formats\n> > + */\n> > +\n> > +#ifdef GL_ES\n> > +precision highp float;\n> > +#endif\n> > +\n> > +varying vec2 textureOut;\n> > +uniform sampler2D tex_y;\n> > +uniform sampler2D tex_u;\n> > +\n> > +void main(void)\n> > +{\n> > +\tvec3 yuv;\n> > +\tvec3 rgb;\n> > +\tmat3 convert_mat = mat3(\n> \n> Maybe yuv2rgb or yuv2rgb_mat instead of convert_mat ? Or yuv2rgb_bt601\n> or yuv2rgb_bt601_mat ?\n> \n> > +\t\t\t\t\t\t\tvec3(1.1640625, 1.1640625, 1.1640625),\n> > +\t\t\t\t\t\t\tvec3(0.0, -0.390625, 2.015625),\n> > +\t\t\t\t\t\t\tvec3(1.5975625, -0.8125, 0.0)\n> \n> Where are the coefficients from ? https://en.wikipedia.org/wiki/YCbCr\n> lists values very similar but slightly different. Are the values here\n> rounded to a multiple of 1/256 on purpose (except for 1.5975625) ?\n> \n> > +\t\t\t\t\t\t );\n> \n> The indentation is a bit weird.\n> \n> > +\n> > +\tyuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> \n> Is 0.0625 (16/256) the right value, or should it be 16/255 ? The Y range\n> on 8-bit is 0-255, which is mapped to 0.0-1.0, right ?\n> \n> > +\tyuv.y = texture2D(tex_u, textureOut).r - 0.5;\n> > +\tyuv.z = texture2D(tex_u, textureOut).g - 0.5;\n> > +\n> > +\trgb = convert_mat * yuv;\n> > +\tgl_FragColor = vec4(rgb, 1.0);\n> > +}\n> > diff --git a/src/qcam/assets/shader/NV_2_planes_VU_f.glsl b/src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> > new file mode 100644\n> > index 0000000..aae12de\n> > --- /dev/null\n> > +++ b/src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> > @@ -0,0 +1,32 @@\n> > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > +/*\n> > + * Copyright (C) 2020, Linaro\n> > + *\n> > + * NV_2_planes_VU_f.glsl - Fragment shader code for NV21, NV61 and NV42 formats\n> > + */\n> > +\n> > +#ifdef GL_ES\n> > +precision highp float;\n> > +#endif\n> > +\n> > +varying vec2 textureOut;\n> > +uniform sampler2D tex_y;\n> > +uniform sampler2D tex_u;\n> > +\n> > +void main(void)\n> > +{\n> > +\tvec3 yuv;\n> > +\tvec3 rgb;\n> > +\tmat3 convert_mat = mat3(\n> > +\t\t\t\t\t\t\tvec3(1.1640625, 1.1640625, 1.1640625),\n> > +\t\t\t\t\t\t\tvec3(0.0, -0.390625, 2.015625),\n> > +\t\t\t\t\t\t\tvec3(1.5975625, -0.8125, 0.0)\n> > +\t\t\t\t\t\t );\n> \n> Here too.\n> \n> > +\n> > +\tyuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > +\tyuv.y = texture2D(tex_u, textureOut).g - 0.5;\n> > +\tyuv.z = texture2D(tex_u, textureOut).r - 0.5;\n> > +\t\n> \n> And there are some white spaces at the end of lines (below too).\n> \n> We'll have to make the coefficients configurable to support different\n> colorspaces, but that's for later.\n> \n> > +\trgb = convert_mat * yuv;\n> > +\tgl_FragColor = vec4(rgb, 1.0);\n> > +}\n> > diff --git a/src/qcam/assets/shader/NV_3_planes_UV_f.glsl b/src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> > new file mode 100644\n> > index 0000000..21fff3a\n> > --- /dev/null\n> > +++ b/src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> > @@ -0,0 +1,33 @@\n> > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > +/*\n> > + * Copyright (C) 2020, Linaro\n> > + *\n> > + * NV_3_planes_UV_f.glsl - Fragment shader code for YUV420 format\n> > + */\n> > +\n> > +#ifdef GL_ES\n> > +precision highp float;\n> > +#endif\n> > +\n> > +varying vec2 textureOut;\n> > +uniform sampler2D tex_y;\n> > +uniform sampler2D tex_u;\n> > +uniform sampler2D tex_v;\n> > +\n> > +void main(void)\n> > +{\n> > +\tvec3 yuv;\n> > +\tvec3 rgb;\n> > +\tmat3 convert_mat = mat3(\n> > +\t\t\t\t\t\t\tvec3(1.1640625, 1.1640625, 1.1640625),\n> > +\t\t\t\t\t\t\tvec3(0.0, -0.390625, 2.015625),\n> > +\t\t\t\t\t\t\tvec3(1.5975625, -0.8125, 0.0)\n> > +\t\t\t\t\t\t );\n> > +\n> > +\tyuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > +\tyuv.y = texture2D(tex_u, textureOut).r - 0.5;\n> > +\tyuv.z = texture2D(tex_v, textureOut).g - 0.5;\n> \n> I'm not sure to understand the .g here, with YUV420 having three planes,\n> shouldn't it be .r ? I could be wrong, my GLSL knowledge is very very\n> rudimentary :-)\n> \n> > +\t\n> > +\trgb = convert_mat * yuv;\n> > +\tgl_FragColor = vec4(rgb, 1.0);\n> > +}\n> > diff --git a/src/qcam/assets/shader/NV_3_planes_VU_f.glsl b/src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> > new file mode 100644\n> > index 0000000..df00170\n> > --- /dev/null\n> > +++ b/src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> > @@ -0,0 +1,33 @@\n> > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > +/*\n> > + * Copyright (C) 2020, Linaro\n> > + *\n> > + * NV_3_planes_VU_f.glsl - Fragment shader code for YVU420 format\n> > + */\n> > +\n> > +#ifdef GL_ES\n> > +precision highp float;\n> > +#endif\n> > +\n> > +varying vec2 textureOut;\n> > +uniform sampler2D tex_y;\n> > +uniform sampler2D tex_u;\n> > +uniform sampler2D tex_v;\n> > +\n> > +void main(void)\n> > +{\n> > +\tvec3 yuv;\n> > +\tvec3 rgb;\n> > +\tmat3 convert_mat = mat3(\n> > +\t\t\t\t\t\t\tvec3(1.1640625, 1.1640625, 1.1640625),\n> > +\t\t\t\t\t\t\tvec3(0.0, -0.390625, 2.015625),\n> > +\t\t\t\t\t\t\tvec3(1.5975625, -0.8125, 0.0)\n> > +\t\t\t\t\t\t );\n> > +\n> > +\tyuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > +\tyuv.y = texture2D(tex_u, textureOut).g - 0.5;\n> > +\tyuv.z = texture2D(tex_v, textureOut).r - 0.5;\n> \n> Same here, I was expecting\n> \n> \tyuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> \tyuv.y = texture2D(tex_v, textureOut).r - 0.5;\n> \tyuv.z = texture2D(tex_u, textureOut).r - 0.5;\n> \n> > +\t\n> > +\trgb = convert_mat * yuv;\n> > +\tgl_FragColor = vec4(rgb, 1.0);\n> > +}\n> > diff --git a/src/qcam/assets/shader/NV_vertex_shader.glsl b/src/qcam/assets/shader/NV_vertex_shader.glsl\n> > new file mode 100644\n> > index 0000000..403b791\n> > --- /dev/null\n> > +++ b/src/qcam/assets/shader/NV_vertex_shader.glsl\n> > @@ -0,0 +1,16 @@\n> > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > +/*\n> > + * Copyright (C) 2020, Linaro\n> > + *\n> > + * NV_vertex_shader.glsl - Vertex shader code for NV family\n> > + */\n> > +\n> > +attribute vec4 vertexIn;\n> > +attribute vec2 textureIn;\n> > +varying vec2 textureOut;\n> > +\n> > +void main(void)\n> > +{\n> > +\tgl_Position = vertexIn;\n> > +\ttextureOut = textureIn;\n> > +}\n> > diff --git a/src/qcam/assets/shader/shaders.qrc b/src/qcam/assets/shader/shaders.qrc\n> > new file mode 100644\n> > index 0000000..6fe4c7f\n> > --- /dev/null\n> > +++ b/src/qcam/assets/shader/shaders.qrc\n> > @@ -0,0 +1,10 @@\n> > +\n> > +\n> > +\n> > +./NV_vertex_shader.glsl\n> > +./NV_2_planes_UV_f.glsl\n> > +./NV_2_planes_VU_f.glsl\n> > +./NV_3_planes_UV_f.glsl\n> > +./NV_3_planes_VU_f.glsl\n> > +\n> > +\n> > diff --git a/src/qcam/meson.build b/src/qcam/meson.build\n> > index 6ea886a..e0c6f26 100644\n> > --- a/src/qcam/meson.build\n> > +++ b/src/qcam/meson.build\n> > @@ -16,6 +16,7 @@ qcam_moc_headers = files([\n> > \n> > qcam_resources = files([\n> > 'assets/feathericons/feathericons.qrc',\n> > + 'assets/shader/shaders.qrc'\n> > ])\n> > \n> > qt5 = import('qt5')","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 37607BE173\n\tfor ;\n\tMon, 24 Aug 2020 00:07:43 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A20F3627B5;\n\tMon, 24 Aug 2020 02:07:42 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 6ADE260381\n\tfor ;\n\tMon, 24 Aug 2020 02:07:41 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id EF647279;\n\tMon, 24 Aug 2020 02:07:40 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"LivC5Qce\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1598227661;\n\tbh=1n7rM/fXDTojQ84Wh6q59vDha8BY8OjphO7ymluepeo=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=LivC5QceQrjOj43TCxWDWQf3535bJ8j4XgBxqu9eIrKZIu02XXFIjPlJeU8odaamT\n\t/G0JczqhATutui21sHgUf6jZ+5nbjknzqN1qwiYJOkIbNC5wYq28s9oO+hFvpp1J96\n\t7AQU7481w069njz6Sr/rVdlwoAVau3m5rWzA8nWk=","Date":"Mon, 24 Aug 2020 03:07:22 +0300","From":"Laurent Pinchart ","To":"Show Liu ","Message-ID":"<20200824000722.GA11768@pendragon.ideasonboard.com>","References":"<20200821161602.5093-1-show.liu@linaro.org>\n\t<20200821161602.5093-2-show.liu@linaro.org>\n\t<20200824000455.GQ6002@pendragon.ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200824000455.GQ6002@pendragon.ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v4 1/3] qcam: add OpenGL shader code\n\tas QT resource","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12145,"web_url":"https://patchwork.libcamera.org/comment/12145/","msgid":"","date":"2020-08-25T09:38:55","subject":"Re: [libcamera-devel] [PATCH v4 1/3] qcam: add OpenGL shader code\n\tas QT resource","submitter":{"id":24,"url":"https://patchwork.libcamera.org/api/people/24/","name":"Show Liu","email":"show.liu@linaro.org"},"content":"Hi Laurent,\n\nOn Mon, Aug 24, 2020 at 8:07 AM Laurent Pinchart <\nlaurent.pinchart@ideasonboard.com> wrote:\n\n> On Mon, Aug 24, 2020 at 03:04:56AM +0300, Laurent Pinchart wrote:\n> > Hi Shaw,\n>\n> Apologies for the typo.\n>\n> > Thank you for the patch.\n> >\n> > On Sat, Aug 22, 2020 at 12:16:00AM +0800, Show Liu wrote:\n> > > qcam: add OpenGL shader code as QT resource\n> >\n> > s/QT/Qt/\n> >\n> > > Signed-off-by: Show Liu \n> > > ---\n> > > src/qcam/assets/shader/NV_2_planes_UV_f.glsl | 32 +++++++++++++++++++\n> > > src/qcam/assets/shader/NV_2_planes_VU_f.glsl | 32 +++++++++++++++++++\n> > > src/qcam/assets/shader/NV_3_planes_UV_f.glsl | 33 ++++++++++++++++++++\n> > > src/qcam/assets/shader/NV_3_planes_VU_f.glsl | 33 ++++++++++++++++++++\n> > > src/qcam/assets/shader/NV_vertex_shader.glsl | 16 ++++++++++\n> > > src/qcam/assets/shader/shaders.qrc | 10 ++++++\n> > > src/qcam/meson.build | 1 +\n> > > 7 files changed, 157 insertions(+)\n> > > create mode 100644 src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> > > create mode 100644 src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> > > create mode 100644 src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> > > create mode 100644 src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> > > create mode 100644 src/qcam/assets/shader/NV_vertex_shader.glsl\n> > > create mode 100644 src/qcam/assets/shader/shaders.qrc\n> > >\n> > > diff --git a/src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> b/src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> > > new file mode 100644\n> > > index 0000000..32c6e90\n> > > --- /dev/null\n> > > +++ b/src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> > > @@ -0,0 +1,32 @@\n> > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > +/*\n> > > + * Copyright (C) 2020, Linaro\n> > > + *\n> > > + * NV_2_planes_UV_f.glsl - Fragment shader code for NV12, NV16 and\n> NV24 formats\n> > > + */\n> > > +\n> > > +#ifdef GL_ES\n> > > +precision highp float;\n> > > +#endif\n> > > +\n> > > +varying vec2 textureOut;\n> > > +uniform sampler2D tex_y;\n> > > +uniform sampler2D tex_u;\n> > > +\n> > > +void main(void)\n> > > +{\n> > > + vec3 yuv;\n> > > + vec3 rgb;\n> > > + mat3 convert_mat = mat3(\n> >\n> > Maybe yuv2rgb or yuv2rgb_mat instead of convert_mat ? Or yuv2rgb_bt601\n> > or yuv2rgb_bt601_mat ?\n>\nsure, I will fix it.\n\n> >\n> > > + vec3(1.1640625,\n> 1.1640625, 1.1640625),\n> > > + vec3(0.0,\n> -0.390625, 2.015625),\n> > > + vec3(1.5975625,\n> -0.8125, 0.0)\n> >\n> > Where are the coefficients from ? https://en.wikipedia.org/wiki/YCbCr\n> > lists values very similar but slightly different. Are the values here\n> > rounded to a multiple of 1/256 on purpose (except for 1.5975625) ?\n>\nThe coefficients matrix refer from the yuv_to_rgb function in\nformat_converter.cpp file\n298/256=1.1640625\nI just conert the elements into floating by dividing 256.\n\n> >\n> > > + );\n> >\n> > The indentation is a bit weird.\n> >\n> > > +\n> > > + yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> >\n> > Is 0.0625 (16/256) the right value, or should it be 16/255 ? The Y range\n> > on 8-bit is 0-255, which is mapped to 0.0-1.0, right ?\n> >\n> > > + yuv.y = texture2D(tex_u, textureOut).r - 0.5;\n> > > + yuv.z = texture2D(tex_u, textureOut).g - 0.5;\n> > > +\n> > > + rgb = convert_mat * yuv;\n> > > + gl_FragColor = vec4(rgb, 1.0);\n> > > +}\n> > > diff --git a/src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> b/src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> > > new file mode 100644\n> > > index 0000000..aae12de\n> > > --- /dev/null\n> > > +++ b/src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> > > @@ -0,0 +1,32 @@\n> > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > +/*\n> > > + * Copyright (C) 2020, Linaro\n> > > + *\n> > > + * NV_2_planes_VU_f.glsl - Fragment shader code for NV21, NV61 and\n> NV42 formats\n> > > + */\n> > > +\n> > > +#ifdef GL_ES\n> > > +precision highp float;\n> > > +#endif\n> > > +\n> > > +varying vec2 textureOut;\n> > > +uniform sampler2D tex_y;\n> > > +uniform sampler2D tex_u;\n> > > +\n> > > +void main(void)\n> > > +{\n> > > + vec3 yuv;\n> > > + vec3 rgb;\n> > > + mat3 convert_mat = mat3(\n> > > + vec3(1.1640625,\n> 1.1640625, 1.1640625),\n> > > + vec3(0.0,\n> -0.390625, 2.015625),\n> > > + vec3(1.5975625,\n> -0.8125, 0.0)\n> > > + );\n> >\n> > Here too.\n> >\n> > > +\n> > > + yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > > + yuv.y = texture2D(tex_u, textureOut).g - 0.5;\n> > > + yuv.z = texture2D(tex_u, textureOut).r - 0.5;\n> > > +\n> >\n> > And there are some white spaces at the end of lines (below too).\n>\nok. I will remove it.\n\n> >\n> > We'll have to make the coefficients configurable to support different\n> > colorspaces, but that's for later.\n> >\n> > > + rgb = convert_mat * yuv;\n> > > + gl_FragColor = vec4(rgb, 1.0);\n> > > +}\n> > > diff --git a/src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> b/src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> > > new file mode 100644\n> > > index 0000000..21fff3a\n> > > --- /dev/null\n> > > +++ b/src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> > > @@ -0,0 +1,33 @@\n> > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > +/*\n> > > + * Copyright (C) 2020, Linaro\n> > > + *\n> > > + * NV_3_planes_UV_f.glsl - Fragment shader code for YUV420 format\n> > > + */\n> > > +\n> > > +#ifdef GL_ES\n> > > +precision highp float;\n> > > +#endif\n> > > +\n> > > +varying vec2 textureOut;\n> > > +uniform sampler2D tex_y;\n> > > +uniform sampler2D tex_u;\n> > > +uniform sampler2D tex_v;\n> > > +\n> > > +void main(void)\n> > > +{\n> > > + vec3 yuv;\n> > > + vec3 rgb;\n> > > + mat3 convert_mat = mat3(\n> > > + vec3(1.1640625,\n> 1.1640625, 1.1640625),\n> > > + vec3(0.0,\n> -0.390625, 2.015625),\n> > > + vec3(1.5975625,\n> -0.8125, 0.0)\n> > > + );\n> > > +\n> > > + yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > > + yuv.y = texture2D(tex_u, textureOut).r - 0.5;\n> > > + yuv.z = texture2D(tex_v, textureOut).g - 0.5;\n> >\n> > I'm not sure to understand the .g here, with YUV420 having three planes,\n> > shouldn't it be .r ?\n\nYes. you are right. it's a problem here. I will fix it.\n\n> I could be wrong, my GLSL knowledge is very very\n> > rudimentary :-)\n>\nYou definitely are a GLSL expert...don't cheat me, I know that. :-)\n\n> >\n> > > +\n> > > + rgb = convert_mat * yuv;\n> > > + gl_FragColor = vec4(rgb, 1.0);\n> > > +}\n> > > diff --git a/src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> b/src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> > > new file mode 100644\n> > > index 0000000..df00170\n> > > --- /dev/null\n> > > +++ b/src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> > > @@ -0,0 +1,33 @@\n> > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > +/*\n> > > + * Copyright (C) 2020, Linaro\n> > > + *\n> > > + * NV_3_planes_VU_f.glsl - Fragment shader code for YVU420 format\n> > > + */\n> > > +\n> > > +#ifdef GL_ES\n> > > +precision highp float;\n> > > +#endif\n> > > +\n> > > +varying vec2 textureOut;\n> > > +uniform sampler2D tex_y;\n> > > +uniform sampler2D tex_u;\n> > > +uniform sampler2D tex_v;\n> > > +\n> > > +void main(void)\n> > > +{\n> > > + vec3 yuv;\n> > > + vec3 rgb;\n> > > + mat3 convert_mat = mat3(\n> > > + vec3(1.1640625,\n> 1.1640625, 1.1640625),\n> > > + vec3(0.0,\n> -0.390625, 2.015625),\n> > > + vec3(1.5975625,\n> -0.8125, 0.0)\n> > > + );\n> > > +\n> > > + yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > > + yuv.y = texture2D(tex_u, textureOut).g - 0.5;\n> > > + yuv.z = texture2D(tex_v, textureOut).r - 0.5;\n> >\n> > Same here, I was expecting\n>\nwill fix it.\n\n> >\n> > yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > yuv.y = texture2D(tex_v, textureOut).r - 0.5;\n> > yuv.z = texture2D(tex_u, textureOut).r - 0.5;\n> >\n> > > +\n> > > + rgb = convert_mat * yuv;\n> > > + gl_FragColor = vec4(rgb, 1.0);\n> > > +}\n> > > diff --git a/src/qcam/assets/shader/NV_vertex_shader.glsl\n> b/src/qcam/assets/shader/NV_vertex_shader.glsl\n> > > new file mode 100644\n> > > index 0000000..403b791\n> > > --- /dev/null\n> > > +++ b/src/qcam/assets/shader/NV_vertex_shader.glsl\n> > > @@ -0,0 +1,16 @@\n> > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > +/*\n> > > + * Copyright (C) 2020, Linaro\n> > > + *\n> > > + * NV_vertex_shader.glsl - Vertex shader code for NV family\n> > > + */\n> > > +\n> > > +attribute vec4 vertexIn;\n> > > +attribute vec2 textureIn;\n> > > +varying vec2 textureOut;\n> > > +\n> > > +void main(void)\n> > > +{\n> > > + gl_Position = vertexIn;\n> > > + textureOut = textureIn;\n> > > +}\n> > > diff --git a/src/qcam/assets/shader/shaders.qrc\n> b/src/qcam/assets/shader/shaders.qrc\n> > > new file mode 100644\n> > > index 0000000..6fe4c7f\n> > > --- /dev/null\n> > > +++ b/src/qcam/assets/shader/shaders.qrc\n> > > @@ -0,0 +1,10 @@\n> > > +\n> > > +\n> > > +\n> > > +./NV_vertex_shader.glsl\n> > > +./NV_2_planes_UV_f.glsl\n> > > +./NV_2_planes_VU_f.glsl\n> > > +./NV_3_planes_UV_f.glsl\n> > > +./NV_3_planes_VU_f.glsl\n> > > +\n> > > +\n> > > diff --git a/src/qcam/meson.build b/src/qcam/meson.build\n> > > index 6ea886a..e0c6f26 100644\n> > > --- a/src/qcam/meson.build\n> > > +++ b/src/qcam/meson.build\n> > > @@ -16,6 +16,7 @@ qcam_moc_headers = files([\n> > >\n> > > qcam_resources = files([\n> > > 'assets/feathericons/feathericons.qrc',\n> > > + 'assets/shader/shaders.qrc'\n> > > ])\n> > >\n> > > qt5 = import('qt5')\n>\n\nBest Regards,\nShow Liu\n\n>\n> --\n> Regards,\n>\n> Laurent Pinchart\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 7D20DBD87C\n\tfor ;\n\tTue, 25 Aug 2020 09:39:10 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D61AB62862;\n\tTue, 25 Aug 2020 11:39:09 +0200 (CEST)","from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com\n\t[IPv6:2607:f8b0:4864:20::1044])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A046F6037B\n\tfor ;\n\tTue, 25 Aug 2020 11:39:07 +0200 (CEST)","by mail-pj1-x1044.google.com with SMTP id nv17so951169pjb.3\n\tfor ;\n\tTue, 25 Aug 2020 02:39:07 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"Rq2mID7o\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=xdw2bMkpGy6yRtLng6xSxjJXWB+pPmp7SKbD6HPa+I4=;\n\tb=Rq2mID7oNcMQm0fdFlFvYbEF8azPkJnVFlmKGZmv8SauT6rkfnDSsNI0B9diizrzeI\n\t3sNU5EK4HU8moTt5jwxFgfMrM1V6wJ/ZjbpDMkYQzdtQJheRmt0CR3qcaR8cIQh4Rs8S\n\toCriOyS2k9q1CRIY1nioakPoY3MwtJmNL/O7tRal9p79jOaCdBrUI+XFa50ef7droCoZ\n\t4tgQ/6Vg+S8M8r0w1TVTl+76UAJxnumMbhwPYCwgPVLcX0ic2OeModiMBQPaSEYhPtHN\n\tFaR7Nu/zik8DD0jFlnIso5fRS68G7Cudy9K2a2A54VQae9RC7KliTWC1rJmqC8qBoEJ8\n\tD9jg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=xdw2bMkpGy6yRtLng6xSxjJXWB+pPmp7SKbD6HPa+I4=;\n\tb=VPomT3h4+Wd4AQV24F0T3tsLUGwZZK9kyHWkaI3Y3mRr8h3cn2MWN3QCKL/WR1wIJ9\n\tD+vbDUDZUrF/EIO/D3btS7ufSZXXbFJWN+hffFiBwWTV60rvXbqproumoZ1Yw2EuVtvd\n\tlxavUbP1ovNXfDcuKBW6+5B23hJF15JX39ZO5k2M93qlphmkaMT2T8VHY6NbLdYh7oA+\n\tUeXqiKGj/nNKGkmqYCjy+yDJUREs6qz4+TF+++gHyniD1OlimL+P0aGCXNwZw5jBHMc9\n\tOCFNtzvoNbY6xyQnibpoGeFW7OdcsPLYyVBBR9JjGpNqu7peVIgP3oNqDbZKzTfa7lxD\n\tB9cQ==","X-Gm-Message-State":"AOAM532dQJMKO8/6TlAje7wLrIAMOUd7czcWTq+81LTTUF/uZvnQWM6u\n\tBwloFfHY2HfAs7Taib0L3vj9FL/myZ397AZbpyir/w==","X-Google-Smtp-Source":"ABdhPJylD/V4OkiR3n4Ls6ty7BqnAtqCgIQ0HmUtnqJURY6l9RzJiPh+W1bYpva8E2ghGILBZTIpqulyiJgRny66IRw=","X-Received":"by 2002:a17:90b:a54:: with SMTP id\n\tgw20mr848150pjb.183.1598348346024; \n\tTue, 25 Aug 2020 02:39:06 -0700 (PDT)","MIME-Version":"1.0","References":"<20200821161602.5093-1-show.liu@linaro.org>\n\t<20200821161602.5093-2-show.liu@linaro.org>\n\t<20200824000455.GQ6002@pendragon.ideasonboard.com>\n\t<20200824000722.GA11768@pendragon.ideasonboard.com>","In-Reply-To":"<20200824000722.GA11768@pendragon.ideasonboard.com>","From":"Show Liu ","Date":"Tue, 25 Aug 2020 17:38:55 +0800","Message-ID":"","To":"Laurent Pinchart ","Subject":"Re: [libcamera-devel] [PATCH v4 1/3] qcam: add OpenGL shader code\n\tas QT resource","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"multipart/mixed;\n\tboundary=\"===============2816724327003459529==\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12147,"web_url":"https://patchwork.libcamera.org/comment/12147/","msgid":"<20200825193722.GJ6767@pendragon.ideasonboard.com>","date":"2020-08-25T19:37:22","subject":"Re: [libcamera-devel] [PATCH v4 1/3] qcam: add OpenGL shader code\n\tas QT resource","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Show,\n\nOn Tue, Aug 25, 2020 at 05:38:55PM +0800, Show Liu wrote:\n> On Mon, Aug 24, 2020 at 8:07 AM Laurent Pinchart wrote:\n> > On Mon, Aug 24, 2020 at 03:04:56AM +0300, Laurent Pinchart wrote:\n> > > On Sat, Aug 22, 2020 at 12:16:00AM +0800, Show Liu wrote:\n> > > > qcam: add OpenGL shader code as QT resource\n> > >\n> > > s/QT/Qt/\n> > >\n> > > > Signed-off-by: Show Liu \n> > > > ---\n> > > > src/qcam/assets/shader/NV_2_planes_UV_f.glsl | 32 +++++++++++++++++++\n> > > > src/qcam/assets/shader/NV_2_planes_VU_f.glsl | 32 +++++++++++++++++++\n> > > > src/qcam/assets/shader/NV_3_planes_UV_f.glsl | 33 ++++++++++++++++++++\n> > > > src/qcam/assets/shader/NV_3_planes_VU_f.glsl | 33 ++++++++++++++++++++\n> > > > src/qcam/assets/shader/NV_vertex_shader.glsl | 16 ++++++++++\n> > > > src/qcam/assets/shader/shaders.qrc | 10 ++++++\n> > > > src/qcam/meson.build | 1 +\n> > > > 7 files changed, 157 insertions(+)\n> > > > create mode 100644 src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> > > > create mode 100644 src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> > > > create mode 100644 src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> > > > create mode 100644 src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> > > > create mode 100644 src/qcam/assets/shader/NV_vertex_shader.glsl\n> > > > create mode 100644 src/qcam/assets/shader/shaders.qrc\n> > > >\n> > > > diff --git a/src/qcam/assets/shader/NV_2_planes_UV_f.glsl b/src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> > > > new file mode 100644\n> > > > index 0000000..32c6e90\n> > > > --- /dev/null\n> > > > +++ b/src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> > > > @@ -0,0 +1,32 @@\n> > > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > > +/*\n> > > > + * Copyright (C) 2020, Linaro\n> > > > + *\n> > > > + * NV_2_planes_UV_f.glsl - Fragment shader code for NV12, NV16 and NV24 formats\n> > > > + */\n> > > > +\n> > > > +#ifdef GL_ES\n> > > > +precision highp float;\n> > > > +#endif\n> > > > +\n> > > > +varying vec2 textureOut;\n> > > > +uniform sampler2D tex_y;\n> > > > +uniform sampler2D tex_u;\n> > > > +\n> > > > +void main(void)\n> > > > +{\n> > > > + vec3 yuv;\n> > > > + vec3 rgb;\n> > > > + mat3 convert_mat = mat3(\n> > >\n> > > Maybe yuv2rgb or yuv2rgb_mat instead of convert_mat ? Or yuv2rgb_bt601\n> > > or yuv2rgb_bt601_mat ?\n>\n> sure, I will fix it.\n> \n> > > > + vec3(1.1640625, 1.1640625, 1.1640625),\n> > > > + vec3(0.0, -0.390625, 2.015625),\n> > > > + vec3(1.5975625, -0.8125, 0.0)\n> > >\n> > > Where are the coefficients from ? https://en.wikipedia.org/wiki/YCbCr\n> > > lists values very similar but slightly different. Are the values here\n> > > rounded to a multiple of 1/256 on purpose (except for 1.5975625) ?\n>\n> The coefficients matrix refer from the yuv_to_rgb function in\n> format_converter.cpp file\n> 298/256=1.1640625\n> I just conert the elements into floating by dividing 256.\n\nOf course. I should have looked there :-) That code uses fixed-point\ncalculations to speed things up. As we can use floating point values\nhere, let's specify the exact values where possible.\n\nCan we use arithemtic expression to let the shader compiler pick the\nhighest precision ?\n\n\t\tvec3(255.0/219.0, 255.0/219.0, 255.0/219.0),\n\t\tvec3(0.0, -255.0/224.0*1.772*0.114/0.587, 255.0/224.0*1.772),\n\t\tvec3(255.0/224.0*1.402, -255.0/224.0*1.402*0.299/0.587, 0.0)\n\nor should we use numerical values directly ? I suppose a 8-bit precision\nwill be enough in practice as that's what we have in the input image and\nwhat we will render, so rounding after 3 decimal points should be fine\nand would probably be more readable.\n\n\t\tvec3(1.164, 1.164, 1.164),\n\t\tvec3(0.000, -0.392, 2.017),\n\t\tvec3(1.596, -0.813, 0.000)\n\nWhat do you think ?\n\n> > > > + );\n> > >\n> > > The indentation is a bit weird.\n> > >\n> > > > +\n> > > > + yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > >\n> > > Is 0.0625 (16/256) the right value, or should it be 16/255 ? The Y range\n> > > on 8-bit is 0-255, which is mapped to 0.0-1.0, right ?\n> > >\n> > > > + yuv.y = texture2D(tex_u, textureOut).r - 0.5;\n> > > > + yuv.z = texture2D(tex_u, textureOut).g - 0.5;\n> > > > +\n> > > > + rgb = convert_mat * yuv;\n> > > > + gl_FragColor = vec4(rgb, 1.0);\n> > > > +}\n> > > > diff --git a/src/qcam/assets/shader/NV_2_planes_VU_f.glsl b/src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> > > > new file mode 100644\n> > > > index 0000000..aae12de\n> > > > --- /dev/null\n> > > > +++ b/src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> > > > @@ -0,0 +1,32 @@\n> > > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > > +/*\n> > > > + * Copyright (C) 2020, Linaro\n> > > > + *\n> > > > + * NV_2_planes_VU_f.glsl - Fragment shader code for NV21, NV61 and NV42 formats\n> > > > + */\n> > > > +\n> > > > +#ifdef GL_ES\n> > > > +precision highp float;\n> > > > +#endif\n> > > > +\n> > > > +varying vec2 textureOut;\n> > > > +uniform sampler2D tex_y;\n> > > > +uniform sampler2D tex_u;\n> > > > +\n> > > > +void main(void)\n> > > > +{\n> > > > + vec3 yuv;\n> > > > + vec3 rgb;\n> > > > + mat3 convert_mat = mat3(\n> > > > + vec3(1.1640625, 1.1640625, 1.1640625),\n> > > > + vec3(0.0, -0.390625, 2.015625),\n> > > > + vec3(1.5975625, -0.8125, 0.0)\n> > > > + );\n> > >\n> > > Here too.\n> > >\n> > > > +\n> > > > + yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > > > + yuv.y = texture2D(tex_u, textureOut).g - 0.5;\n> > > > + yuv.z = texture2D(tex_u, textureOut).r - 0.5;\n> > > > +\n> > >\n> > > And there are some white spaces at the end of lines (below too).\n>\n> ok. I will remove it.\n> \n> > >\n> > > We'll have to make the coefficients configurable to support different\n> > > colorspaces, but that's for later.\n> > >\n> > > > + rgb = convert_mat * yuv;\n> > > > + gl_FragColor = vec4(rgb, 1.0);\n> > > > +}\n> > > > diff --git a/src/qcam/assets/shader/NV_3_planes_UV_f.glsl b/src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> > > > new file mode 100644\n> > > > index 0000000..21fff3a\n> > > > --- /dev/null\n> > > > +++ b/src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> > > > @@ -0,0 +1,33 @@\n> > > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > > +/*\n> > > > + * Copyright (C) 2020, Linaro\n> > > > + *\n> > > > + * NV_3_planes_UV_f.glsl - Fragment shader code for YUV420 format\n> > > > + */\n> > > > +\n> > > > +#ifdef GL_ES\n> > > > +precision highp float;\n> > > > +#endif\n> > > > +\n> > > > +varying vec2 textureOut;\n> > > > +uniform sampler2D tex_y;\n> > > > +uniform sampler2D tex_u;\n> > > > +uniform sampler2D tex_v;\n> > > > +\n> > > > +void main(void)\n> > > > +{\n> > > > + vec3 yuv;\n> > > > + vec3 rgb;\n> > > > + mat3 convert_mat = mat3(\n> > > > + vec3(1.1640625, 1.1640625, 1.1640625),\n> > > > + vec3(0.0, -0.390625, 2.015625),\n> > > > + vec3(1.5975625, -0.8125, 0.0)\n> > > > + );\n> > > > +\n> > > > + yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > > > + yuv.y = texture2D(tex_u, textureOut).r - 0.5;\n> > > > + yuv.z = texture2D(tex_v, textureOut).g - 0.5;\n> > >\n> > > I'm not sure to understand the .g here, with YUV420 having three planes,\n> > > shouldn't it be .r ?\n> \n> Yes. you are right. it's a problem here. I will fix it.\n> \n> > I could be wrong, my GLSL knowledge is very very rudimentary :-)\n>\n> You definitely are a GLSL expert...don't cheat me, I know that. :-)\n> \n> > > > +\n> > > > + rgb = convert_mat * yuv;\n> > > > + gl_FragColor = vec4(rgb, 1.0);\n> > > > +}\n> > > > diff --git a/src/qcam/assets/shader/NV_3_planes_VU_f.glsl b/src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> > > > new file mode 100644\n> > > > index 0000000..df00170\n> > > > --- /dev/null\n> > > > +++ b/src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> > > > @@ -0,0 +1,33 @@\n> > > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > > +/*\n> > > > + * Copyright (C) 2020, Linaro\n> > > > + *\n> > > > + * NV_3_planes_VU_f.glsl - Fragment shader code for YVU420 format\n> > > > + */\n> > > > +\n> > > > +#ifdef GL_ES\n> > > > +precision highp float;\n> > > > +#endif\n> > > > +\n> > > > +varying vec2 textureOut;\n> > > > +uniform sampler2D tex_y;\n> > > > +uniform sampler2D tex_u;\n> > > > +uniform sampler2D tex_v;\n> > > > +\n> > > > +void main(void)\n> > > > +{\n> > > > + vec3 yuv;\n> > > > + vec3 rgb;\n> > > > + mat3 convert_mat = mat3(\n> > > > + vec3(1.1640625, 1.1640625, 1.1640625),\n> > > > + vec3(0.0, -0.390625, 2.015625),\n> > > > + vec3(1.5975625, -0.8125, 0.0)\n> > > > + );\n> > > > +\n> > > > + yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > > > + yuv.y = texture2D(tex_u, textureOut).g - 0.5;\n> > > > + yuv.z = texture2D(tex_v, textureOut).r - 0.5;\n> > >\n> > > Same here, I was expecting\n>\n> will fix it.\n> \n> > > yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > > yuv.y = texture2D(tex_v, textureOut).r - 0.5;\n> > > yuv.z = texture2D(tex_u, textureOut).r - 0.5;\n> > >\n> > > > +\n> > > > + rgb = convert_mat * yuv;\n> > > > + gl_FragColor = vec4(rgb, 1.0);\n> > > > +}\n> > > > diff --git a/src/qcam/assets/shader/NV_vertex_shader.glsl\n> > b/src/qcam/assets/shader/NV_vertex_shader.glsl\n> > > > new file mode 100644\n> > > > index 0000000..403b791\n> > > > --- /dev/null\n> > > > +++ b/src/qcam/assets/shader/NV_vertex_shader.glsl\n> > > > @@ -0,0 +1,16 @@\n> > > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > > +/*\n> > > > + * Copyright (C) 2020, Linaro\n> > > > + *\n> > > > + * NV_vertex_shader.glsl - Vertex shader code for NV family\n> > > > + */\n> > > > +\n> > > > +attribute vec4 vertexIn;\n> > > > +attribute vec2 textureIn;\n> > > > +varying vec2 textureOut;\n> > > > +\n> > > > +void main(void)\n> > > > +{\n> > > > + gl_Position = vertexIn;\n> > > > + textureOut = textureIn;\n> > > > +}\n> > > > diff --git a/src/qcam/assets/shader/shaders.qrc\n> > b/src/qcam/assets/shader/shaders.qrc\n> > > > new file mode 100644\n> > > > index 0000000..6fe4c7f\n> > > > --- /dev/null\n> > > > +++ b/src/qcam/assets/shader/shaders.qrc\n> > > > @@ -0,0 +1,10 @@\n> > > > +\n> > > > +\n> > > > +\n> > > > +./NV_vertex_shader.glsl\n> > > > +./NV_2_planes_UV_f.glsl\n> > > > +./NV_2_planes_VU_f.glsl\n> > > > +./NV_3_planes_UV_f.glsl\n> > > > +./NV_3_planes_VU_f.glsl\n> > > > +\n> > > > +\n> > > > diff --git a/src/qcam/meson.build b/src/qcam/meson.build\n> > > > index 6ea886a..e0c6f26 100644\n> > > > --- a/src/qcam/meson.build\n> > > > +++ b/src/qcam/meson.build\n> > > > @@ -16,6 +16,7 @@ qcam_moc_headers = files([\n> > > >\n> > > > qcam_resources = files([\n> > > > 'assets/feathericons/feathericons.qrc',\n> > > > + 'assets/shader/shaders.qrc'\n> > > > ])\n> > > >\n> > > > qt5 = import('qt5')","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id CBACEBE173\n\tfor ;\n\tTue, 25 Aug 2020 19:37:44 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 5FE9962882;\n\tTue, 25 Aug 2020 21:37:44 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C1D826037C\n\tfor ;\n\tTue, 25 Aug 2020 21:37:42 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 3D7D59A5;\n\tTue, 25 Aug 2020 21:37:42 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"EUAxwhQv\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1598384262;\n\tbh=O8H4BqmRQNWmkzpDoPPGwiMwwb4zyae9o/xf5wXEHYM=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=EUAxwhQvagqV4vn/m2xwZSCdd8y3zoZO1sojg1GIa90j+rKmQ/PlPL3mDHPFShR0/\n\tG6gQVnVj8xHewli0g/3Qe8+hO0F0oggZVvpp/KVTvqyYewNCCLUQcufSRc+R9RynNq\n\tSNJ6GFkiIQbv1bh7wdtpwP38Ls8C0LZizLk16dlw=","Date":"Tue, 25 Aug 2020 22:37:22 +0300","From":"Laurent Pinchart ","To":"Show Liu ","Message-ID":"<20200825193722.GJ6767@pendragon.ideasonboard.com>","References":"<20200821161602.5093-1-show.liu@linaro.org>\n\t<20200821161602.5093-2-show.liu@linaro.org>\n\t<20200824000455.GQ6002@pendragon.ideasonboard.com>\n\t<20200824000722.GA11768@pendragon.ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH v4 1/3] qcam: add OpenGL shader code\n\tas QT resource","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12155,"web_url":"https://patchwork.libcamera.org/comment/12155/","msgid":"","date":"2020-08-26T06:34:45","subject":"Re: [libcamera-devel] [PATCH v4 1/3] qcam: add OpenGL shader code\n\tas QT resource","submitter":{"id":24,"url":"https://patchwork.libcamera.org/api/people/24/","name":"Show Liu","email":"show.liu@linaro.org"},"content":"Hi Laurent,\n\n\nOn Wed, Aug 26, 2020 at 3:37 AM Laurent Pinchart <\nlaurent.pinchart@ideasonboard.com> wrote:\n\n> Hi Show,\n>\n> On Tue, Aug 25, 2020 at 05:38:55PM +0800, Show Liu wrote:\n> > On Mon, Aug 24, 2020 at 8:07 AM Laurent Pinchart wrote:\n> > > On Mon, Aug 24, 2020 at 03:04:56AM +0300, Laurent Pinchart wrote:\n> > > > On Sat, Aug 22, 2020 at 12:16:00AM +0800, Show Liu wrote:\n> > > > > qcam: add OpenGL shader code as QT resource\n> > > >\n> > > > s/QT/Qt/\n> > > >\n> > > > > Signed-off-by: Show Liu \n> > > > > ---\n> > > > > src/qcam/assets/shader/NV_2_planes_UV_f.glsl | 32\n> +++++++++++++++++++\n> > > > > src/qcam/assets/shader/NV_2_planes_VU_f.glsl | 32\n> +++++++++++++++++++\n> > > > > src/qcam/assets/shader/NV_3_planes_UV_f.glsl | 33\n> ++++++++++++++++++++\n> > > > > src/qcam/assets/shader/NV_3_planes_VU_f.glsl | 33\n> ++++++++++++++++++++\n> > > > > src/qcam/assets/shader/NV_vertex_shader.glsl | 16 ++++++++++\n> > > > > src/qcam/assets/shader/shaders.qrc | 10 ++++++\n> > > > > src/qcam/meson.build | 1 +\n> > > > > 7 files changed, 157 insertions(+)\n> > > > > create mode 100644 src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> > > > > create mode 100644 src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> > > > > create mode 100644 src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> > > > > create mode 100644 src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> > > > > create mode 100644 src/qcam/assets/shader/NV_vertex_shader.glsl\n> > > > > create mode 100644 src/qcam/assets/shader/shaders.qrc\n> > > > >\n> > > > > diff --git a/src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> b/src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> > > > > new file mode 100644\n> > > > > index 0000000..32c6e90\n> > > > > --- /dev/null\n> > > > > +++ b/src/qcam/assets/shader/NV_2_planes_UV_f.glsl\n> > > > > @@ -0,0 +1,32 @@\n> > > > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > > > +/*\n> > > > > + * Copyright (C) 2020, Linaro\n> > > > > + *\n> > > > > + * NV_2_planes_UV_f.glsl - Fragment shader code for NV12, NV16\n> and NV24 formats\n> > > > > + */\n> > > > > +\n> > > > > +#ifdef GL_ES\n> > > > > +precision highp float;\n> > > > > +#endif\n> > > > > +\n> > > > > +varying vec2 textureOut;\n> > > > > +uniform sampler2D tex_y;\n> > > > > +uniform sampler2D tex_u;\n> > > > > +\n> > > > > +void main(void)\n> > > > > +{\n> > > > > + vec3 yuv;\n> > > > > + vec3 rgb;\n> > > > > + mat3 convert_mat = mat3(\n> > > >\n> > > > Maybe yuv2rgb or yuv2rgb_mat instead of convert_mat ? Or\n> yuv2rgb_bt601\n> > > > or yuv2rgb_bt601_mat ?\n> >\n> > sure, I will fix it.\n> >\n> > > > > +\n> vec3(1.1640625, 1.1640625, 1.1640625),\n> > > > > + vec3(0.0,\n> -0.390625, 2.015625),\n> > > > > +\n> vec3(1.5975625, -0.8125, 0.0)\n> > > >\n> > > > Where are the coefficients from ?\n> https://en.wikipedia.org/wiki/YCbCr\n> > > > lists values very similar but slightly different. Are the values here\n> > > > rounded to a multiple of 1/256 on purpose (except for 1.5975625) ?\n> >\n> > The coefficients matrix refer from the yuv_to_rgb function in\n> > format_converter.cpp file\n> > 298/256=1.1640625\n> > I just conert the elements into floating by dividing 256.\n>\n> Of course. I should have looked there :-) That code uses fixed-point\n> calculations to speed things up. As we can use floating point values\n> here, let's specify the exact values where possible.\n>\n> Can we use arithemtic expression to let the shader compiler pick the\n> highest precision ?\n>\n> vec3(255.0/219.0, 255.0/219.0, 255.0/219.0),\n> vec3(0.0, -255.0/224.0*1.772*0.114/0.587,\n> 255.0/224.0*1.772),\n> vec3(255.0/224.0*1.402, -255.0/224.0*1.402*0.299/0.587,\n> 0.0)\n>\n> or should we use numerical values directly ? I suppose a 8-bit precision\n> will be enough in practice as that's what we have in the input image and\n> what we will render, so rounding after 3 decimal points should be fine\n> and would probably be more readable.\n>\n> vec3(1.164, 1.164, 1.164),\n> vec3(0.000, -0.392, 2.017),\n> vec3(1.596, -0.813, 0.000)\n>\n> What do you think ?\n>\n\nSure. I will try to apply it on the next version.\n\n\nThanks,\nShow\n\n>\n> > > > > + );\n> > > >\n> > > > The indentation is a bit weird.\n> > > >\n> > > > > +\n> > > > > + yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > > >\n> > > > Is 0.0625 (16/256) the right value, or should it be 16/255 ? The Y\n> range\n> > > > on 8-bit is 0-255, which is mapped to 0.0-1.0, right ?\n> > > >\n> > > > > + yuv.y = texture2D(tex_u, textureOut).r - 0.5;\n> > > > > + yuv.z = texture2D(tex_u, textureOut).g - 0.5;\n> > > > > +\n> > > > > + rgb = convert_mat * yuv;\n> > > > > + gl_FragColor = vec4(rgb, 1.0);\n> > > > > +}\n> > > > > diff --git a/src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> b/src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> > > > > new file mode 100644\n> > > > > index 0000000..aae12de\n> > > > > --- /dev/null\n> > > > > +++ b/src/qcam/assets/shader/NV_2_planes_VU_f.glsl\n> > > > > @@ -0,0 +1,32 @@\n> > > > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > > > +/*\n> > > > > + * Copyright (C) 2020, Linaro\n> > > > > + *\n> > > > > + * NV_2_planes_VU_f.glsl - Fragment shader code for NV21, NV61\n> and NV42 formats\n> > > > > + */\n> > > > > +\n> > > > > +#ifdef GL_ES\n> > > > > +precision highp float;\n> > > > > +#endif\n> > > > > +\n> > > > > +varying vec2 textureOut;\n> > > > > +uniform sampler2D tex_y;\n> > > > > +uniform sampler2D tex_u;\n> > > > > +\n> > > > > +void main(void)\n> > > > > +{\n> > > > > + vec3 yuv;\n> > > > > + vec3 rgb;\n> > > > > + mat3 convert_mat = mat3(\n> > > > > +\n> vec3(1.1640625, 1.1640625, 1.1640625),\n> > > > > + vec3(0.0,\n> -0.390625, 2.015625),\n> > > > > +\n> vec3(1.5975625, -0.8125, 0.0)\n> > > > > + );\n> > > >\n> > > > Here too.\n> > > >\n> > > > > +\n> > > > > + yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > > > > + yuv.y = texture2D(tex_u, textureOut).g - 0.5;\n> > > > > + yuv.z = texture2D(tex_u, textureOut).r - 0.5;\n> > > > > +\n> > > >\n> > > > And there are some white spaces at the end of lines (below too).\n> >\n> > ok. I will remove it.\n> >\n> > > >\n> > > > We'll have to make the coefficients configurable to support different\n> > > > colorspaces, but that's for later.\n> > > >\n> > > > > + rgb = convert_mat * yuv;\n> > > > > + gl_FragColor = vec4(rgb, 1.0);\n> > > > > +}\n> > > > > diff --git a/src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> b/src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> > > > > new file mode 100644\n> > > > > index 0000000..21fff3a\n> > > > > --- /dev/null\n> > > > > +++ b/src/qcam/assets/shader/NV_3_planes_UV_f.glsl\n> > > > > @@ -0,0 +1,33 @@\n> > > > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > > > +/*\n> > > > > + * Copyright (C) 2020, Linaro\n> > > > > + *\n> > > > > + * NV_3_planes_UV_f.glsl - Fragment shader code for YUV420 format\n> > > > > + */\n> > > > > +\n> > > > > +#ifdef GL_ES\n> > > > > +precision highp float;\n> > > > > +#endif\n> > > > > +\n> > > > > +varying vec2 textureOut;\n> > > > > +uniform sampler2D tex_y;\n> > > > > +uniform sampler2D tex_u;\n> > > > > +uniform sampler2D tex_v;\n> > > > > +\n> > > > > +void main(void)\n> > > > > +{\n> > > > > + vec3 yuv;\n> > > > > + vec3 rgb;\n> > > > > + mat3 convert_mat = mat3(\n> > > > > +\n> vec3(1.1640625, 1.1640625, 1.1640625),\n> > > > > + vec3(0.0,\n> -0.390625, 2.015625),\n> > > > > +\n> vec3(1.5975625, -0.8125, 0.0)\n> > > > > + );\n> > > > > +\n> > > > > + yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > > > > + yuv.y = texture2D(tex_u, textureOut).r - 0.5;\n> > > > > + yuv.z = texture2D(tex_v, textureOut).g - 0.5;\n> > > >\n> > > > I'm not sure to understand the .g here, with YUV420 having three\n> planes,\n> > > > shouldn't it be .r ?\n> >\n> > Yes. you are right. it's a problem here. I will fix it.\n> >\n> > > I could be wrong, my GLSL knowledge is very very rudimentary :-)\n> >\n> > You definitely are a GLSL expert...don't cheat me, I know that. :-)\n> >\n> > > > > +\n> > > > > + rgb = convert_mat * yuv;\n> > > > > + gl_FragColor = vec4(rgb, 1.0);\n> > > > > +}\n> > > > > diff --git a/src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> b/src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> > > > > new file mode 100644\n> > > > > index 0000000..df00170\n> > > > > --- /dev/null\n> > > > > +++ b/src/qcam/assets/shader/NV_3_planes_VU_f.glsl\n> > > > > @@ -0,0 +1,33 @@\n> > > > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > > > +/*\n> > > > > + * Copyright (C) 2020, Linaro\n> > > > > + *\n> > > > > + * NV_3_planes_VU_f.glsl - Fragment shader code for YVU420 format\n> > > > > + */\n> > > > > +\n> > > > > +#ifdef GL_ES\n> > > > > +precision highp float;\n> > > > > +#endif\n> > > > > +\n> > > > > +varying vec2 textureOut;\n> > > > > +uniform sampler2D tex_y;\n> > > > > +uniform sampler2D tex_u;\n> > > > > +uniform sampler2D tex_v;\n> > > > > +\n> > > > > +void main(void)\n> > > > > +{\n> > > > > + vec3 yuv;\n> > > > > + vec3 rgb;\n> > > > > + mat3 convert_mat = mat3(\n> > > > > +\n> vec3(1.1640625, 1.1640625, 1.1640625),\n> > > > > + vec3(0.0,\n> -0.390625, 2.015625),\n> > > > > +\n> vec3(1.5975625, -0.8125, 0.0)\n> > > > > + );\n> > > > > +\n> > > > > + yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > > > > + yuv.y = texture2D(tex_u, textureOut).g - 0.5;\n> > > > > + yuv.z = texture2D(tex_v, textureOut).r - 0.5;\n> > > >\n> > > > Same here, I was expecting\n> >\n> > will fix it.\n> >\n> > > > yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\n> > > > yuv.y = texture2D(tex_v, textureOut).r - 0.5;\n> > > > yuv.z = texture2D(tex_u, textureOut).r - 0.5;\n> > > >\n> > > > > +\n> > > > > + rgb = convert_mat * yuv;\n> > > > > + gl_FragColor = vec4(rgb, 1.0);\n> > > > > +}\n> > > > > diff --git a/src/qcam/assets/shader/NV_vertex_shader.glsl\n> > > b/src/qcam/assets/shader/NV_vertex_shader.glsl\n> > > > > new file mode 100644\n> > > > > index 0000000..403b791\n> > > > > --- /dev/null\n> > > > > +++ b/src/qcam/assets/shader/NV_vertex_shader.glsl\n> > > > > @@ -0,0 +1,16 @@\n> > > > > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > > > > +/*\n> > > > > + * Copyright (C) 2020, Linaro\n> > > > > + *\n> > > > > + * NV_vertex_shader.glsl - Vertex shader code for NV family\n> > > > > + */\n> > > > > +\n> > > > > +attribute vec4 vertexIn;\n> > > > > +attribute vec2 textureIn;\n> > > > > +varying vec2 textureOut;\n> > > > > +\n> > > > > +void main(void)\n> > > > > +{\n> > > > > + gl_Position = vertexIn;\n> > > > > + textureOut = textureIn;\n> > > > > +}\n> > > > > diff --git a/src/qcam/assets/shader/shaders.qrc\n> > > b/src/qcam/assets/shader/shaders.qrc\n> > > > > new file mode 100644\n> > > > > index 0000000..6fe4c7f\n> > > > > --- /dev/null\n> > > > > +++ b/src/qcam/assets/shader/shaders.qrc\n> > > > > @@ -0,0 +1,10 @@\n> > > > > +\n> > > > > +\n> > > > > +\n> > > > > +./NV_vertex_shader.glsl\n> > > > > +./NV_2_planes_UV_f.glsl\n> > > > > +./NV_2_planes_VU_f.glsl\n> > > > > +./NV_3_planes_UV_f.glsl\n> > > > > +./NV_3_planes_VU_f.glsl\n> > > > > +\n> > > > > +\n> > > > > diff --git a/src/qcam/meson.build b/src/qcam/meson.build\n> > > > > index 6ea886a..e0c6f26 100644\n> > > > > --- a/src/qcam/meson.build\n> > > > > +++ b/src/qcam/meson.build\n> > > > > @@ -16,6 +16,7 @@ qcam_moc_headers = files([\n> > > > >\n> > > > > qcam_resources = files([\n> > > > > 'assets/feathericons/feathericons.qrc',\n> > > > > + 'assets/shader/shaders.qrc'\n> > > > > ])\n> > > > >\n> > > > > qt5 = import('qt5')\n>\n> --\n> Regards,\n>\n> Laurent Pinchart\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 4CC0BBD87E\n\tfor ;\n\tWed, 26 Aug 2020 06:35:00 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C294C628E6;\n\tWed, 26 Aug 2020 08:34:59 +0200 (CEST)","from mail-pl1-x644.google.com (mail-pl1-x644.google.com\n\t[IPv6:2607:f8b0:4864:20::644])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id CEFFC61EA0\n\tfor ;\n\tWed, 26 Aug 2020 08:34:57 +0200 (CEST)","by mail-pl1-x644.google.com with SMTP id k13so420270plk.13\n\tfor ;\n\tTue, 25 Aug 2020 23:34:57 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"BFeVuYde\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=eLKJr44Pz3Em0OIp44nNj6jz+DLHa+uOSCdsBBreKe0=;\n\tb=BFeVuYdet4/NmvJqoEN8K7U8aG41BMUpB7tnx0xbtRE6s9sWeHn0n2TD1Ds39odvYk\n\tFV3kvfdk9TGQaaDZRuY0gRQT5GizZoGmZf7iWJaqu8CM5AscsFG6lnxUH5CNaTTqPXHv\n\tEd1jME6sdDUO5TEJgR+5AfBQifjIVIlcXkxQB7nJPygJ6Ve7h6a08u1Xijym+tzyb8AP\n\tNE5xyC6BlVofmu71bqHRBU/eZJs36tfuiOyJq/cIVaKx9huHluvzGpU4FfLLpzRasmRk\n\t0S1KViRDRb9Te1gJSM8DCqemtjW7Cswy6tCUHLwf48r2dcsQvySKX6bL6NdyoivoPKT0\n\t7gOw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=eLKJr44Pz3Em0OIp44nNj6jz+DLHa+uOSCdsBBreKe0=;\n\tb=TV3QMTeAx2C6kmnYiICd02DAqJW32J1ykavwZm1kEIEwkDrTGGCEEMPDQAd7vynMdm\n\tvkcgmmtDDchDuajTAXmwHuwEZd1NdoYmbJVbfFDHg5Wmq+62RcYmgAe+wQFeUGhw12J4\n\tvXMXaEZU1mryHMI+8GIZk1MVqo3JlyExkLP6uH6yAx40bRPhVWFclNvJA7b6cSQ3Ns/x\n\tTUEWAX1r4X8mQHFLT9zWRC6MJ2maODpCmlx5Ip1bQAbqc2PG4U7vuO4RVO01rzQYkJPQ\n\t1UkMpmmdAomedqSBQiUvob0woc75/tn1U0nzzOTygaxQNRq7LcVXqIlFS4SNnu+ZnK0z\n\td3GQ==","X-Gm-Message-State":"AOAM533XkwDc6EeIjaMQj+aQ4p7ZGRfE9VO60Sght/CX5SOK/YMvkZDD\n\tgBi0SFlz/1IuJnvJ+N6xCjINe5YFrfN+ymzaLUPdS9LtCRI=","X-Google-Smtp-Source":"ABdhPJxgAdLeZASyNaNCpRsfJQ3/tUvHj9dhnSFS1OM2uO6Qrj4duSmJyqXyTUIPomHNY5Mpn93vX37XILxFUxkiN1o=","X-Received":"by 2002:a17:902:d702:: with SMTP id\n\tw2mr10589857ply.53.1598423695944; \n\tTue, 25 Aug 2020 23:34:55 -0700 (PDT)","MIME-Version":"1.0","References":"<20200821161602.5093-1-show.liu@linaro.org>\n\t<20200821161602.5093-2-show.liu@linaro.org>\n\t<20200824000455.GQ6002@pendragon.ideasonboard.com>\n\t<20200824000722.GA11768@pendragon.ideasonboard.com>\n\t\n\t<20200825193722.GJ6767@pendragon.ideasonboard.com>","In-Reply-To":"<20200825193722.GJ6767@pendragon.ideasonboard.com>","From":"Show Liu ","Date":"Wed, 26 Aug 2020 14:34:45 +0800","Message-ID":"","To":"Laurent Pinchart ","Subject":"Re: [libcamera-devel] [PATCH v4 1/3] qcam: add OpenGL shader code\n\tas QT resource","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"multipart/mixed;\n\tboundary=\"===============1052457303300829875==\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]