[{"id":18506,"web_url":"https://patchwork.libcamera.org/comment/18506/","msgid":"","date":"2021-08-03T09:31:23","subject":"Re: [libcamera-devel] [PATCH] qcam: Support OpenGL ES 2.0","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Laurent,\n\nOn 03/08/2021 10:23, Laurent Pinchart wrote:\n> The GL_RG and GL_RED texture formats are not supported in OpenGL ES\n> prior to 3.0. In order to be compatible with OpenGL ES 2.0, use\n> GL_LUMINANCE_ALPHA and GL_LUMINANCE instead. The shader code needs to be\n> updated accordingly for GL_RG, as the second component is now stored in\n\n/in alpha/in the alpha/\n\n> alpha component instead of the green component. Usage of the red\n> component is fine, the luminance value is stored in the red, green and\n> blue components.\n> \n\nSo these seem to be just fairly arbitrary identifiers of the texture\nmaps to use, which as long as they are consistently referenced that's\nall that matters?\n\n\n> Signed-off-by: Laurent Pinchart \n> ---\n> Tested with vivid, using NV12, YV420, YVU420 and SGRBG8, which should\n> cover all code paths.\n> ---\n> src/qcam/assets/shader/YUV_2_planes.frag | 4 +--\n> src/qcam/viewfinder_gl.cpp | 40 ++++++++++++------------\n> 2 files changed, 22 insertions(+), 22 deletions(-)\n> \n> diff --git a/src/qcam/assets/shader/YUV_2_planes.frag b/src/qcam/assets/shader/YUV_2_planes.frag\n> index 125f1c850a33..254463c05cac 100644\n> --- a/src/qcam/assets/shader/YUV_2_planes.frag\n> +++ b/src/qcam/assets/shader/YUV_2_planes.frag\n> @@ -26,9 +26,9 @@ void main(void)\n> \tyuv.x = texture2D(tex_y, textureOut).r - 0.063;\n> #if defined(YUV_PATTERN_UV)\n> \tyuv.y = texture2D(tex_u, textureOut).r - 0.500;\n> -\tyuv.z = texture2D(tex_u, textureOut).g - 0.500;\n> +\tyuv.z = texture2D(tex_u, textureOut).a - 0.500;\n> #elif defined(YUV_PATTERN_VU)\n> -\tyuv.y = texture2D(tex_u, textureOut).g - 0.500;\n> +\tyuv.y = texture2D(tex_u, textureOut).a - 0.500;\n> \tyuv.z = texture2D(tex_u, textureOut).r - 0.500;\n> #else\n> #error Invalid pattern\n> diff --git a/src/qcam/viewfinder_gl.cpp b/src/qcam/viewfinder_gl.cpp\n> index e7c8620c7024..77a6437e56fd 100644\n> --- a/src/qcam/viewfinder_gl.cpp\n> +++ b/src/qcam/viewfinder_gl.cpp\n> @@ -481,11 +481,11 @@ void ViewFinderGL::doRender()\n> \t\tconfigureTexture(*textures_[0]);\n> \t\tglTexImage2D(GL_TEXTURE_2D,\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t size_.width(),\n> \t\t\t size_.height(),\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t GL_UNSIGNED_BYTE,\n> \t\t\t data_);\n> \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n> @@ -495,11 +495,11 @@ void ViewFinderGL::doRender()\n> \t\tconfigureTexture(*textures_[1]);\n> \t\tglTexImage2D(GL_TEXTURE_2D,\n> \t\t\t 0,\n> -\t\t\t GL_RG,\n> +\t\t\t GL_LUMINANCE_ALPHA,\n> \t\t\t size_.width() / horzSubSample_,\n> \t\t\t size_.height() / vertSubSample_,\n> \t\t\t 0,\n> -\t\t\t GL_RG,\n> +\t\t\t GL_LUMINANCE_ALPHA,\n> \t\t\t GL_UNSIGNED_BYTE,\n> \t\t\t data_ + size_.width() * size_.height());\n> \t\tshaderProgram_.setUniformValue(textureUniformU_, 1);\n> @@ -511,11 +511,11 @@ void ViewFinderGL::doRender()\n> \t\tconfigureTexture(*textures_[0]);\n> \t\tglTexImage2D(GL_TEXTURE_2D,\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t size_.width(),\n> \t\t\t size_.height(),\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t GL_UNSIGNED_BYTE,\n> \t\t\t data_);\n> \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n> @@ -525,11 +525,11 @@ void ViewFinderGL::doRender()\n> \t\tconfigureTexture(*textures_[1]);\n> \t\tglTexImage2D(GL_TEXTURE_2D,\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t size_.width() / horzSubSample_,\n> \t\t\t size_.height() / vertSubSample_,\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t GL_UNSIGNED_BYTE,\n> \t\t\t data_ + size_.width() * size_.height());\n> \t\tshaderProgram_.setUniformValue(textureUniformU_, 1);\n> @@ -539,11 +539,11 @@ void ViewFinderGL::doRender()\n> \t\tconfigureTexture(*textures_[2]);\n> \t\tglTexImage2D(GL_TEXTURE_2D,\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t size_.width() / horzSubSample_,\n> \t\t\t size_.height() / vertSubSample_,\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t GL_UNSIGNED_BYTE,\n> \t\t\t data_ + size_.width() * size_.height() * 5 / 4);\n> \t\tshaderProgram_.setUniformValue(textureUniformV_, 2);\n> @@ -555,11 +555,11 @@ void ViewFinderGL::doRender()\n> \t\tconfigureTexture(*textures_[0]);\n> \t\tglTexImage2D(GL_TEXTURE_2D,\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t size_.width(),\n> \t\t\t size_.height(),\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t GL_UNSIGNED_BYTE,\n> \t\t\t data_);\n> \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n> @@ -569,11 +569,11 @@ void ViewFinderGL::doRender()\n> \t\tconfigureTexture(*textures_[2]);\n> \t\tglTexImage2D(GL_TEXTURE_2D,\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t size_.width() / horzSubSample_,\n> \t\t\t size_.height() / vertSubSample_,\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t GL_UNSIGNED_BYTE,\n> \t\t\t data_ + size_.width() * size_.height());\n> \t\tshaderProgram_.setUniformValue(textureUniformV_, 2);\n> @@ -583,11 +583,11 @@ void ViewFinderGL::doRender()\n> \t\tconfigureTexture(*textures_[1]);\n> \t\tglTexImage2D(GL_TEXTURE_2D,\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t size_.width() / horzSubSample_,\n> \t\t\t size_.height() / vertSubSample_,\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t GL_UNSIGNED_BYTE,\n> \t\t\t data_ + size_.width() * size_.height() * 5 / 4);\n> \t\tshaderProgram_.setUniformValue(textureUniformU_, 1);\n> @@ -674,18 +674,18 @@ void ViewFinderGL::doRender()\n> \tcase libcamera::formats::SRGGB12_CSI2P:\n> \t\t/*\n> \t\t * Raw Bayer 8-bit, and packed raw Bayer 10-bit/12-bit formats\n> -\t\t * are stored in GL_RED texture.\n> -\t\t * The texture width is equal to the stride.\n> +\t\t * are stored in GL_LUMINANCE texture. The texture width is\n\nI know you're modifying an existing sentence, but should this read either:\n\n - are stored in a GL_LUMINANCE texture\nor\n - are stored in the GL_LUMINANCE texture\n\ndepending on if there is only one GL_LUMINANCE texture, or if there\nmight be more and this is specifically allocated.\n\n(That extra clarification would help me understand those identifiers\ntoo... such power from a single word addition).\n\nAnyway, aside from / with that:\n\nReviewed-by: Kieran Bingham \n\n\n> +\t\t * equal to the stride.\n> \t\t */\n> \t\tglActiveTexture(GL_TEXTURE0);\n> \t\tconfigureTexture(*textures_[0]);\n> \t\tglTexImage2D(GL_TEXTURE_2D,\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t stride_,\n> \t\t\t size_.height(),\n> \t\t\t 0,\n> -\t\t\t GL_RED,\n> +\t\t\t GL_LUMINANCE,\n> \t\t\t GL_UNSIGNED_BYTE,\n> \t\t\t data_);\n> \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 0B146C3235\n\tfor ;\n\tTue, 3 Aug 2021 09:31:29 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 7A5FA687CF;\n\tTue, 3 Aug 2021 11:31:28 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E390A6026D\n\tfor ;\n\tTue, 3 Aug 2021 11:31:26 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 568CCDD;\n\tTue, 3 Aug 2021 11:31:26 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"QkF885ti\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1627983086;\n\tbh=DsI9ClGH9F7AwlN+iv6C5/THD6VfmVFarWqBwviItBE=;\n\th=From:To:Cc:References:Subject:Date:In-Reply-To:From;\n\tb=QkF885tiSfsd/isDgDFrYrlR5jIvvrpr9cpKSWkZ+BYWC4LA51h868hm1YfWOzyeB\n\tOsRop2MfoftjT5JhV9Z8gwXxMJGP8Gl+hhZx8Phnu2yHR7bEmBaPQ6vzEy6mp9ZNmG\n\tgujaGcbCAd7XgbpRsg9J3q+Qw3SK5FxLqZjdW5w8=","From":"Kieran Bingham ","To":"Laurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20210803092310.11956-1-laurent.pinchart@ideasonboard.com>","Message-ID":"","Date":"Tue, 3 Aug 2021 10:31:23 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.11.0","MIME-Version":"1.0","In-Reply-To":"<20210803092310.11956-1-laurent.pinchart@ideasonboard.com>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-GB","Content-Transfer-Encoding":"8bit","Subject":"Re: [libcamera-devel] [PATCH] qcam: Support OpenGL ES 2.0","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":18507,"web_url":"https://patchwork.libcamera.org/comment/18507/","msgid":"","date":"2021-08-03T09:41:57","subject":"Re: [libcamera-devel] [PATCH] qcam: Support OpenGL ES 2.0","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Kieran,\n\nOn Tue, Aug 03, 2021 at 10:31:23AM +0100, Kieran Bingham wrote:\n> On 03/08/2021 10:23, Laurent Pinchart wrote:\n> > The GL_RG and GL_RED texture formats are not supported in OpenGL ES\n> > prior to 3.0. In order to be compatible with OpenGL ES 2.0, use\n> > GL_LUMINANCE_ALPHA and GL_LUMINANCE instead. The shader code needs to be\n> > updated accordingly for GL_RG, as the second component is now stored in\n> \n> /in alpha/in the alpha/\n> \n> > alpha component instead of the green component. Usage of the red\n> > component is fine, the luminance value is stored in the red, green and\n> > blue components.\n> \n> So these seem to be just fairly arbitrary identifiers of the texture\n> maps to use, which as long as they are consistently referenced that's\n> all that matters?\n\nThey describe how the texture is stored in memory, and how it's\nconverted to RGBA before going through the shaders. See\nhttps://docs.gl/es2/glTexImage2D for more information. In particular,\n\nGL_LUMINANCE\n\n Each element is a single luminance value. The GL converts it to\n floating point, then assembles it into an RGBA element by\n replicating the luminance value three times for red, green, and blue\n and attaching 1 for alpha. Each component is then clamped to the\n range [0,1].\n\nGL_LUMINANCE_ALPHA\n\n Each element is a luminance/alpha pair. The GL converts it to\n floating point, then assembles it into an RGBA element by\n replicating the luminance value three times for red, green, and\n blue. Each component is then clamped to the range [0,1].\n\nIn ES3, we additionally have\n(https://www.khronos.org/registry/OpenGL-Refpages/es3.0/html/glTexImage2D.xhtml),\n\nGL_RED\n\n Each element is a single red component. For fixed point normalized\n components, the GL converts it to floating point, clamps to the\n range [0,1], and assembles it into an RGBA element by attaching 0.0\n for green and blue, and 1.0 for alpha.\n\nGL_RG\n\n Each element is a red/green double. For fixed point normalized\n components, the GL converts each component to floating point, clamps\n to the range [0,1], and assembles them into an RGBA element by\n attaching 0.0 for blue, and 1.0 for alpha.\n\nGL_RED maps the value to [v, 0.0, 0.0, 1.0] while GL_LUMINANCE maps it\nto [v, v, v, 1.0]. As the shaders use the red component only, they're\nequivalent. For GL_RG, however, when moving to GL_LUMINANCE_ALPHA, we\nneed to replace usage for the green component with the alpha component.\n\n> > Signed-off-by: Laurent Pinchart \n> > ---\n> > Tested with vivid, using NV12, YV420, YVU420 and SGRBG8, which should\n> > cover all code paths.\n> > ---\n> > src/qcam/assets/shader/YUV_2_planes.frag | 4 +--\n> > src/qcam/viewfinder_gl.cpp | 40 ++++++++++++------------\n> > 2 files changed, 22 insertions(+), 22 deletions(-)\n> > \n> > diff --git a/src/qcam/assets/shader/YUV_2_planes.frag b/src/qcam/assets/shader/YUV_2_planes.frag\n> > index 125f1c850a33..254463c05cac 100644\n> > --- a/src/qcam/assets/shader/YUV_2_planes.frag\n> > +++ b/src/qcam/assets/shader/YUV_2_planes.frag\n> > @@ -26,9 +26,9 @@ void main(void)\n> > \tyuv.x = texture2D(tex_y, textureOut).r - 0.063;\n> > #if defined(YUV_PATTERN_UV)\n> > \tyuv.y = texture2D(tex_u, textureOut).r - 0.500;\n> > -\tyuv.z = texture2D(tex_u, textureOut).g - 0.500;\n> > +\tyuv.z = texture2D(tex_u, textureOut).a - 0.500;\n> > #elif defined(YUV_PATTERN_VU)\n> > -\tyuv.y = texture2D(tex_u, textureOut).g - 0.500;\n> > +\tyuv.y = texture2D(tex_u, textureOut).a - 0.500;\n> > \tyuv.z = texture2D(tex_u, textureOut).r - 0.500;\n> > #else\n> > #error Invalid pattern\n> > diff --git a/src/qcam/viewfinder_gl.cpp b/src/qcam/viewfinder_gl.cpp\n> > index e7c8620c7024..77a6437e56fd 100644\n> > --- a/src/qcam/viewfinder_gl.cpp\n> > +++ b/src/qcam/viewfinder_gl.cpp\n> > @@ -481,11 +481,11 @@ void ViewFinderGL::doRender()\n> > \t\tconfigureTexture(*textures_[0]);\n> > \t\tglTexImage2D(GL_TEXTURE_2D,\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t size_.width(),\n> > \t\t\t size_.height(),\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t GL_UNSIGNED_BYTE,\n> > \t\t\t data_);\n> > \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n> > @@ -495,11 +495,11 @@ void ViewFinderGL::doRender()\n> > \t\tconfigureTexture(*textures_[1]);\n> > \t\tglTexImage2D(GL_TEXTURE_2D,\n> > \t\t\t 0,\n> > -\t\t\t GL_RG,\n> > +\t\t\t GL_LUMINANCE_ALPHA,\n> > \t\t\t size_.width() / horzSubSample_,\n> > \t\t\t size_.height() / vertSubSample_,\n> > \t\t\t 0,\n> > -\t\t\t GL_RG,\n> > +\t\t\t GL_LUMINANCE_ALPHA,\n> > \t\t\t GL_UNSIGNED_BYTE,\n> > \t\t\t data_ + size_.width() * size_.height());\n> > \t\tshaderProgram_.setUniformValue(textureUniformU_, 1);\n> > @@ -511,11 +511,11 @@ void ViewFinderGL::doRender()\n> > \t\tconfigureTexture(*textures_[0]);\n> > \t\tglTexImage2D(GL_TEXTURE_2D,\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t size_.width(),\n> > \t\t\t size_.height(),\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t GL_UNSIGNED_BYTE,\n> > \t\t\t data_);\n> > \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n> > @@ -525,11 +525,11 @@ void ViewFinderGL::doRender()\n> > \t\tconfigureTexture(*textures_[1]);\n> > \t\tglTexImage2D(GL_TEXTURE_2D,\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t size_.width() / horzSubSample_,\n> > \t\t\t size_.height() / vertSubSample_,\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t GL_UNSIGNED_BYTE,\n> > \t\t\t data_ + size_.width() * size_.height());\n> > \t\tshaderProgram_.setUniformValue(textureUniformU_, 1);\n> > @@ -539,11 +539,11 @@ void ViewFinderGL::doRender()\n> > \t\tconfigureTexture(*textures_[2]);\n> > \t\tglTexImage2D(GL_TEXTURE_2D,\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t size_.width() / horzSubSample_,\n> > \t\t\t size_.height() / vertSubSample_,\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t GL_UNSIGNED_BYTE,\n> > \t\t\t data_ + size_.width() * size_.height() * 5 / 4);\n> > \t\tshaderProgram_.setUniformValue(textureUniformV_, 2);\n> > @@ -555,11 +555,11 @@ void ViewFinderGL::doRender()\n> > \t\tconfigureTexture(*textures_[0]);\n> > \t\tglTexImage2D(GL_TEXTURE_2D,\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t size_.width(),\n> > \t\t\t size_.height(),\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t GL_UNSIGNED_BYTE,\n> > \t\t\t data_);\n> > \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n> > @@ -569,11 +569,11 @@ void ViewFinderGL::doRender()\n> > \t\tconfigureTexture(*textures_[2]);\n> > \t\tglTexImage2D(GL_TEXTURE_2D,\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t size_.width() / horzSubSample_,\n> > \t\t\t size_.height() / vertSubSample_,\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t GL_UNSIGNED_BYTE,\n> > \t\t\t data_ + size_.width() * size_.height());\n> > \t\tshaderProgram_.setUniformValue(textureUniformV_, 2);\n> > @@ -583,11 +583,11 @@ void ViewFinderGL::doRender()\n> > \t\tconfigureTexture(*textures_[1]);\n> > \t\tglTexImage2D(GL_TEXTURE_2D,\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t size_.width() / horzSubSample_,\n> > \t\t\t size_.height() / vertSubSample_,\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t GL_UNSIGNED_BYTE,\n> > \t\t\t data_ + size_.width() * size_.height() * 5 / 4);\n> > \t\tshaderProgram_.setUniformValue(textureUniformU_, 1);\n> > @@ -674,18 +674,18 @@ void ViewFinderGL::doRender()\n> > \tcase libcamera::formats::SRGGB12_CSI2P:\n> > \t\t/*\n> > \t\t * Raw Bayer 8-bit, and packed raw Bayer 10-bit/12-bit formats\n> > -\t\t * are stored in GL_RED texture.\n> > -\t\t * The texture width is equal to the stride.\n> > +\t\t * are stored in GL_LUMINANCE texture. The texture width is\n> \n> I know you're modifying an existing sentence, but should this read either:\n> \n> - are stored in a GL_LUMINANCE texture\n> or\n> - are stored in the GL_LUMINANCE texture\n> \n> depending on if there is only one GL_LUMINANCE texture, or if there\n> might be more and this is specifically allocated.\n\nI'll fix it, using \"a GL_LUMINANCE texture\".\n\n> (That extra clarification would help me understand those identifiers\n> too... such power from a single word addition).\n> \n> Anyway, aside from / with that:\n> \n> Reviewed-by: Kieran Bingham \n> \n> > +\t\t * equal to the stride.\n> > \t\t */\n> > \t\tglActiveTexture(GL_TEXTURE0);\n> > \t\tconfigureTexture(*textures_[0]);\n> > \t\tglTexImage2D(GL_TEXTURE_2D,\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t stride_,\n> > \t\t\t size_.height(),\n> > \t\t\t 0,\n> > -\t\t\t GL_RED,\n> > +\t\t\t GL_LUMINANCE,\n> > \t\t\t GL_UNSIGNED_BYTE,\n> > \t\t\t data_);\n> > \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n> >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 48B39C3232\n\tfor ;\n\tTue, 3 Aug 2021 09:42:13 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id AA3D6687CF;\n\tTue, 3 Aug 2021 11:42:12 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 890DC6026D\n\tfor ;\n\tTue, 3 Aug 2021 11:42:11 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id A8F8F4A3;\n\tTue, 3 Aug 2021 11:42:10 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"KaQNZsvN\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1627983731;\n\tbh=cK6B9IGiieve3EN3+h/+VXrWBm1wekL/9iFNbhPSyBE=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=KaQNZsvNmDPadBLMkqaNj2auVYj+aYlkBnBHcipQsomhpgaDSuHE+P2tP2Gdmk+E+\n\tj+hOZCSPF+EGQMoYloCxte+1oseAlRXtkz+CRN76qdr/Ei+oideAg/OEPTDLOcFEYp\n\tM7/XJcWvWWBoIYPvgyEnjkWHw6um5PD/zU0T+Yis=","Date":"Tue, 3 Aug 2021 12:41:57 +0300","From":"Laurent Pinchart ","To":"Kieran Bingham ","Message-ID":"","References":"<20210803092310.11956-1-laurent.pinchart@ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH] qcam: Support OpenGL ES 2.0","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":18527,"web_url":"https://patchwork.libcamera.org/comment/18527/","msgid":"<0bfae557-d31a-5bb6-d028-44bcde45f01a@linaro.org>","date":"2021-08-03T20:28:30","subject":"Re: [libcamera-devel] [PATCH] qcam: Support OpenGL ES 2.0","submitter":{"id":25,"url":"https://patchwork.libcamera.org/api/people/25/","name":"Andrey Konovalov","email":"andrey.konovalov@linaro.org"},"content":"Hi Laurent,\n\nMaking qcam to work with OpenGL ES 2.0 is a nice improvement!\n\nOn 03.08.2021 12:41, Laurent Pinchart wrote:\n> Hi Kieran,\n> \n> On Tue, Aug 03, 2021 at 10:31:23AM +0100, Kieran Bingham wrote:\n>> On 03/08/2021 10:23, Laurent Pinchart wrote:\n>>> The GL_RG and GL_RED texture formats are not supported in OpenGL ES\n>>> prior to 3.0. In order to be compatible with OpenGL ES 2.0, use\n>>> GL_LUMINANCE_ALPHA and GL_LUMINANCE instead. The shader code needs to be\n>>> updated accordingly for GL_RG, as the second component is now stored in\n>>\n>> /in alpha/in the alpha/\n>>\n>>> alpha component instead of the green component. Usage of the red\n>>> component is fine, the luminance value is stored in the red, green and\n>>> blue components.\n>>\n>> So these seem to be just fairly arbitrary identifiers of the texture\n>> maps to use, which as long as they are consistently referenced that's\n>> all that matters?\n> \n> They describe how the texture is stored in memory, and how it's\n> converted to RGBA before going through the shaders. See\n> https://docs.gl/es2/glTexImage2D for more information. In particular,\n> \n> GL_LUMINANCE\n> \n> Each element is a single luminance value. The GL converts it to\n> floating point, then assembles it into an RGBA element by\n> replicating the luminance value three times for red, green, and blue\n> and attaching 1 for alpha. Each component is then clamped to the\n> range [0,1].\n> \n> GL_LUMINANCE_ALPHA\n> \n> Each element is a luminance/alpha pair. The GL converts it to\n> floating point, then assembles it into an RGBA element by\n> replicating the luminance value three times for red, green, and\n> blue. Each component is then clamped to the range [0,1].\n> \n> In ES3, we additionally have\n> (https://www.khronos.org/registry/OpenGL-Refpages/es3.0/html/glTexImage2D.xhtml),\n> \n> GL_RED\n> \n> Each element is a single red component. For fixed point normalized\n> components, the GL converts it to floating point, clamps to the\n> range [0,1], and assembles it into an RGBA element by attaching 0.0\n> for green and blue, and 1.0 for alpha.\n> \n> GL_RG\n> \n> Each element is a red/green double. For fixed point normalized\n> components, the GL converts each component to floating point, clamps\n> to the range [0,1], and assembles them into an RGBA element by\n> attaching 0.0 for blue, and 1.0 for alpha.\n> \n> GL_RED maps the value to [v, 0.0, 0.0, 1.0] while GL_LUMINANCE maps it\n> to [v, v, v, 1.0]. As the shaders use the red component only, they're\n> equivalent. For GL_RG, however, when moving to GL_LUMINANCE_ALPHA, we\n> need to replace usage for the green component with the alpha component.\n\nThanks for the explanation and the docs links!\n\n>>> Signed-off-by: Laurent Pinchart \n>>> ---\n>>> Tested with vivid, using NV12, YV420, YVU420 and SGRBG8, which should\n>>> cover all code paths.\n\nReviewed-by: Andrey Konovalov \n\n(Tested on RB3 board with camss using SGRBG10_CSI2P - just in case)\n\nThanks,\nAndrey\n\n>>> ---\n>>> src/qcam/assets/shader/YUV_2_planes.frag | 4 +--\n>>> src/qcam/viewfinder_gl.cpp | 40 ++++++++++++------------\n>>> 2 files changed, 22 insertions(+), 22 deletions(-)\n>>>\n>>> diff --git a/src/qcam/assets/shader/YUV_2_planes.frag b/src/qcam/assets/shader/YUV_2_planes.frag\n>>> index 125f1c850a33..254463c05cac 100644\n>>> --- a/src/qcam/assets/shader/YUV_2_planes.frag\n>>> +++ b/src/qcam/assets/shader/YUV_2_planes.frag\n>>> @@ -26,9 +26,9 @@ void main(void)\n>>> \tyuv.x = texture2D(tex_y, textureOut).r - 0.063;\n>>> #if defined(YUV_PATTERN_UV)\n>>> \tyuv.y = texture2D(tex_u, textureOut).r - 0.500;\n>>> -\tyuv.z = texture2D(tex_u, textureOut).g - 0.500;\n>>> +\tyuv.z = texture2D(tex_u, textureOut).a - 0.500;\n>>> #elif defined(YUV_PATTERN_VU)\n>>> -\tyuv.y = texture2D(tex_u, textureOut).g - 0.500;\n>>> +\tyuv.y = texture2D(tex_u, textureOut).a - 0.500;\n>>> \tyuv.z = texture2D(tex_u, textureOut).r - 0.500;\n>>> #else\n>>> #error Invalid pattern\n>>> diff --git a/src/qcam/viewfinder_gl.cpp b/src/qcam/viewfinder_gl.cpp\n>>> index e7c8620c7024..77a6437e56fd 100644\n>>> --- a/src/qcam/viewfinder_gl.cpp\n>>> +++ b/src/qcam/viewfinder_gl.cpp\n>>> @@ -481,11 +481,11 @@ void ViewFinderGL::doRender()\n>>> \t\tconfigureTexture(*textures_[0]);\n>>> \t\tglTexImage2D(GL_TEXTURE_2D,\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t size_.width(),\n>>> \t\t\t size_.height(),\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t GL_UNSIGNED_BYTE,\n>>> \t\t\t data_);\n>>> \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n>>> @@ -495,11 +495,11 @@ void ViewFinderGL::doRender()\n>>> \t\tconfigureTexture(*textures_[1]);\n>>> \t\tglTexImage2D(GL_TEXTURE_2D,\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RG,\n>>> +\t\t\t GL_LUMINANCE_ALPHA,\n>>> \t\t\t size_.width() / horzSubSample_,\n>>> \t\t\t size_.height() / vertSubSample_,\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RG,\n>>> +\t\t\t GL_LUMINANCE_ALPHA,\n>>> \t\t\t GL_UNSIGNED_BYTE,\n>>> \t\t\t data_ + size_.width() * size_.height());\n>>> \t\tshaderProgram_.setUniformValue(textureUniformU_, 1);\n>>> @@ -511,11 +511,11 @@ void ViewFinderGL::doRender()\n>>> \t\tconfigureTexture(*textures_[0]);\n>>> \t\tglTexImage2D(GL_TEXTURE_2D,\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t size_.width(),\n>>> \t\t\t size_.height(),\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t GL_UNSIGNED_BYTE,\n>>> \t\t\t data_);\n>>> \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n>>> @@ -525,11 +525,11 @@ void ViewFinderGL::doRender()\n>>> \t\tconfigureTexture(*textures_[1]);\n>>> \t\tglTexImage2D(GL_TEXTURE_2D,\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t size_.width() / horzSubSample_,\n>>> \t\t\t size_.height() / vertSubSample_,\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t GL_UNSIGNED_BYTE,\n>>> \t\t\t data_ + size_.width() * size_.height());\n>>> \t\tshaderProgram_.setUniformValue(textureUniformU_, 1);\n>>> @@ -539,11 +539,11 @@ void ViewFinderGL::doRender()\n>>> \t\tconfigureTexture(*textures_[2]);\n>>> \t\tglTexImage2D(GL_TEXTURE_2D,\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t size_.width() / horzSubSample_,\n>>> \t\t\t size_.height() / vertSubSample_,\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t GL_UNSIGNED_BYTE,\n>>> \t\t\t data_ + size_.width() * size_.height() * 5 / 4);\n>>> \t\tshaderProgram_.setUniformValue(textureUniformV_, 2);\n>>> @@ -555,11 +555,11 @@ void ViewFinderGL::doRender()\n>>> \t\tconfigureTexture(*textures_[0]);\n>>> \t\tglTexImage2D(GL_TEXTURE_2D,\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t size_.width(),\n>>> \t\t\t size_.height(),\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t GL_UNSIGNED_BYTE,\n>>> \t\t\t data_);\n>>> \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n>>> @@ -569,11 +569,11 @@ void ViewFinderGL::doRender()\n>>> \t\tconfigureTexture(*textures_[2]);\n>>> \t\tglTexImage2D(GL_TEXTURE_2D,\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t size_.width() / horzSubSample_,\n>>> \t\t\t size_.height() / vertSubSample_,\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t GL_UNSIGNED_BYTE,\n>>> \t\t\t data_ + size_.width() * size_.height());\n>>> \t\tshaderProgram_.setUniformValue(textureUniformV_, 2);\n>>> @@ -583,11 +583,11 @@ void ViewFinderGL::doRender()\n>>> \t\tconfigureTexture(*textures_[1]);\n>>> \t\tglTexImage2D(GL_TEXTURE_2D,\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t size_.width() / horzSubSample_,\n>>> \t\t\t size_.height() / vertSubSample_,\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t GL_UNSIGNED_BYTE,\n>>> \t\t\t data_ + size_.width() * size_.height() * 5 / 4);\n>>> \t\tshaderProgram_.setUniformValue(textureUniformU_, 1);\n>>> @@ -674,18 +674,18 @@ void ViewFinderGL::doRender()\n>>> \tcase libcamera::formats::SRGGB12_CSI2P:\n>>> \t\t/*\n>>> \t\t * Raw Bayer 8-bit, and packed raw Bayer 10-bit/12-bit formats\n>>> -\t\t * are stored in GL_RED texture.\n>>> -\t\t * The texture width is equal to the stride.\n>>> +\t\t * are stored in GL_LUMINANCE texture. The texture width is\n>>\n>> I know you're modifying an existing sentence, but should this read either:\n>>\n>> - are stored in a GL_LUMINANCE texture\n>> or\n>> - are stored in the GL_LUMINANCE texture\n>>\n>> depending on if there is only one GL_LUMINANCE texture, or if there\n>> might be more and this is specifically allocated.\n> \n> I'll fix it, using \"a GL_LUMINANCE texture\".\n> \n>> (That extra clarification would help me understand those identifiers\n>> too... such power from a single word addition).\n>>\n>> Anyway, aside from / with that:\n>>\n>> Reviewed-by: Kieran Bingham \n>>\n>>> +\t\t * equal to the stride.\n>>> \t\t */\n>>> \t\tglActiveTexture(GL_TEXTURE0);\n>>> \t\tconfigureTexture(*textures_[0]);\n>>> \t\tglTexImage2D(GL_TEXTURE_2D,\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t stride_,\n>>> \t\t\t size_.height(),\n>>> \t\t\t 0,\n>>> -\t\t\t GL_RED,\n>>> +\t\t\t GL_LUMINANCE,\n>>> \t\t\t GL_UNSIGNED_BYTE,\n>>> \t\t\t data_);\n>>> \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n>>>\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 573C8C3235\n\tfor ;\n\tTue, 3 Aug 2021 20:28:35 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8CF17687CC;\n\tTue, 3 Aug 2021 22:28:34 +0200 (CEST)","from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com\n\t[IPv6:2a00:1450:4864:20::12b])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E50A46026A\n\tfor ;\n\tTue, 3 Aug 2021 22:28:32 +0200 (CEST)","by mail-lf1-x12b.google.com with SMTP id y34so627364lfa.8\n\tfor ;\n\tTue, 03 Aug 2021 13:28:32 -0700 (PDT)","from [192.168.88.254] ([85.249.40.209])\n\tby smtp.gmail.com with ESMTPSA id\n\ts15sm1336481lfp.216.2021.08.03.13.28.31\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tTue, 03 Aug 2021 13:28:31 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"icUqzC1/\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=eS7vqzBU51PCyoGduA7J48um3d2WcTQlf+fX7EbtXh0=;\n\tb=icUqzC1/Oy3Aq0HCQUxlkQZBuxbyRdA84PWTVx2EwGC7emaQu0HT/hB866gKzhEyEl\n\teUJI7OIztDNiyGBaVHyDr3i+2CpjiocVCCA+z0HjnfQDogW6fEi1VtvSmwWMnBrptKYL\n\tini9rmUglFAPcL3tsy5ISHzrTJfYFDXeP9UuUVeWThdFKZm1cgyYhul8v55P632M557S\n\twhqrArEt5G072fWPF23gH7aA7YoFZy+QH90VwEQ5oxUUb1THYh8bNgtDXRQApr1zhPux\n\tIob3aQQXoqCyUK+VUq/+pbft2u2yc95dB8NYuPhKxYEbAvugPeonSKJYTuR/RdNGS2Dl\n\t47nQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=eS7vqzBU51PCyoGduA7J48um3d2WcTQlf+fX7EbtXh0=;\n\tb=DcMf1OgJ1SGTkLLLKpQ3v2o8ZVtjKDOOcZgOGtpslTE+PKddFACZNFM3VBQZeWKMyu\n\tijjQDXMOw1T4MLO/dJLnY2Jk3KNxwxIpwpYyLC3cLMeVsEi36BCNso6kWv7FtFCS92Wb\n\tJm4WCm+6p/6igFD4lzHRt+kGxEllTKOOLoQplAEjGpyMQe64FEpDlSo20Ca0a/C598N/\n\tBzJTYps/ZHIJYEBGhsNJyQXStfnD08f0VsL4+9B8X12Cx0gI2sMJUaxWRI7g/gVZboBh\n\tz0e17rSVp5g8qRaTY0Ukc7ar4vAyjFhN6+EwgXnAL3oc7m7IVQeTl5wN5UWVRczJ4n/u\n\t54wA==","X-Gm-Message-State":"AOAM531t/AT4KxADJ07aEAhnVxvHamuorenJ/JDuyaP+nnY6wULqDL6d\n\tT3WG0rTSf9v0bNdysYA7vI5JZg==","X-Google-Smtp-Source":"ABdhPJzPEogaa5re/50Mo+b2B7ZC8WZRa4pMoaDUaoQMK3hvBaWGk7mOvxbdwoWb+djycRuqMmypEg==","X-Received":"by 2002:a05:6512:20e:: with SMTP id\n\ta14mr17535141lfo.112.1628022512103; \n\tTue, 03 Aug 2021 13:28:32 -0700 (PDT)","To":"Laurent Pinchart ","References":"<20210803092310.11956-1-laurent.pinchart@ideasonboard.com>\n\t\n\t","From":"Andrey Konovalov ","Message-ID":"<0bfae557-d31a-5bb6-d028-44bcde45f01a@linaro.org>","Date":"Tue, 3 Aug 2021 23:28:30 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.11.0","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [PATCH] qcam: Support OpenGL ES 2.0","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":18528,"web_url":"https://patchwork.libcamera.org/comment/18528/","msgid":"","date":"2021-08-03T20:38:19","subject":"Re: [libcamera-devel] [PATCH] qcam: Support OpenGL ES 2.0","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Andrey,\n\nOn Tue, Aug 03, 2021 at 11:28:30PM +0300, Andrey Konovalov wrote:\n> Hi Laurent,\n> \n> Making qcam to work with OpenGL ES 2.0 is a nice improvement!\n> \n> On 03.08.2021 12:41, Laurent Pinchart wrote:\n> > On Tue, Aug 03, 2021 at 10:31:23AM +0100, Kieran Bingham wrote:\n> >> On 03/08/2021 10:23, Laurent Pinchart wrote:\n> >>> The GL_RG and GL_RED texture formats are not supported in OpenGL ES\n> >>> prior to 3.0. In order to be compatible with OpenGL ES 2.0, use\n> >>> GL_LUMINANCE_ALPHA and GL_LUMINANCE instead. The shader code needs to be\n> >>> updated accordingly for GL_RG, as the second component is now stored in\n> >>\n> >> /in alpha/in the alpha/\n> >>\n> >>> alpha component instead of the green component. Usage of the red\n> >>> component is fine, the luminance value is stored in the red, green and\n> >>> blue components.\n> >>\n> >> So these seem to be just fairly arbitrary identifiers of the texture\n> >> maps to use, which as long as they are consistently referenced that's\n> >> all that matters?\n> > \n> > They describe how the texture is stored in memory, and how it's\n> > converted to RGBA before going through the shaders. See\n> > https://docs.gl/es2/glTexImage2D for more information. In particular,\n> > \n> > GL_LUMINANCE\n> > \n> > Each element is a single luminance value. The GL converts it to\n> > floating point, then assembles it into an RGBA element by\n> > replicating the luminance value three times for red, green, and blue\n> > and attaching 1 for alpha. Each component is then clamped to the\n> > range [0,1].\n> > \n> > GL_LUMINANCE_ALPHA\n> > \n> > Each element is a luminance/alpha pair. The GL converts it to\n> > floating point, then assembles it into an RGBA element by\n> > replicating the luminance value three times for red, green, and\n> > blue. Each component is then clamped to the range [0,1].\n> > \n> > In ES3, we additionally have\n> > (https://www.khronos.org/registry/OpenGL-Refpages/es3.0/html/glTexImage2D.xhtml),\n> > \n> > GL_RED\n> > \n> > Each element is a single red component. For fixed point normalized\n> > components, the GL converts it to floating point, clamps to the\n> > range [0,1], and assembles it into an RGBA element by attaching 0.0\n> > for green and blue, and 1.0 for alpha.\n> > \n> > GL_RG\n> > \n> > Each element is a red/green double. For fixed point normalized\n> > components, the GL converts each component to floating point, clamps\n> > to the range [0,1], and assembles them into an RGBA element by\n> > attaching 0.0 for blue, and 1.0 for alpha.\n> > \n> > GL_RED maps the value to [v, 0.0, 0.0, 1.0] while GL_LUMINANCE maps it\n> > to [v, v, v, 1.0]. As the shaders use the red component only, they're\n> > equivalent. For GL_RG, however, when moving to GL_LUMINANCE_ALPHA, we\n> > need to replace usage for the green component with the alpha component.\n> \n> Thanks for the explanation and the docs links!\n> \n> >>> Signed-off-by: Laurent Pinchart \n> >>> ---\n> >>> Tested with vivid, using NV12, YV420, YVU420 and SGRBG8, which should\n> >>> cover all code paths.\n> \n> Reviewed-by: Andrey Konovalov \n> \n> (Tested on RB3 board with camss using SGRBG10_CSI2P - just in case)\n\nThat's the only format family that I haven't been able to test :-)\nThanks.\n\n> >>> ---\n> >>> src/qcam/assets/shader/YUV_2_planes.frag | 4 +--\n> >>> src/qcam/viewfinder_gl.cpp | 40 ++++++++++++------------\n> >>> 2 files changed, 22 insertions(+), 22 deletions(-)\n> >>>\n> >>> diff --git a/src/qcam/assets/shader/YUV_2_planes.frag b/src/qcam/assets/shader/YUV_2_planes.frag\n> >>> index 125f1c850a33..254463c05cac 100644\n> >>> --- a/src/qcam/assets/shader/YUV_2_planes.frag\n> >>> +++ b/src/qcam/assets/shader/YUV_2_planes.frag\n> >>> @@ -26,9 +26,9 @@ void main(void)\n> >>> \tyuv.x = texture2D(tex_y, textureOut).r - 0.063;\n> >>> #if defined(YUV_PATTERN_UV)\n> >>> \tyuv.y = texture2D(tex_u, textureOut).r - 0.500;\n> >>> -\tyuv.z = texture2D(tex_u, textureOut).g - 0.500;\n> >>> +\tyuv.z = texture2D(tex_u, textureOut).a - 0.500;\n> >>> #elif defined(YUV_PATTERN_VU)\n> >>> -\tyuv.y = texture2D(tex_u, textureOut).g - 0.500;\n> >>> +\tyuv.y = texture2D(tex_u, textureOut).a - 0.500;\n> >>> \tyuv.z = texture2D(tex_u, textureOut).r - 0.500;\n> >>> #else\n> >>> #error Invalid pattern\n> >>> diff --git a/src/qcam/viewfinder_gl.cpp b/src/qcam/viewfinder_gl.cpp\n> >>> index e7c8620c7024..77a6437e56fd 100644\n> >>> --- a/src/qcam/viewfinder_gl.cpp\n> >>> +++ b/src/qcam/viewfinder_gl.cpp\n> >>> @@ -481,11 +481,11 @@ void ViewFinderGL::doRender()\n> >>> \t\tconfigureTexture(*textures_[0]);\n> >>> \t\tglTexImage2D(GL_TEXTURE_2D,\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t size_.width(),\n> >>> \t\t\t size_.height(),\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t GL_UNSIGNED_BYTE,\n> >>> \t\t\t data_);\n> >>> \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n> >>> @@ -495,11 +495,11 @@ void ViewFinderGL::doRender()\n> >>> \t\tconfigureTexture(*textures_[1]);\n> >>> \t\tglTexImage2D(GL_TEXTURE_2D,\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RG,\n> >>> +\t\t\t GL_LUMINANCE_ALPHA,\n> >>> \t\t\t size_.width() / horzSubSample_,\n> >>> \t\t\t size_.height() / vertSubSample_,\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RG,\n> >>> +\t\t\t GL_LUMINANCE_ALPHA,\n> >>> \t\t\t GL_UNSIGNED_BYTE,\n> >>> \t\t\t data_ + size_.width() * size_.height());\n> >>> \t\tshaderProgram_.setUniformValue(textureUniformU_, 1);\n> >>> @@ -511,11 +511,11 @@ void ViewFinderGL::doRender()\n> >>> \t\tconfigureTexture(*textures_[0]);\n> >>> \t\tglTexImage2D(GL_TEXTURE_2D,\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t size_.width(),\n> >>> \t\t\t size_.height(),\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t GL_UNSIGNED_BYTE,\n> >>> \t\t\t data_);\n> >>> \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n> >>> @@ -525,11 +525,11 @@ void ViewFinderGL::doRender()\n> >>> \t\tconfigureTexture(*textures_[1]);\n> >>> \t\tglTexImage2D(GL_TEXTURE_2D,\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t size_.width() / horzSubSample_,\n> >>> \t\t\t size_.height() / vertSubSample_,\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t GL_UNSIGNED_BYTE,\n> >>> \t\t\t data_ + size_.width() * size_.height());\n> >>> \t\tshaderProgram_.setUniformValue(textureUniformU_, 1);\n> >>> @@ -539,11 +539,11 @@ void ViewFinderGL::doRender()\n> >>> \t\tconfigureTexture(*textures_[2]);\n> >>> \t\tglTexImage2D(GL_TEXTURE_2D,\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t size_.width() / horzSubSample_,\n> >>> \t\t\t size_.height() / vertSubSample_,\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t GL_UNSIGNED_BYTE,\n> >>> \t\t\t data_ + size_.width() * size_.height() * 5 / 4);\n> >>> \t\tshaderProgram_.setUniformValue(textureUniformV_, 2);\n> >>> @@ -555,11 +555,11 @@ void ViewFinderGL::doRender()\n> >>> \t\tconfigureTexture(*textures_[0]);\n> >>> \t\tglTexImage2D(GL_TEXTURE_2D,\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t size_.width(),\n> >>> \t\t\t size_.height(),\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t GL_UNSIGNED_BYTE,\n> >>> \t\t\t data_);\n> >>> \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n> >>> @@ -569,11 +569,11 @@ void ViewFinderGL::doRender()\n> >>> \t\tconfigureTexture(*textures_[2]);\n> >>> \t\tglTexImage2D(GL_TEXTURE_2D,\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t size_.width() / horzSubSample_,\n> >>> \t\t\t size_.height() / vertSubSample_,\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t GL_UNSIGNED_BYTE,\n> >>> \t\t\t data_ + size_.width() * size_.height());\n> >>> \t\tshaderProgram_.setUniformValue(textureUniformV_, 2);\n> >>> @@ -583,11 +583,11 @@ void ViewFinderGL::doRender()\n> >>> \t\tconfigureTexture(*textures_[1]);\n> >>> \t\tglTexImage2D(GL_TEXTURE_2D,\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t size_.width() / horzSubSample_,\n> >>> \t\t\t size_.height() / vertSubSample_,\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t GL_UNSIGNED_BYTE,\n> >>> \t\t\t data_ + size_.width() * size_.height() * 5 / 4);\n> >>> \t\tshaderProgram_.setUniformValue(textureUniformU_, 1);\n> >>> @@ -674,18 +674,18 @@ void ViewFinderGL::doRender()\n> >>> \tcase libcamera::formats::SRGGB12_CSI2P:\n> >>> \t\t/*\n> >>> \t\t * Raw Bayer 8-bit, and packed raw Bayer 10-bit/12-bit formats\n> >>> -\t\t * are stored in GL_RED texture.\n> >>> -\t\t * The texture width is equal to the stride.\n> >>> +\t\t * are stored in GL_LUMINANCE texture. The texture width is\n> >>\n> >> I know you're modifying an existing sentence, but should this read either:\n> >>\n> >> - are stored in a GL_LUMINANCE texture\n> >> or\n> >> - are stored in the GL_LUMINANCE texture\n> >>\n> >> depending on if there is only one GL_LUMINANCE texture, or if there\n> >> might be more and this is specifically allocated.\n> > \n> > I'll fix it, using \"a GL_LUMINANCE texture\".\n> > \n> >> (That extra clarification would help me understand those identifiers\n> >> too... such power from a single word addition).\n> >>\n> >> Anyway, aside from / with that:\n> >>\n> >> Reviewed-by: Kieran Bingham \n> >>\n> >>> +\t\t * equal to the stride.\n> >>> \t\t */\n> >>> \t\tglActiveTexture(GL_TEXTURE0);\n> >>> \t\tconfigureTexture(*textures_[0]);\n> >>> \t\tglTexImage2D(GL_TEXTURE_2D,\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t stride_,\n> >>> \t\t\t size_.height(),\n> >>> \t\t\t 0,\n> >>> -\t\t\t GL_RED,\n> >>> +\t\t\t GL_LUMINANCE,\n> >>> \t\t\t GL_UNSIGNED_BYTE,\n> >>> \t\t\t data_);\n> >>> \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n> >>>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 34375C3232\n\tfor ;\n\tTue, 3 Aug 2021 20:38:34 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A7F0168536;\n\tTue, 3 Aug 2021 22:38:33 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 890516026A\n\tfor ;\n\tTue, 3 Aug 2021 22:38:32 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 0DDDB3F0;\n\tTue, 3 Aug 2021 22:38:31 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Rksh3pge\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1628023112;\n\tbh=09yG/Ntok0IzOTOAzIpzr/aPsA5FcZZBe+I0pQjTtAE=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=Rksh3pge+v4DGFkmXDSGHsEDE8aNVRCLBy3qyUmXWBCB7C2x3n6x8JBQJSxnO4Nw8\n\toiRnNjJ58Sk0vv+zfOztL5JFKC/WTV01FznQp2RrrWXS9uxrOT0xpAyavP1hqSLu5F\n\t6rEtRHKwTpkd5EoCjN634K6sXRaojvQs4ojj2OPI=","Date":"Tue, 3 Aug 2021 23:38:19 +0300","From":"Laurent Pinchart ","To":"Andrey Konovalov ","Message-ID":"","References":"<20210803092310.11956-1-laurent.pinchart@ideasonboard.com>\n\t\n\t\n\t<0bfae557-d31a-5bb6-d028-44bcde45f01a@linaro.org>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<0bfae557-d31a-5bb6-d028-44bcde45f01a@linaro.org>","Subject":"Re: [libcamera-devel] [PATCH] qcam: Support OpenGL ES 2.0","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]