{"id":4198,"url":"https://patchwork.libcamera.org/api/1.1/patches/4198/?format=json","web_url":"https://patchwork.libcamera.org/patch/4198/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20200624073705.14737-2-show.liu@linaro.org>","date":"2020-06-24T07:37:05","name":"[libcamera-devel,v3,1/1] qcam: Render YUV formats frame by OpenGL shader","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"10125f36fb469689704f699501fb29003340842f","submitter":{"id":24,"url":"https://patchwork.libcamera.org/api/1.1/people/24/?format=json","name":"Show Liu","email":"show.liu@linaro.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/4198/mbox/","series":[{"id":1035,"url":"https://patchwork.libcamera.org/api/1.1/series/1035/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=1035","date":"2020-06-24T07:37:04","name":"qcam: Render YUV formats frame by OpenGL shader","version":3,"mbox":"https://patchwork.libcamera.org/series/1035/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/4198/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/4198/checks/","tags":{},"headers":{"Return-Path":"","Received":["from mail-pg1-x544.google.com (mail-pg1-x544.google.com\n\t[IPv6:2607:f8b0:4864:20::544])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 97E83609B3\n\tfor ;\n\tWed, 24 Jun 2020 09:37:26 +0200 (CEST)","by mail-pg1-x544.google.com with SMTP id t6so1025435pgq.1\n\tfor ;\n\tWed, 24 Jun 2020 00:37:26 -0700 (PDT)","from localhost.localdomain (211-20-20-223.HINET-IP.hinet.net.\n\t[211.20.20.223]) by smtp.gmail.com with ESMTPSA id\n\tb19sm4281226pjo.57.2020.06.24.00.37.22\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tWed, 24 Jun 2020 00:37:23 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=linaro.org\n\theader.i=@linaro.org header.b=\"jyUAi5+e\"; \n\tdkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=from:to:cc:subject:date:message-id:in-reply-to:references\n\t:mime-version:content-transfer-encoding;\n\tbh=pSZyXUI79qqOOKSmYAcge+eR8C6r79IJIuFPZ+hzHoo=;\n\tb=jyUAi5+eO7VwN34X7M3jQERlFqlCy7zPcfMkt1qeAd9Hu9OXzZRvBEuWXMrx6yfUoZ\n\tooppCGOPAIt3hrdXu9p8W6trw/UdCxfyB8xBSbDlXu0kxJUetblw2kvhQDRxWyWAB9MG\n\tssv2KT3b13xcOt0fMfLMcxjR8DHhTRHkzpL4hPMCOFksQXOvFnYk3Vk9L1dnC9MIjs5E\n\ti/oOuubKA7Ns7XyMC2I82q9aEqZi24XmBEX9JnewnR++KoUrI/0+KBqQaGwkAOvQeGZb\n\tSiuGVDbsvusBYb5FH4g+ojC4z9NRU1oaGjTft/ef4Ads+cU61D93ub8KcESvjYcDxYGc\n\tuWxA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references:mime-version:content-transfer-encoding;\n\tbh=pSZyXUI79qqOOKSmYAcge+eR8C6r79IJIuFPZ+hzHoo=;\n\tb=MlNAqxJ/EyNPW2KccHWUUTh5Ob9V8XVVl+yvy1YW/p8IPd9CARL7iGCbM3etMmvY+t\n\tiLwG703P5Z/DaR+BctHo5GAceuKgQndK5+T7YLzKrrKMTFAqr8i6luugJxYmc+3F73Z6\n\ttL1vY0BcNCKvLVCd46qLn57YXu99Q3jL97etOGDLaNA0koZOEQwHvfczvrnWA+/2qzwo\n\tFcNUX4AOHEhCCZFGwLOmNgzMOgbKcMyRBX991ivbwu5wKhvL3kFgkMdyVfuDXtSs/j5G\n\thpg2dslz/wrlQiPXtwSldQr8gh5dX81Hi4VyuX/1KUGQB8QMx9rnvk/PbBQdSD8KACY5\n\ttF2g==","X-Gm-Message-State":"AOAM532QJaEFeOdUJkhGXBseeulpfhZtvwyUE1jbJZOvDeYi2Rz87va8\n\tl1u8OqDbOTz79cQ3d28I19r9fBC4FZ7/wg==","X-Google-Smtp-Source":"ABdhPJwFkxHr68s9j7UAfjBq92iPIt2lWXceqU6hTZnET32B8b6xQU4ymn9uFmhHnBRAGlCpBuh1/g==","X-Received":"by 2002:a62:ce46:: with SMTP id\n\ty67mr3557920pfg.118.1592984244226; \n\tWed, 24 Jun 2020 00:37:24 -0700 (PDT)","From":"Show Liu ","To":"libcamera-devel@lists.libcamera.org","Date":"Wed, 24 Jun 2020 15:37:05 +0800","Message-Id":"<20200624073705.14737-2-show.liu@linaro.org>","X-Mailer":"git-send-email 2.20.1","In-Reply-To":"<20200624073705.14737-1-show.liu@linaro.org>","References":"<20200624073705.14737-1-show.liu@linaro.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v3 1/1] qcam: Render YUV formats frame by\n\tOpenGL shader","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Wed, 24 Jun 2020 07:37:27 -0000"},"content":"The patch is to render the NV family YUV formats by OpenGL shader.\nV3: refine the fragment shader for better pixel color.\n\nSigned-off-by: Show Liu \n---\n src/qcam/main.cpp | 2 +\n src/qcam/main_window.cpp | 19 ++-\n src/qcam/main_window.h | 3 +-\n src/qcam/meson.build | 2 +\n src/qcam/shader.h | 104 ++++++++++++\n src/qcam/viewfinder.cpp | 18 +-\n src/qcam/viewfinder.h | 23 ++-\n src/qcam/viewfinderGL.cpp | 335 ++++++++++++++++++++++++++++++++++++++\n src/qcam/viewfinderGL.h | 101 ++++++++++++\n 9 files changed, 593 insertions(+), 14 deletions(-)\n create mode 100644 src/qcam/shader.h\n create mode 100644 src/qcam/viewfinderGL.cpp\n create mode 100644 src/qcam/viewfinderGL.h","diff":"diff --git a/src/qcam/main.cpp b/src/qcam/main.cpp\nindex b3468cb..a3ea471 100644\n--- a/src/qcam/main.cpp\n+++ b/src/qcam/main.cpp\n@@ -35,6 +35,8 @@ OptionsParser::Options parseOptions(int argc, char *argv[])\n \t\t\t \"help\");\n \tparser.addOption(OptStream, &streamKeyValue,\n \t\t\t \"Set configuration of a camera stream\", \"stream\", true);\n+\tparser.addOption(OptOpenGL, OptionNone, \"Render YUV formats frame via OpenGL shader\",\n+\t\t\t \"opengl\");\n \n \tOptionsParser::Options options = parser.parse(argc, argv);\n \tif (options.isSet(OptHelp))\ndiff --git a/src/qcam/main_window.cpp b/src/qcam/main_window.cpp\nindex 7bc1360..6edf370 100644\n--- a/src/qcam/main_window.cpp\n+++ b/src/qcam/main_window.cpp\n@@ -28,6 +28,9 @@\n #include \n \n #include \"dng_writer.h\"\n+#include \"main_window.h\"\n+#include \"viewfinder.h\"\n+#include \"viewfinderGL.h\"\n \n using namespace libcamera;\n \n@@ -105,10 +108,18 @@ MainWindow::MainWindow(CameraManager *cm, const OptionsParser::Options &options)\n \tsetWindowTitle(title_);\n \tconnect(&titleTimer_, SIGNAL(timeout()), this, SLOT(updateTitle()));\n \n-\tviewfinder_ = new ViewFinder(this);\n-\tconnect(viewfinder_, &ViewFinder::renderComplete,\n-\t\tthis, &MainWindow::queueRequest);\n-\tsetCentralWidget(viewfinder_);\n+\tif (options_.isSet(OptOpenGL)) {\n+\t\tviewfinder_ = new ViewFinderGL(this);\n+\t\tconnect(dynamic_cast(viewfinder_), &ViewFinderGL::renderComplete,\n+\t\t\t\tthis, &MainWindow::queueRequest);\n+\t\tsetCentralWidget(dynamic_cast(viewfinder_));\n+\t} else {\n+\t\tviewfinder_ = new ViewFinder(this);\n+\t\tconnect(dynamic_cast(viewfinder_), &ViewFinder::renderComplete,\n+\t\t\t\tthis, &MainWindow::queueRequest);\n+\t\tsetCentralWidget(dynamic_cast(viewfinder_));\n+\t}\n+\n \tadjustSize();\n \n \t/* Hotplug/unplug support */\ndiff --git a/src/qcam/main_window.h b/src/qcam/main_window.h\nindex 4606fe4..a852ef4 100644\n--- a/src/qcam/main_window.h\n+++ b/src/qcam/main_window.h\n@@ -38,6 +38,7 @@ enum {\n \tOptCamera = 'c',\n \tOptHelp = 'h',\n \tOptStream = 's',\n+\tOptOpenGL = 'o',\n };\n \n class CaptureRequest\n@@ -102,7 +103,7 @@ private:\n \tQAction *startStopAction_;\n \tQComboBox *cameraCombo_;\n \tQAction *saveRaw_;\n-\tViewFinder *viewfinder_;\n+\tViewFinderHandler *viewfinder_;\n \n \tQIcon iconPlay_;\n \tQIcon iconStop_;\ndiff --git a/src/qcam/meson.build b/src/qcam/meson.build\nindex 045db52..6a58947 100644\n--- a/src/qcam/meson.build\n+++ b/src/qcam/meson.build\n@@ -7,11 +7,13 @@ qcam_sources = files([\n 'main.cpp',\n 'main_window.cpp',\n 'viewfinder.cpp',\n+ 'viewfinderGL.cpp'\n ])\n \n qcam_moc_headers = files([\n 'main_window.h',\n 'viewfinder.h',\n+ 'viewfinderGL.h'\n ])\n \n qcam_resources = files([\ndiff --git a/src/qcam/shader.h b/src/qcam/shader.h\nnew file mode 100644\nindex 0000000..f54c264\n--- /dev/null\n+++ b/src/qcam/shader.h\n@@ -0,0 +1,104 @@\n+/* SPDX-License-Identifier: GPL-2.0-or-later */\n+/*\n+ * Copyright (C) 2020, Linaro\n+ *\n+ * shader.h - YUV format converter shader source code\n+ */\n+#ifndef __SHADER_H__\n+#define __SHADER_H__\n+\n+/* Vertex shader for NV formats */\n+char NV_Vertex_shader[] = \"attribute vec4 vertexIn;\\n\"\n+ \"attribute vec2 textureIn;\\n\"\n+ \"varying vec2 textureOut;\\n\"\n+ \"void main(void)\\n\"\n+ \"{\\n\"\n+ \"gl_Position = vertexIn;\\n\"\n+ \"textureOut = textureIn;\\n\"\n+ \"}\\n\";\n+\n+/* Fragment shader for NV12, NV16 and NV24 */\n+char NV_2_planes_UV[] = \"#ifdef GL_ES\\n\"\n+ \"precision highp float;\\n\"\n+ \"#endif\\n\"\n+ \"varying vec2 textureOut;\\n\"\n+ \"uniform sampler2D tex_y;\\n\"\n+ \"uniform sampler2D tex_u;\\n\"\n+ \"void main(void)\\n\"\n+ \"{\\n\"\n+ \"vec3 yuv;\\n\"\n+ \"vec3 rgb;\\n\"\n+ \"mat3 convert_mat = mat3(vec3(1.1640625, 1.1640625, 1.1640625),\\n\"\n+ \" vec3(0.0, -0.390625, 2.015625),\\n\"\n+ \" vec3(1.5975625, -0.8125, 0.0));\\n\"\n+ \"yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\\n\"\n+ \"yuv.y = texture2D(tex_u, textureOut).r - 0.5;\\n\"\n+ \"yuv.z = texture2D(tex_u, textureOut).g - 0.5;\\n\"\n+ \"rgb = convert_mat * yuv;\\n\"\n+ \"gl_FragColor = vec4(rgb, 1.0);\\n\"\n+ \"}\\n\";\n+\n+/* Fragment shader for NV21, NV61 and NV42 */\n+char NV_2_planes_VU[] = \"#ifdef GL_ES\\n\"\n+ \"precision highp float;\\n\"\n+ \"#endif\\n\"\n+ \"varying vec2 textureOut;\\n\"\n+ \"uniform sampler2D tex_y;\\n\"\n+ \"uniform sampler2D tex_u;\\n\"\n+ \"void main(void)\\n\"\n+ \"{\\n\"\n+ \"vec3 yuv;\\n\"\n+ \"vec3 rgb;\\n\"\n+ \"mat3 convert_mat = mat3(vec3(1.1640625, 1.1640625, 1.1640625),\\n\"\n+ \" vec3(0.0, -0.390625, 2.015625),\\n\"\n+ \" vec3(1.5975625, -0.8125, 0.0));\\n\"\n+ \"yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\\n\"\n+ \"yuv.y = texture2D(tex_u, textureOut).g - 0.5;\\n\"\n+ \"yuv.z = texture2D(tex_u, textureOut).r - 0.5;\\n\"\n+ \"rgb = convert_mat * yuv;\\n\"\n+ \"gl_FragColor = vec4(rgb, 1.0);\\n\"\n+ \"}\\n\";\n+\n+/* Fragment shader for YUV420 */\n+char NV_3_planes_UV[] = \"#ifdef GL_ES\\n\"\n+ \"precision highp float;\\n\"\n+ \"#endif\\n\"\n+ \"varying vec2 textureOut;\\n\"\n+ \"uniform sampler2D tex_y;\\n\"\n+ \"uniform sampler2D tex_u;\\n\"\n+ \"uniform sampler2D tex_v;\\n\"\n+ \"void main(void)\\n\"\n+ \"{\\n\"\n+ \"vec3 yuv;\\n\"\n+ \"vec3 rgb;\\n\"\n+ \"mat3 convert_mat = mat3(vec3(1.1640625, 1.1640625, 1.1640625),\\n\"\n+ \" vec3(0.0, -0.390625, 2.015625),\\n\"\n+ \" vec3(1.5975625, -0.8125, 0.0));\\n\"\n+ \"yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\\n\"\n+ \"yuv.y = texture2D(tex_u, textureOut).r - 0.5;\\n\"\n+ \"yuv.z = texture2D(tex_v, textureOut).g - 0.5;\\n\"\n+ \"rgb = convert_mat * yuv;\\n\"\n+ \"gl_FragColor = vec4(rgb, 1);\\n\"\n+ \"}\\n\";\n+\n+/* Fragment shader for YVU420 */\n+char NV_3_planes_VU[] = \"precision highp float;\\n\"\n+ \"#endif\\n\"\n+ \"varying vec2 textureOut;\\n\"\n+ \"uniform sampler2D tex_y;\\n\"\n+ \"uniform sampler2D tex_u;\\n\"\n+ \"uniform sampler2D tex_v;\\n\"\n+ \"void main(void)\\n\"\n+ \"{\\n\"\n+ \"vec3 yuv;\\n\"\n+ \"vec3 rgb;\\n\"\n+ \"mat3 convert_mat = mat3(vec3(1.1640625, 1.1640625, 1.1640625),\\n\"\n+ \" vec3(0.0, -0.390625, 2.015625),\\n\"\n+ \" vec3(1.5975625, -0.8125, 0.0));\\n\"\n+ \"yuv.x = texture2D(tex_y, textureOut).r - 0.0625;\\n\"\n+ \"yuv.y = texture2D(tex_v, textureOut).g - 0.5;\\n\"\n+ \"yuv.z = texture2D(tex_u, textureOut).r - 0.5;\\n\"\n+ \"rgb = convert_mat * yuv;\\n\"\n+ \"gl_FragColor = vec4(rgb, 1);\\n\"\n+ \"}\\n\";\n+#endif // __SHADER_H__\ndiff --git a/src/qcam/viewfinder.cpp b/src/qcam/viewfinder.cpp\nindex dcf8a15..d3a2422 100644\n--- a/src/qcam/viewfinder.cpp\n+++ b/src/qcam/viewfinder.cpp\n@@ -33,22 +33,30 @@ static const QMap nativeFormats\n \t{ libcamera::formats::BGR888, QImage::Format_RGB888 },\n };\n \n-ViewFinder::ViewFinder(QWidget *parent)\n-\t: QWidget(parent), buffer_(nullptr)\n+ViewFinderHandler::ViewFinderHandler()\n {\n-\ticon_ = QIcon(\":camera-off.svg\");\n }\n \n-ViewFinder::~ViewFinder()\n+ViewFinderHandler::~ViewFinderHandler()\n {\n }\n \n-const QList &ViewFinder::nativeFormats() const\n+const QList &ViewFinderHandler::nativeFormats() const\n {\n \tstatic const QList formats = ::nativeFormats.keys();\n \treturn formats;\n }\n \n+ViewFinder::ViewFinder(QWidget *parent)\n+\t: QWidget(parent), buffer_(nullptr)\n+{\n+\ticon_ = QIcon(\":camera-off.svg\");\n+}\n+\n+ViewFinder::~ViewFinder()\n+{\n+}\n+\n int ViewFinder::setFormat(const libcamera::PixelFormat &format,\n \t\t\t const QSize &size)\n {\ndiff --git a/src/qcam/viewfinder.h b/src/qcam/viewfinder.h\nindex 26a1320..741d694 100644\n--- a/src/qcam/viewfinder.h\n+++ b/src/qcam/viewfinder.h\n@@ -13,6 +13,8 @@\n #include \n #include \n #include \n+#include \n+#include \n #include \n #include \n \n@@ -28,7 +30,23 @@ struct MappedBuffer {\n \tsize_t size;\n };\n \n-class ViewFinder : public QWidget\n+class ViewFinderHandler\n+{\n+public:\n+\tViewFinderHandler();\n+\tvirtual ~ViewFinderHandler();\n+\n+\tconst QList &nativeFormats() const;\n+\n+\tvirtual int setFormat(const libcamera::PixelFormat &format, const QSize &size) =0;\n+\tvirtual void render(libcamera::FrameBuffer *buffer, MappedBuffer *map) =0;\n+\tvirtual void stop() =0;\n+\n+\tvirtual QImage getCurrentImage() =0;\n+\n+};\n+\n+class ViewFinder : public QWidget, public ViewFinderHandler\n {\n \tQ_OBJECT\n \n@@ -36,8 +54,6 @@ public:\n \tViewFinder(QWidget *parent);\n \t~ViewFinder();\n \n-\tconst QList &nativeFormats() const;\n-\n \tint setFormat(const libcamera::PixelFormat &format, const QSize &size);\n \tvoid render(libcamera::FrameBuffer *buffer, MappedBuffer *map);\n \tvoid stop();\n@@ -67,5 +83,4 @@ private:\n \tQImage image_;\n \tQMutex mutex_; /* Prevent concurrent access to image_ */\n };\n-\n #endif /* __QCAM_VIEWFINDER__ */\ndiff --git a/src/qcam/viewfinderGL.cpp b/src/qcam/viewfinderGL.cpp\nnew file mode 100644\nindex 0000000..7b47beb\n--- /dev/null\n+++ b/src/qcam/viewfinderGL.cpp\n@@ -0,0 +1,335 @@\n+/* SPDX-License-Identifier: GPL-2.0-or-later */\n+/*\n+ * Copyright (C) 2020, Linaro\n+ *\n+ * viewfinderGL.cpp - Render YUV format frame by OpenGL shader\n+ */\n+\n+#include \"shader.h\"\n+#include \"viewfinderGL.h\"\n+\n+#include \n+\n+#include \n+\n+#define ATTRIB_VERTEX 0\n+#define ATTRIB_TEXTURE 1\n+\n+ViewFinderGL::ViewFinderGL(QWidget *parent)\n+\t: QOpenGLWidget(parent),\n+\tglBuffer(QOpenGLBuffer::VertexBuffer),\n+\tpFShader(nullptr),\n+\tpVShader(nullptr),\n+\ttextureU(QOpenGLTexture::Target2D),\n+\ttextureV(QOpenGLTexture::Target2D),\n+\ttextureY(QOpenGLTexture::Target2D),\n+\tyuvDataPtr(nullptr)\n+\n+{\n+}\n+\n+ViewFinderGL::~ViewFinderGL()\n+{\n+\tremoveShader();\n+\n+\tif(textureY.isCreated())\n+\t\ttextureY.destroy();\n+\n+\tif(textureU.isCreated())\n+\t\ttextureU.destroy();\n+\n+\tif(textureV.isCreated())\n+\t\ttextureV.destroy();\n+\n+\tglBuffer.destroy();\n+}\n+\n+int ViewFinderGL::setFormat(const libcamera::PixelFormat &format,\n+\t\t\t const QSize &size)\n+{\n+\tformat_ = format;\n+\tsize_ = size;\n+\n+\tupdateGeometry();\n+\treturn 0;\n+}\n+\n+void ViewFinderGL::stop()\n+{\n+\tif (buffer_) {\n+\t\trenderComplete(buffer_);\n+\t\tbuffer_ = nullptr;\n+\t}\n+}\n+\n+QImage ViewFinderGL::getCurrentImage()\n+{\n+\tQMutexLocker locker(&mutex_);\n+\n+\treturn(grabFramebuffer());\n+}\n+\n+void ViewFinderGL::render(libcamera::FrameBuffer *buffer, MappedBuffer *map)\n+{\n+\tif (buffer->planes().size() != 1) {\n+\t\tqWarning() << \"Multi-planar buffers are not supported\";\n+\t\treturn;\n+\t}\n+\n+\tunsigned char *memory = static_cast(map->memory);\n+\tif(memory) {\n+\t\tyuvDataPtr = memory;\n+\t\tupdate();\n+\t\tbuffer_ = buffer;\n+\t}\n+\n+\tif (buffer)\n+\t\trenderComplete(buffer);\n+}\n+\n+void ViewFinderGL::updateFrame(unsigned char *buffer)\n+{\n+\tif(buffer) {\n+\t\tyuvDataPtr = buffer;\n+\t\tupdate();\n+\t}\n+}\n+\n+void ViewFinderGL::setFrameSize(int width, int height)\n+{\n+\tif(width > 0 && height > 0) {\n+\t\twidth_ = width;\n+\t\theight_ = height;\n+\t}\n+}\n+\n+char *ViewFinderGL::selectFragmentShader(unsigned int format)\n+{\n+\tchar *fsrc = nullptr;\n+\n+\tswitch(format) {\n+\tcase libcamera::formats::NV12:\n+\t\thorzSubSample_ = 2;\n+\t\tvertSubSample_ = 2;\n+\t\tfsrc = NV_2_planes_UV;\n+\t\tbreak;\n+\tcase libcamera::formats::NV21:\n+\t\thorzSubSample_ = 2;\n+\t\tvertSubSample_ = 2;\n+\t\tfsrc = NV_2_planes_VU;\n+\t\tbreak;\n+\tcase libcamera::formats::NV16:\n+\t\thorzSubSample_ = 2;\n+\t\tvertSubSample_ = 1;\n+\t\tfsrc = NV_2_planes_UV;\n+\t\tbreak;\n+\tcase libcamera::formats::NV61:\n+\t\thorzSubSample_ = 2;\n+\t\tvertSubSample_ = 1;\n+\t\tfsrc = NV_2_planes_VU;\n+\t\tbreak;\n+\tcase libcamera::formats::NV24:\n+\t\thorzSubSample_ = 1;\n+\t\tvertSubSample_ = 1;\n+\t\tfsrc = NV_2_planes_UV;\n+\t\tbreak;\n+\tcase libcamera::formats::NV42:\n+\t\thorzSubSample_ = 1;\n+\t\tvertSubSample_ = 1;\n+\t\tfsrc = NV_2_planes_VU;\n+\t\tbreak;\n+\tcase libcamera::formats::YUV420:\n+\t\thorzSubSample_ = 2;\n+\t\tvertSubSample_ = 2;\n+\t\tfsrc = NV_3_planes_UV;\n+\t\tbreak;\n+\tdefault:\n+\t\tbreak;\n+\t};\n+\n+\tif(fsrc == nullptr) {\n+\t\tqDebug() << __func__ << \"[ERROR]:\" <<\" No suitable fragment shader can be select.\";\n+\t}\n+\treturn fsrc;\n+}\n+\n+void ViewFinderGL::createFragmentShader()\n+{\n+\tbool bCompile;\n+\n+\tpFShader = new QOpenGLShader(QOpenGLShader::Fragment, this);\n+\tchar *fsrc = selectFragmentShader(format_);\n+\n+\tbCompile = pFShader->compileSourceCode(fsrc);\n+\tif(!bCompile)\n+\t{\n+\t\tqDebug() << __func__ << \":\" << pFShader->log();\n+\t}\n+\n+\tshaderProgram.addShader(pFShader);\n+\n+\t// Link shader pipeline\n+\tif (!shaderProgram.link()) {\n+\t\tqDebug() << __func__ << \": shader program link failed.\\n\" << shaderProgram.log();\n+\t\tclose();\n+\t}\n+\n+\t// Bind shader pipeline for use\n+\tif (!shaderProgram.bind()) {\n+\t\tqDebug() << __func__ << \": shader program binding failed.\\n\" << shaderProgram.log();\n+\t\tclose();\n+\t}\n+\n+\tshaderProgram.enableAttributeArray(ATTRIB_VERTEX);\n+\tshaderProgram.enableAttributeArray(ATTRIB_TEXTURE);\n+\n+\tshaderProgram.setAttributeBuffer(ATTRIB_VERTEX,GL_FLOAT,0,2,2*sizeof(GLfloat));\n+\tshaderProgram.setAttributeBuffer(ATTRIB_TEXTURE,GL_FLOAT,8*sizeof(GLfloat),2,2*sizeof(GLfloat));\n+\n+\ttextureUniformY = shaderProgram.uniformLocation(\"tex_y\");\n+\ttextureUniformU = shaderProgram.uniformLocation(\"tex_u\");\n+\ttextureUniformV = shaderProgram.uniformLocation(\"tex_v\");\n+\n+\tif(!textureY.isCreated())\n+\t\ttextureY.create();\n+\n+\tif(!textureU.isCreated())\n+\t\ttextureU.create();\n+\n+\tif(!textureV.isCreated())\n+\t\ttextureV.create();\n+\n+\tid_y = textureY.textureId();\n+\tid_u = textureU.textureId();\n+\tid_v = textureV.textureId();\n+}\n+\n+void ViewFinderGL::configureTexture(unsigned int id)\n+{\n+\tglBindTexture(GL_TEXTURE_2D, id);\n+\tglTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);\n+\tglTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);\n+\tglTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);\n+\tglTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);\n+}\n+\n+void ViewFinderGL::removeShader()\n+{\n+\tif (shaderProgram.isLinked()) {\n+\t\tshaderProgram.release();\n+\t\tshaderProgram.removeAllShaders();\n+\t}\n+\n+\tif(pFShader)\n+\t\tdelete pFShader;\n+\n+\tif(pVShader)\n+\t\tdelete pVShader;\n+}\n+\n+void ViewFinderGL::initializeGL()\n+{\n+\tbool bCompile;\n+\n+\tinitializeOpenGLFunctions();\n+\tglEnable(GL_TEXTURE_2D);\n+\tglDisable(GL_DEPTH_TEST);\n+\n+\tstatic const GLfloat vertices[] {\n+\t\t-1.0f,-1.0f,\n+\t\t-1.0f,+1.0f,\n+\t\t+1.0f,+1.0f,\n+\t\t+1.0f,-1.0f,\n+\t\t0.0f,1.0f,\n+\t\t0.0f,0.0f,\n+\t\t1.0f,0.0f,\n+\t\t1.0f,1.0f,\n+\t};\n+\n+\tglBuffer.create();\n+\tglBuffer.bind();\n+\tglBuffer.allocate(vertices,sizeof(vertices));\n+\n+\t/* Create Vertex Shader */\n+\tpVShader = new QOpenGLShader(QOpenGLShader::Vertex, this);\n+\tchar *vsrc = NV_Vertex_shader;\n+\n+\tbCompile = pVShader->compileSourceCode(vsrc);\n+\tif(!bCompile) {\n+\t\tqDebug() << __func__<< \":\" << pVShader->log();\n+\t}\n+\n+\tshaderProgram.addShader(pVShader);\n+\n+\tglClearColor(1.0f, 1.0f, 1.0f, 0.0f);\n+}\n+\n+void ViewFinderGL::render()\n+{\n+\tswitch(format_) {\n+\tcase libcamera::formats::NV12:\n+\tcase libcamera::formats::NV21:\n+\tcase libcamera::formats::NV16:\n+\tcase libcamera::formats::NV61:\n+\tcase libcamera::formats::NV24:\n+\tcase libcamera::formats::NV42:\n+\t\t/* activate texture 0 */\n+\t\tglActiveTexture(GL_TEXTURE0);\n+\t\tconfigureTexture(id_y);\n+\t\tglTexImage2D(GL_TEXTURE_2D, 0, GL_RED, size_.width(), size_.height(), 0, GL_RED, GL_UNSIGNED_BYTE, yuvDataPtr);\n+\t\tglUniform1i(textureUniformY, 0);\n+\n+\t\t/* activate texture 1 */\n+\t\tglActiveTexture(GL_TEXTURE1);\n+\t\tconfigureTexture(id_u);\n+\t\tglTexImage2D(GL_TEXTURE_2D, 0, GL_RG, size_.width()/horzSubSample_, size_.height()/vertSubSample_, 0, GL_RG, GL_UNSIGNED_BYTE, (char*)yuvDataPtr+size_.width()*size_.height());\n+\t\tglUniform1i(textureUniformU, 1);\n+\t\tbreak;\n+\tcase libcamera::formats::YUV420:\n+\t\t/* activate texture 0 */\n+\t\tglActiveTexture(GL_TEXTURE0);\n+\t\tconfigureTexture(id_y);\n+\t\tglTexImage2D(GL_TEXTURE_2D, 0, GL_RED, size_.width(), size_.height(), 0, GL_RED, GL_UNSIGNED_BYTE, yuvDataPtr);\n+\t\tglUniform1i(textureUniformY, 0);\n+\n+\t\t/* activate texture 1 */\n+\t\tglActiveTexture(GL_TEXTURE1);\n+\t\tconfigureTexture(id_u);\n+\t\tglTexImage2D(GL_TEXTURE_2D, 0, GL_RG, size_.width()/horzSubSample_, size_.height()/vertSubSample_, 0, GL_RG, GL_UNSIGNED_BYTE, (char*)yuvDataPtr+size_.width()*size_.height());\n+\t\tglUniform1i(textureUniformU, 1);\n+\n+\t\t/* activate texture 2 */\n+\t\tglActiveTexture(GL_TEXTURE2);\n+\t\tconfigureTexture(id_v);\n+\t\tglTexImage2D(GL_TEXTURE_2D, 0, GL_RG, size_.width()/horzSubSample_, size_.height()/vertSubSample_, 0, GL_RG, GL_UNSIGNED_BYTE, (char*)yuvDataPtr+size_.width()*size_.height()*5/4);\n+\t\tglUniform1i(textureUniformV, 2);\n+\t\tbreak;\n+\tdefault:\n+\t\tbreak;\n+\t};\n+}\n+\n+void ViewFinderGL::paintGL()\n+{\n+\tif(pFShader == nullptr)\n+\t\tcreateFragmentShader();\n+\n+\tif(yuvDataPtr)\n+\t{\n+\t\tglClearColor(0.0, 0.0, 0.0, 1.0);\n+\t\tglClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT );\n+\n+\t\trender();\n+\t\tglDrawArrays(GL_TRIANGLE_FAN, 0, 4);\n+\t}\n+}\n+\n+void ViewFinderGL::resizeGL(int w, int h)\n+{\n+\tglViewport(0,0,w,h);\n+}\n+\n+QSize ViewFinderGL::sizeHint() const\n+{\n+\treturn size_.isValid() ? size_ : QSize(640, 480);\n+}\ndiff --git a/src/qcam/viewfinderGL.h b/src/qcam/viewfinderGL.h\nnew file mode 100644\nindex 0000000..08662ca\n--- /dev/null\n+++ b/src/qcam/viewfinderGL.h\n@@ -0,0 +1,101 @@\n+/* SPDX-License-Identifier: GPL-2.0-or-later */\n+/*\n+ * Copyright (C) 2020, Linaro\n+ *\n+ * viewfinderGL.h - Render YUV format frame by OpenGL shader\n+ */\n+#ifndef __VIEWFINDERGL_H__\n+#define __VIEWFINDERGL_H__\n+\n+#include \n+#include \n+#include \n+\n+#include \n+#include \n+#include \n+\n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+\n+#include \n+#include \n+#include \"viewfinder.h\"\n+\n+class ViewFinderGL : public QOpenGLWidget,\n+\t\t\t\t\t public ViewFinderHandler,\n+\t\t\t\t\t protected QOpenGLFunctions\n+{\n+\tQ_OBJECT\n+\n+public:\n+\tViewFinderGL(QWidget *parent = 0);\n+\t~ViewFinderGL();\n+\n+\tint setFormat(const libcamera::PixelFormat &format, const QSize &size);\n+\tvoid render(libcamera::FrameBuffer *buffer, MappedBuffer *map);\n+\tvoid stop();\n+\n+\tQImage getCurrentImage();\n+\n+\tvoid setFrameSize(int width, int height);\n+\tvoid updateFrame(unsigned char *buffer);\n+\n+\tchar *selectFragmentShader(unsigned int format);\n+\tvoid createFragmentShader();\n+\tvoid render();\n+\n+Q_SIGNALS:\n+\tvoid renderComplete(libcamera::FrameBuffer *buffer);\n+\n+protected:\n+\tvoid initializeGL() Q_DECL_OVERRIDE;\n+\tvoid paintGL() Q_DECL_OVERRIDE;\n+\tvoid resizeGL(int w, int h) Q_DECL_OVERRIDE;\n+\tQSize sizeHint() const override;\n+\n+private:\n+\n+\tvoid configureTexture(unsigned int id);\n+\tvoid removeShader();\n+\n+\t/* Captured image size, format and buffer */\n+\tlibcamera::FrameBuffer *buffer_;\n+\tlibcamera::PixelFormat format_;\n+\tQOpenGLBuffer glBuffer;\n+\tQSize size_;\n+\n+\tGLuint id_u;\n+\tGLuint id_v;\n+\tGLuint id_y;\n+\n+\tQMutex mutex_; /* Prevent concurrent access to image_ */\n+\n+\tQOpenGLShader *pFShader;\n+\tQOpenGLShader *pVShader;\n+\tQOpenGLShaderProgram shaderProgram;\n+\n+\tGLuint textureUniformU;\n+\tGLuint textureUniformV;\n+\tGLuint textureUniformY;\n+\n+\tQOpenGLTexture textureU;\n+\tQOpenGLTexture textureV;\n+\tQOpenGLTexture textureY;\n+\n+\tunsigned int width_;\n+\tunsigned int height_;\n+\n+\tunsigned char* yuvDataPtr;\n+\n+\t/* NV parameters */\n+\tunsigned int horzSubSample_ ;\n+\tunsigned int vertSubSample_;\n+};\n+#endif // __VIEWFINDERGL_H__\n","prefixes":["libcamera-devel","v3","1/1"]}