From patchwork Fri Sep 11 08:55:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Show Liu X-Patchwork-Id: 9568 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 15C54BDB1D for ; Fri, 11 Sep 2020 08:55:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D5B0E62D53; Fri, 11 Sep 2020 10:55:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="IRuM0hsA"; dkim-atps=neutral Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A2F8A62C8C for ; Fri, 11 Sep 2020 10:55:41 +0200 (CEST) Received: by mail-pg1-x541.google.com with SMTP id 34so6145363pgo.13 for ; Fri, 11 Sep 2020 01:55:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0omJwhTWOXd4elXOuaIzrGc5lHhq8k1FeAGhy5jES0A=; b=IRuM0hsAeZccZaszJ+Hfd9PFwaJhmrNDg9JbNqk6/i4nXubJ9Ntfr1aM/6ETqFWnK9 XewMQ5pb+r73WmtAZr4TcrwJ8td953fsb7iVi3RticoUYnHlYM8A4itjble1z3x5Y1JX TPQApnqardcsWFTjpuvWbm5lkwz51dK1JpEG8XnYWcyHDRxg0ns9GqwhTmeThsnaPlqm EeDqWM4R6UTwA1chJIJgxMQxhlhdk/4m9YSxQHUkqXj4XlPndJQ0BQzmG04kgWiiupxU zIgc/N6b4PepJWIWRlo2yFpQsCxs/g0wn5be2lqtXyuzOAljx5zcateqT66RleCGT82D L5HQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0omJwhTWOXd4elXOuaIzrGc5lHhq8k1FeAGhy5jES0A=; b=CpfVDWg975e/6Z5NBxIes0eEtXRmiKxYMf4OoerXKCe3is2E4Z10J9vBbxSA+EAUZ8 xkMMDiX1QJriPjs9mvM42wQDJvzul9PwBuuzLHbY6AYBvh9hmXgyGtX09ofyOt2Q/CmJ JSoVSdmjWmYeOWUz8Xe7ihaWksD7lsfzxLy72isJXWCSuZy7rYA4BUnIq1eiSGLYoZaf yL9/ytVnUigmhVGG9lH37bpHLdYLwAszZvDLO74Msn6bOoviaGIcppVBUjGJ3ARdy6+w KsphewL8nmo0GEPMhzNRnc3zWlFk4iKmISwgoZ8op70QMzz7Lqsj1+aoanRMwMlKJ8fI YM3w== X-Gm-Message-State: AOAM53329a1aAERBim/IPrZJG2bEE+0Kc9NIoGFyr12ILxKqmdCqnEhJ a3n/QeaoawhaYtrX83vA6NDAXiNk3BlFag== X-Google-Smtp-Source: ABdhPJwc0lLO2qV4zQqkbIqXFGZ8MMbKlMPBT13P/t3auI6DvyFKn9h9XeDXiIA7VweLhZOxj+fQXQ== X-Received: by 2002:a62:7c82:0:b029:13c:1611:6532 with SMTP id x124-20020a627c820000b029013c16116532mr1278583pfc.4.1599814539831; Fri, 11 Sep 2020 01:55:39 -0700 (PDT) Received: from localhost.localdomain (211-20-20-223.HINET-IP.hinet.net. [211.20.20.223]) by smtp.gmail.com with ESMTPSA id w192sm1616543pfd.156.2020.09.11.01.55.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Sep 2020 01:55:39 -0700 (PDT) From: Show Liu To: libcamera-devel@lists.libcamera.org Date: Fri, 11 Sep 2020 16:55:11 +0800 Message-Id: <20200911085514.30021-2-show.liu@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200911085514.30021-1-show.liu@linaro.org> References: <20200911085514.30021-1-show.liu@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 1/4] qcam: add OpenGL shader code as Qt resource X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add OpenGL fragment and vertex shaders to convert two- and tri-planar YUV formats to RGB. This will be used to accelerate YUV image rendering. Signed-off-by: Show Liu Reviewed-by: Laurent Pinchart --- src/qcam/assets/shader/NV_2_planes_UV_f.glsl | 32 +++++++++++++++++++ src/qcam/assets/shader/NV_2_planes_VU_f.glsl | 32 +++++++++++++++++++ src/qcam/assets/shader/NV_3_planes_f.glsl | 33 ++++++++++++++++++++ src/qcam/assets/shader/NV_vertex_shader.glsl | 16 ++++++++++ src/qcam/assets/shader/shaders.qrc | 9 ++++++ src/qcam/meson.build | 1 + 6 files changed, 123 insertions(+) create mode 100644 src/qcam/assets/shader/NV_2_planes_UV_f.glsl create mode 100644 src/qcam/assets/shader/NV_2_planes_VU_f.glsl create mode 100644 src/qcam/assets/shader/NV_3_planes_f.glsl create mode 100644 src/qcam/assets/shader/NV_vertex_shader.glsl create mode 100644 src/qcam/assets/shader/shaders.qrc diff --git a/src/qcam/assets/shader/NV_2_planes_UV_f.glsl b/src/qcam/assets/shader/NV_2_planes_UV_f.glsl new file mode 100644 index 0000000..80478c5 --- /dev/null +++ b/src/qcam/assets/shader/NV_2_planes_UV_f.glsl @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Linaro + * + * NV_2_planes_UV_f.glsl - Fragment shader code for NV12, NV16 and NV24 formats + */ + +#ifdef GL_ES +precision mediump float; +#endif + +varying vec2 textureOut; +uniform sampler2D tex_y; +uniform sampler2D tex_u; + +void main(void) +{ + vec3 yuv; + vec3 rgb; + mat3 yuv2rgb_bt601_mat = mat3( + vec3(1.164, 1.164, 1.164), + vec3(0.000, -0.392, 2.017), + vec3(1.596, -0.813, 0.000) + ); + + yuv.x = texture2D(tex_y, textureOut).r - 0.063; + yuv.y = texture2D(tex_u, textureOut).r - 0.500; + yuv.z = texture2D(tex_u, textureOut).g - 0.500; + + rgb = yuv2rgb_bt601_mat * yuv; + gl_FragColor = vec4(rgb, 1.0); +} diff --git a/src/qcam/assets/shader/NV_2_planes_VU_f.glsl b/src/qcam/assets/shader/NV_2_planes_VU_f.glsl new file mode 100644 index 0000000..3794be8 --- /dev/null +++ b/src/qcam/assets/shader/NV_2_planes_VU_f.glsl @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Linaro + * + * NV_2_planes_VU_f.glsl - Fragment shader code for NV21, NV61 and NV42 formats + */ + +#ifdef GL_ES +precision mediump float; +#endif + +varying vec2 textureOut; +uniform sampler2D tex_y; +uniform sampler2D tex_u; + +void main(void) +{ + vec3 yuv; + vec3 rgb; + mat3 yuv2rgb_bt601_mat = mat3( + vec3(1.164, 1.164, 1.164), + vec3(0.000, -0.392, 2.017), + vec3(1.596, -0.813, 0.000) + ); + + yuv.x = texture2D(tex_y, textureOut).r - 0.063; + yuv.y = texture2D(tex_u, textureOut).g - 0.500; + yuv.z = texture2D(tex_u, textureOut).r - 0.500; + + rgb = yuv2rgb_bt601_mat * yuv; + gl_FragColor = vec4(rgb, 1.0); +} diff --git a/src/qcam/assets/shader/NV_3_planes_f.glsl b/src/qcam/assets/shader/NV_3_planes_f.glsl new file mode 100644 index 0000000..fca9b65 --- /dev/null +++ b/src/qcam/assets/shader/NV_3_planes_f.glsl @@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Linaro + * + * NV_3_planes_UV_f.glsl - Fragment shader code for YUV420 format + */ + +#ifdef GL_ES +precision mediump float; +#endif + +varying vec2 textureOut; +uniform sampler2D tex_y; +uniform sampler2D tex_u; +uniform sampler2D tex_v; + +void main(void) +{ + vec3 yuv; + vec3 rgb; + mat3 yuv2rgb_bt601_mat = mat3( + vec3(1.164, 1.164, 1.164), + vec3(0.000, -0.392, 2.017), + vec3(1.596, -0.813, 0.000) + ); + + yuv.x = texture2D(tex_y, textureOut).r - 0.063; + yuv.y = texture2D(tex_u, textureOut).r - 0.500; + yuv.z = texture2D(tex_v, textureOut).r - 0.500; + + rgb = yuv2rgb_bt601_mat * yuv; + gl_FragColor = vec4(rgb, 1.0); +} diff --git a/src/qcam/assets/shader/NV_vertex_shader.glsl b/src/qcam/assets/shader/NV_vertex_shader.glsl new file mode 100644 index 0000000..12e791e --- /dev/null +++ b/src/qcam/assets/shader/NV_vertex_shader.glsl @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Linaro + * + * NV_vertex_shader.glsl - Vertex shader code for NV family + */ + +attribute vec4 vertexIn; +attribute vec2 textureIn; +varying vec2 textureOut; + +void main(void) +{ + gl_Position = vertexIn; + textureOut = textureIn; +} diff --git a/src/qcam/assets/shader/shaders.qrc b/src/qcam/assets/shader/shaders.qrc new file mode 100644 index 0000000..33eab27 --- /dev/null +++ b/src/qcam/assets/shader/shaders.qrc @@ -0,0 +1,9 @@ + + + +./NV_vertex_shader.glsl +./NV_2_planes_UV_f.glsl +./NV_2_planes_VU_f.glsl +./NV_3_planes_f.glsl + + diff --git a/src/qcam/meson.build b/src/qcam/meson.build index 6ea886a..e0c6f26 100644 --- a/src/qcam/meson.build +++ b/src/qcam/meson.build @@ -16,6 +16,7 @@ qcam_moc_headers = files([ qcam_resources = files([ 'assets/feathericons/feathericons.qrc', + 'assets/shader/shaders.qrc' ]) qt5 = import('qt5') From patchwork Fri Sep 11 08:55:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Show Liu X-Patchwork-Id: 9569 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9D5BABDB1D for ; Fri, 11 Sep 2020 08:55:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6803962D6E; Fri, 11 Sep 2020 10:55:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="H0lo110b"; dkim-atps=neutral Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4676B62D5B for ; Fri, 11 Sep 2020 10:55:44 +0200 (CEST) Received: by mail-pg1-x543.google.com with SMTP id g29so6194174pgl.2 for ; Fri, 11 Sep 2020 01:55:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BWIcO9CF7ffOFNxyBar68aMjB+PR1UiV1lXSBRWX9k8=; b=H0lo110btfID3OM6i5dD9ZVNRwBf3Ysj1mC00CLeqzdrjEHygEB/oo0OTKBaRfZvlk Bgs3UiA6y9pNI+Ld4QCk9J1W559mwlscKMXpA4L7f2FYuGuBj4HD6khdQP2hYy921sgY CayOxW/M7RAqjVdu96bUO+b3KV7jvuCTeUNDYKLdOE/kmbW9vFC8PyVF6JogNe1VXTTE IlvHer+P4bqhwP/fNwZh6mbjyap8GDTxd6GIu30+QrKNn2t+QKRvWwPMBGtfOpd7IVnM cWR4NK6h2n0mu/ZXAHc0t/DwBx/1XRlxh/d2QCjWe3ybcEvqniwy3C6fm5kO7zKlyIu5 xj0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BWIcO9CF7ffOFNxyBar68aMjB+PR1UiV1lXSBRWX9k8=; b=JctE9mZYS1D/irjLHgU5//DoyMlITJX+4XnLvC1VLe8QEj3C+o9COa/hQDP2ymYlD6 cdEQ1DgF/BK3dg4viDF6BVHbLRYtvUQNb+Ik1huk21DtC7C1F7BqRGPXXCXsNFbwV+/L zH3nG5G4MAHhOLUm0rDPUvT6wy7L4yWagB++pqHpGwbCFth7deS2cK53Knzdw+UcVcKU +Ldl10Lmg7dKiSmgx+FzWfs2eHbF4by5hzsbat3cobRlWOy1PgwIfo2zMLg2f6IP1uk/ /IpvPTWEDIzqEkYnC3KHDFpAvBfRVC2J15cyqABxLRruu0NAlug1LaVyd2CcGGdL/Tky TEzQ== X-Gm-Message-State: AOAM532w2Idb7QjxosrRJNRg4oAGBE1/Eqms/g56/79/2xYQ53ACTpQv lb8VcAuhf5u57SYVolJhalK3jMFm2UUlog== X-Google-Smtp-Source: ABdhPJyEglQOwehYA9/+pA55FnWp1CWeRwvlgCoFghh97tyctFDI5V7pJc9ou4KuEnJXpMxGUNVIKw== X-Received: by 2002:a62:93:0:b029:13e:d13d:a085 with SMTP id 141-20020a6200930000b029013ed13da085mr1260818pfa.28.1599814542653; Fri, 11 Sep 2020 01:55:42 -0700 (PDT) Received: from localhost.localdomain (211-20-20-223.HINET-IP.hinet.net. [211.20.20.223]) by smtp.gmail.com with ESMTPSA id w192sm1616543pfd.156.2020.09.11.01.55.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Sep 2020 01:55:42 -0700 (PDT) From: Show Liu To: libcamera-devel@lists.libcamera.org Date: Fri, 11 Sep 2020 16:55:12 +0800 Message-Id: <20200911085514.30021-3-show.liu@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200911085514.30021-1-show.liu@linaro.org> References: <20200911085514.30021-1-show.liu@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 2/4] qcam: new viewfinder hierarchy X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Create viewfinder base class and rename the original viewfinder as QPainter-based ViewFinder Signed-off-by: Show Liu Reviewed-by: Laurent Pinchart --- src/qcam/meson.build | 4 +- src/qcam/viewfinder.h | 57 ++++------------- .../{viewfinder.cpp => viewfinder_qt.cpp} | 24 +++---- src/qcam/viewfinder_qt.h | 64 +++++++++++++++++++ 4 files changed, 89 insertions(+), 60 deletions(-) rename src/qcam/{viewfinder.cpp => viewfinder_qt.cpp} (86%) create mode 100644 src/qcam/viewfinder_qt.h diff --git a/src/qcam/meson.build b/src/qcam/meson.build index e0c6f26..a4bad0a 100644 --- a/src/qcam/meson.build +++ b/src/qcam/meson.build @@ -6,12 +6,12 @@ qcam_sources = files([ 'format_converter.cpp', 'main.cpp', 'main_window.cpp', - 'viewfinder.cpp', + 'viewfinder_qt.cpp', ]) qcam_moc_headers = files([ 'main_window.h', - 'viewfinder.h', + 'viewfinder_qt.h', ]) qcam_resources = files([ diff --git a/src/qcam/viewfinder.h b/src/qcam/viewfinder.h index 26a1320..67da1df 100644 --- a/src/qcam/viewfinder.h +++ b/src/qcam/viewfinder.h @@ -2,70 +2,35 @@ /* * Copyright (C) 2019, Google Inc. * - * viewfinder.h - qcam - Viewfinder + * viewfinder.h - qcam - Viewfinder base class */ #ifndef __QCAM_VIEWFINDER_H__ #define __QCAM_VIEWFINDER_H__ -#include - -#include -#include #include -#include +#include #include -#include #include -#include - -#include "format_converter.h" - -class QImage; +#include struct MappedBuffer { void *memory; size_t size; }; -class ViewFinder : public QWidget +class ViewFinder { - Q_OBJECT - public: - ViewFinder(QWidget *parent); - ~ViewFinder(); - - const QList &nativeFormats() const; - - int setFormat(const libcamera::PixelFormat &format, const QSize &size); - void render(libcamera::FrameBuffer *buffer, MappedBuffer *map); - void stop(); - - QImage getCurrentImage(); - -Q_SIGNALS: - void renderComplete(libcamera::FrameBuffer *buffer); - -protected: - void paintEvent(QPaintEvent *) override; - QSize sizeHint() const override; - -private: - FormatConverter converter_; + virtual ~ViewFinder() {} - libcamera::PixelFormat format_; - QSize size_; + virtual const QList &nativeFormats() const = 0; - /* Camera stopped icon */ - QSize vfSize_; - QIcon icon_; - QPixmap pixmap_; + virtual int setFormat(const libcamera::PixelFormat &format, const QSize &size) = 0; + virtual void render(libcamera::FrameBuffer *buffer, MappedBuffer *map) = 0; + virtual void stop() = 0; - /* Buffer and render image */ - libcamera::FrameBuffer *buffer_; - QImage image_; - QMutex mutex_; /* Prevent concurrent access to image_ */ + virtual QImage getCurrentImage() = 0; }; -#endif /* __QCAM_VIEWFINDER__ */ +#endif /* __QCAM_VIEWFINDER_H__ */ diff --git a/src/qcam/viewfinder.cpp b/src/qcam/viewfinder_qt.cpp similarity index 86% rename from src/qcam/viewfinder.cpp rename to src/qcam/viewfinder_qt.cpp index dcf8a15..e436714 100644 --- a/src/qcam/viewfinder.cpp +++ b/src/qcam/viewfinder_qt.cpp @@ -2,10 +2,10 @@ /* * Copyright (C) 2019, Google Inc. * - * viewfinder.cpp - qcam - Viewfinder + * viewfinder_qt.cpp - qcam - QPainter-based ViewFinder */ -#include "viewfinder.h" +#include "viewfinder_qt.h" #include #include @@ -33,24 +33,24 @@ static const QMap nativeFormats { libcamera::formats::BGR888, QImage::Format_RGB888 }, }; -ViewFinder::ViewFinder(QWidget *parent) +ViewFinderQt::ViewFinderQt(QWidget *parent) : QWidget(parent), buffer_(nullptr) { icon_ = QIcon(":camera-off.svg"); } -ViewFinder::~ViewFinder() +ViewFinderQt::~ViewFinderQt() { } -const QList &ViewFinder::nativeFormats() const +const QList &ViewFinderQt::nativeFormats() const { static const QList formats = ::nativeFormats.keys(); return formats; } -int ViewFinder::setFormat(const libcamera::PixelFormat &format, - const QSize &size) +int ViewFinderQt::setFormat(const libcamera::PixelFormat &format, + const QSize &size) { image_ = QImage(); @@ -78,7 +78,7 @@ int ViewFinder::setFormat(const libcamera::PixelFormat &format, return 0; } -void ViewFinder::render(libcamera::FrameBuffer *buffer, MappedBuffer *map) +void ViewFinderQt::render(libcamera::FrameBuffer *buffer, MappedBuffer *map) { if (buffer->planes().size() != 1) { qWarning() << "Multi-planar buffers are not supported"; @@ -121,7 +121,7 @@ void ViewFinder::render(libcamera::FrameBuffer *buffer, MappedBuffer *map) renderComplete(buffer); } -void ViewFinder::stop() +void ViewFinderQt::stop() { image_ = QImage(); @@ -133,14 +133,14 @@ void ViewFinder::stop() update(); } -QImage ViewFinder::getCurrentImage() +QImage ViewFinderQt::getCurrentImage() { QMutexLocker locker(&mutex_); return image_.copy(); } -void ViewFinder::paintEvent(QPaintEvent *) +void ViewFinderQt::paintEvent(QPaintEvent *) { QPainter painter(this); @@ -175,7 +175,7 @@ void ViewFinder::paintEvent(QPaintEvent *) painter.drawPixmap(point, pixmap_); } -QSize ViewFinder::sizeHint() const +QSize ViewFinderQt::sizeHint() const { return size_.isValid() ? size_ : QSize(640, 480); } diff --git a/src/qcam/viewfinder_qt.h b/src/qcam/viewfinder_qt.h new file mode 100644 index 0000000..d755428 --- /dev/null +++ b/src/qcam/viewfinder_qt.h @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2019, Google Inc. + * + * viewfinder_qt.h - qcam - QPainter-based ViewFinder + */ +#ifndef __QCAM_VIEWFINDER_QT_H__ +#define __QCAM_VIEWFINDER_QT_H__ + +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "format_converter.h" +#include "viewfinder.h" + +class ViewFinderQt : public QWidget, public ViewFinder +{ + Q_OBJECT + +public: + ViewFinderQt(QWidget *parent); + ~ViewFinderQt(); + + const QList &nativeFormats() const override; + + int setFormat(const libcamera::PixelFormat &format, const QSize &size) override; + void render(libcamera::FrameBuffer *buffer, MappedBuffer *map) override; + void stop() override; + + QImage getCurrentImage() override; + +Q_SIGNALS: + void renderComplete(libcamera::FrameBuffer *buffer); + +protected: + void paintEvent(QPaintEvent *) override; + QSize sizeHint() const override; + +private: + FormatConverter converter_; + + libcamera::PixelFormat format_; + QSize size_; + + /* Camera stopped icon */ + QSize vfSize_; + QIcon icon_; + QPixmap pixmap_; + + /* Buffer and render image */ + libcamera::FrameBuffer *buffer_; + QImage image_; + QMutex mutex_; /* Prevent concurrent access to image_ */ +}; + +#endif /* __QCAM_VIEWFINDER_QT_H__ */ From patchwork Fri Sep 11 08:55:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Show Liu X-Patchwork-Id: 9570 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 11E0FBDB1D for ; Fri, 11 Sep 2020 08:55:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D24D862D53; Fri, 11 Sep 2020 10:55:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="o2mf0K7u"; dkim-atps=neutral Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 132BF62D5B for ; Fri, 11 Sep 2020 10:55:47 +0200 (CEST) Received: by mail-pf1-x429.google.com with SMTP id v196so6934137pfc.1 for ; Fri, 11 Sep 2020 01:55:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5O4q8wP4WZ8rtOxrQxfssO+wmURgF1l82eXdHikeIOY=; b=o2mf0K7uJqSC8NuKBdfG0qA/Ah3/HKAF13Y8AwA3uPDxvhHmYvUsdaJpPIw27KeZKK y0W0A+UfFDY5ebb4Lcq8I/VO7QuhBX3+DOMm5efp4Izd/Gd/UnxlDqchVLKML8b7Pntk 7+pSXYX89zmYm0fM+TtyjaO0NtIbnzKDl4Cr4zCrWYBC8YAEe5bltt2adExONKduhgsL JhZfwUA/sX3ehcFIJ2snmmBujXgvP7nVIS//0BVYAMfLQI4eixqRtKfP6u6Efsc+2Ptq bUM+ArR9J1uDWVnhVceL8kJPEHpxgEgUBz8MR1BOgPJmisXa2unpNApMSP2DL2obB9r0 m37Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5O4q8wP4WZ8rtOxrQxfssO+wmURgF1l82eXdHikeIOY=; b=UG1gQ1qO/VikxjfeRfuVVtBD1Mp5LCKVhSffbT8i8AVpKY3M14xMsUIKLaeo5OwIyS roycoYlsjVPbiUKr/iJ2ckuYf3fZgCX2CkUIpdhVp0bOlcSN9zKknBE2L8UVQbDWQgXT 8RKXxqCaePFMlk53AbBeqjsqlTmmMQy7p5sV2Rb4DW8SaOkjEuIo3xjJlv2y+CtsRRgq 5Jm6WxnaBaDoPuv8mlJJUQGMi60EDtVBqA6qKURH04tQ+Ki+61XbU+gVMpSBK5MDoO4t gPSZ9cyMrHMJmsq5EN92RyT+MGHQuT/Xr+NaKKyoo//cYKIZdW0RJEP055LW78lhNR3+ BJew== X-Gm-Message-State: AOAM532xWOUW5NPTqjiCGbo57NeVfK4zJsZhCcC9LZFKbMxOFNdOlZF+ YJBe1hTz1AuVqFcJ4yg+HYSHHv87sOm4AA== X-Google-Smtp-Source: ABdhPJxTfnisZeBE0NAx+kSM3VwjeB6/mkXzKQIfl37jqZNwaM7X6v/21+iYSjZUsmBqH3aok6gPMQ== X-Received: by 2002:a17:902:a5cb:: with SMTP id t11mr1076741plq.102.1599814545006; Fri, 11 Sep 2020 01:55:45 -0700 (PDT) Received: from localhost.localdomain (211-20-20-223.HINET-IP.hinet.net. [211.20.20.223]) by smtp.gmail.com with ESMTPSA id w192sm1616543pfd.156.2020.09.11.01.55.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Sep 2020 01:55:44 -0700 (PDT) From: Show Liu To: libcamera-devel@lists.libcamera.org Date: Fri, 11 Sep 2020 16:55:13 +0800 Message-Id: <20200911085514.30021-4-show.liu@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200911085514.30021-1-show.liu@linaro.org> References: <20200911085514.30021-1-show.liu@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 3/4] qcam: add viewfinderGL class to accelerate the format convert X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" the viewfinderGL accelerates the format conversion by using OpenGL ES shader Signed-off-by: Show Liu Reviewed-by: Laurent Pinchart --- meson.build | 1 + src/qcam/meson.build | 17 +- src/qcam/viewfinder_gl.cpp | 456 +++++++++++++++++++++++++++++++++++++ src/qcam/viewfinder_gl.h | 96 ++++++++ 4 files changed, 568 insertions(+), 2 deletions(-) create mode 100644 src/qcam/viewfinder_gl.cpp create mode 100644 src/qcam/viewfinder_gl.h diff --git a/meson.build b/meson.build index 1ea35e9..c58d458 100644 --- a/meson.build +++ b/meson.build @@ -26,6 +26,7 @@ libcamera_version = libcamera_git_version.split('+')[0] # Configure the build environment. cc = meson.get_compiler('c') +cxx = meson.get_compiler('cpp') config_h = configuration_data() if cc.has_header_symbol('execinfo.h', 'backtrace') diff --git a/src/qcam/meson.build b/src/qcam/meson.build index a4bad0a..9bb48c0 100644 --- a/src/qcam/meson.build +++ b/src/qcam/meson.build @@ -16,14 +16,14 @@ qcam_moc_headers = files([ qcam_resources = files([ 'assets/feathericons/feathericons.qrc', - 'assets/shader/shaders.qrc' ]) qt5 = import('qt5') qt5_dep = dependency('qt5', method : 'pkg-config', modules : ['Core', 'Gui', 'Widgets'], - required : get_option('qcam')) + required : get_option('qcam'), + version : '>=5.4') if qt5_dep.found() qcam_deps = [ @@ -42,6 +42,19 @@ if qt5_dep.found() ]) endif + if cxx.has_header_symbol('QOpenGLWidget', 'QOpenGLWidget', + dependencies : qt5_dep, args : '-fPIC') + qcam_sources += files([ + 'viewfinder_gl.cpp', + ]) + qcam_moc_headers += files([ + 'viewfinder_gl.h', + ]) + qcam_resources += files([ + 'assets/shader/shaders.qrc' + ]) + endif + # gcc 9 introduced a deprecated-copy warning that is triggered by Qt until # Qt 5.13. clang 10 introduced the same warning, but detects more issues # that are not fixed in Qt yet. Disable the warning manually in both cases. diff --git a/src/qcam/viewfinder_gl.cpp b/src/qcam/viewfinder_gl.cpp new file mode 100644 index 0000000..84f4866 --- /dev/null +++ b/src/qcam/viewfinder_gl.cpp @@ -0,0 +1,456 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Linaro + * + * viewfinderGL.cpp - OpenGL Viewfinder for rendering by OpenGL shader + */ + +#include "viewfinder_gl.h" + +#include + +#include + +static const QList supportedFormats{ + libcamera::formats::NV12, + libcamera::formats::NV21, + libcamera::formats::NV16, + libcamera::formats::NV61, + libcamera::formats::NV24, + libcamera::formats::NV42, + libcamera::formats::YUV420, + libcamera::formats::YVU420 +}; + +ViewFinderGL::ViewFinderGL(QWidget *parent) + : QOpenGLWidget(parent), buffer_(nullptr), yuvData_(nullptr), + fragmentShader_(nullptr), vertexShader_(nullptr), + vertexBuffer_(QOpenGLBuffer::VertexBuffer), + textureU_(QOpenGLTexture::Target2D), + textureV_(QOpenGLTexture::Target2D), + textureY_(QOpenGLTexture::Target2D) +{ +} + +ViewFinderGL::~ViewFinderGL() +{ + removeShader(); + + if (vertexBuffer_.isCreated()) + vertexBuffer_.destroy(); +} + +const QList &ViewFinderGL::nativeFormats() const +{ + return supportedFormats; +} + +int ViewFinderGL::setFormat(const libcamera::PixelFormat &format, + const QSize &size) +{ + int ret = 0; + + /* If the fragment is ceeated remove it and create a new one */ + if (fragmentShader_) { + if (shaderProgram_.isLinked()) { + shaderProgram_.release(); + shaderProgram_.removeShader(fragmentShader_); + delete fragmentShader_; + } + } + + if (selectFormat(format)) { + format_ = format; + size_ = size; + } else { + ret = -1; + } + updateGeometry(); + return ret; +} + +void ViewFinderGL::stop() +{ + if (buffer_) { + renderComplete(buffer_); + buffer_ = nullptr; + } +} + +QImage ViewFinderGL::getCurrentImage() +{ + QMutexLocker locker(&mutex_); + + return grabFramebuffer(); +} + +void ViewFinderGL::render(libcamera::FrameBuffer *buffer, MappedBuffer *map) +{ + if (buffer->planes().size() != 1) { + qWarning() << "Multi-planar buffers are not supported"; + return; + } + + if (buffer_) + renderComplete(buffer_); + + yuvData_ = static_cast(map->memory); + update(); + buffer_ = buffer; +} + +bool ViewFinderGL::selectFormat(const libcamera::PixelFormat &format) +{ + bool ret = true; + switch (format) { + case libcamera::formats::NV12: + horzSubSample_ = 2; + vertSubSample_ = 2; + vertexShaderSrc_ = ":NV_vertex_shader.glsl"; + fragmentShaderSrc_ = ":NV_2_planes_UV_f.glsl"; + break; + case libcamera::formats::NV21: + horzSubSample_ = 2; + vertSubSample_ = 2; + vertexShaderSrc_ = ":NV_vertex_shader.glsl"; + fragmentShaderSrc_ = ":NV_2_planes_VU_f.glsl"; + break; + case libcamera::formats::NV16: + horzSubSample_ = 2; + vertSubSample_ = 1; + vertexShaderSrc_ = ":NV_vertex_shader.glsl"; + fragmentShaderSrc_ = ":NV_2_planes_UV_f.glsl"; + break; + case libcamera::formats::NV61: + horzSubSample_ = 2; + vertSubSample_ = 1; + vertexShaderSrc_ = ":NV_vertex_shader.glsl"; + fragmentShaderSrc_ = ":NV_2_planes_VU_f.glsl"; + break; + case libcamera::formats::NV24: + horzSubSample_ = 1; + vertSubSample_ = 1; + vertexShaderSrc_ = ":NV_vertex_shader.glsl"; + fragmentShaderSrc_ = ":NV_2_planes_UV_f.glsl"; + break; + case libcamera::formats::NV42: + horzSubSample_ = 1; + vertSubSample_ = 1; + vertexShaderSrc_ = ":NV_vertex_shader.glsl"; + fragmentShaderSrc_ = ":NV_2_planes_VU_f.glsl"; + break; + case libcamera::formats::YUV420: + horzSubSample_ = 2; + vertSubSample_ = 2; + vertexShaderSrc_ = ":NV_vertex_shader.glsl"; + fragmentShaderSrc_ = ":NV_3_planes_f.glsl"; + break; + case libcamera::formats::YVU420: + horzSubSample_ = 2; + vertSubSample_ = 2; + vertexShaderSrc_ = ":NV_vertex_shader.glsl"; + fragmentShaderSrc_ = ":NV_3_planes_f.glsl"; + break; + default: + ret = false; + qWarning() << "[ViewFinderGL]:" + << "format not supported."; + break; + }; + + return ret; +} + +bool ViewFinderGL::createVertexShader() +{ + /* Create Vertex Shader */ + vertexShader_ = new QOpenGLShader(QOpenGLShader::Vertex, this); + + /* Compile the vertex shader */ + if (!vertexShader_->compileSourceFile(vertexShaderSrc_)) { + qWarning() << "[ViewFinderGL]:" << vertexShader_->log(); + return false; + } + + shaderProgram_.addShader(vertexShader_); + return true; +} + +bool ViewFinderGL::createFragmentShader() +{ + int attributeVertex; + int attributeTexture; + + /* Create Fragment Shader */ + fragmentShader_ = new QOpenGLShader(QOpenGLShader::Fragment, this); + + /* Compile the fragment shader */ + if (!fragmentShader_->compileSourceFile(fragmentShaderSrc_)) { + qWarning() << "[ViewFinderGL]:" << fragmentShader_->log(); + return false; + } + + shaderProgram_.addShader(fragmentShader_); + + /* Link shader pipeline */ + if (!shaderProgram_.link()) { + qWarning() << "[ViewFinderGL]:" << shaderProgram_.log(); + close(); + } + + /* Bind shader pipeline for use */ + if (!shaderProgram_.bind()) { + qWarning() << "[ViewFinderGL]:" << shaderProgram_.log(); + close(); + } + + attributeVertex = shaderProgram_.attributeLocation("vertexIn"); + attributeTexture = shaderProgram_.attributeLocation("textureIn"); + + shaderProgram_.enableAttributeArray(attributeVertex); + shaderProgram_.setAttributeBuffer(attributeVertex, + GL_FLOAT, + 0, + 2, + 2 * sizeof(GLfloat)); + + shaderProgram_.enableAttributeArray(attributeTexture); + shaderProgram_.setAttributeBuffer(attributeTexture, + GL_FLOAT, + 8 * sizeof(GLfloat), + 2, + 2 * sizeof(GLfloat)); + + textureUniformY_ = shaderProgram_.uniformLocation("tex_y"); + textureUniformU_ = shaderProgram_.uniformLocation("tex_u"); + textureUniformV_ = shaderProgram_.uniformLocation("tex_v"); + + if (!textureY_.isCreated()) + textureY_.create(); + + if (!textureU_.isCreated()) + textureU_.create(); + + if (!textureV_.isCreated()) + textureV_.create(); + + id_y_ = textureY_.textureId(); + id_u_ = textureU_.textureId(); + id_v_ = textureV_.textureId(); + return true; +} + +void ViewFinderGL::configureTexture(unsigned int id) +{ + glBindTexture(GL_TEXTURE_2D, id); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); +} + +void ViewFinderGL::removeShader() +{ + if (shaderProgram_.isLinked()) { + shaderProgram_.release(); + shaderProgram_.removeAllShaders(); + } + + if (fragmentShader_) + delete fragmentShader_; + + if (vertexShader_) + delete vertexShader_; +} + +void ViewFinderGL::initializeGL() +{ + initializeOpenGLFunctions(); + glEnable(GL_TEXTURE_2D); + glDisable(GL_DEPTH_TEST); + + static const GLfloat coordinates[2][4][2]{ + { + /* Vertex coordinates */ + { -1.0f, -1.0f }, + { -1.0f, +1.0f }, + { +1.0f, +1.0f }, + { +1.0f, -1.0f }, + }, + { + /* Texture coordinates */ + { 1.0f, 0.0f }, + { 1.0f, 1.0f }, + { 0.0f, 1.0f }, + { 0.0f, 0.0f }, + }, + }; + + vertexBuffer_.create(); + vertexBuffer_.bind(); + vertexBuffer_.allocate(coordinates, sizeof(coordinates)); + + /* Create Vertex Shader */ + if (!createVertexShader()) + qWarning() << "[ViewFinderGL]: create vertex shader failed."; + + glClearColor(1.0f, 1.0f, 1.0f, 0.0f); +} + +void ViewFinderGL::doRender() +{ + switch (format_) { + case libcamera::formats::NV12: + case libcamera::formats::NV21: + case libcamera::formats::NV16: + case libcamera::formats::NV61: + case libcamera::formats::NV24: + case libcamera::formats::NV42: + /* activate texture Y */ + glActiveTexture(GL_TEXTURE0); + configureTexture(id_y_); + glTexImage2D(GL_TEXTURE_2D, + 0, + GL_RED, + size_.width(), + size_.height(), + 0, + GL_RED, + GL_UNSIGNED_BYTE, + yuvData_); + shaderProgram_.setUniformValue(textureUniformY_, 0); + + /* activate texture UV/VU */ + glActiveTexture(GL_TEXTURE1); + configureTexture(id_u_); + glTexImage2D(GL_TEXTURE_2D, + 0, + GL_RG, + size_.width() / horzSubSample_, + size_.height() / vertSubSample_, + 0, + GL_RG, + GL_UNSIGNED_BYTE, + (char *)yuvData_ + size_.width() * size_.height()); + shaderProgram_.setUniformValue(textureUniformU_, 1); + break; + + case libcamera::formats::YUV420: + /* activate texture Y */ + glActiveTexture(GL_TEXTURE0); + configureTexture(id_y_); + glTexImage2D(GL_TEXTURE_2D, + 0, + GL_RED, + size_.width(), + size_.height(), + 0, + GL_RED, + GL_UNSIGNED_BYTE, + yuvData_); + shaderProgram_.setUniformValue(textureUniformY_, 0); + + /* activate texture U */ + glActiveTexture(GL_TEXTURE1); + configureTexture(id_u_); + glTexImage2D(GL_TEXTURE_2D, + 0, + GL_RED, + size_.width() / horzSubSample_, + size_.height() / vertSubSample_, + 0, + GL_RED, + GL_UNSIGNED_BYTE, + (char *)yuvData_ + size_.width() * size_.height()); + shaderProgram_.setUniformValue(textureUniformU_, 1); + + /* activate texture V */ + glActiveTexture(GL_TEXTURE2); + configureTexture(id_v_); + glTexImage2D(GL_TEXTURE_2D, + 0, + GL_RED, + size_.width() / horzSubSample_, + size_.height() / vertSubSample_, + 0, + GL_RED, + GL_UNSIGNED_BYTE, + (char *)yuvData_ + size_.width() * size_.height() * 5 / 4); + shaderProgram_.setUniformValue(textureUniformV_, 2); + break; + + case libcamera::formats::YVU420: + /* activate texture Y */ + glActiveTexture(GL_TEXTURE0); + configureTexture(id_y_); + glTexImage2D(GL_TEXTURE_2D, + 0, + GL_RED, + size_.width(), + size_.height(), + 0, + GL_RED, + GL_UNSIGNED_BYTE, + yuvData_); + shaderProgram_.setUniformValue(textureUniformY_, 0); + + /* activate texture V */ + glActiveTexture(GL_TEXTURE2); + configureTexture(id_v_); + glTexImage2D(GL_TEXTURE_2D, + 0, + GL_RED, + size_.width() / horzSubSample_, + size_.height() / vertSubSample_, + 0, + GL_RED, + GL_UNSIGNED_BYTE, + (char *)yuvData_ + size_.width() * size_.height()); + shaderProgram_.setUniformValue(textureUniformV_, 1); + + /* activate texture U */ + glActiveTexture(GL_TEXTURE1); + configureTexture(id_u_); + glTexImage2D(GL_TEXTURE_2D, + 0, + GL_RED, + size_.width() / horzSubSample_, + size_.height() / vertSubSample_, + 0, + GL_RED, + GL_UNSIGNED_BYTE, + (char *)yuvData_ + size_.width() * size_.height() * 5 / 4); + shaderProgram_.setUniformValue(textureUniformU_, 2); + break; + + default: + break; + }; +} + +void ViewFinderGL::paintGL() +{ + if (!fragmentShader_) + if (!createFragmentShader()) { + qWarning() << "[ViewFinderGL]:" + << "create fragment shader failed."; + } + + if (yuvData_) { + glClearColor(0.0, 0.0, 0.0, 1.0); + glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); + + doRender(); + glDrawArrays(GL_TRIANGLE_FAN, 0, 4); + } +} + +void ViewFinderGL::resizeGL(int w, int h) +{ + glViewport(0, 0, w, h); +} + +QSize ViewFinderGL::sizeHint() const +{ + return size_.isValid() ? size_ : QSize(640, 480); +} diff --git a/src/qcam/viewfinder_gl.h b/src/qcam/viewfinder_gl.h new file mode 100644 index 0000000..69502b7 --- /dev/null +++ b/src/qcam/viewfinder_gl.h @@ -0,0 +1,96 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Linaro + * + * viewfinder_GL.h - OpenGL Viewfinder for rendering by OpenGL shader + * + */ +#ifndef __VIEWFINDER_GL_H__ +#define __VIEWFINDER_GL_H__ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "viewfinder.h" + +class ViewFinderGL : public QOpenGLWidget, + public ViewFinder, + protected QOpenGLFunctions +{ + Q_OBJECT + +public: + ViewFinderGL(QWidget *parent = nullptr); + ~ViewFinderGL(); + + const QList &nativeFormats() const override; + + int setFormat(const libcamera::PixelFormat &format, const QSize &size) override; + void render(libcamera::FrameBuffer *buffer, MappedBuffer *map) override; + void stop() override; + + QImage getCurrentImage() override; + +Q_SIGNALS: + void renderComplete(libcamera::FrameBuffer *buffer); + +protected: + void initializeGL() override; + void paintGL() override; + void resizeGL(int w, int h) override; + QSize sizeHint() const override; + +private: + bool selectFormat(const libcamera::PixelFormat &format); + + void configureTexture(unsigned int id); + bool createFragmentShader(); + bool createVertexShader(); + void removeShader(); + void doRender(); + + /* Captured image size, format and buffer */ + libcamera::FrameBuffer *buffer_; + libcamera::PixelFormat format_; + QSize size_; + unsigned char *yuvData_; + + /* OpenGL components for rendering */ + QOpenGLShader *fragmentShader_; + QOpenGLShader *vertexShader_; + QOpenGLShaderProgram shaderProgram_; + + /* Vertex buffer */ + QOpenGLBuffer vertexBuffer_; + + /* Fragment and Vertex shader file name */ + QString fragmentShaderSrc_; + QString vertexShaderSrc_; + + /* YUV texture planars and parameters */ + GLuint id_u_; + GLuint id_v_; + GLuint id_y_; + GLuint textureUniformU_; + GLuint textureUniformV_; + GLuint textureUniformY_; + QOpenGLTexture textureU_; + QOpenGLTexture textureV_; + QOpenGLTexture textureY_; + unsigned int horzSubSample_; + unsigned int vertSubSample_; + + QMutex mutex_; /* Prevent concurrent access to image_ */ +}; + +#endif /* __VIEWFINDER_GL_H__ */ From patchwork Fri Sep 11 08:55:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Show Liu X-Patchwork-Id: 9571 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6B9B8BDB1D for ; Fri, 11 Sep 2020 08:55:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 387DC62D81; Fri, 11 Sep 2020 10:55:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="OPKwd4VK"; dkim-atps=neutral Received: from mail-pg1-x535.google.com (mail-pg1-x535.google.com [IPv6:2607:f8b0:4864:20::535]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E2FF662D38 for ; Fri, 11 Sep 2020 10:55:48 +0200 (CEST) Received: by mail-pg1-x535.google.com with SMTP id f2so4149463pgd.3 for ; Fri, 11 Sep 2020 01:55:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pplAyIWzx/jJzkQ8ErheBqp/JOxZ8UPVrHihID0UcyI=; b=OPKwd4VKgIIIKRsjxFjXBz/Cx8xMaKCS7IMlPL6NmoZ+bvnl8Vt4g4oUxoPSwWP70k r3Bs9uNGFPxvwHwPxBLpd+BipXuQvOpq4/ydzLBNNDYoxrP7oVxNS1oyc/GMtpkgME6u jch0RJZ3VxGFVqcVvIZfciVmgqETiHv8AofTfxPrqU+WjJ2vRDtn99SpIwgo9kNKIIcC F11GjezY/rLuUmm+AoZgUKzhrUBa+DVL1WVQY1YmPngkXaBp+0tX3e0YueP9X+z08m3L zYgX5BI4uiXrwUNWhnhqmwCq09Dc7Xlv9r6j8xWTWipXH66nTp3k3f2divjoxW8cXREf Oo2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pplAyIWzx/jJzkQ8ErheBqp/JOxZ8UPVrHihID0UcyI=; b=d9ZVe/i7SUagwabulx8HjXj049yVPHDyfC/2PgrRWDVcxqb4VWrU/1AC7WFZ07+1+Y hcAmaCglDzG7vdjd/0QXWO3x2hI66vashIl4hI6qcFwuLMNT7+2JsaqbfP/MNwTv9wjW RApgh7DF3sEI2ZgK3KkoLYvfdw8PA+R58kK6ItMvSVDMIKx7Yy99PhRwaSYeTW36FaNY s7LxiKkUN2ey0lt8QB6cSlu5+uGMjuKqKVAdlTztru6xny9dxnzPugeplzNENg4xAnaj VHD5isQL8EqCddWCuIL2xKRQ3py5Hs/XtfwBX+7iZHzddpPTKsb2ly98EfAG56FU3m9k +6fA== X-Gm-Message-State: AOAM5303BNrBxZuCGMLjbE0h9NnjPkNbMW780s1Y+rUE8yThFpMl4O3x YdHBpKQSR+O6ztRW6C9St7VY45XBSETsrg== X-Google-Smtp-Source: ABdhPJxNpzy/hFw2SihqXYSN4Jiv3Fnl7fcsHMR1c4DvczcKMArA0UbcyT5HIDoKw9tA8ytJEJgxtA== X-Received: by 2002:a63:1356:: with SMTP id 22mr964104pgt.108.1599814547320; Fri, 11 Sep 2020 01:55:47 -0700 (PDT) Received: from localhost.localdomain (211-20-20-223.HINET-IP.hinet.net. [211.20.20.223]) by smtp.gmail.com with ESMTPSA id w192sm1616543pfd.156.2020.09.11.01.55.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Sep 2020 01:55:47 -0700 (PDT) From: Show Liu To: libcamera-devel@lists.libcamera.org Date: Fri, 11 Sep 2020 16:55:14 +0800 Message-Id: <20200911085514.30021-5-show.liu@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200911085514.30021-1-show.liu@linaro.org> References: <20200911085514.30021-1-show.liu@linaro.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 4/4] qcam: add additional command line option to select the render type X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add new option "--render=qt|gles" to select the render type, "--render=gles" to accelerate format conversion and rendering "--render=qt" is the original Qt rendering Signed-off-by: Show Liu Reviewed-by: Laurent Pinchart --- src/qcam/main.cpp | 3 +++ src/qcam/main_window.cpp | 31 +++++++++++++++++++++++++++---- src/qcam/main_window.h | 3 +++ 3 files changed, 33 insertions(+), 4 deletions(-) diff --git a/src/qcam/main.cpp b/src/qcam/main.cpp index bae358d..4b7d041 100644 --- a/src/qcam/main.cpp +++ b/src/qcam/main.cpp @@ -33,6 +33,9 @@ OptionsParser::Options parseOptions(int argc, char *argv[]) ArgumentRequired, "camera"); parser.addOption(OptHelp, OptionNone, "Display this help message", "help"); + parser.addOption(OptRendered, OptionString, + "Choose the renderer type {qt,gles} (default: qt)", "renderer", + ArgumentRequired, "render"); parser.addOption(OptStream, &streamKeyValue, "Set configuration of a camera stream", "stream", true); diff --git a/src/qcam/main_window.cpp b/src/qcam/main_window.cpp index 612d978..315102c 100644 --- a/src/qcam/main_window.cpp +++ b/src/qcam/main_window.cpp @@ -95,6 +95,9 @@ MainWindow::MainWindow(CameraManager *cm, const OptionsParser::Options &options) { int ret; + /* Render Type Qt or GLES, and set Qt by default */ + std::string renderType_ = "qt"; + /* * Initialize the UI: Create the toolbar, set the window title and * create the viewfinder widget. @@ -105,10 +108,30 @@ MainWindow::MainWindow(CameraManager *cm, const OptionsParser::Options &options) setWindowTitle(title_); connect(&titleTimer_, SIGNAL(timeout()), this, SLOT(updateTitle())); - viewfinder_ = new ViewFinder(this); - connect(viewfinder_, &ViewFinder::renderComplete, - this, &MainWindow::queueRequest); - setCentralWidget(viewfinder_); + if (options_.isSet(OptRendered)) + renderType_ = options_[OptRendered].toString(); + + if (renderType_ == "qt") { + ViewFinderQt *viewfinder = new ViewFinderQt(this); + connect(viewfinder, &ViewFinderQt::renderComplete, + this, &MainWindow::queueRequest); + viewfinder_ = viewfinder; + setCentralWidget(viewfinder); +#ifndef QT_NO_OPENGL + } else if (renderType_ == "gles") { + ViewFinderGL *viewfinder = new ViewFinderGL(this); + connect(viewfinder, &ViewFinderGL::renderComplete, + this, &MainWindow::queueRequest); + viewfinder_ = viewfinder; + setCentralWidget(viewfinder); +#endif + } else { + qWarning() << "Invalid render type" + << QString::fromStdString(renderType_); + quit(); + return; + } + adjustSize(); /* Hotplug/unplug support */ diff --git a/src/qcam/main_window.h b/src/qcam/main_window.h index 3d21779..251f78b 100644 --- a/src/qcam/main_window.h +++ b/src/qcam/main_window.h @@ -26,6 +26,8 @@ #include "../cam/stream_options.h" #include "viewfinder.h" +#include "viewfinder_gl.h" +#include "viewfinder_qt.h" using namespace libcamera; @@ -37,6 +39,7 @@ class HotplugEvent; enum { OptCamera = 'c', OptHelp = 'h', + OptRendered = 'r', OptStream = 's', };