From patchwork Fri Aug 21 16:15:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Show Liu X-Patchwork-Id: 9354 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 51340BD87C for ; Fri, 21 Aug 2020 16:16:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DE82A61B4F; Fri, 21 Aug 2020 18:16:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="GNMN6dXA"; dkim-atps=neutral Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B3D7E60383 for ; Fri, 21 Aug 2020 18:16:23 +0200 (CEST) Received: by mail-pg1-x541.google.com with SMTP id p37so1226770pgl.3 for ; Fri, 21 Aug 2020 09:16:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=a3ihhj+rdi8Xq1qvKPZrdYxJiOIQgqdgyotDTl6IK/8=; b=GNMN6dXABOtz/4NBP75x4ir9FTP/TjVyzXzs+aIST2SiMSmqej1LfqHjgMCpqfKiWN YAkeJD4FHYT/sM3xqMEf/SfibqAmQU6UKZCJ61NHSwba5GZz4VRowfA1adU8qZz5s4VS AVgjigNbvyOv+222PwqQDFM+2aDC0eoTid+UEh+FHN/maG5vIOpDqlcSxG/8RcEkep1X hkwEgkhjFD5KepuuvXVFZzuNAufGwG/vv4nko8sKO/e8XC6gYqTRKuuq6E3gV/0yLZyZ Tvs/oF7yZEcli1uiomK4PLYh/X/471iPXUzf8aDFsbtSNkhVcJzMw7Qnq7uMJG6/fmIb LEEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=a3ihhj+rdi8Xq1qvKPZrdYxJiOIQgqdgyotDTl6IK/8=; b=fpRDkLUMEuys/MMniq3F5RUtOrp3rAj2dNJyGY6Tv0hOYblv7+jRPC5doHghdlZe27 bI5tsGF3ZjZ6BePv66QID4zvMqbB8/UyKjj8YcFCF4624Ewm/pBOnhoiXIT32htLRnyO 4Xyi9Fh5Zv/+GarEyPgd8gdCHrv+m0I+QMW6DYFmG34hl6sNwAcf2ia9VYhG5qlo5N0q FPfu+kXZDdWa46splXZrgkCA/iEHfnH9j91h1cAJwhexFsM3FjgulpETlIpJFK+yzmFy RNzV4R2btHomwB5uUbztmWqQyay6XO/KASOQpO4a7dEUVe2zw6h5bbJye4gcgUSnBWrM kY1w== X-Gm-Message-State: AOAM532QTfYtHwvFHxi7ucfWJWuEsPDLFKq9XlwpyCPYvon/QMDuLNYL QrjPCYBT/ojdEq258mtredkTcNWaWuuGDw== X-Google-Smtp-Source: ABdhPJxw0aFRyEHIjcXErk0WJGK+Vn70CHjMRoRfe2tfhSuGLrIJIQrcNRVW8FB1DUoctHrLIZd6ww== X-Received: by 2002:a63:5b05:: with SMTP id p5mr2772635pgb.143.1598026581790; Fri, 21 Aug 2020 09:16:21 -0700 (PDT) Received: from localhost.localdomain (2001-b011-200c-3405-5522-d8a8-1333-f09d.dynamic-ip6.hinet.net. [2001:b011:200c:3405:5522:d8a8:1333:f09d]) by smtp.gmail.com with ESMTPSA id h15sm2524184pjf.54.2020.08.21.09.16.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Aug 2020 09:16:21 -0700 (PDT) From: Show Liu To: libcamera-devel@lists.libcamera.org Date: Sat, 22 Aug 2020 00:15:59 +0800 Message-Id: <20200821161602.5093-1-show.liu@linaro.org> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 0/3] qcam: NV family YUV format conversion by using OpenGL shader X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hi, This is the version 4 patchset for qcam to accelerate the YUV format conversion by OpenGL shader. Most of the modifications are according to the comments and suggestions from V3. And the big change is using the QOpenGLFramebufferObject with QOffscreenSurface instead of the QOpenGLWidget, the main reason that use dynamic_cast to get the viewfinder_ is really weird. In this version, OpenGL rendering enabled by --render=gles, and if any error occurs during OpenGL configuration, it will return to use QT rendering. But compared with the previous version, this version performance dropped a lot due to I using the toImage() function to get the image when rendering complete, it's really an inefficient operation. the framerate down to 16.x ~ 18.x fps(1280 x 800). I am still trying to improve this performance issue. Any suggestions are welcome. BR, Show Liu Show Liu (3): qcam: add OpenGL shader code as QT resource qcam: add OpenGL renderer qcam: use the OpenGL renderer as NV family YUV format converter src/qcam/assets/shader/NV_2_planes_UV_f.glsl | 32 ++ src/qcam/assets/shader/NV_2_planes_VU_f.glsl | 32 ++ src/qcam/assets/shader/NV_3_planes_UV_f.glsl | 33 ++ src/qcam/assets/shader/NV_3_planes_VU_f.glsl | 33 ++ src/qcam/assets/shader/NV_vertex_shader.glsl | 16 + src/qcam/assets/shader/shaders.qrc | 10 + src/qcam/main.cpp | 3 + src/qcam/main_window.cpp | 2 + src/qcam/main_window.h | 1 + src/qcam/meson.build | 3 + src/qcam/renderer.cpp | 346 +++++++++++++++++++ src/qcam/renderer.h | 81 +++++ src/qcam/viewfinder.cpp | 46 ++- src/qcam/viewfinder.h | 10 + 14 files changed, 640 insertions(+), 8 deletions(-) create mode 100644 src/qcam/assets/shader/NV_2_planes_UV_f.glsl create mode 100644 src/qcam/assets/shader/NV_2_planes_VU_f.glsl create mode 100644 src/qcam/assets/shader/NV_3_planes_UV_f.glsl create mode 100644 src/qcam/assets/shader/NV_3_planes_VU_f.glsl create mode 100644 src/qcam/assets/shader/NV_vertex_shader.glsl create mode 100644 src/qcam/assets/shader/shaders.qrc create mode 100644 src/qcam/renderer.cpp create mode 100644 src/qcam/renderer.h