From patchwork Tue Sep 10 23:46:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 21225 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2629FBF415 for ; Tue, 10 Sep 2024 23:47:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B04EF63502; Wed, 11 Sep 2024 01:47:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="IbNTiej5"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A144634F5 for ; Wed, 11 Sep 2024 01:47:24 +0200 (CEST) Received: from pendragon.ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 325B3BEB; Wed, 11 Sep 2024 01:46:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726011967; bh=KGfAVXmSPWZE+yattAsCkhFdb5Uewtihi2avsou+KGw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IbNTiej5kez5BfcmivdpS7xMOQnhLeI8crmjKNhQf8//URl9iuXsRIZtd4ZBPjOjN ya/Vtwosbj98Ld096HDYaHMyoBeKgulCAaamTLpSCvgZu6GAJOPXkfLUJw44u9kJWI KQEHdYTXd2t0ToscxbNWFer4USH4QK273AM8vF2I= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: Neal Gompa Subject: [PATCH 1/3] qcam: viewfinder_gl: Fix binding of vertex buffer and shader program Date: Wed, 11 Sep 2024 02:46:47 +0300 Message-ID: <20240910234649.28591-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.44.2 In-Reply-To: <20240910234649.28591-1-laurent.pinchart@ideasonboard.com> References: <20240910234649.28591-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Starting in Qt 6.7.0, vertex buffers and shader programs are unbound just before calling QOpenGLWidget::paintGL(). This breaks rendering in the GL viewfinder in two ways. First, we bind the vertex buffer only once at initialization time. There is therefore no vertex buffer mapped at rendering time, preventing both the vertex shader from having access to the vertex and texture coordinates. Then, we bind the shader program only when rendering the first frame. There is thus no shader program bound for all subsequent frames, breaking rendering. Fix this by binding the vertex buffer where needed, when setting attribute buffers for the shader program, and binding the shader program for every frame. As we use a single vertex buffer, we could bind it at the beginning of paintGL() and keep it bound indefinitely. That would however fail to clearly indicate in the source code where the vertex buffer is needed, making the code more difficult to understand as it would rely on implicit assumptions. Release the vertex buffer explicitly when we don't need it anymore to avoid this. While at it, fix a coding style violation by adding missing curly brackets. Signed-off-by: Laurent Pinchart --- src/apps/qcam/viewfinder_gl.cpp | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/src/apps/qcam/viewfinder_gl.cpp b/src/apps/qcam/viewfinder_gl.cpp index 9d2a69600db1..5be7bec539bf 100644 --- a/src/apps/qcam/viewfinder_gl.cpp +++ b/src/apps/qcam/viewfinder_gl.cpp @@ -443,15 +443,11 @@ bool ViewFinderGL::createFragmentShader() close(); } - /* Bind shader pipeline for use */ - if (!shaderProgram_.bind()) { - qWarning() << "[ViewFinderGL]:" << shaderProgram_.log(); - close(); - } - attributeVertex = shaderProgram_.attributeLocation("vertexIn"); attributeTexture = shaderProgram_.attributeLocation("textureIn"); + vertexBuffer_.bind(); + shaderProgram_.enableAttributeArray(attributeVertex); shaderProgram_.setAttributeBuffer(attributeVertex, GL_FLOAT, @@ -466,6 +462,8 @@ bool ViewFinderGL::createFragmentShader() 2, 2 * sizeof(GLfloat)); + vertexBuffer_.release(); + textureUniformY_ = shaderProgram_.uniformLocation("tex_y"); textureUniformU_ = shaderProgram_.uniformLocation("tex_u"); textureUniformV_ = shaderProgram_.uniformLocation("tex_v"); @@ -809,11 +807,18 @@ void ViewFinderGL::doRender() void ViewFinderGL::paintGL() { - if (!fragmentShader_) + if (!fragmentShader_) { if (!createFragmentShader()) { qWarning() << "[ViewFinderGL]:" << "create fragment shader failed."; } + } + + /* Bind shader pipeline for use. */ + if (!shaderProgram_.bind()) { + qWarning() << "[ViewFinderGL]:" << shaderProgram_.log(); + close(); + } if (image_) { glClearColor(0.0, 0.0, 0.0, 1.0); From patchwork Tue Sep 10 23:46:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 21226 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4B6F4BF415 for ; Tue, 10 Sep 2024 23:47:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 002C6634FD; Wed, 11 Sep 2024 01:47:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nVy5jsmA"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9917F634FC for ; Wed, 11 Sep 2024 01:47:25 +0200 (CEST) Received: from pendragon.ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3BA92108D; Wed, 11 Sep 2024 01:46:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726011968; bh=tBB/fm/QHR7uGKEdy9aFeQXVkHB9SG45Rlj9Mz3TBII=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nVy5jsmA/iT8Jv9kag9i3HuuusPhCulpgqrYx/V8DlASw4QN5DFrKHPno4osvFE7I tnbkFSQZI4pY7pJRTRXXHPvJsCdPv/GfnWAuW/0eHUgtZpo5qwEsvZXxELTErbVwXz CJlZROsqlm8w8YvUO8COIeCFa0iXle4Jqyxev5Z4= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: Neal Gompa Subject: [PATCH 2/3] qcam: viewfinder_gl: Drop duplicate glClearColor() Date: Wed, 11 Sep 2024 02:46:48 +0300 Message-ID: <20240910234649.28591-3-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.44.2 In-Reply-To: <20240910234649.28591-1-laurent.pinchart@ideasonboard.com> References: <20240910234649.28591-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" There's no need to call glClearColor() twice before drawing any GL content. Drop the first call. This doesn't introduce any functional change. While at it, pass floats instead of doubles to glClearColor(), as required by the function. Signed-off-by: Laurent Pinchart --- src/apps/qcam/viewfinder_gl.cpp | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/src/apps/qcam/viewfinder_gl.cpp b/src/apps/qcam/viewfinder_gl.cpp index 5be7bec539bf..b2096faf859f 100644 --- a/src/apps/qcam/viewfinder_gl.cpp +++ b/src/apps/qcam/viewfinder_gl.cpp @@ -533,8 +533,6 @@ void ViewFinderGL::initializeGL() /* Create Vertex Shader */ if (!createVertexShader()) qWarning() << "[ViewFinderGL]: create vertex shader failed."; - - glClearColor(1.0f, 1.0f, 1.0f, 0.0f); } void ViewFinderGL::doRender() @@ -821,7 +819,7 @@ void ViewFinderGL::paintGL() } if (image_) { - glClearColor(0.0, 0.0, 0.0, 1.0); + glClearColor(0.0f, 0.0f, 0.0f, 1.0f); glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); doRender(); From patchwork Tue Sep 10 23:46:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 21227 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 75272C3259 for ; Tue, 10 Sep 2024 23:47:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CF0D863501; Wed, 11 Sep 2024 01:47:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FrmEDzEd"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 547F6634FD for ; Wed, 11 Sep 2024 01:47:26 +0200 (CEST) Received: from pendragon.ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 36EDB11D6; Wed, 11 Sep 2024 01:46:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726011969; bh=g48epqajRJ9hCS8P7kBtlFSX96b9dh8PpIeMSWb2c2o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FrmEDzEdzfIcZPstYUz4Xhc99D5JyXXjiBqTO78xFOzRwoWeHXjaS7cXVaGjs/LQs c1sLioQorBSfaCi/+2LYChns+6r0wf3/cCfkeVqXC/COQgDinn6v7lm9+0SHGhfdAg Mx6KmYjxn6g6mrLm6fUly9F+WRn5an3zK2iDc/Go= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: Neal Gompa Subject: [PATCH 3/3] qcam: viewfinder_gl: Render image centered in letterbox Date: Wed, 11 Sep 2024 02:46:49 +0300 Message-ID: <20240910234649.28591-4-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.44.2 In-Reply-To: <20240910234649.28591-1-laurent.pinchart@ideasonboard.com> References: <20240910234649.28591-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Mimic the letterbox behaviour of the Qt viewfinder by rendering the image centered. This is done by adding a projection matrix to the vertex shader to scale the rendered rectangle. Another option would have been to keep using glViewport() (which would have needed to be moved to paintGL(), as Qt resets the viewport to span the full widget before calling). Hidpi displays would then need special handling of the device pixel ratio, which is done automatically by Qt when it sets the default viewport. Using a projection matrix avoids this complication. Signed-off-by: Laurent Pinchart --- src/apps/qcam/assets/shader/bayer_8.vert | 4 ++- src/apps/qcam/assets/shader/identity.vert | 3 ++- src/apps/qcam/viewfinder_gl.cpp | 31 +++++++++++++++-------- src/apps/qcam/viewfinder_gl.h | 2 +- 4 files changed, 26 insertions(+), 14 deletions(-) diff --git a/src/apps/qcam/assets/shader/bayer_8.vert b/src/apps/qcam/assets/shader/bayer_8.vert index 3695a5e910c9..fb5109eee3f5 100644 --- a/src/apps/qcam/assets/shader/bayer_8.vert +++ b/src/apps/qcam/assets/shader/bayer_8.vert @@ -19,6 +19,8 @@ Copyright (C) 2021, Linaro attribute vec4 vertexIn; attribute vec2 textureIn; +uniform mat4 proj_matrix; + uniform vec2 tex_size; /* The texture size in pixels */ uniform vec2 tex_step; @@ -47,5 +49,5 @@ void main(void) { yCoord = center.y + vec4(-2.0 * tex_step.y, -tex_step.y, tex_step.y, 2.0 * tex_step.y); - gl_Position = vertexIn; + gl_Position = proj_matrix * vertexIn; } diff --git a/src/apps/qcam/assets/shader/identity.vert b/src/apps/qcam/assets/shader/identity.vert index 12c41377cfe7..907e8741fae7 100644 --- a/src/apps/qcam/assets/shader/identity.vert +++ b/src/apps/qcam/assets/shader/identity.vert @@ -9,10 +9,11 @@ attribute vec4 vertexIn; attribute vec2 textureIn; varying vec2 textureOut; +uniform mat4 proj_matrix; uniform float stride_factor; void main(void) { - gl_Position = vertexIn; + gl_Position = proj_matrix * vertexIn; textureOut = vec2(textureIn.x * stride_factor, textureIn.y); } diff --git a/src/apps/qcam/viewfinder_gl.cpp b/src/apps/qcam/viewfinder_gl.cpp index b2096faf859f..f31956ff0504 100644 --- a/src/apps/qcam/viewfinder_gl.cpp +++ b/src/apps/qcam/viewfinder_gl.cpp @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -464,6 +465,7 @@ bool ViewFinderGL::createFragmentShader() vertexBuffer_.release(); + projMatrixUniform_ = shaderProgram_.uniformLocation("proj_matrix"); textureUniformY_ = shaderProgram_.uniformLocation("tex_y"); textureUniformU_ = shaderProgram_.uniformLocation("tex_u"); textureUniformV_ = shaderProgram_.uniformLocation("tex_v"); @@ -509,7 +511,7 @@ void ViewFinderGL::initializeGL() glEnable(GL_TEXTURE_2D); glDisable(GL_DEPTH_TEST); - static const GLfloat coordinates[2][4][2]{ + const GLfloat coordinates[2][4][2]{ { /* Vertex coordinates */ { -1.0f, -1.0f }, @@ -801,6 +803,17 @@ void ViewFinderGL::doRender() shaderProgram_.setUniformValue(textureUniformStrideFactor_, static_cast(size_.width() - 1) / (stridePixels - 1)); + + /* + * Place the viewfinder in the centre of the widget, preserving the + * aspect ratio of the image. + */ + QMatrix4x4 projMatrix; + QSizeF scaledSize = size_.scaled(size(), Qt::KeepAspectRatio); + projMatrix.scale(scaledSize.width() / size().width(), + scaledSize.height() / size().height()); + + shaderProgram_.setUniformValue(projMatrixUniform_, projMatrix); } void ViewFinderGL::paintGL() @@ -818,18 +831,14 @@ void ViewFinderGL::paintGL() close(); } - if (image_) { - glClearColor(0.0f, 0.0f, 0.0f, 1.0f); - glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); + if (!image_) + return; - doRender(); - glDrawArrays(GL_TRIANGLE_FAN, 0, 4); - } -} + glClearColor(0.0f, 0.0f, 0.0f, 1.0f); + glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); -void ViewFinderGL::resizeGL(int w, int h) -{ - glViewport(0, 0, w, h); + doRender(); + glDrawArrays(GL_TRIANGLE_FAN, 0, 4); } QSize ViewFinderGL::sizeHint() const diff --git a/src/apps/qcam/viewfinder_gl.h b/src/apps/qcam/viewfinder_gl.h index 23744b411c86..23c657bcfae2 100644 --- a/src/apps/qcam/viewfinder_gl.h +++ b/src/apps/qcam/viewfinder_gl.h @@ -51,7 +51,6 @@ Q_SIGNALS: protected: void initializeGL() override; void paintGL() override; - void resizeGL(int w, int h) override; QSize sizeHint() const override; private: @@ -88,6 +87,7 @@ private: /* Common texture parameters */ GLuint textureMinMagFilters_; + GLuint projMatrixUniform_; /* YUV texture parameters */ GLuint textureUniformU_;