{"id":12576,"url":"https://patchwork.libcamera.org/api/1.1/patches/12576/?format=json","web_url":"https://patchwork.libcamera.org/patch/12576/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20210611162726.824789-5-andrey.konovalov@linaro.org>","date":"2021-06-11T16:27:26","name":"[libcamera-devel,v3,4/4] qcam: viewfinder_gl: Add support for RAW8 Bayer formats","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"c63b68748e60a00935bb7aa4f511ff874af99568","submitter":{"id":25,"url":"https://patchwork.libcamera.org/api/1.1/people/25/?format=json","name":"Andrey Konovalov","email":"andrey.konovalov@linaro.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/12576/mbox/","series":[{"id":2124,"url":"https://patchwork.libcamera.org/api/1.1/series/2124/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2124","date":"2021-06-11T16:27:22","name":"qcam: viewfinder_gl: add RAW8, RAW10P and RAW12P formats","version":3,"mbox":"https://patchwork.libcamera.org/series/2124/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/12576/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/12576/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 209A0C320B\n\tfor ;\n\tFri, 11 Jun 2021 16:27:53 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D229E6892E;\n\tFri, 11 Jun 2021 18:27:52 +0200 (CEST)","from mail-lf1-x134.google.com (mail-lf1-x134.google.com\n\t[IPv6:2a00:1450:4864:20::134])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A77CA68939\n\tfor ;\n\tFri, 11 Jun 2021 18:27:50 +0200 (CEST)","by mail-lf1-x134.google.com with SMTP id bp38so9458063lfb.0\n\tfor ;\n\tFri, 11 Jun 2021 09:27:50 -0700 (PDT)","from localhost.localdomain ([85.249.44.185])\n\tby smtp.googlemail.com with ESMTPSA id\n\tl2sm777773lji.70.2021.06.11.09.27.49\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tFri, 11 Jun 2021 09:27:49 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"s02B/qfo\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=from:to:cc:subject:date:message-id:in-reply-to:references\n\t:mime-version:content-transfer-encoding;\n\tbh=v1yLVlRyd9h1kPxiadXi+BkUjG64afCsCJ0wPzLg6w8=;\n\tb=s02B/qfotlj8wbHWdX0T5qb7X13do/eE6IhBPkm09oRJKbOOf6BdliEv+wPWGlNNnb\n\t/uvS0Ldi7X+BrpD8UnEz2tUORm3aaH3nCGqOkujW0BqfxlcLjr1O1ceIKQPW8JvlxwTa\n\tDxKc08lEsR49mmVbYfefRv7J4WxSsBPwYA590s686nhKF1hgGIoGLsXmuzOhrvg93KdQ\n\tWFPmXKjUv5lmPZv/YG/6nptbvb6n6xpxMlfC0MfVClxWivXDYBkHDJVjzWfFewppA0I0\n\t4xif7sgllLHoEAvUzyuNdNBnmiOTg7iv0lzsYU6fkJzEwHLffV2yaYx74DpFAoSDjbcc\n\t8qfg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references:mime-version:content-transfer-encoding;\n\tbh=v1yLVlRyd9h1kPxiadXi+BkUjG64afCsCJ0wPzLg6w8=;\n\tb=SfhqMpeq+fFBP24j6EDSTs4ouP1qWS3Mtl7NlYWLFek4IiKE5vYE3qer4REpCc29Kp\n\t2PErXfRzsFpOQHtR9EjMu7WyO2+kBv0laczdtH6BoaNaMVUIecKR/wPKrk3Y24dYA3KM\n\tn9+pEKQZwJxe+EjA0Ccv8z76r+5bB9s59yPvCqbGDpeoGuC1PMUMT37lkS+3S6aD9FL0\n\t4OKspBmgblvr089oow6E6qG5lG4u7lFNjssVKa4e/3j9M2Djz4rD9htd0mcx3RKUWx25\n\t0buQDX5iW6KXte5p+K8FMxwZNTUFoCWqRWb7QSLtlrwjgkXPEq0otRiIta2kl6TZgbk0\n\t/Phw==","X-Gm-Message-State":"AOAM5310b1gp/HeWnpFHy/myJi6Rl/FcxlpsLS0Q99/t/MHWfgSd6Lcb\n\twpaSGvozDhRsGfaW+iFXZoFOzCV7dUpaJA==","X-Google-Smtp-Source":"ABdhPJwwZzq9qqH1Awjl/hoV8m8GBwGcAuc6cvIaGdfZ/xB40qnxwAD/h7g7+DJ/QlPFZXFuQWORVA==","X-Received":"by 2002:ac2:544a:: with SMTP id\n\td10mr3213024lfn.656.1623428870062; \n\tFri, 11 Jun 2021 09:27:50 -0700 (PDT)","From":"Andrey Konovalov ","To":"libcamera-devel@lists.libcamera.org,\n\tlaurent.pinchart@ideasonboard.com","Date":"Fri, 11 Jun 2021 19:27:26 +0300","Message-Id":"<20210611162726.824789-5-andrey.konovalov@linaro.org>","X-Mailer":"git-send-email 2.25.1","In-Reply-To":"<20210611162726.824789-1-andrey.konovalov@linaro.org>","References":"<20210611162726.824789-1-andrey.konovalov@linaro.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v3 4/4] qcam: viewfinder_gl: Add support\n\tfor RAW8 Bayer formats","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"morgan@casual-effects.com","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"All the four Bayer orders are supported.\n\nThe texture coordinates passed to the fragment shader are ajusted\nto point to the nearest pixel in the image. This prevents artifacts\nwhen the image is scaled from the frame resolution to the window size.\n\nSigned-off-by: Andrey Konovalov \n---\n src/qcam/assets/shader/bayer_8.frag | 136 ++++++++++++++++++++++++++++\n src/qcam/assets/shader/shaders.qrc | 1 +\n src/qcam/viewfinder_gl.cpp | 37 +++++++-\n 3 files changed, 172 insertions(+), 2 deletions(-)\n create mode 100644 src/qcam/assets/shader/bayer_8.frag","diff":"diff --git a/src/qcam/assets/shader/bayer_8.frag b/src/qcam/assets/shader/bayer_8.frag\nnew file mode 100644\nindex 00000000..d93ef1da\n--- /dev/null\n+++ b/src/qcam/assets/shader/bayer_8.frag\n@@ -0,0 +1,136 @@\n+/* SPDX-License-Identifier: BSD-2-Clause */\n+/*\n+From http://jgt.akpeters.com/papers/McGuire08/\n+\n+Efficient, High-Quality Bayer Demosaic Filtering on GPUs\n+\n+Morgan McGuire\n+\n+This paper appears in issue Volume 13, Number 4.\n+---------------------------------------------------------\n+Copyright (c) 2008, Morgan McGuire. All rights reserved.\n+\n+\n+Modified by Linaro Ltd to integrate it into libcamera, and to\n+fix the artifacts due to pixel coordinates interpolation.\n+Copyright (C) 2021, Linaro\n+*/\n+\n+//Pixel Shader\n+\n+varying vec2 textureOut;\n+\n+/* The texture size: tex_size.xy is in bytes, tex_size.zw is in pixels */\n+uniform vec4 tex_size;\n+uniform vec2 tex_step;\n+\n+/** Pixel position of the first red pixel in the */\n+/** Bayer pattern. [{0,1}, {0, 1}]*/\n+uniform vec2 tex_bayer_first_red;\n+\n+/** Monochrome RGBA or GL_LUMINANCE Bayer encoded texture.*/\n+uniform sampler2D tex_raw;\n+\n+void main(void) {\n+ #define fetch(x, y) texture2D(tex_raw, vec2(x, y)).r\n+\n+ /** .xy = Pixel being sampled in the fragment shader on the range [0, 1]\n+ .zw = ...on the range [0, sourceSize], offset by firstRed */\n+ vec4 center;\n+\n+ /** center.x + (-2/w, -1/w, 1/w, 2/w); These are the x-positions */\n+ /** of the adjacent pixels.*/\n+ vec4 xCoord;\n+\n+ /** center.y + (-2/h, -1/h, 1/h, 2/h); These are the y-positions */\n+ /** of the adjacent pixels.*/\n+ vec4 yCoord;\n+\n+ /* Align the center coordinates to the nearest pixel */\n+ center.zw = floor(textureOut * tex_size.zw);\n+ center.xy = center.zw * tex_step;\n+ center.zw += tex_bayer_first_red;\n+\n+ xCoord = center.x + vec4(-2.0 * tex_step.x,\n+ -tex_step.x, tex_step.x, 2.0 * tex_step.x);\n+ yCoord = center.y + vec4(-2.0 * tex_step.y,\n+ -tex_step.y, tex_step.y, 2.0 * tex_step.y);\n+\n+ float C = texture2D(tex_raw, center.xy).r; // ( 0, 0)\n+ const vec4 kC = vec4( 4.0, 6.0, 5.0, 5.0) / 8.0;\n+\n+ // Determine which of four types of pixels we are on.\n+ vec2 alternate = mod(floor(center.zw), 2.0);\n+\n+ vec4 Dvec = vec4(\n+ fetch(xCoord[1], yCoord[1]), // (-1,-1)\n+ fetch(xCoord[1], yCoord[2]), // (-1, 1)\n+ fetch(xCoord[2], yCoord[1]), // ( 1,-1)\n+ fetch(xCoord[2], yCoord[2])); // ( 1, 1)\n+\n+ vec4 PATTERN = (kC.xyz * C).xyzz;\n+\n+ // Can also be a dot product with (1,1,1,1) on hardware where that is\n+ // specially optimized.\n+ // Equivalent to: D = Dvec[0] + Dvec[1] + Dvec[2] + Dvec[3];\n+ Dvec.xy += Dvec.zw;\n+ Dvec.x += Dvec.y;\n+\n+ vec4 value = vec4(\n+ fetch(center.x, yCoord[0]), // ( 0,-2)\n+ fetch(center.x, yCoord[1]), // ( 0,-1)\n+ fetch(xCoord[0], center.y), // (-2, 0)\n+ fetch(xCoord[1], center.y)); // (-1, 0)\n+\n+ vec4 temp = vec4(\n+ fetch(center.x, yCoord[3]), // ( 0, 2)\n+ fetch(center.x, yCoord[2]), // ( 0, 1)\n+ fetch(xCoord[3], center.y), // ( 2, 0)\n+ fetch(xCoord[2], center.y)); // ( 1, 0)\n+\n+ // Even the simplest compilers should be able to constant-fold these to\n+ // avoid the division.\n+ // Note that on scalar processors these constants force computation of some\n+ // identical products twice.\n+ const vec4 kA = vec4(-1.0, -1.5, 0.5, -1.0) / 8.0;\n+ const vec4 kB = vec4( 2.0, 0.0, 0.0, 4.0) / 8.0;\n+ const vec4 kD = vec4( 0.0, 2.0, -1.0, -1.0) / 8.0;\n+\n+ // Conserve constant registers and take advantage of free swizzle on load\n+ #define kE (kA.xywz)\n+ #define kF (kB.xywz)\n+\n+ value += temp;\n+\n+ // There are five filter patterns (identity, cross, checker,\n+ // theta, phi). Precompute the terms from all of them and then\n+ // use swizzles to assign to color channels.\n+ //\n+ // Channel Matches\n+ // x cross (e.g., EE G)\n+ // y checker (e.g., EE B)\n+ // z theta (e.g., EO R)\n+ // w phi (e.g., EO R)\n+ #define A (value[0])\n+ #define B (value[1])\n+ #define D (Dvec.x)\n+ #define E (value[2])\n+ #define F (value[3])\n+\n+ // Avoid zero elements. On a scalar processor this saves two MADDs\n+ // and it has no effect on a vector processor.\n+ PATTERN.yzw += (kD.yz * D).xyy;\n+\n+ PATTERN += (kA.xyz * A).xyzx + (kE.xyw * E).xyxz;\n+ PATTERN.xw += kB.xw * B;\n+ PATTERN.xz += kF.xz * F;\n+\n+ vec3 rgb = (alternate.y == 0.0) ?\n+ ((alternate.x == 0.0) ?\n+ vec3(C, PATTERN.xy) :\n+ vec3(PATTERN.z, C, PATTERN.w)) :\n+ ((alternate.x == 0.0) ?\n+ vec3(PATTERN.w, C, PATTERN.z) :\n+ vec3(PATTERN.yx, C));\n+ gl_FragColor = vec4(rgb, 1.0);\n+}\ndiff --git a/src/qcam/assets/shader/shaders.qrc b/src/qcam/assets/shader/shaders.qrc\nindex d76d65c5..79f44a30 100644\n--- a/src/qcam/assets/shader/shaders.qrc\n+++ b/src/qcam/assets/shader/shaders.qrc\n@@ -5,6 +5,7 @@\n \tYUV_2_planes.frag\n \tYUV_3_planes.frag\n \tYUV_packed.frag\n+\tbayer_8.frag\n \tbayer_1x_packed.frag\n \tidentity.vert\n \ndiff --git a/src/qcam/viewfinder_gl.cpp b/src/qcam/viewfinder_gl.cpp\nindex dcfaf973..98b6b39d 100644\n--- a/src/qcam/viewfinder_gl.cpp\n+++ b/src/qcam/viewfinder_gl.cpp\n@@ -36,6 +36,11 @@ static const QList supportedFormats{\n \tlibcamera::formats::RGBA8888,\n \tlibcamera::formats::BGR888,\n \tlibcamera::formats::RGB888,\n+\t/* Raw Bayer 8-bit */\n+\tlibcamera::formats::SBGGR8,\n+\tlibcamera::formats::SGBRG8,\n+\tlibcamera::formats::SGRBG8,\n+\tlibcamera::formats::SRGGB8,\n \t/* Raw Bayer 10-bit packed */\n \tlibcamera::formats::SBGGR10_CSI2P,\n \tlibcamera::formats::SGBRG10_CSI2P,\n@@ -220,6 +225,30 @@ bool ViewFinderGL::selectFormat(const libcamera::PixelFormat &format)\n \t\tfragmentShaderDefines_.append(\"#define RGB_PATTERN bgr\");\n \t\tfragmentShaderFile_ = \":RGB.frag\";\n \t\tbreak;\n+\tcase libcamera::formats::SBGGR8:\n+\t\tfirstRed_.setX(1.0);\n+\t\tfirstRed_.setY(1.0);\n+\t\tfragmentShaderFile_ = \":bayer_8.frag\";\n+\t\ttextureMinMagFilters_ = GL_NEAREST;\n+\t\tbreak;\n+\tcase libcamera::formats::SGBRG8:\n+\t\tfirstRed_.setX(0.0);\n+\t\tfirstRed_.setY(1.0);\n+\t\tfragmentShaderFile_ = \":bayer_8.frag\";\n+\t\ttextureMinMagFilters_ = GL_NEAREST;\n+\t\tbreak;\n+\tcase libcamera::formats::SGRBG8:\n+\t\tfirstRed_.setX(1.0);\n+\t\tfirstRed_.setY(0.0);\n+\t\tfragmentShaderFile_ = \":bayer_8.frag\";\n+\t\ttextureMinMagFilters_ = GL_NEAREST;\n+\t\tbreak;\n+\tcase libcamera::formats::SRGGB8:\n+\t\tfirstRed_.setX(0.0);\n+\t\tfirstRed_.setY(0.0);\n+\t\tfragmentShaderFile_ = \":bayer_8.frag\";\n+\t\ttextureMinMagFilters_ = GL_NEAREST;\n+\t\tbreak;\n \tcase libcamera::formats::SBGGR10_CSI2P:\n \t\tfirstRed_.setX(1.0);\n \t\tfirstRed_.setY(1.0);\n@@ -624,6 +653,10 @@ void ViewFinderGL::doRender()\n \t\tshaderProgram_.setUniformValue(textureUniformY_, 0);\n \t\tbreak;\n \n+\tcase libcamera::formats::SBGGR8:\n+\tcase libcamera::formats::SGBRG8:\n+\tcase libcamera::formats::SGRBG8:\n+\tcase libcamera::formats::SRGGB8:\n \tcase libcamera::formats::SBGGR10_CSI2P:\n \tcase libcamera::formats::SGBRG10_CSI2P:\n \tcase libcamera::formats::SGRBG10_CSI2P:\n@@ -633,8 +666,8 @@ void ViewFinderGL::doRender()\n \tcase libcamera::formats::SGRBG12_CSI2P:\n \tcase libcamera::formats::SRGGB12_CSI2P:\n \t\t/*\n-\t\t * Packed raw Bayer 10-bit and 12-bit formats are stored in\n-\t\t * GL_RED texture.\n+\t\t * Raw Bayer 8-bit, and packed raw Bayer 10-bit/12-bit formats\n+\t\t * are stored in GL_RED texture.\n \t\t * The texture width is equal to the stride.\n \t\t */\n \t\tglActiveTexture(GL_TEXTURE0);\n","prefixes":["libcamera-devel","v3","4/4"]}