From patchwork Sun Aug 14 16:07:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kunal Agarwal X-Patchwork-Id: 17116 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D3F7DC3272 for ; Sun, 14 Aug 2022 16:08:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 24D0B61FC1; Sun, 14 Aug 2022 18:08:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1660493291; bh=CFCCm3qhsYlV/cSMq0SuqhAn9aXHQPSameYXDa/7Guk=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=fTzog0ciAyB8GsjO/bAwQt4e0BiVqDSKoJaMr1QuLmzxoAB2Cl1+Wc0qfAd53iCwC nDyn9vQ/EoG7BRTQHz6ikc0ojWKbJdgyvFMZu6fv1HGZREOXXLQCvn6n/CPrSlqCaS 447GVgmyK97yNDFtU/raRUw6E0FLWNUoEVrcZ8H3Qanr2nLk0ribt9Xf0aqd9NF0Vt wQr6I8dv0LSswNwnmzEfXu9bBJc0409v3/AsdeiKEJpaGC1kqT4pjsetJKpr9vnGrv kshNwdRKmovvbIJV/pF3oFhDGAHwXgMyokcxZpsAOjvUX3MQpLeOANx9YNbkzFyM90 6hkZvblriL5Eg== Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E26F361FBA for ; Sun, 14 Aug 2022 18:08:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="eScTQl+w"; dkim-atps=neutral Received: by mail-pj1-x1034.google.com with SMTP id w11-20020a17090a380b00b001f73f75a1feso12545050pjb.2 for ; Sun, 14 Aug 2022 09:08:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc; bh=Mv2/OGgWITkybWi8yKAUa6aKvJhjxQYpPZCXqcbgkr0=; b=eScTQl+wmZU+rKw5uN/VVhMtZhdq8HZ6yeSR+fksErP7NXZw504IUzBiqSdmphH/W/ r2Bvbrdf6undZNg4dyvQuCkcRdsprdi3EMq5n3VWg2t2pjtp8hda8qMDjIwyh3M0Clt7 gIY5rSvHwtCtkZoiT6ECBVHqaYHy6ie9qdjwdW4I6FZfQ/qn+FVRXv9HYu+bJmA6jW8A Oc3N/9cE623DnL/+Ij69Yfvq061WXykSRq8ouLNuIg0b5rqwq6Np0UqmHnXliOKyezOh ZM///cXiIIc7Mf61JQI2N0mK738AJf6FfsRu6IvwSxsGegxRB2jhJs2sUrnEWDwHe1SJ nVTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc; bh=Mv2/OGgWITkybWi8yKAUa6aKvJhjxQYpPZCXqcbgkr0=; b=WgbHy+IXBYWVuxmtPaqTdPgCa9TaNNwDdKfl4ukHSWGIaajVEIem9tAzDaFxJmnxVk L8uSL67iQXp81izViRPQy6XrLZHHXEjs8tPYl8SlGkccHNKHAO0Ke9gaZ1Kl7zQ/bG1i UU9Q+YXXTVD+ehSZvOG5iB4ohjOFHky+CTV68z7diNrj4M1Y1Rw4H8mq0EJdFgSs4rGy sWjzdgU35hm9/YgwZQHOHNOPDnWpGsOTck45AlYSWAjDTMG3lUAhWPPfbfMdmsQwsM4r dJtYF4xADrgUQJE064USIJVtvTSbM8s9OdiRTykTFxOpmvtT3p/GKHgDIUvUj2fRafm5 qiCA== X-Gm-Message-State: ACgBeo2oHsKN0dZZgzuYeueE9u75BgOIkogWtFvJoEXyI0vb9AomfgxT HUgHZpxqAyvIvrsf9c+sfawyzumwjsE= X-Google-Smtp-Source: AA6agR7P4Fzas4Gqq8cKMB8swQyW0DGru32OGlG/tRU0ooyoQjuIfc6peaKGrUL1edZ4Z1W9oWKb7g== X-Received: by 2002:a17:902:820d:b0:16e:c853:ddd1 with SMTP id x13-20020a170902820d00b0016ec853ddd1mr13097273pln.31.1660493287725; Sun, 14 Aug 2022 09:08:07 -0700 (PDT) Received: from pop-os.localdomain ([115.96.217.20]) by smtp.googlemail.com with ESMTPSA id j6-20020a170902da8600b001709aea1516sm5576653plx.276.2022.08.14.09.08.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 14 Aug 2022 09:08:07 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Sun, 14 Aug 2022 21:37:42 +0530 Message-Id: <20220814160747.52093-1-kunalagarwal1072002@gmail.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/6] pipeline: simple: shader: Shaders for debayering X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kunal Agarwal via libcamera-devel From: Kunal Agarwal Reply-To: Kunal Agarwal Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Default and Bayer shaders which performs debayering on raw input data Signed-off-by: Kunal Agarwal --- .../pipeline/simple/shader/bayer_8.frag | 107 ++++++++++++++++++ .../pipeline/simple/shader/bayer_8.vert | 51 +++++++++ .../pipeline/simple/shader/default.frag | 16 +++ .../pipeline/simple/shader/default.vert | 23 ++++ 4 files changed, 197 insertions(+) create mode 100644 src/libcamera/pipeline/simple/shader/bayer_8.frag create mode 100644 src/libcamera/pipeline/simple/shader/bayer_8.vert create mode 100644 src/libcamera/pipeline/simple/shader/default.frag create mode 100644 src/libcamera/pipeline/simple/shader/default.vert diff --git a/src/libcamera/pipeline/simple/shader/bayer_8.frag b/src/libcamera/pipeline/simple/shader/bayer_8.frag new file mode 100644 index 00000000..7e35ca88 --- /dev/null +++ b/src/libcamera/pipeline/simple/shader/bayer_8.frag @@ -0,0 +1,107 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* +From http://jgt.akpeters.com/papers/McGuire08/ + +Efficient, High-Quality Bayer Demosaic Filtering on GPUs + +Morgan McGuire + +This paper appears in issue Volume 13, Number 4. +--------------------------------------------------------- +Copyright (c) 2008, Morgan McGuire. All rights reserved. + +Modified by Linaro Ltd to integrate it into libcamera. +Copyright (C) 2021, Linaro +*/ + +//Pixel Shader +#ifdef GL_ES +precision mediump float; +#endif + +/** Monochrome RGBA or GL_LUMINANCE Bayer encoded texture.*/ +uniform sampler2D tex_y; +varying vec4 center; +varying vec4 yCoord; +varying vec4 xCoord; + +void main(void) { + #define fetch(x, y) texture2D(tex_y, vec2(x, y)).r + + float C = texture2D(tex_y, center.xy).r; // ( 0, 0) + const vec4 kC = vec4( 4.0, 6.0, 5.0, 5.0) / 8.0; + + // Determine which of four types of pixels we are on. + vec2 alternate = mod(floor(center.zw), 2.0); + + vec4 Dvec = vec4( + fetch(xCoord[1], yCoord[1]), // (-1,-1) + fetch(xCoord[1], yCoord[2]), // (-1, 1) + fetch(xCoord[2], yCoord[1]), // ( 1,-1) + fetch(xCoord[2], yCoord[2])); // ( 1, 1) + + vec4 PATTERN = (kC.xyz * C).xyzz; + + // Can also be a dot product with (1,1,1,1) on hardware where that is + // specially optimized. + // Equivalent to: D = Dvec[0] + Dvec[1] + Dvec[2] + Dvec[3]; + Dvec.xy += Dvec.zw; + Dvec.x += Dvec.y; + + vec4 value = vec4( + fetch(center.x, yCoord[0]), // ( 0,-2) + fetch(center.x, yCoord[1]), // ( 0,-1) + fetch(xCoord[0], center.y), // (-2, 0) + fetch(xCoord[1], center.y)); // (-1, 0) + + vec4 temp = vec4( + fetch(center.x, yCoord[3]), // ( 0, 2) + fetch(center.x, yCoord[2]), // ( 0, 1) + fetch(xCoord[3], center.y), // ( 2, 0) + fetch(xCoord[2], center.y)); // ( 1, 0) + + // Even the simplest compilers should be able to constant-fold these to + // avoid the division. + // Note that on scalar processors these constants force computation of some + // identical products twice. + const vec4 kA = vec4(-1.0, -1.5, 0.5, -1.0) / 8.0; + const vec4 kB = vec4( 2.0, 0.0, 0.0, 4.0) / 8.0; + const vec4 kD = vec4( 0.0, 2.0, -1.0, -1.0) / 8.0; + + // Conserve constant registers and take advantage of free swizzle on load + #define kE (kA.xywz) + #define kF (kB.xywz) + + value += temp; + + // There are five filter patterns (identity, cross, checker, + // theta, phi). Precompute the terms from all of them and then + // use swizzles to assign to color channels. + // + // Channel Matches + // x cross (e.g., EE G) + // y checker (e.g., EE B) + // z theta (e.g., EO R) + // w phi (e.g., EO R) + #define A (value[0]) + #define B (value[1]) + #define D (Dvec.x) + #define E (value[2]) + #define F (value[3]) + + // Avoid zero elements. On a scalar processor this saves two MADDs + // and it has no effect on a vector processor. + PATTERN.yzw += (kD.yz * D).xyy; + + PATTERN += (kA.xyz * A).xyzx + (kE.xyw * E).xyxz; + PATTERN.xw += kB.xw * B; + PATTERN.xz += kF.xz * F; + + gl_FragColor.rgb = (alternate.y == 0.0) ? + ((alternate.x == 0.0) ? + vec3(C, PATTERN.xy) : + vec3(PATTERN.z, C, PATTERN.w)) : + ((alternate.x == 0.0) ? + vec3(PATTERN.w, C, PATTERN.z) : + vec3(PATTERN.yx, C)); +} diff --git a/src/libcamera/pipeline/simple/shader/bayer_8.vert b/src/libcamera/pipeline/simple/shader/bayer_8.vert new file mode 100644 index 00000000..3695a5e9 --- /dev/null +++ b/src/libcamera/pipeline/simple/shader/bayer_8.vert @@ -0,0 +1,51 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* +From http://jgt.akpeters.com/papers/McGuire08/ + +Efficient, High-Quality Bayer Demosaic Filtering on GPUs + +Morgan McGuire + +This paper appears in issue Volume 13, Number 4. +--------------------------------------------------------- +Copyright (c) 2008, Morgan McGuire. All rights reserved. + +Modified by Linaro Ltd to integrate it into libcamera. +Copyright (C) 2021, Linaro +*/ + +//Vertex Shader + +attribute vec4 vertexIn; +attribute vec2 textureIn; + +uniform vec2 tex_size; /* The texture size in pixels */ +uniform vec2 tex_step; + +/** Pixel position of the first red pixel in the */ +/** Bayer pattern. [{0,1}, {0, 1}]*/ +uniform vec2 tex_bayer_first_red; + +/** .xy = Pixel being sampled in the fragment shader on the range [0, 1] + .zw = ...on the range [0, sourceSize], offset by firstRed */ +varying vec4 center; + +/** center.x + (-2/w, -1/w, 1/w, 2/w); These are the x-positions */ +/** of the adjacent pixels.*/ +varying vec4 xCoord; + +/** center.y + (-2/h, -1/h, 1/h, 2/h); These are the y-positions */ +/** of the adjacent pixels.*/ +varying vec4 yCoord; + +void main(void) { + center.xy = textureIn; + center.zw = textureIn * tex_size + tex_bayer_first_red; + + xCoord = center.x + vec4(-2.0 * tex_step.x, + -tex_step.x, tex_step.x, 2.0 * tex_step.x); + yCoord = center.y + vec4(-2.0 * tex_step.y, + -tex_step.y, tex_step.y, 2.0 * tex_step.y); + + gl_Position = vertexIn; +} diff --git a/src/libcamera/pipeline/simple/shader/default.frag b/src/libcamera/pipeline/simple/shader/default.frag new file mode 100644 index 00000000..5899f8fc --- /dev/null +++ b/src/libcamera/pipeline/simple/shader/default.frag @@ -0,0 +1,16 @@ +#version 310 es +precision mediump float; +// Outputs colors in RGBA +out vec4 FragColor; + +//Inputs color from the Vertex Shader +in vec3 color; +// Inputs the texture coordinates from the Vertex Shader +in vec2 texCoord; + +uniform sampler2D tex0; + +void main() +{ + FragColor = texture(tex0,texCoord); +} diff --git a/src/libcamera/pipeline/simple/shader/default.vert b/src/libcamera/pipeline/simple/shader/default.vert new file mode 100644 index 00000000..1a24284b --- /dev/null +++ b/src/libcamera/pipeline/simple/shader/default.vert @@ -0,0 +1,23 @@ +#version 310 es +precision mediump float; +//Positions/Coordinates +layout (location = 0) in vec3 aPos; +//Colors +layout (location = 1) in vec3 aColor; +// Texture Coordinates +layout (location = 2) in vec2 aTex; + +//Outputs the color for the Fragment Shader +out vec3 color; +// Outputs the texture coordinates to the fragment shader +out vec2 texCoord; + + +void main() +{ + gl_Position = vec4(aPos.x, aPos.y, aPos.z, 1.0); + //Assigns the color from the vertex Data to "color" + color = aColor; + // Assigns the texture coordinates from the Vertex Data to "texCoord" + texCoord = aTex; +} \ No newline at end of file