{"id":13296,"url":"https://patchwork.libcamera.org/api/patches/13296/?format=json","web_url":"https://patchwork.libcamera.org/patch/13296/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"","date":"2021-08-11T06:12:56","name":"[libcamera-devel,RFC,v2,2/4] libcamera: swisp: The software ISP class","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"76e5e75345fcb032319b0e654cb5b71911c265c2","submitter":{"id":88,"url":"https://patchwork.libcamera.org/api/people/88/?format=json","name":"Siyuan Fan","email":"siyuan.fan@foxmail.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/13296/mbox/","series":[{"id":2336,"url":"https://patchwork.libcamera.org/api/series/2336/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2336","date":"2021-08-11T06:12:54","name":"pipeline: isp: The software ISP module","version":2,"mbox":"https://patchwork.libcamera.org/series/2336/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/13296/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/13296/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id CA9C3C3240\n\tfor ;\n\tWed, 11 Aug 2021 06:13:51 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 7F2BF68887;\n\tWed, 11 Aug 2021 08:13:51 +0200 (CEST)","from out203-205-221-239.mail.qq.com (out203-205-221-239.mail.qq.com\n\t[203.205.221.239])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id BA3E760261\n\tfor ;\n\tWed, 11 Aug 2021 08:13:48 +0200 (CEST)","from localhost.localdomain ([123.126.82.8])\n\tby newxmesmtplogicsvrszc9.qq.com (NewEsmtp) with SMTP\n\tid 3569928F; Wed, 11 Aug 2021 14:13:22 +0800"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=foxmail.com header.i=@foxmail.com\n\theader.b=\"lYfkTV9m\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com;\n\ts=s201512; t=1628662425;\n\tbh=UUxf8YzWkicuLPPbKmd5HQVEu9LR0W1YIB+C7AjdmxU=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References;\n\tb=lYfkTV9mk8hDGIuPxiABXgcxj0UCu7blevgkZhs6nlGCmDhS9N4yVZtTpPtJktR0D\n\tabWk15cDwfVKaMknsl0p2iKm7zIPGhWBCyfONIbwcQMtPPaEuzJh2grH2hxYpjFv1Q\n\tOkmqQc9qbVHHQIA+d5Zj5bs2b9b/RNQBpXObs8xQ=","X-QQ-mid":"xmsmtpt1628662405tls74qumv","Message-ID":"","X-QQ-XMAILINFO":"N4ZILu3POr3OC73HyoXAn++rILiOp5rnqKOnh9dMrO6TaOj4ygobtWTpvUHWpL\n\twdHSYea8kL2YpOxxPhL4DnrIpFpsOoaxmDNmlDj/rAJK67jjdcx08q/T+vzaguLzQuRwqXLHHx1Z\n\tUXKzLEwnzK3wtvzh7zVuB8xVJ3KT9fg0H6rHygEWiiHWjAlSvqzH3WNCw6YoDgRO9x5dOKJAKp/Y\n\tTh5ov1u/mgZtOk/rdf7uixzGDDwEFaoT61dnCFJ5PIeFPzJK/3/WUCqDMxB5PjbhPM1YY3WF+9Ne\n\tqZ8X0R+J93ajLnuZrm7PRrW2WViOxAJh2Ng5jrf3JDf2Z/GBzxohYgLJHwaMhtduQyiI4V5rw50z\n\tTecauHf0/6sAxHbu+ChLNeLUo3tp1aApwqxz4kc6VOc3tTHA95/qXGbxtjh/CuFOb4z2McQbVUVF\n\taeG2EKFsx1PB5qYfq9L/8cnUleQdHsPHxT3JMHQG/v2WBncIjwhn//sbwP/Ip4TROJl7/PQ90Ib3\n\tokzL7oE1FvSimZ1FYQSijXX/GhsjJDB+fQ+giEEqO8mLzIoAFnVLRtOUUPImQ/kBg6iqQ5c+El7y\n\tYJLUWuxmhoV+yyItV/buI1ZEYL5Opo2WV15V76SvNRuXAjWc4urvG3NqYY55x7nggFi68Wr1B7fW\n\tbhFvWmqZJE+eeiWqH41DoTN6x6Ja6V1QOl91FG1Pt5btopXJfBTUDeogXZStw8l4NHBEBHFNr4bl\n\tGVmEhCPzFbchOqgDSKwWewAbO/LRPW2SK94wspv68YvUDQxfoXaJPf5yYUmi1P1JNZFMGiLJgyVP\n\tnDnZ7oE8ymSN3wR5LeSL4DFEp72soM4zNHGeOojPd2G9e9ze58dq0GY0+momPda4gkOeQDUKesLH\n\t6Q9T00pOR2U66dsxOs6GiS9zVGZbQT9RxpbvI64tiaT5rbhg+kQYsE9UElOXhXfdAV2MfcZUOYdI\n\tgE83llGVI71cRbiXcr6w==","From":"Siyuan Fan ","To":"libcamera-devel@lists.libcamera.org","Date":"Wed, 11 Aug 2021 07:12:56 +0100","X-OQ-MSGID":"<20210811061258.7421-3-siyuan.fan@foxmail.com>","X-Mailer":"git-send-email 2.20.1","In-Reply-To":"<20210811061258.7421-1-siyuan.fan@foxmail.com>","References":"<20210811061258.7421-1-siyuan.fan@foxmail.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [RFC PATCH v2 2/4] libcamera: swisp: The software\n\tISP class","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"From: Fan Siyuan \n\n Changes in V2:\n -Design the abstract class ISP, CPU_ISP and future GPU_ISP inherit from it.\n -Add the rgb buf alloc and thread control member function\n\n Now the ISP must implement the below process:\n BLC-->demosaic-->AWB-->tone mapping-->gamma-->Noise reduction\n\n Plan to add:\n lens shading correct-->Color correct-->color space transform\n\nSigned-off-by: Fan Siyuan \n---\n src/libcamera/swisp/isp.cpp | 643 ++++++++++++++++++++++++++++++++++++\n src/libcamera/swisp/isp.h | 92 ++++++\n 2 files changed, 735 insertions(+)\n create mode 100644 src/libcamera/swisp/isp.cpp\n create mode 100644 src/libcamera/swisp/isp.h","diff":"diff --git a/src/libcamera/swisp/isp.cpp b/src/libcamera/swisp/isp.cpp\nnew file mode 100644\nindex 00000000..6d073040\n--- /dev/null\n+++ b/src/libcamera/swisp/isp.cpp\n@@ -0,0 +1,643 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2021, Siyuan Fan \n+ *\n+ * isp.cpp - The software ISP class\n+ */\n+\n+#include \"isp.h\"\n+\n+#include \n+#include \n+#include \n+#include \n+#include \n+\n+#include \n+#include \n+\n+#include \"libcamera/base/log.h\"\n+\n+namespace libcamera{\n+\n+LOG_DECLARE_CATEGORY(ISP)\n+\n+void CPU_ISP::autoContrast(uint16_t *data, float lowCut, float highCut, int width, int height)\n+{\n+ int blue, gr, gb, red;\n+ int histBlue[1024] = {0}, histGb[1024] = {0}, histGr[1024] = {0}, histRed[1024] = {0};\n+\n+ int index;\n+ for (int i = 0; i < height; i++) {\n+ index = i * width;\n+ for (int j = 0; j < width; j++) {\n+ if (i % 2 == 0 && j % 2 == 0) {\n+ blue = data[index];\n+ histBlue[blue]++;\n+ }\n+ else if ((i % 2 == 0 && j % 2 == 1)) {\n+ gb = data[index];\n+ histGb[gb]++;\n+ }\n+ else if ((i % 2 == 1 && j % 2 == 0)) {\n+ gr = data[index];\n+ histGr[gr]++;\n+ }\n+ else {\n+ red = data[index];\n+ histRed[red]++;\n+ }\n+ index++;\n+ } \n+ }\n+\n+ int pixelAmount = width * height;\n+ int sum = 0;\n+ int minBlue;\n+ for (int i = 0; i < 1024; i++){\n+ sum = sum + histBlue[i];\n+ if (sum >= pixelAmount * lowCut * 0.01) {\n+ minBlue = i;\n+ break;\n+ }\n+ }\n+\n+ sum = 0;\n+ int maxBlue;\n+ for (int i = 1023; i >= 0; i--){\n+ sum = sum + histBlue[i];\n+ if (sum >= pixelAmount * highCut * 0.01) {\n+ maxBlue = i;\n+ break;\n+ }\n+ }\n+\n+ sum = 0;\n+ int minGb;\n+ for (int i = 0; i < 1024; i++){\n+ sum = sum + histGb[i];\n+ if (sum >= pixelAmount * lowCut * 0.01) {\n+ minGb = i;\n+ break;\n+ }\n+ }\n+\n+ sum = 0;\n+ int maxGb;\n+ for (int i = 1023; i >= 0; i--){\n+ sum = sum + histGb[i];\n+ if (sum >= pixelAmount * highCut * 0.01) {\n+ maxGb = i;\n+ break;\n+ }\n+ }\n+\n+ sum = 0;\n+ int minGr;\n+ for (int i = 0; i < 1024; i++){\n+ sum = sum + histGr[i];\n+ if (sum >= pixelAmount * lowCut * 0.01) {\n+ minGr = i;\n+ break;\n+ }\n+ }\n+\n+ sum = 0;\n+ int maxGr;\n+ for (int i = 1023; i >= 0; i--){\n+ sum = sum + histGr[i];\n+ if (sum >= pixelAmount * highCut * 0.01) {\n+ maxGr = i;\n+ break;\n+ }\n+ }\n+\n+ sum = 0;\n+ int minRed;\n+ for (int i = 0; i < 1024; i++){\n+ sum = sum + histRed[i];\n+ if (sum >= pixelAmount * lowCut * 0.01) {\n+ minRed = i;\n+ break;\n+ }\n+ }\n+\n+ sum = 0;\n+ int maxRed;\n+ for (int i = 1023; i >= 0; i--){\n+ sum = sum + histRed[i];\n+ if (sum >= pixelAmount * highCut * 0.01) {\n+ maxRed = i;\n+ break;\n+ }\n+ }\n+\n+ int blueMap[1024];\n+ float norb = 1.0 / (maxBlue - minBlue);\n+ for (int i = 0; i < 1024; i++) {\n+ if (i < minBlue) {\n+ blueMap[i] = 0;\n+ }\n+ else if (i > maxBlue) {\n+ blueMap[i] = 1023;\n+ }\n+ else {\n+ blueMap[i] = (i - minBlue) * norb * 1023;\n+ }\n+ if (blueMap[i] > 1023) blueMap[i] = 1023;\n+ }\n+\n+ int gbMap[1024];\n+ float norgb = 1.0 / (maxGb - minGb);\n+ for (int i = 0; i < 1024; i++) {\n+ if (i < minGb) {\n+ gbMap[i] = 0;\n+ }\n+ else if (i > maxGb) {\n+ gbMap[i] = 1023;\n+ }\n+ else {\n+ gbMap[i] = (i - minGb) * norgb * 1023;\n+ }\n+ if (gbMap[i] > 1023) gbMap[i] = 1023;\n+ }\n+\n+ int grMap[1024];\n+ float norgr = 1.0 / (maxGr - minGr);\n+ for (int i = 0; i < 1024; i++) {\n+ if (i < minGr) {\n+ grMap[i] = 0;\n+ }\n+ else if (i > maxGr) {\n+ grMap[i] = 1023;\n+ }\n+ else {\n+ grMap[i] = (i - minGr) * norgr * 1023;\n+ }\n+ if (grMap[i] > 1023) grMap[i] = 1023;\n+ }\n+\n+ int redMap[1024];\n+ float norr = 1.0 / (maxRed - minRed);\n+ for (int i = 0; i < 1024; i++) {\n+ if (i < minRed) {\n+ redMap[i] = 0;\n+ }\n+ else if (i > maxRed) {\n+ redMap[i] = 1023;\n+ }\n+ else{\n+ redMap[i] = (i - minRed) * norr * 1023;\n+ }\n+ if (redMap[i] > 1023) redMap[i] = 1023;\n+ }\n+\n+ for (int i = 0;i < height; i++) {\n+ for (int j = 0; j < width; j++){\n+ index = i * width;\n+ if (i % 2 == 0 && j % 2 == 0) {\n+ data[index] = blueMap[data[index]];\n+ } \n+ else if (i % 2 == 0 && j % 2 == 1) {\n+ data[index] = gbMap[data[index]];\n+ } \n+ else if (i % 2 == 1 && j % 2 == 0) {\n+ data[index] = grMap[data[index]];\n+ } \n+ else {\n+ data[index] = redMap[data[index]];\n+ }\n+ index++;\n+ }\n+ }\n+}\n+\n+void CPU_ISP::blackLevelCorrect(uint16_t *data, uint16_t offset, int width, int height)\n+{\n+ int len = width * height;\n+ for(int i = 0; i < len; i++) {\n+ if (data[i] < offset){\n+ data[i] = 0;\n+ }\n+ else {\n+ data[i] -= offset;\n+ }\n+ }\n+}\n+\n+void CPU_ISP::readChannels(uint16_t *data, uint16_t *R, uint16_t *G, uint16_t *B,\n+ int width, int height)\n+{\n+ int index;\n+ for (int i = 0; i < height; i++) {\n+ index = i * width;\n+ for (int j = 0; j < width; j++) {\n+ if (i % 2 == 0 && j % 2 == 0) {\n+ B[index] = data[index];\n+ }\n+ else if ((i % 2 == 0 && j % 2 == 1) || (i % 2 == 1 && j % 2 == 0)){\n+ G[index] = data[index];\n+ }\n+ else {\n+ R[index] = data[index];\n+ }\n+ index++;\n+ }\n+ }\n+}\n+\n+void CPU_ISP::firstPixelInsert(uint16_t *src, uint16_t *dst, int width, int height)\n+{\n+ int index;\n+ for (int i = 0; i < height; i++) {\n+ index = i * width;\n+ for (int j = 0; j < width; j++){\n+ if (i % 2 == 0 && j % 2 == 1) {\n+ if (j == (width - 1)) {\n+ dst[index] = src[index - 1];\n+ }\n+ else {\n+ dst[index] = (src[index - 1] +\n+ src[index + 1]) >> 1;\n+ }\n+ }\n+\n+ if (i % 2 == 1 && j % 2 == 0) {\n+ if(i == height - 1) {\n+ dst[index] = src[index - width];\n+ }\n+ else {\n+ dst[index] = (src[index - width]+\n+ src[index + width]) >> 1;\n+ }\n+ }\n+ \n+ if (i % 2 == 1 && j % 2 == 1) {\n+ if (j < width - 1 && i < height - 1) {\n+ dst[index] = (src[index - width - 1] +\n+ src[index - width + 1] +\n+ src[index + width - 1] +\n+ src[index + width + 1]) >> 2;\n+ }\n+ else if (i == height - 1 && j < width - 1) {\n+ dst[index] = (src[index - width - 1] +\n+ src[index - width + 1]) >> 1;\n+ }\n+ else if (i < height - 1 && j == width - 1) {\n+ dst[index] = (src[index - width - 1] +\n+ src[index + width - 1]) >> 1;\n+ }\n+ else {\n+ dst[index] = src[index - width - 1];\n+ } \n+ }\n+ index++;\n+ }\n+ }\n+}\n+\n+void CPU_ISP::twoPixelInsert(uint16_t *src, uint16_t *dst, int width, int height)\n+{\n+ int index;\n+ for (int i = 0; i < height; i++) {\n+ index = i * width;\n+ for (int j = 0; j < width; j++) {\n+ if (i == 0 && j == 0) {\n+ dst[index] = (src[index + width] +\n+ src[index + 1]) >> 1;\n+ }\n+ else if (i == 0 && j > 0 && j % 2 == 0) {\n+ dst[index] = (src[index - 1] +\n+ src[index + width] +\n+ src[index + 1]) / 3;\n+ }\n+ else if (i > 0 && j == 0 && i % 2 == 0) {\n+ dst[index] = (src[index - width] +\n+ src[index + 1] +\n+ src[index + width]) / 3;\n+ }\n+ else if (i == (height - 1) && j < (width - 1) && j % 2 == 1) {\n+ dst[index] = (src[index - 1] +\n+ src[index - width] +\n+ src[index + 1]) / 3;\n+ }\n+ else if (i < (height - 1) && j == (width - 1) && i % 2 == 1) {\n+ dst[index] = (src[index - width] +\n+ src[index - 1] +\n+ src[index + width]) / 3;\n+ }\n+ else if (i == (height - 1) && j == (width - 1)) {\n+ dst[index] = (src[index - width] +\n+ src[index - 1]) >> 1;\n+ }\n+ else if ((i % 2 == 0 && j % 2 == 0) || (i % 2 == 1 && j % 2 == 1)) {\n+ dst[index] = (src[index - 1] +\n+ src[index + 1] +\n+ src[index - width] +\n+ src[index + width]) / 4;\n+ }\n+ index++;\n+ }\n+ }\n+}\n+\n+void CPU_ISP::lastPixelInsert(uint16_t *src, uint16_t *dst, int width, int height)\n+{\n+ int index;\n+ for (int i = 0; i < height; i++) {\n+ index = i * width;\n+ for (int j = 0; j < width; j++){\n+ if (i % 2 == 1 && j % 2 == 0) {\n+ if (j == 0) {\n+ dst[index] = src[index + 1];\n+ }\n+ else {\n+ dst[index] = (src[index - 1] +\n+ src[index + 1]) >> 1;\n+ }\n+ }\n+\n+ if (i % 2 == 0 && j % 2 == 1) {\n+ if(i == 0) {\n+ dst[index] = src[index + width];\n+ }\n+ else {\n+ dst[index] = (src[index - width]+\n+ src[index + width]) >> 1;\n+ }\n+ }\n+\n+ if (i % 2 == 0 && j % 2 == 0) {\n+ if (i > 0 && j > 0) {\n+ dst[index] = (src[index - width - 1] +\n+ src[index - width + 1] +\n+ src[index + width - 1] +\n+ src[index + width + 1]) >> 2;\n+ }\n+ else if (i == 0 && j > 0) {\n+ dst[index] = (src[index + width - 1] +\n+ src[index + width + 1]) >> 1;\n+ }\n+ else if (i > 0 && j == 0) {\n+ dst[index] = (src[index - width + 1] +\n+ src[index + width + 1]) >> 1;\n+ }\n+ else {\n+ dst[index] = src[index + width + 1];\n+ }\n+ }\n+ index++;\n+ }\n+ }\n+}\n+\n+void CPU_ISP::demosaic(uint16_t *data, uint16_t *R, uint16_t *G, uint16_t *B,\n+ int width, int height)\n+{\n+ firstPixelInsert(data, B, width, height);\n+ twoPixelInsert(data, G, width, height);\n+ lastPixelInsert(data, R, width, height);\n+}\n+\n+void CPU_ISP::autoWhiteBalance(uint16_t *R, uint16_t *G, uint16_t *B, int width, int height)\n+{\n+ float aveB = 0, aveG = 0, aveR = 0;\n+ float Kb, Kg, Kr;\n+\n+ for (int i = 0; i < width * height; i++) {\n+ aveB += 1.0 * B[i];\n+ aveG += 1.0 * G[i];\n+ aveR += 1.0 * R[i];\n+ }\n+\n+ aveB *= (1.0 / (width * height));\n+ aveG *= (1.0 / (width * height));\n+ aveR *= (1.0 / (width * height));\n+\n+ Kr = (aveB + aveG + aveR) / aveR * (1.0 / 3.0);\n+ Kg = (aveB + aveG + aveR) / aveG * (1.0 / 3.0);\n+ Kb = (aveB + aveG + aveR) / aveB * (1.0 / 3.0);\n+\n+ for (int i = 0; i < width * height; i++) {\n+ B[i] = B[i] * Kb;\n+ G[i] = G[i] * Kg;\n+ R[i] = R[i] * Kr;\n+\n+ if (R[i] > 1023) R[i] = 1023;\n+ if (G[i] > 1023) G[i] = 1023;\n+ if (R[i] > 1023) B[i] = 1023;\n+ }\n+}\n+\n+void CPU_ISP::gammaCorrect(uint16_t *R, uint16_t *G, uint16_t *B, float val, int width, int height)\n+{\n+ float nor = 1.0 / 1023.0;\n+ float gamma = 1.0 / val;\n+ for (int i = 0; i < width * height; i++) {\n+ R[i] = pow(R[i] * nor, gamma) * 1023;\n+ G[i] = pow(G[i] * nor, gamma) * 1023;\n+ B[i] = pow(B[i] * nor, gamma) * 1023;\n+\n+ if (R[i] > 1023) R[i] = 1023;\n+ if (G[i] > 1023) G[i] = 1023;\n+ if (B[i] > 1023) B[i] = 1023;\n+ }\n+}\n+\n+void CPU_ISP::compress_10bit_to_8bit (uint16_t *src, uint8_t *dst, int width, int height)\n+{\n+ for (int i = 0; i < width * height; i++) {\n+ dst[i] = src[i] >> 2 & 0xff;\n+ }\n+}\n+\n+float CPU_ISP::distance(int x, int y, int i, int j)\n+{\n+ return float(sqrt(pow(x - i, 2) + pow(y - j, 2)));\n+}\n+\n+double CPU_ISP::gaussian(float x, double sigma)\n+{\n+ return exp(-(pow(x, 2)) / (2 * pow(sigma, 2))) / (2 * 3.1415926 * pow(sigma, 2));\n+}\n+\n+void CPU_ISP::bilateralFilter(uint16_t *R, uint16_t *G, uint16_t *B,\n+ int diameter, double sigmaI,\n+ double sigmaS, int width, int height)\n+{\n+ for (int i = 2; i < height - 2; i++) {\n+ for (int j = 2; j < width - 2; j++) {\n+ double iFiltered = 0;\n+ double wp = 0;\n+ int neighbor_x = 0;\n+ int neighbor_y = 0;\n+ int half = diameter / 2;\n+\n+ for (int k = 0; k < diameter; k++) {\n+ for (int l = 0; l < diameter; l++) {\n+ neighbor_x = i - (half - k);\n+ neighbor_y = j - (half - l);\n+ double gi = gaussian(R[neighbor_x * width + neighbor_y] - R[i * width +j], sigmaI);\n+ double gs = gaussian(distance(i, j, neighbor_x, neighbor_y), sigmaS);\n+ double w = gi * gs;\n+ iFiltered = iFiltered + R[neighbor_x * width + neighbor_y] * w;\n+ wp = wp + w;\n+ } \n+ }\n+\n+ iFiltered = iFiltered / wp;\n+ R[i * width + j] = iFiltered;\n+ }\n+ }\n+\n+ for (int i = 2; i < height - 2; i++) {\n+ for (int j = 2; j < width - 2; j++) {\n+ double iFiltered = 0;\n+ double wp = 0;\n+ int neighbor_x = 0;\n+ int neighbor_y = 0;\n+ int half = diameter / 2;\n+\n+ for (int k = 0; k < diameter; k++) {\n+ for (int l = 0; l < diameter; l++) {\n+ neighbor_x = i - (half - k);\n+ neighbor_y = j - (half - l);\n+ double gi = gaussian(G[neighbor_x * width + neighbor_y] - G[i * width +j], sigmaI);\n+ double gs = gaussian(distance(i, j, neighbor_x, neighbor_y), sigmaS);\n+ double w = gi * gs;\n+ iFiltered = iFiltered + G[neighbor_x * width + neighbor_y] * w;\n+ wp = wp + w;\n+ } \n+ }\n+\n+ iFiltered = iFiltered / wp;\n+ G[i * width + j] = iFiltered;\n+ }\n+ }\n+\n+ for (int i = 2; i < height - 2; i++) {\n+ for (int j = 2; j < width - 2; j++) {\n+ double iFiltered = 0;\n+ double wp = 0;\n+ int neighbor_x = 0;\n+ int neighbor_y = 0;\n+ int half = diameter / 2;\n+\n+ for (int k = 0; k < diameter; k++) {\n+ for (int l = 0; l < diameter; l++) {\n+ neighbor_x = i - (half - k);\n+ neighbor_y = j - (half - l);\n+ double gi = gaussian(B[neighbor_x * width + neighbor_y] - B[i * width +j], sigmaI);\n+ double gs = gaussian(distance(i, j, neighbor_x, neighbor_y), sigmaS);\n+ double w = gi * gs;\n+ iFiltered = iFiltered + B[neighbor_x * width + neighbor_y] * w;\n+ wp = wp + w;\n+ } \n+ }\n+\n+ iFiltered = iFiltered / wp;\n+ B[i * width + j] = iFiltered;\n+ }\n+ }\n+}\n+\n+void CPU_ISP::noiseReduction(uint16_t *R, uint16_t *G, uint16_t *B, int width, int height)\n+{\n+ bilateralFilter(R, G, B, 5, 24.0, 32.0, width, height);\n+}\n+\n+void CPU_ISP::processing(FrameBuffer *srcBuffer, FrameBuffer *dstBuffer, int width, int height)\n+{\n+ uint8_t *rgb_buf;\n+ uint16_t *rawData;\n+ uint16_t *rgbData = new std::uint16_t[width * height * 3];\n+\n+ uint16_t *rData = rgbData;\n+ uint16_t *gData = rData + width * height;\n+ uint16_t *bData = gData + width * height;\n+ memset(rgbData, 0x0, width * height * 3);\n+\n+ const FrameBuffer::Plane &plane = srcBuffer->planes()[0]; \n+ rawData = (uint16_t *)mmap(NULL, plane.length, PROT_READ|PROT_WRITE, MAP_SHARED, plane.fd.fd(), 0);\n+ LOG(ISP, Debug) << plane.length << \" \" << plane.fd.fd();\n+ if (rawData == MAP_FAILED) {\n+ LOG(ISP, Error) << \"Read raw data failed\";\n+ ispCompleted.emit(dstBuffer);\n+ }\n+\n+ blackLevelCorrect(rawData, 16, width, height);\n+ readChannels(rawData, rData, gData, bData, width, height);\n+ demosaic(rawData, rData, gData, bData, width, height);\n+ autoWhiteBalance(rData, gData, bData, width, height);\n+ autoContrast(rData, 0.01, 0.01, width, height);\n+ autoContrast(gData, 0.01, 0.01, width, height);\n+ autoContrast(bData, 0.01, 0.01, width, height);\n+ gammaCorrect(rData, gData, bData, 2.2, width, height);\n+ //bilateralFilter(rData, gData, bData, 5, 24.0, 32.0, width, height);\n+ \n+ const FrameBuffer::Plane &rgbPlane = dstBuffer->planes()[0];\n+ rgb_buf = (uint8_t *)mmap(NULL, rgbPlane.length, PROT_READ|PROT_WRITE, MAP_SHARED, rgbPlane.fd.fd(), 0);\n+ if (rgb_buf == MAP_FAILED) {\n+ LOG(ISP, Error) << \"Read rgb data failed\";\n+ ispCompleted.emit(dstBuffer);\n+ }\n+\n+ uint8_t *rData_8 = rgb_buf;\n+ uint8_t *gData_8 = rData_8 + width * height;\n+ uint8_t *bData_8 = gData_8 + width * height;\n+\n+ compress_10bit_to_8bit(rData, rData_8, width, height);\n+ compress_10bit_to_8bit(gData, gData_8, width, height);\n+ compress_10bit_to_8bit(bData, bData_8, width, height);\n+\n+ dstBuffer->metadata_.status = srcBuffer->metadata().status;\n+ dstBuffer->metadata_.sequence = srcBuffer->metadata().sequence;\n+ dstBuffer->metadata_.timestamp = srcBuffer->metadata().timestamp;\n+\n+ dstBuffer->metadata_.planes.clear();\n+ dstBuffer->metadata_.planes.push_back({rgbPlane.length});\n+\n+ delete[] rgbData;\n+\n+ ispCompleted.emit(dstBuffer);\n+}\n+\n+int CPU_ISP::exportBuffers(std::vector> *buffers,\n+ unsigned int count, int width, int height)\n+{\n+ int bufferByte = width * height * 3;\n+\n+ for (unsigned int i = 0; i < count; i++) {\n+ std::string name = \"frame-\" + std::to_string(i);\n+\n+ const int isp_fd = memfd_create(name.c_str(), 0);\n+ int ret = ftruncate(isp_fd, bufferByte);\n+ if (ret < 0) {\n+ LOG(ISP, Error) << \"Failed to resize memfd\" << strerror(-ret);\n+ return ret;\n+ }\n+\n+ FrameBuffer::Plane rgbPlane;\n+ rgbPlane.fd = FileDescriptor(std::move(isp_fd));\n+ rgbPlane.length = bufferByte;\n+\n+ std::vector planes{rgbPlane};\n+ buffers->emplace_back(std::make_unique(std::move(planes)));\n+ }\n+\n+ return count;\n+}\n+\n+void CPU_ISP::startThreadISP()\n+{\n+ moveToThread(&thread_);\n+ thread_.start();\n+}\n+\n+void CPU_ISP::stopThreadISP()\n+{\n+ thread_.exit();\n+ thread_.wait();\n+}\n+\n+} /* namespace libcamera */\n\\ No newline at end of file\ndiff --git a/src/libcamera/swisp/isp.h b/src/libcamera/swisp/isp.h\nnew file mode 100644\nindex 00000000..4099e2c6\n--- /dev/null\n+++ b/src/libcamera/swisp/isp.h\n@@ -0,0 +1,92 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2021, Siyuan Fan \n+ *\n+ * isp.h - The software ISP class\n+ */\n+#ifndef __LIBCAMERA_SWISP_ISP_H__\n+#define __LIBCAMERA_SWISP_ISP_H__\n+\n+#include \n+\n+#include \"libcamera/base/object.h\"\n+#include \"libcamera/base/signal.h\"\n+#include \"libcamera/base/thread.h\"\n+\n+namespace libcamera{\n+\n+using std::uint16_t;\n+using std::uint8_t;\n+\n+class ISP : public Object\n+{\n+public:\n+ ISP() {}\n+\n+ virtual void blackLevelCorrect(uint16_t *data, uint16_t offset, int width, int height) = 0;\n+\n+ virtual void demosaic(uint16_t *data, uint16_t *R, uint16_t *G, uint16_t *B,\n+ int width, int height) = 0;\n+\n+ virtual void autoWhiteBalance(uint16_t *R, uint16_t *G, uint16_t *B, int width, int height) = 0;\n+\n+ virtual void gammaCorrect(uint16_t *R, uint16_t *G, uint16_t *B, float val, int width, int height) = 0;\n+\n+ virtual void noiseReduction(uint16_t *R, uint16_t *G, uint16_t *B, int width, int height) = 0;\n+\n+ virtual ~ISP() {}\n+};\n+\n+class CPU_ISP : public ISP\n+{\n+public:\n+ void autoContrast(uint16_t *data, float lowCut, float highCut, int width, int height);\n+\n+ void blackLevelCorrect(uint16_t *data, uint16_t offset, int width, int height) override;\n+\n+ void readChannels(uint16_t *data, uint16_t *R, uint16_t *G, uint16_t *B,\n+ int width, int height);\n+\n+ void firstPixelInsert(uint16_t *src, uint16_t *dst, int width, int height);\n+\n+ void twoPixelInsert(uint16_t *src, uint16_t *dst, int width, int height);\n+\n+ void lastPixelInsert(uint16_t *src, uint16_t *dst, int width, int height);\n+\n+ void demosaic(uint16_t *data, uint16_t *R, uint16_t *G, uint16_t *B,\n+ int width, int height) override;\n+\n+ void autoWhiteBalance(uint16_t *R, uint16_t *G, uint16_t *B, int width, int height) override;\n+\n+ void gammaCorrect(uint16_t *R, uint16_t *G, uint16_t *B, float val, int width, int height) override;\n+\n+ float distance(int x, int y, int i, int j);\n+\n+ double gaussian(float x, double sigma);\n+\n+ void bilateralFilter(uint16_t *R, uint16_t *G, uint16_t *B,\n+ int diameter, double sigmaI, double sigmaS,\n+ int width, int height);\n+\n+ void noiseReduction(uint16_t *R, uint16_t *G, uint16_t *B, int width, int height) override;\n+\n+ void compress_10bit_to_8bit(uint16_t *src, uint8_t *dst, int width, int height); \n+\n+ void processing(FrameBuffer *srcBuffer, FrameBuffer *dstBuffer, int width, int height);\n+\n+ int exportBuffers(std::vector> *buffers,\n+ unsigned int count, int width, int height);\n+\n+ void startThreadISP();\n+\n+ void stopThreadISP();\n+\n+ Signal ispCompleted;\n+\n+private:\n+ Thread thread_;\n+};\n+\n+} /* namespace libcamera */\n+\n+#endif /* __LIBCAMERA_SWISP_ISP_H__ */\n\\ No newline at end of file\n","prefixes":["libcamera-devel","RFC","v2","2/4"]}