{"id":19402,"url":"https://patchwork.libcamera.org/api/patches/19402/?format=json","web_url":"https://patchwork.libcamera.org/patch/19402/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240113142218.28063-13-hdegoede@redhat.com>","date":"2024-01-13T14:22:12","name":"[libcamera-devel,v2,12/18] libcamera: ipa: Soft IPA: add a Simple Soft IPA implementation","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"e5a1026ebea5eb58ee471d14a15c5db21f3faec1","submitter":{"id":102,"url":"https://patchwork.libcamera.org/api/people/102/?format=json","name":"Hans de Goede","email":"hdegoede@redhat.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/19402/mbox/","series":[{"id":4142,"url":"https://patchwork.libcamera.org/api/series/4142/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4142","date":"2024-01-13T14:22:00","name":"[libcamera-devel,v2,01/18] libcamera: pipeline: simple: fix size adjustment in validate()","version":2,"mbox":"https://patchwork.libcamera.org/series/4142/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/19402/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/19402/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 34BA8C32BE\n\tfor ;\n\tSat, 13 Jan 2024 14:23:04 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B16D4628BF;\n\tSat, 13 Jan 2024 15:23:03 +0100 (CET)","from us-smtp-delivery-124.mimecast.com\n\t(us-smtp-delivery-124.mimecast.com [170.10.129.124])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2190E61D57\n\tfor ;\n\tSat, 13 Jan 2024 15:23:01 +0100 (CET)","from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com\n\t[66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS\n\t(version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n\tus-mta-597-oe9j5ER5NHyQyi8cg6ZK4Q-1; Sat, 13 Jan 2024 09:22:56 -0500","from smtp.corp.redhat.com\n\t(int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\tkey-exchange X25519 server-signature RSA-PSS (2048 bits)\n\tserver-digest SHA256) (No client certificate requested)\n\tby mimecast-mx02.redhat.com (Postfix) with ESMTPS id 172EC85A589;\n\tSat, 13 Jan 2024 14:22:56 +0000 (UTC)","from localhost.localdomain (unknown [10.39.192.58])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id 258313C27;\n\tSat, 13 Jan 2024 14:22:54 +0000 (UTC)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1705155783;\n\tbh=qYHImG/8hkP2dGeHdAFzhWMnVGByl02Gb/Et/BSOEOM=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=DSmpBCxkuUgUwwkI1xpR/hmibFiKgDs3DaBtGa+XMNsc4U/ePy6eD/1vWkTUpPJo9\n\tnBWpO6f8VcFp4CvH5n7BQF1k/LvrMn+t+r9NsVFfX/T4j458jN8XvaBg5Kbhl8gYk8\n\tiiTj/ojyWXTftq3bnpO5xzhvDnvngjK09KgaeAmZ+cUg0lcYm5bBs8I3HX1fd+YE+N\n\tOBAUB2IbyBaq+zVnP3T6ZwUOW+xDSayPjWa4cq7W38PvyPuL9caAZedDndgQdjVbF7\n\twC786C8rD2sWFYr7/TKlG9Ke9EqxI1CQrQe/+zOG9i36pgv9YtJhMtbvpzqJl+Fimm\n\tv0wOsLL4B2fkA==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n\ts=mimecast20190719; t=1705155780;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tcontent-transfer-encoding:content-transfer-encoding:\n\tin-reply-to:in-reply-to:references:references;\n\tbh=32ULhuHZBgqbGMDZ9lo/6o9mYPPrfHjbHiasMs4eVXs=;\n\tb=G5r1nZuoPFvU8RXMueJM8i3e9hM/60O5RzIXT7RhqlHicebP+nhkNef9DD7xbyQkqdm3QD\n\tYzbYOgYQ8ZKdIzrNCiFAdinQZWU5XMOwflp6njg3Ho6GGIihSmYs/RlaMoQH/YR10inqXj\n\tF/00kRvciaaJCnTLO2Tizc5ubyPw33A="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=redhat.com\n\theader.i=@redhat.com header.b=\"G5r1nZuo\"; \n\tdkim-atps=neutral","X-MC-Unique":"oe9j5ER5NHyQyi8cg6ZK4Q-1","To":"libcamera-devel@lists.libcamera.org,\n\tAndrey Konovalov ","Date":"Sat, 13 Jan 2024 15:22:12 +0100","Message-ID":"<20240113142218.28063-13-hdegoede@redhat.com>","In-Reply-To":"<20240113142218.28063-1-hdegoede@redhat.com>","References":"<20240113142218.28063-1-hdegoede@redhat.com>","MIME-Version":"1.0","X-Scanned-By":"MIMEDefang 3.4.1 on 10.11.54.1","X-Mimecast-Spam-Score":"0","X-Mimecast-Originator":"redhat.com","Content-Transfer-Encoding":"8bit","Content-Type":"text/plain; charset=\"US-ASCII\"; x-default=true","Subject":"[libcamera-devel] [PATCH v2 12/18] libcamera: ipa: Soft IPA: add a\n\tSimple Soft IPA implementation","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Hans de Goede via libcamera-devel ","Reply-To":"Hans de Goede ","Cc":"Maxime Ripard , g.martti@gmail.com,\n\tt.langendam@gmail.com, srinivas.kandagatla@linaro.org,\n\tPavel Machek ,\n\tBryan O'Donoghue , admin@dennisbonke.com","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"From: Andrey Konovalov \n\nAuto exposure/gain and AWB implementation by Dennis, Toon and Martti.\n\nCo-authored-by: Dennis Bonke \nSigned-off-by: Dennis Bonke \nCo-authored-by: Marttico \nSigned-off-by: Marttico \nCo-authored-by: Toon Langendam \nSigned-off-by: Toon Langendam \nSigned-off-by: Andrey Konovalov \nSigned-off-by: Hans de Goede \nTested-by: Bryan O'Donoghue # sc8280xp Lenovo x13s\nTested-by: Pavel Machek \n---\n meson_options.txt | 3 +-\n src/ipa/simple/meson.build | 9 +\n src/ipa/simple/simple/data/meson.build | 9 +\n src/ipa/simple/simple/data/soft.conf | 3 +\n src/ipa/simple/simple/meson.build | 26 +++\n src/ipa/simple/simple/soft_simple.cpp | 273 +++++++++++++++++++++++++\n 6 files changed, 322 insertions(+), 1 deletion(-)\n create mode 100644 src/ipa/simple/simple/data/meson.build\n create mode 100644 src/ipa/simple/simple/data/soft.conf\n create mode 100644 src/ipa/simple/simple/meson.build\n create mode 100644 src/ipa/simple/simple/soft_simple.cpp","diff":"diff --git a/meson_options.txt b/meson_options.txt\nindex 5fdc7be8..8ec08658 100644\n--- a/meson_options.txt\n+++ b/meson_options.txt\n@@ -27,7 +27,7 @@ option('gstreamer',\n \n option('ipas',\n type : 'array',\n- choices : ['ipu3', 'rkisp1', 'rpi/vc4', 'vimc'],\n+ choices : ['ipu3', 'rkisp1', 'rpi/vc4', 'simple/simple', 'vimc'],\n description : 'Select which IPA modules to build')\n \n option('lc-compliance',\n@@ -46,6 +46,7 @@ option('pipelines',\n 'rkisp1',\n 'rpi/vc4',\n 'simple',\n+ 'simple/simple',\n 'uvcvideo',\n 'vimc'\n ],\ndiff --git a/src/ipa/simple/meson.build b/src/ipa/simple/meson.build\nindex 9688bbdb..14be5dc2 100644\n--- a/src/ipa/simple/meson.build\n+++ b/src/ipa/simple/meson.build\n@@ -1,3 +1,12 @@\n # SPDX-License-Identifier: CC0-1.0\n \n subdir('common')\n+\n+foreach pipeline : pipelines\n+ pipeline = pipeline.split('/')\n+ if pipeline.length() < 2 or pipeline[0] != 'simple'\n+ continue\n+ endif\n+\n+ subdir(pipeline[1])\n+endforeach\ndiff --git a/src/ipa/simple/simple/data/meson.build b/src/ipa/simple/simple/data/meson.build\nnew file mode 100644\nindex 00000000..33548cc6\n--- /dev/null\n+++ b/src/ipa/simple/simple/data/meson.build\n@@ -0,0 +1,9 @@\n+# SPDX-License-Identifier: CC0-1.0\n+\n+conf_files = files([\n+ 'soft.conf',\n+])\n+\n+install_data(conf_files,\n+ install_dir : ipa_data_dir / 'soft',\n+ install_tag : 'runtime')\ndiff --git a/src/ipa/simple/simple/data/soft.conf b/src/ipa/simple/simple/data/soft.conf\nnew file mode 100644\nindex 00000000..0c70e7c0\n--- /dev/null\n+++ b/src/ipa/simple/simple/data/soft.conf\n@@ -0,0 +1,3 @@\n+# SPDX-License-Identifier: LGPL-2.1-or-later\n+#\n+# Dummy configuration file for the soft IPA.\ndiff --git a/src/ipa/simple/simple/meson.build b/src/ipa/simple/simple/meson.build\nnew file mode 100644\nindex 00000000..8b5d76b5\n--- /dev/null\n+++ b/src/ipa/simple/simple/meson.build\n@@ -0,0 +1,26 @@\n+# SPDX-License-Identifier: CC0-1.0\n+\n+ipa_name = 'ipa_soft_simple'\n+\n+mod = shared_module(ipa_name,\n+ ['soft_simple.cpp', libcamera_generated_ipa_headers],\n+ name_prefix : '',\n+ include_directories : [ipa_includes, libipa_includes, '..'],\n+ dependencies : libcamera_private,\n+ link_with : libipa,\n+ link_whole : soft_ipa_common_lib,\n+ install : true,\n+ install_dir : ipa_install_dir)\n+\n+if ipa_sign_module\n+ custom_target(ipa_name + '.so.sign',\n+ input : mod,\n+ output : ipa_name + '.so.sign',\n+ command : [ipa_sign, ipa_priv_key, '@INPUT@', '@OUTPUT@'],\n+ install : false,\n+ build_by_default : true)\n+endif\n+\n+subdir('data')\n+\n+ipa_names += ipa_name\ndiff --git a/src/ipa/simple/simple/soft_simple.cpp b/src/ipa/simple/simple/soft_simple.cpp\nnew file mode 100644\nindex 00000000..93fc1545\n--- /dev/null\n+++ b/src/ipa/simple/simple/soft_simple.cpp\n@@ -0,0 +1,273 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2023, Linaro Ltd\n+ *\n+ * soft_simple.cpp - Simple Software Image Processing Algorithm module\n+ */\n+\n+#include \n+\n+#include \n+#include \n+\n+#include \n+\n+#include \n+#include \n+\n+#include \"libcamera/internal/camera_sensor.h\"\n+#include \"libcamera/internal/software_isp/debayer_params.h\"\n+#include \"libcamera/internal/software_isp/swisp_stats.h\"\n+\n+#include \"common/soft_base.h\"\n+\n+#define EXPOSURE_OPTIMAL_VALUE 2.5\n+#define EXPOSURE_SATISFACTORY_OFFSET 0.2\n+\n+namespace libcamera {\n+\n+LOG_DECLARE_CATEGORY(IPASoft)\n+\n+namespace ipa::soft {\n+\n+class IPASoftSimple final : public IPASoftBase\n+{\n+public:\n+\tIPASoftSimple()\n+\t\t: IPASoftBase(), ignore_updates_(0)\n+\t{\n+\t}\n+\n+\t~IPASoftSimple()\n+\t{\n+\t\tif (stats_)\n+\t\t\tmunmap(stats_, sizeof(SwIspStats));\n+\t\tif (params_)\n+\t\t\tmunmap(params_, sizeof(DebayerParams));\n+\t}\n+\n+\tint platformInit(const ControlInfoMap &sensorInfoMap) override;\n+\tint platformConfigure(const ControlInfoMap &sensorInfoMap) override;\n+\tint platformStart() override;\n+\tvoid platformStop() override;\n+\tvoid platformProcessStats(const ControlList &sensorControls) override;\n+\n+private:\n+\tvoid update_exposure(double exposuremsv);\n+\n+\tDebayerParams *params_;\n+\tSwIspStats *stats_;\n+\tint exposure_min_, exposure_max_;\n+\tint again_min_, again_max_;\n+\tint again_, exposure_;\n+\tint ignore_updates_;\n+};\n+\n+int IPASoftSimple::platformInit(const ControlInfoMap &sensorInfoMap)\n+{\n+\tparams_ = static_cast(mmap(nullptr, sizeof(DebayerParams),\n+\t\t\t\t\t\t PROT_WRITE, MAP_SHARED,\n+\t\t\t\t\t\t fdParams_.get(), 0));\n+\tif (!params_) {\n+\t\tLOG(IPASoft, Error) << \"Unable to map Parameters\";\n+\t\treturn -ENODEV;\n+\t}\n+\n+\tstats_ = static_cast(mmap(nullptr, sizeof(SwIspStats),\n+\t\t\t\t\t\tPROT_READ, MAP_SHARED,\n+\t\t\t\t\t\tfdStats_.get(), 0));\n+\tif (!stats_) {\n+\t\tLOG(IPASoft, Error) << \"Unable to map Statistics\";\n+\t\treturn -ENODEV;\n+\t}\n+\n+\tif (sensorInfoMap.find(V4L2_CID_EXPOSURE) == sensorInfoMap.end()) {\n+\t\tLOG(IPASoft, Error) << \"Don't have exposure control\";\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tif (sensorInfoMap.find(V4L2_CID_ANALOGUE_GAIN) == sensorInfoMap.end()) {\n+\t\tLOG(IPASoft, Error) << \"Don't have gain control\";\n+\t\treturn -EINVAL;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+int IPASoftSimple::platformConfigure(const ControlInfoMap &sensorInfoMap)\n+{\n+\tconst ControlInfo &exposure_info = sensorInfoMap.find(V4L2_CID_EXPOSURE)->second;\n+\tconst ControlInfo &gain_info = sensorInfoMap.find(V4L2_CID_ANALOGUE_GAIN)->second;\n+\n+\texposure_min_ = exposure_info.min().get();\n+\tif (!exposure_min_) {\n+\t\tLOG(IPASoft, Warning) << \"Minimum exposure is zero, that can't be linear\";\n+\t\texposure_min_ = 1;\n+\t}\n+\texposure_max_ = exposure_info.max().get();\n+\tagain_min_ = gain_info.min().get();\n+\tif (!again_min_) {\n+\t\tLOG(IPASoft, Warning) << \"Minimum gain is zero, that can't be linear\";\n+\t\tagain_min_ = 100;\n+\t}\n+\tagain_max_ = gain_info.max().get();\n+\n+\tLOG(IPASoft, Info) << \"Exposure \" << exposure_min_ << \"-\" << exposure_max_\n+\t\t\t << \", gain \" << again_min_ << \"-\" << again_max_;\n+\n+\treturn 0;\n+}\n+\n+int IPASoftSimple::platformStart()\n+{\n+\treturn 0;\n+}\n+\n+void IPASoftSimple::platformStop()\n+{\n+}\n+\n+void IPASoftSimple::platformProcessStats(const ControlList &sensorControls)\n+{\n+\t/*\n+\t * Calculate red and blue gains for AWB.\n+\t * Clamp max gain at 4.0, this also avoids 0 division.\n+\t */\n+\tif (stats_->sumR_ <= stats_->sumG_ / 4)\n+\t\tparams_->gainR = 1024;\n+\telse\n+\t\tparams_->gainR = 256 * stats_->sumG_ / stats_->sumR_;\n+\n+\tif (stats_->sumB_ <= stats_->sumG_ / 4)\n+\t\tparams_->gainB = 1024;\n+\telse\n+\t\tparams_->gainB = 256 * stats_->sumG_ / stats_->sumB_;\n+\n+\t/* Green gain and gamma values are fixed */\n+\tparams_->gainG = 256;\n+\tparams_->gamma = 0.5;\n+\n+\tsetIspParams.emit(0);\n+\n+\t/*\n+\t * AE / AGC, use 2 frames delay to make sure that the exposure and\n+\t * the gain set have applied to the camera sensor.\n+\t */\n+\tif (ignore_updates_ > 0) {\n+\t\t--ignore_updates_;\n+\t\treturn;\n+\t}\n+\n+\tunsigned int denom = 0;\n+\tunsigned int num = 0;\n+\tunsigned int y_histogramSmall[5] = {};\n+\n+\tfor (int i = 0; i < 16; i++)\n+\t\ty_histogramSmall[(i - i / 8) / 3] += stats_->y_histogram[i];\n+\n+\tfor (int i = 0; i < 5; i++) {\n+\t\tLOG(IPASoft, Debug) << i << \": \" << y_histogramSmall[i];\n+\t\tdenom += y_histogramSmall[i];\n+\t\tnum += y_histogramSmall[i] * (i + 1);\n+\t}\n+\n+\tfloat exposuremsv = (float)num / denom;\n+\n+\t/* sanity check */\n+\tif (!sensorControls.contains(V4L2_CID_EXPOSURE) ||\n+\t !sensorControls.contains(V4L2_CID_ANALOGUE_GAIN)) {\n+\t\tLOG(IPASoft, Error) << \"Control(s) missing\";\n+\t\treturn;\n+\t}\n+\n+\tControlList ctrls(sensorControls);\n+\n+\texposure_ = ctrls.get(V4L2_CID_EXPOSURE).get();\n+\tagain_ = ctrls.get(V4L2_CID_ANALOGUE_GAIN).get();\n+\n+\tupdate_exposure(exposuremsv);\n+\n+\tctrls.set(V4L2_CID_EXPOSURE, exposure_);\n+\tctrls.set(V4L2_CID_ANALOGUE_GAIN, again_);\n+\n+\tignore_updates_ = 2;\n+\n+\tsetSensorControls.emit(ctrls);\n+\n+\tLOG(IPASoft, Debug) << \"exposuremsv \" << exposuremsv\n+\t\t\t << \" exp \" << exposure_ << \" again \" << again_\n+\t\t\t << \" gain R/B \" << params_->gainR << \"/\" << params_->gainB;\n+}\n+\n+/* DENOMINATOR of 10 gives ~10% increment/decrement; DENOMINATOR of 5 - about ~20% */\n+#define DENOMINATOR 10\n+#define UP_NUMERATOR (DENOMINATOR + 1)\n+#define DOWN_NUMERATOR (DENOMINATOR - 1)\n+\n+void IPASoftSimple::update_exposure(double exposuremsv)\n+{\n+\tint next;\n+\n+\tif (exposuremsv < EXPOSURE_OPTIMAL_VALUE - EXPOSURE_SATISFACTORY_OFFSET) {\n+\t\tnext = exposure_ * UP_NUMERATOR / DENOMINATOR;\n+\t\tif (next - exposure_ < 1)\n+\t\t\texposure_ += 1;\n+\t\telse\n+\t\t\texposure_ = next;\n+\t\tif (exposure_ >= exposure_max_) {\n+\t\t\tnext = again_ * UP_NUMERATOR / DENOMINATOR;\n+\t\t\tif (next - again_ < 1)\n+\t\t\t\tagain_ += 1;\n+\t\t\telse\n+\t\t\t\tagain_ = next;\n+\t\t}\n+\t}\n+\n+\tif (exposuremsv > EXPOSURE_OPTIMAL_VALUE + EXPOSURE_SATISFACTORY_OFFSET) {\n+\t\tif (exposure_ == exposure_max_ && again_ != again_min_) {\n+\t\t\tnext = again_ * DOWN_NUMERATOR / DENOMINATOR;\n+\t\t\tif (again_ - next < 1)\n+\t\t\t\tagain_ -= 1;\n+\t\t\telse\n+\t\t\t\tagain_ = next;\n+\t\t} else {\n+\t\t\tnext = exposure_ * DOWN_NUMERATOR / DENOMINATOR;\n+\t\t\tif (exposure_ - next < 1)\n+\t\t\t\texposure_ -= 1;\n+\t\t\telse\n+\t\t\t\texposure_ = next;\n+\t\t}\n+\t}\n+\n+\tif (exposure_ > exposure_max_)\n+\t\texposure_ = exposure_max_;\n+\telse if (exposure_ < exposure_min_)\n+\t\texposure_ = exposure_min_;\n+\n+\tif (again_ > again_max_)\n+\t\tagain_ = again_max_;\n+\telse if (again_ < again_min_)\n+\t\tagain_ = again_min_;\n+}\n+\n+} /* namespace ipa::soft */\n+\n+/*\n+ * External IPA module interface\n+ */\n+extern \"C\" {\n+const struct IPAModuleInfo ipaModuleInfo = {\n+\tIPA_MODULE_API_VERSION,\n+\t0,\n+\t\"SimplePipelineHandler\",\n+\t\"soft/simple\",\n+};\n+\n+IPAInterface *ipaCreate()\n+{\n+\treturn new ipa::soft::IPASoftSimple();\n+}\n+\n+} /* extern \"C\" */\n+\n+} /* namespace libcamera */\n","prefixes":["libcamera-devel","v2","12/18"]}