{"id":10772,"url":"https://patchwork.libcamera.org/api/patches/10772/?format=json","web_url":"https://patchwork.libcamera.org/patch/10772/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20201229160318.77536-7-niklas.soderlund@ragnatech.se>","date":"2020-12-29T16:03:13","name":"[libcamera-devel,v2,06/11] libcamera: ipa: ipu3: Add an IPA skeleton for the IPU3 pipeline","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"fec679455addec0e7e54db1c3f8936a463a414a2","submitter":{"id":5,"url":"https://patchwork.libcamera.org/api/people/5/?format=json","name":"Niklas Söderlund","email":"niklas.soderlund@ragnatech.se"},"delegate":{"id":16,"url":"https://patchwork.libcamera.org/api/users/16/?format=json","username":"neg","first_name":"Niklas","last_name":"Söderlund","email":"niklas.soderlund@ragnatech.se"},"mbox":"https://patchwork.libcamera.org/patch/10772/mbox/","series":[{"id":1559,"url":"https://patchwork.libcamera.org/api/series/1559/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=1559","date":"2020-12-29T16:03:07","name":"libcamera: ipu3: Attach to an skeleton IPA","version":2,"mbox":"https://patchwork.libcamera.org/series/1559/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/10772/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/10772/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A8F27C0F1A\n\tfor ;\n\tTue, 29 Dec 2020 16:04:11 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8010462009;\n\tTue, 29 Dec 2020 17:04:11 +0100 (CET)","from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net\n\t[195.74.38.228])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 726A9615D5\n\tfor ;\n\tTue, 29 Dec 2020 17:04:09 +0100 (CET)","from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de\n\t[79.202.36.88])\n\tby bin-vsp-out-01.atm.binero.net (Halon) with ESMTPA\n\tid 7ad0eff3-49ef-11eb-a542-005056917a89;\n\tTue, 29 Dec 2020 17:04:07 +0100 (CET)"],"X-Halon-ID":"7ad0eff3-49ef-11eb-a542-005056917a89","Authorized-sender":"niklas.soderlund@fsdn.se","From":"=?utf-8?q?Niklas_S=C3=B6derlund?= ","To":"libcamera-devel@lists.libcamera.org","Date":"Tue, 29 Dec 2020 17:03:13 +0100","Message-Id":"<20201229160318.77536-7-niklas.soderlund@ragnatech.se>","X-Mailer":"git-send-email 2.29.2","In-Reply-To":"<20201229160318.77536-1-niklas.soderlund@ragnatech.se>","References":"<20201229160318.77536-1-niklas.soderlund@ragnatech.se>","MIME-Version":"1.0","Subject":"[libcamera-devel] [PATCH v2 06/11] libcamera: ipa: ipu3: Add an IPA\n\tskeleton for the IPU3 pipeline","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Add an empty IPA skeleton for the IPU3 pipeline. The skeleton IPA\nhandles the flow of parameter and statistic buffers but does not read or\nwrite anything in the buffers. It also allows the IPA to set sensor\ncontrols but does not implement any logic to set optimal values and\ninstead sets the V4L2 exposure and gain controls to max and keeps them\nat that setting.\n\nThis IPA is meant as a base to allow the pipeline handler to be wired up\nto an IPA. The image algorithms can then later be added to the IPA\nindependently from also having to add plumbing to the pipeline handler.\n\nSigned-off-by: Niklas Söderlund \nAcked-by: Kieran Bingham \n---\n* Changes since v1\n- Updated commit message.\n- Rename IPU3_IPA_EVENT_PARSE_STAT to IPU3_IPA_EVENT_PARSE_STAT.\n- Sort and drop unused headers.\n- Fix style issue in meson.build and add dependency on libatomic.\n- Switch to MappedFrameBuffer interface.\n- Add result of IPA configuration status.\n---\n include/libcamera/ipa/ipu3.h | 23 ++++\n src/ipa/ipu3/ipu3.cpp | 236 +++++++++++++++++++++++++++++++++++\n src/ipa/ipu3/meson.build | 21 ++++\n src/ipa/meson.build | 2 +-\n 4 files changed, 281 insertions(+), 1 deletion(-)\n create mode 100644 include/libcamera/ipa/ipu3.h\n create mode 100644 src/ipa/ipu3/ipu3.cpp\n create mode 100644 src/ipa/ipu3/meson.build","diff":"diff --git a/include/libcamera/ipa/ipu3.h b/include/libcamera/ipa/ipu3.h\nnew file mode 100644\nindex 0000000000000000..1bcf3591f9fe36a9\n--- /dev/null\n+++ b/include/libcamera/ipa/ipu3.h\n@@ -0,0 +1,23 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2020, Google Inc.\n+ *\n+ * ipu3.h - Image Processing Algorithm interface for IPU3\n+ */\n+#ifndef __LIBCAMERA_IPA_INTERFACE_IPU3_H__\n+#define __LIBCAMERA_IPA_INTERFACE_IPU3_H__\n+\n+#ifndef __DOXYGEN__\n+\n+enum IPU3Operations {\n+\tIPU3_IPA_STATUS_CONFIGURATION = 1,\n+\tIPU3_IPA_ACTION_SET_SENSOR_CONTROLS = 2,\n+\tIPU3_IPA_ACTION_PARAM_FILLED = 3,\n+\tIPU3_IPA_ACTION_METADATA_READY = 4,\n+\tIPU3_IPA_EVENT_STAT_READY = 5,\n+\tIPU3_IPA_EVENT_FILL_PARAMS = 6,\n+};\n+\n+#endif /* __DOXYGEN__ */\n+\n+#endif /* __LIBCAMERA_IPA_INTERFACE_IPU3_H__ */\ndiff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp\nnew file mode 100644\nindex 0000000000000000..809a3eec398a8e57\n--- /dev/null\n+++ b/src/ipa/ipu3/ipu3.cpp\n@@ -0,0 +1,236 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2020, Google Inc.\n+ *\n+ * ipu3.cpp - IPU3 Image Processing Algorithms\n+ */\n+\n+#include \n+\n+#include \n+\n+#include \n+#include \n+\n+#include \n+#include \n+#include \n+#include \n+#include \n+\n+#include \n+\n+#include \"libcamera/internal/buffer.h\"\n+#include \"libcamera/internal/log.h\"\n+\n+namespace libcamera {\n+\n+LOG_DEFINE_CATEGORY(IPAIPU3)\n+\n+class IPAIPU3 : public IPAInterface\n+{\n+public:\n+\tint init([[maybe_unused]] const IPASettings &settings) override\n+\t{\n+\t\treturn 0;\n+\t}\n+\tint start([[maybe_unused]] const IPAOperationData &data,\n+\t\t [[maybe_unused]] IPAOperationData *result) override { return 0; }\n+\tvoid stop() override {}\n+\n+\tvoid configure(const CameraSensorInfo &info,\n+\t\t const std::map &streamConfig,\n+\t\t const std::map &entityControls,\n+\t\t const IPAOperationData &ipaConfig,\n+\t\t IPAOperationData *response) override;\n+\tvoid mapBuffers(const std::vector &buffers) override;\n+\tvoid unmapBuffers(const std::vector &ids) override;\n+\tvoid processEvent(const IPAOperationData &event) override;\n+\n+private:\n+\tvoid fillParams(unsigned int frame, ipu3_uapi_params *params,\n+\t\t\tconst ControlList &controls);\n+\n+\tvoid parseStatistics(unsigned int frame,\n+\t\t\t const ipu3_uapi_stats_3a *stats);\n+\n+\tvoid setControls(unsigned int frame);\n+\n+\tstd::map buffers_;\n+\n+\tControlInfoMap ctrls_;\n+\n+\t/* Camera sensor controls. */\n+\tuint32_t exposure_;\n+\tuint32_t minExposure_;\n+\tuint32_t maxExposure_;\n+\tuint32_t gain_;\n+\tuint32_t minGain_;\n+\tuint32_t maxGain_;\n+};\n+\n+void IPAIPU3::configure([[maybe_unused]] const CameraSensorInfo &info,\n+\t\t\t[[maybe_unused]] const std::map &streamConfig,\n+\t\t\tconst std::map &entityControls,\n+\t\t\t[[maybe_unused]] const IPAOperationData &ipaConfig,\n+\t\t\t[[maybe_unused]] IPAOperationData *result)\n+{\n+\tresult->operation = IPU3_IPA_STATUS_CONFIGURATION;\n+\n+\tif (entityControls.empty())\n+\t\treturn;\n+\n+\tctrls_ = entityControls.at(0);\n+\n+\tconst auto itExp = ctrls_.find(V4L2_CID_EXPOSURE);\n+\tif (itExp == ctrls_.end()) {\n+\t\tLOG(IPAIPU3, Error) << \"Can't find exposure control\";\n+\t\treturn;\n+\t}\n+\n+\tconst auto itGain = ctrls_.find(V4L2_CID_ANALOGUE_GAIN);\n+\tif (itGain == ctrls_.end()) {\n+\t\tLOG(IPAIPU3, Error) << \"Can't find gain control\";\n+\t\treturn;\n+\t}\n+\n+\tminExposure_ = std::max(itExp->second.min().get(), 1);\n+\tmaxExposure_ = itExp->second.max().get();\n+\texposure_ = maxExposure_;\n+\n+\tminGain_ = std::max(itGain->second.min().get(), 1);\n+\tmaxGain_ = itGain->second.max().get();\n+\tgain_ = maxGain_;\n+\n+\tsetControls(0);\n+\n+\tresult->data.push_back(1);\n+}\n+\n+void IPAIPU3::mapBuffers(const std::vector &buffers)\n+{\n+\tfor (const IPABuffer &buffer : buffers) {\n+\t\tconst FrameBuffer fb(buffer.planes);\n+\t\tbuffers_.emplace(buffer.id,\n+\t\t\t\t MappedFrameBuffer(&fb, PROT_READ | PROT_WRITE));\n+\t}\n+}\n+\n+void IPAIPU3::unmapBuffers(const std::vector &ids)\n+{\n+\tfor (unsigned int id : ids) {\n+\t\tauto it = buffers_.find(id);\n+\t\tif (it == buffers_.end())\n+\t\t\tcontinue;\n+\n+\t\tbuffers_.erase(id);\n+\t}\n+}\n+\n+void IPAIPU3::processEvent(const IPAOperationData &event)\n+{\n+\tswitch (event.operation) {\n+\tcase IPU3_IPA_EVENT_STAT_READY: {\n+\t\tunsigned int frame = event.data[0];\n+\t\tunsigned int bufferId = event.data[1];\n+\n+\t\tauto it = buffers_.find(bufferId);\n+\t\tif (it == buffers_.end()) {\n+\t\t\tLOG(IPAIPU3, Error) << \"Could not find stats buffer!\";\n+\t\t\treturn;\n+\t\t}\n+\n+\t\tSpan mem = it->second.maps()[0];\n+\t\tconst ipu3_uapi_stats_3a *stats =\n+\t\t\treinterpret_cast(mem.data());\n+\n+\t\tparseStatistics(frame, stats);\n+\t\tbreak;\n+\t}\n+\tcase IPU3_IPA_EVENT_FILL_PARAMS: {\n+\t\tunsigned int frame = event.data[0];\n+\t\tunsigned int bufferId = event.data[1];\n+\n+\t\tauto it = buffers_.find(bufferId);\n+\t\tif (it == buffers_.end()) {\n+\t\t\tLOG(IPAIPU3, Error) << \"Could not find param buffer!\";\n+\t\t\treturn;\n+\t\t}\n+\n+\t\tSpan mem = it->second.maps()[0];\n+\t\tipu3_uapi_params *params =\n+\t\t\treinterpret_cast(mem.data());\n+\n+\t\tfillParams(frame, params, event.controls[0]);\n+\t\tbreak;\n+\t}\n+\tdefault:\n+\t\tLOG(IPAIPU3, Error) << \"Unknown event \" << event.operation;\n+\t\tbreak;\n+\t}\n+}\n+\n+void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params,\n+\t\t\t [[maybe_unused]] const ControlList &controls)\n+{\n+\t/* Prepare parameters buffer. */\n+\tmemset(params, 0, sizeof(*params));\n+\n+\t/* \\todo Fill in parameters buffer. */\n+\n+\tIPAOperationData op;\n+\top.operation = IPU3_IPA_ACTION_PARAM_FILLED;\n+\n+\tqueueFrameAction.emit(frame, op);\n+\n+\t/* \\todo Calculate new values for exposure_ and gain_. */\n+\tsetControls(frame);\n+}\n+\n+void IPAIPU3::parseStatistics(unsigned int frame,\n+\t\t\t [[maybe_unused]] const ipu3_uapi_stats_3a *stats)\n+{\n+\tControlList ctrls(controls::controls);\n+\n+\t/* \\todo React to statistics and update internal state machine. */\n+\t/* \\todo Add meta-data information to ctrls. */\n+\n+\tIPAOperationData op;\n+\top.operation = IPU3_IPA_ACTION_METADATA_READY;\n+\top.controls.push_back(ctrls);\n+\n+\tqueueFrameAction.emit(frame, op);\n+}\n+\n+void IPAIPU3::setControls(unsigned int frame)\n+{\n+\tIPAOperationData op;\n+\top.operation = IPU3_IPA_ACTION_SET_SENSOR_CONTROLS;\n+\n+\tControlList ctrls(ctrls_);\n+\tctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_));\n+\tctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_));\n+\top.controls.push_back(ctrls);\n+\n+\tqueueFrameAction.emit(frame, op);\n+}\n+\n+/*\n+ * External IPA module interface\n+ */\n+\n+extern \"C\" {\n+const struct IPAModuleInfo ipaModuleInfo = {\n+\tIPA_MODULE_API_VERSION,\n+\t1,\n+\t\"PipelineHandlerIPU3\",\n+\t\"ipu3\",\n+};\n+\n+struct ipa_context *ipaCreate()\n+{\n+\treturn new IPAInterfaceWrapper(std::make_unique());\n+}\n+}\n+\n+} /* namespace libcamera */\ndiff --git a/src/ipa/ipu3/meson.build b/src/ipa/ipu3/meson.build\nnew file mode 100644\nindex 0000000000000000..444c82453eac42ff\n--- /dev/null\n+++ b/src/ipa/ipu3/meson.build\n@@ -0,0 +1,21 @@\n+# SPDX-License-Identifier: CC0-1.0\n+\n+ipa_name = 'ipa_ipu3'\n+\n+mod = shared_module(ipa_name,\n+ 'ipu3.cpp',\n+ name_prefix : '',\n+ include_directories : [ ipa_includes, libipa_includes ],\n+ dependencies : [ libatomic, libcamera_dep ],\n+ link_with : libipa,\n+ install : true,\n+ install_dir : ipa_install_dir)\n+\n+if ipa_sign_module\n+ custom_target(ipa_name + '.so.sign',\n+ input : mod,\n+ output : ipa_name + '.so.sign',\n+ command : [ ipa_sign, ipa_priv_key, '@INPUT@', '@OUTPUT@' ],\n+ install : false,\n+ build_by_default : true)\n+endif\ndiff --git a/src/ipa/meson.build b/src/ipa/meson.build\nindex 5a5de267c1477d24..9d623f227a1f9feb 100644\n--- a/src/ipa/meson.build\n+++ b/src/ipa/meson.build\n@@ -19,7 +19,7 @@ subdir('libipa')\n \n ipa_sign = files('ipa-sign.sh')\n \n-ipas = ['raspberrypi', 'rkisp1', 'vimc']\n+ipas = ['ipu3', 'raspberrypi', 'rkisp1', 'vimc']\n ipa_names = []\n \n foreach pipeline : get_option('pipelines')\n","prefixes":["libcamera-devel","v2","06/11"]}