{"id":19324,"url":"https://patchwork.libcamera.org/api/1.1/patches/19324/?format=json","web_url":"https://patchwork.libcamera.org/patch/19324/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20231214121350.206015-10-hdegoede@redhat.com>","date":"2023-12-14T12:13:47","name":"[libcamera-devel,09/11] libcamera: software_isp: add Simple SoftwareIsp implementation","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"82f69ad47281aeffd48f62274412be8a209f9577","submitter":{"id":102,"url":"https://patchwork.libcamera.org/api/1.1/people/102/?format=json","name":"Hans de Goede","email":"hdegoede@redhat.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/19324/mbox/","series":[{"id":4117,"url":"https://patchwork.libcamera.org/api/1.1/series/4117/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4117","date":"2023-12-14T12:13:38","name":"libcamera: introduce Software ISP and Software IPA","version":1,"mbox":"https://patchwork.libcamera.org/series/4117/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/19324/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/19324/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8F67BC3292\n\tfor ;\n\tThu, 14 Dec 2023 12:14:12 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 4611F62B61;\n\tThu, 14 Dec 2023 13:14:12 +0100 (CET)","from us-smtp-delivery-124.mimecast.com\n\t(us-smtp-delivery-124.mimecast.com [170.10.129.124])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C60AF62B61\n\tfor ;\n\tThu, 14 Dec 2023 13:14:10 +0100 (CET)","from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73])\n\tby relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3,\n\tcipher=TLS_AES_256_GCM_SHA384) id us-mta-173-vGhwkzODNaWFO5mdrr15ZA-1;\n\tThu, 14 Dec 2023 07:14:06 -0500","from smtp.corp.redhat.com\n\t(int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\tkey-exchange X25519 server-signature RSA-PSS (2048 bits)\n\tserver-digest SHA256) (No client certificate requested)\n\tby mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0D7043C23FE8;\n\tThu, 14 Dec 2023 12:14:06 +0000 (UTC)","from shalem.redhat.com (unknown [10.39.194.72])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id BC30D1121306;\n\tThu, 14 Dec 2023 12:14:04 +0000 (UTC)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1702556052;\n\tbh=AzPbf99kmRo4L7Zn43l8oEDzEJoEC/Bv9o1ZyvIB0LY=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=3KobMLIQepuMLByMytZyjHMV6AF5ieDNdhtyIruya5OV3bRHw9tf4z1R0suUBWDHz\n\t/k2H8I5SyA0Wngpg5LL4mFdwXq6WFz8WaS5g1Zox+s3R0tez0dC9BUkP5/3dlA95/6\n\tIi5kFItYCjtcZkobD4kO4MowsVVNrmm7RepriQp9/MwWszTRuk1ekNx7pQsgLX4o/w\n\tHLS4z84aQzoMP2nynX+AJOvJrmMahudqOx+EnyjctgiKyl8RELPZqD30AbRdGR/wSG\n\tLS+2oEUjCNMhXNcKRnKNSVlefGJ4hirvOxVi3HYl2uQ5pWS27hsA4eXD9s5hdCJPXW\n\t3GzBdh/VwG2xw==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n\ts=mimecast20190719; t=1702556049;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tcontent-transfer-encoding:content-transfer-encoding:\n\tin-reply-to:in-reply-to:references:references;\n\tbh=yDVEm3pYLfHh77GPI/fz95A+9YFc6sPF3hsMkf+AWCM=;\n\tb=LoQiLexx2KsQ9baRu8cVFT0wo5SC+SMECWXq0xKYMJ5t9XiVdIkACMXjsvj6XpB+dI1vu9\n\t9UtdUa0eqE8hEIgFh2xOBK7uFPp6ApQ9FPcmonZ2wXMh5RAg7TZop/YokXOkl3egeqtZgJ\n\tOkcBdCa72wC9blTFZn43wFLXyZOOMQI="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=redhat.com\n\theader.i=@redhat.com header.b=\"LoQiLexx\"; \n\tdkim-atps=neutral","X-MC-Unique":"vGhwkzODNaWFO5mdrr15ZA-1","To":"libcamera-devel@lists.libcamera.org,\n\tAndrey Konovalov ","Date":"Thu, 14 Dec 2023 13:13:47 +0100","Message-ID":"<20231214121350.206015-10-hdegoede@redhat.com>","In-Reply-To":"<20231214121350.206015-1-hdegoede@redhat.com>","References":"<20231214121350.206015-1-hdegoede@redhat.com>","MIME-Version":"1.0","X-Scanned-By":"MIMEDefang 3.4.1 on 10.11.54.3","X-Mimecast-Spam-Score":"0","X-Mimecast-Originator":"redhat.com","Content-Transfer-Encoding":"8bit","Content-Type":"text/plain; charset=\"US-ASCII\"; x-default=true","Subject":"[libcamera-devel] [PATCH 09/11] libcamera: software_isp: add Simple\n\tSoftwareIsp implementation","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Hans de Goede via libcamera-devel ","Reply-To":"Hans de Goede ","Cc":"Maxime Ripard , g.martti@gmail.com,\n\tt.langendam@gmail.com, srinivas.kandagatla@linaro.org,\n\tBryan O'Donoghue , admin@dennisbonke.com","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"From: Andrey Konovalov \n\nThe implementation of SoftwareIsp handles creation of Soft IPA\nand interactions with it, so that the pipeline handler wouldn't\nneed to care about the Soft IPA.\n\nSigned-off-by: Andrey Konovalov \nCo-authored-by: Hans de Goede \nSigned-off-by: Hans de Goede \n---\n include/libcamera/internal/meson.build | 1 +\n .../internal/software_isp/meson.build | 1 +\n .../internal/software_isp/swisp_simple.h | 67 ++++++\n src/libcamera/meson.build | 1 +\n src/libcamera/software_isp/meson.build | 19 ++\n src/libcamera/software_isp/swisp_simple.cpp | 219 ++++++++++++++++++\n 6 files changed, 308 insertions(+)\n create mode 100644 include/libcamera/internal/software_isp/swisp_simple.h\n create mode 100644 src/libcamera/software_isp/swisp_simple.cpp","diff":"diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build\nindex b780777c..eeae801c 100644\n--- a/include/libcamera/internal/meson.build\n+++ b/include/libcamera/internal/meson.build\n@@ -50,3 +50,4 @@ libcamera_internal_headers = files([\n ])\n \n subdir('converter')\n+subdir('software_isp')\ndiff --git a/include/libcamera/internal/software_isp/meson.build b/include/libcamera/internal/software_isp/meson.build\nindex b5a0d737..cf21ace5 100644\n--- a/include/libcamera/internal/software_isp/meson.build\n+++ b/include/libcamera/internal/software_isp/meson.build\n@@ -4,6 +4,7 @@ libcamera_internal_headers += files([\n 'debayer.h',\n 'debayer_cpu.h',\n 'debayer_params.h',\n+ 'swisp_simple.h',\n 'swisp_stats.h',\n 'swstats.h',\n 'swstats_cpu.h',\ndiff --git a/include/libcamera/internal/software_isp/swisp_simple.h b/include/libcamera/internal/software_isp/swisp_simple.h\nnew file mode 100644\nindex 00000000..ee66e193\n--- /dev/null\n+++ b/include/libcamera/internal/software_isp/swisp_simple.h\n@@ -0,0 +1,67 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2023, Linaro Ltd\n+ *\n+ * swisp_simple.h - Simple software ISP implementation\n+ */\n+\n+#pragma once\n+\n+#include \n+#include \n+\n+#include \n+\n+#include \n+#include \n+\n+#include \"libcamera/internal/software_isp.h\"\n+#include \"libcamera/internal/software_isp/debayer_cpu.h\"\n+\n+namespace libcamera {\n+\n+class SwIspSimple : public SoftwareIsp\n+{\n+public:\n+\tSwIspSimple(PipelineHandler *pipe, const ControlInfoMap &sensorControls);\n+\t~SwIspSimple() {}\n+\n+\tint loadConfiguration([[maybe_unused]] const std::string &filename) { return 0; }\n+\tbool isValid() const;\n+\n+\tstd::vector formats(PixelFormat input);\n+\tSizeRange sizes(PixelFormat inputFormat, const Size &inputSize);\n+\n+\tstd::tuple\n+\tstrideAndFrameSize(const PixelFormat &outputFormat, const Size &size);\n+\n+\tint configure(const StreamConfiguration &inputCfg,\n+\t\t const std::vector> &outputCfgs);\n+\tint exportBuffers(unsigned int output, unsigned int count,\n+\t\t\t std::vector> *buffers);\n+\tvoid processStats(const ControlList &sensorControls);\n+\n+\tint start();\n+\tvoid stop();\n+\n+\tint queueBuffers(FrameBuffer *input,\n+\t\t\t const std::map &outputs);\n+\n+\tSignal &getSignalSetSensorControls();\n+\n+\tvoid process(FrameBuffer *input, FrameBuffer *output);\n+\n+private:\n+\tvoid statsReady(int dummy);\n+\tvoid inputReady(FrameBuffer *input);\n+\tvoid outputReady(FrameBuffer *output);\n+\n+\tstd::unique_ptr debayer_;\n+\tThread ispWorkerThread_;\n+\t/* FIXME debayerParams should come from the IPA */\n+\tDebayerParams debayerParams_;\n+\n+\tstd::unique_ptr ipa_;\n+};\n+\n+} /* namespace libcamera */\ndiff --git a/src/libcamera/meson.build b/src/libcamera/meson.build\nindex b3606969..e758ac9c 100644\n--- a/src/libcamera/meson.build\n+++ b/src/libcamera/meson.build\n@@ -70,6 +70,7 @@ subdir('converter')\n subdir('ipa')\n subdir('pipeline')\n subdir('proxy')\n+subdir('software_isp')\n \n null_dep = dependency('', required : false)\n \ndiff --git a/src/libcamera/software_isp/meson.build b/src/libcamera/software_isp/meson.build\nindex 6d7a44d7..9464f2fd 100644\n--- a/src/libcamera/software_isp/meson.build\n+++ b/src/libcamera/software_isp/meson.build\n@@ -6,3 +6,22 @@ libcamera_sources += files([\n \t'swstats.cpp',\n \t'swstats_cpu.cpp',\n ])\n+\n+# Software ISP is enabled for 'simple' pipeline handler.\n+# The 'pipelines' option value should be\n+# 'simple/' e.g.:\n+# -Dpipelines=simple/simple\n+# The source file should be named swisp_.cpp,\n+# e.g. 'swisp_simple.cpp'.\n+\n+foreach pipeline : pipelines\n+ pipeline = pipeline.split('/')\n+ if pipeline.length() == 2 and pipeline[0] == 'simple'\n+ libcamera_sources += files([\n+ 'swisp_' + pipeline[1] + '.cpp',\n+ ])\n+ # the 'break' below can be removed if/when multiple\n+ # Software ISP implementations are allowed in single build\n+ break\n+ endif\n+endforeach\ndiff --git a/src/libcamera/software_isp/swisp_simple.cpp b/src/libcamera/software_isp/swisp_simple.cpp\nnew file mode 100644\nindex 00000000..ff05b6fe\n--- /dev/null\n+++ b/src/libcamera/software_isp/swisp_simple.cpp\n@@ -0,0 +1,219 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2023, Linaro Ltd\n+ *\n+ * swisp_simple.cpp - Simple software ISP implementation\n+ */\n+\n+#include \"libcamera/internal/software_isp/swisp_simple.h\"\n+\n+#include \n+#include \n+#include \n+\n+#include \n+#include \n+\n+#include \"libcamera/internal/bayer_format.h\"\n+#include \"libcamera/internal/framebuffer.h\"\n+#include \"libcamera/internal/ipa_manager.h\"\n+#include \"libcamera/internal/mapped_framebuffer.h\"\n+\n+namespace libcamera {\n+\n+SwIspSimple::SwIspSimple(PipelineHandler *pipe, const ControlInfoMap &sensorControls)\n+\t: SoftwareIsp(pipe, sensorControls), debayer_(nullptr), debayerParams_{256, 256, 0.5f}\n+{\n+\tstd::unique_ptr stats;\n+\n+\tstats = std::make_unique();\n+\tif (!stats) {\n+\t\tLOG(SoftwareIsp, Error) << \"Failed to create SwStatsCpu object\";\n+\t\treturn;\n+\t}\n+\n+\tstats->statsReady.connect(this, &SwIspSimple::statsReady);\n+\n+\tdebayer_ = std::make_unique(std::move(stats));\n+\tif (!debayer_) {\n+\t\tLOG(SoftwareIsp, Error) << \"Failed to create DebayerCpu object\";\n+\t\treturn;\n+\t}\n+\n+\tdebayer_->inputBufferReady.connect(this, &SwIspSimple::inputReady);\n+\tdebayer_->outputBufferReady.connect(this, &SwIspSimple::outputReady);\n+\n+\tipa_ = IPAManager::createIPA(pipe, 0, 0);\n+\tif (!ipa_) {\n+\t\tLOG(SoftwareIsp, Error)\n+\t\t\t<< \"Creating IPA for software ISP failed\";\n+\t\tdebayer_.reset();\n+\t\treturn;\n+\t}\n+\n+\tint ret = ipa_->init(IPASettings{ \"No cfg file\", \"No sensor model\" },\n+\t\t\t debayer_->getStatsFD(),\n+\t\t\t sensorControls);\n+\tif (ret) {\n+\t\tLOG(SoftwareIsp, Error) << \"IPA init failed\";\n+\t\tdebayer_.reset();\n+\t\treturn;\n+\t}\n+\n+\tipa_->configure(sensorControls);\n+\n+\tdebayer_->moveToThread(&ispWorkerThread_);\n+}\n+\n+void SwIspSimple::processStats(const ControlList &sensorControls)\n+{\n+\tASSERT(ipa_);\n+\tipa_->processStats(sensorControls);\n+}\n+\n+Signal &SwIspSimple::getSignalSetSensorControls()\n+{\n+\tASSERT(ipa_);\n+\treturn ipa_->setSensorControls;\n+}\n+\n+bool SwIspSimple::isValid() const\n+{\n+\treturn !!debayer_;\n+}\n+\n+std::vector SwIspSimple::formats(PixelFormat inputFormat)\n+{\n+\tASSERT(debayer_ != nullptr);\n+\n+\treturn debayer_->formats(inputFormat);\n+}\n+\n+SizeRange SwIspSimple::sizes(PixelFormat inputFormat, const Size &inputSize)\n+{\n+\tASSERT(debayer_ != nullptr);\n+\n+\treturn debayer_->sizes(inputFormat, inputSize);\n+}\n+\n+std::tuple\n+SwIspSimple::strideAndFrameSize(const PixelFormat &outputFormat, const Size &size)\n+{\n+\tASSERT(debayer_ != nullptr);\n+\n+\treturn debayer_->strideAndFrameSize(outputFormat, size);\n+}\n+\n+int SwIspSimple::configure(const StreamConfiguration &inputCfg,\n+\t\t\t const std::vector> &outputCfgs)\n+{\n+\tASSERT(debayer_ != nullptr);\n+\n+\treturn debayer_->configure(inputCfg, outputCfgs);\n+}\n+\n+int SwIspSimple::exportBuffers(unsigned int output, unsigned int count,\n+\t\t\t std::vector> *buffers)\n+{\n+\tASSERT(debayer_ != nullptr);\n+\n+\t/* single output for now */\n+\tif (output >= 1)\n+\t\treturn -EINVAL;\n+\n+\t/* TODO: allocate from dma_heap; memfd buffs aren't allowed in FrameBuffer */\n+\tfor (unsigned int i = 0; i < count; i++) {\n+\t\tstd::string name = \"frame-\" + std::to_string(i);\n+\n+\t\tconst int ispFd = memfd_create(name.c_str(), 0);\n+\t\tint ret = ftruncate(ispFd, debayer_->frameSize());\n+\t\tif (ret < 0) {\n+\t\t\tLOG(SoftwareIsp, Error)\n+\t\t\t\t<< \"ftruncate() for memfd failed \"\n+\t\t\t\t<< strerror(-ret);\n+\t\t\treturn ret;\n+\t\t}\n+\n+\t\tFrameBuffer::Plane outPlane;\n+\t\toutPlane.fd = SharedFD(std::move(ispFd));\n+\t\toutPlane.offset = 0;\n+\t\toutPlane.length = debayer_->frameSize();\n+\n+\t\tstd::vector planes{ outPlane };\n+\t\tbuffers->emplace_back(std::make_unique(std::move(planes)));\n+\t}\n+\n+\treturn count;\n+}\n+\n+int SwIspSimple::queueBuffers(FrameBuffer *input,\n+\t\t\t const std::map &outputs)\n+{\n+\tunsigned int mask = 0;\n+\n+\t/*\n+\t * Validate the outputs as a sanity check: at least one output is\n+\t * required, all outputs must reference a valid stream and no two\n+\t * outputs can reference the same stream.\n+\t */\n+\tif (outputs.empty())\n+\t\treturn -EINVAL;\n+\n+\tfor (auto [index, buffer] : outputs) {\n+\t\tif (!buffer)\n+\t\t\treturn -EINVAL;\n+\t\tif (index >= 1) /* only single stream atm */\n+\t\t\treturn -EINVAL;\n+\t\tif (mask & (1 << index))\n+\t\t\treturn -EINVAL;\n+\n+\t\tmask |= 1 << index;\n+\t}\n+\n+\tprocess(input, outputs.at(0));\n+\n+\treturn 0;\n+}\n+\n+int SwIspSimple::start()\n+{\n+\tint ret = ipa_->start();\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tispWorkerThread_.start();\n+\treturn 0;\n+}\n+\n+void SwIspSimple::stop()\n+{\n+\tispWorkerThread_.exit();\n+\tispWorkerThread_.wait();\n+\n+\tipa_->stop();\n+}\n+\n+void SwIspSimple::process(FrameBuffer *input, FrameBuffer *output)\n+{\n+\tdebayer_->invokeMethod(&DebayerCpu::process,\n+\t\t\t ConnectionTypeQueued, input, output, debayerParams_);\n+}\n+\n+void SwIspSimple::statsReady(int dummy)\n+{\n+\tispStatsReady.emit(dummy);\n+}\n+\n+void SwIspSimple::inputReady(FrameBuffer *input)\n+{\n+\tinputBufferReady.emit(input);\n+}\n+\n+void SwIspSimple::outputReady(FrameBuffer *output)\n+{\n+\toutputBufferReady.emit(output);\n+}\n+\n+REGISTER_SOFTWAREISP(SwIspSimple)\n+\n+} /* namespace libcamera */\n","prefixes":["libcamera-devel","09/11"]}