{"id":26229,"url":"https://patchwork.libcamera.org/api/1.1/patches/26229/?format=json","web_url":"https://patchwork.libcamera.org/patch/26229/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20260223160930.27913-4-johannes.goede@oss.qualcomm.com>","date":"2026-02-23T16:09:29","name":"[v2,3/4] software_isp: debayer_cpu: Add multi-threading support","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"1d71bc12e1bfb2ee81e2060f275c9019e3e3793e","submitter":{"id":242,"url":"https://patchwork.libcamera.org/api/1.1/people/242/?format=json","name":"Hans de Goede","email":"johannes.goede@oss.qualcomm.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/26229/mbox/","series":[{"id":5809,"url":"https://patchwork.libcamera.org/api/1.1/series/5809/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5809","date":"2026-02-23T16:09:27","name":"software_isp: debayer_cpu: Add multi-threading support","version":2,"mbox":"https://patchwork.libcamera.org/series/5809/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/26229/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/26229/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C0BD8C32EA\n\tfor ;\n\tMon, 23 Feb 2026 16:09:43 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 5D09662299;\n\tMon, 23 Feb 2026 17:09:42 +0100 (CET)","from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com\n\t[205.220.180.131])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B739E62287\n\tfor ;\n\tMon, 23 Feb 2026 17:09:38 +0100 (CET)","from pps.filterd (m0279870.ppops.net [127.0.0.1])\n\tby mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id\n\t61NAYcGQ322186 for ;\n\tMon, 23 Feb 2026 16:09:37 GMT","from mail-qk1-f199.google.com (mail-qk1-f199.google.com\n\t[209.85.222.199])\n\tby mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4cgn8y1145-1\n\t(version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT)\n\tfor ;\n\tMon, 23 Feb 2026 16:09:37 +0000 (GMT)","by mail-qk1-f199.google.com with SMTP id\n\taf79cd13be357-8cb4d191ef1so652118185a.0\n\tfor ;\n\tMon, 23 Feb 2026 08:09:37 -0800 (PST)","from t14s.vakwerkhuis.com ([78.108.130.194])\n\tby smtp.gmail.com with ESMTPSA id\n\ta640c23a62f3a-b9084c60b91sm338284066b.14.2026.02.23.08.09.34\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tMon, 23 Feb 2026 08:09:35 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=qualcomm.com header.i=@qualcomm.com\n\theader.b=\"b3oOvW2d\"; dkim=pass (2048-bit key;\n\tunprotected) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com\n\theader.b=\"C5ZPSBiS\"; dkim-atps=neutral","DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h=\n\tcc:content-transfer-encoding:date:from:in-reply-to:message-id\n\t:mime-version:references:subject:to; s=qcppdkim1; bh=pp4GDY/S4RF\n\tspnVUNwPz0HW9rkYc2d5geNOIcGEDaMI=; b=b3oOvW2dFLo2IQY9QZmWMxMkBig\n\th93be8jE/tDzK1GBARkLEwZwWDp3syi+BnACXUHLCrntkSNeNJ+GffRcOPm4ClcL\n\tvBmK77l7xXEiGi37KpKHPlRU/j9B5pkoPicpVg8DDAJfE1pD3Iksb6KJzos1CGg5\n\t4g7E8Rw5fMDQgFH9h9wpOMYCYyA11zwJ/gU9L/QfuMvth2qnHKRUmFm7pRCIFhcE\n\tzkG00JepyeDQxM1NN0/2euI+ppVjipHhwKKXsZXxUmHUszYZjixsevcMNYHIuHUY\n\tTg16abZgjO0UenfSHUUzEm2ilOjfiyxbICLdcwXcLkoGFK8PEO13mYDm0BA==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=oss.qualcomm.com; s=google; t=1771862976; x=1772467776;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=pp4GDY/S4RFspnVUNwPz0HW9rkYc2d5geNOIcGEDaMI=;\n\tb=C5ZPSBiSNS0j/DoEUsQM9nXEPhPA48gk/uXdZhSP8C8NZlTSzpcr7vbGUCtscgmPTa\n\tgR8C9BVhvEzmRRiYNMkQ2re4ryXXoFSsE+gMkG3JJsAPYg0Y3x1wIgZzHCEb0PHYiesU\n\tWGdjF8JilwZHa1BAXF3slBNji9l6ey8zkggzOqNdvjkipICjolF/qTfpLgFgGwo5W+lU\n\tl2xB+cVcMDfBjEXniS//wV2s9QIL2+Gx6QXm8i9fMNyWlCPwBsXJPjolUTmjNXSPQDLV\n\tFBYUalwvssKQpfpkdwgUi+dy34CuMzqIJK9N9E2mhmD4jGmElfT67balPyL/b3fzh8N2\n\tajrw=="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1771862976; x=1772467776;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from\n\t:to:cc:subject:date:message-id:reply-to;\n\tbh=pp4GDY/S4RFspnVUNwPz0HW9rkYc2d5geNOIcGEDaMI=;\n\tb=I03XSONgVW9i+3QewGSUdCpT6ces68xEVMVMPaLCMlIwpw9hKoB504C4ZsrACoI4pu\n\tpJk/FRiYlAtEPcAxx3bc5pSHAaynE4zAJt/U+L2lBUSu3YNzSnNYwETXAnId6b39R6bV\n\tNaxel1tjFEb+WtOWkQHHNPOhVWWEjT+NrbjQH9jg4yMNoiXXLEzCyL4Y0LnmoFT4M9mC\n\tUpXo+42AvgW3TOADN0y7d0bZz1uf3Y4DevHP3XdMCqde42QO0jXPCJ9iNGK+Gf3nc00A\n\t4CHjh9XdwxNtwDz7CSsjuuz9PvmBVQYxyb+pco8sRMvOTtBLKDneAL/04OFkB5FK7oCL\n\tDkdw==","X-Gm-Message-State":"AOJu0Yx3CSDSpoUiiRzz1b0ISsbpqJRXhnPHq9zG/72a6gv2bZb8aTpz\n\tk74JKvoVRCMiJkgZm5NcrSfBEOJV0ss3OpOirGdDs5//NSxy2HtGtmd4lsdRXslMKBsOoXe/Sbj\n\tPUhkquVpNcGxVK565N8yFKyDy+V4l/0P33JIL7XEpSUppCt+QTfY4UGXtziwvZemeE2VqgqvzhP\n\tLvhk4XdudB","X-Gm-Gg":"AZuq6aIOfURWBbYpW0Ffz2iClKS6fXb4R3ihjPGb+UG47J0DA/b4UPVn6EcjmIDqdAN\n\tFgvuyWGQVCX9IQf42LFerixDGzRYDVWEOmJShZq4rAj58Wf1++2Y5zHbbKCGXyL5MXq1V7GfTTW\n\tWYy+T2cD08K1xetH8b54XDBmKchyvOsw2lXyshJ+Wj9Ptu3oUVO7isKdoad7i52vEiAfKhicJLL\n\tTXVw7SMUBG1LzEkXqVAKDn4ek1U+xVCG3lMrw8Sj6bxiYMGI4nQaoY+Rq+Gt8v4S26llMh0+9YW\n\t718QiTKLd8XltaWHrSpuvcKKwnj09ThIDk7ATYjNhY/dtfcWNT9P2VGWNCZShCZQvLelRejgNRw\n\tttkQ6HwrUMzyRFS0ncyQ1FY1bXBcISIoui79k5qpFDK5a3QMHLw==","X-Received":["by 2002:a05:620a:29c5:b0:8cb:4ba9:ccec with SMTP id\n\taf79cd13be357-8cb8ca92639mr1060160885a.72.1771862976286; \n\tMon, 23 Feb 2026 08:09:36 -0800 (PST)","by 2002:a05:620a:29c5:b0:8cb:4ba9:ccec with SMTP id\n\taf79cd13be357-8cb8ca92639mr1060154585a.72.1771862975641; \n\tMon, 23 Feb 2026 08:09:35 -0800 (PST)"],"From":"Hans de Goede ","To":"libcamera-devel@lists.libcamera.org","Cc":"Milan Zamazal ,\n\tHans de Goede ","Subject":"[PATCH v2 3/4] software_isp: debayer_cpu: Add multi-threading\n\tsupport","Date":"Mon, 23 Feb 2026 17:09:29 +0100","Message-ID":"<20260223160930.27913-4-johannes.goede@oss.qualcomm.com>","X-Mailer":"git-send-email 2.52.0","In-Reply-To":"<20260223160930.27913-1-johannes.goede@oss.qualcomm.com>","References":"<20260223160930.27913-1-johannes.goede@oss.qualcomm.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-Proofpoint-Spam-Details-Enc":"AW1haW4tMjYwMjIzMDEzOSBTYWx0ZWRfX+2BIC4oSdKV+\n\tls8YZLbjOAL7aVU3sQDWeVa7weM/BbCo5TBtJJNPn6T27VuXmDCg2wagkuM6OaULC/rF/JKdWuZ\n\t5yx98uhcohbtMopb4uWA75eMbWLhVW+JSTk262ZsH/nwSfxbOWa4hMgLeFyzRWwfTJ+c6cZvRlR\n\tsN6EOGzCgnvF5rb6fCa0HPBPIwg5nlLa5ESwAa658SJx4Ma5eNbHrjTTxCE/D5aImW7eDSZZeHE\n\tY/FT2ejrXlvdJe79cfh5hGAdUyMclWK7CWmwFt0rEfCz/cDfUJC1LAbn/rtCeHoEURePpjQM0DD\n\t8iiHI3HoJmeqUZi+47q8tvCkjK+hrkGveQoTRufhPRa8V0sfpXuatans+mO2TVRirK9IK1Mlta1\n\tEkfLnq77cgHZb3vN7pGBSLTq2tPhe0RIXE+87cDDVN0whTdhmK04iB0OOUKOs3qZn5/AFg/f4jp\n\t68x+HaN+ACfq17E97zw==","X-Authority-Analysis":"v=2.4 cv=edYwvrEH c=1 sm=1 tr=0 ts=699c7bc1 cx=c_pps\n\ta=HLyN3IcIa5EE8TELMZ618Q==:117 a=rrvG0T/C2D967D07Ol03YQ==:17\n\ta=HzLeVaNsDn8A:10 a=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22\n\ta=u7WPNUs3qKkmUXheDGA7:22 a=gowsoOTTUOVcmtlkKump:22 a=EUspDBNiAAAA:8\n\ta=wcwPUnDRDUmv0zfwghEA:9 a=bTQJ7kPSJx9SKPbeHEYW:22","X-Proofpoint-GUID":"VRgBxlBvilEpD_39Ykk9USmFBYyqWytG","X-Proofpoint-ORIG-GUID":"VRgBxlBvilEpD_39Ykk9USmFBYyqWytG","X-Proofpoint-Virus-Version":"vendor=baseguard\n\tengine=ICAP:2.0.293, Aquarius:18.0.1121, Hydra:6.1.51,\n\tFMLib:17.12.100.49\n\tdefinitions=2026-02-23_03,2026-02-23_03,2025-10-01_01","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tadultscore=0 malwarescore=0 phishscore=0 clxscore=1015\n\tpriorityscore=1501\n\tspamscore=0 impostorscore=0 bulkscore=0 suspectscore=0\n\tlowpriorityscore=0\n\tclassifier=typeunknown authscore=0 authtc= authcc= route=outbound\n\tadjust=0\n\treason=mlx scancount=1 engine=8.22.0-2602130000\n\tdefinitions=main-2602230139","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Add CPU soft ISP multi-threading support.\n\nBenchmark results for the Arduino Uno-Q with a weak CPU which is good for\nperformance testing, all numbers with an IMX219 running at\n3280x2464 -> 3272x2464:\n\n1 thread : 147ms / frame, ~6.5 fps\n2 threads: 80ms / frame, ~12.5 fps\n3 threads: 65ms / frame, ~15 fps\n\nAdding a 4th thread does not improve performance.\n\nSigned-off-by: Hans de Goede \n---\nChanges in v2:\n- Adjust to use the new DebayerCpuThread class introduced in the v2 patch-series\n- Re-use threads instead of starting new threads every frame\n---\n src/libcamera/software_isp/debayer_cpu.cpp | 53 ++++++++++++++++++++--\n src/libcamera/software_isp/debayer_cpu.h | 6 +++\n 2 files changed, 55 insertions(+), 4 deletions(-)","diff":"diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp\nindex 122bfbb05..ea1b17c1f 100644\n--- a/src/libcamera/software_isp/debayer_cpu.cpp\n+++ b/src/libcamera/software_isp/debayer_cpu.cpp\n@@ -18,6 +18,8 @@\n \n #include \n \n+#include \n+\n #include \n \n #include \"libcamera/internal/bayer_format.h\"\n@@ -50,13 +52,15 @@ public:\n \tunsigned int lineBufferIndex_;\n \tstd::vector lineBuffers_[DebayerCpu::kMaxLineBuffers];\n \tbool enableInputMemcpy_;\n+\tThread worker_;\n };\n \n DebayerCpuThread::DebayerCpuThread(DebayerCpu *debayer, unsigned int threadIndex,\n \t\t\t\t bool enableInputMemcpy)\n \t: debayer_(debayer), threadIndex_(threadIndex),\n-\t enableInputMemcpy_(enableInputMemcpy)\n+\t enableInputMemcpy_(enableInputMemcpy), worker_(\"DebayerWorker\")\n {\n+\tthis->moveToThread(&worker_);\n }\n \n /**\n@@ -88,8 +92,10 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats, const GlobalConfigurat\n \tbool enableInputMemcpy =\n \t\tconfiguration.option({ \"software_isp\", \"copy_input_buffer\" }).value_or(true);\n \n-\t/* Just one thread object for now, which will be called inline rather than async */\n-\tthreads_.resize(1);\n+\tunsigned int threadCount =\n+\t\tconfiguration.option({ \"software_isp\", \"threads\" }).value_or(2);\n+\tthreadCount = std::clamp(threadCount, 1u, 8u);\n+\tthreads_.resize(threadCount);\n \n \tfor (unsigned int i = 0; i < threads_.size(); i++)\n \t\tthreads_[i] = new DebayerCpuThread(this, i, enableInputMemcpy);\n@@ -714,6 +720,11 @@ void DebayerCpuThread::process(uint32_t frame, const uint8_t *src, uint8_t *dst)\n \t\tprocess2(frame, src, dst);\n \telse\n \t\tprocess4(frame, src, dst);\n+\n+\tdebayer_->workPendingMutex_.lock();\n+\tdebayer_->workPending_ &= ~(1 << threadIndex_);\n+\tdebayer_->workPendingMutex_.unlock();\n+\tdebayer_->workPendingCv_.notify_one();\n }\n \n void DebayerCpuThread::process2(uint32_t frame, const uint8_t *src, uint8_t *dst)\n@@ -953,7 +964,24 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output\n \n \tstats_->startFrame(frame);\n \n-\tthreads_[0]->process(frame, in.planes()[0].data(), out.planes()[0].data());\n+\tworkPendingMutex_.lock();\n+\tworkPending_ = (1 << threads_.size()) - 1;\n+\tworkPendingMutex_.unlock();\n+\n+\tfor (unsigned int i = 0; i < threads_.size(); i++)\n+\t\tthreads_[i]->invokeMethod(&DebayerCpuThread::process,\n+\t\t\t\t\t ConnectionTypeQueued, frame,\n+\t\t\t\t\t in.planes()[0].data(), out.planes()[0].data());\n+\n+\t{\n+\t\tMutexLocker locker(workPendingMutex_);\n+\n+\t\tauto workPending = ([&]() LIBCAMERA_TSA_REQUIRES(workPendingMutex_) {\n+\t\t\treturn workPending_ == 0;\n+\t\t});\n+\n+\t\tworkPendingCv_.wait(locker, workPending);\n+\t}\n \n \tmetadata.planes()[0].bytesused = out.planes()[0].size();\n \n@@ -972,6 +1000,23 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output\n \tinputBufferReady.emit(input);\n }\n \n+int DebayerCpu::start()\n+{\n+\tfor (unsigned int i = 0; i < threads_.size(); i++)\n+\t\tthreads_[i]->worker_.start();\n+\n+\treturn 0;\n+}\n+\n+void DebayerCpu::stop()\n+{\n+\tfor (unsigned int i = 0; i < threads_.size(); i++)\n+\t\tthreads_[i]->worker_.exit();\n+\n+\tfor (unsigned int i = 0; i < threads_.size(); i++)\n+\t\tthreads_[i]->worker_.wait();\n+}\n+\n SizeRange DebayerCpu::sizes(PixelFormat inputFormat, const Size &inputSize)\n {\n \tSize patternSize = this->patternSize(inputFormat);\ndiff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h\nindex 7196dcdd0..2c84f8e40 100644\n--- a/src/libcamera/software_isp/debayer_cpu.h\n+++ b/src/libcamera/software_isp/debayer_cpu.h\n@@ -16,6 +16,7 @@\n #include \n \n #include \n+#include \n \n #include \"libcamera/internal/bayer_format.h\"\n #include \"libcamera/internal/global_configuration.h\"\n@@ -41,6 +42,8 @@ public:\n \tstd::tuple\n \tstrideAndFrameSize(const PixelFormat &outputFormat, const Size &size);\n \tvoid process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms);\n+\tint start();\n+\tvoid stop();\n \tSizeRange sizes(PixelFormat inputFormat, const Size &inputSize);\n \tconst SharedFD &getStatsFD() { return stats_->getStatsFD(); }\n \n@@ -147,6 +150,9 @@ private:\n \tstd::unique_ptr stats_;\n \tunsigned int xShift_; /* Offset of 0/1 applied to window_.x */\n \n+\tunsigned int workPending_ LIBCAMERA_TSA_GUARDED_BY(workPendingMutex_);\n+\tMutex workPendingMutex_;\n+\tConditionVariable workPendingCv_;\n \tstd::vectorthreads_;\n };\n \n","prefixes":["v2","3/4"]}