{"id":26246,"url":"https://patchwork.libcamera.org/api/1.1/patches/26246/?format=json","web_url":"https://patchwork.libcamera.org/patch/26246/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20260303111741.17417-4-johannes.goede@oss.qualcomm.com>","date":"2026-03-03T11:17:40","name":"[v4,3/4] software_isp: debayer_cpu: Add multi-threading support","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"33e18d6aec4df6df15ba7bb280208bd0dbbbfd0b","submitter":{"id":242,"url":"https://patchwork.libcamera.org/api/1.1/people/242/?format=json","name":"Hans de Goede","email":"johannes.goede@oss.qualcomm.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/26246/mbox/","series":[{"id":5816,"url":"https://patchwork.libcamera.org/api/1.1/series/5816/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5816","date":"2026-03-03T11:17:37","name":"software_isp: debayer_cpu: Add multi-threading support","version":4,"mbox":"https://patchwork.libcamera.org/series/5816/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/26246/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/26246/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8A662C32EA\n\tfor ;\n\tTue, 3 Mar 2026 11:17:56 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id BD928623A2;\n\tTue, 3 Mar 2026 12:17:55 +0100 (CET)","from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com\n\t[205.220.168.131])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3547662395\n\tfor ;\n\tTue, 3 Mar 2026 12:17:52 +0100 (CET)","from pps.filterd (m0279867.ppops.net [127.0.0.1])\n\tby mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id\n\t6239moKV2733596 for ;\n\tTue, 3 Mar 2026 11:17:50 GMT","from mail-qk1-f199.google.com (mail-qk1-f199.google.com\n\t[209.85.222.199])\n\tby mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4cnswe129k-1\n\t(version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT)\n\tfor ;\n\tTue, 03 Mar 2026 11:17:50 +0000 (GMT)","by mail-qk1-f199.google.com with SMTP id\n\taf79cd13be357-8cb3ff05c73so3645742785a.0\n\tfor ;\n\tTue, 03 Mar 2026 03:17:50 -0800 (PST)","from t14s\n\t(2001-1c00-0c32-7800-beb3-9058-f5fe-3f2e.cable.dynamic.v6.ziggo.nl.\n\t[2001:1c00:c32:7800:beb3:9058:f5fe:3f2e])\n\tby smtp.gmail.com with ESMTPSA id\n\t4fb4d7f45d1cf-66003479178sm2958144a12.9.2026.03.03.03.17.47\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tTue, 03 Mar 2026 03:17:47 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=qualcomm.com header.i=@qualcomm.com\n\theader.b=\"pJXKCwHp\"; dkim=pass (2048-bit key;\n\tunprotected) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com\n\theader.b=\"LwpMicCV\"; dkim-atps=neutral","DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h=\n\tcc:content-transfer-encoding:date:from:in-reply-to:message-id\n\t:mime-version:references:subject:to; s=qcppdkim1; bh=61zLKSb3u/2\n\tBf+JVWEk65x3S2OYd22ZVRPj0PFmBxuE=; b=pJXKCwHpTsuNuQtSW3Asgq/Fxm2\n\tNkHLpg83OQp0NZg78EqVzjaBUA3i7ucq7DSLjaE1eZr5E/opZK88iL3IFKMKtjQU\n\tj7pBBhIZCwfiGnvrwiub/ShVhfudTwb0Ot1Cynjex3WcUZJL62Z/2v3XubaUfa39\n\t0sf0JVyab1DPfO6yV5+fE/zgqAJS+SnOt6o1Avd5x2dISvuQnmjbVWcK6wuDFt7/\n\tAM4R+VCIni7f3TO6WiOoRnBFtqjAdwAuAqlqf3HhNh/LNKDuCM1qbpsnvl7t2Arr\n\tCkW9IDk4xZkpuw7TDRjLkUtkX7djUrW/8PYHQTUrV5hOkpSCvakCSqzY9KA==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=oss.qualcomm.com; s=google; t=1772536669; x=1773141469;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=61zLKSb3u/2Bf+JVWEk65x3S2OYd22ZVRPj0PFmBxuE=;\n\tb=LwpMicCVH4M4BviWWDbSjQU9X9Z3epjOdgkPChjhgY4jg73zp8gJllJnta0d+SFuiG\n\tBEkRxPTOI6xkHft94P3wy0MJR9kcekXq/pi2oodew1sMPWibeLTUiSFZQzxQ2BxUSUQV\n\tY7peqIxqfQ0Uw+xr76GCF4fqNGQaIfrGS1zWiZHx4Z/Pssx8TB2ehDuAkA4OzCMVKKWn\n\tVwt5KrgJmHwB5zy3D+KZbMGz0qbiwcSPW4pmgNwNj15hseOicHe4/bo78wonrFMpNA+f\n\tP8U5c2gTT+Wx5jHLZYmp/SUzyQyxYjSTaqgfWR6HvlRgO82CgtcPd/a6/UUnSQiswNai\n\tYu0w=="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1772536669; x=1773141469;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from\n\t:to:cc:subject:date:message-id:reply-to;\n\tbh=61zLKSb3u/2Bf+JVWEk65x3S2OYd22ZVRPj0PFmBxuE=;\n\tb=KT8/mhAU4EIKqMjn0nd/OdD2AT2rwZu+b3SUXX2iFpn2ytJnHDDsNSpc7UpcUUnbnP\n\tt/gedDMveij3dH90lH6B0Pk9M69886rpGhqE5MniYX2uAsDVqpddWX/ZtKaXLU4lh1k4\n\tDlzPdesDVFyLbvqtnL2WW5XPWO/uRXksL/jLNWEPiK+Udh9BKCpsm8t3ZcUtlXUQ4Gxg\n\tHvrv5XVlaCdc4MVbP04z//GwMkbnTJaq8uYQUXdfujZ6+qMLZZjAqphVFjPuFcjolv4U\n\toqcsmsFtkavjdmssZ9iG+JB+CKtvXWqtRXT8AuEtwLL9jnvRwwWWC91iH52w/XVtlhLQ\n\t9b9A==","X-Gm-Message-State":"AOJu0Yx0+Wph8b8CoPczeMosApoElvibolqXyNdvHuLgSs5sYB3dcXsR\n\t3N8eDrCJfZ7IdrrsArRFesKib2xGm4Z8/bT3bhCCD9X67jCAKB8vJVrB/P8W+Y96zO5zDjuD5vV\n\tTVa3tjjDA2wn7pBT8KXpyPKuePl0152rRzLtr4viEyFp0U1QKx8LtDdpXEK2XGZD0vlrnOEF1SX\n\tWN17wyfuZA","X-Gm-Gg":"ATEYQzyIzHe+ilRDMIIjtIWAzXMbzJ+Zof6B7wI3yi0cn1eNRWflKLs0BsjvohRhDoy\n\tDqDGP1QTBNe84dSSSpoPUyTszAGyWuE58GMY2ALmCir/Ci/oAC4P0fNo9hznQTg1smPxmlCXZrt\n\tzB1z6aMiOPVocoqopu4ZM/PtKcNDeOtIWqqDt/q1lt/Gl731Sm8i40m9WjczFCLgw9TvzXwyYY+\n\tANXt56AwWc4N3LMyOjuQKngN0gW8qfZON0waiymonogjJNWaxw0DUBoh48Y6tevun8X0Bu6V3Sc\n\tbTkqIDMt2HNtRAVx97GFETxuITFzambnmTkr48LpGUIJzfbXzoD60lF6MiSLcQjv/StXsZlv+Pl\n\tI0FT9r5gVs73qZtpKZuMGLruIwl+sR5KsJvisOeuX0oEiXKnpu01AofLdheuBl/vuRbK4FfR2qo\n\tIOglny79Ep9dx8iAOB9JGAZxk8aa8GwrANSQ==","X-Received":["by 2002:a05:620a:25c7:b0:8ca:3c67:891d with SMTP id\n\taf79cd13be357-8cbc8f1af5emr1859077585a.54.1772536669088; \n\tTue, 03 Mar 2026 03:17:49 -0800 (PST)","by 2002:a05:620a:25c7:b0:8ca:3c67:891d with SMTP id\n\taf79cd13be357-8cbc8f1af5emr1859074685a.54.1772536668537; \n\tTue, 03 Mar 2026 03:17:48 -0800 (PST)"],"From":"Hans de Goede ","To":"libcamera-devel@lists.libcamera.org","Cc":"Milan Zamazal ,\n\tHans de Goede ","Subject":"[PATCH v4 3/4] software_isp: debayer_cpu: Add multi-threading\n\tsupport","Date":"Tue, 3 Mar 2026 12:17:40 +0100","Message-ID":"<20260303111741.17417-4-johannes.goede@oss.qualcomm.com>","X-Mailer":"git-send-email 2.52.0","In-Reply-To":"<20260303111741.17417-1-johannes.goede@oss.qualcomm.com>","References":"<20260303111741.17417-1-johannes.goede@oss.qualcomm.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-Proofpoint-ORIG-GUID":"WPnRoZim_gyi0BujCbYdFpjfD0LbFb00","X-Proofpoint-Spam-Details-Enc":"AW1haW4tMjYwMzAzMDA4NiBTYWx0ZWRfX94nF592oVPcr\n\t2p7Ue/HOZiimTJFor5DZYX2GA962AQ+46pwssysULQrblnTjUbhcNsBJ07RWQ2lX6lOSZiMNiRP\n\ttAz5a7rF4k7IBtTVbmhw09UiPQ7KDwXH1nev+hEFx6kVeapNoC+4Ki4nFKwjovCKF7/JgO1ZVeA\n\tWG8zIy+jGLrh/kbta2o77K2LtuPJ12OlUlFQlsmR4sMkFzHSJj7TlsRen/QPbI+XTH4M6qYBPjd\n\tL2YCA7Y55NRuuSarKBXNJxXodZCpI2uy3K/fzvBnmMc/OY1V/XgS1dbK/bepHp/Q6JR6rJxIE/B\n\t2HrBncoSOIuCih27EPdF001CIuL/2sDWIWNhlL/FpL7CYS3/W/3DC/vbR5PlR3O5fsYXiafV7Ss\n\trQhDBIdmcGvMVp4RnZumQkBTz3eHSV1LR8pThcRVuVRZXEMPo9FJi5S3OqnNKvHPKiXndqLtz0v\n\tuo7oUikJhGGYyLSseMg==","X-Authority-Analysis":"v=2.4 cv=TtHrRTXh c=1 sm=1 tr=0 ts=69a6c35e cx=c_pps\n\ta=HLyN3IcIa5EE8TELMZ618Q==:117 a=xqWC_Br6kY4A:10 a=Yq5XynenixoA:10\n\ta=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=u7WPNUs3qKkmUXheDGA7:22\n\ta=eoimf2acIAo5FJnRuUoq:22 a=EUspDBNiAAAA:8 a=wcwPUnDRDUmv0zfwghEA:9\n\ta=bTQJ7kPSJx9SKPbeHEYW:22","X-Proofpoint-GUID":"WPnRoZim_gyi0BujCbYdFpjfD0LbFb00","X-Proofpoint-Virus-Version":"vendor=baseguard\n\tengine=ICAP:2.0.293, Aquarius:18.0.1121, Hydra:6.1.51,\n\tFMLib:17.12.100.49\n\tdefinitions=2026-03-02_05,2026-03-03_01,2025-10-01_01","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tlowpriorityscore=0 malwarescore=0 phishscore=0 suspectscore=0\n\tadultscore=0\n\tpriorityscore=1501 spamscore=0 impostorscore=0 bulkscore=0\n\tclxscore=1015\n\tclassifier=typeunknown authscore=0 authtc= authcc= route=outbound\n\tadjust=0\n\treason=mlx scancount=1 engine=8.22.0-2602130000\n\tdefinitions=main-2603030086","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Add CPU soft ISP multi-threading support.\n\nBenchmark results for the Arduino Uno-Q with a weak CPU which is good for\nperformance testing, all numbers with an IMX219 running at\n3280x2464 -> 3272x2464:\n\n1 thread : 147ms / frame, ~6.5 fps\n2 threads: 80ms / frame, ~12.5 fps\n3 threads: 65ms / frame, ~15 fps\n\nAdding a 4th thread does not improve performance.\n\nSigned-off-by: Hans de Goede \n---\nChanges in v4:\n- Document \"software_isp\", \"threads\" option in runtime_configuration.rst\n- Add an use constants for min/max/default number of threads\n\nChanges in v3:\n- Adjust for DebayerCpuThread now inheriting from Thread\n- Use for (auto &thread : threads_)\n\nChanges in v2:\n- Adjust to use the new DebayerCpuThread class introduced in the v2 patch-series\n- Re-use threads instead of starting new threads every frame\n---\n Documentation/runtime_configuration.rst | 1 +\n src/libcamera/software_isp/debayer_cpu.cpp | 45 ++++++++++++++++++++--\n src/libcamera/software_isp/debayer_cpu.h | 10 +++++\n 3 files changed, 53 insertions(+), 3 deletions(-)","diff":"diff --git a/Documentation/runtime_configuration.rst b/Documentation/runtime_configuration.rst\nindex e99ef2fb9..07d1a9123 100644\n--- a/Documentation/runtime_configuration.rst\n+++ b/Documentation/runtime_configuration.rst\n@@ -51,6 +51,7 @@ file structure:\n measure:\n skip: # non-negative integer, frames to skip initially\n number: # non-negative integer, frames to measure\n+ threads: # integer >= 1, number of render threads to use, default 2\n \n Configuration file example\n --------------------------\ndiff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp\nindex d57d640df..f7c3e1751 100644\n--- a/src/libcamera/software_isp/debayer_cpu.cpp\n+++ b/src/libcamera/software_isp/debayer_cpu.cpp\n@@ -76,6 +76,7 @@ DebayerCpuThread::DebayerCpuThread(DebayerCpu *debayer, unsigned int threadIndex\n \t debayer_(debayer), threadIndex_(threadIndex),\n \t enableInputMemcpy_(enableInputMemcpy)\n {\n+\tmoveToThread(this);\n }\n \n /**\n@@ -107,8 +108,10 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats, const GlobalConfigurat\n \tbool enableInputMemcpy =\n \t\tconfiguration.option({ \"software_isp\", \"copy_input_buffer\" }).value_or(true);\n \n-\t/* Just one thread object for now, which will be called inline rather than async */\n-\tthreads_.resize(1);\n+\tunsigned int threadCount =\n+\t\tconfiguration.option({ \"software_isp\", \"threads\" }).value_or(kDefaultThreads);\n+\tthreadCount = std::clamp(threadCount, kMinThreads, kMaxThreads);\n+\tthreads_.resize(threadCount);\n \n \tfor (unsigned int i = 0; i < threads_.size(); i++)\n \t\tthreads_[i] = std::make_unique(this, i, enableInputMemcpy);\n@@ -746,6 +749,11 @@ void DebayerCpuThread::process(uint32_t frame, const uint8_t *src, uint8_t *dst)\n \t\tprocess2(frame, src, dst);\n \telse\n \t\tprocess4(frame, src, dst);\n+\n+\tdebayer_->workPendingMutex_.lock();\n+\tdebayer_->workPending_ &= ~(1 << threadIndex_);\n+\tdebayer_->workPendingMutex_.unlock();\n+\tdebayer_->workPendingCv_.notify_one();\n }\n \n void DebayerCpuThread::process2(uint32_t frame, const uint8_t *src, uint8_t *dst)\n@@ -985,7 +993,21 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output\n \n \tstats_->startFrame(frame);\n \n-\tthreads_[0]->process(frame, in.planes()[0].data(), out.planes()[0].data());\n+\tworkPendingMutex_.lock();\n+\tworkPending_ = (1 << threads_.size()) - 1;\n+\tworkPendingMutex_.unlock();\n+\n+\tfor (auto &thread : threads_)\n+\t\tthread->invokeMethod(&DebayerCpuThread::process,\n+\t\t\t\t ConnectionTypeQueued, frame,\n+\t\t\t\t in.planes()[0].data(), out.planes()[0].data());\n+\n+\t{\n+\t\tMutexLocker locker(workPendingMutex_);\n+\t\tworkPendingCv_.wait(locker, [&]() LIBCAMERA_TSA_REQUIRES(workPendingMutex_) {\n+\t\t\treturn workPending_ == 0;\n+\t\t});\n+\t}\n \n \tmetadata.planes()[0].bytesused = out.planes()[0].size();\n \n@@ -1004,6 +1026,23 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output\n \tinputBufferReady.emit(input);\n }\n \n+int DebayerCpu::start()\n+{\n+\tfor (auto &thread : threads_)\n+\t\tthread->start();\n+\n+\treturn 0;\n+}\n+\n+void DebayerCpu::stop()\n+{\n+\tfor (auto &thread : threads_)\n+\t\tthread->exit();\n+\n+\tfor (auto &thread : threads_)\n+\t\tthread->wait();\n+}\n+\n SizeRange DebayerCpu::sizes(PixelFormat inputFormat, const Size &inputSize)\n {\n \tSize patternSize = this->patternSize(inputFormat);\ndiff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h\nindex 780576090..11280c0a1 100644\n--- a/src/libcamera/software_isp/debayer_cpu.h\n+++ b/src/libcamera/software_isp/debayer_cpu.h\n@@ -16,6 +16,7 @@\n #include \n \n #include \n+#include \n \n #include \"libcamera/internal/bayer_format.h\"\n #include \"libcamera/internal/global_configuration.h\"\n@@ -41,6 +42,8 @@ public:\n \tstd::tuple\n \tstrideAndFrameSize(const PixelFormat &outputFormat, const Size &size);\n \tvoid process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms);\n+\tint start();\n+\tvoid stop();\n \tSizeRange sizes(PixelFormat inputFormat, const Size &inputSize);\n \tconst SharedFD &getStatsFD() { return stats_->getStatsFD(); }\n \n@@ -144,6 +147,13 @@ private:\n \tstd::unique_ptr stats_;\n \tunsigned int xShift_; /* Offset of 0/1 applied to window_.x */\n \n+\tstatic constexpr unsigned int kMinThreads = 1;\n+\tstatic constexpr unsigned int kMaxThreads = 8;\n+\tstatic constexpr unsigned int kDefaultThreads = 2;\n+\n+\tunsigned int workPending_ LIBCAMERA_TSA_GUARDED_BY(workPendingMutex_);\n+\tMutex workPendingMutex_;\n+\tConditionVariable workPendingCv_;\n \tstd::vector>threads_;\n };\n \n","prefixes":["v4","3/4"]}