{"id":26235,"url":"https://patchwork.libcamera.org/api/patches/26235/?format=json","web_url":"https://patchwork.libcamera.org/patch/26235/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20260224193745.106186-4-johannes.goede@oss.qualcomm.com>","date":"2026-02-24T19:37:44","name":"[v3,3/4] software_isp: debayer_cpu: Add multi-threading support","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"827d921ea6226136bec18acf2a8d6e41f23838bc","submitter":{"id":242,"url":"https://patchwork.libcamera.org/api/people/242/?format=json","name":"Hans de Goede","email":"johannes.goede@oss.qualcomm.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/26235/mbox/","series":[{"id":5811,"url":"https://patchwork.libcamera.org/api/series/5811/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5811","date":"2026-02-24T19:37:41","name":"software_isp: debayer_cpu: Add multi-threading support","version":3,"mbox":"https://patchwork.libcamera.org/series/5811/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/26235/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/26235/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 708E5C0DA4\n\tfor ;\n\tTue, 24 Feb 2026 19:37:58 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 23DB1622B7;\n\tTue, 24 Feb 2026 20:37:58 +0100 (CET)","from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com\n\t[205.220.180.131])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 23F9D622A2\n\tfor ;\n\tTue, 24 Feb 2026 20:37:53 +0100 (CET)","from pps.filterd (m0279873.ppops.net [127.0.0.1])\n\tby mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id\n\t61OFO0WH2432436 for ;\n\tTue, 24 Feb 2026 19:37:51 GMT","from mail-qk1-f200.google.com (mail-qk1-f200.google.com\n\t[209.85.222.200])\n\tby mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4chekj8ygx-1\n\t(version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT)\n\tfor ;\n\tTue, 24 Feb 2026 19:37:51 +0000 (GMT)","by mail-qk1-f200.google.com with SMTP id\n\taf79cd13be357-8cb4817f3c8so4472312685a.3\n\tfor ;\n\tTue, 24 Feb 2026 11:37:51 -0800 (PST)","from shalem\n\t(2001-1c00-0c32-7800-5bfa-a036-83f0-f9ec.cable.dynamic.v6.ziggo.nl.\n\t[2001:1c00:c32:7800:5bfa:a036:83f0:f9ec])\n\tby smtp.gmail.com with ESMTPSA id\n\ta640c23a62f3a-b9084e8cb3fsm458232866b.48.2026.02.24.11.37.49\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tTue, 24 Feb 2026 11:37:49 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=qualcomm.com header.i=@qualcomm.com\n\theader.b=\"fdb6yK38\"; dkim=pass (2048-bit key;\n\tunprotected) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com\n\theader.b=\"Gwekh/f7\"; dkim-atps=neutral","DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h=\n\tcc:content-transfer-encoding:date:from:in-reply-to:message-id\n\t:mime-version:references:subject:to; s=qcppdkim1; bh=Dg+H+cXUvva\n\tSQunQpgmYy3lZMM8Jgoeiy1NGwXPXAos=; b=fdb6yK38Xrbp5RlwYqLQ1CR/Mpb\n\tvfU3Xdl4XhBs6QE9cqpewmgpvDJUEX5YpNyXbdSJx1UTc+4FCebuIK/4NxVAahJ3\n\tlnl9y3Ytqm33bVARB9i0Ynwzan4aMFV9t3BgAi6L6/EJDT55uAr9ehWm8fBQHqFH\n\tJ4ZxibxnudF2aziwb3H8MaFxDsnlbY/WTaPdiIRJMJuWRk9niEireT9JwiKfHg1u\n\tfArjixlkBy+x6Cg+H+2PUk3+4VZb/rbjX4qAFOhX0n5lUTr3++up3ihKOC71XkaI\n\tlpKuTxKSyylBHOEmAqWfxOKh3nnSkXskK6q0t6FQgKu7ZvI2rcVSzL907lw==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=oss.qualcomm.com; s=google; t=1771961871; x=1772566671;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=Dg+H+cXUvvaSQunQpgmYy3lZMM8Jgoeiy1NGwXPXAos=;\n\tb=Gwekh/f77ArA7xrXlPmac9BExCao+9ndJBB2X7bbPELTR4vYWKrFOxtFZFMQkgJKmL\n\tsun3CM89511a+OlKbO3QZBOoxgNhnBSqwNPb9FA88XJBrOdC6so7VSHv2meLiZt37g/s\n\tAmTf69GEc/0KAa3sEMXlPQjWzE3QFMF7+vSzppXnIqaAj7bDYqfELTsSLJnayqRrLTuM\n\txyMiOkJy2p5H00Irk19qFSO9TrPnImqUKNOLwF2Bz55J+QtlUOrJTuf74UFXVA08ZIcA\n\tCaTxk/a2zbQiYFW6yqP6K2N9gPbvAlJMHFH54nIWDeRNDyJPbDdU/Em3WtWu/39MWj9i\n\tO/zQ=="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1771961871; x=1772566671;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from\n\t:to:cc:subject:date:message-id:reply-to;\n\tbh=Dg+H+cXUvvaSQunQpgmYy3lZMM8Jgoeiy1NGwXPXAos=;\n\tb=pKj4309VbZ7l/UTf2EQgSlblsMq9vqqUZKW4NT1gibXem341CqYjp34A/K0Pid1oZ8\n\tLwWU2wEORoPbdG2s3jHjSZIcr5ReEX4rtgWfZtNX9zprEn0WOHUXKVGHyDjW4t/Czd3o\n\txiuqceI3lP1cWbbIrNRaTK3FWj5D0BWucErtFA2X56S9L332oCniFpQrj0IsOfqHd6WK\n\t8gTYRNuECZQrssJg0ObSdwd1YITmsCt6eaKW5ICmNB6h0/n5eZ+RsfSENI5uwsfpPnOf\n\t2H8Z5AZXnabdXzHn+w4hldqagMKs3iUTEx5IiEV3LPIUjJX6D2E1DNJKpLo6hLQMPJs8\n\t2JLA==","X-Gm-Message-State":"AOJu0Ywt6xRfTY1juRL8XEATBwPggOFSl8AmjggBWx79YiBrzow2ylyk\n\tGm6xhnut18P+VKGPHvHdlVVko4oyHNcLkkEPdNTGGpRlVS2Ox5Mt8S+Q0tdfIlq9WjP9J4vFd/T\n\t97leX/PSUYEIFA1E7Fxl2FRmJlHnJtDR9/LKZIYtmJcxMf191yv97VrBpvgPDC9slVtBGwmxSNC\n\tiOxWtiGXN5","X-Gm-Gg":"AZuq6aIowf2MRuy9t6wXjn4xo5E7AAoWZ/s47P2+cKyL5OH6TqvgEPwdVJPQKemIjaA\n\tQ63PCYY3E6KmvAGMiRE3WCrjBpW6EKI+E/3OGGSRcW8B6n7I7k8bBjaxumaiXJD3G2BmZGTtca+\n\t+nJgnFiLqWYQUvaLfyYTglNndwQrBxFuHNVjlzaGotaow32plJUevRBbVKQHhJwxDMDZAr13SEu\n\t3L5G2LUAP8z5oG7wbkaxXyiMc044U+vF8bhhIZKXh6NtTgbA4otwI3sTkDdgP1Qpnll85Y1K2lG\n\tJAdV0Pwl2qj5cd7pv5izWCnMOqDDWmW+ihhos25hS15VL+y7tyLVSZ+kNVtObvMyFxiif3R1eQz\n\t+CkOWGd38gdEC5LFX/8xu3x24bZiTx0XxECEWBXp9QxN0yK1ZKhmmDzIcMNJxQSMntT4lreZt8Q\n\tON1UH0U6h4kFKdksK7nw+Frlpv2iczUyZcip2K","X-Received":["by 2002:a05:620a:4449:b0:8cb:4d64:e972 with SMTP id\n\taf79cd13be357-8cb8c9cdc56mr1454365385a.9.1771961870574; \n\tTue, 24 Feb 2026 11:37:50 -0800 (PST)","by 2002:a05:620a:4449:b0:8cb:4d64:e972 with SMTP id\n\taf79cd13be357-8cb8c9cdc56mr1454362385a.9.1771961870052; \n\tTue, 24 Feb 2026 11:37:50 -0800 (PST)"],"From":"Hans de Goede ","To":"libcamera-devel@lists.libcamera.org, Milan Zamazal ","Cc":"Hans de Goede ","Subject":"[PATCH v3 3/4] software_isp: debayer_cpu: Add multi-threading\n\tsupport","Date":"Tue, 24 Feb 2026 20:37:44 +0100","Message-ID":"<20260224193745.106186-4-johannes.goede@oss.qualcomm.com>","X-Mailer":"git-send-email 2.52.0","In-Reply-To":"<20260224193745.106186-1-johannes.goede@oss.qualcomm.com>","References":"<20260224193745.106186-1-johannes.goede@oss.qualcomm.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-Authority-Analysis":"v=2.4 cv=RNe+3oi+ c=1 sm=1 tr=0 ts=699dfe0f cx=c_pps\n\ta=hnmNkyzTK/kJ09Xio7VxxA==:117 a=xqWC_Br6kY4A:10 a=HzLeVaNsDn8A:10\n\ta=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=u7WPNUs3qKkmUXheDGA7:22\n\ta=rJkE3RaqiGZ5pbrm-msn:22 a=EUspDBNiAAAA:8 a=uyevB83Nksalfc49Bg4A:9\n\ta=PEH46H7Ffwr30OY-TuGO:22","X-Proofpoint-ORIG-GUID":"WgMGUl3KVxKeQaaHLsIkBcIFUhhhTmKC","X-Proofpoint-Spam-Details-Enc":"AW1haW4tMjYwMjI0MDE2OCBTYWx0ZWRfX58vuduKb+Msb\n\tPxOk93PCPgDqqxDNKvWjBq+OcJ6f7Aj0bM+jVC5JOZTZXfUknlW64l8+tq0sXXZBUOOKlybnqqc\n\t2oPsaw+zkyM4mfX7XfzNcVLOYeX5DfGCr2dpvfXjeUYlnLi0PEjO69h+mSXeEfLuSYWrWTR/TxX\n\tdssyaUFFXdiovtIlVKNxh1cTlvU9cFGRbc5y3Tfl1IvS9gVUPF8pt4lhIfzmzJc0nL5tbvXrzLi\n\tPtrOo4nVEnx2pbi97syxqVQtWUOL3ITYpjD578aLlUMo64EOyO9K0alMcUoGlRdwVbnrHnsS1H/\n\tywL8yhu8n0jh+6s3HMhe9Mm8jG1CGYM35voLllAzFpdSedYG9hEOlGFaLlYvj9Qn0es5C7AQNXR\n\tF9xAwCo24iq8SKA5Bp6q/hLTN9VIXT5e7r5C+1EP13TqEOKCayKSlmaZWciyyPz7v6xzJWlp1m1\n\turzUcoJcO93uLeLg6+w==","X-Proofpoint-GUID":"WgMGUl3KVxKeQaaHLsIkBcIFUhhhTmKC","X-Proofpoint-Virus-Version":"vendor=baseguard\n\tengine=ICAP:2.0.293, Aquarius:18.0.1121, Hydra:6.1.51,\n\tFMLib:17.12.100.49\n\tdefinitions=2026-02-24_02,2026-02-23_03,2025-10-01_01","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tbulkscore=0 lowpriorityscore=0 adultscore=0 priorityscore=1501\n\tphishscore=0\n\tclxscore=1015 spamscore=0 impostorscore=0 suspectscore=0\n\tmalwarescore=0\n\tclassifier=typeunknown authscore=0 authtc= authcc= route=outbound\n\tadjust=0\n\treason=mlx scancount=1 engine=8.22.0-2602130000\n\tdefinitions=main-2602240168","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Add CPU soft ISP multi-threading support.\n\nBenchmark results for the Arduino Uno-Q with a weak CPU which is good for\nperformance testing, all numbers with an IMX219 running at\n3280x2464 -> 3272x2464:\n\n1 thread : 147ms / frame, ~6.5 fps\n2 threads: 80ms / frame, ~12.5 fps\n3 threads: 65ms / frame, ~15 fps\n\nAdding a 4th thread does not improve performance.\n\nSigned-off-by: Hans de Goede \n---\nChanges in v3:\n- Adjust for DebayerCpuThread now inheriting from Thread\n- Use for (auto &thread : threads_)\n\nChanges in v2:\n- Adjust to use the new DebayerCpuThread class introduced in the v2 patch-series\n- Re-use threads instead of starting new threads every frame\n---\n src/libcamera/software_isp/debayer_cpu.cpp | 45 ++++++++++++++++++++--\n src/libcamera/software_isp/debayer_cpu.h | 6 +++\n 2 files changed, 48 insertions(+), 3 deletions(-)","diff":"diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp\nindex 36b7881b..cf16f00b 100644\n--- a/src/libcamera/software_isp/debayer_cpu.cpp\n+++ b/src/libcamera/software_isp/debayer_cpu.cpp\n@@ -73,6 +73,7 @@ DebayerCpuThread::DebayerCpuThread(DebayerCpu *debayer, unsigned int threadIndex\n \t debayer_(debayer), threadIndex_(threadIndex),\n \t enableInputMemcpy_(enableInputMemcpy)\n {\n+\tmoveToThread(this);\n }\n \n /**\n@@ -104,8 +105,10 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats, const GlobalConfigurat\n \tbool enableInputMemcpy =\n \t\tconfiguration.option({ \"software_isp\", \"copy_input_buffer\" }).value_or(true);\n \n-\t/* Just one thread object for now, which will be called inline rather than async */\n-\tthreads_.resize(1);\n+\tunsigned int threadCount =\n+\t\tconfiguration.option({ \"software_isp\", \"threads\" }).value_or(2);\n+\tthreadCount = std::clamp(threadCount, 1u, 8u);\n+\tthreads_.resize(threadCount);\n \n \tfor (unsigned int i = 0; i < threads_.size(); i++)\n \t\tthreads_[i] = std::make_unique(this, i, enableInputMemcpy);\n@@ -743,6 +746,11 @@ void DebayerCpuThread::process(uint32_t frame, const uint8_t *src, uint8_t *dst)\n \t\tprocess2(frame, src, dst);\n \telse\n \t\tprocess4(frame, src, dst);\n+\n+\tdebayer_->workPendingMutex_.lock();\n+\tdebayer_->workPending_ &= ~(1 << threadIndex_);\n+\tdebayer_->workPendingMutex_.unlock();\n+\tdebayer_->workPendingCv_.notify_one();\n }\n \n void DebayerCpuThread::process2(uint32_t frame, const uint8_t *src, uint8_t *dst)\n@@ -982,7 +990,21 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output\n \n \tstats_->startFrame(frame);\n \n-\tthreads_[0]->process(frame, in.planes()[0].data(), out.planes()[0].data());\n+\tworkPendingMutex_.lock();\n+\tworkPending_ = (1 << threads_.size()) - 1;\n+\tworkPendingMutex_.unlock();\n+\n+\tfor (auto &thread : threads_)\n+\t\tthread->invokeMethod(&DebayerCpuThread::process,\n+\t\t\t\t ConnectionTypeQueued, frame,\n+\t\t\t\t in.planes()[0].data(), out.planes()[0].data());\n+\n+\t{\n+\t\tMutexLocker locker(workPendingMutex_);\n+\t\tworkPendingCv_.wait(locker, [&]() LIBCAMERA_TSA_REQUIRES(workPendingMutex_) {\n+\t\t\treturn workPending_ == 0;\n+\t\t});\n+\t}\n \n \tmetadata.planes()[0].bytesused = out.planes()[0].size();\n \n@@ -1001,6 +1023,23 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output\n \tinputBufferReady.emit(input);\n }\n \n+int DebayerCpu::start()\n+{\n+\tfor (auto &thread : threads_)\n+\t\tthread->start();\n+\n+\treturn 0;\n+}\n+\n+void DebayerCpu::stop()\n+{\n+\tfor (auto &thread : threads_)\n+\t\tthread->exit();\n+\n+\tfor (auto &thread : threads_)\n+\t\tthread->wait();\n+}\n+\n SizeRange DebayerCpu::sizes(PixelFormat inputFormat, const Size &inputSize)\n {\n \tSize patternSize = this->patternSize(inputFormat);\ndiff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h\nindex 1074bc9c..eb52f101 100644\n--- a/src/libcamera/software_isp/debayer_cpu.h\n+++ b/src/libcamera/software_isp/debayer_cpu.h\n@@ -16,6 +16,7 @@\n #include \n \n #include \n+#include \n \n #include \"libcamera/internal/bayer_format.h\"\n #include \"libcamera/internal/global_configuration.h\"\n@@ -41,6 +42,8 @@ public:\n \tstd::tuple\n \tstrideAndFrameSize(const PixelFormat &outputFormat, const Size &size);\n \tvoid process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms);\n+\tint start();\n+\tvoid stop();\n \tSizeRange sizes(PixelFormat inputFormat, const Size &inputSize);\n \tconst SharedFD &getStatsFD() { return stats_->getStatsFD(); }\n \n@@ -147,6 +150,9 @@ private:\n \tstd::unique_ptr stats_;\n \tunsigned int xShift_; /* Offset of 0/1 applied to window_.x */\n \n+\tunsigned int workPending_ LIBCAMERA_TSA_GUARDED_BY(workPendingMutex_);\n+\tMutex workPendingMutex_;\n+\tConditionVariable workPendingCv_;\n \tstd::vector>threads_;\n };\n \n","prefixes":["v3","3/4"]}