{"id":26248,"url":"https://patchwork.libcamera.org/api/covers/26248/?format=json","web_url":"https://patchwork.libcamera.org/cover/26248/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20260304075052.11599-1-johannes.goede@oss.qualcomm.com>","date":"2026-03-04T07:50:47","name":"[v5,0/5] software_isp: debayer_cpu: Add multi-threading support","submitter":{"id":242,"url":"https://patchwork.libcamera.org/api/people/242/?format=json","name":"Hans de Goede","email":"johannes.goede@oss.qualcomm.com"},"mbox":"https://patchwork.libcamera.org/cover/26248/mbox/","series":[{"id":5817,"url":"https://patchwork.libcamera.org/api/series/5817/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5817","date":"2026-03-04T07:50:47","name":"software_isp: debayer_cpu: Add multi-threading support","version":5,"mbox":"https://patchwork.libcamera.org/series/5817/mbox/"}],"comments":"https://patchwork.libcamera.org/api/covers/26248/comments/","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 12B67BE086\n\tfor ;\n\tWed, 4 Mar 2026 07:51:00 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 41A8F6239A;\n\tWed, 4 Mar 2026 08:50:59 +0100 (CET)","from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com\n\t[205.220.180.131])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7095D62080\n\tfor ;\n\tWed, 4 Mar 2026 08:50:57 +0100 (CET)","from pps.filterd (m0279868.ppops.net [127.0.0.1])\n\tby mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id\n\t6245SdNr957613 for ;\n\tWed, 4 Mar 2026 07:50:56 GMT","from mail-qk1-f200.google.com (mail-qk1-f200.google.com\n\t[209.85.222.200])\n\tby mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4cp6qg9xcv-1\n\t(version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT)\n\tfor ;\n\tWed, 04 Mar 2026 07:50:55 +0000 (GMT)","by mail-qk1-f200.google.com with SMTP id\n\taf79cd13be357-8cb3ff05c73so3969448685a.0\n\tfor ;\n\tTue, 03 Mar 2026 23:50:55 -0800 (PST)","from shalem\n\t(2001-1c00-0c32-7800-5bfa-a036-83f0-f9ec.cable.dynamic.v6.ziggo.nl.\n\t[2001:1c00:c32:7800:5bfa:a036:83f0:f9ec])\n\tby smtp.gmail.com with ESMTPSA id\n\ta640c23a62f3a-b935ac73a5dsm693263366b.25.2026.03.03.23.50.53\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tTue, 03 Mar 2026 23:50:53 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=qualcomm.com header.i=@qualcomm.com\n\theader.b=\"iX3/2XR+\"; dkim=pass (2048-bit key;\n\tunprotected) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com\n\theader.b=\"Nuecm7zu\"; dkim-atps=neutral","DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h=\n\tcc:content-transfer-encoding:date:from:message-id:mime-version\n\t:subject:to; s=qcppdkim1; bh=r6yfsDZD7okJiDZWb/qTRzX4We5MW4Nw87f\n\t9JlQudus=; b=iX3/2XR+554L/JNbIfFuu/AG5wHVBPmagDKwY31vs27i5iEjrUu\n\tpep6o9uVF4WnjIGhIQ4HsSqdrNvsFi5cuybLXiaTHi4HUS+2iyUTheO2u98VU/kQ\n\tYEaUzox1vp22gZqY6LIdLy4AaPkb4+Vd2almK5RsS7T+H8xna6o32Hn21BmvwINU\n\tcx66z+Yp/iRwZJ55Biauc2PJjOePcSCt5KjoaVoshCH5LvrGePYbLiV+OIGhQNng\n\te+iuZSTv5WA+hPJLpQ+U0MyFUbv3e4JVieAw5ugIIdvTPZ4iua9XlXU991w4Tsgc\n\twWCC7cl2q2WNm6ivjNui1an4U9+CflkjPhA==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=oss.qualcomm.com; s=google; t=1772610655; x=1773215455;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:message-id:date:subject:cc\n\t:to:from:from:to:cc:subject:date:message-id:reply-to;\n\tbh=r6yfsDZD7okJiDZWb/qTRzX4We5MW4Nw87f9JlQudus=;\n\tb=Nuecm7zucNZgSO5cfq2egoE6Wrt4b2Mw5D42s2OhKQWWkzV00GXyt5lKvj3uHp2ucG\n\t3TBE4FjFrx/pYc5cym0DEkBQ/XzxEwVTDGB+GD9pqSpbiVYEP9GvDs+MenJp8Ewsq+w4\n\tfsGjTSOKE7U5VnJZ+sBZVpiFUfP34Y3yiu90mNbHz54uAoJ9jCLW0wZW4kCUz+DkfN+4\n\tgLWUU/PuNmZ0Wgo8MbZBo8VBu6lyWqQ+GO2zKo17ayrslf8AQMegVyHVvKoJ5pctN4i/\n\tfACtvYgQJ1WU0tPvW6wJP4NlSXofCS2+FJt/WcVJ/yBg4J09vi0AdCTtuSk/DErT0/4e\n\tTJxg=="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1772610655; x=1773215455;\n\th=content-transfer-encoding:mime-version:message-id:date:subject:cc\n\t:to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=r6yfsDZD7okJiDZWb/qTRzX4We5MW4Nw87f9JlQudus=;\n\tb=eS9QHN9tsq7KTMztCNbfMcNKfq39174G0Kw1oxaL7a09mWfI7zD82SmiCk3qUaqZcK\n\tuymJMHCZrY4xcFwZ1DGpB4bv5sYK8ao+ALOXvQvSgwe2ax61hRhsCNZ2fUoAxcOHPP3M\n\tMv2kW1AZitbvOaR/EA9xUVXT/JonwOG1J5Ik3Y4LurtVxFDT9is/HIZQxh+SFHprD/sH\n\tZVj9WPnPO98M7wg7C/xwEd26MnsPLFpV+EF6GeZ0JnHibVeWTWd4UvpL2c8+JD6L92wP\n\tkHjQteBq+zPV3LHQP7ZkAjeleENMx9B821QXkb7Ped9Ade0+UeNmvMmXPTBPvSosxWUh\n\tXkiQ==","X-Gm-Message-State":"AOJu0Yy9z6qs1VQR+moFxjq3ch8T2Z4X38onUZXtmwbqv0yulF86LaEP\n\teF2O2HZxRBs/4SvRlyTQqnVx5AqV3rLNN22nZKtGZ+hrjCusukWG24VdubABEy1YBtq1uC7rhhf\n\tZBQhJU+wTJVuCQR5t7uH3UOGWKMWqiWXzuDoCRdVsiPpIBZH2jXjQdtgE6rgfeqlJkR7iSnLYya\n\tBbv6opFSkD","X-Gm-Gg":"ATEYQzzeOtHqqGViLv/gPIEwmiUkn4v7h+di5AE5ak9ySAcnEp53QDTGO3w6bGa888r\n\tEo1oW/aW8rHtTUwGOWHTd7caHUwG3XpI6dO8fSBlrSQ3d3QnZELgAwtmOJQcMInJnCtHTfD+TAX\n\tN+vGniyt9th3wiIWJ5DJBc3dywMAlXQbrirGoAGCSNpjS6YMZYL3uhHuXbrkuPO+cwF9oEGP+CM\n\tKaswcpDa3sqlJ5hPtyAFipwWPnC1mqoU6elQgtjFdXDdLGUTCI0rXDc3tI2scmAUpBAAxSPDXVk\n\tvwABnYRczT2lHAJnlxMsjaEQmbmc1GfUXVvm0G2c6wP/eZW9CEqXgM2yUZ8WQUY5dQhgyzEHVj/\n\toQrbjtBI+0BDrhu4z66N6O2l6l7TcB0WErFXBO8eMhmavjFHdTJ16igwYPvZoxB+smtbmLCMWN6\n\tcv0uzv6ihHFCdezcwqhKzKbCuRb5oK5JmI9Rh6","X-Received":["by 2002:a05:620a:4546:b0:8c7:140a:7dbf with SMTP id\n\taf79cd13be357-8cd5afc3269mr123736585a.77.1772610654936; \n\tTue, 03 Mar 2026 23:50:54 -0800 (PST)","by 2002:a05:620a:4546:b0:8c7:140a:7dbf with SMTP id\n\taf79cd13be357-8cd5afc3269mr123735585a.77.1772610654479; \n\tTue, 03 Mar 2026 23:50:54 -0800 (PST)"],"From":"Hans de Goede ","To":"libcamera-devel@lists.libcamera.org, Milan Zamazal ","Cc":"Hans de Goede ","Subject":"[PATCH v5 0/5] software_isp: debayer_cpu: Add multi-threading\n\tsupport","Date":"Wed, 4 Mar 2026 08:50:47 +0100","Message-ID":"<20260304075052.11599-1-johannes.goede@oss.qualcomm.com>","X-Mailer":"git-send-email 2.52.0","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-Proofpoint-Spam-Details-Enc":"AW1haW4tMjYwMzA0MDA2MiBTYWx0ZWRfXzN5etPNgAUkO\n\tcco9CHI49qiY5ZLk7T/H+aj66OVdzM2iFROtlQJvAhy48CtSSj+bsyf3dUXigpXBp+pAXJ/vlQM\n\tkQaLq+N34Ah6yPV6Fq5ksJjXoPNNUYquwM6fM/1JaeqJ0/kHoW6ez2nh5Zisb/eoUJ44MK9OTFo\n\tWdCw55un6ywXsQGsQYM459NI6qjFUpQbm2Wqd6ZZkUtJ8ZboSNAckr47v122HGuZy2nNGpe34eI\n\tFAOFK9V2cFwas5+BS5z+H8qBzb9uLjIuXG5bUp1t2C/RlbDTHKLpUgbT1JbG49Ev6VXW8bDpx8J\n\tUQs6jAdJ6sPD+z6XuKfvGvhKgzWWDQmLglab4dCqp4grJuHdbmepaNOw5HsrxD9lkWct0nGLazC\n\tqsvcibw0Yn71SO1alK2kKzCZSJFPNc+bYykAp/epVk2kpUCqwRrs7lvHl0+DHD/YMmTmDHbg04j\n\tMW9M9yTz8I7glJVGd9Q==","X-Authority-Analysis":"v=2.4 cv=UJ3Q3Sfy c=1 sm=1 tr=0 ts=69a7e45f cx=c_pps\n\ta=hnmNkyzTK/kJ09Xio7VxxA==:117 a=xqWC_Br6kY4A:10 a=Yq5XynenixoA:10\n\ta=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=u7WPNUs3qKkmUXheDGA7:22\n\ta=ZpdpYltYx_vBUK5n70dp:22 a=_ldmQgw09SqRUz8hUs0A:9\n\ta=PEH46H7Ffwr30OY-TuGO:22","X-Proofpoint-ORIG-GUID":"IW6-oPTAc9n3USsLbyddmmw7alyLwFGr","X-Proofpoint-GUID":"IW6-oPTAc9n3USsLbyddmmw7alyLwFGr","X-Proofpoint-Virus-Version":"vendor=baseguard\n\tengine=ICAP:2.0.293, Aquarius:18.0.1121, Hydra:6.1.51,\n\tFMLib:17.12.100.49\n\tdefinitions=2026-03-04_02,2026-03-03_01,2025-10-01_01","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tsuspectscore=0 priorityscore=1501 clxscore=1015 malwarescore=0\n\tspamscore=0\n\tadultscore=0 bulkscore=0 phishscore=0 impostorscore=0\n\tlowpriorityscore=0\n\tclassifier=typeunknown authscore=0 authtc= authcc= route=outbound\n\tadjust=0\n\treason=mlx scancount=1 engine=8.22.0-2602130000\n\tdefinitions=main-2603040062","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Hi All,\n\nThe QCM2290 SoC used on the Arduino Uno-Q seems to have a weak GPU, so weak\nthat it is barely faster then a single CPU core (CPU without CCM).\n\nThis has made me code-up the long envisioned multi-threading support\nfor the CPU softISP.\n\nChanges in v5:\n- Extend software_isp.threads docs in runtime_configuration.rst\n- New patch: \"Documentation/runtime_configuration: Add missing\n software_isp.mode doc\"\n\nChanges in v4:\n- Use const in for (const auto &s : stats_) {} in SwStatsCpu::finishFrame()\n- Move kMaxLineBuffers constant to DebayerCpuThread class\n- Document software_isp.threads option in runtime_configuration.rst\n- Add an use constants for min/max/default number of threads\n\nChanges in v3:\n- Use std::unique_ptr for the DebayerCpuThread pointers\n- Document new DebayerCpuThread class\n- Make DebayerCpuThread inherit from both Thread and Object\n- Use for (auto &thread : threads_)\n- Use for (auto &s : stats_) {}\n- Move input format logging from DebayerCpu::configure() to\n SoftwareIsp::configure()\n\nChanges in v2:\n- Quite a bit of refactoring based on v1 feedback, dropped patch 3/5 and 4/5\n from v2 since these now no longer make sense\n- Move the allocation of the vector of SwIspStats objects to inside\n the SwStatsCpu class, controlled by a configure() arguments instead\n of making the caller allocate the objects\n- Replace the DebayerCpuThreadData struct from v1 with a DebayerCpuThread\n class, derived from Object to allow calling invokeMethod for thread re-use\n in followup patches\n- As part of this also move a bunch of methods which primarily deal with\n per thread data: setupInputMemcpy(), shiftLinePointers(), memcpyNextLine(),\n process*() to the new DebayerCpuThread class\n- Re-use threads instead of starting new threads every frame\n- Add a new patch adding some extra DebayerCpu input format logging\n\nBenchmark results for the Uno-Q + IMX219 running at 3280x2464 -> 3272x2464\nwithout CCM:\n\n1 thread : 147ms / frame, ~6.5 fps\n2 threads: 80ms / frame, ~12.5 fps\n3 threads: 65ms / frame, ~15 fps\nGPU: 130ms / frame, ~7,5 fps\nGPU 0-copy: 110ms / frame, ~9.5 fps (requires pipeline + camss hacks)\nGPU lite: 85ms / frame, ~12 fps (CCM, contrast and gamma disabled)\n\nRegards,\n\nHans\n\n\nHans de Goede (5):\n software_isp: swstats_cpu: Prepare for multi-threading support\n software_isp: debayer_cpu: Add DebayerCpuThread class\n software_isp: debayer_cpu: Add multi-threading support\n software_isp: Log input config from configure()\n Documentation/runtime_configuration: Add missing software_isp.mode doc\n\n Documentation/runtime_configuration.rst | 20 ++\n .../internal/software_isp/swstats_cpu.h | 25 +-\n src/libcamera/software_isp/debayer_cpu.cpp | 286 ++++++++++++++----\n src/libcamera/software_isp/debayer_cpu.h | 33 +-\n src/libcamera/software_isp/software_isp.cpp | 12 +-\n src/libcamera/software_isp/swstats_cpu.cpp | 54 ++--\n 6 files changed, 321 insertions(+), 109 deletions(-)"}