From patchwork Mon Feb 23 16:09:29 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans de Goede X-Patchwork-Id: 26229 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C0BD8C32EA for ; Mon, 23 Feb 2026 16:09:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5D09662299; Mon, 23 Feb 2026 17:09:42 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=qualcomm.com header.i=@qualcomm.com header.b="b3oOvW2d"; dkim=pass (2048-bit key; unprotected) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b="C5ZPSBiS"; dkim-atps=neutral Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B739E62287 for ; Mon, 23 Feb 2026 17:09:38 +0100 (CET) Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 61NAYcGQ322186 for ; Mon, 23 Feb 2026 16:09:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h= cc:content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=qcppdkim1; bh=pp4GDY/S4RF spnVUNwPz0HW9rkYc2d5geNOIcGEDaMI=; b=b3oOvW2dFLo2IQY9QZmWMxMkBig h93be8jE/tDzK1GBARkLEwZwWDp3syi+BnACXUHLCrntkSNeNJ+GffRcOPm4ClcL vBmK77l7xXEiGi37KpKHPlRU/j9B5pkoPicpVg8DDAJfE1pD3Iksb6KJzos1CGg5 4g7E8Rw5fMDQgFH9h9wpOMYCYyA11zwJ/gU9L/QfuMvth2qnHKRUmFm7pRCIFhcE zkG00JepyeDQxM1NN0/2euI+ppVjipHhwKKXsZXxUmHUszYZjixsevcMNYHIuHUY Tg16abZgjO0UenfSHUUzEm2ilOjfiyxbICLdcwXcLkoGFK8PEO13mYDm0BA== Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4cgn8y1145-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Mon, 23 Feb 2026 16:09:37 +0000 (GMT) Received: by mail-qk1-f199.google.com with SMTP id af79cd13be357-8cb4d191ef1so652118185a.0 for ; Mon, 23 Feb 2026 08:09:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oss.qualcomm.com; s=google; t=1771862976; x=1772467776; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pp4GDY/S4RFspnVUNwPz0HW9rkYc2d5geNOIcGEDaMI=; b=C5ZPSBiSNS0j/DoEUsQM9nXEPhPA48gk/uXdZhSP8C8NZlTSzpcr7vbGUCtscgmPTa gR8C9BVhvEzmRRiYNMkQ2re4ryXXoFSsE+gMkG3JJsAPYg0Y3x1wIgZzHCEb0PHYiesU WGdjF8JilwZHa1BAXF3slBNji9l6ey8zkggzOqNdvjkipICjolF/qTfpLgFgGwo5W+lU l2xB+cVcMDfBjEXniS//wV2s9QIL2+Gx6QXm8i9fMNyWlCPwBsXJPjolUTmjNXSPQDLV FBYUalwvssKQpfpkdwgUi+dy34CuMzqIJK9N9E2mhmD4jGmElfT67balPyL/b3fzh8N2 ajrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771862976; x=1772467776; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=pp4GDY/S4RFspnVUNwPz0HW9rkYc2d5geNOIcGEDaMI=; b=I03XSONgVW9i+3QewGSUdCpT6ces68xEVMVMPaLCMlIwpw9hKoB504C4ZsrACoI4pu pJk/FRiYlAtEPcAxx3bc5pSHAaynE4zAJt/U+L2lBUSu3YNzSnNYwETXAnId6b39R6bV Naxel1tjFEb+WtOWkQHHNPOhVWWEjT+NrbjQH9jg4yMNoiXXLEzCyL4Y0LnmoFT4M9mC UpXo+42AvgW3TOADN0y7d0bZz1uf3Y4DevHP3XdMCqde42QO0jXPCJ9iNGK+Gf3nc00A 4CHjh9XdwxNtwDz7CSsjuuz9PvmBVQYxyb+pco8sRMvOTtBLKDneAL/04OFkB5FK7oCL Dkdw== X-Gm-Message-State: AOJu0Yx3CSDSpoUiiRzz1b0ISsbpqJRXhnPHq9zG/72a6gv2bZb8aTpz k74JKvoVRCMiJkgZm5NcrSfBEOJV0ss3OpOirGdDs5//NSxy2HtGtmd4lsdRXslMKBsOoXe/Sbj PUhkquVpNcGxVK565N8yFKyDy+V4l/0P33JIL7XEpSUppCt+QTfY4UGXtziwvZemeE2VqgqvzhP Lvhk4XdudB X-Gm-Gg: AZuq6aIOfURWBbYpW0Ffz2iClKS6fXb4R3ihjPGb+UG47J0DA/b4UPVn6EcjmIDqdAN FgvuyWGQVCX9IQf42LFerixDGzRYDVWEOmJShZq4rAj58Wf1++2Y5zHbbKCGXyL5MXq1V7GfTTW WYy+T2cD08K1xetH8b54XDBmKchyvOsw2lXyshJ+Wj9Ptu3oUVO7isKdoad7i52vEiAfKhicJLL TXVw7SMUBG1LzEkXqVAKDn4ek1U+xVCG3lMrw8Sj6bxiYMGI4nQaoY+Rq+Gt8v4S26llMh0+9YW 718QiTKLd8XltaWHrSpuvcKKwnj09ThIDk7ATYjNhY/dtfcWNT9P2VGWNCZShCZQvLelRejgNRw ttkQ6HwrUMzyRFS0ncyQ1FY1bXBcISIoui79k5qpFDK5a3QMHLw== X-Received: by 2002:a05:620a:29c5:b0:8cb:4ba9:ccec with SMTP id af79cd13be357-8cb8ca92639mr1060160885a.72.1771862976286; Mon, 23 Feb 2026 08:09:36 -0800 (PST) X-Received: by 2002:a05:620a:29c5:b0:8cb:4ba9:ccec with SMTP id af79cd13be357-8cb8ca92639mr1060154585a.72.1771862975641; Mon, 23 Feb 2026 08:09:35 -0800 (PST) Received: from t14s.vakwerkhuis.com ([78.108.130.194]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9084c60b91sm338284066b.14.2026.02.23.08.09.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Feb 2026 08:09:35 -0800 (PST) From: Hans de Goede To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Hans de Goede Subject: [PATCH v2 3/4] software_isp: debayer_cpu: Add multi-threading support Date: Mon, 23 Feb 2026 17:09:29 +0100 Message-ID: <20260223160930.27913-4-johannes.goede@oss.qualcomm.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260223160930.27913-1-johannes.goede@oss.qualcomm.com> References: <20260223160930.27913-1-johannes.goede@oss.qualcomm.com> MIME-Version: 1.0 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMjIzMDEzOSBTYWx0ZWRfX+2BIC4oSdKV+ ls8YZLbjOAL7aVU3sQDWeVa7weM/BbCo5TBtJJNPn6T27VuXmDCg2wagkuM6OaULC/rF/JKdWuZ 5yx98uhcohbtMopb4uWA75eMbWLhVW+JSTk262ZsH/nwSfxbOWa4hMgLeFyzRWwfTJ+c6cZvRlR sN6EOGzCgnvF5rb6fCa0HPBPIwg5nlLa5ESwAa658SJx4Ma5eNbHrjTTxCE/D5aImW7eDSZZeHE Y/FT2ejrXlvdJe79cfh5hGAdUyMclWK7CWmwFt0rEfCz/cDfUJC1LAbn/rtCeHoEURePpjQM0DD 8iiHI3HoJmeqUZi+47q8tvCkjK+hrkGveQoTRufhPRa8V0sfpXuatans+mO2TVRirK9IK1Mlta1 EkfLnq77cgHZb3vN7pGBSLTq2tPhe0RIXE+87cDDVN0whTdhmK04iB0OOUKOs3qZn5/AFg/f4jp 68x+HaN+ACfq17E97zw== X-Authority-Analysis: v=2.4 cv=edYwvrEH c=1 sm=1 tr=0 ts=699c7bc1 cx=c_pps a=HLyN3IcIa5EE8TELMZ618Q==:117 a=rrvG0T/C2D967D07Ol03YQ==:17 a=HzLeVaNsDn8A:10 a=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=u7WPNUs3qKkmUXheDGA7:22 a=gowsoOTTUOVcmtlkKump:22 a=EUspDBNiAAAA:8 a=wcwPUnDRDUmv0zfwghEA:9 a=bTQJ7kPSJx9SKPbeHEYW:22 X-Proofpoint-GUID: VRgBxlBvilEpD_39Ykk9USmFBYyqWytG X-Proofpoint-ORIG-GUID: VRgBxlBvilEpD_39Ykk9USmFBYyqWytG X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293, Aquarius:18.0.1121, Hydra:6.1.51, FMLib:17.12.100.49 definitions=2026-02-23_03,2026-02-23_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 malwarescore=0 phishscore=0 clxscore=1015 priorityscore=1501 spamscore=0 impostorscore=0 bulkscore=0 suspectscore=0 lowpriorityscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2602130000 definitions=main-2602230139 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add CPU soft ISP multi-threading support. Benchmark results for the Arduino Uno-Q with a weak CPU which is good for performance testing, all numbers with an IMX219 running at 3280x2464 -> 3272x2464: 1 thread : 147ms / frame, ~6.5 fps 2 threads: 80ms / frame, ~12.5 fps 3 threads: 65ms / frame, ~15 fps Adding a 4th thread does not improve performance. Signed-off-by: Hans de Goede --- Changes in v2: - Adjust to use the new DebayerCpuThread class introduced in the v2 patch-series - Re-use threads instead of starting new threads every frame --- src/libcamera/software_isp/debayer_cpu.cpp | 53 ++++++++++++++++++++-- src/libcamera/software_isp/debayer_cpu.h | 6 +++ 2 files changed, 55 insertions(+), 4 deletions(-) diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 122bfbb05..ea1b17c1f 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -18,6 +18,8 @@ #include +#include + #include #include "libcamera/internal/bayer_format.h" @@ -50,13 +52,15 @@ public: unsigned int lineBufferIndex_; std::vector lineBuffers_[DebayerCpu::kMaxLineBuffers]; bool enableInputMemcpy_; + Thread worker_; }; DebayerCpuThread::DebayerCpuThread(DebayerCpu *debayer, unsigned int threadIndex, bool enableInputMemcpy) : debayer_(debayer), threadIndex_(threadIndex), - enableInputMemcpy_(enableInputMemcpy) + enableInputMemcpy_(enableInputMemcpy), worker_("DebayerWorker") { + this->moveToThread(&worker_); } /** @@ -88,8 +92,10 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats, const GlobalConfigurat bool enableInputMemcpy = configuration.option({ "software_isp", "copy_input_buffer" }).value_or(true); - /* Just one thread object for now, which will be called inline rather than async */ - threads_.resize(1); + unsigned int threadCount = + configuration.option({ "software_isp", "threads" }).value_or(2); + threadCount = std::clamp(threadCount, 1u, 8u); + threads_.resize(threadCount); for (unsigned int i = 0; i < threads_.size(); i++) threads_[i] = new DebayerCpuThread(this, i, enableInputMemcpy); @@ -714,6 +720,11 @@ void DebayerCpuThread::process(uint32_t frame, const uint8_t *src, uint8_t *dst) process2(frame, src, dst); else process4(frame, src, dst); + + debayer_->workPendingMutex_.lock(); + debayer_->workPending_ &= ~(1 << threadIndex_); + debayer_->workPendingMutex_.unlock(); + debayer_->workPendingCv_.notify_one(); } void DebayerCpuThread::process2(uint32_t frame, const uint8_t *src, uint8_t *dst) @@ -953,7 +964,24 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output stats_->startFrame(frame); - threads_[0]->process(frame, in.planes()[0].data(), out.planes()[0].data()); + workPendingMutex_.lock(); + workPending_ = (1 << threads_.size()) - 1; + workPendingMutex_.unlock(); + + for (unsigned int i = 0; i < threads_.size(); i++) + threads_[i]->invokeMethod(&DebayerCpuThread::process, + ConnectionTypeQueued, frame, + in.planes()[0].data(), out.planes()[0].data()); + + { + MutexLocker locker(workPendingMutex_); + + auto workPending = ([&]() LIBCAMERA_TSA_REQUIRES(workPendingMutex_) { + return workPending_ == 0; + }); + + workPendingCv_.wait(locker, workPending); + } metadata.planes()[0].bytesused = out.planes()[0].size(); @@ -972,6 +1000,23 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output inputBufferReady.emit(input); } +int DebayerCpu::start() +{ + for (unsigned int i = 0; i < threads_.size(); i++) + threads_[i]->worker_.start(); + + return 0; +} + +void DebayerCpu::stop() +{ + for (unsigned int i = 0; i < threads_.size(); i++) + threads_[i]->worker_.exit(); + + for (unsigned int i = 0; i < threads_.size(); i++) + threads_[i]->worker_.wait(); +} + SizeRange DebayerCpu::sizes(PixelFormat inputFormat, const Size &inputSize) { Size patternSize = this->patternSize(inputFormat); diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 7196dcdd0..2c84f8e40 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -16,6 +16,7 @@ #include #include +#include #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/global_configuration.h" @@ -41,6 +42,8 @@ public: std::tuple strideAndFrameSize(const PixelFormat &outputFormat, const Size &size); void process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms); + int start(); + void stop(); SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); const SharedFD &getStatsFD() { return stats_->getStatsFD(); } @@ -147,6 +150,9 @@ private: std::unique_ptr stats_; unsigned int xShift_; /* Offset of 0/1 applied to window_.x */ + unsigned int workPending_ LIBCAMERA_TSA_GUARDED_BY(workPendingMutex_); + Mutex workPendingMutex_; + ConditionVariable workPendingCv_; std::vectorthreads_; };