From patchwork Tue Feb 24 19:37:44 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans de Goede X-Patchwork-Id: 26235 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 708E5C0DA4 for ; Tue, 24 Feb 2026 19:37:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 23DB1622B7; Tue, 24 Feb 2026 20:37:58 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=qualcomm.com header.i=@qualcomm.com header.b="fdb6yK38"; dkim=pass (2048-bit key; unprotected) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b="Gwekh/f7"; dkim-atps=neutral Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 23F9D622A2 for ; Tue, 24 Feb 2026 20:37:53 +0100 (CET) Received: from pps.filterd (m0279873.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 61OFO0WH2432436 for ; Tue, 24 Feb 2026 19:37:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h= cc:content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=qcppdkim1; bh=Dg+H+cXUvva SQunQpgmYy3lZMM8Jgoeiy1NGwXPXAos=; b=fdb6yK38Xrbp5RlwYqLQ1CR/Mpb vfU3Xdl4XhBs6QE9cqpewmgpvDJUEX5YpNyXbdSJx1UTc+4FCebuIK/4NxVAahJ3 lnl9y3Ytqm33bVARB9i0Ynwzan4aMFV9t3BgAi6L6/EJDT55uAr9ehWm8fBQHqFH J4ZxibxnudF2aziwb3H8MaFxDsnlbY/WTaPdiIRJMJuWRk9niEireT9JwiKfHg1u fArjixlkBy+x6Cg+H+2PUk3+4VZb/rbjX4qAFOhX0n5lUTr3++up3ihKOC71XkaI lpKuTxKSyylBHOEmAqWfxOKh3nnSkXskK6q0t6FQgKu7ZvI2rcVSzL907lw== Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4chekj8ygx-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Tue, 24 Feb 2026 19:37:51 +0000 (GMT) Received: by mail-qk1-f200.google.com with SMTP id af79cd13be357-8cb4817f3c8so4472312685a.3 for ; Tue, 24 Feb 2026 11:37:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oss.qualcomm.com; s=google; t=1771961871; x=1772566671; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Dg+H+cXUvvaSQunQpgmYy3lZMM8Jgoeiy1NGwXPXAos=; b=Gwekh/f77ArA7xrXlPmac9BExCao+9ndJBB2X7bbPELTR4vYWKrFOxtFZFMQkgJKmL sun3CM89511a+OlKbO3QZBOoxgNhnBSqwNPb9FA88XJBrOdC6so7VSHv2meLiZt37g/s AmTf69GEc/0KAa3sEMXlPQjWzE3QFMF7+vSzppXnIqaAj7bDYqfELTsSLJnayqRrLTuM xyMiOkJy2p5H00Irk19qFSO9TrPnImqUKNOLwF2Bz55J+QtlUOrJTuf74UFXVA08ZIcA CaTxk/a2zbQiYFW6yqP6K2N9gPbvAlJMHFH54nIWDeRNDyJPbDdU/Em3WtWu/39MWj9i O/zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771961871; x=1772566671; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Dg+H+cXUvvaSQunQpgmYy3lZMM8Jgoeiy1NGwXPXAos=; b=pKj4309VbZ7l/UTf2EQgSlblsMq9vqqUZKW4NT1gibXem341CqYjp34A/K0Pid1oZ8 LwWU2wEORoPbdG2s3jHjSZIcr5ReEX4rtgWfZtNX9zprEn0WOHUXKVGHyDjW4t/Czd3o xiuqceI3lP1cWbbIrNRaTK3FWj5D0BWucErtFA2X56S9L332oCniFpQrj0IsOfqHd6WK 8gTYRNuECZQrssJg0ObSdwd1YITmsCt6eaKW5ICmNB6h0/n5eZ+RsfSENI5uwsfpPnOf 2H8Z5AZXnabdXzHn+w4hldqagMKs3iUTEx5IiEV3LPIUjJX6D2E1DNJKpLo6hLQMPJs8 2JLA== X-Gm-Message-State: AOJu0Ywt6xRfTY1juRL8XEATBwPggOFSl8AmjggBWx79YiBrzow2ylyk Gm6xhnut18P+VKGPHvHdlVVko4oyHNcLkkEPdNTGGpRlVS2Ox5Mt8S+Q0tdfIlq9WjP9J4vFd/T 97leX/PSUYEIFA1E7Fxl2FRmJlHnJtDR9/LKZIYtmJcxMf191yv97VrBpvgPDC9slVtBGwmxSNC iOxWtiGXN5 X-Gm-Gg: AZuq6aIowf2MRuy9t6wXjn4xo5E7AAoWZ/s47P2+cKyL5OH6TqvgEPwdVJPQKemIjaA Q63PCYY3E6KmvAGMiRE3WCrjBpW6EKI+E/3OGGSRcW8B6n7I7k8bBjaxumaiXJD3G2BmZGTtca+ +nJgnFiLqWYQUvaLfyYTglNndwQrBxFuHNVjlzaGotaow32plJUevRBbVKQHhJwxDMDZAr13SEu 3L5G2LUAP8z5oG7wbkaxXyiMc044U+vF8bhhIZKXh6NtTgbA4otwI3sTkDdgP1Qpnll85Y1K2lG JAdV0Pwl2qj5cd7pv5izWCnMOqDDWmW+ihhos25hS15VL+y7tyLVSZ+kNVtObvMyFxiif3R1eQz +CkOWGd38gdEC5LFX/8xu3x24bZiTx0XxECEWBXp9QxN0yK1ZKhmmDzIcMNJxQSMntT4lreZt8Q ON1UH0U6h4kFKdksK7nw+Frlpv2iczUyZcip2K X-Received: by 2002:a05:620a:4449:b0:8cb:4d64:e972 with SMTP id af79cd13be357-8cb8c9cdc56mr1454365385a.9.1771961870574; Tue, 24 Feb 2026 11:37:50 -0800 (PST) X-Received: by 2002:a05:620a:4449:b0:8cb:4d64:e972 with SMTP id af79cd13be357-8cb8c9cdc56mr1454362385a.9.1771961870052; Tue, 24 Feb 2026 11:37:50 -0800 (PST) Received: from shalem (2001-1c00-0c32-7800-5bfa-a036-83f0-f9ec.cable.dynamic.v6.ziggo.nl. [2001:1c00:c32:7800:5bfa:a036:83f0:f9ec]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b9084e8cb3fsm458232866b.48.2026.02.24.11.37.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Feb 2026 11:37:49 -0800 (PST) From: Hans de Goede To: libcamera-devel@lists.libcamera.org, Milan Zamazal Cc: Hans de Goede Subject: [PATCH v3 3/4] software_isp: debayer_cpu: Add multi-threading support Date: Tue, 24 Feb 2026 20:37:44 +0100 Message-ID: <20260224193745.106186-4-johannes.goede@oss.qualcomm.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260224193745.106186-1-johannes.goede@oss.qualcomm.com> References: <20260224193745.106186-1-johannes.goede@oss.qualcomm.com> MIME-Version: 1.0 X-Authority-Analysis: v=2.4 cv=RNe+3oi+ c=1 sm=1 tr=0 ts=699dfe0f cx=c_pps a=hnmNkyzTK/kJ09Xio7VxxA==:117 a=xqWC_Br6kY4A:10 a=HzLeVaNsDn8A:10 a=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=u7WPNUs3qKkmUXheDGA7:22 a=rJkE3RaqiGZ5pbrm-msn:22 a=EUspDBNiAAAA:8 a=uyevB83Nksalfc49Bg4A:9 a=PEH46H7Ffwr30OY-TuGO:22 X-Proofpoint-ORIG-GUID: WgMGUl3KVxKeQaaHLsIkBcIFUhhhTmKC X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMjI0MDE2OCBTYWx0ZWRfX58vuduKb+Msb PxOk93PCPgDqqxDNKvWjBq+OcJ6f7Aj0bM+jVC5JOZTZXfUknlW64l8+tq0sXXZBUOOKlybnqqc 2oPsaw+zkyM4mfX7XfzNcVLOYeX5DfGCr2dpvfXjeUYlnLi0PEjO69h+mSXeEfLuSYWrWTR/TxX dssyaUFFXdiovtIlVKNxh1cTlvU9cFGRbc5y3Tfl1IvS9gVUPF8pt4lhIfzmzJc0nL5tbvXrzLi PtrOo4nVEnx2pbi97syxqVQtWUOL3ITYpjD578aLlUMo64EOyO9K0alMcUoGlRdwVbnrHnsS1H/ ywL8yhu8n0jh+6s3HMhe9Mm8jG1CGYM35voLllAzFpdSedYG9hEOlGFaLlYvj9Qn0es5C7AQNXR F9xAwCo24iq8SKA5Bp6q/hLTN9VIXT5e7r5C+1EP13TqEOKCayKSlmaZWciyyPz7v6xzJWlp1m1 urzUcoJcO93uLeLg6+w== X-Proofpoint-GUID: WgMGUl3KVxKeQaaHLsIkBcIFUhhhTmKC X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293, Aquarius:18.0.1121, Hydra:6.1.51, FMLib:17.12.100.49 definitions=2026-02-24_02,2026-02-23_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 lowpriorityscore=0 adultscore=0 priorityscore=1501 phishscore=0 clxscore=1015 spamscore=0 impostorscore=0 suspectscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2602130000 definitions=main-2602240168 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add CPU soft ISP multi-threading support. Benchmark results for the Arduino Uno-Q with a weak CPU which is good for performance testing, all numbers with an IMX219 running at 3280x2464 -> 3272x2464: 1 thread : 147ms / frame, ~6.5 fps 2 threads: 80ms / frame, ~12.5 fps 3 threads: 65ms / frame, ~15 fps Adding a 4th thread does not improve performance. Signed-off-by: Hans de Goede --- Changes in v3: - Adjust for DebayerCpuThread now inheriting from Thread - Use for (auto &thread : threads_) Changes in v2: - Adjust to use the new DebayerCpuThread class introduced in the v2 patch-series - Re-use threads instead of starting new threads every frame --- src/libcamera/software_isp/debayer_cpu.cpp | 45 ++++++++++++++++++++-- src/libcamera/software_isp/debayer_cpu.h | 6 +++ 2 files changed, 48 insertions(+), 3 deletions(-) diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 36b7881b..cf16f00b 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -73,6 +73,7 @@ DebayerCpuThread::DebayerCpuThread(DebayerCpu *debayer, unsigned int threadIndex debayer_(debayer), threadIndex_(threadIndex), enableInputMemcpy_(enableInputMemcpy) { + moveToThread(this); } /** @@ -104,8 +105,10 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats, const GlobalConfigurat bool enableInputMemcpy = configuration.option({ "software_isp", "copy_input_buffer" }).value_or(true); - /* Just one thread object for now, which will be called inline rather than async */ - threads_.resize(1); + unsigned int threadCount = + configuration.option({ "software_isp", "threads" }).value_or(2); + threadCount = std::clamp(threadCount, 1u, 8u); + threads_.resize(threadCount); for (unsigned int i = 0; i < threads_.size(); i++) threads_[i] = std::make_unique(this, i, enableInputMemcpy); @@ -743,6 +746,11 @@ void DebayerCpuThread::process(uint32_t frame, const uint8_t *src, uint8_t *dst) process2(frame, src, dst); else process4(frame, src, dst); + + debayer_->workPendingMutex_.lock(); + debayer_->workPending_ &= ~(1 << threadIndex_); + debayer_->workPendingMutex_.unlock(); + debayer_->workPendingCv_.notify_one(); } void DebayerCpuThread::process2(uint32_t frame, const uint8_t *src, uint8_t *dst) @@ -982,7 +990,21 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output stats_->startFrame(frame); - threads_[0]->process(frame, in.planes()[0].data(), out.planes()[0].data()); + workPendingMutex_.lock(); + workPending_ = (1 << threads_.size()) - 1; + workPendingMutex_.unlock(); + + for (auto &thread : threads_) + thread->invokeMethod(&DebayerCpuThread::process, + ConnectionTypeQueued, frame, + in.planes()[0].data(), out.planes()[0].data()); + + { + MutexLocker locker(workPendingMutex_); + workPendingCv_.wait(locker, [&]() LIBCAMERA_TSA_REQUIRES(workPendingMutex_) { + return workPending_ == 0; + }); + } metadata.planes()[0].bytesused = out.planes()[0].size(); @@ -1001,6 +1023,23 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output inputBufferReady.emit(input); } +int DebayerCpu::start() +{ + for (auto &thread : threads_) + thread->start(); + + return 0; +} + +void DebayerCpu::stop() +{ + for (auto &thread : threads_) + thread->exit(); + + for (auto &thread : threads_) + thread->wait(); +} + SizeRange DebayerCpu::sizes(PixelFormat inputFormat, const Size &inputSize) { Size patternSize = this->patternSize(inputFormat); diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 1074bc9c..eb52f101 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -16,6 +16,7 @@ #include #include +#include #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/global_configuration.h" @@ -41,6 +42,8 @@ public: std::tuple strideAndFrameSize(const PixelFormat &outputFormat, const Size &size); void process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, const DebayerParams ¶ms); + int start(); + void stop(); SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); const SharedFD &getStatsFD() { return stats_->getStatsFD(); } @@ -147,6 +150,9 @@ private: std::unique_ptr stats_; unsigned int xShift_; /* Offset of 0/1 applied to window_.x */ + unsigned int workPending_ LIBCAMERA_TSA_GUARDED_BY(workPendingMutex_); + Mutex workPendingMutex_; + ConditionVariable workPendingCv_; std::vector>threads_; };