From patchwork Wed Oct 28 21:43:07 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans de Goede X-Patchwork-Id: 28660 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F0AF7C324C for ; Wed, 28 Oct 2026 21:43:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A037069086; Wed, 28 Oct 2026 22:43:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=qualcomm.com header.i=@qualcomm.com header.b="pVRjv5JG"; dkim=pass (2048-bit key; unprotected) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b="S1zDgnOT"; dkim-atps=neutral Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8679E6907C for ; Wed, 28 Oct 2026 22:43:13 +0100 (CET) Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 69SIjEMb3022004 for ; Wed, 28 Oct 2026 21:43:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h= cc:content-transfer-encoding:date:from:message-id:mime-version :subject:to; s=qcppdkim1; bh=FTBCJHXGl09jK/Lw4lC3GU5f8RAXzu9pl1g TM6oFPwI=; b=pVRjv5JGx8ssvcDlwk2hnRlR1DuaTAa4vhmmvTSQetUT/GJnf46 MPh7VkzWjJWibSnVP8CXL+perOU0HhAE73xPbC+VigMbRZSB7r3grKkiFkHvd3Jz RUA508N9eGUgkal62bS6/o3LPtkGfYWE+FfTi/qTcW9nahAVjlAb+3+1H/JiYtWS yUDxqklh/5tT/UzuF8CE/jyrr3H7Q9ibWXhwjU4KVfrSxZQY7Rwbs0w4sVSO/fbN mGnuP5nrpPAzItCnv524IHVICnGe8lDbTMPyAZmLyuN7OU/YTfFZFiJxXGCiguhL ZZXUkFQc4mhwgueJHBraorGlZSsmgAogXtg== Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4hkkehskuc-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Wed, 28 Oct 2026 21:43:10 +0000 (GMT) Received: by mail-qv1-f72.google.com with SMTP id 6a1803df08f44-91c01778cefso30376706d6.2 for ; Wed, 28 Oct 2026 14:43:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oss.qualcomm.com; s=google; t=1793223790; x=1793828590; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to:content-type; bh=FTBCJHXGl09jK/Lw4lC3GU5f8RAXzu9pl1gTM6oFPwI=; b=S1zDgnOTNFtqEw/T7W+Fe0531Hu8YEtK0gYDl14AQo+XTYr3JIkuVJwUT4bDAx3iRU imwzHFmOR3hIWosHQYozt7DPmsEogh0beDTOdbOdRVSGMCZK9CnJcw3BLTiqhzmGek38 DOEvzBz8k5enAydy0cs0FYONo5eTKCOlnqFSGHB2Tvijyr8YlM6VX8Rt1uv2rZG5TwiR p/WBom0grTo+5+G4/lAZ9pjuKkiRFOZu6Loj96f20TaEWQcEjQoWBRuJyPVT2gfKnS2L fg6g7jW3gjvTx88je8kQyGPSMhv78CC/5xRS7xHhebNhBN15eKq/m54oHHhzftJZ2cQV k7jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1793223790; x=1793828590; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=FTBCJHXGl09jK/Lw4lC3GU5f8RAXzu9pl1gTM6oFPwI=; b=TwzaJPO7Lb0xvTxDx1IT7Rt2IbzRux8Mbvc/U1W1L1uiP/U4aB1GkRlAm3OzxE1HNI STFFCgvHPnmblfeJ+btBnZ6xPK0VgxlYfiPaqVxOSHmFXAeEoPUFixefOaEw1IoVRoIL GBD+j7e5oqtNyUMdEu0FpifyMjEV7fO1iC1qZD1t8VTRZPJxgTY1gDPUjIHXUYJt451c r8H6x8vg6NOOepAGT6IKgbGPoKBeQxt9lXYsTpAsPR0fz9qzOIjbZ1dShjtaigNGmRoC KRGNQx/JWI5bTCuUqdETREO9p7RL1UOn+dlZB+HN9HVUk1vx0wf4JLo4Yj77VmT5nLVD k9tA== X-Gm-Message-State: AFq9FYLS+TpmRt2CUu7iXa7Yp0YwG4heOucAHr2DSK3EZxZS7zCDrqpc gzENFmkl8lvEW4YKb+aGt1LMr1mFD+a9H56+T+JNpQYYdhgg0jjWpMqdmE+4LV5SQeyViZ8zGW1 m1euW0V3HUpHSwYmW9fQHTNC2nJiX1kdyUN+yuL3jKgH2NWKPQd8QkhGtMnhn5dNUUTtVXQWuFt dPdjICfeV7 X-Gm-Gg: AYBFou3ES2yeB9Hmqmcl5uYq9y2/bsj2AjeBOESeimVRhh2wqba/uW8rMu1kAHc9oGL m7v8xZHLvSiGILOi7L+KGOb0CajiroysfeSL42RxNnHB+9e74znQlyzQ8f9OQagfROfMkXW9n7G eP1iWnJi5gGHWB5a/FgFDfk4GOAMr41iFew8bqF9akDgUEO0HFua66g8u7HNXzEw818/oCD9V7I mNSaD6Fk0naZZkt6zqVnt74/FIids27weaxoeNVkhIXFIkl/H8LJGX5o4DATHwjF8tfyYKdALid JeRbG460Z+kFOyHJ5tMfE2mvul3+gYoveTG0sU88Ls0a+nCLxSmpwXFYi0znoIun7hFkf9NPMsT ddEmI8W/SAR5A0TpsBzsxutiJs/6BDumeniHu9LZLTLq3ATnJ3OmChFXWW+ASqmBEQVukFUVf8N Ppiy2CiedsLck5xHVtUX7lfYba8T7+TAaSEWMJp5MFOikoxPWJYe6O1T1U/RecFXUB X-Received: by 2002:a05:620a:2629:b0:93f:735:6c83 with SMTP id af79cd13be357-94408d6a291mr528409185a.56.1793223789509; Wed, 28 Oct 2026 14:43:09 -0700 (PDT) X-Received: by 2002:a05:620a:2629:b0:93f:735:6c83 with SMTP id af79cd13be357-94408d6a291mr528405385a.56.1793223788953; Wed, 28 Oct 2026 14:43:08 -0700 (PDT) Received: from shalem (2001-1c00-0c32-7800-5bfa-a036-83f0-f9ec.cable.dynamic.v6.ziggo.nl. [2001:1c00:c32:7800:5bfa:a036:83f0:f9ec]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c46bd7abf6asm647141366b.81.2026.10.28.14.43.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Oct 2026 14:43:08 -0700 (PDT) From: Hans de Goede To: libcamera-devel@lists.libcamera.org, Kieran Bingham Cc: Hans de Goede Subject: [PATCH v3] libcamera: pipeline: simple: Prefer CSI2 packed bayer formats over unpacket formats Date: Wed, 28 Oct 2026 22:43:07 +0100 Message-ID: <20261028214307.158415-1-johannes.goede@oss.qualcomm.com> X-Mailer: git-send-email 2.56.0 MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: ezDIfqfkxWbjAj6bP02QkWyXlxh3EaoD X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYxMDI4MDA4NiBTYWx0ZWRfXzrubIbzMqgI6 c+8BSUj1SD5JTkN8yHCcmBwWssUDtaaQIW5rG1smko4wLIugYYtkI2oiz0zyUkAYkTpdOb4szm+ pAx0vaTjMKMVn2eZmf2r7+MW/bLD7lVVmXTYTnAC16VJs1U+3SlAaBizaCRIKr9Kq4zVDi76+fz 4OAyIBymZoUgUgkupcxasroFJMgromjKTaQZjEUnNjutfx+LKF3Rqn8rzUsQnYHrguqCxGA8qbP jXn10hDqJuwRELRJrSBSGn87yp6423XV18DSfjlQvftT9CWAdiRQ5pbHWmxJUuwocMss2x1w2xA SD5XxytmFG+reroWX2jIb+btYpxHYqtN7g7hCfGjYBFyDEWvOUD/DZMhtmbcXzWUdsuGq5yOw+D LobEtqaWZXCz+U3DXuLWay+natuTpn88XrWUoRwp46Bbiosq+wf+Xqai3/NfRyInjE9klJZcUls zbxPqfvARqm4UuD58Mw== X-Proofpoint-GUID: ezDIfqfkxWbjAj6bP02QkWyXlxh3EaoD X-Proofpoint-Spam-Info: AW1haW4tMjYxMDI4MDA4NiBTYWx0ZWRfX12Sv+33lqaMc KPkuMVou5NHZA5VN6VIQqmymgZrXuFGFGrvAkqrCVySGVweOxKgKqlAAdzA57IGCATir7qUIe+J nnQ7alLhRDJhBriNa3LJFL6OBVEhkaI= X-Authority-Analysis: v=2.4 cv=bYv+w+PB c=1 sm=1 tr=0 ts=6ae26c6e cx=c_pps a=7E5Bxpl4vBhpaufnMqZlrw==:117 a=xqWC_Br6kY4A:10 a=660iZSQnnn4A:10 a=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=u7WPNUs3qKkmUXheDGA7:22 a=eoimf2acIAo5FJnRuUoq:22 a=e5mUnYsNAAAA:8 a=EUspDBNiAAAA:8 a=1czY_G0o6ylMawb2oE0A:9 a=pJ04lnu7RYOZP9TFuWaZ:22 a=Vxmtnl_E_bksehYqCbjh:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293, Aquarius:18.0.1176, Hydra:6.1.138, FMLib:17.12.100.49 definitions=2026-10-28_06,2026-10-21_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 impostorscore=0 malwarescore=0 bulkscore=0 spamscore=0 phishscore=0 lowpriorityscore=0 clxscore=1015 adultscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2610190000 definitions=main-2610280086 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Some CSI2 receivers like the IPU6 isys can store e.g. 10 bpp Bayer formats in either CSI2 packed (5 bytes / 4 pixels) or unpacked (2 bytes / pixel) formats. Before this change the simple-pipeline handler would pick whatever format was enumerated first as capture format in this case. Which on IPU6 boards comes down to picking unpacked. Packed formats use less memory and typically perform better due to reduced RAM bandwidth requirements. E.g. on a Lenovo ThinkPad X1 Yoga Gen 8 with i7-1370P CPU and a ov2740 sensor running at full sensor (FHD) resolution, 3 consecutive benchmark runs using libcamera's software ISP's builtin Debayer benchmark gives the following results: Unpacked EGL us/frame: 6398 / 7393 / 7524 CSI2 pack EGL us/frame: 5682 / 5536 / 5634 Unpacked CPU us/frame: 7653 / 8147 / 8614 CSI2 pack CPU us/frame: 7183 / 7063 / 6318 Note the EGL comparison is not entirely fair, the unpacked debayer shader uses the "McGuire" Debayer algorithm where as the packed one uses simpler linear filtering. Modify the simple pipeline handler to prefer the more performant CSI2 packed format when both formats are available. This also fixes a problem with unpacked formats on IPU6 where sometimes at the start of a stream the IPU6 isys CSI2 receiver may return a corrupt unpacked frame with high bits set which causes libcamera to crash with an out of bound array access as the software ISP code expects the high bits of unpacked formats to always be 0. Closes: https://gitlab.freedesktop.org/camera/libcamera/-/work_items/328 Signed-off-by: Hans de Goede --- Changes in v3: - s/support/supported/ in the comment Changes in v2: - Add note to the commit message about the EGL benchmark comparison not being fair due to Debayer algorithm diferences - Extend the comment to mention that this works around a bug on ipu6/7 so that if the decision is made in the future to prefer unpacked it will be clear to the person making such a change this might be a problem --- src/libcamera/pipeline/simple/simple.cpp | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 35c29ceca..5f7e21463 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1216,6 +1216,15 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * Then pick, among the possible configuration for the pixel format, * the smallest sensor resolution that can accommodate all streams * without upscaling. + * + * If both packed and unpacked Bayer capture formats are supported + * prefer packed for lower memory usage and better software ISP + * performance. + * + * Note preferring packed also works around a bug on ipu6/7 setups where + * the CSI receives sometimes hands over corrupt frames. With unpacked + * formats this causes OOB array accesses in the software ISP, due to + * some of the unused high bits being 1 in these corrupt frames. */ const SimpleCameraData::Configuration *maxPipeConfig = nullptr; pipeConfig_ = nullptr; @@ -1223,12 +1232,25 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() for (const SimpleCameraData::Configuration *pipeConfig : *configs) { const Size &captureSize = pipeConfig->captureSize; const Size &maxOutputSize = pipeConfig->outputSizes.max; + bool curCfgPacked = false; + bool newCfgPacked = false; + + if (pipeConfig_) { + BayerFormat curBayerFmt = BayerFormat::fromPixelFormat(pipeConfig_->captureFormat); + BayerFormat newBayerFmt = BayerFormat::fromPixelFormat(pipeConfig->captureFormat); + /* Only compare packed vs unpacked if both formats are Bayer */ + if (curBayerFmt.isValid() && newBayerFmt.isValid()) { + curCfgPacked = curBayerFmt.packing == BayerFormat::Packing::CSI2; + newCfgPacked = newBayerFmt.packing == BayerFormat::Packing::CSI2; + } + } if (maxOutputSize.width >= maxProcessedStreamSize.width && maxOutputSize.height >= maxProcessedStreamSize.height && captureSize.width >= maxRawStreamSize.width && captureSize.height >= maxRawStreamSize.height) { - if (!pipeConfig_ || captureSize < pipeConfig_->captureSize) + if (!pipeConfig_ || captureSize < pipeConfig_->captureSize || + (captureSize <= pipeConfig_->captureSize && !curCfgPacked && newCfgPacked)) pipeConfig_ = pipeConfig; }