From patchwork Wed Oct 28 20:43:01 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans de Goede X-Patchwork-Id: 28659 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B70F0C3261 for ; Wed, 28 Oct 2026 20:43:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 701EF69081; Wed, 28 Oct 2026 21:43:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=qualcomm.com header.i=@qualcomm.com header.b="GYEG46cS"; dkim=pass (2048-bit key; unprotected) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b="GQLnktFe"; dkim-atps=neutral Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0B41F68FC6 for ; Wed, 28 Oct 2026 21:43:09 +0100 (CET) Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 69SIPjZK848561 for ; Wed, 28 Oct 2026 20:43:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h= cc:content-transfer-encoding:date:from:message-id:mime-version :subject:to; s=qcppdkim1; bh=VA9DNybN5jlFpdl+dQTGUR0dvgmE4048Ee9 QkLyJGyE=; b=GYEG46cSqDjDbs/jiElJjYpMg8WVLUN2XA4TDKYUyXuBEbypqlM /brghotmU91S9XJGdmzPcVVxjEx6zpRrmvYpudaD96+WvemXGuXilv6OOdx//HN6 7NMylX8A2Y6k/A8SiKWYmuWRJUzListOAH0niVAETQzGF4mC5coVJZAzVjpBGB9U W0/oQ6bc2rk6ZW0XXcIhZNatXI8HIQXJU+xgPqyYFaxnzOv9vvjazfG4x34X7UPN i2hJr65SVLPR7ZIRBxd8qe2CRCJDCHvMMVLoplfHB5Z7L77gwwl1RqbRAhXzY8XV bEzHFk9M13WDl0WpWRRdgE1N4w82bzUHleQ== Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4hkdsp2tuw-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Wed, 28 Oct 2026 20:43:08 +0000 (GMT) Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-91c71966998so19689586d6.1 for ; Wed, 28 Oct 2026 13:43:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oss.qualcomm.com; s=google; t=1793220185; x=1793824985; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to:content-type; bh=VA9DNybN5jlFpdl+dQTGUR0dvgmE4048Ee9QkLyJGyE=; b=GQLnktFe88skw5x4MR5fhPTSrqZNPV7cGlV9SrwRbrp80odf4cmD15bpixHmTQh3Ue iihNu//6nwBbEaSPL2zx1qhkauo574DwMdhtIKuCkEKT0xLf9p1aqjaECq5Tot3j0+MG 2Thu9eikDQFK0uxZWubxBQI1J3vr5D1oTgceXc5pU9kzvAoGWtjlutZj6scRXJDTE5wO lI2zRiu93rV7tvc48X9nyimS3rKqnHtk+zv1fKEgZfhWvYxk3DdqjyBDuq/jT5UyLIFy VTiJEb8bpp4TDBylrmvgauEZoZgcLaa19CF1qWEjNmopHM/L4iyEx+bPlCDj2aAf6Emi aMyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1793220185; x=1793824985; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=VA9DNybN5jlFpdl+dQTGUR0dvgmE4048Ee9QkLyJGyE=; b=j6821Vccvlm8rzZxBUJNFAoUBR/oAijDESXtH9vJdsOZIQi23UzeEttn96y8/dc/MR /cm9X19L0kEL9tI9/NAYSrAzLNa59y3hxseCQQ03abLTxbkVBbPl7u9NcyKraXt9lDoP xKD+KfYPokxUWWAGIDHqHThF3ZIixl4KgApiNbZ6gR5GwJduHImVHHC6ivvjP6jQm/b/ zxjfkOqGjAi8yzdkp3rviC0ycjt16e6YFArc+nHrrk6XJq6i9+qSylK7vkwAQvZMXePV vaMuyks5tMUUWUUcxC18xIOMd1V6HhY329a51oSHTGv9eGp8esbhRYSlAHCQkkEoM3nn mzNA== X-Gm-Message-State: AFq9FYLkHtsnq9MdOZVPVIAQuL1zwbu8TGFRETzbAnxD+bYOVEALOmQs pNuziit2pAgHX9wtdeLAw07jqc+ql4L5dcv+zCCCBV55HP2anwMg7sKmr8ZFbaeuMfnNkwGrtvv WfBfUFC5SaZP3tMGRiUROCtAqp7d/x5Uq3Z6EGHxR45FcayWbb8GFI93jtcBtKvZ2bL40l0/NMY pomvFLRMx7 X-Gm-Gg: AYBFou2+FWQgsil+33m7mdf5sSvrRyKZIk8JkgjTYzWCoJ3E6nCuL4b1H+lEhd5MFLP 7bCZ7uSRvIj4G9MRXI8pVl/NWLcAUzov0IOpR5QA6FdOgLzO3uKckxjAU5vCXw4NBuc3Hg7poDf hkZqQFb9R0GClassShrKLonH+tdrL60Eqa9+0dsefitpqP2JpRe7iBVBQHa7sjvRV9DxifdAcwX FY87cqVnbzMPRukhmfd7YGDcTnw/OXrV12FZfCqondcbEN4ExjNjKofWuVMyGIkC/fqlUMP4kQW ldV/jY1doyqhoCsz4NKPSMPJWPg0qWPNvhhV2cQJb/f340d0Q/07W4ddEqoqRcQP2edPpGaElPq t2KhY0vgtQ51/6irB2uHjt/L1cQK0elKCqrm8xtQoc9+EUyR1QPCIXOKPaPRYvzgl98JokvlzRv bCeu/DQ5Fa2Qjxo+9BFKY0W3Eq4U906aUENt5ESv1CDDyyv8Vr456suMpvn+tu+sgV X-Received: by 2002:a05:620a:4723:b0:93e:c12c:fbcf with SMTP id af79cd13be357-94408d56345mr516300785a.75.1793220185330; Wed, 28 Oct 2026 13:43:05 -0700 (PDT) X-Received: by 2002:a05:620a:4723:b0:93e:c12c:fbcf with SMTP id af79cd13be357-94408d56345mr516295985a.75.1793220184535; Wed, 28 Oct 2026 13:43:04 -0700 (PDT) Received: from shalem (2001-1c00-0c32-7800-5bfa-a036-83f0-f9ec.cable.dynamic.v6.ziggo.nl. [2001:1c00:c32:7800:5bfa:a036:83f0:f9ec]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-6b496435d40sm1831625a12.27.2026.10.28.13.43.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Oct 2026 13:43:02 -0700 (PDT) From: Hans de Goede To: libcamera-devel@lists.libcamera.org, Kieran Bingham Cc: Hans de Goede Subject: [PATCH v2] libcamera: pipeline: simple: Prefer CSI2 packed bayer formats over unpacket formats Date: Wed, 28 Oct 2026 21:43:01 +0100 Message-ID: <20261028204301.152898-1-johannes.goede@oss.qualcomm.com> X-Mailer: git-send-email 2.56.0 MIME-Version: 1.0 X-Proofpoint-GUID: SonE3J7BuEa9YbsVdwAqRK1Up9du0pfx X-Proofpoint-ORIG-GUID: SonE3J7BuEa9YbsVdwAqRK1Up9du0pfx X-Authority-Analysis: v=2.4 cv=Cssq4X4D c=1 sm=1 tr=0 ts=6ae25e5c cx=c_pps a=oc9J++0uMp73DTRD5QyR2A==:117 a=xqWC_Br6kY4A:10 a=660iZSQnnn4A:10 a=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=u7WPNUs3qKkmUXheDGA7:22 a=yx91gb_oNiZeI1HMLzn7:22 a=e5mUnYsNAAAA:8 a=EUspDBNiAAAA:8 a=1czY_G0o6ylMawb2oE0A:9 a=iYH6xdkBrDN1Jqds4HTS:22 a=Vxmtnl_E_bksehYqCbjh:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYxMDI4MDA4MiBTYWx0ZWRfX8nGH7ZtWVchv MelaVjZ/h1rcXuwnf9SjLDQyI9Gjfv3ECd2337cuNn0vp+ZvRqtyU3nqhT0c+1Og0r+4FhUtNG9 ESKiSEcx2XJmby95AHzDkHVlclLIIKD+qjQwMTrbyCOHxDqSOGrtYm2/ivaB8wwqhRl4R27+cOJ 7kuxz3smh9WrKFaudk/MF2YDUBIyAYXO0SgxKYZWJFC0n6e5+TrxrFjCzfRFwOQtPYv6uBzqv5t g/W3DiNxaktdaYdFyA52M+mSgeNBTz97LWpPLR6BFc+XETf83JzjNSgPWjSkmZsBun+mxAkI3Cw Ws6Q5z6AFP5cIaDOo6XMsXm3Zjldr3Y0Ff9aTC2QidCuPwcEpwhKoEu13PglkRYERUPyZtRs5+o RqgEFQ3doOmHFJpuhO0irCW8hE4ZqPmMN/GZOIc1KmhTtojKBaiCTNsCm/6yj2gqYhZubUAoQH1 /o+WmEOAN+2VcZW6D5A== X-Proofpoint-Spam-Info: AW1haW4tMjYxMDI4MDA4MiBTYWx0ZWRfX64wfN2XbFzXc Nm9ZYRUjp3qErylbk5VmwGf4NfFMk3x3xeiO73p1mAJG0WBVZVylJ8UAKaRMIp7jn52DE6+G5dK GByRyPB3EwXQR6/xVDcWFcwIOKyGJuY= X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293, Aquarius:18.0.1176, Hydra:6.1.138, FMLib:17.12.100.49 definitions=2026-10-28_06,2026-10-21_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 lowpriorityscore=0 priorityscore=1501 adultscore=0 impostorscore=0 malwarescore=0 bulkscore=0 phishscore=0 clxscore=1015 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2610190000 definitions=main-2610280082 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Some CSI2 receivers like the IPU6 isys can store e.g. 10 bpp Bayer formats in either CSI2 packed (5 bytes / 4 pixels) or unpacked (2 bytes / pixel) formats. Before this change the simple-pipeline handler would pick whatever format was enumerated first as capture format in this case. Which on IPU6 boards comes down to picking unpacked. Packed formats use less memory and typically perform better due to reduced RAM bandwidth requirements. E.g. on a Lenovo ThinkPad X1 Yoga Gen 8 with i7-1370P CPU and a ov2740 sensor running at full sensor (FHD) resolution, 3 consecutive benchmark runs using libcamera's software ISP's builtin Debayer benchmark gives the following results: Unpacked EGL us/frame: 6398 / 7393 / 7524 CSI2 pack EGL us/frame: 5682 / 5536 / 5634 Unpacked CPU us/frame: 7653 / 8147 / 8614 CSI2 pack CPU us/frame: 7183 / 7063 / 6318 Note the EGL comparison is not entirely fair, the unpacked debayer shader uses the "McGuire" Debayer algorithm where as the packed one uses simpler linear filtering. Modify the simple pipeline handler to prefer the more performant CSI2 packed format when both formats are available. This also fixes a problem with unpacked formats on IPU6 where sometimes at the start of a stream the IPU6 isys CSI2 receiver may return a corrupt unpacked frame with high bits set which causes libcamera to crash with an out of bound array access as the software ISP code expects the high bits of unpacked formats to always be 0. Closes: https://gitlab.freedesktop.org/camera/libcamera/-/work_items/328 Signed-off-by: Hans de Goede --- Changes in v2: - Add note to the commit message about the EGL benchmark comparison not being fair due to Debayer algorithm diferences - Extend the comment to mention that this works around a bug on ipu6/7 so that if the decision is made in the future to prefer unpacked it will be clear to the person making such a change this might be a problem --- src/libcamera/pipeline/simple/simple.cpp | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 35c29ceca..a53f1e152 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1216,6 +1216,14 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * Then pick, among the possible configuration for the pixel format, * the smallest sensor resolution that can accommodate all streams * without upscaling. + * + * If both packed and unpacked Bayer capture formats are support prefer + * packed for lower memory usage and better software ISP performance. + * + * Note preferring packed also works around a bug on ipu6/7 setups where + * the CSI receives sometimes hands over corrupt frames. With unpacked + * formats this causes OOB array accesses in the software ISP, due to + * some of the unused high bits being 1 in these corrupt frames. */ const SimpleCameraData::Configuration *maxPipeConfig = nullptr; pipeConfig_ = nullptr; @@ -1223,12 +1231,25 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() for (const SimpleCameraData::Configuration *pipeConfig : *configs) { const Size &captureSize = pipeConfig->captureSize; const Size &maxOutputSize = pipeConfig->outputSizes.max; + bool curCfgPacked = false; + bool newCfgPacked = false; + + if (pipeConfig_) { + BayerFormat curBayerFmt = BayerFormat::fromPixelFormat(pipeConfig_->captureFormat); + BayerFormat newBayerFmt = BayerFormat::fromPixelFormat(pipeConfig->captureFormat); + /* Only compare packed vs unpacked if both formats are Bayer */ + if (curBayerFmt.isValid() && newBayerFmt.isValid()) { + curCfgPacked = curBayerFmt.packing == BayerFormat::Packing::CSI2; + newCfgPacked = newBayerFmt.packing == BayerFormat::Packing::CSI2; + } + } if (maxOutputSize.width >= maxProcessedStreamSize.width && maxOutputSize.height >= maxProcessedStreamSize.height && captureSize.width >= maxRawStreamSize.width && captureSize.height >= maxRawStreamSize.height) { - if (!pipeConfig_ || captureSize < pipeConfig_->captureSize) + if (!pipeConfig_ || captureSize < pipeConfig_->captureSize || + (captureSize <= pipeConfig_->captureSize && !curCfgPacked && newCfgPacked)) pipeConfig_ = pipeConfig; }