From patchwork Wed Nov 12 09:09:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans de Goede X-Patchwork-Id: 25007 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A74BAC32DB for ; Wed, 12 Nov 2025 09:09:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C2E5360A80; Wed, 12 Nov 2025 10:09:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=qualcomm.com header.i=@qualcomm.com header.b="aPBPUolI"; dkim=pass (2048-bit key; unprotected) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b="VXvSd/Cw"; dkim-atps=neutral Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 20B66609D8 for ; Wed, 12 Nov 2025 10:09:36 +0100 (CET) Received: from pps.filterd (m0279868.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 5AC1iuTf4078255 for ; Wed, 12 Nov 2025 09:09:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h= cc:content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=qcppdkim1; bh=yWkX+wFbzlV IzdVuQeKrJhvTcMMtgyiYzvzYlPJBMU8=; b=aPBPUolI3xPkApUkw1rj47FFw5y kNptog/o07WA4nE39QuKusLls+V8YkzFIjXdMRNoOW4NmbFEUR7FCfjo+OF0UwQ+ QUvjr4kkrfHmkivjxufpIMjczaF3HIGwM0ZPrnb9qensU1Jcv09ANQTNdjqwJ1mV eG+I5aZkKYnWsaF74yhCgOud7USbP9J15BUerLEdtXU9uqPf9bPCRAMSy6X6co/G 8C2BgqZ3lo4HtaxCBE6MFwbRQWCPbdvg63XAX43AaNDtnBB4yGEpjQALXi9nSiHu vLWcWpXXIsiHhD6Af0STu9WJY1HQBtexCNz14b9c8tFpBZxlbuHi+6+Gm/Q== Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4acguah518-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Wed, 12 Nov 2025 09:09:34 +0000 (GMT) Received: by mail-qt1-f200.google.com with SMTP id d75a77b69052e-4edb84fc9bbso15924371cf.0 for ; Wed, 12 Nov 2025 01:09:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oss.qualcomm.com; s=google; t=1762938573; x=1763543373; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yWkX+wFbzlVIzdVuQeKrJhvTcMMtgyiYzvzYlPJBMU8=; b=VXvSd/CwizG5Yetod06WCNBQs2d6AE25T39EIITeaoP/ZxsBbfecZ7HP+02mVH2nCN qC51xjTzzuf9iFsCMLFpf02sQirwyTJkG29kJ53hR7Kkb1q3IqR+cRSJ5wszXfz/nPuJ WVRv9GGh+cT1lwT91V5viLrIjOuNV0zuMriUjP4Bl1kom3vFCRigGV5WvkUONOsYrJ88 CSRAktqfW6SFoDahYWXmSLPp2I/jAoH4WnwIBTCL+qxyehebp0ybA1myW0a17MfOJijv qRimTbWVCa6fefAW4dDRA+FzDYMNhdCj4oJEyMd4P74blxJFZE02gf8FlVhhXuCtNdj0 Qz8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762938573; x=1763543373; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=yWkX+wFbzlVIzdVuQeKrJhvTcMMtgyiYzvzYlPJBMU8=; b=try7f1Gk2MO4tLhV7yp0dj73ejxJunUQYYmV6cpdgTxxwkQPngm3j2yyIEfYHPqgNT kgOj7hUGVEDxxL+Iyy3Gn5yg3AmtxgvB7pbm33pfkCRXTDaSL9aqwdVgsAEW0sYRLzaF hmBizRuEEiqbTiuHQnwRygBo31XiDbJ6fU4wO8t/B3WZ9D2LhkH4QlPLn2uRlzHlVFMa gTeK+nOpmEi5nJefkNzzJc7kp6nolNkUne36nrePtpg2vCkBRacGR8lPRKWpVqv/C2d3 OZSw8gW7yaIieTjd9Ji8DEjswVw0pek+y4E1eaDHZDNbrCksZCREQLcuJjw4QslhqNNa R0jw== X-Gm-Message-State: AOJu0YytooIIdCvDoARqfNPksKzZI9q1TXZ2iHftD9sEhH+0//M2h/K8 95IrCFhRl1oG2DC4+DBCyCGucroY2Df5aVztjVZj0QhMa1hDq4U1TdZNhUweUM3XAuoOgOY+jxJ Dg6ius8gYdtccLdDdCcL2aPsOKAVD6z8es/hlV9na/fPxWZo6LSbcGwR0pDKNEuaRfaZuOFdjB3 D94XOIJhAf X-Gm-Gg: ASbGncsrRptohnkwsZk1FfIHMfnGTGmpoMkXftPlE5ziyHXo31yuYiAZgOqUQxFeX7b A3OyllBHVFNDZohTe7TjRAgG6eUB1rYF9n2Y5z/em8NXUfN0wqrRhX1d0P1CdTq73o9fnFN4NFc 5KWgML8mGnFn0W+ZzkvJU/DuB9zC4OPqbQuyDpPZstFMdsxJVIkE3fsGaojxGOoNezbBH9BHY1f pSLGeEQ3nt3g66UjycvB8zaD/RhBOahvSbL0mAU62y3oYqHG01OFSKukML+CF0G7GV6Da/RlAv4 0JGjapotLOklSMUJKde5jVW06uAzv+duQp6xdWCaetJ06+fYDEmyNv2+8FIPJxa9DWNFmQ/G0uf e6zHeJ/MIloc+bNUkNTcODqlwAJ6UaoKXoKQAsL6nEkOR1fy3aAwCcsIwcqT975T+NxReo6TKaj bDJRyWd1n+H0YKKw== X-Received: by 2002:ac8:5783:0:b0:4ec:ee54:b141 with SMTP id d75a77b69052e-4edcab4dafamr79801831cf.26.1762938573404; Wed, 12 Nov 2025 01:09:33 -0800 (PST) X-Google-Smtp-Source: AGHT+IGe8d0FaFe7QdAJaRNml9QGzqJ2/F/EvCxAVGZpOo4PxlmzICV42GcPgVg6YUppJBfwFc+3fQ== X-Received: by 2002:ac8:5783:0:b0:4ec:ee54:b141 with SMTP id d75a77b69052e-4edcab4dafamr79801551cf.26.1762938572931; Wed, 12 Nov 2025 01:09:32 -0800 (PST) Received: from shalem (2001-1c00-0c32-7800-5bfa-a036-83f0-f9ec.cable.dynamic.v6.ziggo.nl. [2001:1c00:c32:7800:5bfa:a036:83f0:f9ec]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b72bfa11367sm1530114066b.68.2025.11.12.01.09.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Nov 2025 01:09:32 -0800 (PST) From: Hans de Goede To: libcamera-devel@lists.libcamera.org Cc: Hans de Goede Subject: [RFC 1/1] libcamera: debayer_cpu: Mask out unused bits from > 8bpp non packed src data Date: Wed, 12 Nov 2025 10:09:24 +0100 Message-ID: <20251112090924.46295-2-johannes.goede@oss.qualcomm.com> X-Mailer: git-send-email 2.51.1 In-Reply-To: <20251112090924.46295-1-johannes.goede@oss.qualcomm.com> References: <20251112090924.46295-1-johannes.goede@oss.qualcomm.com> MIME-Version: 1.0 X-Proofpoint-GUID: n5m8rE-1LY9_65w0ZsP1QyypHie1cACk X-Authority-Analysis: v=2.4 cv=ao2/yCZV c=1 sm=1 tr=0 ts=69144ece cx=c_pps a=JbAStetqSzwMeJznSMzCyw==:117 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=20KFwNOVAAAA:8 a=EUspDBNiAAAA:8 a=YkkIcgtYq-VEA5p9qpYA:9 a=uxP6HrT_eTzRwkO_Te1X:22 X-Proofpoint-ORIG-GUID: n5m8rE-1LY9_65w0ZsP1QyypHie1cACk X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEyMDA3MiBTYWx0ZWRfX+WMOv7hDko9z xLyx+SLDw3TfR+u9kB/C/eJt3Uo3WDS4LOrV7l/V5vfGWWAlJzyHU6m89o5d+gAkRc6qdCMkXo/ JTzIjNC+O6N4gnlZMjlpc/ClQ0IBuVikkB9LHaXqidK4dH10g2ZIso5FkFekfxzMvkyTIvIDLWX Z4Bx9jye6bvrXRNeHx0oFzB05gTdDM/iroN2fTM5zqVqkcC4nCb+EqrhK3J8tPKjd/fq7gDcvCQ Adg+WVlMKAul9VHomotaCvmOxi1fT4iRwtNBEvihZyWlgqzW4NV1QfFQ3oB0v98BRRqew5l7+i1 c1Q6QcqkJc9bhouDB35YcflMpoctcGqezBFle0PX6ZBCB4LwQx34vQjDZGyjmwe0sz5kEwGOj4D nm1FQxNkDMK7xcKrLWoERvrJAgbUqA== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293, Aquarius:18.0.1121, Hydra:6.1.9, FMLib:17.12.100.49 definitions=2025-11-12_03,2025-11-11_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 clxscore=1015 spamscore=0 phishscore=0 adultscore=0 priorityscore=1501 impostorscore=0 malwarescore=0 suspectscore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2510240001 definitions=main-2511120072 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Users have been reporting invalid array access assert errors in statsBGGR10Line0 inside the SWSTATS_ACCUMULATE_LINE_STATS() macro caused by out of bounds accesses to the yHistogram array. Another case of the same problem is out of bounds accesses to the various lookup arrays used in the DebayerCpu code. Both of these are caused by 10 bpp sparse (stored in 16 bit words) input frames containing pixels values > 1023 leading to out of bounds array accesses. IOW bits 15-10 of the 16 bit word are not 0 as they should be. This can only happen if we somehow get a corrupt frame from the kernel. The reported crashes show that this can actually happen so we will need to harden the software ISP code to deal with this. The cheapest (in CPU time) way to fix this is to simply mask out the high unused bits of the words for sparse input formats directly after memcpy-ing a line to one of the lineBuffers. This also means that the software ISP must always use the memcpy path for sparse 10/12 bpp input data (it should never write to the input buffer). Add a new forceInputMemcpy_ for the setting coming from the config file and set enableInputMemcpy_ to true if either forceInputMemcpy_ is set or the input format is sparse 10/12 bpp. Link: https://bugzilla.redhat.com/show_bug.cgi?id=2402746#c20 Signed-off-by: Hans de Goede --- src/libcamera/software_isp/debayer_cpu.cpp | 41 ++++++++++++++++++++-- src/libcamera/software_isp/debayer_cpu.h | 6 ++++ 2 files changed, 45 insertions(+), 2 deletions(-) diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 00738c56..616e4f13 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -47,14 +47,14 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats, const GlobalConfigurat * Reading from uncached buffers may be very slow. * In such a case, it's better to copy input buffer data to normal memory. * But in case of cached buffers, copying the data is unnecessary overhead. - * enable_input_memcpy_ makes this behavior configurable. At the moment, we + * forceInputMemcpy_ makes this behavior configurable. At the moment, we * always set it to true as the safer choice but this should be changed in * future. * * \todo Make memcpy automatic based on runtime detection of platform * capabilities. */ - enableInputMemcpy_ = + forceInputMemcpy_ = configuration.option({ "software_isp", "copy_input_buffer" }).value_or(true); } @@ -288,6 +288,30 @@ void DebayerCpu::debayer10P_RGRG_BGR888(uint8_t *dst, const uint8_t *src[]) } } +/* + * Functions to mask out high unused bits from input buffers. These bits should + * already by 0, but sometimes with corrupt input frames these are not 0 causing + * out-of-bounds accesses to various lookup tables. These functions explicitly + * set the unused high bits to 0 to avoid corrupt frames causing crashes. + */ +void DebayerCpu::inputMask10(uint8_t *data, unsigned int len) +{ + /* Everything is aligned to 2 16 bit pixels, mask 2 pixels at a time */ + uint32_t *input, *end = (uint32_t *)(data + len); + + for (input = (uint32_t *)data; input < end; input++) + *input &= 0x03ff03ff; +} + +void DebayerCpu::inputMask12(uint8_t *data, unsigned int len) +{ + /* Everything is aligned to 2 16 bit pixels, mask 2 pixels at a time */ + uint32_t *input, *end = (uint32_t *)(data + len); + + for (input = (uint32_t *)data; input < end; input++) + *input &= 0x0fff0fff; +} + /* * Setup the Debayer object according to the passed in parameters. * Return 0 on success, a negative errno value on failure @@ -395,6 +419,8 @@ int DebayerCpu::setDebayerFunctions(PixelFormat inputFormat, xShift_ = 0; swapRedBlueGains_ = false; + inputMask_ = NULL; + enableInputMemcpy_ = forceInputMemcpy_; auto invalidFmt = []() -> int { LOG(Debayer, Error) << "Unsupported input output format combination"; @@ -446,12 +472,17 @@ int DebayerCpu::setDebayerFunctions(PixelFormat inputFormat, break; case 10: SET_DEBAYER_METHODS(debayer10_BGBG_BGR888, debayer10_GRGR_BGR888) + inputMask_ = &DebayerCpu::inputMask10; break; case 12: SET_DEBAYER_METHODS(debayer12_BGBG_BGR888, debayer12_GRGR_BGR888) + inputMask_ = &DebayerCpu::inputMask12; break; } setupStandardBayerOrder(bayerFormat.order); + if (inputMask_) + enableInputMemcpy_ = true; + return 0; } @@ -601,6 +632,8 @@ void DebayerCpu::setupInputMemcpy(const uint8_t *linePointers[]) memcpy(lineBuffers_[i].data(), linePointers[i + 1] - lineBufferPadding_, lineBufferLength_); + if (inputMask_) + (this->*inputMask_)(lineBuffers_[i].data(), lineBufferLength_); linePointers[i + 1] = lineBuffers_[i].data() + lineBufferPadding_; } @@ -629,6 +662,10 @@ void DebayerCpu::memcpyNextLine(const uint8_t *linePointers[]) memcpy(lineBuffers_[lineBufferIndex_].data(), linePointers[patternHeight] - lineBufferPadding_, lineBufferLength_); + if (inputMask_) + (this->*inputMask_)(lineBuffers_[lineBufferIndex_].data(), + lineBufferLength_); + linePointers[patternHeight] = lineBuffers_[lineBufferIndex_].data() + lineBufferPadding_; lineBufferIndex_ = (lineBufferIndex_ + 1) % (patternHeight + 1); diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 3bf34ac3..a5feefa2 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -79,6 +79,8 @@ private: */ using debayerFn = void (DebayerCpu::*)(uint8_t *dst, const uint8_t *src[]); + using inputMaskFn = void (DebayerCpu::*)(uint8_t *data, unsigned int len); + /* 8-bit raw bayer format */ template void debayer8_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]); @@ -89,11 +91,13 @@ private: void debayer10_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]); template void debayer10_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]); + void inputMask10(uint8_t *data, unsigned int len); /* unpacked 12-bit raw bayer format */ template void debayer12_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]); template void debayer12_GRGR_BGR888(uint8_t *dst, const uint8_t *src[]); + void inputMask12(uint8_t *data, unsigned int len); /* CSI-2 packed 10-bit raw bayer format (all the 4 orders) */ template void debayer10P_BGBG_BGR888(uint8_t *dst, const uint8_t *src[]); @@ -123,6 +127,7 @@ private: debayerFn debayer1_; debayerFn debayer2_; debayerFn debayer3_; + inputMaskFn inputMask_; Rectangle window_; std::unique_ptr stats_; std::vector lineBuffers_[kMaxLineBuffers]; @@ -130,6 +135,7 @@ private: unsigned int lineBufferPadding_; unsigned int lineBufferIndex_; unsigned int xShift_; /* Offset of 0/1 applied to window_.x */ + bool forceInputMemcpy_; bool enableInputMemcpy_; };