{"id":25006,"url":"https://patchwork.libcamera.org/api/1.1/covers/25006/?format=json","web_url":"https://patchwork.libcamera.org/cover/25006/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20251112090924.46295-1-johannes.goede@oss.qualcomm.com>","date":"2025-11-12T09:09:23","name":"[RFC,0/1] Fix softISP crash on 10/12bpp sparse input frames","submitter":{"id":242,"url":"https://patchwork.libcamera.org/api/1.1/people/242/?format=json","name":"Hans de Goede","email":"johannes.goede@oss.qualcomm.com"},"mbox":"https://patchwork.libcamera.org/cover/25006/mbox/","series":[{"id":5583,"url":"https://patchwork.libcamera.org/api/1.1/series/5583/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5583","date":"2025-11-12T09:09:23","name":"Fix softISP crash on 10/12bpp sparse input frames","version":1,"mbox":"https://patchwork.libcamera.org/series/5583/mbox/"}],"comments":"https://patchwork.libcamera.org/api/covers/25006/comments/","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 15CD8C3263\n\tfor ;\n\tWed, 12 Nov 2025 09:09:39 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 15C4A60A9D;\n\tWed, 12 Nov 2025 10:09:38 +0100 (CET)","from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com\n\t[205.220.168.131])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id BAD6E606E6\n\tfor ;\n\tWed, 12 Nov 2025 10:09:35 +0100 (CET)","from pps.filterd (m0279867.ppops.net [127.0.0.1])\n\tby mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id\n\t5AC6SjZ7350148 for ;\n\tWed, 12 Nov 2025 09:09:34 GMT","from mail-qt1-f199.google.com (mail-qt1-f199.google.com\n\t[209.85.160.199])\n\tby mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4acn0nrf23-1\n\t(version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT)\n\tfor ;\n\tWed, 12 Nov 2025 09:09:33 +0000 (GMT)","by mail-qt1-f199.google.com with SMTP id\n\td75a77b69052e-4e8984d8833so28677951cf.0\n\tfor ;\n\tWed, 12 Nov 2025 01:09:33 -0800 (PST)","from shalem\n\t(2001-1c00-0c32-7800-5bfa-a036-83f0-f9ec.cable.dynamic.v6.ziggo.nl.\n\t[2001:1c00:c32:7800:5bfa:a036:83f0:f9ec])\n\tby smtp.gmail.com with ESMTPSA id\n\ta640c23a62f3a-b72bfa11367sm1530114066b.68.2025.11.12.01.09.25\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tWed, 12 Nov 2025 01:09:25 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=qualcomm.com header.i=@qualcomm.com\n\theader.b=\"eo91wK3x\"; dkim=pass (2048-bit key;\n\tunprotected) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com\n\theader.b=\"ay3EH0EM\"; dkim-atps=neutral","DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h=\n\tcc:content-transfer-encoding:date:from:message-id:mime-version\n\t:subject:to; s=qcppdkim1; bh=RkffvbpRnRcOvReYMPahCz1Agckc7PO+hYW\n\tNzhFRvqA=; b=eo91wK3x3Asyf+TAifxLT8DUGwMxJazBbIr9ax9M08jnxSNIqWL\n\t+rbin8wpQQklHmY/ROJIZYl8F16Evobr0itwm60g4RaZoQA9wJ/tHR63S8Bjk9L5\n\tJ0DQx7zCz48dNOXXoWoPHZ6+gC+FiQsG46OLSY+5zCza4bfZvkgpBNVq7shXWWx3\n\tr7Ae7O/sqrvAO5DQ4NKHE2/FGRd+rbyHmnLniHcOa1MUoaMF4Ha+fJqPRqulQsJf\n\t9BAbV6x/YzniQ82TQoXkC5Z4BR9xf1Vx66yAEmhGnoBHXGPZW7GJdXcsR+q8YNnR\n\t3JovnnmhqQ5/jb1/mCYMWT1I0FN0HuIKW1w==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=oss.qualcomm.com; s=google; t=1762938572; x=1763543372;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:message-id:date:subject:cc\n\t:to:from:from:to:cc:subject:date:message-id:reply-to;\n\tbh=RkffvbpRnRcOvReYMPahCz1Agckc7PO+hYWNzhFRvqA=;\n\tb=ay3EH0EM+u3ZnCrsIfN2ArIx1CA9ZIDiKNAMc1zI/rGONG9hRhwCzs4BGeR24zR2aB\n\tV6wi6MZONfCu4sR5ZpyriupDBG0I3zsJHAnLSXwxretAf+WYyvPOsQpk00DluapASCgD\n\tObz/WbW1qnAul1BVMildmIm/GOb/j7WWn+BOyTEuCZvAt355qLCBl9O7lCj7CScrI67X\n\tbAsyXbdAXWViZeRKHYGBKoPw5IZD5pc3zBklXU3fMjFTR3na5w1SBqzLXQqA2iM+f/hB\n\to2DkF6j3AJR9sxcF3uVqnmFIDEAsbFkzkntc7ZT0xLoe3crOMIIIerYM1N4WUV7wHDRw\n\t4vGw=="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1762938572; x=1763543372;\n\th=content-transfer-encoding:mime-version:message-id:date:subject:cc\n\t:to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=RkffvbpRnRcOvReYMPahCz1Agckc7PO+hYWNzhFRvqA=;\n\tb=O7ocBdAKpK59n7ahI23BptCOhlIisaHhFtRc+aZeB8vuY6K/YfwAKhTY2wxQjd1TK6\n\tR3faYmKM3w8Fjt0h0VnyEHJIy1yHXyGsr7J3LfnXPhtrtgr1w6oooXEk2dio7qPEG3VV\n\tztjkT7gU/nEpUtSW/MliU5o3+Fqc5spYcfU2fWPSXXNS6kCbzs22Qjx6cQ6Nts5Gcmmp\n\t3gjsia9pu3r6QKlwka7JqxffGVSI9fUWvt2N3xutrGD/xPEOPHw/Ro+C8yqfJY6GteyC\n\t8pDT9/o/ZmRu3RGcx4hfUi3XQSPQQ0zuYdMoCQxouc322qdYzGob7UtglgRNaN3jfB+9\n\tqP3w==","X-Gm-Message-State":"AOJu0Yy9/T9ogOckAf253BGcs8tP99ffq9q62jCddeutD1uYSKScV4y/\n\tnVRUsRZe2TvTCiJY5M46UJ2vwIyoS6FPuKgEYR+JT36J3xMueRbVyiWJTVvcZn9sNBBsj9ZUhxN\n\tFuI9ijFlV3BI8Ye0/0Och6Wgg9zcxnX1HTY0MG4Dua8TxmeboiFcyKINn3M4+Q+QKEpSGtOTLKP\n\t1tPJbaF0q/","X-Gm-Gg":"ASbGncukKu/OZe4su4jV6iblCu6cqx6SBV01DAnQXebcelQKg/dJEG49h1O0GfamgQt\n\tg3NCbmW6JiAgxIXH4PWyF/VClrg6FCJ8rtANyYPCHzza6bbMzbzpts6k+Qe4T/PK5ukf/8EiTDY\n\tax0IGZKrlinY5YkA4mUXe/3hcXQQ9aC4WZfcQkT722b9q+12jeEn8Zx+geFrqMcs1r01e5c6qJA\n\tnI0ifvSm7o7h6uwMIn+L07trFtQfYae1vys7Zb8OWNlIOEcu6rr1f11yb7fAVsZikzXUabwq8A+\n\tBVxiUWw3R+pKFipalnOpjO8snJ/ea9B9vEAJBtF2p/0gKMNxo9CduDnqwycmX7rPqtu3wrH3YRW\n\tZ5xSFu5WOV6WbPVALfJLStdWXV1DUEUlVDdI/ExLRk5N2Xb1qjQijscQqrZZM9G5Y0KwGZg1VQT\n\t7/1OxQUkx8x/Cz9A==","X-Received":["by 2002:a05:622a:10a:b0:4ed:aa7b:e1b6 with SMTP id\n\td75a77b69052e-4eddbc7c12emr28707571cf.12.1762938572408; \n\tWed, 12 Nov 2025 01:09:32 -0800 (PST)","by 2002:a05:622a:10a:b0:4ed:aa7b:e1b6 with SMTP id\n\td75a77b69052e-4eddbc7c12emr28704141cf.12.1762938565925; \n\tWed, 12 Nov 2025 01:09:25 -0800 (PST)"],"X-Google-Smtp-Source":"AGHT+IHNyXiMlexaZjKTQcy4vqYlHGr/TJIcLjeSJvGNWH0Bl7j0ofMvibGMpVUMpZkwaVk2wDKlGQ==","From":"Hans de Goede ","To":"libcamera-devel@lists.libcamera.org","Cc":"Hans de Goede ","Subject":"[RFC 0/1] Fix softISP crash on 10/12bpp sparse input frames","Date":"Wed, 12 Nov 2025 10:09:23 +0100","Message-ID":"<20251112090924.46295-1-johannes.goede@oss.qualcomm.com>","X-Mailer":"git-send-email 2.51.1","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-Proofpoint-GUID":"Zn-rfebyE1RK5F7YSshbBq4OrpdvQcOj","X-Proofpoint-ORIG-GUID":"Zn-rfebyE1RK5F7YSshbBq4OrpdvQcOj","X-Authority-Analysis":"v=2.4 cv=acRsXBot c=1 sm=1 tr=0 ts=69144ecd cx=c_pps\n\ta=WeENfcodrlLV9YRTxbY/uA==:117 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10\n\ta=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=20KFwNOVAAAA:8\n\ta=Q41pJdeu3oAlCBGPiJAA:9 a=kacYvNCVWA4VmyqE58fU:22","X-Proofpoint-Spam-Details-Enc":"AW1haW4tMjUxMTEyMDA3MiBTYWx0ZWRfX+ymBv/md0XV2\n\tQxuQ24WCJx+k6+U9S9NZQYoxBW0VHiHq5u7V7Vqh1zMDVGT2q4MfxU3KlZdY9csI5z9HaVvYz8/\n\tYKYzL2Ojler5mAvpfqVU95ARenGIyU2wCIyS/usS+8Ms2Rh5fBcyQ6bxSs1DrLlfWzjREIsImP+\n\tS9jh80Ol8gOQ+dKpihM4xXvEOFUAxD+MsUB0G9E9oYXMA6uWyKjZ0Qb/M4aZNFlpO71htG5IAea\n\tSpZGWKD9ZRDCn4KSP9KsTmEe5g+BjlFucg4CMQMakcMwxhHQRzc9n+4ghzwd1M92p4VCGmTg8DG\n\txurXH4I4C6Tk/8KVKZV7l/bNlV7cC6oaEtmjZddXl+h8QmAnA3UE5pDLkFrKMxhFtMNJcjVHlCm\n\toVfTChdlLKBGMw2vnInbPADUTGltew==","X-Proofpoint-Virus-Version":"vendor=baseguard\n\tengine=ICAP:2.0.293, Aquarius:18.0.1121, Hydra:6.1.9,\n\tFMLib:17.12.100.49\n\tdefinitions=2025-11-12_03,2025-11-11_03,2025-10-01_01","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tsuspectscore=0 phishscore=0 bulkscore=0 clxscore=1015 spamscore=0\n\tlowpriorityscore=0 malwarescore=0 adultscore=0 impostorscore=0\n\tpriorityscore=1501 classifier=typeunknown authscore=0 authtc= authcc=\n\troute=outbound adjust=0 reason=mlx scancount=1\n\tengine=8.22.0-2510240001\n\tdefinitions=main-2511120072","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Hi All,\n\nAs reported here https://bugzilla.redhat.com/show_bug.cgi?id=2402746#c20\nthere are several places where the swstats / debayer CPU code will\ndo out of bounds array accesses when processing a corrupt input frame\nin 10/12 bpp sparse format. The issue is that these corrupt frames\nwhich store 10 or 12 bpp pixels in 16 bit words may have the high\nbits set leading to e.g. pixel values > 1023 for 10 bpp input data,\nwhich in turn leads to out of bounds array accesses.\n\nHere are 2 example backtraces:\n\n#4 0x00007fba4900b084 in std::__glibcxx_assert_fail (file=file@entry=0x7fba495b7344 \"/usr/include/c++/15/array\", line=line@entry=210, \n function=function@entry=0x7fba495b5d68 \"constexpr std::array<_Tp, _Nm>::value_type& std::array<_Tp, _Nm>::operator[](size_type) [with _Tp = unsigned int; long unsigned int _Nm = 64; reference = unsigned int&; size_type = long unsigned int]\", condition=condition@entry=0x7fba495b6df2 \"__n < this->size()\") at ../../../../../libstdc++-v3/src/c++11/assert_fail.cc:41\n#5 0x00007fba49423d15 in std::array::operator[] (__n=, this=) at /usr/include/c++/15/array:210\n#6 0x00007fba49423d1b in std::array::operator[] (this=, __n=) at /usr/include/c++/15/array:210\n#7 libcamera::SwStatsCpu::statsBGGR10Line0 (this=, src=) at ../src/libcamera/software_isp/swstats_cpu.cpp:219\n#8 0x00007fba4951d13b in libcamera::SwStatsCpu::processLine0 (frame=0, y=0, this=, src=0x7fba427f89c0) at ../src/libcamera/software_isp/swstats_cpu.h:63\n#9 libcamera::SwStatsCpu::processLine0 (this=, frame=0, y=0, src=0x7fba427f89c0) at ../src/libcamera/software_isp/swstats_cpu.h:54\n#10 libcamera::DebayerCpu::process2 (this=this@entry=0x7fba38053c20, frame=frame@entry=0, src=0x7fba48089d08 , \n\nWhere swstats_cpu.cpp:219 points SWSTATS_ACCUMULATE_LINE_STATS() which\naccesses the yHistogram array.\n\n#4 0x00007f936ca0b084 in std::__glibcxx_assert_fail (file=file@entry=0x7f936cfb727c \"/usr/include/c++/15/array\", \n line=line@entry=210, \n function=function@entry=0x7f936cfb5760 \"constexpr std::array<_Tp, _Nm>::value_type& std::array<_Tp, _Nm>::operator[](size_type) [with _Tp = unsigned char; long unsigned int _Nm = 256; reference = unsigned char&; size_type = long unsigned in\"..., condition=condition@entry=0x7f936cfb6d2a \"__n < this->size()\") at ../../../../../libstdc++-v3/src/c++11/assert_fail.cc:41\n#5 0x00007f936ce22fd3 in std::array::operator[] (__n=, this=)\n at /usr/include/c++/15/array:210\n#6 0x00007f936cf21102 in std::array::operator[] (this=, __n=)\n at ../src/libcamera/software_isp/debayer_cpu.cpp:158\n#7 libcamera::DebayerCpu::debayer10_BGBG_BGR888 (this=, dst=, \n src=) at ../src/libcamera/software_isp/debayer_cpu.cpp:164\n\nWhere debayer_cpu.cpp:164 ends up calling the STORE_PIXEL() macro which\naccesses various lookup tables.\n\nThe single patch in this series should fix this. This is marked as a RFC\nfor now because I'm waiting on testing feedback from the reporter.\n\nRegards,\n\nHans\n\n\nHans de Goede (1):\n libcamera: debayer_cpu: Mask out unused bits from > 8bpp non packed\n src data\n\n src/libcamera/software_isp/debayer_cpu.cpp | 41 ++++++++++++++++++++--\n src/libcamera/software_isp/debayer_cpu.h | 6 ++++\n 2 files changed, 45 insertions(+), 2 deletions(-)"}