[{"id":32026,"web_url":"https://patchwork.libcamera.org/comment/32026/","msgid":"<173080183359.3353069.11449653144311665785@ping.linuxembedded.co.uk>","date":"2024-11-05T10:17:13","subject":"Re: [PATCH 2/2] test: py: LD_PRELOAD the C++ standard library when\n\tusing ASan","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Laurent Pinchart (2024-11-05 01:49:49)\n> When the ASan runtime is linked using --as-needed, its dependency on the\n> C++ standard library is stripped. This results to a failure to properly\n> handled exceptions when a C++ dynamically loaded .so is used, as in the\n> Python unit tests that load the libcamera Python module:\n> \n> AddressSanitizer: CHECK failed: asan_interceptors.cpp:335 \"((__interception::real___cxa_throw)) != (0)\" (0x0, 0x0) (tid=32679)\n> #0 0x7fa2f32e6c19 in CheckUnwind /var/tmp/portage/sys-devel/gcc-13.3.1_p20241025/work/gcc-13-20241025/libsanitizer/asan/asan_rtl.cpp:69\n> #1 0x7fa2f330c9fd in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /var/tmp/portage/sys-devel/gcc-13.3.1_p20241025/work/gcc-13-20241025/libsanitizer/sanitizer_common/sanitizer_termination.cpp:86\n> #2 0x7fa2f3247824 in __interceptor___cxa_throw /var/tmp/portage/sys-devel/gcc-13.3.1_p20241025/work/gcc-13-20241025/libsanitizer/asan/asan_interceptors.cpp:335\n> #3 0x7fa2f3247824 in __interceptor___cxa_throw /var/tmp/portage/sys-devel/gcc-13.3.1_p20241025/work/gcc-13-20241025/libsanitizer/asan/asan_interceptors.cpp:334\n> #4 0x7fa2efb6da8b in operator() ../../src/py/libcamera/py_main.cpp:157\n> [...]\n> \n> The issue has been reported in [1] and so far remains unfixed. Work\n> around it by preloading the C++ standard library.\n> \n> [1] https://github.com/google/sanitizers/issues/934\n> \n> Signed-off-by: Laurent Pinchart \n> ---\n> meson.build | 3 +++\n> test/py/meson.build | 6 ++++++\n> 2 files changed, 9 insertions(+)\n> \n> diff --git a/meson.build b/meson.build\n> index 63e45465da9c..e3052260f13e 100644\n> --- a/meson.build\n> +++ b/meson.build\n> @@ -112,6 +112,8 @@ common_arguments = [\n> c_arguments = []\n> cpp_arguments = []\n> \n> +cxx_stdlib = 'libstdc++'\n> +\n> if cc.get_id() == 'clang'\n> if cc.version().version_compare('<9')\n> error('clang version is too old, libcamera requires 9.0 or newer')\n> @@ -137,6 +139,7 @@ if cc.get_id() == 'clang'\n> cpp_arguments += [\n> '-stdlib=libc++',\n> ]\n> + cxx_stdlib = 'libc++'\n> endif\n> \n> cpp_arguments += [\n> diff --git a/test/py/meson.build b/test/py/meson.build\n> index 481bb481c110..b922e8578c29 100644\n> --- a/test/py/meson.build\n> +++ b/test/py/meson.build\n> @@ -24,6 +24,12 @@ py_env.append('PYTHONPATH', pypathdir)\n> if asan_enabled\n> py_env.append('LD_PRELOAD', asan_runtime)\n> \n> + # Preload the C++ standard library to work around a bug in ASan when\n> + # dynamically loading C++ .so modules.\n> + stdlib = run_command(cxx, '-print-file-name=' + cxx_stdlib + '.so',\n> + check : true).stdout().strip()\n> + py_env.append('LD_PRELOAD', stdlib)\n> +\n\nI assume it is, but have never used LD_PRELOAD this way - is it safe to\nhave multiple instances of the environment variable?\n\n\nDoes this make a space separated list? Or does it make\nLD_PRELOAD=file1.so LD_PRELOAD=file2.so appear in the env?\n\nI presume whatever it does it works as you'll have used it ...\n\nso presumptively:\n\n\nReviewed-by: Kieran Bingham \n\nBut I'm interested to know which form is used by meson when appending\nthe same variable to an environment for future use cases.\n\n--\nKieran\n\n\n> # Disable leak detection as the Python interpreter is full of leaks.\n> py_env.append('ASAN_OPTIONS', 'detect_leaks=0')\n> endif\n> -- \n> Regards,\n> \n> Laurent Pinchart\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id E3982BDB1C\n\tfor ;\n\tTue, 5 Nov 2024 10:17:18 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 204EE653EE;\n\tTue, 5 Nov 2024 11:17:18 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id D9F15653E7\n\tfor ;\n\tTue, 5 Nov 2024 11:17:16 +0100 (CET)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id EF866D52;\n\tTue, 5 Nov 2024 11:17:08 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"GMOu+Xc7\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1730801829;\n\tbh=Bgk6jNgnPhW4euvqTXLNa9a6aeeXqUAzBxC9LWxtYIw=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=GMOu+Xc70j0PLIKNwMqiKsjlmaObpNZ8j9i5Ww6gNkeWda0towYE6Wtxm1RFlxJVX\n\toap4BdorNm884U2117zf61U2aWq00hkdBVnU6nHif3/ZIkw0MI4LMv7rKXWS/8X87o\n\tgcBmazKyVQlRDQytbu0zTu/gqY0O0mIIHoZU6pZA=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20241105014949.32244-3-laurent.pinchart@ideasonboard.com>","References":"<20241105014949.32244-1-laurent.pinchart@ideasonboard.com>\n\t<20241105014949.32244-3-laurent.pinchart@ideasonboard.com>","Subject":"Re: [PATCH 2/2] test: py: LD_PRELOAD the C++ standard library when\n\tusing ASan","From":"Kieran Bingham ","Cc":"Tomi Valkeinen ","To":"Laurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Tue, 05 Nov 2024 10:17:13 +0000","Message-ID":"<173080183359.3353069.11449653144311665785@ping.linuxembedded.co.uk>","User-Agent":"alot/0.10","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32029,"web_url":"https://patchwork.libcamera.org/comment/32029/","msgid":"<20241105104008.GM27775@pendragon.ideasonboard.com>","date":"2024-11-05T10:40:08","subject":"Re: [PATCH 2/2] test: py: LD_PRELOAD the C++ standard library when\n\tusing ASan","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Tue, Nov 05, 2024 at 10:17:13AM +0000, Kieran Bingham wrote:\n> Quoting Laurent Pinchart (2024-11-05 01:49:49)\n> > When the ASan runtime is linked using --as-needed, its dependency on the\n> > C++ standard library is stripped. This results to a failure to properly\n> > handled exceptions when a C++ dynamically loaded .so is used, as in the\n> > Python unit tests that load the libcamera Python module:\n> > \n> > AddressSanitizer: CHECK failed: asan_interceptors.cpp:335 \"((__interception::real___cxa_throw)) != (0)\" (0x0, 0x0) (tid=32679)\n> > #0 0x7fa2f32e6c19 in CheckUnwind /var/tmp/portage/sys-devel/gcc-13.3.1_p20241025/work/gcc-13-20241025/libsanitizer/asan/asan_rtl.cpp:69\n> > #1 0x7fa2f330c9fd in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /var/tmp/portage/sys-devel/gcc-13.3.1_p20241025/work/gcc-13-20241025/libsanitizer/sanitizer_common/sanitizer_termination.cpp:86\n> > #2 0x7fa2f3247824 in __interceptor___cxa_throw /var/tmp/portage/sys-devel/gcc-13.3.1_p20241025/work/gcc-13-20241025/libsanitizer/asan/asan_interceptors.cpp:335\n> > #3 0x7fa2f3247824 in __interceptor___cxa_throw /var/tmp/portage/sys-devel/gcc-13.3.1_p20241025/work/gcc-13-20241025/libsanitizer/asan/asan_interceptors.cpp:334\n> > #4 0x7fa2efb6da8b in operator() ../../src/py/libcamera/py_main.cpp:157\n> > [...]\n> > \n> > The issue has been reported in [1] and so far remains unfixed. Work\n> > around it by preloading the C++ standard library.\n> > \n> > [1] https://github.com/google/sanitizers/issues/934\n> > \n> > Signed-off-by: Laurent Pinchart \n> > ---\n> > meson.build | 3 +++\n> > test/py/meson.build | 6 ++++++\n> > 2 files changed, 9 insertions(+)\n> > \n> > diff --git a/meson.build b/meson.build\n> > index 63e45465da9c..e3052260f13e 100644\n> > --- a/meson.build\n> > +++ b/meson.build\n> > @@ -112,6 +112,8 @@ common_arguments = [\n> > c_arguments = []\n> > cpp_arguments = []\n> > \n> > +cxx_stdlib = 'libstdc++'\n> > +\n> > if cc.get_id() == 'clang'\n> > if cc.version().version_compare('<9')\n> > error('clang version is too old, libcamera requires 9.0 or newer')\n> > @@ -137,6 +139,7 @@ if cc.get_id() == 'clang'\n> > cpp_arguments += [\n> > '-stdlib=libc++',\n> > ]\n> > + cxx_stdlib = 'libc++'\n> > endif\n> > \n> > cpp_arguments += [\n> > diff --git a/test/py/meson.build b/test/py/meson.build\n> > index 481bb481c110..b922e8578c29 100644\n> > --- a/test/py/meson.build\n> > +++ b/test/py/meson.build\n> > @@ -24,6 +24,12 @@ py_env.append('PYTHONPATH', pypathdir)\n> > if asan_enabled\n> > py_env.append('LD_PRELOAD', asan_runtime)\n> > \n> > + # Preload the C++ standard library to work around a bug in ASan when\n> > + # dynamically loading C++ .so modules.\n> > + stdlib = run_command(cxx, '-print-file-name=' + cxx_stdlib + '.so',\n> > + check : true).stdout().strip()\n> > + py_env.append('LD_PRELOAD', stdlib)\n> > +\n> \n> I assume it is, but have never used LD_PRELOAD this way - is it safe to\n> have multiple instances of the environment variable?\n> \n> \n> Does this make a space separated list? Or does it make\n> LD_PRELOAD=file1.so LD_PRELOAD=file2.so appear in the env?\n> \n> I presume whatever it does it works as you'll have used it ...\n\nenvironment.append() appends a new value to an existing environment\nvariable, with a separator. The separator defaults to ':' on Linux. It\ncan be overridden with an optional argument to environment.append() if\nneeded, but in this case ':' is what we want.\n\n> so presumptively:\n> \n> \n> Reviewed-by: Kieran Bingham \n> \n> But I'm interested to know which form is used by meson when appending\n> the same variable to an environment for future use cases.\n> \n> > # Disable leak detection as the Python interpreter is full of leaks.\n> > py_env.append('ASAN_OPTIONS', 'detect_leaks=0')\n> > endif","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 6FFA2BDB1C\n\tfor ;\n\tTue, 5 Nov 2024 10:40:20 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A92F4653F1;\n\tTue, 5 Nov 2024 11:40:19 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 725316035B\n\tfor ;\n\tTue, 5 Nov 2024 11:40:18 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id A343B4AD;\n\tTue, 5 Nov 2024 11:40:10 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"CCQFVwOG\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1730803210;\n\tbh=29fYx35sY/yiLqDHXPNTByf83M9DBtVCrvJkt9E3jd4=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=CCQFVwOGI0IMj+AZZtGLnnIw5u7Y8SDNAuAWcZWEP4RbDcixCzHJzpv5L4k7zeR1P\n\tGzkrubry12RgFqQ8iRVbvZE6n+HkPe368lpIXqPjhwSYG+wsE4rn1FYfMJ9KrmPG6o\n\tX6bBmcCGPbOTuRGi+E8quYo8OrgwI1AmB2/erhWA=","Date":"Tue, 5 Nov 2024 12:40:08 +0200","From":"Laurent Pinchart ","To":"Kieran Bingham ","Cc":"libcamera-devel@lists.libcamera.org,\n\tTomi Valkeinen ","Subject":"Re: [PATCH 2/2] test: py: LD_PRELOAD the C++ standard library when\n\tusing ASan","Message-ID":"<20241105104008.GM27775@pendragon.ideasonboard.com>","References":"<20241105014949.32244-1-laurent.pinchart@ideasonboard.com>\n\t<20241105014949.32244-3-laurent.pinchart@ideasonboard.com>\n\t<173080183359.3353069.11449653144311665785@ping.linuxembedded.co.uk>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<173080183359.3353069.11449653144311665785@ping.linuxembedded.co.uk>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]