[{"id":19259,"web_url":"https://patchwork.libcamera.org/comment/19259/","msgid":"<20210901095736.GA968527@pyrite.rasen.tech>","date":"2021-09-01T09:57:36","subject":"Re: [libcamera-devel] [PATCH] libcamera: base: log: Don't crash\n\twhen logging after Logger destruction","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"content":"Hi Laurent,\n\nOn Fri, Aug 27, 2021 at 06:58:41PM +0300, Laurent Pinchart wrote:\n> libcamera isn't supposed to log messages after the logger is destroyed,\n> as the global logger instance is destroyed after the main() function\n> returns, and the camera manager is supposed to have been stopped and\n> destroyed before that.\n> \n> This rule is difficult to enforce in the V4L2 compat implementation, as\n> there is no location where we can destroy the camera manager manually\n> before the logger is destroyed. This results in a use-after-free\n> condition when the camera manager gets stopped during destruction.\n> \n> Fix it by not trying to print log messages when the global logger\n> instance has been destroyed.\n> \n> This is a bit of a hack, but hopefully not too bad. There could be race\n> conditions when using a CameraManager instance that is destroyed as part\n> of the destruction of global variables (like the V4L2 compat layer does,\n> it wraps CameraManager in a singleton V4L2CompatManager class, and\n> destroys it when V4L2CompatManager is destroyed) as the CameraManager\n> thread will still be running when the logger gets destroyed, but this\n> doesn't cause any regression as we destroy the logger without any\n> safeguard measure today anyway.\n\nThis looks like a good protection in general.\n\n> \n> There are other options that could be considered. Forcing destruction of\n> the logger after the camera manager in the V4L2 compat layer is one of\n> them, but turned out to be difficult. For instance care would need to be\n> taken *not* to log any message in the mmap() wrapper if the fd doesn't\n> match a wrapped camera, as mmap() is called very early in the\n> initialization process, before libcamera and the logger get initialized.\n> The resulting implementation would likely be fairly complex.\n> \n> Another option could be to wrap the logger with a shared pointer, and\n> keep a reference to it in CameraManager. That's more intrusive, and it's\n> not clear if it would be worth it.\n> \n> Signed-off-by: Laurent Pinchart \n> ---\n> src/libcamera/base/log.cpp | 18 ++++++++++++++++--\n> 1 file changed, 16 insertions(+), 2 deletions(-)\n> \n> diff --git a/src/libcamera/base/log.cpp b/src/libcamera/base/log.cpp\n> index 073b7c34b75e..a3e3f9ea2712 100644\n> --- a/src/libcamera/base/log.cpp\n> +++ b/src/libcamera/base/log.cpp\n> @@ -270,12 +270,16 @@ private:\n> \tfriend LogCategory;\n> \tvoid registerCategory(LogCategory *category);\n> \n> +\tstatic bool destroyed_;\n> +\n> \tstd::unordered_set categories_;\n> \tstd::list> levels_;\n> \n> \tstd::shared_ptr output_;\n> };\n> \n> +bool Logger::destroyed_ = false;\n\nWhat's the merit of putting this here instead of in the constructor?\n\n\nPaul\n\n> +\n> /**\n> * \\enum LoggingTarget\n> * \\brief Log destination type\n> @@ -372,6 +376,8 @@ void logSetLevel(const char *category, const char *level)\n> \n> Logger::~Logger()\n> {\n> +\tdestroyed_ = true;\n> +\n> \tfor (LogCategory *category : categories_)\n> \t\tdelete category;\n> }\n> @@ -387,6 +393,10 @@ Logger::~Logger()\n> Logger *Logger::instance()\n> {\n> \tstatic Logger instance;\n> +\n> +\tif (destroyed_)\n> +\t\treturn nullptr;\n> +\n> \treturn &instance;\n> }\n> \n> @@ -808,13 +818,17 @@ LogMessage::~LogMessage()\n> \tif (severity_ == LogInvalid)\n> \t\treturn;\n> \n> +\tLogger *logger = Logger::instance();\n> +\tif (!logger)\n> +\t\treturn;\n> +\n> \tmsgStream_ << std::endl;\n> \n> \tif (severity_ >= category_.severity())\n> -\t\tLogger::instance()->write(*this);\n> +\t\tlogger->write(*this);\n> \n> \tif (severity_ == LogSeverity::LogFatal) {\n> -\t\tLogger::instance()->backtrace();\n> +\t\tlogger->backtrace();\n> \t\tstd::abort();\n> \t}\n> }\n> -- \n> Regards,\n> \n> Laurent Pinchart\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 55418BD87D\n\tfor ;\n\tWed, 1 Sep 2021 09:57:46 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id BE7BE6916A;\n\tWed, 1 Sep 2021 11:57:45 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4C0C860253\n\tfor ;\n\tWed, 1 Sep 2021 11:57:44 +0200 (CEST)","from pyrite.rasen.tech (unknown\n\t[IPv6:2400:4051:61:600:2c71:1b79:d06d:5032])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id CC004559;\n\tWed, 1 Sep 2021 11:57:42 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"psMkhRuL\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1630490264;\n\tbh=79llcvR9J95Zf/bN1ZnhwS4sGAeome5rP2KUitPevMY=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=psMkhRuLP7N3F775efiW6zmIYEI2sgr0fRpNtAUrMAvRclxCl6AIt83Yc/OcGL5hN\n\tzAKAtcgkzfceXI3v9yDVXzXG2yUIUo2RFJA4hKcw3P9ThzRugMEkNI/6gM9irMrUPQ\n\tGef50ZKMqmgiRQLCBT25t6LtZwfTC1LOmtAtsVvU=","Date":"Wed, 1 Sep 2021 18:57:36 +0900","From":"paul.elder@ideasonboard.com","To":"Laurent Pinchart ","Message-ID":"<20210901095736.GA968527@pyrite.rasen.tech>","References":"<20210827155841.17511-1-laurent.pinchart@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20210827155841.17511-1-laurent.pinchart@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH] libcamera: base: log: Don't crash\n\twhen logging after Logger destruction","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":19260,"web_url":"https://patchwork.libcamera.org/comment/19260/","msgid":"","date":"2021-09-01T10:00:46","subject":"Re: [libcamera-devel] [PATCH] libcamera: base: log: Don't crash\n\twhen logging after Logger destruction","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Paul,\n\nOn Wed, Sep 01, 2021 at 06:57:36PM +0900, paul.elder@ideasonboard.com wrote:\n> On Fri, Aug 27, 2021 at 06:58:41PM +0300, Laurent Pinchart wrote:\n> > libcamera isn't supposed to log messages after the logger is destroyed,\n> > as the global logger instance is destroyed after the main() function\n> > returns, and the camera manager is supposed to have been stopped and\n> > destroyed before that.\n> > \n> > This rule is difficult to enforce in the V4L2 compat implementation, as\n> > there is no location where we can destroy the camera manager manually\n> > before the logger is destroyed. This results in a use-after-free\n> > condition when the camera manager gets stopped during destruction.\n> > \n> > Fix it by not trying to print log messages when the global logger\n> > instance has been destroyed.\n> > \n> > This is a bit of a hack, but hopefully not too bad. There could be race\n> > conditions when using a CameraManager instance that is destroyed as part\n> > of the destruction of global variables (like the V4L2 compat layer does,\n> > it wraps CameraManager in a singleton V4L2CompatManager class, and\n> > destroys it when V4L2CompatManager is destroyed) as the CameraManager\n> > thread will still be running when the logger gets destroyed, but this\n> > doesn't cause any regression as we destroy the logger without any\n> > safeguard measure today anyway.\n> \n> This looks like a good protection in general.\n> \n> > There are other options that could be considered. Forcing destruction of\n> > the logger after the camera manager in the V4L2 compat layer is one of\n> > them, but turned out to be difficult. For instance care would need to be\n> > taken *not* to log any message in the mmap() wrapper if the fd doesn't\n> > match a wrapped camera, as mmap() is called very early in the\n> > initialization process, before libcamera and the logger get initialized.\n> > The resulting implementation would likely be fairly complex.\n> > \n> > Another option could be to wrap the logger with a shared pointer, and\n> > keep a reference to it in CameraManager. That's more intrusive, and it's\n> > not clear if it would be worth it.\n> > \n> > Signed-off-by: Laurent Pinchart \n> > ---\n> > src/libcamera/base/log.cpp | 18 ++++++++++++++++--\n> > 1 file changed, 16 insertions(+), 2 deletions(-)\n> > \n> > diff --git a/src/libcamera/base/log.cpp b/src/libcamera/base/log.cpp\n> > index 073b7c34b75e..a3e3f9ea2712 100644\n> > --- a/src/libcamera/base/log.cpp\n> > +++ b/src/libcamera/base/log.cpp\n> > @@ -270,12 +270,16 @@ private:\n> > \tfriend LogCategory;\n> > \tvoid registerCategory(LogCategory *category);\n> > \n> > +\tstatic bool destroyed_;\n> > +\n> > \tstd::unordered_set categories_;\n> > \tstd::list> levels_;\n> > \n> > \tstd::shared_ptr output_;\n> > };\n> > \n> > +bool Logger::destroyed_ = false;\n> \n> What's the merit of putting this here instead of in the constructor?\n\nIt's a static class member, so it's effectively a global variable, not a\nper-instance member variable.\n\n> > +\n> > /**\n> > * \\enum LoggingTarget\n> > * \\brief Log destination type\n> > @@ -372,6 +376,8 @@ void logSetLevel(const char *category, const char *level)\n> > \n> > Logger::~Logger()\n> > {\n> > +\tdestroyed_ = true;\n> > +\n> > \tfor (LogCategory *category : categories_)\n> > \t\tdelete category;\n> > }\n> > @@ -387,6 +393,10 @@ Logger::~Logger()\n> > Logger *Logger::instance()\n> > {\n> > \tstatic Logger instance;\n> > +\n> > +\tif (destroyed_)\n> > +\t\treturn nullptr;\n> > +\n> > \treturn &instance;\n> > }\n> > \n> > @@ -808,13 +818,17 @@ LogMessage::~LogMessage()\n> > \tif (severity_ == LogInvalid)\n> > \t\treturn;\n> > \n> > +\tLogger *logger = Logger::instance();\n> > +\tif (!logger)\n> > +\t\treturn;\n> > +\n> > \tmsgStream_ << std::endl;\n> > \n> > \tif (severity_ >= category_.severity())\n> > -\t\tLogger::instance()->write(*this);\n> > +\t\tlogger->write(*this);\n> > \n> > \tif (severity_ == LogSeverity::LogFatal) {\n> > -\t\tLogger::instance()->backtrace();\n> > +\t\tlogger->backtrace();\n> > \t\tstd::abort();\n> > \t}\n> > }","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 136D2BD87D\n\tfor ;\n\tWed, 1 Sep 2021 10:01:04 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id AE11D6916A;\n\tWed, 1 Sep 2021 12:01:03 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1641C60288\n\tfor ;\n\tWed, 1 Sep 2021 12:01:03 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 9461D559;\n\tWed, 1 Sep 2021 12:01:02 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"UoaH1u1g\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1630490462;\n\tbh=ROV3Romv87fpt7oradFK5NIoS3CuP62apGkTR4lg9ds=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=UoaH1u1gY2fCZRWkcuvF1uVwjLOu8xYaC+63MUHxz7JrqT3m/FyhzFI75dXaXCP4n\n\tCgbrVE9U9MxtRnHrFAM9rIAK5ZAtWNh57w5X/B7/PNqzvnWLP7bK5SBd1zlLaeGEMM\n\tV04jtmbZqK2E7AsEjTuL6b7EVEOrmJ/mvVs31eG8=","Date":"Wed, 1 Sep 2021 13:00:46 +0300","From":"Laurent Pinchart ","To":"paul.elder@ideasonboard.com","Message-ID":"","References":"<20210827155841.17511-1-laurent.pinchart@ideasonboard.com>\n\t<20210901095736.GA968527@pyrite.rasen.tech>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20210901095736.GA968527@pyrite.rasen.tech>","Subject":"Re: [libcamera-devel] [PATCH] libcamera: base: log: Don't crash\n\twhen logging after Logger destruction","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":19261,"web_url":"https://patchwork.libcamera.org/comment/19261/","msgid":"<20210901100222.GB968527@pyrite.rasen.tech>","date":"2021-09-01T10:02:22","subject":"Re: [libcamera-devel] [PATCH] libcamera: base: log: Don't crash\n\twhen logging after Logger destruction","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"content":"Hi Laurent,\n\nOn Wed, Sep 01, 2021 at 01:00:46PM +0300, Laurent Pinchart wrote:\n> Hi Paul,\n> \n> On Wed, Sep 01, 2021 at 06:57:36PM +0900, paul.elder@ideasonboard.com wrote:\n> > On Fri, Aug 27, 2021 at 06:58:41PM +0300, Laurent Pinchart wrote:\n> > > libcamera isn't supposed to log messages after the logger is destroyed,\n> > > as the global logger instance is destroyed after the main() function\n> > > returns, and the camera manager is supposed to have been stopped and\n> > > destroyed before that.\n> > > \n> > > This rule is difficult to enforce in the V4L2 compat implementation, as\n> > > there is no location where we can destroy the camera manager manually\n> > > before the logger is destroyed. This results in a use-after-free\n> > > condition when the camera manager gets stopped during destruction.\n> > > \n> > > Fix it by not trying to print log messages when the global logger\n> > > instance has been destroyed.\n> > > \n> > > This is a bit of a hack, but hopefully not too bad. There could be race\n> > > conditions when using a CameraManager instance that is destroyed as part\n> > > of the destruction of global variables (like the V4L2 compat layer does,\n> > > it wraps CameraManager in a singleton V4L2CompatManager class, and\n> > > destroys it when V4L2CompatManager is destroyed) as the CameraManager\n> > > thread will still be running when the logger gets destroyed, but this\n> > > doesn't cause any regression as we destroy the logger without any\n> > > safeguard measure today anyway.\n> > \n> > This looks like a good protection in general.\n> > \n> > > There are other options that could be considered. Forcing destruction of\n> > > the logger after the camera manager in the V4L2 compat layer is one of\n> > > them, but turned out to be difficult. For instance care would need to be\n> > > taken *not* to log any message in the mmap() wrapper if the fd doesn't\n> > > match a wrapped camera, as mmap() is called very early in the\n> > > initialization process, before libcamera and the logger get initialized.\n> > > The resulting implementation would likely be fairly complex.\n> > > \n> > > Another option could be to wrap the logger with a shared pointer, and\n> > > keep a reference to it in CameraManager. That's more intrusive, and it's\n> > > not clear if it would be worth it.\n> > > \n> > > Signed-off-by: Laurent Pinchart \n> > > ---\n> > > src/libcamera/base/log.cpp | 18 ++++++++++++++++--\n> > > 1 file changed, 16 insertions(+), 2 deletions(-)\n> > > \n> > > diff --git a/src/libcamera/base/log.cpp b/src/libcamera/base/log.cpp\n> > > index 073b7c34b75e..a3e3f9ea2712 100644\n> > > --- a/src/libcamera/base/log.cpp\n> > > +++ b/src/libcamera/base/log.cpp\n> > > @@ -270,12 +270,16 @@ private:\n> > > \tfriend LogCategory;\n> > > \tvoid registerCategory(LogCategory *category);\n> > > \n> > > +\tstatic bool destroyed_;\n> > > +\n> > > \tstd::unordered_set categories_;\n> > > \tstd::list> levels_;\n> > > \n> > > \tstd::shared_ptr output_;\n> > > };\n> > > \n> > > +bool Logger::destroyed_ = false;\n> > \n> > What's the merit of putting this here instead of in the constructor?\n> \n> It's a static class member, so it's effectively a global variable, not a\n> per-instance member variable.\n\nOh right, I see. Thanks for the clarification.\n\nReviewed-by: Paul Elder \n\n> \n> > > +\n> > > /**\n> > > * \\enum LoggingTarget\n> > > * \\brief Log destination type\n> > > @@ -372,6 +376,8 @@ void logSetLevel(const char *category, const char *level)\n> > > \n> > > Logger::~Logger()\n> > > {\n> > > +\tdestroyed_ = true;\n> > > +\n> > > \tfor (LogCategory *category : categories_)\n> > > \t\tdelete category;\n> > > }\n> > > @@ -387,6 +393,10 @@ Logger::~Logger()\n> > > Logger *Logger::instance()\n> > > {\n> > > \tstatic Logger instance;\n> > > +\n> > > +\tif (destroyed_)\n> > > +\t\treturn nullptr;\n> > > +\n> > > \treturn &instance;\n> > > }\n> > > \n> > > @@ -808,13 +818,17 @@ LogMessage::~LogMessage()\n> > > \tif (severity_ == LogInvalid)\n> > > \t\treturn;\n> > > \n> > > +\tLogger *logger = Logger::instance();\n> > > +\tif (!logger)\n> > > +\t\treturn;\n> > > +\n> > > \tmsgStream_ << std::endl;\n> > > \n> > > \tif (severity_ >= category_.severity())\n> > > -\t\tLogger::instance()->write(*this);\n> > > +\t\tlogger->write(*this);\n> > > \n> > > \tif (severity_ == LogSeverity::LogFatal) {\n> > > -\t\tLogger::instance()->backtrace();\n> > > +\t\tlogger->backtrace();\n> > > \t\tstd::abort();\n> > > \t}\n> > > }\n> \n> -- \n> Regards,\n> \n> Laurent Pinchart","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 77051BD87D\n\tfor ;\n\tWed, 1 Sep 2021 10:02:30 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 306DB6916A;\n\tWed, 1 Sep 2021 12:02:30 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 22D4760288\n\tfor ;\n\tWed, 1 Sep 2021 12:02:29 +0200 (CEST)","from pyrite.rasen.tech (unknown\n\t[IPv6:2400:4051:61:600:2c71:1b79:d06d:5032])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id D39D8559;\n\tWed, 1 Sep 2021 12:02:27 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"UyGiYoBg\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1630490548;\n\tbh=SpHlumRSwKWIK9eMeAY/5t9XQDAItK5GQlKP6RnfSqg=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=UyGiYoBg0l7+iWv6PUwLo7/DSUsmGpCchU7TxmQlu6w7D4fwx6apO6pPzOs0o5/RI\n\t2TFe4QUiExJWZGvdF4Wwd5wnKAfZRdn0uzZ/HoMg9rtCEId1W1APQk8QjwMVjJPgW/\n\t8kgXT7ASYroM8zuWgZ7/4lyTfBoGaLch3SOQLw9Q=","Date":"Wed, 1 Sep 2021 19:02:22 +0900","From":"paul.elder@ideasonboard.com","To":"Laurent Pinchart ","Message-ID":"<20210901100222.GB968527@pyrite.rasen.tech>","References":"<20210827155841.17511-1-laurent.pinchart@ideasonboard.com>\n\t<20210901095736.GA968527@pyrite.rasen.tech>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH] libcamera: base: log: Don't crash\n\twhen logging after Logger destruction","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]