[{"id":25648,"web_url":"https://patchwork.libcamera.org/comment/25648/","msgid":"<166695266727.2560709.1182888743581387057@Monstersaurus>","date":"2022-10-28T10:24:27","subject":"Re: [libcamera-devel] Concurrent camera usage in libcamera","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Naushir Patuck via libcamera-devel (2022-10-28 10:42:55)\n> Hi all,\n> \n> I wonder if I could get some thoughts regarding a problem we are seeing\n> with running multiple cameras in separate libcamera processes on Raspberry\n> Pi (this probably affects any/all platforms though).\n> \n> In my scenario, I have libcamera process 1 running \"cam -c 1 -C\" and\n> streaming from imx477 using the Raspberry Pi pipeline handler. In process\n> 2, I have \"cam -c 2 -C\" running streaming from a USB camera using the\n> uvcvideo pipeline handler. If I exit process 2, I get the following error\n> messages:\n> \n> [0:01:33.155169911] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241\n> /dev/video2[16:cap]: Unable to request 0 buffers: Device or resource busy\n> [0:01:33.155359910] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241\n> /dev/video3[17:cap]: Unable to request 0 buffers: Device or resource busy\n> [0:01:33.155460595] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241\n> /dev/video13[18:out]: Unable to request 0 buffers: Device or resource busy\n> [0:01:33.155564354] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241\n> /dev/video14[19:cap]: Unable to request 0 buffers: Device or resource busy\n> [0:01:33.155664150] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241\n> /dev/video15[20:cap]: Unable to request 0 buffers: Device or resource busy\n> [0:01:33.155763483] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241\n> /dev/video16[21:cap]: Unable to request 0 buffers: Device or resource busy\n> \n> These errors come up because the V4L2VideoDevice destructor in process 2 is\n> indirectly calling ioctl(REQBUFS, 0) on all the Unicam and ISP device nodes\n> while the device driver buffer queues are actively owned and used by\n> process 1. This does not actively interfere with process 1 running, it\n> continues happily enough.\n> \n> The error here stems from the fact that the RPi pipeline handler\n> PipelineHandler::match() opens all the device nodes for every Camera object\n> registered through PipelineHandler::registerCamera(). So even if the\n> Camera was never used by a process, when terminating the V4L2VideoDevice\n> destructor will call ioctl(REQBUFS, 0) for every single device. This\n> behavior is exactly the same for the ipu3 and rkisp1 pipeline handlers.\n> \n> So the question is how to fix this and avoid the error messages?\n> \n> 1) The V4L2VideoDevice destructor should not call ioctl(REQBUFS, 0) if it\n> has not allocated buffers for the device:\n> \n> diff --git a/src/libcamera/v4l2_videodevice.cpp\n> b/src/libcamera/v4l2_videodevice.cpp\n> index e30858c9fa02..108e60f035ab 100644\n> --- a/src/libcamera/v4l2_videodevice.cpp\n> +++ b/src/libcamera/v4l2_videodevice.cpp\n> @@ -758,7 +758,8 @@ void V4L2VideoDevice::close()\n> if (!isOpen())\n> return;\n> \n> - releaseBuffers();\n> + if (cache_)\n> + releaseBuffers();\n> delete fdBufferNotifier_;\n> \n> 2) All pipeline handlers need to change and avoid calling device open() in\n> PipelineHandler::match(), and defer this call to when configure() gets\n> called.\n> \n> Option 2 is not trivial, as (at least in the RPi pipeline handler) a bunch\n> of validation occurs in PipelineHandler::match() that needs the device\n> nodes to be open. In my opinion, option 1 seems to be the correct thing to\n> do.\n> \n> Any other suggestions or thoughts?\n\nAs a quick reply, without looking through code - then yes, opeion 1\nseems most reasonable. We should not be trying to release things that we\nhaven't claimed in that process.\n\nAt least the kernel is preventing it from actually releasing any memory\n- (I believe) - but indeed - it's not a good position to be in, trying\nto free someone elses buffers.\n\nI'd check that releaseBuffers() isn't called anywhere else explicitly (I\nexpect this is just the destructor code path?) which might make the\ncache_ check better suited in releaseBuffers() itself ... but otherwise\nit seems sane to make sure we only release resources we acquire.\n\n--\nKieran\n\n\n\n> \n> Regards,\n> Naush","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A6F3BBDB16\n\tfor ;\n\tFri, 28 Oct 2022 10:24:31 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 126C062F97;\n\tFri, 28 Oct 2022 12:24:31 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 0D02061F4A\n\tfor ;\n\tFri, 28 Oct 2022 12:24:30 +0200 (CEST)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 927646BE;\n\tFri, 28 Oct 2022 12:24:29 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1666952671;\n\tbh=gD4DrrEW8/DyK/sP77GwOfqffGxKh6/DS6KnumRgAAw=;\n\th=In-Reply-To:References:To:Date:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=0qHuclP2DiLhtalqBowi7KlUorEeTm4bL5QGaXkaPY+YzmDW4iiYmKwUc+IUHUoHu\n\tiwdhaZ64tnziv5j6DpMDzx9WpMxwSL9AjEfn0cta0+3nHGjACm8rRtPMuEI62/DA32\n\t0LrfNX0UoPRuCAtOrJMn36Gxul+xZF52bFVsmxJ8/9JrX0M4OTK+0hOgFjAcpz+H07\n\tkHIOgVsdssdUJG7KQcoJcgWNlL+BgVh70yPUrNDKH8EOcy1IhNnDPEPwF5erZwOstA\n\tF6tk8s+tv1jhLeEJ5mUInGAsiafHmMbqrvAGoexULWOmHt2sElt5jkGBxKw6ciQCLW\n\tXoBG17xNsJ/FQ==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1666952669;\n\tbh=gD4DrrEW8/DyK/sP77GwOfqffGxKh6/DS6KnumRgAAw=;\n\th=In-Reply-To:References:Subject:From:To:Date:From;\n\tb=ZZ0XkqdLXr2SJ6/BLUNguEKEpyyWwvlkkJ6p6oxUs5PMU7uybpvKSEgaAMFyVXXPW\n\tDEmnpEXE1GvQTgWWYWjb/9hFbaUv4ip28gWTFsv5Kd6W8VqDEYQ1UyEVyydiP1+3Ec\n\tgQGPzjrEDw1e1Qm6REwBNNrScd+jKKu5A1bJr0jI="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"ZZ0XkqdL\"; dkim-atps=neutral","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"","References":"","To":"Naushir Patuck ,\n\tlibcamera devel ","Date":"Fri, 28 Oct 2022 11:24:27 +0100","Message-ID":"<166695266727.2560709.1182888743581387057@Monstersaurus>","User-Agent":"alot/0.10","Subject":"Re: [libcamera-devel] Concurrent camera usage in libcamera","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Kieran Bingham via libcamera-devel\n\t","Reply-To":"Kieran Bingham ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":25658,"web_url":"https://patchwork.libcamera.org/comment/25658/","msgid":"","date":"2022-10-28T11:31:24","subject":"Re: [libcamera-devel] Concurrent camera usage in libcamera","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"content":"On Fri, 28 Oct 2022 at 11:24, Kieran Bingham <\nkieran.bingham@ideasonboard.com> wrote:\n\n> Quoting Naushir Patuck via libcamera-devel (2022-10-28 10:42:55)\n> > Hi all,\n> >\n> > I wonder if I could get some thoughts regarding a problem we are seeing\n> > with running multiple cameras in separate libcamera processes on\n> Raspberry\n> > Pi (this probably affects any/all platforms though).\n> >\n> > In my scenario, I have libcamera process 1 running \"cam -c 1 -C\" and\n> > streaming from imx477 using the Raspberry Pi pipeline handler. In\n> process\n> > 2, I have \"cam -c 2 -C\" running streaming from a USB camera using the\n> > uvcvideo pipeline handler. If I exit process 2, I get the following\n> error\n> > messages:\n> >\n> > [0:01:33.155169911] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241\n> > /dev/video2[16:cap]: Unable to request 0 buffers: Device or resource busy\n> > [0:01:33.155359910] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241\n> > /dev/video3[17:cap]: Unable to request 0 buffers: Device or resource busy\n> > [0:01:33.155460595] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241\n> > /dev/video13[18:out]: Unable to request 0 buffers: Device or resource\n> busy\n> > [0:01:33.155564354] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241\n> > /dev/video14[19:cap]: Unable to request 0 buffers: Device or resource\n> busy\n> > [0:01:33.155664150] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241\n> > /dev/video15[20:cap]: Unable to request 0 buffers: Device or resource\n> busy\n> > [0:01:33.155763483] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241\n> > /dev/video16[21:cap]: Unable to request 0 buffers: Device or resource\n> busy\n> >\n> > These errors come up because the V4L2VideoDevice destructor in process 2\n> is\n> > indirectly calling ioctl(REQBUFS, 0) on all the Unicam and ISP device\n> nodes\n> > while the device driver buffer queues are actively owned and used by\n> > process 1. This does not actively interfere with process 1 running, it\n> > continues happily enough.\n> >\n> > The error here stems from the fact that the RPi pipeline handler\n> > PipelineHandler::match() opens all the device nodes for every Camera\n> object\n> > registered through PipelineHandler::registerCamera(). So even if the\n> > Camera was never used by a process, when terminating the V4L2VideoDevice\n> > destructor will call ioctl(REQBUFS, 0) for every single device. This\n> > behavior is exactly the same for the ipu3 and rkisp1 pipeline handlers.\n> >\n> > So the question is how to fix this and avoid the error messages?\n> >\n> > 1) The V4L2VideoDevice destructor should not call ioctl(REQBUFS, 0) if it\n> > has not allocated buffers for the device:\n> >\n> > diff --git a/src/libcamera/v4l2_videodevice.cpp\n> > b/src/libcamera/v4l2_videodevice.cpp\n> > index e30858c9fa02..108e60f035ab 100644\n> > --- a/src/libcamera/v4l2_videodevice.cpp\n> > +++ b/src/libcamera/v4l2_videodevice.cpp\n> > @@ -758,7 +758,8 @@ void V4L2VideoDevice::close()\n> > if (!isOpen())\n> > return;\n> >\n> > - releaseBuffers();\n> > + if (cache_)\n> > + releaseBuffers();\n> > delete fdBufferNotifier_;\n> >\n> > 2) All pipeline handlers need to change and avoid calling device open()\n> in\n> > PipelineHandler::match(), and defer this call to when configure() gets\n> > called.\n> >\n> > Option 2 is not trivial, as (at least in the RPi pipeline handler) a\n> bunch\n> > of validation occurs in PipelineHandler::match() that needs the device\n> > nodes to be open. In my opinion, option 1 seems to be the correct thing\n> to\n> > do.\n> >\n> > Any other suggestions or thoughts?\n>\n> As a quick reply, without looking through code - then yes, opeion 1\n> seems most reasonable. We should not be trying to release things that we\n> haven't claimed in that process.\n>\n> At least the kernel is preventing it from actually releasing any memory\n> - (I believe) - but indeed - it's not a good position to be in, trying\n> to free someone elses buffers.\n>\n\nThe kernel handles this correctly, so nothing stops working thankfully!\n\n\n>\n> I'd check that releaseBuffers() isn't called anywhere else explicitly (I\n> expect this is just the destructor code path?) which might make the\n> cache_ check better suited in releaseBuffers() itself ... but otherwise\n> it seems sane to make sure we only release resources we acquire.\n>\n\nreleaseBuffers() only ever gets called from close(), but I'll put the\ncache_ check in releaseBuffers() to be safe from possible future changes.\n\nNaush\n\n\n\n> --\n> Kieran\n>\n>\n>\n> >\n> > Regards,\n> > Naush\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 6BCA3BD16B\n\tfor ;\n\tFri, 28 Oct 2022 11:31:44 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A1D2D62FD6;\n\tFri, 28 Oct 2022 13:31:43 +0200 (CEST)","from mail-qt1-x835.google.com (mail-qt1-x835.google.com\n\t[IPv6:2607:f8b0:4864:20::835])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 9423661F4A\n\tfor ;\n\tFri, 28 Oct 2022 13:31:41 +0200 (CEST)","by mail-qt1-x835.google.com with SMTP id w29so3239133qtv.9\n\tfor ;\n\tFri, 28 Oct 2022 04:31:41 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1666956703;\n\tbh=UquZJZvQP2BlldUz4zLqmCxjM6Is1vEiQnakrdYM4+U=;\n\th=References:In-Reply-To:Date:To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=fJI5FFkwvDPL1Yh60EW+GTXS6HAbnmfu+kp4FVrgZw3q+rcE3OZ19SaLWw3p0wrsF\n\tG6Vm09h//as+uJ0C8KzMXkLJEfUiZfccF9bGp3CCaGjCvIZp+q++V239naw0sdLFyp\n\t90OsJqO+InlndVsQYmDUGNZx7BL0YygfER1HQGULu6jljhUmFCf4QiaYIvmXQxt4bh\n\t7/pfQTDNpsgkVR66vYWAAfnyVwYX1KQWd525Rs5Ra5K3qjKX7nbeaGqR+RxBJNwLZV\n\tkjgcu9b2eRwH2OaRoHag+5Ac83vO83u/o+v/AcFlnnYhQCthFBIBGRNaRxW8i8LCG1\n\tNCyvj1YoXJhrA==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:from:to:cc:subject:date:message-id:reply-to;\n\tbh=UNroj0381dC9XjxmST5b78SjC+TNzEu5M19dHW64cDs=;\n\tb=n9rz4wCedaGjCGeHb0diMd3gK7cqiEFwSITwYCE3eloHuI986MwdUpQvadcU+yCaSU\n\td01/0yHuFVqZN2RXIal23WMccewt6S92EhKNq9QVAW7PhMCtccy+fYSttiJgzcfw1si2\n\tPeX7PF/7X5kuLADY0VokJ7ppglad5e3Yb1hskxApU8iBAjVJIMiaEfW936pftI2L0Zrl\n\tiRhUGZowtXjpVPgvfOcpgkoQfo0/URGmP83ltCSub0A050XWeIut60UPVxwgNFqnJ0qC\n\tSFR0igte2KX93FrKIwvWZEc1rrNQLzCYAhEQjqM7x+wnrlNAwzMddv0GLf9WLhYwSyr3\n\t41qg=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=raspberrypi.com\n\theader.i=@raspberrypi.com\n\theader.b=\"n9rz4wCe\"; dkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:x-gm-message-state:from:to:cc:subject:date:message-id\n\t:reply-to;\n\tbh=UNroj0381dC9XjxmST5b78SjC+TNzEu5M19dHW64cDs=;\n\tb=e4KQ+MmwgPC50HjV3wKF0REEfeaxuMNdb+zQh5IZX+F3A1NYkSi3xWuXn7KRci3GkX\n\tt7FZHM4xnm/DdydHghXssupg8LNmxVtmxNdwO4JomMdFu/RTC3c1k8xfqZAZL+sDtaQI\n\t+EPwldJOuqcSGrVlfQDnIJkJdqf+PK+MnnrSDkMaIIizwHOCuosmnuFaUThv1nYwsvOg\n\t/ezPGeTmStela2w3GAgfuusU9foIpEDqzW8ckoPXVgqHPg99lryblJpYoyObnxSEZwEX\n\t0TvSXm+mdIfzxgSaTZNTJqv+xpmUSNQm/9bE2nGWr91wJDqi8kSk+65GnFt4QH8Tobp4\n\tBbVw==","X-Gm-Message-State":"ACrzQf0OJfWjOQUmofVwYIz0G+BYA9Sm5XEWqdp3lzyfkLkNlp6IddQs\n\tCHMSGlsoqpBgNXE1oyd8o86b6wrBKnRkunyeo9OO+X/ZNP0=","X-Google-Smtp-Source":"AMsMyM5yxZG7vonZ/BdUFH2/YhuMxLfonCGaZvd72JbOCFRY7mckPAadbFDz4BWoyGA1aL/lApioeSlsyVCYVGz3F4o=","X-Received":"by 2002:ac8:7dc4:0:b0:39c:f95f:57fe with SMTP id\n\tc4-20020ac87dc4000000b0039cf95f57femr42585721qte.612.1666956700500;\n\tFri, 28 Oct 2022 04:31:40 -0700 (PDT)","MIME-Version":"1.0","References":"\n\t<166695266727.2560709.1182888743581387057@Monstersaurus>","In-Reply-To":"<166695266727.2560709.1182888743581387057@Monstersaurus>","Date":"Fri, 28 Oct 2022 12:31:24 +0100","Message-ID":"","To":"Kieran Bingham ","Content-Type":"multipart/alternative; boundary=\"000000000000fd6ce505ec1699fb\"","Subject":"Re: [libcamera-devel] Concurrent camera usage in libcamera","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Naushir Patuck via libcamera-devel\n\t","Reply-To":"Naushir Patuck ","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]