[libcamera-devel] Concurrent camera usage in libcamera
diff mbox series

Message ID CAEmqJPqRYY8T6u1vBFv1aXjkaH60fK3WhkMZSEGQEhoiruk0Zw@mail.gmail.com
State Not Applicable
Headers show
Series
  • [libcamera-devel] Concurrent camera usage in libcamera
Related show

Commit Message

Naushir Patuck Oct. 28, 2022, 9:42 a.m. UTC 
Hi all,

I wonder if I could get some thoughts regarding a problem we are seeing
with running multiple cameras in separate libcamera processes on Raspberry
Pi (this probably affects any/all platforms though).

In my scenario, I have libcamera process 1 running "cam -c 1 -C" and
streaming from imx477 using the Raspberry Pi pipeline handler.  In process
2, I have "cam -c 2 -C" running streaming from a USB camera using the
uvcvideo pipeline handler.  If I exit process 2, I get the following error
messages:

[0:01:33.155169911] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
/dev/video2[16:cap]: Unable to request 0 buffers: Device or resource busy
[0:01:33.155359910] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
/dev/video3[17:cap]: Unable to request 0 buffers: Device or resource busy
[0:01:33.155460595] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
/dev/video13[18:out]: Unable to request 0 buffers: Device or resource busy
[0:01:33.155564354] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
/dev/video14[19:cap]: Unable to request 0 buffers: Device or resource busy
[0:01:33.155664150] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
/dev/video15[20:cap]: Unable to request 0 buffers: Device or resource busy
[0:01:33.155763483] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
/dev/video16[21:cap]: Unable to request 0 buffers: Device or resource busy

These errors come up because the V4L2VideoDevice destructor in process 2 is
indirectly calling ioctl(REQBUFS, 0) on all the Unicam and ISP device nodes
while the device driver buffer queues are actively owned and used by
process 1. This does not actively interfere with process 1 running, it
continues happily enough.

The error here stems from the fact that the RPi pipeline handler
PipelineHandler::match() opens all the device nodes for every Camera object
registered through PipelineHandler::registerCamera().  So even if the
Camera was never used by a process, when terminating the V4L2VideoDevice
destructor will call ioctl(REQBUFS, 0) for every single device.  This
behavior is exactly the same for the ipu3 and rkisp1 pipeline handlers.

So the question is how to fix this and avoid the error messages?

1) The V4L2VideoDevice destructor should not call ioctl(REQBUFS, 0) if it
has not allocated buffers for the device:

PipelineHandler::match(), and defer this call to when configure() gets
called.

Option 2 is not trivial, as (at least in the RPi pipeline handler) a bunch
of validation occurs in PipelineHandler::match() that needs the device
nodes to be open. In my opinion, option 1 seems to be the correct thing to
do.

Any other suggestions or thoughts?

Regards,
Naush

Comments

Kieran Bingham Oct. 28, 2022, 10:24 a.m. UTC | #1 
Quoting Naushir Patuck via libcamera-devel (2022-10-28 10:42:55)
> Hi all,
> 
> I wonder if I could get some thoughts regarding a problem we are seeing
> with running multiple cameras in separate libcamera processes on Raspberry
> Pi (this probably affects any/all platforms though).
> 
> In my scenario, I have libcamera process 1 running "cam -c 1 -C" and
> streaming from imx477 using the Raspberry Pi pipeline handler.  In process
> 2, I have "cam -c 2 -C" running streaming from a USB camera using the
> uvcvideo pipeline handler.  If I exit process 2, I get the following error
> messages:
> 
> [0:01:33.155169911] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
> /dev/video2[16:cap]: Unable to request 0 buffers: Device or resource busy
> [0:01:33.155359910] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
> /dev/video3[17:cap]: Unable to request 0 buffers: Device or resource busy
> [0:01:33.155460595] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
> /dev/video13[18:out]: Unable to request 0 buffers: Device or resource busy
> [0:01:33.155564354] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
> /dev/video14[19:cap]: Unable to request 0 buffers: Device or resource busy
> [0:01:33.155664150] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
> /dev/video15[20:cap]: Unable to request 0 buffers: Device or resource busy
> [0:01:33.155763483] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
> /dev/video16[21:cap]: Unable to request 0 buffers: Device or resource busy
> 
> These errors come up because the V4L2VideoDevice destructor in process 2 is
> indirectly calling ioctl(REQBUFS, 0) on all the Unicam and ISP device nodes
> while the device driver buffer queues are actively owned and used by
> process 1. This does not actively interfere with process 1 running, it
> continues happily enough.
> 
> The error here stems from the fact that the RPi pipeline handler
> PipelineHandler::match() opens all the device nodes for every Camera object
> registered through PipelineHandler::registerCamera().  So even if the
> Camera was never used by a process, when terminating the V4L2VideoDevice
> destructor will call ioctl(REQBUFS, 0) for every single device.  This
> behavior is exactly the same for the ipu3 and rkisp1 pipeline handlers.
> 
> So the question is how to fix this and avoid the error messages?
> 
> 1) The V4L2VideoDevice destructor should not call ioctl(REQBUFS, 0) if it
> has not allocated buffers for the device:
> 
> diff --git a/src/libcamera/v4l2_videodevice.cpp
> b/src/libcamera/v4l2_videodevice.cpp
> index e30858c9fa02..108e60f035ab 100644
> --- a/src/libcamera/v4l2_videodevice.cpp
> +++ b/src/libcamera/v4l2_videodevice.cpp
> @@ -758,7 +758,8 @@ void V4L2VideoDevice::close()
>         if (!isOpen())
>                 return;
> 
> -       releaseBuffers();
> +       if (cache_)
> +               releaseBuffers();
>         delete fdBufferNotifier_;
> 
> 2) All pipeline handlers need to change and avoid calling device open() in
> PipelineHandler::match(), and defer this call to when configure() gets
> called.
> 
> Option 2 is not trivial, as (at least in the RPi pipeline handler) a bunch
> of validation occurs in PipelineHandler::match() that needs the device
> nodes to be open. In my opinion, option 1 seems to be the correct thing to
> do.
> 
> Any other suggestions or thoughts?

As a quick reply, without looking through code - then yes, opeion 1
seems most reasonable. We should not be trying to release things that we
haven't claimed in that process.

At least the kernel is preventing it from actually releasing any memory
- (I believe) - but indeed - it's not a good position to be in, trying
to free someone elses buffers.

I'd check that releaseBuffers() isn't called anywhere else explicitly (I
expect this is just the destructor code path?) which might make the
cache_ check better suited in releaseBuffers() itself ... but otherwise
it seems sane to make sure we only release resources we acquire.

--
Kieran



> 
> Regards,
> Naush
Naushir Patuck Oct. 28, 2022, 11:31 a.m. UTC | #2 
On Fri, 28 Oct 2022 at 11:24, Kieran Bingham <
kieran.bingham@ideasonboard.com> wrote:

> Quoting Naushir Patuck via libcamera-devel (2022-10-28 10:42:55)
> > Hi all,
> >
> > I wonder if I could get some thoughts regarding a problem we are seeing
> > with running multiple cameras in separate libcamera processes on
> Raspberry
> > Pi (this probably affects any/all platforms though).
> >
> > In my scenario, I have libcamera process 1 running "cam -c 1 -C" and
> > streaming from imx477 using the Raspberry Pi pipeline handler.  In
> process
> > 2, I have "cam -c 2 -C" running streaming from a USB camera using the
> > uvcvideo pipeline handler.  If I exit process 2, I get the following
> error
> > messages:
> >
> > [0:01:33.155169911] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
> > /dev/video2[16:cap]: Unable to request 0 buffers: Device or resource busy
> > [0:01:33.155359910] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
> > /dev/video3[17:cap]: Unable to request 0 buffers: Device or resource busy
> > [0:01:33.155460595] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
> > /dev/video13[18:out]: Unable to request 0 buffers: Device or resource
> busy
> > [0:01:33.155564354] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
> > /dev/video14[19:cap]: Unable to request 0 buffers: Device or resource
> busy
> > [0:01:33.155664150] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
> > /dev/video15[20:cap]: Unable to request 0 buffers: Device or resource
> busy
> > [0:01:33.155763483] [1392] ERROR V4L2 v4l2_videodevice.cpp:1241
> > /dev/video16[21:cap]: Unable to request 0 buffers: Device or resource
> busy
> >
> > These errors come up because the V4L2VideoDevice destructor in process 2
> is
> > indirectly calling ioctl(REQBUFS, 0) on all the Unicam and ISP device
> nodes
> > while the device driver buffer queues are actively owned and used by
> > process 1. This does not actively interfere with process 1 running, it
> > continues happily enough.
> >
> > The error here stems from the fact that the RPi pipeline handler
> > PipelineHandler::match() opens all the device nodes for every Camera
> object
> > registered through PipelineHandler::registerCamera().  So even if the
> > Camera was never used by a process, when terminating the V4L2VideoDevice
> > destructor will call ioctl(REQBUFS, 0) for every single device.  This
> > behavior is exactly the same for the ipu3 and rkisp1 pipeline handlers.
> >
> > So the question is how to fix this and avoid the error messages?
> >
> > 1) The V4L2VideoDevice destructor should not call ioctl(REQBUFS, 0) if it
> > has not allocated buffers for the device:
> >
> > diff --git a/src/libcamera/v4l2_videodevice.cpp
> > b/src/libcamera/v4l2_videodevice.cpp
> > index e30858c9fa02..108e60f035ab 100644
> > --- a/src/libcamera/v4l2_videodevice.cpp
> > +++ b/src/libcamera/v4l2_videodevice.cpp
> > @@ -758,7 +758,8 @@ void V4L2VideoDevice::close()
> >         if (!isOpen())
> >                 return;
> >
> > -       releaseBuffers();
> > +       if (cache_)
> > +               releaseBuffers();
> >         delete fdBufferNotifier_;
> >
> > 2) All pipeline handlers need to change and avoid calling device open()
> in
> > PipelineHandler::match(), and defer this call to when configure() gets
> > called.
> >
> > Option 2 is not trivial, as (at least in the RPi pipeline handler) a
> bunch
> > of validation occurs in PipelineHandler::match() that needs the device
> > nodes to be open. In my opinion, option 1 seems to be the correct thing
> to
> > do.
> >
> > Any other suggestions or thoughts?
>
> As a quick reply, without looking through code - then yes, opeion 1
> seems most reasonable. We should not be trying to release things that we
> haven't claimed in that process.
>
> At least the kernel is preventing it from actually releasing any memory
> - (I believe) - but indeed - it's not a good position to be in, trying
> to free someone elses buffers.
>

The kernel handles this correctly, so nothing stops working thankfully!


>
> I'd check that releaseBuffers() isn't called anywhere else explicitly (I
> expect this is just the destructor code path?) which might make the
> cache_ check better suited in releaseBuffers() itself ... but otherwise
> it seems sane to make sure we only release resources we acquire.
>

releaseBuffers() only ever gets called from close(), but I'll put the
cache_ check in releaseBuffers() to be safe from possible future changes.

Naush



> --
> Kieran
>
>
>
> >
> > Regards,
> > Naush
>

Patch
diff mbox series 

diff --git a/src/libcamera/v4l2_videodevice.cpp
b/src/libcamera/v4l2_videodevice.cpp
index e30858c9fa02..108e60f035ab 100644
--- a/src/libcamera/v4l2_videodevice.cpp
+++ b/src/libcamera/v4l2_videodevice.cpp
@@ -758,7 +758,8 @@  void V4L2VideoDevice::close()
        if (!isOpen())
                return;

-       releaseBuffers();
+       if (cache_)
+               releaseBuffers();
        delete fdBufferNotifier_;

2) All pipeline handlers need to change and avoid calling device open() in