[libcamera-devel,0/3] Sequence Observer

15750 mbox series

This fell out of some more work I did in the past catching and detecting
frame drops in streams. While the method of reporting this event to
userspace is debated, this pulls out some core functionality and wraps
some generic code into a Sequence Observer to provide an easy to use
check on any monotonic sequence.

For now - it simply reports a warning if a V4L2 device detects a frame
drop, which is more notification that we have currently... as we are
silently dropping frames at the moment.

Kieran Bingham (3):
  libcamera: Add sequence value observer
  test: Add Sequence observer tests
  libcamera: v4l2: Detect frame drops on V4L2VideoDevice

 include/libcamera/internal/v4l2_videodevice.h |  2 +
 include/libcamera/meson.build                 |  1 +
 include/libcamera/sequence.h                  | 20 +++++
 src/libcamera/meson.build                     |  1 +
 src/libcamera/sequence.cpp                    | 66 +++++++++++++++
 src/libcamera/v4l2_videodevice.cpp            | 10 +++
 test/meson.build                              |  1 +
 test/sequence.cpp                             | 80 +++++++++++++++++++
 8 files changed, 181 insertions(+)
 create mode 100644 include/libcamera/sequence.h
 create mode 100644 src/libcamera/sequence.cpp
 create mode 100644 test/sequence.cpp

Hi Kieran,

On Fri, Apr 29, 2022 at 09:34:31PM +0200, Kieran Bingham via libcamera-devel wrote:
> This fell out of some more work I did in the past catching and detecting
> frame drops in streams. While the method of reporting this event to
> userspace is debated, this pulls out some core functionality and wraps
> some generic code into a Sequence Observer to provide an easy to use
> check on any monotonic sequence.
> 
> For now - it simply reports a warning if a V4L2 device detects a frame
> drop, which is more notification that we have currently... as we are
> silently dropping frames at the moment.

Even if we haven't finalized the way to handle frame drops in the
application-facing API, I think it's useful to be able to be notified of
frame drops already. I'm however thinking that the sequence observer
class is a bit overkill at this point. I'd recommend starting with an
ad-hoc solution in V4L2VideoDevice only, and later extend it if
required.

> Kieran Bingham (3):
>   libcamera: Add sequence value observer
>   test: Add Sequence observer tests
>   libcamera: v4l2: Detect frame drops on V4L2VideoDevice
> 
>  include/libcamera/internal/v4l2_videodevice.h |  2 +
>  include/libcamera/meson.build                 |  1 +
>  include/libcamera/sequence.h                  | 20 +++++
>  src/libcamera/meson.build                     |  1 +
>  src/libcamera/sequence.cpp                    | 66 +++++++++++++++
>  src/libcamera/v4l2_videodevice.cpp            | 10 +++
>  test/meson.build                              |  1 +
>  test/sequence.cpp                             | 80 +++++++++++++++++++
>  8 files changed, 181 insertions(+)
>  create mode 100644 include/libcamera/sequence.h
>  create mode 100644 src/libcamera/sequence.cpp
>  create mode 100644 test/sequence.cpp

Quoting Laurent Pinchart (2022-05-09 14:33:00)
> Hi Kieran,
> 
> On Fri, Apr 29, 2022 at 09:34:31PM +0200, Kieran Bingham via libcamera-devel wrote:
> > This fell out of some more work I did in the past catching and detecting
> > frame drops in streams. While the method of reporting this event to
> > userspace is debated, this pulls out some core functionality and wraps
> > some generic code into a Sequence Observer to provide an easy to use
> > check on any monotonic sequence.
> > 
> > For now - it simply reports a warning if a V4L2 device detects a frame
> > drop, which is more notification that we have currently... as we are
> > silently dropping frames at the moment.
> 
> Even if we haven't finalized the way to handle frame drops in the
> application-facing API, I think it's useful to be able to be notified of
> frame drops already. I'm however thinking that the sequence observer
> class is a bit overkill at this point. I'd recommend starting with an
> ad-hoc solution in V4L2VideoDevice only, and later extend it if
> required.

Of course this did start of as an ad-hoc solution in V4L2VideoDevice and
then became the observer because I had the same snippets of
functionality in 4 different places. (But I think you don't like those
either so ... we'll find out later).

But anyway, that's why I split this out as I thought it was a helpful
... helper.

As it happens, I need to put more thought and consideration into this
anyway, as a warning print really is too noisy anyway - so I don't think
it can even suffice as a starting point to print a warning (Perhaps a
Debug though)

(I am partialy concerned that we can have so many frame drops that a
warning print can be too noisy, but I think that it probably would be
expected behaviour in some scenarios, even if we weren't using
libcamera).

> 
> > Kieran Bingham (3):
> >   libcamera: Add sequence value observer
> >   test: Add Sequence observer tests
> >   libcamera: v4l2: Detect frame drops on V4L2VideoDevice
> > 
> >  include/libcamera/internal/v4l2_videodevice.h |  2 +
> >  include/libcamera/meson.build                 |  1 +
> >  include/libcamera/sequence.h                  | 20 +++++
> >  src/libcamera/meson.build                     |  1 +
> >  src/libcamera/sequence.cpp                    | 66 +++++++++++++++
> >  src/libcamera/v4l2_videodevice.cpp            | 10 +++
> >  test/meson.build                              |  1 +
> >  test/sequence.cpp                             | 80 +++++++++++++++++++
> >  8 files changed, 181 insertions(+)
> >  create mode 100644 include/libcamera/sequence.h
> >  create mode 100644 src/libcamera/sequence.cpp
> >  create mode 100644 test/sequence.cpp
> 
> -- 
> Regards,
> 
> Laurent Pinchart