[libcamera-devel,00/13] Support SensorTimestamp metadata
mbox series

Message ID 20210419131433.22920-1-jacopo@jmondi.org
Headers show
Series
  • Support SensorTimestamp metadata
Related show

Message

Jacopo Mondi April 19, 2021, 1:14 p.m. UTC
Hello,
  v2 grow quite significatively compared to v1 :/

Reason is that now all the pipeline handlers report SensorTimestamp, each
one with its own specificities.

Here and there a few drive-by fixes, and two patches for cam to add support
for listing all the metadata associated with a completed Request.

v1->v2:
- controls: Refer to CLOCK_BOOTTIME explicitely
- pipeline:
  - Add support for SensorTimestamp in all pipelines
- HAL
  - Assume SensorTimestamp is always available.

In future, it might be beneficial to add to lc-compliance a test to make
sure SensorTimestamp is reported. Maybe a 'mandatory' flag can be added to
the controls definition for that purpose.

Thanks
   j

Jacopo Mondi (13):
  libcamera: controls: Destage 'SensorTimestamp'
  libcamera: ipu3: Report sensor timestamp
  libcamera: uvc: Report sensor timestamp
  libcamera: rkisp1: Do not over-write metadata
  libcamera: rkisp1: Report sensor timestamp
  libcamera: simple: Report sensor timestamp
  libcamera: vimc: Report sensor timestamp
  libcamera: buffer: Re-work setRequest() documentation
  libcamera: raspberrypi: Do not over-write metadata
  libcamera: raspberry: Report sensor timestamp
  cam: Add option to print the Request metadata
  cam: Implement OptMetadata
  android: camera_device: Use controls::SensorTimestamp

 src/android/camera_device.cpp                 | 19 +++++-----------
 src/android/camera_device.h                   |  3 +--
 src/cam/capture.cpp                           | 15 ++++++++++++-
 src/cam/capture.h                             |  1 +
 src/cam/main.cpp                              |  3 +++
 src/cam/main.h                                |  1 +
 src/libcamera/buffer.cpp                      | 11 ++++++++--
 src/libcamera/control_ids.yaml                | 21 ++++++++++++------
 src/libcamera/pipeline/ipu3/ipu3.cpp          |  9 ++++++++
 .../pipeline/raspberrypi/raspberrypi.cpp      | 22 +++++++++++++++++--
 src/libcamera/pipeline/rkisp1/rkisp1.cpp      | 13 ++++++++++-
 src/libcamera/pipeline/simple/simple.cpp      | 11 ++++++++++
 src/libcamera/pipeline/uvcvideo/uvcvideo.cpp  |  3 +++
 src/libcamera/pipeline/vimc/vimc.cpp          |  4 ++++
 14 files changed, 108 insertions(+), 28 deletions(-)

--
2.31.1

Comments

Kieran Bingham April 19, 2021, 1:31 p.m. UTC | #1
Hi Jacopo,

On 19/04/2021 14:14, Jacopo Mondi wrote:
> Hello,
>   v2 grow quite significatively compared to v1 :/
> 
> Reason is that now all the pipeline handlers report SensorTimestamp, each
> one with its own specificities.
> 
> Here and there a few drive-by fixes, and two patches for cam to add support
> for listing all the metadata associated with a completed Request.
> 
> v1->v2:
> - controls: Refer to CLOCK_BOOTTIME explicitely
> - pipeline:
>   - Add support for SensorTimestamp in all pipelines
> - HAL
>   - Assume SensorTimestamp is always available.
> 
> In future, it might be beneficial to add to lc-compliance a test to make
> sure SensorTimestamp is reported. Maybe a 'mandatory' flag can be added to
> the controls definition for that purpose.

Aha, I came here specifically to say ... shouldn't we add a test to
lc-compliance to make sure the timestamp is set ...

:-)

I'm not sure if a mandatory flag makes sense yet - but perhaps it could
simplify things.

Are there other expected mandatory metadata values in requests?

--
Kieran


> Thanks
>    j
> 
> Jacopo Mondi (13):
>   libcamera: controls: Destage 'SensorTimestamp'
>   libcamera: ipu3: Report sensor timestamp
>   libcamera: uvc: Report sensor timestamp
>   libcamera: rkisp1: Do not over-write metadata
>   libcamera: rkisp1: Report sensor timestamp
>   libcamera: simple: Report sensor timestamp
>   libcamera: vimc: Report sensor timestamp
>   libcamera: buffer: Re-work setRequest() documentation
>   libcamera: raspberrypi: Do not over-write metadata
>   libcamera: raspberry: Report sensor timestamp
>   cam: Add option to print the Request metadata
>   cam: Implement OptMetadata
>   android: camera_device: Use controls::SensorTimestamp
> 
>  src/android/camera_device.cpp                 | 19 +++++-----------
>  src/android/camera_device.h                   |  3 +--
>  src/cam/capture.cpp                           | 15 ++++++++++++-
>  src/cam/capture.h                             |  1 +
>  src/cam/main.cpp                              |  3 +++
>  src/cam/main.h                                |  1 +
>  src/libcamera/buffer.cpp                      | 11 ++++++++--
>  src/libcamera/control_ids.yaml                | 21 ++++++++++++------
>  src/libcamera/pipeline/ipu3/ipu3.cpp          |  9 ++++++++
>  .../pipeline/raspberrypi/raspberrypi.cpp      | 22 +++++++++++++++++--
>  src/libcamera/pipeline/rkisp1/rkisp1.cpp      | 13 ++++++++++-
>  src/libcamera/pipeline/simple/simple.cpp      | 11 ++++++++++
>  src/libcamera/pipeline/uvcvideo/uvcvideo.cpp  |  3 +++
>  src/libcamera/pipeline/vimc/vimc.cpp          |  4 ++++
>  14 files changed, 108 insertions(+), 28 deletions(-)
> 
> --
> 2.31.1
> 
> _______________________________________________
> libcamera-devel mailing list
> libcamera-devel@lists.libcamera.org
> https://lists.libcamera.org/listinfo/libcamera-devel
>
Laurent Pinchart April 20, 2021, 8:48 p.m. UTC | #2
Hi Jacopo,

On Mon, Apr 19, 2021 at 03:14:20PM +0200, Jacopo Mondi wrote:
> Hello,
>   v2 grow quite significatively compared to v1 :/
> 
> Reason is that now all the pipeline handlers report SensorTimestamp, each
> one with its own specificities.
> 
> Here and there a few drive-by fixes, and two patches for cam to add support
> for listing all the metadata associated with a completed Request.
> 
> v1->v2:
> - controls: Refer to CLOCK_BOOTTIME explicitely
> - pipeline:
>   - Add support for SensorTimestamp in all pipelines
> - HAL
>   - Assume SensorTimestamp is always available.
> 
> In future, it might be beneficial to add to lc-compliance a test to make
> sure SensorTimestamp is reported. Maybe a 'mandatory' flag can be added to
> the controls definition for that purpose.

That's a good idea. I foresee that libcamera will define device profiles
in the future, similar to the Android camera HAL hardware levels, and
controls will then be documented as optional/mandatory for the different
profiles. We can start with a simple mandatory/required flag.

> Jacopo Mondi (13):
>   libcamera: controls: Destage 'SensorTimestamp'
>   libcamera: ipu3: Report sensor timestamp
>   libcamera: uvc: Report sensor timestamp
>   libcamera: rkisp1: Do not over-write metadata
>   libcamera: rkisp1: Report sensor timestamp
>   libcamera: simple: Report sensor timestamp
>   libcamera: vimc: Report sensor timestamp
>   libcamera: buffer: Re-work setRequest() documentation
>   libcamera: raspberrypi: Do not over-write metadata
>   libcamera: raspberry: Report sensor timestamp
>   cam: Add option to print the Request metadata
>   cam: Implement OptMetadata
>   android: camera_device: Use controls::SensorTimestamp
> 
>  src/android/camera_device.cpp                 | 19 +++++-----------
>  src/android/camera_device.h                   |  3 +--
>  src/cam/capture.cpp                           | 15 ++++++++++++-
>  src/cam/capture.h                             |  1 +
>  src/cam/main.cpp                              |  3 +++
>  src/cam/main.h                                |  1 +
>  src/libcamera/buffer.cpp                      | 11 ++++++++--
>  src/libcamera/control_ids.yaml                | 21 ++++++++++++------
>  src/libcamera/pipeline/ipu3/ipu3.cpp          |  9 ++++++++
>  .../pipeline/raspberrypi/raspberrypi.cpp      | 22 +++++++++++++++++--
>  src/libcamera/pipeline/rkisp1/rkisp1.cpp      | 13 ++++++++++-
>  src/libcamera/pipeline/simple/simple.cpp      | 11 ++++++++++
>  src/libcamera/pipeline/uvcvideo/uvcvideo.cpp  |  3 +++
>  src/libcamera/pipeline/vimc/vimc.cpp          |  4 ++++
>  14 files changed, 108 insertions(+), 28 deletions(-)
Laurent Pinchart April 20, 2021, 8:50 p.m. UTC | #3
Hi Jacopo,

On Mon, Apr 19, 2021 at 02:31:42PM +0100, Kieran Bingham wrote:
> On 19/04/2021 14:14, Jacopo Mondi wrote:
> > Hello,
> >   v2 grow quite significatively compared to v1 :/
> > 
> > Reason is that now all the pipeline handlers report SensorTimestamp, each
> > one with its own specificities.
> > 
> > Here and there a few drive-by fixes, and two patches for cam to add support
> > for listing all the metadata associated with a completed Request.
> > 
> > v1->v2:
> > - controls: Refer to CLOCK_BOOTTIME explicitely
> > - pipeline:
> >   - Add support for SensorTimestamp in all pipelines
> > - HAL
> >   - Assume SensorTimestamp is always available.
> > 
> > In future, it might be beneficial to add to lc-compliance a test to make
> > sure SensorTimestamp is reported. Maybe a 'mandatory' flag can be added to
> > the controls definition for that purpose.
> 
> Aha, I came here specifically to say ... shouldn't we add a test to
> lc-compliance to make sure the timestamp is set ...

Could you file a feature request for lc-compliance in bugs.libcamera.org
?

> :-)
> 
> I'm not sure if a mandatory flag makes sense yet - but perhaps it could
> simplify things.
> 
> Are there other expected mandatory metadata values in requests?
> 
> --
> Kieran
> 
> 
> > Thanks
> >    j
> > 
> > Jacopo Mondi (13):
> >   libcamera: controls: Destage 'SensorTimestamp'
> >   libcamera: ipu3: Report sensor timestamp
> >   libcamera: uvc: Report sensor timestamp
> >   libcamera: rkisp1: Do not over-write metadata
> >   libcamera: rkisp1: Report sensor timestamp
> >   libcamera: simple: Report sensor timestamp
> >   libcamera: vimc: Report sensor timestamp
> >   libcamera: buffer: Re-work setRequest() documentation
> >   libcamera: raspberrypi: Do not over-write metadata
> >   libcamera: raspberry: Report sensor timestamp
> >   cam: Add option to print the Request metadata
> >   cam: Implement OptMetadata
> >   android: camera_device: Use controls::SensorTimestamp
> > 
> >  src/android/camera_device.cpp                 | 19 +++++-----------
> >  src/android/camera_device.h                   |  3 +--
> >  src/cam/capture.cpp                           | 15 ++++++++++++-
> >  src/cam/capture.h                             |  1 +
> >  src/cam/main.cpp                              |  3 +++
> >  src/cam/main.h                                |  1 +
> >  src/libcamera/buffer.cpp                      | 11 ++++++++--
> >  src/libcamera/control_ids.yaml                | 21 ++++++++++++------
> >  src/libcamera/pipeline/ipu3/ipu3.cpp          |  9 ++++++++
> >  .../pipeline/raspberrypi/raspberrypi.cpp      | 22 +++++++++++++++++--
> >  src/libcamera/pipeline/rkisp1/rkisp1.cpp      | 13 ++++++++++-
> >  src/libcamera/pipeline/simple/simple.cpp      | 11 ++++++++++
> >  src/libcamera/pipeline/uvcvideo/uvcvideo.cpp  |  3 +++
> >  src/libcamera/pipeline/vimc/vimc.cpp          |  4 ++++
> >  14 files changed, 108 insertions(+), 28 deletions(-)
Jacopo Mondi April 21, 2021, 2:44 p.m. UTC | #4
Hello,

On Tue, Apr 20, 2021 at 11:50:12PM +0300, Laurent Pinchart wrote:
> Hi Jacopo,
>
> On Mon, Apr 19, 2021 at 02:31:42PM +0100, Kieran Bingham wrote:
> > On 19/04/2021 14:14, Jacopo Mondi wrote:
> > > Hello,
> > >   v2 grow quite significatively compared to v1 :/
> > >
> > > Reason is that now all the pipeline handlers report SensorTimestamp, each
> > > one with its own specificities.
> > >
> > > Here and there a few drive-by fixes, and two patches for cam to add support
> > > for listing all the metadata associated with a completed Request.
> > >
> > > v1->v2:
> > > - controls: Refer to CLOCK_BOOTTIME explicitely
> > > - pipeline:
> > >   - Add support for SensorTimestamp in all pipelines
> > > - HAL
> > >   - Assume SensorTimestamp is always available.
> > >
> > > In future, it might be beneficial to add to lc-compliance a test to make
> > > sure SensorTimestamp is reported. Maybe a 'mandatory' flag can be added to
> > > the controls definition for that purpose.
> >
> > Aha, I came here specifically to say ... shouldn't we add a test to
> > lc-compliance to make sure the timestamp is set ...
>
> Could you file a feature request for lc-compliance in bugs.libcamera.org
> ?
>

Done
https://bugs.libcamera.org/show_bug.cgi?id=27

> > :-)
> >
> > I'm not sure if a mandatory flag makes sense yet - but perhaps it could
> > simplify things.
> >
> > Are there other expected mandatory metadata values in requests?
> >
> > --
> > Kieran
> >
> >
> > > Thanks
> > >    j
> > >
> > > Jacopo Mondi (13):
> > >   libcamera: controls: Destage 'SensorTimestamp'
> > >   libcamera: ipu3: Report sensor timestamp
> > >   libcamera: uvc: Report sensor timestamp
> > >   libcamera: rkisp1: Do not over-write metadata
> > >   libcamera: rkisp1: Report sensor timestamp
> > >   libcamera: simple: Report sensor timestamp
> > >   libcamera: vimc: Report sensor timestamp
> > >   libcamera: buffer: Re-work setRequest() documentation
> > >   libcamera: raspberrypi: Do not over-write metadata
> > >   libcamera: raspberry: Report sensor timestamp
> > >   cam: Add option to print the Request metadata
> > >   cam: Implement OptMetadata
> > >   android: camera_device: Use controls::SensorTimestamp
> > >
> > >  src/android/camera_device.cpp                 | 19 +++++-----------
> > >  src/android/camera_device.h                   |  3 +--
> > >  src/cam/capture.cpp                           | 15 ++++++++++++-
> > >  src/cam/capture.h                             |  1 +
> > >  src/cam/main.cpp                              |  3 +++
> > >  src/cam/main.h                                |  1 +
> > >  src/libcamera/buffer.cpp                      | 11 ++++++++--
> > >  src/libcamera/control_ids.yaml                | 21 ++++++++++++------
> > >  src/libcamera/pipeline/ipu3/ipu3.cpp          |  9 ++++++++
> > >  .../pipeline/raspberrypi/raspberrypi.cpp      | 22 +++++++++++++++++--
> > >  src/libcamera/pipeline/rkisp1/rkisp1.cpp      | 13 ++++++++++-
> > >  src/libcamera/pipeline/simple/simple.cpp      | 11 ++++++++++
> > >  src/libcamera/pipeline/uvcvideo/uvcvideo.cpp  |  3 +++
> > >  src/libcamera/pipeline/vimc/vimc.cpp          |  4 ++++
> > >  14 files changed, 108 insertions(+), 28 deletions(-)
>
> --
> Regards,
>
> Laurent Pinchart