[libcamera-devel,4/8] cam: Use Sensor sequence numbers and detect frame drop
diff mbox series

Message ID 20211206233948.1351206-5-kieran.bingham@ideasonboard.com
State New
Delegated to: Kieran Bingham
Headers show
Series
  • Request metadata: SensorSequence
Related show

Commit Message

Kieran Bingham Dec. 6, 2021, 11:39 p.m. UTC
The stream buffer sequence numbers might produce sequential
monotonic sequence numbers from an ISP producing a frame for every
input.

This however, doesn't capture pipeline stalls that cause us to miss or
drop frames from the sensor.

Use the SensorSequence metadata to report sequence information, and add
a warning to the summary line if a frame drop is detected.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
---
 src/cam/camera_session.cpp | 17 +++++++++++++++--
 src/cam/camera_session.h   |  1 +
 2 files changed, 16 insertions(+), 2 deletions(-)

Comments

Umang Jain Dec. 7, 2021, 1:56 p.m. UTC | #1
Hi Kieran,

On 12/7/21 5:09 AM, Kieran Bingham wrote:
> The stream buffer sequence numbers might produce sequential
> monotonic sequence numbers from an ISP producing a frame for every
> input.
>
> This however, doesn't capture pipeline stalls that cause us to miss or
> drop frames from the sensor.
>
> Use the SensorSequence metadata to report sequence information, and add
> a warning to the summary line if a frame drop is detected.
>
> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
> ---
>   src/cam/camera_session.cpp | 17 +++++++++++++++--
>   src/cam/camera_session.h   |  1 +
>   2 files changed, 16 insertions(+), 2 deletions(-)
>
> diff --git a/src/cam/camera_session.cpp b/src/cam/camera_session.cpp
> index 50170723c30f..f9495f7f7e42 100644
> --- a/src/cam/camera_session.cpp
> +++ b/src/cam/camera_session.cpp
> @@ -29,8 +29,8 @@ CameraSession::CameraSession(CameraManager *cm,
>   			     unsigned int cameraIndex,
>   			     const OptionsParser::Options &options)
>   	: options_(options), cameraIndex_(cameraIndex), last_(0),
> -	  queueCount_(0), captureCount_(0), captureLimit_(0),
> -	  printMetadata_(false)
> +	  sequence_(0), queueCount_(0), captureCount_(0),
> +	  captureLimit_(0), printMetadata_(false)
>   {
>   	char *endptr;
>   	unsigned long index = strtoul(cameraId.c_str(), &endptr, 10);
> @@ -375,6 +375,11 @@ void CameraSession::processRequest(Request *request)
>   	     << std::setw(6) << std::setfill('0') << ts / 1000 % 1000000
>   	     << " (" << std::fixed << std::setprecision(2) << fps << " fps)";
>   
> +	uint64_t sequence = request->metadata().get(controls::SensorSequence);
> +	// Report capture sequence number


no checkstyle complain here?

Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>

> +	info << " [" << std::setw(6) << std::setfill('0')
> +	     << sequence << "]";
> +
>   	for (auto it = buffers.begin(); it != buffers.end(); ++it) {
>   		const Stream *stream = it->first;
>   		FrameBuffer *buffer = it->second;
> @@ -398,6 +403,14 @@ void CameraSession::processRequest(Request *request)
>   			requeue = false;
>   	}
>   
> +	/* Handle basic frame drop detection and reporting. */
> +	if (sequence_ == 0)
> +		sequence_ = sequence - 1;
> +	unsigned int drops = sequence - sequence_ - 1;
> +	if (drops)
> +		info << " *" << drops << " frame drops detected* ";
> +	sequence_ = sequence;
> +
>   	std::cout << info.str() << std::endl;
>   
>   	if (printMetadata_) {
> diff --git a/src/cam/camera_session.h b/src/cam/camera_session.h
> index bf966bd15ab0..14dc1421da4d 100644
> --- a/src/cam/camera_session.h
> +++ b/src/cam/camera_session.h
> @@ -65,6 +65,7 @@ private:
>   	unsigned int cameraIndex_;
>   
>   	uint64_t last_;
> +	uint64_t sequence_;
>   
>   	unsigned int queueCount_;
>   	unsigned int captureCount_;

Patch
diff mbox series

diff --git a/src/cam/camera_session.cpp b/src/cam/camera_session.cpp
index 50170723c30f..f9495f7f7e42 100644
--- a/src/cam/camera_session.cpp
+++ b/src/cam/camera_session.cpp
@@ -29,8 +29,8 @@  CameraSession::CameraSession(CameraManager *cm,
 			     unsigned int cameraIndex,
 			     const OptionsParser::Options &options)
 	: options_(options), cameraIndex_(cameraIndex), last_(0),
-	  queueCount_(0), captureCount_(0), captureLimit_(0),
-	  printMetadata_(false)
+	  sequence_(0), queueCount_(0), captureCount_(0),
+	  captureLimit_(0), printMetadata_(false)
 {
 	char *endptr;
 	unsigned long index = strtoul(cameraId.c_str(), &endptr, 10);
@@ -375,6 +375,11 @@  void CameraSession::processRequest(Request *request)
 	     << std::setw(6) << std::setfill('0') << ts / 1000 % 1000000
 	     << " (" << std::fixed << std::setprecision(2) << fps << " fps)";
 
+	uint64_t sequence = request->metadata().get(controls::SensorSequence);
+	// Report capture sequence number
+	info << " [" << std::setw(6) << std::setfill('0')
+	     << sequence << "]";
+
 	for (auto it = buffers.begin(); it != buffers.end(); ++it) {
 		const Stream *stream = it->first;
 		FrameBuffer *buffer = it->second;
@@ -398,6 +403,14 @@  void CameraSession::processRequest(Request *request)
 			requeue = false;
 	}
 
+	/* Handle basic frame drop detection and reporting. */
+	if (sequence_ == 0)
+		sequence_ = sequence - 1;
+	unsigned int drops = sequence - sequence_ - 1;
+	if (drops)
+		info << " *" << drops << " frame drops detected* ";
+	sequence_ = sequence;
+
 	std::cout << info.str() << std::endl;
 
 	if (printMetadata_) {
diff --git a/src/cam/camera_session.h b/src/cam/camera_session.h
index bf966bd15ab0..14dc1421da4d 100644
--- a/src/cam/camera_session.h
+++ b/src/cam/camera_session.h
@@ -65,6 +65,7 @@  private:
 	unsigned int cameraIndex_;
 
 	uint64_t last_;
+	uint64_t sequence_;
 
 	unsigned int queueCount_;
 	unsigned int captureCount_;