[libcamera-devel,v1,08/10] pipeline: raspberrypi: Allow pipeline handler to always use the newest frame

17604 diff mbox series

Add a pipeline config parameter "return_newest_frames" to always use the
most recently captured Unicam frame when processing a request. This effectively
stops the pipeline handler from queuing Unicam buffers and processing requests
using the buffer at the front of the queue.

Note that setting this parameter might incur unnecessary frame drops during
times of high transient CPU loads where the application might not be able to
provide requests quick enough.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
---
 .../pipeline/raspberrypi/data/default.json    |  7 ++++++-
 .../pipeline/raspberrypi/raspberrypi.cpp      | 19 +++++++++++++++++++
 2 files changed, 25 insertions(+), 1 deletion(-)

Hi Naush

Thanks for the patch!

On Fri, 14 Oct 2022 at 14:19, Naushir Patuck via libcamera-devel
<libcamera-devel@lists.libcamera.org> wrote:
>
> Add a pipeline config parameter "return_newest_frames" to always use the
> most recently captured Unicam frame when processing a request. This effectively
> stops the pipeline handler from queuing Unicam buffers and processing requests
> using the buffer at the front of the queue.
>
> Note that setting this parameter might incur unnecessary frame drops during
> times of high transient CPU loads where the application might not be able to
> provide requests quick enough.
>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> ---
>  .../pipeline/raspberrypi/data/default.json    |  7 ++++++-
>  .../pipeline/raspberrypi/raspberrypi.cpp      | 19 +++++++++++++++++++
>  2 files changed, 25 insertions(+), 1 deletion(-)
>
> diff --git a/src/libcamera/pipeline/raspberrypi/data/default.json b/src/libcamera/pipeline/raspberrypi/data/default.json
> index 707414bcc5c5..f900ca824523 100644
> --- a/src/libcamera/pipeline/raspberrypi/data/default.json
> +++ b/src/libcamera/pipeline/raspberrypi/data/default.json
> @@ -18,6 +18,11 @@
>                  "num_output0_buffers": 1,
>
>                  # Override any request from the IPA to drop a number of startup frames.
> -                "disable_startup_frame_drops": false
> +                "disable_startup_frame_drops": false,
> +
> +                # Always process a pending request with the last captured sensor frame.
> +                # Note that this might lead to avoidable frame drops during periods
> +                # of transient heavey CPU loading.

s/heavey/heavy/    (sorry!)

Reviewed-by: David Plowman <david.plowman@raspberrypi.com>

Thanks!
David

> +                "return_newest_frames": false
>          }
>  }
> diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> index 135948d82f41..d9bea35c88a3 100644
> --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> @@ -301,6 +301,7 @@ public:
>                 unsigned int minTotalUnicamBuffers;
>                 unsigned int numOutput0Buffers;
>                 bool disableStartupFrameDrops;
> +               bool returnNewestFrames;
>         };
>
>         Config config_;
> @@ -1432,6 +1433,7 @@ int PipelineHandlerRPi::configurePipelineHandler(RPiCameraData *data)
>                 .minTotalUnicamBuffers = 4,
>                 .numOutput0Buffers = 1,
>                 .disableStartupFrameDrops = false,
> +               .returnNewestFrames = false,
>         };
>
>         char const *configFromEnv = utils::secure_getenv("LIBCAMERA_RPI_CONFIG_FILE");
> @@ -1468,6 +1470,8 @@ int PipelineHandlerRPi::configurePipelineHandler(RPiCameraData *data)
>                 phConfig["num_output0_buffers"].get<unsigned int>(config.numOutput0Buffers);
>         config.disableStartupFrameDrops =
>                 phConfig["disable_startup_frame_drops"].get<bool>(config.disableStartupFrameDrops);
> +       config.returnNewestFrames =
> +               phConfig["return_newest_frames"].get<bool>(config.returnNewestFrames);
>
>         if (config.minTotalUnicamBuffers < config.minUnicamBuffers || config.minTotalUnicamBuffers < 1) {
>                 LOG(RPI, Error) << "Invalid Unicam buffer configuration used!";
> @@ -2320,6 +2324,21 @@ bool RPiCameraData::findMatchingBuffers(BayerFrame &bayerFrame, FrameBuffer *&em
>         if (bayerQueue_.empty())
>                 return false;
>
> +       /*
> +        * If the pipeline is configured to only ever return the most recently
> +        * captured frame, empty the buffer queue until a single element is
> +        * left, corresponding to the most recent buffer. Note that this will
> +        * likely result in possibly avoidable dropped frames.
> +        */
> +       if (config_.returnNewestFrames && !unicam_[Unicam::Image].isExternal()) {
> +               while (bayerQueue_.size() > 1) {
> +                       FrameBuffer *bayer = bayerQueue_.front().buffer;
> +
> +                       unicam_[Unicam::Image].returnBuffer(bayer);
> +                       bayerQueue_.pop();
> +               }
> +       }
> +
>         /*
>          * Find the embedded data buffer with a matching timestamp to pass to
>          * the IPA. Any embedded buffers with a timestamp lower than the
> --
> 2.25.1
>