[libcamera-devel,v2,0/7] Implement Android Partial Result Featrue
mbox series

Message ID 20220812090838.1784703-1-hanlinchen@chromium.org
Headers show
Series
  • Implement Android Partial Result Featrue
Related show

Message

Hanlin Chen Aug. 12, 2022, 9:08 a.m. UTC 
Hi,

The series is to implement Android partial result feature, which allows
pipeline handler returns early buffers and metadata back to application before
request completed.

The feature is as a preparation of implementing the following description of
Android camera3 spec:

* The order of returning metadata and buffers for a single result does not
* matter, but buffers for a given stream must be returned in FIFO order. So
* the buffer for request 5 for stream A must always be returned before the
* buffer for request 6 for stream A. This also applies to the result
* metadata; the metadata for request 5 must be returned before the metadata
* for request 6.
*
* However, different streams are independent of each other, so it is
* acceptable and expected that the buffer for request 5 for stream A may be
* returned after the buffer for request 6 for stream B is. And it is
* acceptable that the result metadata for request 6 for stream B is
* returned before the buffer for request 5 for stream A is.

The spec is critical for ZSL implementation, because the still capture
can be return in an order independent to pending preview requests. This will
enhance still capture performance by:
1. Use cached raw frame for still capture, skip pending sensor exposure.
2. Skip pending preview requests and return still capture concurently.
2. HAL can pick 3A stable raw frame, so the application doesn't need to
wait for the process of stablizing 3A (by AEPrecapture and AFTrigger).

Because Android relies on metadata to process buffer properly, bufferCompleted
signal is not sufficient to report partial result to framework. The
patch series allows returning metadata earlier for pipeline handler and
refactor Android adaptor to return partial result as soon as possible.

V2:
  1. Add patch to for flush() waiting for all requests done before
  return.
  2. Change the final result returns in its o submission order.
  3. Reorganize the reuturning order of the results to meet Android's
  requirement that the buffers of the same stream returns in FIFO.

Han-Lin Chen (7):
  libcamera: Camera: Add RequestCompletionMode to configure the
    completion order
  libcamera: Camera: Add signals for completion of metadata and partial
    result
  libcamera: ipu3: returning partial results
  android: Move StreamBuffer out of Camera3RequestDescriptor
  android: Add JpegExifMetadata to store tags setting into Exif
  android: Implement partial result feature based on
    partialResultCompleted signal
  android: function flush() waits for all requests completed before
    return

 include/libcamera/camera.h                    |  10 +
 include/libcamera/internal/camera.h           |   4 +
 include/libcamera/internal/pipeline_handler.h |   4 +
 include/libcamera/request.h                   |  40 +
 src/android/camera_capabilities.cpp           |   2 +-
 src/android/camera_capabilities.h             |   2 +
 src/android/camera_device.cpp                 | 884 ++++++++++++------
 src/android/camera_device.h                   |  45 +-
 src/android/camera_request.cpp                |  90 +-
 src/android/camera_request.h                  | 104 ++-
 src/android/camera_stream.cpp                 |  18 +-
 src/android/camera_stream.h                   |  10 +-
 src/android/jpeg/post_processor_jpeg.cpp      |  17 +-
 src/android/jpeg/post_processor_jpeg.h        |   2 +-
 src/android/post_processor.h                  |   4 +-
 src/android/yuv/post_processor_yuv.cpp        |   2 +-
 src/android/yuv/post_processor_yuv.h          |   2 +-
 src/ipa/ipu3/ipu3.cpp                         |   2 -
 src/libcamera/camera.cpp                      |  78 +-
 src/libcamera/pipeline/ipu3/ipu3.cpp          |  65 +-
 src/libcamera/pipeline_handler.cpp            | 141 ++-
 src/libcamera/request.cpp                     | 170 +++-
 22 files changed, 1278 insertions(+), 418 deletions(-)