[libcamera-devel,v5,0/12] libcamera: Replace CameraConfiguration::transform

18963 mbox series

v4->v5:
- Add support in Python layer
- Rebased on master

v3->v4:
- Remove unrelated change in 8/10
- Remove a double test case
- Add David's tags

v2->v3:
- Move Orientation out of CameraConfiguration to a dedicated file
- Change operator*(Transform, Transform)
- Address David's comments in documentation and commit messages

v1->v2:
- Provide a better definitio of Orientation based on two basic operations
- Provide functions to combine Orientation and Transpose
- Provide tests
- Do not adjust properties::Rotation anymore

This series proposes to replace the usage to Transform in the public API in
favour of a new Orientation type, defined based on the EXIF
specification, tag 274, 'orientation'. For reference this is the same as
implemented by gstreamer:
https://gstreamer.freedesktop.org/documentation/gstreamer/gsttaglist.html?gi-language=c#GST_TAG_IMAGE_ORIENTATION

(the names of the enum values are different as gstreamer expresses the
"correction" while we express the actual orientation in memory buffers).

The newly introduced CameraConfiguration::orientation replaces the
existing CameraConfiguration::tranform, and it is meant for application to
express how they would like the images to be oriented, not to tell libcamera
what to do. As an example, passing in 'rotation0' means that the application
expects the images to be rotated upright, and doesn't tell libcamera not to
apply any rotation like passing in "Transform::Identity" did.

The value CameraConfiguration::orientation is set to after a validate() also
differs in meaning, as instead of reporting "what applications have to do
to obtain what they originally asked for" it simply reports the actual
orientation of the stream: this means that if libcamera cannot fully satisfy the
user request it will set ::orientation to report the native images rotation
and the CameraConfiguration::status will be set to Adjusted.

Handling of 90 and 270 degrees rotation has also changed: as the camera sensor
cannot correct rotations that include a transposition, requests for a 90/270
corrections are ignored. This makes it clear and less confusing for applications
that they have to deal with correction fully by themselves. As an example, with
the current implementation if the application requires a Rot270 (HFlip +
Transpose) libcamera will do the HFlip and leave transposition to the upper
layers. There is no clear advantage in doing so, and display compositors are
better suited for handling transpositions and flipping in a single pass instead
of having the library try to handle part of that.

This series clearly breaks the application API as it removes a member from
CameraConfiguration, so it should be introduced probably only when a new release
is cut.

Tested on RPi imx219 with the newly introduced cam option, flip, mirror and
rotation work as expected.

Jacopo Mondi (12):
  libcamera: camera_sensor: Cache rotationTransform_
  libcamera: camera: Introduce Orientation
  Documentation: Add figures to document Orientation
  libcamera: properties: Make 'Rotation' the mounting rotation
  libcamera: transform: Add functions to convert Orientation
  libcamera: transform: Invert operator*() operands
  libcamera: transform: Add operations with Orientation
  test: Add unit test for Transform and Orientation
  py: libcamera: Define and use Orientation
  libcamera: Use CameraConfiguration::orientation
  apps: cam: Add option to set stream orientation
  py: cam: Add option to set stream orientation

 Documentation/Doxyfile.in                     |   2 +
 Documentation/rotation/flip-rotate-0.eps      | 170 +++++++++
 Documentation/rotation/flip-rotate-0.png      | Bin 0 -> 16488 bytes
 Documentation/rotation/flip-rotate-180.eps    | 191 ++++++++++
 Documentation/rotation/flip-rotate-180.png    | Bin 0 -> 22198 bytes
 Documentation/rotation/flip-rotate-270.eps    | 118 +++++++
 Documentation/rotation/flip-rotate-270.png    | Bin 0 -> 9108 bytes
 Documentation/rotation/flip-rotate-90.eps     | 118 +++++++
 Documentation/rotation/flip-rotate-90.png     | Bin 0 -> 10918 bytes
 Documentation/rotation/rotate-0.eps           | 169 +++++++++
 Documentation/rotation/rotate-0.png           | Bin 0 -> 9086 bytes
 Documentation/rotation/rotate-180.eps         | 189 ++++++++++
 Documentation/rotation/rotate-180.png         | Bin 0 -> 22182 bytes
 Documentation/rotation/rotate-270.eps         | 118 +++++++
 Documentation/rotation/rotate-270.png         | Bin 0 -> 9334 bytes
 Documentation/rotation/rotate-90.eps          | 118 +++++++
 Documentation/rotation/rotate-90.png          | Bin 0 -> 9163 bytes
 include/libcamera/camera.h                    |   4 +-
 include/libcamera/internal/camera_sensor.h    |   5 +-
 include/libcamera/meson.build                 |   1 +
 include/libcamera/orientation.h               |  28 ++
 include/libcamera/transform.h                 |   7 +
 src/apps/cam/camera_session.cpp               |  17 +
 src/apps/cam/main.cpp                         |   5 +
 src/apps/cam/main.h                           |   1 +
 src/libcamera/camera.cpp                      |  20 +-
 src/libcamera/camera_sensor.cpp               | 129 ++++---
 src/libcamera/meson.build                     |   1 +
 src/libcamera/orientation.cpp                 | 103 ++++++
 src/libcamera/pipeline/ipu3/ipu3.cpp          |   6 +-
 src/libcamera/pipeline/rkisp1/rkisp1.cpp      |   8 +-
 .../pipeline/rpi/common/pipeline_base.cpp     |   9 +-
 src/libcamera/pipeline/simple/simple.cpp      |   6 +-
 src/libcamera/pipeline/uvcvideo/uvcvideo.cpp  |   4 +-
 src/libcamera/pipeline/vimc/vimc.cpp          |   4 +-
 src/libcamera/property_ids.yaml               |   8 +-
 src/libcamera/transform.cpp                   | 116 +++++-
 src/py/cam/cam.py                             |  20 ++
 src/py/libcamera/py_enums.cpp                 |  10 +
 src/py/libcamera/py_main.cpp                  |   2 +-
 test/meson.build                              |   1 +
 test/transform.cpp                            | 331 ++++++++++++++++++
 42 files changed, 1928 insertions(+), 111 deletions(-)
 create mode 100644 Documentation/rotation/flip-rotate-0.eps
 create mode 100644 Documentation/rotation/flip-rotate-0.png
 create mode 100644 Documentation/rotation/flip-rotate-180.eps
 create mode 100644 Documentation/rotation/flip-rotate-180.png
 create mode 100644 Documentation/rotation/flip-rotate-270.eps
 create mode 100644 Documentation/rotation/flip-rotate-270.png
 create mode 100644 Documentation/rotation/flip-rotate-90.eps
 create mode 100644 Documentation/rotation/flip-rotate-90.png
 create mode 100644 Documentation/rotation/rotate-0.eps
 create mode 100644 Documentation/rotation/rotate-0.png
 create mode 100644 Documentation/rotation/rotate-180.eps
 create mode 100644 Documentation/rotation/rotate-180.png
 create mode 100644 Documentation/rotation/rotate-270.eps
 create mode 100644 Documentation/rotation/rotate-270.png
 create mode 100644 Documentation/rotation/rotate-90.eps
 create mode 100644 Documentation/rotation/rotate-90.png
 create mode 100644 include/libcamera/orientation.h
 create mode 100644 src/libcamera/orientation.cpp
 create mode 100644 test/transform.cpp

--
2.40.1