[libcamera-devel,RFC,0/7] add support for more complex optics
mbox series

Message ID 20230202111252.3366729-1-matthias.fend@emfend.at
Headers show
Series
  • add support for more complex optics
Related show

Message

Matthias Fend Feb. 2, 2023, 11:12 a.m. UTC 
Hello,

as mentioned on the mailing list some time ago, I was wondering how to use more complex optics together with libcamera IPAs.

Currently, only optics with a VCM focus lens are considered in libcamera. With these, an absolute position is set and the lens is almost immediately at the new desired position. Pretty simple.
For a slightly more complex optics there are the following differences:
- There may be multiple lens groups with different functions (zoom, focus, or auxiliary lens).
- The lens groups are often controlled by stepper motors and thus need some time to reach the new position.
- It may be necessary to move some lens group(s) synchronously (in realtime) depending on another lens group(s).
- To implement these requirements, dedicated intelligent controller hardware is often necessary.
- Depending on the application, calibration of the optics or lens groups may be necessary.
- With such an optics controller, the current status or position of the lens group(s) can change at any time, and the application must read this information from it.

In my current case, the control of the optics controller, in analogy to the VCM drivers, was also implemented as a v4l2 device.
Since this controller requires controls that go beyond the currently available v4l2 controls, this driver must use some custom or at least adapted existing v4l2 controls.
The intention is to bring the missing v4l2 controls upstream as well, but if that doesn't fully succeed, there should still be a way to use those controls in an IPA.


Currently, v4l2 controls are used by libcamera to transfer control information for the lens from the IPA to the pipeline handler. This transport form is probably just a utility, but I used the circumstance to transparently bring v4l2 controls from the IPA to the v4l-subdev via the pipeline handler.
The IPA can thus use all possible controls at will and assume that they arrive in the driver exactly the same way.
With this, I was able to meet my (control) requirements with minor changes.

As mentioned at the beginning, it may be that only the dedicated optic controller knows where the lens groups actually are.
To report back the current state or positions of the lens groups, the v4l2 subdev can provide read-only volatile controls.
Something like this was not available in libcamera until now. To have the same interface for this direction, I decided to pass these controls transparently to the IPA as well. So that not all controls must be transferred, I filtered only controls, which can change themselves (volatile).

To give you a little better idea, I have created this series of patches.

The question now is whether this approach can in principle be integrated into libcamera?

I look forward to feedback of all kinds.

Thanks
 ~Matthias

Matthias Fend (7):
  ipa: rkisp1: add basic camera lens support
  libcamera: camera_lens: add method to set controls on subdev
  pipeline: rkisp1: transparently pass lens controls from the IPA to the
    lens
  libcamera: camera_lens: add method to get controls from subdev
  libcamera: v4l2_device: add function to get a list of volatile control
    ids
  libcamera: camera_lens: add method to get a list of volatile controls
  ipa: rkisp1: pass volatile lens controls to IPA

 include/libcamera/internal/camera_lens.h |  3 +++
 include/libcamera/internal/v4l2_device.h |  1 +
 include/libcamera/ipa/rkisp1.mojom       |  7 ++++--
 src/libcamera/camera_lens.cpp            | 24 +++++++++++++++++++++
 src/libcamera/pipeline/rkisp1/rkisp1.cpp | 27 ++++++++++++++++++++----
 src/libcamera/v4l2_device.cpp            | 15 +++++++++++++
 6 files changed, 71 insertions(+), 6 deletions(-)