| Message ID | 20230406-feature-controls-lens-v1-1-543189a680de@wolfvision.net |
|---|---|
| State | Not Applicable |
| Headers | show |
| Series |
|
| Related | show |
Hi Michael Thanks for the patch. I've got a personal interest here as I'd love to be able to control a couple of CCTV lenses that I have. Those have standard motors with potentiometers for position feedback, not stepper motors, but otherwise have the same limitations of slow movement. On Thu, 6 Apr 2023 at 15:31, Michael Riesch via B4 Relay via libcamera-devel <libcamera-devel@lists.libcamera.org> wrote: > > From: Michael Riesch <michael.riesch@wolfvision.net> > > Add the controls V4L2_CID_FOCUS_STATUS and V4L2_CID_ZOOM_STATUS that report > the status of the zoom lens group and the focus lens group, respectively. > The returned data structure contains the current position of the lens group > as well as movement indication flags. > > Signed-off-by: Michael Riesch <michael.riesch@wolfvision.net> > --- > .../userspace-api/media/v4l/ext-ctrls-camera.rst | 48 ++++++++++++++++++++++ > drivers/media/v4l2-core/v4l2-ctrls-core.c | 9 ++++ > drivers/media/v4l2-core/v4l2-ctrls-defs.c | 7 ++++ > include/media/v4l2-ctrls.h | 2 + > include/uapi/linux/v4l2-controls.h | 13 ++++++ > include/uapi/linux/videodev2.h | 2 + > 6 files changed, 81 insertions(+) > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst > index daa4f40869f8..3a270bc63f1a 100644 > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst > @@ -149,6 +149,30 @@ enum v4l2_exposure_metering - > to the camera, negative values towards infinity. This is a > write-only control. > > +``V4L2_CID_FOCUS_STATUS (struct)`` > + The current status of the focus lens group. This is a read-only control. > + The returned data structure contains the current position and movement > + indication flags. The unit of the current position is undefined. Positive > + values move the focus closer to the camera, negative values towards > + infinity. The possible flags are described in the table below. The units are undefined, but positive and negative mean something with respect to some unspecified focal distance represented by 0. That seems a little odd. I was going to suggest that it seems to make sense to follow the same units as V4L2_CID_FOCUS_ABSOLUTE, but on reading that description in [1] it's the same text. I suspect there was a little too much copy/paste from V4L2_CID_FOCUS_RELATIVE, or the intent was that increasing the value brings the focus closer, and decreasing moves it towards infinity. If we did specify that it was the same units as V4L2_CID_FOCUS_ABSOLUTE, then what would that mean for use with V4L2_CID_FOCUS_RELATIVE? Then again the only user of _RELATIVE appears to be ov5693 with atomisp and that just maps it onto _ABSOLUTE, so potentially it's redundant and could be deprecated. [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-camera.html > + > +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}| > + > +.. flat-table:: > + :header-rows: 0 > + :stub-columns: 0 > + > + * - ``V4L2_LENS_STATUS_IDLE`` > + - Focus lens group is at rest. > + * - ``V4L2_LENS_STATUS_BUSY`` > + - Focus lens group is moving. > + * - ``V4L2_LENS_STATUS_REACHED`` > + - Focus lens group has reached its target position. > + * - ``V4L2_LENS_STATUS_FAILED`` > + - Focus lens group has failed to reach its target position. The driver > + will not transition from this state until another action is performed > + by an application. When would the lens state transition from V4L2_LENS_STATUS_REACHED to V4L2_LENS_STATUS_IDLE? If it's reached the position then it is at rest, and being at rest is the definition of V4L2_LENS_STATUS_IDLE. Likewise the lens always has a target position based on the control value, so it's always at V4L2_LENS_STATUS_REACHED when it's not moving. Is there a need to have 2 states? If the position is the same units as V4L2_CID_FOCUS_ABSOLUTE, then do you leave the determination of state to the application? > ``V4L2_CID_FOCUS_AUTO (boolean)`` > Enables continuous automatic focus adjustments. The effect of manual > focus adjustments while this feature is enabled is undefined, > @@ -239,6 +263,30 @@ enum v4l2_auto_focus_range - > movement. A negative value moves the zoom lens group towards the > wide-angle direction. The zoom speed unit is driver-specific. > > +``V4L2_CID_ZOOM_STATUS (struct)`` > + The current status of the zoom lens group. This is a read-only control. > + The returned data structure contains the current position and movement > + indication flags. The unit of the current position is driver-specific and > + its value should be a positive integer. The possible flags are described > + in the table below. > + > +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}| > + > +.. flat-table:: > + :header-rows: 0 > + :stub-columns: 0 > + > + * - ``V4L2_LENS_STATUS_IDLE`` > + - Zoom lens group is at rest. > + * - ``V4L2_LENS_STATUS_BUSY`` > + - Zoom lens group is moving. > + * - ``V4L2_LENS_STATUS_REACHED`` > + - Zoom lens group has reached its target position. > + * - ``V4L2_LENS_STATUS_FAILED`` > + - Zoom lens group has failed to reach its target position. The driver will > + not transition from this state until another action is performed by an > + application. > + > ``V4L2_CID_IRIS_ABSOLUTE (integer)`` > This control sets the camera's aperture to the specified value. The > unit is undefined. Larger values open the iris wider, smaller values > diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c > index 29169170880a..f6ad30f311c5 100644 > --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c > +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c > @@ -350,6 +350,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl) > case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS: > pr_cont("HEVC_DECODE_PARAMS"); > break; > + case V4L2_CTRL_TYPE_LENS_STATUS: > + pr_cont("LENS_STATUS"); > + break; > default: > pr_cont("unknown type %d", ctrl->type); > break; > @@ -918,6 +921,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx, > return -EINVAL; > break; > > + case V4L2_CTRL_TYPE_LENS_STATUS: > + break; > + > default: > return -EINVAL; > } > @@ -1605,6 +1611,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, > case V4L2_CTRL_TYPE_AREA: > elem_size = sizeof(struct v4l2_area); > break; > + case V4L2_CTRL_TYPE_LENS_STATUS: > + elem_size = sizeof(struct v4l2_ctrl_lens_status); > + break; > default: > if (type < V4L2_CTRL_COMPOUND_TYPES) > elem_size = sizeof(s32); > diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c > index 564fedee2c88..9b26a3aa9e9c 100644 > --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c > +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c > @@ -1044,6 +1044,8 @@ const char *v4l2_ctrl_get_name(u32 id) > case V4L2_CID_CAMERA_ORIENTATION: return "Camera Orientation"; > case V4L2_CID_CAMERA_SENSOR_ROTATION: return "Camera Sensor Rotation"; > case V4L2_CID_HDR_SENSOR_MODE: return "HDR Sensor Mode"; > + case V4L2_CID_FOCUS_STATUS: return "Focus, Status"; > + case V4L2_CID_ZOOM_STATUS: return "Zoom, Status"; Is there a need for the comma in the text strings? Cheers Dave > > /* FM Radio Modulator controls */ > /* Keep the order of the 'case's the same as in v4l2-controls.h! */ > @@ -1593,6 +1595,11 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, > *flags |= V4L2_CTRL_FLAG_WRITE_ONLY | > V4L2_CTRL_FLAG_EXECUTE_ON_WRITE; > break; > + case V4L2_CID_FOCUS_STATUS: > + case V4L2_CID_ZOOM_STATUS: > + *type = V4L2_CTRL_TYPE_LENS_STATUS; > + *flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE; > + break; > case V4L2_CID_FLASH_STROBE_STATUS: > case V4L2_CID_AUTO_FOCUS_STATUS: > case V4L2_CID_FLASH_READY: > diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h > index e59d9a234631..f7273ffc20c9 100644 > --- a/include/media/v4l2-ctrls.h > +++ b/include/media/v4l2-ctrls.h > @@ -52,6 +52,7 @@ struct video_device; > * @p_hdr10_cll: Pointer to an HDR10 Content Light Level structure. > * @p_hdr10_mastering: Pointer to an HDR10 Mastering Display structure. > * @p_area: Pointer to an area. > + * @p_lens_status: Pointer to a lens status structure. > * @p: Pointer to a compound value. > * @p_const: Pointer to a constant compound value. > */ > @@ -81,6 +82,7 @@ union v4l2_ctrl_ptr { > struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll; > struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering; > struct v4l2_area *p_area; > + struct v4l2_ctrl_lens_status *p_lens_status; > void *p; > const void *p_const; > }; > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h > index 5e80daa4ffe0..8b037467ba9a 100644 > --- a/include/uapi/linux/v4l2-controls.h > +++ b/include/uapi/linux/v4l2-controls.h > @@ -993,6 +993,19 @@ enum v4l2_auto_focus_range { > > #define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36) > > +struct v4l2_ctrl_lens_status { > + __u32 flags; > + __s32 current_position; > +}; > + > +#define V4L2_LENS_STATUS_IDLE (0 << 0) > +#define V4L2_LENS_STATUS_BUSY (1 << 0) > +#define V4L2_LENS_STATUS_REACHED (1 << 1) > +#define V4L2_LENS_STATUS_FAILED (1 << 2) > + > +#define V4L2_CID_FOCUS_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 37) > +#define V4L2_CID_ZOOM_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 38) > + > /* FM Modulator class control IDs */ > > #define V4L2_CID_FM_TX_CLASS_BASE (V4L2_CTRL_CLASS_FM_TX | 0x900) > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > index 17a9b975177a..256c21c68720 100644 > --- a/include/uapi/linux/videodev2.h > +++ b/include/uapi/linux/videodev2.h > @@ -1888,6 +1888,8 @@ enum v4l2_ctrl_type { > V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272, > V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273, > V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274, > + > + V4L2_CTRL_TYPE_LENS_STATUS = 0x0300, > }; > > /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */ > > -- > 2.30.2 >
Hi Dave, On 4/6/23 17:16, Dave Stevenson wrote: > Hi Michael > > Thanks for the patch. > > I've got a personal interest here as I'd love to be able to control a > couple of CCTV lenses that I have. Those have standard motors with > potentiometers for position feedback, not stepper motors, but > otherwise have the same limitations of slow movement. That's great to hear :-) Thank you for your feedback! > On Thu, 6 Apr 2023 at 15:31, Michael Riesch via B4 Relay via > libcamera-devel <libcamera-devel@lists.libcamera.org> wrote: >> >> From: Michael Riesch <michael.riesch@wolfvision.net> >> >> Add the controls V4L2_CID_FOCUS_STATUS and V4L2_CID_ZOOM_STATUS that report >> the status of the zoom lens group and the focus lens group, respectively. >> The returned data structure contains the current position of the lens group >> as well as movement indication flags. >> >> Signed-off-by: Michael Riesch <michael.riesch@wolfvision.net> >> --- >> .../userspace-api/media/v4l/ext-ctrls-camera.rst | 48 ++++++++++++++++++++++ >> drivers/media/v4l2-core/v4l2-ctrls-core.c | 9 ++++ >> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 7 ++++ >> include/media/v4l2-ctrls.h | 2 + >> include/uapi/linux/v4l2-controls.h | 13 ++++++ >> include/uapi/linux/videodev2.h | 2 + >> 6 files changed, 81 insertions(+) >> >> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst >> index daa4f40869f8..3a270bc63f1a 100644 >> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst >> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst >> @@ -149,6 +149,30 @@ enum v4l2_exposure_metering - >> to the camera, negative values towards infinity. This is a >> write-only control. >> >> +``V4L2_CID_FOCUS_STATUS (struct)`` >> + The current status of the focus lens group. This is a read-only control. >> + The returned data structure contains the current position and movement >> + indication flags. The unit of the current position is undefined. Positive >> + values move the focus closer to the camera, negative values towards >> + infinity. The possible flags are described in the table below. > > The units are undefined, but positive and negative mean something with > respect to some unspecified focal distance represented by 0. That > seems a little odd. > > I was going to suggest that it seems to make sense to follow the same > units as V4L2_CID_FOCUS_ABSOLUTE, but on reading that description in > [1] it's the same text. > I suspect there was a little too much copy/paste from > V4L2_CID_FOCUS_RELATIVE, or the intent was that increasing the value > brings the focus closer, and decreasing moves it towards infinity. > > If we did specify that it was the same units as > V4L2_CID_FOCUS_ABSOLUTE, then what would that mean for use with > V4L2_CID_FOCUS_RELATIVE? Then again the only user of _RELATIVE appears > to be ov5693 with atomisp and that just maps it onto _ABSOLUTE, so > potentially it's redundant and could be deprecated. > > [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-camera.html I think we agree that the _STATUS controls should use the same unit as the corresponding _ABSOLUTE control, whatever that may be. Then, the question is whether the description of FOCUS_ABSOLUTE needs to be revised. I would agree that the intention probably was: "Larger values move the focus closer to the camera, smaller values move the focus to infinity." (cf. the phrasing in the description of IRIS_ABSOLUTE). It would be interesting to know whether zero and negative values are (intentionally?) included. Since they are not explicitly excluded, my driver exposes the position of the lens in motor steps to the user space. If the values were (supposed to be) restricted to positive values like ZOOM_ABSOLUTE, this would not be allowed of course. As to the relation to _RELATIVE, I think it should be clear that all controls should act on the same scale and I don't see any problems here. However, feel free to point out what I am missing :-) >> + >> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}| >> + >> +.. flat-table:: >> + :header-rows: 0 >> + :stub-columns: 0 >> + >> + * - ``V4L2_LENS_STATUS_IDLE`` >> + - Focus lens group is at rest. >> + * - ``V4L2_LENS_STATUS_BUSY`` >> + - Focus lens group is moving. >> + * - ``V4L2_LENS_STATUS_REACHED`` >> + - Focus lens group has reached its target position. >> + * - ``V4L2_LENS_STATUS_FAILED`` >> + - Focus lens group has failed to reach its target position. The driver >> + will not transition from this state until another action is performed >> + by an application. > > When would the lens state transition from V4L2_LENS_STATUS_REACHED to > V4L2_LENS_STATUS_IDLE? > If it's reached the position then it is at rest, and being at rest is > the definition of V4L2_LENS_STATUS_IDLE. > Likewise the lens always has a target position based on the control > value, so it's always at V4L2_LENS_STATUS_REACHED when it's not > moving. > Is there a need to have 2 states? Good point, I need to reconsider that (and possibly remove one of those states). > If the position is the same units as V4L2_CID_FOCUS_ABSOLUTE, then do > you leave the determination of state to the application? I am afraid I don't quite follow. But FWIW the application should read out the flags to find out whether the lens is moving and, if this is not the case, whether the last move command (FOCUS_ABSOLUTE or FOCUS_RELATIVE) has succeeded or failed. The failed state is important as there might be the possibility that depending on the zoom lens position certain focus positions are not available (and vice versa). >> ``V4L2_CID_FOCUS_AUTO (boolean)`` >> Enables continuous automatic focus adjustments. The effect of manual >> focus adjustments while this feature is enabled is undefined, >> @@ -239,6 +263,30 @@ enum v4l2_auto_focus_range - >> movement. A negative value moves the zoom lens group towards the >> wide-angle direction. The zoom speed unit is driver-specific. >> >> +``V4L2_CID_ZOOM_STATUS (struct)`` >> + The current status of the zoom lens group. This is a read-only control. >> + The returned data structure contains the current position and movement >> + indication flags. The unit of the current position is driver-specific and >> + its value should be a positive integer. The possible flags are described >> + in the table below. >> + >> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}| >> + >> +.. flat-table:: >> + :header-rows: 0 >> + :stub-columns: 0 >> + >> + * - ``V4L2_LENS_STATUS_IDLE`` >> + - Zoom lens group is at rest. >> + * - ``V4L2_LENS_STATUS_BUSY`` >> + - Zoom lens group is moving. >> + * - ``V4L2_LENS_STATUS_REACHED`` >> + - Zoom lens group has reached its target position. >> + * - ``V4L2_LENS_STATUS_FAILED`` >> + - Zoom lens group has failed to reach its target position. The driver will >> + not transition from this state until another action is performed by an >> + application. >> + >> ``V4L2_CID_IRIS_ABSOLUTE (integer)`` >> This control sets the camera's aperture to the specified value. The >> unit is undefined. Larger values open the iris wider, smaller values >> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c >> index 29169170880a..f6ad30f311c5 100644 >> --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c >> +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c >> @@ -350,6 +350,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl) >> case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS: >> pr_cont("HEVC_DECODE_PARAMS"); >> break; >> + case V4L2_CTRL_TYPE_LENS_STATUS: >> + pr_cont("LENS_STATUS"); >> + break; >> default: >> pr_cont("unknown type %d", ctrl->type); >> break; >> @@ -918,6 +921,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx, >> return -EINVAL; >> break; >> >> + case V4L2_CTRL_TYPE_LENS_STATUS: >> + break; >> + >> default: >> return -EINVAL; >> } >> @@ -1605,6 +1611,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, >> case V4L2_CTRL_TYPE_AREA: >> elem_size = sizeof(struct v4l2_area); >> break; >> + case V4L2_CTRL_TYPE_LENS_STATUS: >> + elem_size = sizeof(struct v4l2_ctrl_lens_status); >> + break; >> default: >> if (type < V4L2_CTRL_COMPOUND_TYPES) >> elem_size = sizeof(s32); >> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c >> index 564fedee2c88..9b26a3aa9e9c 100644 >> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c >> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c >> @@ -1044,6 +1044,8 @@ const char *v4l2_ctrl_get_name(u32 id) >> case V4L2_CID_CAMERA_ORIENTATION: return "Camera Orientation"; >> case V4L2_CID_CAMERA_SENSOR_ROTATION: return "Camera Sensor Rotation"; >> case V4L2_CID_HDR_SENSOR_MODE: return "HDR Sensor Mode"; >> + case V4L2_CID_FOCUS_STATUS: return "Focus, Status"; >> + case V4L2_CID_ZOOM_STATUS: return "Zoom, Status"; > > Is there a need for the comma in the text strings? Not sure, actually. Some other strings used commas. Monkey see, monkey do. Best regards, Michael > > Cheers > Dave > >> >> /* FM Radio Modulator controls */ >> /* Keep the order of the 'case's the same as in v4l2-controls.h! */ >> @@ -1593,6 +1595,11 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, >> *flags |= V4L2_CTRL_FLAG_WRITE_ONLY | >> V4L2_CTRL_FLAG_EXECUTE_ON_WRITE; >> break; >> + case V4L2_CID_FOCUS_STATUS: >> + case V4L2_CID_ZOOM_STATUS: >> + *type = V4L2_CTRL_TYPE_LENS_STATUS; >> + *flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE; >> + break; >> case V4L2_CID_FLASH_STROBE_STATUS: >> case V4L2_CID_AUTO_FOCUS_STATUS: >> case V4L2_CID_FLASH_READY: >> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h >> index e59d9a234631..f7273ffc20c9 100644 >> --- a/include/media/v4l2-ctrls.h >> +++ b/include/media/v4l2-ctrls.h >> @@ -52,6 +52,7 @@ struct video_device; >> * @p_hdr10_cll: Pointer to an HDR10 Content Light Level structure. >> * @p_hdr10_mastering: Pointer to an HDR10 Mastering Display structure. >> * @p_area: Pointer to an area. >> + * @p_lens_status: Pointer to a lens status structure. >> * @p: Pointer to a compound value. >> * @p_const: Pointer to a constant compound value. >> */ >> @@ -81,6 +82,7 @@ union v4l2_ctrl_ptr { >> struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll; >> struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering; >> struct v4l2_area *p_area; >> + struct v4l2_ctrl_lens_status *p_lens_status; >> void *p; >> const void *p_const; >> }; >> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h >> index 5e80daa4ffe0..8b037467ba9a 100644 >> --- a/include/uapi/linux/v4l2-controls.h >> +++ b/include/uapi/linux/v4l2-controls.h >> @@ -993,6 +993,19 @@ enum v4l2_auto_focus_range { >> >> #define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36) >> >> +struct v4l2_ctrl_lens_status { >> + __u32 flags; >> + __s32 current_position; >> +}; >> + >> +#define V4L2_LENS_STATUS_IDLE (0 << 0) >> +#define V4L2_LENS_STATUS_BUSY (1 << 0) >> +#define V4L2_LENS_STATUS_REACHED (1 << 1) >> +#define V4L2_LENS_STATUS_FAILED (1 << 2) >> + >> +#define V4L2_CID_FOCUS_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 37) >> +#define V4L2_CID_ZOOM_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 38) >> + >> /* FM Modulator class control IDs */ >> >> #define V4L2_CID_FM_TX_CLASS_BASE (V4L2_CTRL_CLASS_FM_TX | 0x900) >> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h >> index 17a9b975177a..256c21c68720 100644 >> --- a/include/uapi/linux/videodev2.h >> +++ b/include/uapi/linux/videodev2.h >> @@ -1888,6 +1888,8 @@ enum v4l2_ctrl_type { >> V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272, >> V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273, >> V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274, >> + >> + V4L2_CTRL_TYPE_LENS_STATUS = 0x0300, >> }; >> >> /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */ >> >> -- >> 2.30.2 >>
Hi Michael On Tue, 11 Apr 2023 at 18:33, Michael Riesch <michael.riesch@wolfvision.net> wrote: > > Hi Dave, > > On 4/6/23 17:16, Dave Stevenson wrote: > > Hi Michael > > > > Thanks for the patch. > > > > I've got a personal interest here as I'd love to be able to control a > > couple of CCTV lenses that I have. Those have standard motors with > > potentiometers for position feedback, not stepper motors, but > > otherwise have the same limitations of slow movement. > > That's great to hear :-) Thank you for your feedback! > > > On Thu, 6 Apr 2023 at 15:31, Michael Riesch via B4 Relay via > > libcamera-devel <libcamera-devel@lists.libcamera.org> wrote: > >> > >> From: Michael Riesch <michael.riesch@wolfvision.net> > >> > >> Add the controls V4L2_CID_FOCUS_STATUS and V4L2_CID_ZOOM_STATUS that report > >> the status of the zoom lens group and the focus lens group, respectively. > >> The returned data structure contains the current position of the lens group > >> as well as movement indication flags. > >> > >> Signed-off-by: Michael Riesch <michael.riesch@wolfvision.net> > >> --- > >> .../userspace-api/media/v4l/ext-ctrls-camera.rst | 48 ++++++++++++++++++++++ > >> drivers/media/v4l2-core/v4l2-ctrls-core.c | 9 ++++ > >> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 7 ++++ > >> include/media/v4l2-ctrls.h | 2 + > >> include/uapi/linux/v4l2-controls.h | 13 ++++++ > >> include/uapi/linux/videodev2.h | 2 + > >> 6 files changed, 81 insertions(+) > >> > >> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst > >> index daa4f40869f8..3a270bc63f1a 100644 > >> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst > >> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst > >> @@ -149,6 +149,30 @@ enum v4l2_exposure_metering - > >> to the camera, negative values towards infinity. This is a > >> write-only control. > >> > >> +``V4L2_CID_FOCUS_STATUS (struct)`` > >> + The current status of the focus lens group. This is a read-only control. > >> + The returned data structure contains the current position and movement > >> + indication flags. The unit of the current position is undefined. Positive > >> + values move the focus closer to the camera, negative values towards > >> + infinity. The possible flags are described in the table below. > > > > The units are undefined, but positive and negative mean something with > > respect to some unspecified focal distance represented by 0. That > > seems a little odd. > > > > I was going to suggest that it seems to make sense to follow the same > > units as V4L2_CID_FOCUS_ABSOLUTE, but on reading that description in > > [1] it's the same text. > > I suspect there was a little too much copy/paste from > > V4L2_CID_FOCUS_RELATIVE, or the intent was that increasing the value > > brings the focus closer, and decreasing moves it towards infinity. > > > > If we did specify that it was the same units as > > V4L2_CID_FOCUS_ABSOLUTE, then what would that mean for use with > > V4L2_CID_FOCUS_RELATIVE? Then again the only user of _RELATIVE appears > > to be ov5693 with atomisp and that just maps it onto _ABSOLUTE, so > > potentially it's redundant and could be deprecated. > > > > [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-camera.html > > I think we agree that the _STATUS controls should use the same unit as > the corresponding _ABSOLUTE control, whatever that may be. > > Then, the question is whether the description of FOCUS_ABSOLUTE needs to > be revised. I would agree that the intention probably was: "Larger > values move the focus closer to the camera, smaller values move the > focus to infinity." (cf. the phrasing in the description of IRIS_ABSOLUTE). > > It would be interesting to know whether zero and negative values are > (intentionally?) included. Since they are not explicitly excluded, my > driver exposes the position of the lens in motor steps to the user > space. If the values were (supposed to be) restricted to positive values > like ZOOM_ABSOLUTE, this would not be allowed of course. > > As to the relation to _RELATIVE, I think it should be clear that all > controls should act on the same scale and I don't see any problems here. > However, feel free to point out what I am missing :-) > > >> + > >> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}| > >> + > >> +.. flat-table:: > >> + :header-rows: 0 > >> + :stub-columns: 0 > >> + > >> + * - ``V4L2_LENS_STATUS_IDLE`` > >> + - Focus lens group is at rest. > >> + * - ``V4L2_LENS_STATUS_BUSY`` > >> + - Focus lens group is moving. > >> + * - ``V4L2_LENS_STATUS_REACHED`` > >> + - Focus lens group has reached its target position. > >> + * - ``V4L2_LENS_STATUS_FAILED`` > >> + - Focus lens group has failed to reach its target position. The driver > >> + will not transition from this state until another action is performed > >> + by an application. > > > > When would the lens state transition from V4L2_LENS_STATUS_REACHED to > > V4L2_LENS_STATUS_IDLE? > > If it's reached the position then it is at rest, and being at rest is > > the definition of V4L2_LENS_STATUS_IDLE. > > Likewise the lens always has a target position based on the control > > value, so it's always at V4L2_LENS_STATUS_REACHED when it's not > > moving. > > Is there a need to have 2 states? > > Good point, I need to reconsider that (and possibly remove one of those > states). > > > If the position is the same units as V4L2_CID_FOCUS_ABSOLUTE, then do > > you leave the determination of state to the application? > > I am afraid I don't quite follow. If the application sets V4L2_CID_FOCUS_ABSOLUTE to 42, and V4L2_CID_FOCUS_STATUS returns that the position is now 42, then the application knows that the lens has reached the requested position. If the two controls return different values then the lens is still moving. What new information does adding an additional status flag give you? (Thinking aloud) I guess you have got my case where your readback is via an ADC so that the position may fluctuate slightly due to conversion noise. The control loop for applying power to the motors will presumably stop at some point and stop trying to adjust the position, so potentially it could be the state returned from that control loop. However the noisy ADC position could also be solved by the position being returned by the control loop instead of giving the raw value. > But FWIW the application should read out the flags to find out whether > the lens is moving and, if this is not the case, whether the last move > command (FOCUS_ABSOLUTE or FOCUS_RELATIVE) has succeeded or failed. The > failed state is important as there might be the possibility that > depending on the zoom lens position certain focus positions are not > available (and vice versa). How would you configure those kinds of restrictions in the kernel driver? Is the kernel driver the best place to do it (so many things are being kicked back to userspace these days)? If it is to be done in the kernel, then shouldn't eg changing the zoom position alter the ranges advertised for the relevant focus controls? If the ranges aren't updated, where should that out-of-range lens movement leave the lens? Dave > >> ``V4L2_CID_FOCUS_AUTO (boolean)`` > >> Enables continuous automatic focus adjustments. The effect of manual > >> focus adjustments while this feature is enabled is undefined, > >> @@ -239,6 +263,30 @@ enum v4l2_auto_focus_range - > >> movement. A negative value moves the zoom lens group towards the > >> wide-angle direction. The zoom speed unit is driver-specific. > >> > >> +``V4L2_CID_ZOOM_STATUS (struct)`` > >> + The current status of the zoom lens group. This is a read-only control. > >> + The returned data structure contains the current position and movement > >> + indication flags. The unit of the current position is driver-specific and > >> + its value should be a positive integer. The possible flags are described > >> + in the table below. > >> + > >> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}| > >> + > >> +.. flat-table:: > >> + :header-rows: 0 > >> + :stub-columns: 0 > >> + > >> + * - ``V4L2_LENS_STATUS_IDLE`` > >> + - Zoom lens group is at rest. > >> + * - ``V4L2_LENS_STATUS_BUSY`` > >> + - Zoom lens group is moving. > >> + * - ``V4L2_LENS_STATUS_REACHED`` > >> + - Zoom lens group has reached its target position. > >> + * - ``V4L2_LENS_STATUS_FAILED`` > >> + - Zoom lens group has failed to reach its target position. The driver will > >> + not transition from this state until another action is performed by an > >> + application. > >> + > >> ``V4L2_CID_IRIS_ABSOLUTE (integer)`` > >> This control sets the camera's aperture to the specified value. The > >> unit is undefined. Larger values open the iris wider, smaller values > >> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c > >> index 29169170880a..f6ad30f311c5 100644 > >> --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c > >> +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c > >> @@ -350,6 +350,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl) > >> case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS: > >> pr_cont("HEVC_DECODE_PARAMS"); > >> break; > >> + case V4L2_CTRL_TYPE_LENS_STATUS: > >> + pr_cont("LENS_STATUS"); > >> + break; > >> default: > >> pr_cont("unknown type %d", ctrl->type); > >> break; > >> @@ -918,6 +921,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx, > >> return -EINVAL; > >> break; > >> > >> + case V4L2_CTRL_TYPE_LENS_STATUS: > >> + break; > >> + > >> default: > >> return -EINVAL; > >> } > >> @@ -1605,6 +1611,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, > >> case V4L2_CTRL_TYPE_AREA: > >> elem_size = sizeof(struct v4l2_area); > >> break; > >> + case V4L2_CTRL_TYPE_LENS_STATUS: > >> + elem_size = sizeof(struct v4l2_ctrl_lens_status); > >> + break; > >> default: > >> if (type < V4L2_CTRL_COMPOUND_TYPES) > >> elem_size = sizeof(s32); > >> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c > >> index 564fedee2c88..9b26a3aa9e9c 100644 > >> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c > >> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c > >> @@ -1044,6 +1044,8 @@ const char *v4l2_ctrl_get_name(u32 id) > >> case V4L2_CID_CAMERA_ORIENTATION: return "Camera Orientation"; > >> case V4L2_CID_CAMERA_SENSOR_ROTATION: return "Camera Sensor Rotation"; > >> case V4L2_CID_HDR_SENSOR_MODE: return "HDR Sensor Mode"; > >> + case V4L2_CID_FOCUS_STATUS: return "Focus, Status"; > >> + case V4L2_CID_ZOOM_STATUS: return "Zoom, Status"; > > > > Is there a need for the comma in the text strings? > > Not sure, actually. Some other strings used commas. Monkey see, monkey do. > > Best regards, > Michael > > > > > Cheers > > Dave > > > >> > >> /* FM Radio Modulator controls */ > >> /* Keep the order of the 'case's the same as in v4l2-controls.h! */ > >> @@ -1593,6 +1595,11 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, > >> *flags |= V4L2_CTRL_FLAG_WRITE_ONLY | > >> V4L2_CTRL_FLAG_EXECUTE_ON_WRITE; > >> break; > >> + case V4L2_CID_FOCUS_STATUS: > >> + case V4L2_CID_ZOOM_STATUS: > >> + *type = V4L2_CTRL_TYPE_LENS_STATUS; > >> + *flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE; > >> + break; > >> case V4L2_CID_FLASH_STROBE_STATUS: > >> case V4L2_CID_AUTO_FOCUS_STATUS: > >> case V4L2_CID_FLASH_READY: > >> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h > >> index e59d9a234631..f7273ffc20c9 100644 > >> --- a/include/media/v4l2-ctrls.h > >> +++ b/include/media/v4l2-ctrls.h > >> @@ -52,6 +52,7 @@ struct video_device; > >> * @p_hdr10_cll: Pointer to an HDR10 Content Light Level structure. > >> * @p_hdr10_mastering: Pointer to an HDR10 Mastering Display structure. > >> * @p_area: Pointer to an area. > >> + * @p_lens_status: Pointer to a lens status structure. > >> * @p: Pointer to a compound value. > >> * @p_const: Pointer to a constant compound value. > >> */ > >> @@ -81,6 +82,7 @@ union v4l2_ctrl_ptr { > >> struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll; > >> struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering; > >> struct v4l2_area *p_area; > >> + struct v4l2_ctrl_lens_status *p_lens_status; > >> void *p; > >> const void *p_const; > >> }; > >> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h > >> index 5e80daa4ffe0..8b037467ba9a 100644 > >> --- a/include/uapi/linux/v4l2-controls.h > >> +++ b/include/uapi/linux/v4l2-controls.h > >> @@ -993,6 +993,19 @@ enum v4l2_auto_focus_range { > >> > >> #define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36) > >> > >> +struct v4l2_ctrl_lens_status { > >> + __u32 flags; > >> + __s32 current_position; > >> +}; > >> + > >> +#define V4L2_LENS_STATUS_IDLE (0 << 0) > >> +#define V4L2_LENS_STATUS_BUSY (1 << 0) > >> +#define V4L2_LENS_STATUS_REACHED (1 << 1) > >> +#define V4L2_LENS_STATUS_FAILED (1 << 2) > >> + > >> +#define V4L2_CID_FOCUS_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 37) > >> +#define V4L2_CID_ZOOM_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 38) > >> + > >> /* FM Modulator class control IDs */ > >> > >> #define V4L2_CID_FM_TX_CLASS_BASE (V4L2_CTRL_CLASS_FM_TX | 0x900) > >> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > >> index 17a9b975177a..256c21c68720 100644 > >> --- a/include/uapi/linux/videodev2.h > >> +++ b/include/uapi/linux/videodev2.h > >> @@ -1888,6 +1888,8 @@ enum v4l2_ctrl_type { > >> V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272, > >> V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273, > >> V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274, > >> + > >> + V4L2_CTRL_TYPE_LENS_STATUS = 0x0300, > >> }; > >> > >> /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */ > >> > >> -- > >> 2.30.2 > >>
Hi Dave, On 4/11/23 20:15, Dave Stevenson wrote: > Hi Michael > > On Tue, 11 Apr 2023 at 18:33, Michael Riesch > <michael.riesch@wolfvision.net> wrote: >> >> Hi Dave, >> >> On 4/6/23 17:16, Dave Stevenson wrote: >>> Hi Michael >>> >>> Thanks for the patch. >>> >>> I've got a personal interest here as I'd love to be able to control a >>> couple of CCTV lenses that I have. Those have standard motors with >>> potentiometers for position feedback, not stepper motors, but >>> otherwise have the same limitations of slow movement. >> >> That's great to hear :-) Thank you for your feedback! >> >>> On Thu, 6 Apr 2023 at 15:31, Michael Riesch via B4 Relay via >>> libcamera-devel <libcamera-devel@lists.libcamera.org> wrote: >>>> >>>> From: Michael Riesch <michael.riesch@wolfvision.net> >>>> >>>> Add the controls V4L2_CID_FOCUS_STATUS and V4L2_CID_ZOOM_STATUS that report >>>> the status of the zoom lens group and the focus lens group, respectively. >>>> The returned data structure contains the current position of the lens group >>>> as well as movement indication flags. >>>> >>>> Signed-off-by: Michael Riesch <michael.riesch@wolfvision.net> >>>> --- >>>> .../userspace-api/media/v4l/ext-ctrls-camera.rst | 48 ++++++++++++++++++++++ >>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 9 ++++ >>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 7 ++++ >>>> include/media/v4l2-ctrls.h | 2 + >>>> include/uapi/linux/v4l2-controls.h | 13 ++++++ >>>> include/uapi/linux/videodev2.h | 2 + >>>> 6 files changed, 81 insertions(+) >>>> >>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst >>>> index daa4f40869f8..3a270bc63f1a 100644 >>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst >>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst >>>> @@ -149,6 +149,30 @@ enum v4l2_exposure_metering - >>>> to the camera, negative values towards infinity. This is a >>>> write-only control. >>>> >>>> +``V4L2_CID_FOCUS_STATUS (struct)`` >>>> + The current status of the focus lens group. This is a read-only control. >>>> + The returned data structure contains the current position and movement >>>> + indication flags. The unit of the current position is undefined. Positive >>>> + values move the focus closer to the camera, negative values towards >>>> + infinity. The possible flags are described in the table below. >>> >>> The units are undefined, but positive and negative mean something with >>> respect to some unspecified focal distance represented by 0. That >>> seems a little odd. >>> >>> I was going to suggest that it seems to make sense to follow the same >>> units as V4L2_CID_FOCUS_ABSOLUTE, but on reading that description in >>> [1] it's the same text. >>> I suspect there was a little too much copy/paste from >>> V4L2_CID_FOCUS_RELATIVE, or the intent was that increasing the value >>> brings the focus closer, and decreasing moves it towards infinity. >>> >>> If we did specify that it was the same units as >>> V4L2_CID_FOCUS_ABSOLUTE, then what would that mean for use with >>> V4L2_CID_FOCUS_RELATIVE? Then again the only user of _RELATIVE appears >>> to be ov5693 with atomisp and that just maps it onto _ABSOLUTE, so >>> potentially it's redundant and could be deprecated. >>> >>> [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-camera.html >> >> I think we agree that the _STATUS controls should use the same unit as >> the corresponding _ABSOLUTE control, whatever that may be. >> >> Then, the question is whether the description of FOCUS_ABSOLUTE needs to >> be revised. I would agree that the intention probably was: "Larger >> values move the focus closer to the camera, smaller values move the >> focus to infinity." (cf. the phrasing in the description of IRIS_ABSOLUTE). >> >> It would be interesting to know whether zero and negative values are >> (intentionally?) included. Since they are not explicitly excluded, my >> driver exposes the position of the lens in motor steps to the user >> space. If the values were (supposed to be) restricted to positive values >> like ZOOM_ABSOLUTE, this would not be allowed of course. >> >> As to the relation to _RELATIVE, I think it should be clear that all >> controls should act on the same scale and I don't see any problems here. >> However, feel free to point out what I am missing :-) >> >>>> + >>>> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}| >>>> + >>>> +.. flat-table:: >>>> + :header-rows: 0 >>>> + :stub-columns: 0 >>>> + >>>> + * - ``V4L2_LENS_STATUS_IDLE`` >>>> + - Focus lens group is at rest. >>>> + * - ``V4L2_LENS_STATUS_BUSY`` >>>> + - Focus lens group is moving. >>>> + * - ``V4L2_LENS_STATUS_REACHED`` >>>> + - Focus lens group has reached its target position. >>>> + * - ``V4L2_LENS_STATUS_FAILED`` >>>> + - Focus lens group has failed to reach its target position. The driver >>>> + will not transition from this state until another action is performed >>>> + by an application. >>> >>> When would the lens state transition from V4L2_LENS_STATUS_REACHED to >>> V4L2_LENS_STATUS_IDLE? >>> If it's reached the position then it is at rest, and being at rest is >>> the definition of V4L2_LENS_STATUS_IDLE. >>> Likewise the lens always has a target position based on the control >>> value, so it's always at V4L2_LENS_STATUS_REACHED when it's not >>> moving. >>> Is there a need to have 2 states? >> >> Good point, I need to reconsider that (and possibly remove one of those >> states). >> >>> If the position is the same units as V4L2_CID_FOCUS_ABSOLUTE, then do >>> you leave the determination of state to the application? >> >> I am afraid I don't quite follow. > > If the application sets V4L2_CID_FOCUS_ABSOLUTE to 42, and > V4L2_CID_FOCUS_STATUS returns that the position is now 42, then the > application knows that the lens has reached the requested position. If > the two controls return different values then the lens is still > moving. > What new information does adding an additional status flag give you? I can think of two situations: - Hardware error: the lens stops at 40 (for whatever reason). Since 40 != 42 the lens is still moving, although a hardware error has occured that may need recovery or at least reporting. - Different controls: If moving = (V4L2_CID_FOCUS_ABSOLUTE == current), then what happens if the application performs a V4L2_CID_FOCUS_RELATIVE with -3? current should reach 39, V4L2_CID_FOCUS_ABSOLUTE is still at 42, the lens is still moving from the application's point of view. Don't get me wrong, a simple control that reports the current position would be way easier. But I feel that we may regret this in the future. Also, I would not use two separate controls "current" and "flags/status" in order to avoid non-atomic access patterns. > (Thinking aloud) I guess you have got my case where your readback is > via an ADC so that the position may fluctuate slightly due to > conversion noise. The control loop for applying power to the motors > will presumably stop at some point and stop trying to adjust the > position, so potentially it could be the state returned from that > control loop. However the noisy ADC position could also be solved by > the position being returned by the control loop instead of giving the > raw value. With stepper motors the readback is not that noisy, but in any case the flags field allows an underlying controller to signal certain conditions without relying on the exact value of the current position. I think that is a plus, especially for noisy readback. >> But FWIW the application should read out the flags to find out whether >> the lens is moving and, if this is not the case, whether the last move >> command (FOCUS_ABSOLUTE or FOCUS_RELATIVE) has succeeded or failed. The >> failed state is important as there might be the possibility that >> depending on the zoom lens position certain focus positions are not >> available (and vice versa). > > How would you configure those kinds of restrictions in the kernel > driver? Is the kernel driver the best place to do it (so many things > are being kicked back to userspace these days)? In our case we have a hardware controller that needs to handle the restrictions anyway in order to avoid mechanical damage. The kernel driver is only responsible for reading out the controller status and transforming it into the V4L2 control. I am not sure how any restrictions can be reliably handled in software, hence I don't have a strong opinion on whether this is done in kernel or user space. > If it is to be done in the kernel, then shouldn't eg changing the zoom > position alter the ranges advertised for the relevant focus controls? While it should be possible to update the minimum and maximum of e.g., FOCUS_ABSOLUTE, I am not sure whether the interface is designed for frequent re-reading of the range. > If the ranges aren't updated, where should that out-of-range lens > movement leave the lens? This is up to the hardware controller, but I would guess it typically stops one step before disaster. Wherever that may be, the error condition and the current position can be read out via this new STATUS control. Does this sound good so far? Best regards, Michael > > Dave > >>>> ``V4L2_CID_FOCUS_AUTO (boolean)`` >>>> Enables continuous automatic focus adjustments. The effect of manual >>>> focus adjustments while this feature is enabled is undefined, >>>> @@ -239,6 +263,30 @@ enum v4l2_auto_focus_range - >>>> movement. A negative value moves the zoom lens group towards the >>>> wide-angle direction. The zoom speed unit is driver-specific. >>>> >>>> +``V4L2_CID_ZOOM_STATUS (struct)`` >>>> + The current status of the zoom lens group. This is a read-only control. >>>> + The returned data structure contains the current position and movement >>>> + indication flags. The unit of the current position is driver-specific and >>>> + its value should be a positive integer. The possible flags are described >>>> + in the table below. >>>> + >>>> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}| >>>> + >>>> +.. flat-table:: >>>> + :header-rows: 0 >>>> + :stub-columns: 0 >>>> + >>>> + * - ``V4L2_LENS_STATUS_IDLE`` >>>> + - Zoom lens group is at rest. >>>> + * - ``V4L2_LENS_STATUS_BUSY`` >>>> + - Zoom lens group is moving. >>>> + * - ``V4L2_LENS_STATUS_REACHED`` >>>> + - Zoom lens group has reached its target position. >>>> + * - ``V4L2_LENS_STATUS_FAILED`` >>>> + - Zoom lens group has failed to reach its target position. The driver will >>>> + not transition from this state until another action is performed by an >>>> + application. >>>> + >>>> ``V4L2_CID_IRIS_ABSOLUTE (integer)`` >>>> This control sets the camera's aperture to the specified value. The >>>> unit is undefined. Larger values open the iris wider, smaller values >>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c >>>> index 29169170880a..f6ad30f311c5 100644 >>>> --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c >>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c >>>> @@ -350,6 +350,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl) >>>> case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS: >>>> pr_cont("HEVC_DECODE_PARAMS"); >>>> break; >>>> + case V4L2_CTRL_TYPE_LENS_STATUS: >>>> + pr_cont("LENS_STATUS"); >>>> + break; >>>> default: >>>> pr_cont("unknown type %d", ctrl->type); >>>> break; >>>> @@ -918,6 +921,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx, >>>> return -EINVAL; >>>> break; >>>> >>>> + case V4L2_CTRL_TYPE_LENS_STATUS: >>>> + break; >>>> + >>>> default: >>>> return -EINVAL; >>>> } >>>> @@ -1605,6 +1611,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, >>>> case V4L2_CTRL_TYPE_AREA: >>>> elem_size = sizeof(struct v4l2_area); >>>> break; >>>> + case V4L2_CTRL_TYPE_LENS_STATUS: >>>> + elem_size = sizeof(struct v4l2_ctrl_lens_status); >>>> + break; >>>> default: >>>> if (type < V4L2_CTRL_COMPOUND_TYPES) >>>> elem_size = sizeof(s32); >>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c >>>> index 564fedee2c88..9b26a3aa9e9c 100644 >>>> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c >>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c >>>> @@ -1044,6 +1044,8 @@ const char *v4l2_ctrl_get_name(u32 id) >>>> case V4L2_CID_CAMERA_ORIENTATION: return "Camera Orientation"; >>>> case V4L2_CID_CAMERA_SENSOR_ROTATION: return "Camera Sensor Rotation"; >>>> case V4L2_CID_HDR_SENSOR_MODE: return "HDR Sensor Mode"; >>>> + case V4L2_CID_FOCUS_STATUS: return "Focus, Status"; >>>> + case V4L2_CID_ZOOM_STATUS: return "Zoom, Status"; >>> >>> Is there a need for the comma in the text strings? >> >> Not sure, actually. Some other strings used commas. Monkey see, monkey do. >> >> Best regards, >> Michael >> >>> >>> Cheers >>> Dave >>> >>>> >>>> /* FM Radio Modulator controls */ >>>> /* Keep the order of the 'case's the same as in v4l2-controls.h! */ >>>> @@ -1593,6 +1595,11 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, >>>> *flags |= V4L2_CTRL_FLAG_WRITE_ONLY | >>>> V4L2_CTRL_FLAG_EXECUTE_ON_WRITE; >>>> break; >>>> + case V4L2_CID_FOCUS_STATUS: >>>> + case V4L2_CID_ZOOM_STATUS: >>>> + *type = V4L2_CTRL_TYPE_LENS_STATUS; >>>> + *flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE; >>>> + break; >>>> case V4L2_CID_FLASH_STROBE_STATUS: >>>> case V4L2_CID_AUTO_FOCUS_STATUS: >>>> case V4L2_CID_FLASH_READY: >>>> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h >>>> index e59d9a234631..f7273ffc20c9 100644 >>>> --- a/include/media/v4l2-ctrls.h >>>> +++ b/include/media/v4l2-ctrls.h >>>> @@ -52,6 +52,7 @@ struct video_device; >>>> * @p_hdr10_cll: Pointer to an HDR10 Content Light Level structure. >>>> * @p_hdr10_mastering: Pointer to an HDR10 Mastering Display structure. >>>> * @p_area: Pointer to an area. >>>> + * @p_lens_status: Pointer to a lens status structure. >>>> * @p: Pointer to a compound value. >>>> * @p_const: Pointer to a constant compound value. >>>> */ >>>> @@ -81,6 +82,7 @@ union v4l2_ctrl_ptr { >>>> struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll; >>>> struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering; >>>> struct v4l2_area *p_area; >>>> + struct v4l2_ctrl_lens_status *p_lens_status; >>>> void *p; >>>> const void *p_const; >>>> }; >>>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h >>>> index 5e80daa4ffe0..8b037467ba9a 100644 >>>> --- a/include/uapi/linux/v4l2-controls.h >>>> +++ b/include/uapi/linux/v4l2-controls.h >>>> @@ -993,6 +993,19 @@ enum v4l2_auto_focus_range { >>>> >>>> #define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36) >>>> >>>> +struct v4l2_ctrl_lens_status { >>>> + __u32 flags; >>>> + __s32 current_position; >>>> +}; >>>> + >>>> +#define V4L2_LENS_STATUS_IDLE (0 << 0) >>>> +#define V4L2_LENS_STATUS_BUSY (1 << 0) >>>> +#define V4L2_LENS_STATUS_REACHED (1 << 1) >>>> +#define V4L2_LENS_STATUS_FAILED (1 << 2) >>>> + >>>> +#define V4L2_CID_FOCUS_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 37) >>>> +#define V4L2_CID_ZOOM_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 38) >>>> + >>>> /* FM Modulator class control IDs */ >>>> >>>> #define V4L2_CID_FM_TX_CLASS_BASE (V4L2_CTRL_CLASS_FM_TX | 0x900) >>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h >>>> index 17a9b975177a..256c21c68720 100644 >>>> --- a/include/uapi/linux/videodev2.h >>>> +++ b/include/uapi/linux/videodev2.h >>>> @@ -1888,6 +1888,8 @@ enum v4l2_ctrl_type { >>>> V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272, >>>> V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273, >>>> V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274, >>>> + >>>> + V4L2_CTRL_TYPE_LENS_STATUS = 0x0300, >>>> }; >>>> >>>> /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */ >>>> >>>> -- >>>> 2.30.2 >>>>
Hi Michael, On Wed, Apr 12, 2023 at 10:00:26AM +0200, Michael Riesch wrote: > - Different controls: If moving = (V4L2_CID_FOCUS_ABSOLUTE == current), > then what happens if the application performs a > V4L2_CID_FOCUS_RELATIVE with -3? current should reach 39, > V4L2_CID_FOCUS_ABSOLUTE is still at 42, the lens is still moving from > the application's point of view. Would there be a reason to implement both of these controls in a single driver? AFAIU, the relative one should be used if there absolute value isn't known to the driver.
Hi Sakari, On 4/12/23 13:50, Sakari Ailus wrote: > Hi Michael, > > On Wed, Apr 12, 2023 at 10:00:26AM +0200, Michael Riesch wrote: >> - Different controls: If moving = (V4L2_CID_FOCUS_ABSOLUTE == current), >> then what happens if the application performs a >> V4L2_CID_FOCUS_RELATIVE with -3? current should reach 39, >> V4L2_CID_FOCUS_ABSOLUTE is still at 42, the lens is still moving from >> the application's point of view. > > Would there be a reason to implement both of these controls in a single > driver? AFAIU, the relative one should be used if there absolute value > isn't known to the driver. Probably not, but on the other hand there is nothing the prevents a driver developer from doing so, right? Point is that should there be a driver which does implement both controls, we are in trouble AFAIU. Best regards, Michael
Hi Michael, On Wed, Apr 12, 2023 at 01:57:36PM +0200, Michael Riesch wrote: > Hi Sakari, > > On 4/12/23 13:50, Sakari Ailus wrote: > > Hi Michael, > > > > On Wed, Apr 12, 2023 at 10:00:26AM +0200, Michael Riesch wrote: > >> - Different controls: If moving = (V4L2_CID_FOCUS_ABSOLUTE == current), > >> then what happens if the application performs a > >> V4L2_CID_FOCUS_RELATIVE with -3? current should reach 39, > >> V4L2_CID_FOCUS_ABSOLUTE is still at 42, the lens is still moving from > >> the application's point of view. > > > > Would there be a reason to implement both of these controls in a single > > driver? AFAIU, the relative one should be used if there absolute value > > isn't known to the driver. > > Probably not, but on the other hand there is nothing the prevents a > driver developer from doing so, right? Point is that should there be a > driver which does implement both controls, we are in trouble AFAIU. I think the documentation should be improved in this regard. Also cc Hans.
Hi Sakari, On 4/12/23 14:05, Sakari Ailus wrote: > Hi Michael, > > On Wed, Apr 12, 2023 at 01:57:36PM +0200, Michael Riesch wrote: >> Hi Sakari, >> >> On 4/12/23 13:50, Sakari Ailus wrote: >>> Hi Michael, >>> >>> On Wed, Apr 12, 2023 at 10:00:26AM +0200, Michael Riesch wrote: >>>> - Different controls: If moving = (V4L2_CID_FOCUS_ABSOLUTE == current), >>>> then what happens if the application performs a >>>> V4L2_CID_FOCUS_RELATIVE with -3? current should reach 39, >>>> V4L2_CID_FOCUS_ABSOLUTE is still at 42, the lens is still moving from >>>> the application's point of view. >>> >>> Would there be a reason to implement both of these controls in a single >>> driver? AFAIU, the relative one should be used if there absolute value >>> isn't known to the driver. >> >> Probably not, but on the other hand there is nothing the prevents a >> driver developer from doing so, right? Point is that should there be a >> driver which does implement both controls, we are in trouble AFAIU. > > I think the documentation should be improved in this regard. The documentation of which control exactly? And what items should be added? Thanks for the pointers! Best regards, Michael > > Also cc Hans. >
Hi Michael On Wed, 12 Apr 2023 at 09:00, Michael Riesch <michael.riesch@wolfvision.net> wrote: > > Hi Dave, > > On 4/11/23 20:15, Dave Stevenson wrote: > > Hi Michael > > > > On Tue, 11 Apr 2023 at 18:33, Michael Riesch > > <michael.riesch@wolfvision.net> wrote: > >> > >> Hi Dave, > >> > >> On 4/6/23 17:16, Dave Stevenson wrote: > >>> Hi Michael > >>> > >>> Thanks for the patch. > >>> > >>> I've got a personal interest here as I'd love to be able to control a > >>> couple of CCTV lenses that I have. Those have standard motors with > >>> potentiometers for position feedback, not stepper motors, but > >>> otherwise have the same limitations of slow movement. > >> > >> That's great to hear :-) Thank you for your feedback! > >> > >>> On Thu, 6 Apr 2023 at 15:31, Michael Riesch via B4 Relay via > >>> libcamera-devel <libcamera-devel@lists.libcamera.org> wrote: > >>>> > >>>> From: Michael Riesch <michael.riesch@wolfvision.net> > >>>> > >>>> Add the controls V4L2_CID_FOCUS_STATUS and V4L2_CID_ZOOM_STATUS that report > >>>> the status of the zoom lens group and the focus lens group, respectively. > >>>> The returned data structure contains the current position of the lens group > >>>> as well as movement indication flags. > >>>> > >>>> Signed-off-by: Michael Riesch <michael.riesch@wolfvision.net> > >>>> --- > >>>> .../userspace-api/media/v4l/ext-ctrls-camera.rst | 48 ++++++++++++++++++++++ > >>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 9 ++++ > >>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 7 ++++ > >>>> include/media/v4l2-ctrls.h | 2 + > >>>> include/uapi/linux/v4l2-controls.h | 13 ++++++ > >>>> include/uapi/linux/videodev2.h | 2 + > >>>> 6 files changed, 81 insertions(+) > >>>> > >>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst > >>>> index daa4f40869f8..3a270bc63f1a 100644 > >>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst > >>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst > >>>> @@ -149,6 +149,30 @@ enum v4l2_exposure_metering - > >>>> to the camera, negative values towards infinity. This is a > >>>> write-only control. > >>>> > >>>> +``V4L2_CID_FOCUS_STATUS (struct)`` > >>>> + The current status of the focus lens group. This is a read-only control. > >>>> + The returned data structure contains the current position and movement > >>>> + indication flags. The unit of the current position is undefined. Positive > >>>> + values move the focus closer to the camera, negative values towards > >>>> + infinity. The possible flags are described in the table below. > >>> > >>> The units are undefined, but positive and negative mean something with > >>> respect to some unspecified focal distance represented by 0. That > >>> seems a little odd. > >>> > >>> I was going to suggest that it seems to make sense to follow the same > >>> units as V4L2_CID_FOCUS_ABSOLUTE, but on reading that description in > >>> [1] it's the same text. > >>> I suspect there was a little too much copy/paste from > >>> V4L2_CID_FOCUS_RELATIVE, or the intent was that increasing the value > >>> brings the focus closer, and decreasing moves it towards infinity. > >>> > >>> If we did specify that it was the same units as > >>> V4L2_CID_FOCUS_ABSOLUTE, then what would that mean for use with > >>> V4L2_CID_FOCUS_RELATIVE? Then again the only user of _RELATIVE appears > >>> to be ov5693 with atomisp and that just maps it onto _ABSOLUTE, so > >>> potentially it's redundant and could be deprecated. > >>> > >>> [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-camera.html > >> > >> I think we agree that the _STATUS controls should use the same unit as > >> the corresponding _ABSOLUTE control, whatever that may be. > >> > >> Then, the question is whether the description of FOCUS_ABSOLUTE needs to > >> be revised. I would agree that the intention probably was: "Larger > >> values move the focus closer to the camera, smaller values move the > >> focus to infinity." (cf. the phrasing in the description of IRIS_ABSOLUTE). > >> > >> It would be interesting to know whether zero and negative values are > >> (intentionally?) included. Since they are not explicitly excluded, my > >> driver exposes the position of the lens in motor steps to the user > >> space. If the values were (supposed to be) restricted to positive values > >> like ZOOM_ABSOLUTE, this would not be allowed of course. > >> > >> As to the relation to _RELATIVE, I think it should be clear that all > >> controls should act on the same scale and I don't see any problems here. > >> However, feel free to point out what I am missing :-) > >> > >>>> + > >>>> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}| > >>>> + > >>>> +.. flat-table:: > >>>> + :header-rows: 0 > >>>> + :stub-columns: 0 > >>>> + > >>>> + * - ``V4L2_LENS_STATUS_IDLE`` > >>>> + - Focus lens group is at rest. > >>>> + * - ``V4L2_LENS_STATUS_BUSY`` > >>>> + - Focus lens group is moving. > >>>> + * - ``V4L2_LENS_STATUS_REACHED`` > >>>> + - Focus lens group has reached its target position. > >>>> + * - ``V4L2_LENS_STATUS_FAILED`` > >>>> + - Focus lens group has failed to reach its target position. The driver > >>>> + will not transition from this state until another action is performed > >>>> + by an application. > >>> > >>> When would the lens state transition from V4L2_LENS_STATUS_REACHED to > >>> V4L2_LENS_STATUS_IDLE? > >>> If it's reached the position then it is at rest, and being at rest is > >>> the definition of V4L2_LENS_STATUS_IDLE. > >>> Likewise the lens always has a target position based on the control > >>> value, so it's always at V4L2_LENS_STATUS_REACHED when it's not > >>> moving. > >>> Is there a need to have 2 states? > >> > >> Good point, I need to reconsider that (and possibly remove one of those > >> states). > >> > >>> If the position is the same units as V4L2_CID_FOCUS_ABSOLUTE, then do > >>> you leave the determination of state to the application? > >> > >> I am afraid I don't quite follow. > > > > If the application sets V4L2_CID_FOCUS_ABSOLUTE to 42, and > > V4L2_CID_FOCUS_STATUS returns that the position is now 42, then the > > application knows that the lens has reached the requested position. If > > the two controls return different values then the lens is still > > moving. > > What new information does adding an additional status flag give you? > > I can think of two situations: > > - Hardware error: the lens stops at 40 (for whatever reason). Since 40 > != 42 the lens is still moving, although a hardware error has occured > that may need recovery or at least reporting. Are you thinking of the lens drive having totally failed, or just that the requested position is out of the achievable range? Looking at the flash controls[1], hardware failures are handled by V4L2_CID_FLASH_FAULT as a separate control. [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-flash.html > - Different controls: If moving = (V4L2_CID_FOCUS_ABSOLUTE == current), > then what happens if the application performs a > V4L2_CID_FOCUS_RELATIVE with -3? current should reach 39, > V4L2_CID_FOCUS_ABSOLUTE is still at 42, the lens is still moving from > the application's point of view. V4L2_CID_FOCUS_RELATIVE really ought to die IMHO, or at least work in co-operation with V4L2_CID_FOCUS_ABSOLUTE such that the absolute current position can always be retrieved. Using _RELATIVE to set a position that is outside of the range defined by _ABSOLUTE is meaningless. Any call to S_CTRL on _RELATIVE almost has to be followed by a call to G_CTRL _ABSOLUTE to know the final result, so why not just make a S_CTRL _ABSOLUTE call in the first place. I guess _RELATIVE may have uses in the case where there is no read-back of the position at all, but in that case I don't see a way that your new controls could be implemented. > Don't get me wrong, a simple control that reports the current position > would be way easier. But I feel that we may regret this in the future. > Also, I would not use two separate controls "current" and "flags/status" > in order to avoid non-atomic access patterns. I have no fixed views, but overcomplicating things is generally bad news. > > (Thinking aloud) I guess you have got my case where your readback is > > via an ADC so that the position may fluctuate slightly due to > > conversion noise. The control loop for applying power to the motors > > will presumably stop at some point and stop trying to adjust the > > position, so potentially it could be the state returned from that > > control loop. However the noisy ADC position could also be solved by > > the position being returned by the control loop instead of giving the > > raw value. > > With stepper motors the readback is not that noisy, but in any case the > flags field allows an underlying controller to signal certain conditions > without relying on the exact value of the current position. I think that > is a plus, especially for noisy readback. > > >> But FWIW the application should read out the flags to find out whether > >> the lens is moving and, if this is not the case, whether the last move > >> command (FOCUS_ABSOLUTE or FOCUS_RELATIVE) has succeeded or failed. The > >> failed state is important as there might be the possibility that > >> depending on the zoom lens position certain focus positions are not > >> available (and vice versa). > > > > How would you configure those kinds of restrictions in the kernel > > driver? Is the kernel driver the best place to do it (so many things > > are being kicked back to userspace these days)? > > In our case we have a hardware controller that needs to handle the > restrictions anyway in order to avoid mechanical damage. The kernel > driver is only responsible for reading out the controller status and > transforming it into the V4L2 control. How does userspace know what the valid ranges are? Do you end up with the same restrictions having to be coded in both your controller and userspace, or are you relying on the driver reporting back this error status with userspace not knowing by how much it got it wrong by? > I am not sure how any restrictions can be reliably handled in software, > hence I don't have a strong opinion on whether this is done in kernel or > user space. > > > If it is to be done in the kernel, then shouldn't eg changing the zoom > > position alter the ranges advertised for the relevant focus controls? > > While it should be possible to update the minimum and maximum of e.g., > FOCUS_ABSOLUTE, I am not sure whether the interface is designed for > frequent re-reading of the range. There's nothing in there that stops you, and several hooks to help you. Updating V4L2_CID_VBLANK on most raw sensors will update the range on V4L2_CID_EXPOSURE, as the exposure period can't be longer than the frame period. It would be similar that each change to ZOOM_ABSOLUTE would result in a range adjustment to FOCUS_ABSOLUTE. Userspace can also use VIDIOC_SUBSCRIBE_EVENT to subscribe to any updates in controls, so it gets told automatically of any updates. > > If the ranges aren't updated, where should that out-of-range lens > > movement leave the lens? > > This is up to the hardware controller, but I would guess it typically > stops one step before disaster. Wherever that may be, the error > condition and the current position can be read out via this new STATUS > control. > > Does this sound good so far? Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or Laurent), and I'm just expressing my views based on the lenses I've encountered. All of my lenses have a single drive for focus, a single drive for zoom, and where there are multiple elements they are all connected mechanically. Your setup sounds far more complex and is likely to need a more extensive driver, but it'd be nice to not unnecessarily overcomplicate the interface. Dave > Best regards, > Michael > > > > > Dave > > > >>>> ``V4L2_CID_FOCUS_AUTO (boolean)`` > >>>> Enables continuous automatic focus adjustments. The effect of manual > >>>> focus adjustments while this feature is enabled is undefined, > >>>> @@ -239,6 +263,30 @@ enum v4l2_auto_focus_range - > >>>> movement. A negative value moves the zoom lens group towards the > >>>> wide-angle direction. The zoom speed unit is driver-specific. > >>>> > >>>> +``V4L2_CID_ZOOM_STATUS (struct)`` > >>>> + The current status of the zoom lens group. This is a read-only control. > >>>> + The returned data structure contains the current position and movement > >>>> + indication flags. The unit of the current position is driver-specific and > >>>> + its value should be a positive integer. The possible flags are described > >>>> + in the table below. > >>>> + > >>>> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}| > >>>> + > >>>> +.. flat-table:: > >>>> + :header-rows: 0 > >>>> + :stub-columns: 0 > >>>> + > >>>> + * - ``V4L2_LENS_STATUS_IDLE`` > >>>> + - Zoom lens group is at rest. > >>>> + * - ``V4L2_LENS_STATUS_BUSY`` > >>>> + - Zoom lens group is moving. > >>>> + * - ``V4L2_LENS_STATUS_REACHED`` > >>>> + - Zoom lens group has reached its target position. > >>>> + * - ``V4L2_LENS_STATUS_FAILED`` > >>>> + - Zoom lens group has failed to reach its target position. The driver will > >>>> + not transition from this state until another action is performed by an > >>>> + application. > >>>> + > >>>> ``V4L2_CID_IRIS_ABSOLUTE (integer)`` > >>>> This control sets the camera's aperture to the specified value. The > >>>> unit is undefined. Larger values open the iris wider, smaller values > >>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c > >>>> index 29169170880a..f6ad30f311c5 100644 > >>>> --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c > >>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c > >>>> @@ -350,6 +350,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl) > >>>> case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS: > >>>> pr_cont("HEVC_DECODE_PARAMS"); > >>>> break; > >>>> + case V4L2_CTRL_TYPE_LENS_STATUS: > >>>> + pr_cont("LENS_STATUS"); > >>>> + break; > >>>> default: > >>>> pr_cont("unknown type %d", ctrl->type); > >>>> break; > >>>> @@ -918,6 +921,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx, > >>>> return -EINVAL; > >>>> break; > >>>> > >>>> + case V4L2_CTRL_TYPE_LENS_STATUS: > >>>> + break; > >>>> + > >>>> default: > >>>> return -EINVAL; > >>>> } > >>>> @@ -1605,6 +1611,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, > >>>> case V4L2_CTRL_TYPE_AREA: > >>>> elem_size = sizeof(struct v4l2_area); > >>>> break; > >>>> + case V4L2_CTRL_TYPE_LENS_STATUS: > >>>> + elem_size = sizeof(struct v4l2_ctrl_lens_status); > >>>> + break; > >>>> default: > >>>> if (type < V4L2_CTRL_COMPOUND_TYPES) > >>>> elem_size = sizeof(s32); > >>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c > >>>> index 564fedee2c88..9b26a3aa9e9c 100644 > >>>> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c > >>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c > >>>> @@ -1044,6 +1044,8 @@ const char *v4l2_ctrl_get_name(u32 id) > >>>> case V4L2_CID_CAMERA_ORIENTATION: return "Camera Orientation"; > >>>> case V4L2_CID_CAMERA_SENSOR_ROTATION: return "Camera Sensor Rotation"; > >>>> case V4L2_CID_HDR_SENSOR_MODE: return "HDR Sensor Mode"; > >>>> + case V4L2_CID_FOCUS_STATUS: return "Focus, Status"; > >>>> + case V4L2_CID_ZOOM_STATUS: return "Zoom, Status"; > >>> > >>> Is there a need for the comma in the text strings? > >> > >> Not sure, actually. Some other strings used commas. Monkey see, monkey do. > >> > >> Best regards, > >> Michael > >> > >>> > >>> Cheers > >>> Dave > >>> > >>>> > >>>> /* FM Radio Modulator controls */ > >>>> /* Keep the order of the 'case's the same as in v4l2-controls.h! */ > >>>> @@ -1593,6 +1595,11 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, > >>>> *flags |= V4L2_CTRL_FLAG_WRITE_ONLY | > >>>> V4L2_CTRL_FLAG_EXECUTE_ON_WRITE; > >>>> break; > >>>> + case V4L2_CID_FOCUS_STATUS: > >>>> + case V4L2_CID_ZOOM_STATUS: > >>>> + *type = V4L2_CTRL_TYPE_LENS_STATUS; > >>>> + *flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE; > >>>> + break; > >>>> case V4L2_CID_FLASH_STROBE_STATUS: > >>>> case V4L2_CID_AUTO_FOCUS_STATUS: > >>>> case V4L2_CID_FLASH_READY: > >>>> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h > >>>> index e59d9a234631..f7273ffc20c9 100644 > >>>> --- a/include/media/v4l2-ctrls.h > >>>> +++ b/include/media/v4l2-ctrls.h > >>>> @@ -52,6 +52,7 @@ struct video_device; > >>>> * @p_hdr10_cll: Pointer to an HDR10 Content Light Level structure. > >>>> * @p_hdr10_mastering: Pointer to an HDR10 Mastering Display structure. > >>>> * @p_area: Pointer to an area. > >>>> + * @p_lens_status: Pointer to a lens status structure. > >>>> * @p: Pointer to a compound value. > >>>> * @p_const: Pointer to a constant compound value. > >>>> */ > >>>> @@ -81,6 +82,7 @@ union v4l2_ctrl_ptr { > >>>> struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll; > >>>> struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering; > >>>> struct v4l2_area *p_area; > >>>> + struct v4l2_ctrl_lens_status *p_lens_status; > >>>> void *p; > >>>> const void *p_const; > >>>> }; > >>>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h > >>>> index 5e80daa4ffe0..8b037467ba9a 100644 > >>>> --- a/include/uapi/linux/v4l2-controls.h > >>>> +++ b/include/uapi/linux/v4l2-controls.h > >>>> @@ -993,6 +993,19 @@ enum v4l2_auto_focus_range { > >>>> > >>>> #define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36) > >>>> > >>>> +struct v4l2_ctrl_lens_status { > >>>> + __u32 flags; > >>>> + __s32 current_position; > >>>> +}; > >>>> + > >>>> +#define V4L2_LENS_STATUS_IDLE (0 << 0) > >>>> +#define V4L2_LENS_STATUS_BUSY (1 << 0) > >>>> +#define V4L2_LENS_STATUS_REACHED (1 << 1) > >>>> +#define V4L2_LENS_STATUS_FAILED (1 << 2) > >>>> + > >>>> +#define V4L2_CID_FOCUS_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 37) > >>>> +#define V4L2_CID_ZOOM_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 38) > >>>> + > >>>> /* FM Modulator class control IDs */ > >>>> > >>>> #define V4L2_CID_FM_TX_CLASS_BASE (V4L2_CTRL_CLASS_FM_TX | 0x900) > >>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > >>>> index 17a9b975177a..256c21c68720 100644 > >>>> --- a/include/uapi/linux/videodev2.h > >>>> +++ b/include/uapi/linux/videodev2.h > >>>> @@ -1888,6 +1888,8 @@ enum v4l2_ctrl_type { > >>>> V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272, > >>>> V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273, > >>>> V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274, > >>>> + > >>>> + V4L2_CTRL_TYPE_LENS_STATUS = 0x0300, > >>>> }; > >>>> > >>>> /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */ > >>>> > >>>> -- > >>>> 2.30.2 > >>>>
Hi Michael, On Wed, Apr 12, 2023 at 03:43:26PM +0200, Michael Riesch wrote: > Hi Sakari, > > On 4/12/23 14:05, Sakari Ailus wrote: > > Hi Michael, > > > > On Wed, Apr 12, 2023 at 01:57:36PM +0200, Michael Riesch wrote: > >> Hi Sakari, > >> > >> On 4/12/23 13:50, Sakari Ailus wrote: > >>> Hi Michael, > >>> > >>> On Wed, Apr 12, 2023 at 10:00:26AM +0200, Michael Riesch wrote: > >>>> - Different controls: If moving = (V4L2_CID_FOCUS_ABSOLUTE == current), > >>>> then what happens if the application performs a > >>>> V4L2_CID_FOCUS_RELATIVE with -3? current should reach 39, > >>>> V4L2_CID_FOCUS_ABSOLUTE is still at 42, the lens is still moving from > >>>> the application's point of view. > >>> > >>> Would there be a reason to implement both of these controls in a single > >>> driver? AFAIU, the relative one should be used if there absolute value > >>> isn't known to the driver. > >> > >> Probably not, but on the other hand there is nothing the prevents a > >> driver developer from doing so, right? Point is that should there be a > >> driver which does implement both controls, we are in trouble AFAIU. > > > > I think the documentation should be improved in this regard. > > The documentation of which control exactly? And what items should be added? Both V4L2_CID_FOCUS_ABSOLUTE and V4L2_CID_FOCUS_RELATIVE. For the former, the sentence "Positive values set the focus closer to the camera, negative values towards infinity." doesn't make much sense in the context. For the latter, what I mentioned earlier, i.e. this should be only implemented if the absolute value isn't known. It's not a driver's job to do simple arithmetics for the user space. In fact, it appears that no driver is using V4L2_CID_FOCUS_RELATIVE at the moment. So we could as well deprecate it (or remove from documentation altogether).
Hi Sakari On Wed, 12 Apr 2023 at 15:39, Sakari Ailus <sakari.ailus@linux.intel.com> wrote: > > Hi Michael, > > On Wed, Apr 12, 2023 at 03:43:26PM +0200, Michael Riesch wrote: > > Hi Sakari, > > > > On 4/12/23 14:05, Sakari Ailus wrote: > > > Hi Michael, > > > > > > On Wed, Apr 12, 2023 at 01:57:36PM +0200, Michael Riesch wrote: > > >> Hi Sakari, > > >> > > >> On 4/12/23 13:50, Sakari Ailus wrote: > > >>> Hi Michael, > > >>> > > >>> On Wed, Apr 12, 2023 at 10:00:26AM +0200, Michael Riesch wrote: > > >>>> - Different controls: If moving = (V4L2_CID_FOCUS_ABSOLUTE == current), > > >>>> then what happens if the application performs a > > >>>> V4L2_CID_FOCUS_RELATIVE with -3? current should reach 39, > > >>>> V4L2_CID_FOCUS_ABSOLUTE is still at 42, the lens is still moving from > > >>>> the application's point of view. > > >>> > > >>> Would there be a reason to implement both of these controls in a single > > >>> driver? AFAIU, the relative one should be used if there absolute value > > >>> isn't known to the driver. > > >> > > >> Probably not, but on the other hand there is nothing the prevents a > > >> driver developer from doing so, right? Point is that should there be a > > >> driver which does implement both controls, we are in trouble AFAIU. > > > > > > I think the documentation should be improved in this regard. > > > > The documentation of which control exactly? And what items should be added? > > Both V4L2_CID_FOCUS_ABSOLUTE and V4L2_CID_FOCUS_RELATIVE. For the former, > the sentence "Positive values set the focus closer to the camera, negative > values towards infinity." doesn't make much sense in the context. For the > latter, what I mentioned earlier, i.e. this should be only implemented if > the absolute value isn't known. It's not a driver's job to do simple > arithmetics for the user space. > > In fact, it appears that no driver is using V4L2_CID_FOCUS_RELATIVE at the > moment. So we could as well deprecate it (or remove from documentation > altogether). Atomisp and ov5693 are using it, but only to manipulate the value behind V4L2_CID_FOCUS_ABSOLUTE. I don't know why elixir.bootlin.com isn't picking up those references. https://github.com/torvalds/linux/blob/master/drivers/staging/media/atomisp/pci/atomisp_ioctl.c#L1766 https://github.com/torvalds/linux/blob/master/drivers/staging/media/atomisp/i2c/ov5693/atomisp-ov5693.c#L966 Dave
Hi Dave, Michael, On Wed, Apr 12, 2023 at 02:55:56PM +0100, Dave Stevenson wrote: > > > If the ranges aren't updated, where should that out-of-range lens > > > movement leave the lens? > > > > This is up to the hardware controller, but I would guess it typically > > stops one step before disaster. Wherever that may be, the error > > condition and the current position can be read out via this new STATUS > > control. > > > > Does this sound good so far? > > Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or > Laurent), and I'm just expressing my views based on the lenses I've > encountered. > All of my lenses have a single drive for focus, a single drive for > zoom, and where there are multiple elements they are all connected > mechanically. Your setup sounds far more complex and is likely to need > a more extensive driver, but it'd be nice to not unnecessarily > overcomplicate the interface. Could we also have a driver that uses these new controls? The controls themselves appear reasonable to me as well. I guess there are changes to be made based on the discussion?
Hi Dave, On 4/12/23 15:55, Dave Stevenson wrote: > Hi Michael > > On Wed, 12 Apr 2023 at 09:00, Michael Riesch > <michael.riesch@wolfvision.net> wrote: >> >> Hi Dave, >> >> On 4/11/23 20:15, Dave Stevenson wrote: >>> Hi Michael >>> >>> On Tue, 11 Apr 2023 at 18:33, Michael Riesch >>> <michael.riesch@wolfvision.net> wrote: >>>> >>>> Hi Dave, >>>> >>>> On 4/6/23 17:16, Dave Stevenson wrote: >>>>> Hi Michael >>>>> >>>>> Thanks for the patch. >>>>> >>>>> I've got a personal interest here as I'd love to be able to control a >>>>> couple of CCTV lenses that I have. Those have standard motors with >>>>> potentiometers for position feedback, not stepper motors, but >>>>> otherwise have the same limitations of slow movement. >>>> >>>> That's great to hear :-) Thank you for your feedback! >>>> >>>>> On Thu, 6 Apr 2023 at 15:31, Michael Riesch via B4 Relay via >>>>> libcamera-devel <libcamera-devel@lists.libcamera.org> wrote: >>>>>> >>>>>> From: Michael Riesch <michael.riesch@wolfvision.net> >>>>>> >>>>>> Add the controls V4L2_CID_FOCUS_STATUS and V4L2_CID_ZOOM_STATUS that report >>>>>> the status of the zoom lens group and the focus lens group, respectively. >>>>>> The returned data structure contains the current position of the lens group >>>>>> as well as movement indication flags. >>>>>> >>>>>> Signed-off-by: Michael Riesch <michael.riesch@wolfvision.net> >>>>>> --- >>>>>> .../userspace-api/media/v4l/ext-ctrls-camera.rst | 48 ++++++++++++++++++++++ >>>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 9 ++++ >>>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 7 ++++ >>>>>> include/media/v4l2-ctrls.h | 2 + >>>>>> include/uapi/linux/v4l2-controls.h | 13 ++++++ >>>>>> include/uapi/linux/videodev2.h | 2 + >>>>>> 6 files changed, 81 insertions(+) >>>>>> >>>>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst >>>>>> index daa4f40869f8..3a270bc63f1a 100644 >>>>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst >>>>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst >>>>>> @@ -149,6 +149,30 @@ enum v4l2_exposure_metering - >>>>>> to the camera, negative values towards infinity. This is a >>>>>> write-only control. >>>>>> >>>>>> +``V4L2_CID_FOCUS_STATUS (struct)`` >>>>>> + The current status of the focus lens group. This is a read-only control. >>>>>> + The returned data structure contains the current position and movement >>>>>> + indication flags. The unit of the current position is undefined. Positive >>>>>> + values move the focus closer to the camera, negative values towards >>>>>> + infinity. The possible flags are described in the table below. >>>>> >>>>> The units are undefined, but positive and negative mean something with >>>>> respect to some unspecified focal distance represented by 0. That >>>>> seems a little odd. >>>>> >>>>> I was going to suggest that it seems to make sense to follow the same >>>>> units as V4L2_CID_FOCUS_ABSOLUTE, but on reading that description in >>>>> [1] it's the same text. >>>>> I suspect there was a little too much copy/paste from >>>>> V4L2_CID_FOCUS_RELATIVE, or the intent was that increasing the value >>>>> brings the focus closer, and decreasing moves it towards infinity. >>>>> >>>>> If we did specify that it was the same units as >>>>> V4L2_CID_FOCUS_ABSOLUTE, then what would that mean for use with >>>>> V4L2_CID_FOCUS_RELATIVE? Then again the only user of _RELATIVE appears >>>>> to be ov5693 with atomisp and that just maps it onto _ABSOLUTE, so >>>>> potentially it's redundant and could be deprecated. >>>>> >>>>> [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-camera.html >>>> >>>> I think we agree that the _STATUS controls should use the same unit as >>>> the corresponding _ABSOLUTE control, whatever that may be. >>>> >>>> Then, the question is whether the description of FOCUS_ABSOLUTE needs to >>>> be revised. I would agree that the intention probably was: "Larger >>>> values move the focus closer to the camera, smaller values move the >>>> focus to infinity." (cf. the phrasing in the description of IRIS_ABSOLUTE). >>>> >>>> It would be interesting to know whether zero and negative values are >>>> (intentionally?) included. Since they are not explicitly excluded, my >>>> driver exposes the position of the lens in motor steps to the user >>>> space. If the values were (supposed to be) restricted to positive values >>>> like ZOOM_ABSOLUTE, this would not be allowed of course. >>>> >>>> As to the relation to _RELATIVE, I think it should be clear that all >>>> controls should act on the same scale and I don't see any problems here. >>>> However, feel free to point out what I am missing :-) >>>> >>>>>> + >>>>>> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}| >>>>>> + >>>>>> +.. flat-table:: >>>>>> + :header-rows: 0 >>>>>> + :stub-columns: 0 >>>>>> + >>>>>> + * - ``V4L2_LENS_STATUS_IDLE`` >>>>>> + - Focus lens group is at rest. >>>>>> + * - ``V4L2_LENS_STATUS_BUSY`` >>>>>> + - Focus lens group is moving. >>>>>> + * - ``V4L2_LENS_STATUS_REACHED`` >>>>>> + - Focus lens group has reached its target position. >>>>>> + * - ``V4L2_LENS_STATUS_FAILED`` >>>>>> + - Focus lens group has failed to reach its target position. The driver >>>>>> + will not transition from this state until another action is performed >>>>>> + by an application. >>>>> >>>>> When would the lens state transition from V4L2_LENS_STATUS_REACHED to >>>>> V4L2_LENS_STATUS_IDLE? >>>>> If it's reached the position then it is at rest, and being at rest is >>>>> the definition of V4L2_LENS_STATUS_IDLE. >>>>> Likewise the lens always has a target position based on the control >>>>> value, so it's always at V4L2_LENS_STATUS_REACHED when it's not >>>>> moving. >>>>> Is there a need to have 2 states? >>>> >>>> Good point, I need to reconsider that (and possibly remove one of those >>>> states). >>>> >>>>> If the position is the same units as V4L2_CID_FOCUS_ABSOLUTE, then do >>>>> you leave the determination of state to the application? >>>> >>>> I am afraid I don't quite follow. >>> >>> If the application sets V4L2_CID_FOCUS_ABSOLUTE to 42, and >>> V4L2_CID_FOCUS_STATUS returns that the position is now 42, then the >>> application knows that the lens has reached the requested position. If >>> the two controls return different values then the lens is still >>> moving. >>> What new information does adding an additional status flag give you? >> >> I can think of two situations: >> >> - Hardware error: the lens stops at 40 (for whatever reason). Since 40 >> != 42 the lens is still moving, although a hardware error has occured >> that may need recovery or at least reporting. > > Are you thinking of the lens drive having totally failed, or just that > the requested position is out of the achievable range? I would say the latter. At least one of our optics (with one focus lens and two zoom lenses) exhibits nontrivial constraints for the range of the individual lenses. > Looking at the flash controls[1], hardware failures are handled by > V4L2_CID_FLASH_FAULT as a separate control. > > [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-flash.html > >> - Different controls: If moving = (V4L2_CID_FOCUS_ABSOLUTE == current), >> then what happens if the application performs a >> V4L2_CID_FOCUS_RELATIVE with -3? current should reach 39, >> V4L2_CID_FOCUS_ABSOLUTE is still at 42, the lens is still moving from >> the application's point of view. > > V4L2_CID_FOCUS_RELATIVE really ought to die IMHO, or at least work in > co-operation with V4L2_CID_FOCUS_ABSOLUTE such that the absolute > current position can always be retrieved. > > Using _RELATIVE to set a position that is outside of the range defined > by _ABSOLUTE is meaningless. Any call to S_CTRL on _RELATIVE almost > has to be followed by a call to G_CTRL _ABSOLUTE to know the final > result, so why not just make a S_CTRL _ABSOLUTE call in the first > place. > > I guess _RELATIVE may have uses in the case where there is no > read-back of the position at all, but in that case I don't see a way > that your new controls could be implemented. I am quite neutral to this. We don't plan to use the _RELATIVE controls and if you want to get rid of them there won't be any objections from our side. If the drivers should only implement either ABSOLUTE or RELATIVE then situation I described above is not an issue, of course. >> Don't get me wrong, a simple control that reports the current position >> would be way easier. But I feel that we may regret this in the future. >> Also, I would not use two separate controls "current" and "flags/status" >> in order to avoid non-atomic access patterns. > > I have no fixed views, but overcomplicating things is generally bad news. > >>> (Thinking aloud) I guess you have got my case where your readback is >>> via an ADC so that the position may fluctuate slightly due to >>> conversion noise. The control loop for applying power to the motors >>> will presumably stop at some point and stop trying to adjust the >>> position, so potentially it could be the state returned from that >>> control loop. However the noisy ADC position could also be solved by >>> the position being returned by the control loop instead of giving the >>> raw value. >> >> With stepper motors the readback is not that noisy, but in any case the >> flags field allows an underlying controller to signal certain conditions >> without relying on the exact value of the current position. I think that >> is a plus, especially for noisy readback. >> >>>> But FWIW the application should read out the flags to find out whether >>>> the lens is moving and, if this is not the case, whether the last move >>>> command (FOCUS_ABSOLUTE or FOCUS_RELATIVE) has succeeded or failed. The >>>> failed state is important as there might be the possibility that >>>> depending on the zoom lens position certain focus positions are not >>>> available (and vice versa). >>> >>> How would you configure those kinds of restrictions in the kernel >>> driver? Is the kernel driver the best place to do it (so many things >>> are being kicked back to userspace these days)? >> >> In our case we have a hardware controller that needs to handle the >> restrictions anyway in order to avoid mechanical damage. The kernel >> driver is only responsible for reading out the controller status and >> transforming it into the V4L2 control. > > How does userspace know what the valid ranges are? Do you end up with > the same restrictions having to be coded in both your controller and > userspace, or are you relying on the driver reporting back this error > status with userspace not knowing by how much it got it wrong by? Maybe I should clarify a bit the use cases here. Most of the time the ZOOM_ABSOLUTE and FOCUS_ABSOLUTE controls will be used to operate the optics. Here, AFAIK there should not be any constraints (or at least, they are handled by the lens controller hardware) and user space can pick any value between the minimum and maximum of the respective control. The situation is more complicated for the calibration. Here, individual lenses can be moved and may be subject to certain constraints. Before the calibration, the limits may be unknown or estimated. Then, one approach could be to set the target position of a lens to INT_MAX and read back the status. The error status plus the current position could then yield the actual limit. Note that such calibration techniques are very specialized use cases, but with the proposed API it should be possible to cover several different approaches. >> I am not sure how any restrictions can be reliably handled in software, >> hence I don't have a strong opinion on whether this is done in kernel or >> user space. >> >>> If it is to be done in the kernel, then shouldn't eg changing the zoom >>> position alter the ranges advertised for the relevant focus controls? >> >> While it should be possible to update the minimum and maximum of e.g., >> FOCUS_ABSOLUTE, I am not sure whether the interface is designed for >> frequent re-reading of the range. > > There's nothing in there that stops you, and several hooks to help you. > Updating V4L2_CID_VBLANK on most raw sensors will update the range on > V4L2_CID_EXPOSURE, as the exposure period can't be longer than the > frame period. It would be similar that each change to ZOOM_ABSOLUTE > would result in a range adjustment to FOCUS_ABSOLUTE. > Userspace can also use VIDIOC_SUBSCRIBE_EVENT to subscribe to any > updates in controls, so it gets told automatically of any updates. OK, maybe we'll give this some more thought! Thanks a lot for your comments! Best regards, Michael >>> If the ranges aren't updated, where should that out-of-range lens >>> movement leave the lens? >> >> This is up to the hardware controller, but I would guess it typically >> stops one step before disaster. Wherever that may be, the error >> condition and the current position can be read out via this new STATUS >> control. >> >> Does this sound good so far? > > Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or > Laurent), and I'm just expressing my views based on the lenses I've > encountered. > All of my lenses have a single drive for focus, a single drive for > zoom, and where there are multiple elements they are all connected > mechanically. Your setup sounds far more complex and is likely to need > a more extensive driver, but it'd be nice to not unnecessarily > overcomplicate the interface. > > Dave > >> Best regards, >> Michael >> >>> >>> Dave >>> >>>>>> ``V4L2_CID_FOCUS_AUTO (boolean)`` >>>>>> Enables continuous automatic focus adjustments. The effect of manual >>>>>> focus adjustments while this feature is enabled is undefined, >>>>>> @@ -239,6 +263,30 @@ enum v4l2_auto_focus_range - >>>>>> movement. A negative value moves the zoom lens group towards the >>>>>> wide-angle direction. The zoom speed unit is driver-specific. >>>>>> >>>>>> +``V4L2_CID_ZOOM_STATUS (struct)`` >>>>>> + The current status of the zoom lens group. This is a read-only control. >>>>>> + The returned data structure contains the current position and movement >>>>>> + indication flags. The unit of the current position is driver-specific and >>>>>> + its value should be a positive integer. The possible flags are described >>>>>> + in the table below. >>>>>> + >>>>>> +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}| >>>>>> + >>>>>> +.. flat-table:: >>>>>> + :header-rows: 0 >>>>>> + :stub-columns: 0 >>>>>> + >>>>>> + * - ``V4L2_LENS_STATUS_IDLE`` >>>>>> + - Zoom lens group is at rest. >>>>>> + * - ``V4L2_LENS_STATUS_BUSY`` >>>>>> + - Zoom lens group is moving. >>>>>> + * - ``V4L2_LENS_STATUS_REACHED`` >>>>>> + - Zoom lens group has reached its target position. >>>>>> + * - ``V4L2_LENS_STATUS_FAILED`` >>>>>> + - Zoom lens group has failed to reach its target position. The driver will >>>>>> + not transition from this state until another action is performed by an >>>>>> + application. >>>>>> + >>>>>> ``V4L2_CID_IRIS_ABSOLUTE (integer)`` >>>>>> This control sets the camera's aperture to the specified value. The >>>>>> unit is undefined. Larger values open the iris wider, smaller values >>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c >>>>>> index 29169170880a..f6ad30f311c5 100644 >>>>>> --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c >>>>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c >>>>>> @@ -350,6 +350,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl) >>>>>> case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS: >>>>>> pr_cont("HEVC_DECODE_PARAMS"); >>>>>> break; >>>>>> + case V4L2_CTRL_TYPE_LENS_STATUS: >>>>>> + pr_cont("LENS_STATUS"); >>>>>> + break; >>>>>> default: >>>>>> pr_cont("unknown type %d", ctrl->type); >>>>>> break; >>>>>> @@ -918,6 +921,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx, >>>>>> return -EINVAL; >>>>>> break; >>>>>> >>>>>> + case V4L2_CTRL_TYPE_LENS_STATUS: >>>>>> + break; >>>>>> + >>>>>> default: >>>>>> return -EINVAL; >>>>>> } >>>>>> @@ -1605,6 +1611,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, >>>>>> case V4L2_CTRL_TYPE_AREA: >>>>>> elem_size = sizeof(struct v4l2_area); >>>>>> break; >>>>>> + case V4L2_CTRL_TYPE_LENS_STATUS: >>>>>> + elem_size = sizeof(struct v4l2_ctrl_lens_status); >>>>>> + break; >>>>>> default: >>>>>> if (type < V4L2_CTRL_COMPOUND_TYPES) >>>>>> elem_size = sizeof(s32); >>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c >>>>>> index 564fedee2c88..9b26a3aa9e9c 100644 >>>>>> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c >>>>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c >>>>>> @@ -1044,6 +1044,8 @@ const char *v4l2_ctrl_get_name(u32 id) >>>>>> case V4L2_CID_CAMERA_ORIENTATION: return "Camera Orientation"; >>>>>> case V4L2_CID_CAMERA_SENSOR_ROTATION: return "Camera Sensor Rotation"; >>>>>> case V4L2_CID_HDR_SENSOR_MODE: return "HDR Sensor Mode"; >>>>>> + case V4L2_CID_FOCUS_STATUS: return "Focus, Status"; >>>>>> + case V4L2_CID_ZOOM_STATUS: return "Zoom, Status"; >>>>> >>>>> Is there a need for the comma in the text strings? >>>> >>>> Not sure, actually. Some other strings used commas. Monkey see, monkey do. >>>> >>>> Best regards, >>>> Michael >>>> >>>>> >>>>> Cheers >>>>> Dave >>>>> >>>>>> >>>>>> /* FM Radio Modulator controls */ >>>>>> /* Keep the order of the 'case's the same as in v4l2-controls.h! */ >>>>>> @@ -1593,6 +1595,11 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, >>>>>> *flags |= V4L2_CTRL_FLAG_WRITE_ONLY | >>>>>> V4L2_CTRL_FLAG_EXECUTE_ON_WRITE; >>>>>> break; >>>>>> + case V4L2_CID_FOCUS_STATUS: >>>>>> + case V4L2_CID_ZOOM_STATUS: >>>>>> + *type = V4L2_CTRL_TYPE_LENS_STATUS; >>>>>> + *flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE; >>>>>> + break; >>>>>> case V4L2_CID_FLASH_STROBE_STATUS: >>>>>> case V4L2_CID_AUTO_FOCUS_STATUS: >>>>>> case V4L2_CID_FLASH_READY: >>>>>> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h >>>>>> index e59d9a234631..f7273ffc20c9 100644 >>>>>> --- a/include/media/v4l2-ctrls.h >>>>>> +++ b/include/media/v4l2-ctrls.h >>>>>> @@ -52,6 +52,7 @@ struct video_device; >>>>>> * @p_hdr10_cll: Pointer to an HDR10 Content Light Level structure. >>>>>> * @p_hdr10_mastering: Pointer to an HDR10 Mastering Display structure. >>>>>> * @p_area: Pointer to an area. >>>>>> + * @p_lens_status: Pointer to a lens status structure. >>>>>> * @p: Pointer to a compound value. >>>>>> * @p_const: Pointer to a constant compound value. >>>>>> */ >>>>>> @@ -81,6 +82,7 @@ union v4l2_ctrl_ptr { >>>>>> struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll; >>>>>> struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering; >>>>>> struct v4l2_area *p_area; >>>>>> + struct v4l2_ctrl_lens_status *p_lens_status; >>>>>> void *p; >>>>>> const void *p_const; >>>>>> }; >>>>>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h >>>>>> index 5e80daa4ffe0..8b037467ba9a 100644 >>>>>> --- a/include/uapi/linux/v4l2-controls.h >>>>>> +++ b/include/uapi/linux/v4l2-controls.h >>>>>> @@ -993,6 +993,19 @@ enum v4l2_auto_focus_range { >>>>>> >>>>>> #define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36) >>>>>> >>>>>> +struct v4l2_ctrl_lens_status { >>>>>> + __u32 flags; >>>>>> + __s32 current_position; >>>>>> +}; >>>>>> + >>>>>> +#define V4L2_LENS_STATUS_IDLE (0 << 0) >>>>>> +#define V4L2_LENS_STATUS_BUSY (1 << 0) >>>>>> +#define V4L2_LENS_STATUS_REACHED (1 << 1) >>>>>> +#define V4L2_LENS_STATUS_FAILED (1 << 2) >>>>>> + >>>>>> +#define V4L2_CID_FOCUS_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 37) >>>>>> +#define V4L2_CID_ZOOM_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 38) >>>>>> + >>>>>> /* FM Modulator class control IDs */ >>>>>> >>>>>> #define V4L2_CID_FM_TX_CLASS_BASE (V4L2_CTRL_CLASS_FM_TX | 0x900) >>>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h >>>>>> index 17a9b975177a..256c21c68720 100644 >>>>>> --- a/include/uapi/linux/videodev2.h >>>>>> +++ b/include/uapi/linux/videodev2.h >>>>>> @@ -1888,6 +1888,8 @@ enum v4l2_ctrl_type { >>>>>> V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272, >>>>>> V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273, >>>>>> V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274, >>>>>> + >>>>>> + V4L2_CTRL_TYPE_LENS_STATUS = 0x0300, >>>>>> }; >>>>>> >>>>>> /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */ >>>>>> >>>>>> -- >>>>>> 2.30.2 >>>>>>
Hi Sakari, On 4/12/23 17:12, Sakari Ailus wrote: > Hi Dave, Michael, > > On Wed, Apr 12, 2023 at 02:55:56PM +0100, Dave Stevenson wrote: >>>> If the ranges aren't updated, where should that out-of-range lens >>>> movement leave the lens? >>> >>> This is up to the hardware controller, but I would guess it typically >>> stops one step before disaster. Wherever that may be, the error >>> condition and the current position can be read out via this new STATUS >>> control. >>> >>> Does this sound good so far? >> >> Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or >> Laurent), and I'm just expressing my views based on the lenses I've >> encountered. >> All of my lenses have a single drive for focus, a single drive for >> zoom, and where there are multiple elements they are all connected >> mechanically. Your setup sounds far more complex and is likely to need >> a more extensive driver, but it'd be nice to not unnecessarily >> overcomplicate the interface. > > Could we also have a driver that uses these new controls? If you are referring to the driver for our custom lens controller, then I have to say that it is under development and simply not ready for release yet. Also, the decision has not yet been made whether or not this will be an open-source driver. A different approach could be the adaptation of the vimc-lens driver, which currently only supports FOCUS_ABSOLUTE. But this would raise several implementation questions and at least for me this would be a nontrivial task. Is it required to have a driver for this interface (in the sense that the patches cannot be accepted otherwise)? > The controls themselves appear reasonable to me as well. I guess there are > changes to be made based on the discussion? I'd summarize that whether or not the status controls are compound controls of the type V4L2_CTRL_TYPE_LENS_STATUS is the open question. As a potential follow-up question I recently asked myself if the struct v4l2_ctrl_lens_status should contain trailing reserved bytes for future extension (no idea, though, what this could be). Alternatively, we could come up with "V4L2_CID_FOCUS_CURRENT (integer)" for the current position and "V4L2_CID_FOCUS_STATUS (bitmask)" (and add further controls when they are needed. Here, we lose atomicity but maybe this can be ignored. One could assume that all relevant controls are read out with a single ioctl which provides at least some level of atomicity. Any comments and/or recommendations to this open question would be much appreciated. Other review comments will be incorporated in the next iteration of this series as well, but they are quite straightforward. Best regards, Michael
Hi Michael, On Mon, Apr 17, 2023 at 02:38:20PM +0200, Michael Riesch wrote: > Hi Sakari, > > On 4/12/23 17:12, Sakari Ailus wrote: > > Hi Dave, Michael, > > > > On Wed, Apr 12, 2023 at 02:55:56PM +0100, Dave Stevenson wrote: > >>>> If the ranges aren't updated, where should that out-of-range lens > >>>> movement leave the lens? > >>> > >>> This is up to the hardware controller, but I would guess it typically > >>> stops one step before disaster. Wherever that may be, the error > >>> condition and the current position can be read out via this new STATUS > >>> control. > >>> > >>> Does this sound good so far? > >> > >> Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or > >> Laurent), and I'm just expressing my views based on the lenses I've > >> encountered. > >> All of my lenses have a single drive for focus, a single drive for > >> zoom, and where there are multiple elements they are all connected > >> mechanically. Your setup sounds far more complex and is likely to need > >> a more extensive driver, but it'd be nice to not unnecessarily > >> overcomplicate the interface. > > > > Could we also have a driver that uses these new controls? > > If you are referring to the driver for our custom lens controller, then > I have to say that it is under development and simply not ready for > release yet. Also, the decision has not yet been made whether or not > this will be an open-source driver. > > A different approach could be the adaptation of the vimc-lens driver, > which currently only supports FOCUS_ABSOLUTE. But this would raise > several implementation questions and at least for me this would be a > nontrivial task. > > Is it required to have a driver for this interface (in the sense that > the patches cannot be accepted otherwise)? That has been traditionally required, and a virtual driver isn't usually considered enough. There are at least two reasons for this. The first one being that if the driver isn't reviewable and targetting upstream it may be difficult to figure out whether the interface changes are the right ones for that driver. This is perhaps a lesser concern here. Secondly, there is also unwillingness to add interface elements that might never be supported by the kernel itself --- this is effectively just dead code. Also cc Hans and Laurent. > > > The controls themselves appear reasonable to me as well. I guess there are > > changes to be made based on the discussion? > > I'd summarize that whether or not the status controls are compound > controls of the type V4L2_CTRL_TYPE_LENS_STATUS is the open question. > > As a potential follow-up question I recently asked myself if the struct > v4l2_ctrl_lens_status should contain trailing reserved bytes for future > extension (no idea, though, what this could be). > > Alternatively, we could come up with "V4L2_CID_FOCUS_CURRENT (integer)" > for the current position and "V4L2_CID_FOCUS_STATUS (bitmask)" (and add > further controls when they are needed. Here, we lose atomicity but maybe > this can be ignored. One could assume that all relevant controls are > read out with a single ioctl which provides at least some level of > atomicity. There might be something that could be done in the control framework to address this. But it's not something that can be expected to happen soon. I'd perhaps keep them separate, not to make it a compound control just for the access reason. But I certainly don't have a strong opinion about it. > > Any comments and/or recommendations to this open question would be much > appreciated. > > Other review comments will be incorporated in the next iteration of this > series as well, but they are quite straightforward.
Hi Sakari, On 4/19/23 11:01, Sakari Ailus wrote: > Hi Michael, > > On Mon, Apr 17, 2023 at 02:38:20PM +0200, Michael Riesch wrote: >> Hi Sakari, >> >> On 4/12/23 17:12, Sakari Ailus wrote: >>> Hi Dave, Michael, >>> >>> On Wed, Apr 12, 2023 at 02:55:56PM +0100, Dave Stevenson wrote: >>>>>> If the ranges aren't updated, where should that out-of-range lens >>>>>> movement leave the lens? >>>>> >>>>> This is up to the hardware controller, but I would guess it typically >>>>> stops one step before disaster. Wherever that may be, the error >>>>> condition and the current position can be read out via this new STATUS >>>>> control. >>>>> >>>>> Does this sound good so far? >>>> >>>> Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or >>>> Laurent), and I'm just expressing my views based on the lenses I've >>>> encountered. >>>> All of my lenses have a single drive for focus, a single drive for >>>> zoom, and where there are multiple elements they are all connected >>>> mechanically. Your setup sounds far more complex and is likely to need >>>> a more extensive driver, but it'd be nice to not unnecessarily >>>> overcomplicate the interface. >>> >>> Could we also have a driver that uses these new controls? >> >> If you are referring to the driver for our custom lens controller, then >> I have to say that it is under development and simply not ready for >> release yet. Also, the decision has not yet been made whether or not >> this will be an open-source driver. >> >> A different approach could be the adaptation of the vimc-lens driver, >> which currently only supports FOCUS_ABSOLUTE. But this would raise >> several implementation questions and at least for me this would be a >> nontrivial task. >> >> Is it required to have a driver for this interface (in the sense that >> the patches cannot be accepted otherwise)? > > That has been traditionally required, and a virtual driver isn't usually > considered enough. There are at least two reasons for this. The first one > being that if the driver isn't reviewable and targetting upstream it may be > difficult to figure out whether the interface changes are the right ones > for that driver. This is perhaps a lesser concern here. Secondly, there is > also unwillingness to add interface elements that might never be supported > by the kernel itself --- this is effectively just dead code. > > Also cc Hans and Laurent. I understand your concerns. Cc: Alexander and Dieter We aim to be an open-source friendly company. If you are OK with us submitting a driver that targets very custom hardware that is only available in integrated form in our products (and not, for instance, available for sale as a standalone device), then we are prepared to submit the driver sources for consideration for inclusion in mainline Linux. Would this be acceptable? As I already stated above, it will take us some time to prepare everything in a form that is suitable for submission. Now should I submit the next iteration(s) of the series at hand as RFC or as regular patch series? >>> The controls themselves appear reasonable to me as well. I guess there are >>> changes to be made based on the discussion? >> >> I'd summarize that whether or not the status controls are compound >> controls of the type V4L2_CTRL_TYPE_LENS_STATUS is the open question. >> >> As a potential follow-up question I recently asked myself if the struct >> v4l2_ctrl_lens_status should contain trailing reserved bytes for future >> extension (no idea, though, what this could be). >> >> Alternatively, we could come up with "V4L2_CID_FOCUS_CURRENT (integer)" >> for the current position and "V4L2_CID_FOCUS_STATUS (bitmask)" (and add >> further controls when they are needed. Here, we lose atomicity but maybe >> this can be ignored. One could assume that all relevant controls are >> read out with a single ioctl which provides at least some level of >> atomicity. > > There might be something that could be done in the control framework to > address this. But it's not something that can be expected to happen soon. > > I'd perhaps keep them separate, not to make it a compound control just for > the access reason. But I certainly don't have a strong opinion about it. After some further considerations, and following Dave's and your comments, I'll keep them separate. Discussion to be continued with v2. Best regards, Michael > >> >> Any comments and/or recommendations to this open question would be much >> appreciated. >> >> Other review comments will be incorporated in the next iteration of this >> series as well, but they are quite straightforward. >
Hi Michael On Wed, 19 Apr 2023 at 12:25, Michael Riesch <michael.riesch@wolfvision.net> wrote: > > Hi Sakari, > > On 4/19/23 11:01, Sakari Ailus wrote: > > Hi Michael, > > > > On Mon, Apr 17, 2023 at 02:38:20PM +0200, Michael Riesch wrote: > >> Hi Sakari, > >> > >> On 4/12/23 17:12, Sakari Ailus wrote: > >>> Hi Dave, Michael, > >>> > >>> On Wed, Apr 12, 2023 at 02:55:56PM +0100, Dave Stevenson wrote: > >>>>>> If the ranges aren't updated, where should that out-of-range lens > >>>>>> movement leave the lens? > >>>>> > >>>>> This is up to the hardware controller, but I would guess it typically > >>>>> stops one step before disaster. Wherever that may be, the error > >>>>> condition and the current position can be read out via this new STATUS > >>>>> control. > >>>>> > >>>>> Does this sound good so far? > >>>> > >>>> Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or > >>>> Laurent), and I'm just expressing my views based on the lenses I've > >>>> encountered. > >>>> All of my lenses have a single drive for focus, a single drive for > >>>> zoom, and where there are multiple elements they are all connected > >>>> mechanically. Your setup sounds far more complex and is likely to need > >>>> a more extensive driver, but it'd be nice to not unnecessarily > >>>> overcomplicate the interface. > >>> > >>> Could we also have a driver that uses these new controls? > >> > >> If you are referring to the driver for our custom lens controller, then > >> I have to say that it is under development and simply not ready for > >> release yet. Also, the decision has not yet been made whether or not > >> this will be an open-source driver. > >> > >> A different approach could be the adaptation of the vimc-lens driver, > >> which currently only supports FOCUS_ABSOLUTE. But this would raise > >> several implementation questions and at least for me this would be a > >> nontrivial task. > >> > >> Is it required to have a driver for this interface (in the sense that > >> the patches cannot be accepted otherwise)? > > > > That has been traditionally required, and a virtual driver isn't usually > > considered enough. There are at least two reasons for this. The first one > > being that if the driver isn't reviewable and targetting upstream it may be > > difficult to figure out whether the interface changes are the right ones > > for that driver. This is perhaps a lesser concern here. Secondly, there is > > also unwillingness to add interface elements that might never be supported > > by the kernel itself --- this is effectively just dead code. > > > > Also cc Hans and Laurent. > > I understand your concerns. Cc: Alexander and Dieter > > We aim to be an open-source friendly company. If you are OK with us > submitting a driver that targets very custom hardware that is only > available in integrated form in our products (and not, for instance, > available for sale as a standalone device), then we are prepared to > submit the driver sources for consideration for inclusion in mainline > Linux. Would this be acceptable? My plan with the motor drive is far simpler with a Pi RP2040 microcontroller on I2C running an ADC for the potentiometers, PWM for motor control, and a PID loop driving it. The MCU code will be open source. It is a spare time project rather than work, so I can't guarantee timescales, but I'll see if I can find some time to progress it. (It could all be driven from the kernel with ADC and PWM, but I don't see such a driver being accepted. Offloading it to an MCU seems to be the easier option). > As I already stated above, it will take us some time to prepare > everything in a form that is suitable for submission. Now should I > submit the next iteration(s) of the series at hand as RFC or as regular > patch series? > > >>> The controls themselves appear reasonable to me as well. I guess there are > >>> changes to be made based on the discussion? > >> > >> I'd summarize that whether or not the status controls are compound > >> controls of the type V4L2_CTRL_TYPE_LENS_STATUS is the open question. > >> > >> As a potential follow-up question I recently asked myself if the struct > >> v4l2_ctrl_lens_status should contain trailing reserved bytes for future > >> extension (no idea, though, what this could be). > >> > >> Alternatively, we could come up with "V4L2_CID_FOCUS_CURRENT (integer)" > >> for the current position and "V4L2_CID_FOCUS_STATUS (bitmask)" (and add > >> further controls when they are needed. Here, we lose atomicity but maybe > >> this can be ignored. One could assume that all relevant controls are > >> read out with a single ioctl which provides at least some level of > >> atomicity. VIDIOC_G_EXT_CTRLS should allow you to read multiple controls in one ioctl call. It would be multiple calls to your g_volatile_ctrl handler, so potentially multiple I2C calls to your lens controller. There is the option of cluster controls, but I think that is largely only applicable for setting controls rather than reading them. Dave > > There might be something that could be done in the control framework to > > address this. But it's not something that can be expected to happen soon. > > > > I'd perhaps keep them separate, not to make it a compound control just for > > the access reason. But I certainly don't have a strong opinion about it. > > After some further considerations, and following Dave's and your > comments, I'll keep them separate. > > Discussion to be continued with v2. > > Best regards, > Michael > > > > >> > >> Any comments and/or recommendations to this open question would be much > >> appreciated. > >> > >> Other review comments will be incorporated in the next iteration of this > >> series as well, but they are quite straightforward. > >
Hi Michael, On Wed, Apr 19, 2023 at 01:24:58PM +0200, Michael Riesch wrote: > Hi Sakari, > > On 4/19/23 11:01, Sakari Ailus wrote: > > Hi Michael, > > > > On Mon, Apr 17, 2023 at 02:38:20PM +0200, Michael Riesch wrote: > >> Hi Sakari, > >> > >> On 4/12/23 17:12, Sakari Ailus wrote: > >>> Hi Dave, Michael, > >>> > >>> On Wed, Apr 12, 2023 at 02:55:56PM +0100, Dave Stevenson wrote: > >>>>>> If the ranges aren't updated, where should that out-of-range lens > >>>>>> movement leave the lens? > >>>>> > >>>>> This is up to the hardware controller, but I would guess it typically > >>>>> stops one step before disaster. Wherever that may be, the error > >>>>> condition and the current position can be read out via this new STATUS > >>>>> control. > >>>>> > >>>>> Does this sound good so far? > >>>> > >>>> Sounds reasonable, but I'm not the gatekeeper (that would be Sakari or > >>>> Laurent), and I'm just expressing my views based on the lenses I've > >>>> encountered. > >>>> All of my lenses have a single drive for focus, a single drive for > >>>> zoom, and where there are multiple elements they are all connected > >>>> mechanically. Your setup sounds far more complex and is likely to need > >>>> a more extensive driver, but it'd be nice to not unnecessarily > >>>> overcomplicate the interface. > >>> > >>> Could we also have a driver that uses these new controls? > >> > >> If you are referring to the driver for our custom lens controller, then > >> I have to say that it is under development and simply not ready for > >> release yet. Also, the decision has not yet been made whether or not > >> this will be an open-source driver. > >> > >> A different approach could be the adaptation of the vimc-lens driver, > >> which currently only supports FOCUS_ABSOLUTE. But this would raise > >> several implementation questions and at least for me this would be a > >> nontrivial task. > >> > >> Is it required to have a driver for this interface (in the sense that > >> the patches cannot be accepted otherwise)? > > > > That has been traditionally required, and a virtual driver isn't usually > > considered enough. There are at least two reasons for this. The first one > > being that if the driver isn't reviewable and targetting upstream it may be > > difficult to figure out whether the interface changes are the right ones > > for that driver. This is perhaps a lesser concern here. Secondly, there is > > also unwillingness to add interface elements that might never be supported > > by the kernel itself --- this is effectively just dead code. > > > > Also cc Hans and Laurent. > > I understand your concerns. Cc: Alexander and Dieter > > We aim to be an open-source friendly company. If you are OK with us > submitting a driver that targets very custom hardware that is only > available in integrated form in our products (and not, for instance, > available for sale as a standalone device), then we are prepared to > submit the driver sources for consideration for inclusion in mainline > Linux. Would this be acceptable? How easily can you run your own kernel on this thing? A somewhat close comparison point to this would be mobile phones that come with raw camera sensors that often are found nowhere else except on that very phone model. These are not always very easy to use actually. It is also true that these sensors _could_ be found elsewhere and sometimes are. I wonder what others think. > > As I already stated above, it will take us some time to prepare > everything in a form that is suitable for submission. Now should I > submit the next iteration(s) of the series at hand as RFC or as regular > patch series? RFC perhaps, unless it comes with a driver? It doesn't necessarily matter much in the end. Sometimes what was labelled as RFC gets merged as-is, at other times there are 20 versions of what was labelled as PATCH to begin with. > > >>> The controls themselves appear reasonable to me as well. I guess there are > >>> changes to be made based on the discussion? > >> > >> I'd summarize that whether or not the status controls are compound > >> controls of the type V4L2_CTRL_TYPE_LENS_STATUS is the open question. > >> > >> As a potential follow-up question I recently asked myself if the struct > >> v4l2_ctrl_lens_status should contain trailing reserved bytes for future > >> extension (no idea, though, what this could be). > >> > >> Alternatively, we could come up with "V4L2_CID_FOCUS_CURRENT (integer)" > >> for the current position and "V4L2_CID_FOCUS_STATUS (bitmask)" (and add > >> further controls when they are needed. Here, we lose atomicity but maybe > >> this can be ignored. One could assume that all relevant controls are > >> read out with a single ioctl which provides at least some level of > >> atomicity. > > > > There might be something that could be done in the control framework to > > address this. But it's not something that can be expected to happen soon. > > > > I'd perhaps keep them separate, not to make it a compound control just for > > the access reason. But I certainly don't have a strong opinion about it. > > After some further considerations, and following Dave's and your > comments, I'll keep them separate. > > Discussion to be continued with v2.
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst index daa4f40869f8..3a270bc63f1a 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst @@ -149,6 +149,30 @@ enum v4l2_exposure_metering - to the camera, negative values towards infinity. This is a write-only control. +``V4L2_CID_FOCUS_STATUS (struct)`` + The current status of the focus lens group. This is a read-only control. + The returned data structure contains the current position and movement + indication flags. The unit of the current position is undefined. Positive + values move the focus closer to the camera, negative values towards + infinity. The possible flags are described in the table below. + +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + + * - ``V4L2_LENS_STATUS_IDLE`` + - Focus lens group is at rest. + * - ``V4L2_LENS_STATUS_BUSY`` + - Focus lens group is moving. + * - ``V4L2_LENS_STATUS_REACHED`` + - Focus lens group has reached its target position. + * - ``V4L2_LENS_STATUS_FAILED`` + - Focus lens group has failed to reach its target position. The driver + will not transition from this state until another action is performed + by an application. + ``V4L2_CID_FOCUS_AUTO (boolean)`` Enables continuous automatic focus adjustments. The effect of manual focus adjustments while this feature is enabled is undefined, @@ -239,6 +263,30 @@ enum v4l2_auto_focus_range - movement. A negative value moves the zoom lens group towards the wide-angle direction. The zoom speed unit is driver-specific. +``V4L2_CID_ZOOM_STATUS (struct)`` + The current status of the zoom lens group. This is a read-only control. + The returned data structure contains the current position and movement + indication flags. The unit of the current position is driver-specific and + its value should be a positive integer. The possible flags are described + in the table below. + +.. tabularcolumns:: |p{6.8cm}|p{10.7cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + + * - ``V4L2_LENS_STATUS_IDLE`` + - Zoom lens group is at rest. + * - ``V4L2_LENS_STATUS_BUSY`` + - Zoom lens group is moving. + * - ``V4L2_LENS_STATUS_REACHED`` + - Zoom lens group has reached its target position. + * - ``V4L2_LENS_STATUS_FAILED`` + - Zoom lens group has failed to reach its target position. The driver will + not transition from this state until another action is performed by an + application. + ``V4L2_CID_IRIS_ABSOLUTE (integer)`` This control sets the camera's aperture to the specified value. The unit is undefined. Larger values open the iris wider, smaller values diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c index 29169170880a..f6ad30f311c5 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c @@ -350,6 +350,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl) case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS: pr_cont("HEVC_DECODE_PARAMS"); break; + case V4L2_CTRL_TYPE_LENS_STATUS: + pr_cont("LENS_STATUS"); + break; default: pr_cont("unknown type %d", ctrl->type); break; @@ -918,6 +921,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx, return -EINVAL; break; + case V4L2_CTRL_TYPE_LENS_STATUS: + break; + default: return -EINVAL; } @@ -1605,6 +1611,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, case V4L2_CTRL_TYPE_AREA: elem_size = sizeof(struct v4l2_area); break; + case V4L2_CTRL_TYPE_LENS_STATUS: + elem_size = sizeof(struct v4l2_ctrl_lens_status); + break; default: if (type < V4L2_CTRL_COMPOUND_TYPES) elem_size = sizeof(s32); diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c index 564fedee2c88..9b26a3aa9e9c 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c @@ -1044,6 +1044,8 @@ const char *v4l2_ctrl_get_name(u32 id) case V4L2_CID_CAMERA_ORIENTATION: return "Camera Orientation"; case V4L2_CID_CAMERA_SENSOR_ROTATION: return "Camera Sensor Rotation"; case V4L2_CID_HDR_SENSOR_MODE: return "HDR Sensor Mode"; + case V4L2_CID_FOCUS_STATUS: return "Focus, Status"; + case V4L2_CID_ZOOM_STATUS: return "Zoom, Status"; /* FM Radio Modulator controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */ @@ -1593,6 +1595,11 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, *flags |= V4L2_CTRL_FLAG_WRITE_ONLY | V4L2_CTRL_FLAG_EXECUTE_ON_WRITE; break; + case V4L2_CID_FOCUS_STATUS: + case V4L2_CID_ZOOM_STATUS: + *type = V4L2_CTRL_TYPE_LENS_STATUS; + *flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_VOLATILE; + break; case V4L2_CID_FLASH_STROBE_STATUS: case V4L2_CID_AUTO_FOCUS_STATUS: case V4L2_CID_FLASH_READY: diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h index e59d9a234631..f7273ffc20c9 100644 --- a/include/media/v4l2-ctrls.h +++ b/include/media/v4l2-ctrls.h @@ -52,6 +52,7 @@ struct video_device; * @p_hdr10_cll: Pointer to an HDR10 Content Light Level structure. * @p_hdr10_mastering: Pointer to an HDR10 Mastering Display structure. * @p_area: Pointer to an area. + * @p_lens_status: Pointer to a lens status structure. * @p: Pointer to a compound value. * @p_const: Pointer to a constant compound value. */ @@ -81,6 +82,7 @@ union v4l2_ctrl_ptr { struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll; struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering; struct v4l2_area *p_area; + struct v4l2_ctrl_lens_status *p_lens_status; void *p; const void *p_const; }; diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h index 5e80daa4ffe0..8b037467ba9a 100644 --- a/include/uapi/linux/v4l2-controls.h +++ b/include/uapi/linux/v4l2-controls.h @@ -993,6 +993,19 @@ enum v4l2_auto_focus_range { #define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36) +struct v4l2_ctrl_lens_status { + __u32 flags; + __s32 current_position; +}; + +#define V4L2_LENS_STATUS_IDLE (0 << 0) +#define V4L2_LENS_STATUS_BUSY (1 << 0) +#define V4L2_LENS_STATUS_REACHED (1 << 1) +#define V4L2_LENS_STATUS_FAILED (1 << 2) + +#define V4L2_CID_FOCUS_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 37) +#define V4L2_CID_ZOOM_STATUS (V4L2_CID_CAMERA_CLASS_BASE + 38) + /* FM Modulator class control IDs */ #define V4L2_CID_FM_TX_CLASS_BASE (V4L2_CTRL_CLASS_FM_TX | 0x900) diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 17a9b975177a..256c21c68720 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -1888,6 +1888,8 @@ enum v4l2_ctrl_type { V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272, V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273, V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274, + + V4L2_CTRL_TYPE_LENS_STATUS = 0x0300, }; /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */