[v3,2/3] libcamera: Add face detection controls
diff mbox series

Message ID 20240830210313.30691-3-chenghaoyang@chromium.org
State Superseded
Headers show
Series
  • Add Face Detection Controls
Related show

Commit Message

Harvey Yang Aug. 30, 2024, 9 p.m. UTC
From: Yudhistira Erlandinata <yerlandinata@chromium.org>

Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,
and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting
FaceDetectFaceLandmark.

Signed-off-by: Yudhistira Erlandinata <yerlandinata@chromium.org>
Co-developed-by: becker hsieh <beckerh@chromium.org>
Co-developed-by: Harvey Yang <chenghaoyang@chromium.org>
---
 include/libcamera/controls.h         |  6 +++
 src/libcamera/control_ids_draft.yaml | 63 ++++++++++++++++++++++++++++
 src/libcamera/controls.cpp           |  6 +++
 3 files changed, 75 insertions(+)

Comments

Jacopo Mondi Aug. 31, 2024, 2:09 p.m. UTC | #1
Hi Harvey
On Fri, Aug 30, 2024 at 09:00:20PM GMT, Harvey Yang wrote:
> From: Yudhistira Erlandinata <yerlandinata@chromium.org>
>
> Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,
> and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting
> FaceDetectFaceLandmark.
>
> Signed-off-by: Yudhistira Erlandinata <yerlandinata@chromium.org>
> Co-developed-by: becker hsieh <beckerh@chromium.org>
> Co-developed-by: Harvey Yang <chenghaoyang@chromium.org>
> ---
>  include/libcamera/controls.h         |  6 +++
>  src/libcamera/control_ids_draft.yaml | 63 ++++++++++++++++++++++++++++
>  src/libcamera/controls.cpp           |  6 +++
>  3 files changed, 75 insertions(+)
>
> diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h
> index 7c2bb2872..bf1b8609c 100644
> --- a/include/libcamera/controls.h
> +++ b/include/libcamera/controls.h
> @@ -34,6 +34,7 @@ enum ControlType {
>  	ControlTypeString,
>  	ControlTypeRectangle,
>  	ControlTypeSize,
> +	ControlTypePoint,
>  };
>
>  namespace details {
> @@ -87,6 +88,11 @@ struct control_type<Size> {
>  	static constexpr ControlType value = ControlTypeSize;
>  };
>
> +template<>
> +struct control_type<Point> {
> +	static constexpr ControlType value = ControlTypePoint;
> +};
> +
>  template<typename T, std::size_t N>
>  struct control_type<Span<T, N>> : public control_type<std::remove_cv_t<T>> {
>  };
> diff --git a/src/libcamera/control_ids_draft.yaml b/src/libcamera/control_ids_draft.yaml
> index 9bef5bf15..23e09bff8 100644
> --- a/src/libcamera/control_ids_draft.yaml
> +++ b/src/libcamera/control_ids_draft.yaml
> @@ -227,4 +227,67 @@ controls:
>              value. All of the custom test patterns will be static (that is the
>              raw image must not vary from frame to frame).
>
> +  - FaceDetectMode:
> +      type: uint8_t
> +      description: |
> +        Reporting mode of face detection.
> +
> +      enum:
> +        - name: FaceDetectModeOff
> +          value: 0
> +          description: |
> +            Pipeline should not report face detection result.
> +        - name: FaceDetectModeSimple
> +          value: 1
> +          description: |
> +            Pipeline should at least report faces boundary rectangles, in
> +            metadata FaceDetectFaceRectangles, and confidence score,
> +            in metadata FaceDetectFaceScores, for each of them.
> +            FaceDetectFaceLandmark is optional.
> +
> +            The number of faces in each list must be the same.
> +
> +            \sa FaceDetectFaceRectangles
> +            \sa FaceDetectFaceScores
> +            \sa FaceDetectFaceLandmark

Currently identical to ANDROID_STATISTICS_FACE_DETECT_MODE ?

> +
> +  - FaceDetectFaceRectangles:
> +      type: Rectangle
> +      description: |
> +        Boundary rectangle of the detected faces in format:
> +        [..., xmin_i, ymin_i, xmax_i, ymax_i, ...], where (0,0) is top-left
> +        of active pixel area.
> +        The number of values should be 3 * the number of faces.
> +
> +        The FaceDetectFaceRectangles control can only be returned in metadata.

Currently indendtical to ANDROID_STATISTICS_FACE_RECTANGLES ?
> +
> +      size: [n]
> +
> +  - FaceDetectFaceScores:
> +      type: uint8_t
> +      description: |
> +        Confidence score of each of the detected faces by face detector.
> +        The range of score is [0, 100].
> +        The FaceDetectFaceScores control can only be returned in metadata.
> +        The number of values should be the number of faces.
> +
> +        Currently identical to ANDROID_STATISTICS_FACE_SCORES.
> +
> +      size: [n]
> +
> +  - FaceDetectFaceLandmark:
> +      type: Point
> +      description: |
> +        Array of human face landmark coordinates in format:
> +        [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],
> +        with i = index of face.
> +        The number of values should be 6 * the number of faces.
> +
> +        The FaceDetectFaceLandmark control can only be returned in metadata.
> +
> +        Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.
> +
> +      size: [n]
> +
> +

Additional empy line

>  ...
> diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp
> index 11d35321c..ce73ae9d7 100644
> --- a/src/libcamera/controls.cpp
> +++ b/src/libcamera/controls.cpp
> @@ -61,6 +61,7 @@ static constexpr size_t ControlValueSize[] = {
>  	[ControlTypeString]		= sizeof(char),
>  	[ControlTypeRectangle]		= sizeof(Rectangle),
>  	[ControlTypeSize]		= sizeof(Size),
> +	[ControlTypePoint]		= sizeof(Point),
>  };
>
>  } /* namespace */
> @@ -255,6 +256,11 @@ std::string ControlValue::toString() const
>  			str += value->toString();
>  			break;
>  		}
> +		case ControlTypePoint: {
> +			const Point *value = reinterpret_cast<const Point *>(data);
> +			str += value->toString();
> +			break;
> +		}
>  		case ControlTypeNone:
>  		case ControlTypeString:
>  			break;
> --
> 2.46.0.469.g59c65b2a67-goog
>
Harvey Yang Sept. 3, 2024, 7:43 a.m. UTC | #2
Hi Jacopo,

The updates will be applied in the next patch.

On Sat, Aug 31, 2024 at 10:19 PM Jacopo Mondi <jacopo.mondi@ideasonboard.com>
wrote:

> Hi
>
> On Fri, Aug 30, 2024 at 09:00:20PM GMT, Harvey Yang wrote:
> > From: Yudhistira Erlandinata <yerlandinata@chromium.org>
> >
> > Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,
> > and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting
> > FaceDetectFaceLandmark.
> >
> > Signed-off-by: Yudhistira Erlandinata <yerlandinata@chromium.org>
> > Co-developed-by: becker hsieh <beckerh@chromium.org>
> > Co-developed-by: Harvey Yang <chenghaoyang@chromium.org>
> > ---
> >  include/libcamera/controls.h         |  6 +++
> >  src/libcamera/control_ids_draft.yaml | 63 ++++++++++++++++++++++++++++
> >  src/libcamera/controls.cpp           |  6 +++
> >  3 files changed, 75 insertions(+)
> >
> > diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h
> > index 7c2bb2872..bf1b8609c 100644
> > --- a/include/libcamera/controls.h
> > +++ b/include/libcamera/controls.h
> > @@ -34,6 +34,7 @@ enum ControlType {
> >       ControlTypeString,
> >       ControlTypeRectangle,
> >       ControlTypeSize,
> > +     ControlTypePoint,
> >  };
> >
> >  namespace details {
> > @@ -87,6 +88,11 @@ struct control_type<Size> {
> >       static constexpr ControlType value = ControlTypeSize;
> >  };
> >
> > +template<>
> > +struct control_type<Point> {
> > +     static constexpr ControlType value = ControlTypePoint;
> > +};
> > +
> >  template<typename T, std::size_t N>
> >  struct control_type<Span<T, N>> : public
> control_type<std::remove_cv_t<T>> {
> >  };
> > diff --git a/src/libcamera/control_ids_draft.yaml
> b/src/libcamera/control_ids_draft.yaml
> > index 9bef5bf15..23e09bff8 100644
> > --- a/src/libcamera/control_ids_draft.yaml
> > +++ b/src/libcamera/control_ids_draft.yaml
> > @@ -227,4 +227,67 @@ controls:
> >              value. All of the custom test patterns will be static (that
> is the
> >              raw image must not vary from frame to frame).
> >
> > +  - FaceDetectMode:
> > +      type: uint8_t
> > +      description: |
> > +        Reporting mode of face detection.
> > +
> > +      enum:
> > +        - name: FaceDetectModeOff
> > +          value: 0
> > +          description: |
> > +            Pipeline should not report face detection result.
> > +        - name: FaceDetectModeSimple
> > +          value: 1
> > +          description: |
> > +            Pipeline should at least report faces boundary rectangles,
> in
> > +            metadata FaceDetectFaceRectangles, and confidence score,
> > +            in metadata FaceDetectFaceScores, for each of them.
> > +            FaceDetectFaceLandmark is optional.
> > +
> > +            The number of faces in each list must be the same.
> > +
> > +            \sa FaceDetectFaceRectangles
> > +            \sa FaceDetectFaceScores
> > +            \sa FaceDetectFaceLandmark
> > +
> > +  - FaceDetectFaceRectangles:
> > +      type: Rectangle
> > +      description: |
> > +        Boundary rectangle of the detected faces in format:
> > +        [..., xmin_i, ymin_i, xmax_i, ymax_i, ...], where (0,0) is
> top-left
> > +        of active pixel area.
> > +        The number of values should be 3 * the number of faces.
>
> Aren't there 4 entries per face ?
>
Right, sorry. Updated.


>
> > +
> > +        The FaceDetectFaceRectangles control can only be returned in
> metadata.
> > +
> > +      size: [n]
> > +
> > +  - FaceDetectFaceScores:
> > +      type: uint8_t
> > +      description: |
> > +        Confidence score of each of the detected faces by face detector.
> > +        The range of score is [0, 100].
> > +        The FaceDetectFaceScores control can only be returned in
> metadata.
> > +        The number of values should be the number of faces.
> > +
> > +        Currently identical to ANDROID_STATISTICS_FACE_SCORES.
> > +
> > +      size: [n]
> > +
> > +  - FaceDetectFaceLandmark:
> > +      type: Point
> > +      description: |
> > +        Array of human face landmark coordinates in format:
> > +        [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],
> > +        with i = index of face.
> > +        The number of values should be 6 * the number of faces.
>
> I count 3. but looking at the implementation each landmark is a point
> in (x, y) form.
>
Right, updated to [left_eye_i_x, left_eye_i_y, ...].


>
> > +
> > +        The FaceDetectFaceLandmark control can only be returned in
> metadata.
> > +
> > +        Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.
> > +
> > +      size: [n]
> > +
> > +
> >  ...
> > diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp
> > index 11d35321c..ce73ae9d7 100644
> > --- a/src/libcamera/controls.cpp
> > +++ b/src/libcamera/controls.cpp
> > @@ -61,6 +61,7 @@ static constexpr size_t ControlValueSize[] = {
> >       [ControlTypeString]             = sizeof(char),
> >       [ControlTypeRectangle]          = sizeof(Rectangle),
> >       [ControlTypeSize]               = sizeof(Size),
> > +     [ControlTypePoint]              = sizeof(Point),
> >  };
> >
> >  } /* namespace */
> > @@ -255,6 +256,11 @@ std::string ControlValue::toString() const
> >                       str += value->toString();
> >                       break;
> >               }
> > +             case ControlTypePoint: {
> > +                     const Point *value = reinterpret_cast<const Point
> *>(data);
> > +                     str += value->toString();
> > +                     break;
> > +             }
> >               case ControlTypeNone:
> >               case ControlTypeString:
> >                       break;
> > --
> > 2.46.0.469.g59c65b2a67-goog
> >
>
Harvey Yang Sept. 3, 2024, 7:59 a.m. UTC | #3
Sorry, Jacopo, I regret:

On Tue, Sep 3, 2024 at 3:43 PM Cheng-Hao Yang <chenghaoyang@chromium.org>
wrote:

> Hi Jacopo,
>
> The updates will be applied in the next patch.
>
> On Sat, Aug 31, 2024 at 10:19 PM Jacopo Mondi <
> jacopo.mondi@ideasonboard.com> wrote:
>
>> Hi
>>
>> On Fri, Aug 30, 2024 at 09:00:20PM GMT, Harvey Yang wrote:
>> > From: Yudhistira Erlandinata <yerlandinata@chromium.org>
>> >
>> > Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,
>> > and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting
>> > FaceDetectFaceLandmark.
>> >
>> > Signed-off-by: Yudhistira Erlandinata <yerlandinata@chromium.org>
>> > Co-developed-by: becker hsieh <beckerh@chromium.org>
>> > Co-developed-by: Harvey Yang <chenghaoyang@chromium.org>
>> > ---
>> >  include/libcamera/controls.h         |  6 +++
>> >  src/libcamera/control_ids_draft.yaml | 63 ++++++++++++++++++++++++++++
>> >  src/libcamera/controls.cpp           |  6 +++
>> >  3 files changed, 75 insertions(+)
>> >
>> > diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h
>> > index 7c2bb2872..bf1b8609c 100644
>> > --- a/include/libcamera/controls.h
>> > +++ b/include/libcamera/controls.h
>> > @@ -34,6 +34,7 @@ enum ControlType {
>> >       ControlTypeString,
>> >       ControlTypeRectangle,
>> >       ControlTypeSize,
>> > +     ControlTypePoint,
>> >  };
>> >
>> >  namespace details {
>> > @@ -87,6 +88,11 @@ struct control_type<Size> {
>> >       static constexpr ControlType value = ControlTypeSize;
>> >  };
>> >
>> > +template<>
>> > +struct control_type<Point> {
>> > +     static constexpr ControlType value = ControlTypePoint;
>> > +};
>> > +
>> >  template<typename T, std::size_t N>
>> >  struct control_type<Span<T, N>> : public
>> control_type<std::remove_cv_t<T>> {
>> >  };
>> > diff --git a/src/libcamera/control_ids_draft.yaml
>> b/src/libcamera/control_ids_draft.yaml
>> > index 9bef5bf15..23e09bff8 100644
>> > --- a/src/libcamera/control_ids_draft.yaml
>> > +++ b/src/libcamera/control_ids_draft.yaml
>> > @@ -227,4 +227,67 @@ controls:
>> >              value. All of the custom test patterns will be static
>> (that is the
>> >              raw image must not vary from frame to frame).
>> >
>> > +  - FaceDetectMode:
>> > +      type: uint8_t
>> > +      description: |
>> > +        Reporting mode of face detection.
>> > +
>> > +      enum:
>> > +        - name: FaceDetectModeOff
>> > +          value: 0
>> > +          description: |
>> > +            Pipeline should not report face detection result.
>> > +        - name: FaceDetectModeSimple
>> > +          value: 1
>> > +          description: |
>> > +            Pipeline should at least report faces boundary rectangles,
>> in
>> > +            metadata FaceDetectFaceRectangles, and confidence score,
>> > +            in metadata FaceDetectFaceScores, for each of them.
>> > +            FaceDetectFaceLandmark is optional.
>> > +
>> > +            The number of faces in each list must be the same.
>> > +
>> > +            \sa FaceDetectFaceRectangles
>> > +            \sa FaceDetectFaceScores
>> > +            \sa FaceDetectFaceLandmark
>> > +
>> > +  - FaceDetectFaceRectangles:
>> > +      type: Rectangle
>> > +      description: |
>> > +        Boundary rectangle of the detected faces in format:
>> > +        [..., xmin_i, ymin_i, xmax_i, ymax_i, ...], where (0,0) is
>> top-left
>> > +        of active pixel area.
>> > +        The number of values should be 3 * the number of faces.
>>
>> Aren't there 4 entries per face ?
>>
> Right, sorry. Updated.
>
It should be 1 * the number of faces, as the unit we use is Rectangle.
We don't need to describe the boundary again here as well, as it's
Android's definition.


>
>
>>
>> > +
>> > +        The FaceDetectFaceRectangles control can only be returned in
>> metadata.
>> > +
>> > +      size: [n]
>> > +
>> > +  - FaceDetectFaceScores:
>> > +      type: uint8_t
>> > +      description: |
>> > +        Confidence score of each of the detected faces by face
>> detector.
>> > +        The range of score is [0, 100].
>> > +        The FaceDetectFaceScores control can only be returned in
>> metadata.
>> > +        The number of values should be the number of faces.
>> > +
>> > +        Currently identical to ANDROID_STATISTICS_FACE_SCORES.
>> > +
>> > +      size: [n]
>> > +
>> > +  - FaceDetectFaceLandmark:
>> > +      type: Point
>> > +      description: |
>> > +        Array of human face landmark coordinates in format:
>> > +        [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],
>> > +        with i = index of face.
>> > +        The number of values should be 6 * the number of faces.
>>
>> I count 3. but looking at the implementation each landmark is a point
>> in (x, y) form.
>>
> Right, updated to [left_eye_i_x, left_eye_i_y, ...].
>
It should be 3, and the format remains as
`[..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...]`.


>
>
>>
>> > +
>> > +        The FaceDetectFaceLandmark control can only be returned in
>> metadata.
>> > +
>> > +        Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.
>> > +
>> > +      size: [n]
>> > +
>> > +
>> >  ...
>> > diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp
>> > index 11d35321c..ce73ae9d7 100644
>> > --- a/src/libcamera/controls.cpp
>> > +++ b/src/libcamera/controls.cpp
>> > @@ -61,6 +61,7 @@ static constexpr size_t ControlValueSize[] = {
>> >       [ControlTypeString]             = sizeof(char),
>> >       [ControlTypeRectangle]          = sizeof(Rectangle),
>> >       [ControlTypeSize]               = sizeof(Size),
>> > +     [ControlTypePoint]              = sizeof(Point),
>> >  };
>> >
>> >  } /* namespace */
>> > @@ -255,6 +256,11 @@ std::string ControlValue::toString() const
>> >                       str += value->toString();
>> >                       break;
>> >               }
>> > +             case ControlTypePoint: {
>> > +                     const Point *value = reinterpret_cast<const Point
>> *>(data);
>> > +                     str += value->toString();
>> > +                     break;
>> > +             }
>> >               case ControlTypeNone:
>> >               case ControlTypeString:
>> >                       break;
>> > --
>> > 2.46.0.469.g59c65b2a67-goog
>> >
>>
>
Harvey Yang Sept. 10, 2024, 4:52 a.m. UTC | #4
Hi Jacopo,

On Sat, Aug 31, 2024 at 10:09 PM Jacopo Mondi <jacopo.mondi@ideasonboard.com>
wrote:

> Hi Harvey
> On Fri, Aug 30, 2024 at 09:00:20PM GMT, Harvey Yang wrote:
> > From: Yudhistira Erlandinata <yerlandinata@chromium.org>
> >
> > Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,
> > and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting
> > FaceDetectFaceLandmark.
> >
> > Signed-off-by: Yudhistira Erlandinata <yerlandinata@chromium.org>
> > Co-developed-by: becker hsieh <beckerh@chromium.org>
> > Co-developed-by: Harvey Yang <chenghaoyang@chromium.org>
> > ---
> >  include/libcamera/controls.h         |  6 +++
> >  src/libcamera/control_ids_draft.yaml | 63 ++++++++++++++++++++++++++++
> >  src/libcamera/controls.cpp           |  6 +++
> >  3 files changed, 75 insertions(+)
> >
> > diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h
> > index 7c2bb2872..bf1b8609c 100644
> > --- a/include/libcamera/controls.h
> > +++ b/include/libcamera/controls.h
> > @@ -34,6 +34,7 @@ enum ControlType {
> >       ControlTypeString,
> >       ControlTypeRectangle,
> >       ControlTypeSize,
> > +     ControlTypePoint,
> >  };
> >
> >  namespace details {
> > @@ -87,6 +88,11 @@ struct control_type<Size> {
> >       static constexpr ControlType value = ControlTypeSize;
> >  };
> >
> > +template<>
> > +struct control_type<Point> {
> > +     static constexpr ControlType value = ControlTypePoint;
> > +};
> > +
> >  template<typename T, std::size_t N>
> >  struct control_type<Span<T, N>> : public
> control_type<std::remove_cv_t<T>> {
> >  };
> > diff --git a/src/libcamera/control_ids_draft.yaml
> b/src/libcamera/control_ids_draft.yaml
> > index 9bef5bf15..23e09bff8 100644
> > --- a/src/libcamera/control_ids_draft.yaml
> > +++ b/src/libcamera/control_ids_draft.yaml
> > @@ -227,4 +227,67 @@ controls:
> >              value. All of the custom test patterns will be static (that
> is the
> >              raw image must not vary from frame to frame).
> >
> > +  - FaceDetectMode:
> > +      type: uint8_t
> > +      description: |
> > +        Reporting mode of face detection.
> > +
> > +      enum:
> > +        - name: FaceDetectModeOff
> > +          value: 0
> > +          description: |
> > +            Pipeline should not report face detection result.
> > +        - name: FaceDetectModeSimple
> > +          value: 1
> > +          description: |
> > +            Pipeline should at least report faces boundary rectangles,
> in
> > +            metadata FaceDetectFaceRectangles, and confidence score,
> > +            in metadata FaceDetectFaceScores, for each of them.
> > +            FaceDetectFaceLandmark is optional.
> > +
> > +            The number of faces in each list must be the same.
> > +
> > +            \sa FaceDetectFaceRectangles
> > +            \sa FaceDetectFaceScores
> > +            \sa FaceDetectFaceLandmark
>
> Currently identical to ANDROID_STATISTICS_FACE_DETECT_MODE ?
>
Hmm, in this patch we lack the full mode. Let's add the full mode in the
control ids, without any pipeline handler supporting it.


>
> > +
> > +  - FaceDetectFaceRectangles:
> > +      type: Rectangle
> > +      description: |
> > +        Boundary rectangle of the detected faces in format:
> > +        [..., xmin_i, ymin_i, xmax_i, ymax_i, ...], where (0,0) is
> top-left
> > +        of active pixel area.
> > +        The number of values should be 3 * the number of faces.
> > +
> > +        The FaceDetectFaceRectangles control can only be returned in
> metadata.
>
> Currently indendtical to ANDROID_STATISTICS_FACE_RECTANGLES ?
>
Done


> > +
> > +      size: [n]
> > +
> > +  - FaceDetectFaceScores:
> > +      type: uint8_t
> > +      description: |
> > +        Confidence score of each of the detected faces by face detector.
> > +        The range of score is [0, 100].
> > +        The FaceDetectFaceScores control can only be returned in
> metadata.
> > +        The number of values should be the number of faces.
> > +
> > +        Currently identical to ANDROID_STATISTICS_FACE_SCORES.
> > +
> > +      size: [n]
> > +
> > +  - FaceDetectFaceLandmark:
> > +      type: Point
> > +      description: |
> > +        Array of human face landmark coordinates in format:
> > +        [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],
> > +        with i = index of face.
> > +        The number of values should be 6 * the number of faces.
> > +
> > +        The FaceDetectFaceLandmark control can only be returned in
> metadata.
> > +
> > +        Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.
> > +
> > +      size: [n]
> > +
> > +
>
> Additional empy line
>
Removed.


>
> >  ...
> > diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp
> > index 11d35321c..ce73ae9d7 100644
> > --- a/src/libcamera/controls.cpp
> > +++ b/src/libcamera/controls.cpp
> > @@ -61,6 +61,7 @@ static constexpr size_t ControlValueSize[] = {
> >       [ControlTypeString]             = sizeof(char),
> >       [ControlTypeRectangle]          = sizeof(Rectangle),
> >       [ControlTypeSize]               = sizeof(Size),
> > +     [ControlTypePoint]              = sizeof(Point),
> >  };
> >
> >  } /* namespace */
> > @@ -255,6 +256,11 @@ std::string ControlValue::toString() const
> >                       str += value->toString();
> >                       break;
> >               }
> > +             case ControlTypePoint: {
> > +                     const Point *value = reinterpret_cast<const Point
> *>(data);
> > +                     str += value->toString();
> > +                     break;
> > +             }
> >               case ControlTypeNone:
> >               case ControlTypeString:
> >                       break;
> > --
> > 2.46.0.469.g59c65b2a67-goog
> >
>

Patch
diff mbox series

diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h
index 7c2bb2872..bf1b8609c 100644
--- a/include/libcamera/controls.h
+++ b/include/libcamera/controls.h
@@ -34,6 +34,7 @@  enum ControlType {
 	ControlTypeString,
 	ControlTypeRectangle,
 	ControlTypeSize,
+	ControlTypePoint,
 };
 
 namespace details {
@@ -87,6 +88,11 @@  struct control_type<Size> {
 	static constexpr ControlType value = ControlTypeSize;
 };
 
+template<>
+struct control_type<Point> {
+	static constexpr ControlType value = ControlTypePoint;
+};
+
 template<typename T, std::size_t N>
 struct control_type<Span<T, N>> : public control_type<std::remove_cv_t<T>> {
 };
diff --git a/src/libcamera/control_ids_draft.yaml b/src/libcamera/control_ids_draft.yaml
index 9bef5bf15..23e09bff8 100644
--- a/src/libcamera/control_ids_draft.yaml
+++ b/src/libcamera/control_ids_draft.yaml
@@ -227,4 +227,67 @@  controls:
             value. All of the custom test patterns will be static (that is the
             raw image must not vary from frame to frame).
 
+  - FaceDetectMode:
+      type: uint8_t
+      description: |
+        Reporting mode of face detection.
+
+      enum:
+        - name: FaceDetectModeOff
+          value: 0
+          description: |
+            Pipeline should not report face detection result.
+        - name: FaceDetectModeSimple
+          value: 1
+          description: |
+            Pipeline should at least report faces boundary rectangles, in
+            metadata FaceDetectFaceRectangles, and confidence score,
+            in metadata FaceDetectFaceScores, for each of them.
+            FaceDetectFaceLandmark is optional.
+
+            The number of faces in each list must be the same.
+
+            \sa FaceDetectFaceRectangles
+            \sa FaceDetectFaceScores
+            \sa FaceDetectFaceLandmark
+
+  - FaceDetectFaceRectangles:
+      type: Rectangle
+      description: |
+        Boundary rectangle of the detected faces in format:
+        [..., xmin_i, ymin_i, xmax_i, ymax_i, ...], where (0,0) is top-left
+        of active pixel area.
+        The number of values should be 3 * the number of faces.
+
+        The FaceDetectFaceRectangles control can only be returned in metadata.
+
+      size: [n]
+
+  - FaceDetectFaceScores:
+      type: uint8_t
+      description: |
+        Confidence score of each of the detected faces by face detector.
+        The range of score is [0, 100].
+        The FaceDetectFaceScores control can only be returned in metadata.
+        The number of values should be the number of faces.
+
+        Currently identical to ANDROID_STATISTICS_FACE_SCORES.
+
+      size: [n]
+
+  - FaceDetectFaceLandmark:
+      type: Point
+      description: |
+        Array of human face landmark coordinates in format:
+        [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],
+        with i = index of face.
+        The number of values should be 6 * the number of faces.
+
+        The FaceDetectFaceLandmark control can only be returned in metadata.
+
+        Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.
+
+      size: [n]
+
+
 ...
diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp
index 11d35321c..ce73ae9d7 100644
--- a/src/libcamera/controls.cpp
+++ b/src/libcamera/controls.cpp
@@ -61,6 +61,7 @@  static constexpr size_t ControlValueSize[] = {
 	[ControlTypeString]		= sizeof(char),
 	[ControlTypeRectangle]		= sizeof(Rectangle),
 	[ControlTypeSize]		= sizeof(Size),
+	[ControlTypePoint]		= sizeof(Point),
 };
 
 } /* namespace */
@@ -255,6 +256,11 @@  std::string ControlValue::toString() const
 			str += value->toString();
 			break;
 		}
+		case ControlTypePoint: {
+			const Point *value = reinterpret_cast<const Point *>(data);
+			str += value->toString();
+			break;
+		}
 		case ControlTypeNone:
 		case ControlTypeString:
 			break;