[{"id":31021,"web_url":"https://patchwork.libcamera.org/comment/31021/","msgid":"","date":"2024-08-31T14:09:03","subject":"Re: [PATCH v3 2/3] libcamera: Add face detection controls","submitter":{"id":143,"url":"https://patchwork.libcamera.org/api/people/143/","name":"Jacopo Mondi","email":"jacopo.mondi@ideasonboard.com"},"content":"Hi Harvey\nOn Fri, Aug 30, 2024 at 09:00:20PM GMT, Harvey Yang wrote:\n> From: Yudhistira Erlandinata \n>\n> Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,\n> and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting\n> FaceDetectFaceLandmark.\n>\n> Signed-off-by: Yudhistira Erlandinata \n> Co-developed-by: becker hsieh \n> Co-developed-by: Harvey Yang \n> ---\n> include/libcamera/controls.h | 6 +++\n> src/libcamera/control_ids_draft.yaml | 63 ++++++++++++++++++++++++++++\n> src/libcamera/controls.cpp | 6 +++\n> 3 files changed, 75 insertions(+)\n>\n> diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h\n> index 7c2bb2872..bf1b8609c 100644\n> --- a/include/libcamera/controls.h\n> +++ b/include/libcamera/controls.h\n> @@ -34,6 +34,7 @@ enum ControlType {\n> \tControlTypeString,\n> \tControlTypeRectangle,\n> \tControlTypeSize,\n> +\tControlTypePoint,\n> };\n>\n> namespace details {\n> @@ -87,6 +88,11 @@ struct control_type {\n> \tstatic constexpr ControlType value = ControlTypeSize;\n> };\n>\n> +template<>\n> +struct control_type {\n> +\tstatic constexpr ControlType value = ControlTypePoint;\n> +};\n> +\n> template\n> struct control_type> : public control_type> {\n> };\n> diff --git a/src/libcamera/control_ids_draft.yaml b/src/libcamera/control_ids_draft.yaml\n> index 9bef5bf15..23e09bff8 100644\n> --- a/src/libcamera/control_ids_draft.yaml\n> +++ b/src/libcamera/control_ids_draft.yaml\n> @@ -227,4 +227,67 @@ controls:\n> value. All of the custom test patterns will be static (that is the\n> raw image must not vary from frame to frame).\n>\n> + - FaceDetectMode:\n> + type: uint8_t\n> + description: |\n> + Reporting mode of face detection.\n> +\n> + enum:\n> + - name: FaceDetectModeOff\n> + value: 0\n> + description: |\n> + Pipeline should not report face detection result.\n> + - name: FaceDetectModeSimple\n> + value: 1\n> + description: |\n> + Pipeline should at least report faces boundary rectangles, in\n> + metadata FaceDetectFaceRectangles, and confidence score,\n> + in metadata FaceDetectFaceScores, for each of them.\n> + FaceDetectFaceLandmark is optional.\n> +\n> + The number of faces in each list must be the same.\n> +\n> + \\sa FaceDetectFaceRectangles\n> + \\sa FaceDetectFaceScores\n> + \\sa FaceDetectFaceLandmark\n\nCurrently identical to ANDROID_STATISTICS_FACE_DETECT_MODE ?\n\n> +\n> + - FaceDetectFaceRectangles:\n> + type: Rectangle\n> + description: |\n> + Boundary rectangle of the detected faces in format:\n> + [..., xmin_i, ymin_i, xmax_i, ymax_i, ...], where (0,0) is top-left\n> + of active pixel area.\n> + The number of values should be 3 * the number of faces.\n> +\n> + The FaceDetectFaceRectangles control can only be returned in metadata.\n\nCurrently indendtical to ANDROID_STATISTICS_FACE_RECTANGLES ?\n> +\n> + size: [n]\n> +\n> + - FaceDetectFaceScores:\n> + type: uint8_t\n> + description: |\n> + Confidence score of each of the detected faces by face detector.\n> + The range of score is [0, 100].\n> + The FaceDetectFaceScores control can only be returned in metadata.\n> + The number of values should be the number of faces.\n> +\n> + Currently identical to ANDROID_STATISTICS_FACE_SCORES.\n> +\n> + size: [n]\n> +\n> + - FaceDetectFaceLandmark:\n> + type: Point\n> + description: |\n> + Array of human face landmark coordinates in format:\n> + [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],\n> + with i = index of face.\n> + The number of values should be 6 * the number of faces.\n> +\n> + The FaceDetectFaceLandmark control can only be returned in metadata.\n> +\n> + Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.\n> +\n> + size: [n]\n> +\n> +\n\nAdditional empy line\n\n> ...\n> diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp\n> index 11d35321c..ce73ae9d7 100644\n> --- a/src/libcamera/controls.cpp\n> +++ b/src/libcamera/controls.cpp\n> @@ -61,6 +61,7 @@ static constexpr size_t ControlValueSize[] = {\n> \t[ControlTypeString]\t\t= sizeof(char),\n> \t[ControlTypeRectangle]\t\t= sizeof(Rectangle),\n> \t[ControlTypeSize]\t\t= sizeof(Size),\n> +\t[ControlTypePoint]\t\t= sizeof(Point),\n> };\n>\n> } /* namespace */\n> @@ -255,6 +256,11 @@ std::string ControlValue::toString() const\n> \t\t\tstr += value->toString();\n> \t\t\tbreak;\n> \t\t}\n> +\t\tcase ControlTypePoint: {\n> +\t\t\tconst Point *value = reinterpret_cast(data);\n> +\t\t\tstr += value->toString();\n> +\t\t\tbreak;\n> +\t\t}\n> \t\tcase ControlTypeNone:\n> \t\tcase ControlTypeString:\n> \t\t\tbreak;\n> --\n> 2.46.0.469.g59c65b2a67-goog\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B710DC323E\n\tfor ;\n\tSat, 31 Aug 2024 14:09:10 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C036663471;\n\tSat, 31 Aug 2024 16:09:09 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 94B1F61900\n\tfor ;\n\tSat, 31 Aug 2024 16:09:07 +0200 (CEST)","from ideasonboard.com (mob-5-90-57-33.net.vodafone.it [5.90.57.33])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 8CE44667;\n\tSat, 31 Aug 2024 16:07:57 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"cq8wmhgl\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1725113277;\n\tbh=8WTIPBtWJX31zhScaGRiYP4U2/ry4HsqroKC1cR00ZI=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=cq8wmhgltZewzxUUgfuO+V55rJmP6JHbAk7+YW95iAPm4NH97waVBUVXMEi6cjEKL\n\tYZaJdQtLJQmHXS5bF0PAt1k1C9+osjtQjJrdFhfzvwPtuE3CoSckg+a+cYfvAl37WO\n\tU0mZ27xSHT+4hbjh4rE7EAt9x+gTNcGDsg1SMHtY=","Date":"Sat, 31 Aug 2024 16:09:03 +0200","From":"Jacopo Mondi ","To":"Harvey Yang ","Cc":"libcamera-devel@lists.libcamera.org, \n\tYudhistira Erlandinata ,\n\tbecker hsieh ","Subject":"Re: [PATCH v3 2/3] libcamera: Add face detection controls","Message-ID":"","References":"<20240830210313.30691-1-chenghaoyang@chromium.org>\n\t<20240830210313.30691-3-chenghaoyang@chromium.org>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20240830210313.30691-3-chenghaoyang@chromium.org>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":31075,"web_url":"https://patchwork.libcamera.org/comment/31075/","msgid":"","date":"2024-09-03T07:43:10","subject":"Re: [PATCH v3 2/3] libcamera: Add face detection controls","submitter":{"id":117,"url":"https://patchwork.libcamera.org/api/people/117/","name":"Cheng-Hao Yang","email":"chenghaoyang@chromium.org"},"content":"Hi Jacopo,\n\nThe updates will be applied in the next patch.\n\nOn Sat, Aug 31, 2024 at 10:19 PM Jacopo Mondi \nwrote:\n\n> Hi\n>\n> On Fri, Aug 30, 2024 at 09:00:20PM GMT, Harvey Yang wrote:\n> > From: Yudhistira Erlandinata \n> >\n> > Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,\n> > and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting\n> > FaceDetectFaceLandmark.\n> >\n> > Signed-off-by: Yudhistira Erlandinata \n> > Co-developed-by: becker hsieh \n> > Co-developed-by: Harvey Yang \n> > ---\n> > include/libcamera/controls.h | 6 +++\n> > src/libcamera/control_ids_draft.yaml | 63 ++++++++++++++++++++++++++++\n> > src/libcamera/controls.cpp | 6 +++\n> > 3 files changed, 75 insertions(+)\n> >\n> > diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h\n> > index 7c2bb2872..bf1b8609c 100644\n> > --- a/include/libcamera/controls.h\n> > +++ b/include/libcamera/controls.h\n> > @@ -34,6 +34,7 @@ enum ControlType {\n> > ControlTypeString,\n> > ControlTypeRectangle,\n> > ControlTypeSize,\n> > + ControlTypePoint,\n> > };\n> >\n> > namespace details {\n> > @@ -87,6 +88,11 @@ struct control_type {\n> > static constexpr ControlType value = ControlTypeSize;\n> > };\n> >\n> > +template<>\n> > +struct control_type {\n> > + static constexpr ControlType value = ControlTypePoint;\n> > +};\n> > +\n> > template\n> > struct control_type> : public\n> control_type> {\n> > };\n> > diff --git a/src/libcamera/control_ids_draft.yaml\n> b/src/libcamera/control_ids_draft.yaml\n> > index 9bef5bf15..23e09bff8 100644\n> > --- a/src/libcamera/control_ids_draft.yaml\n> > +++ b/src/libcamera/control_ids_draft.yaml\n> > @@ -227,4 +227,67 @@ controls:\n> > value. All of the custom test patterns will be static (that\n> is the\n> > raw image must not vary from frame to frame).\n> >\n> > + - FaceDetectMode:\n> > + type: uint8_t\n> > + description: |\n> > + Reporting mode of face detection.\n> > +\n> > + enum:\n> > + - name: FaceDetectModeOff\n> > + value: 0\n> > + description: |\n> > + Pipeline should not report face detection result.\n> > + - name: FaceDetectModeSimple\n> > + value: 1\n> > + description: |\n> > + Pipeline should at least report faces boundary rectangles,\n> in\n> > + metadata FaceDetectFaceRectangles, and confidence score,\n> > + in metadata FaceDetectFaceScores, for each of them.\n> > + FaceDetectFaceLandmark is optional.\n> > +\n> > + The number of faces in each list must be the same.\n> > +\n> > + \\sa FaceDetectFaceRectangles\n> > + \\sa FaceDetectFaceScores\n> > + \\sa FaceDetectFaceLandmark\n> > +\n> > + - FaceDetectFaceRectangles:\n> > + type: Rectangle\n> > + description: |\n> > + Boundary rectangle of the detected faces in format:\n> > + [..., xmin_i, ymin_i, xmax_i, ymax_i, ...], where (0,0) is\n> top-left\n> > + of active pixel area.\n> > + The number of values should be 3 * the number of faces.\n>\n> Aren't there 4 entries per face ?\n>\nRight, sorry. Updated.\n\n\n>\n> > +\n> > + The FaceDetectFaceRectangles control can only be returned in\n> metadata.\n> > +\n> > + size: [n]\n> > +\n> > + - FaceDetectFaceScores:\n> > + type: uint8_t\n> > + description: |\n> > + Confidence score of each of the detected faces by face detector.\n> > + The range of score is [0, 100].\n> > + The FaceDetectFaceScores control can only be returned in\n> metadata.\n> > + The number of values should be the number of faces.\n> > +\n> > + Currently identical to ANDROID_STATISTICS_FACE_SCORES.\n> > +\n> > + size: [n]\n> > +\n> > + - FaceDetectFaceLandmark:\n> > + type: Point\n> > + description: |\n> > + Array of human face landmark coordinates in format:\n> > + [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],\n> > + with i = index of face.\n> > + The number of values should be 6 * the number of faces.\n>\n> I count 3. but looking at the implementation each landmark is a point\n> in (x, y) form.\n>\nRight, updated to [left_eye_i_x, left_eye_i_y, ...].\n\n\n>\n> > +\n> > + The FaceDetectFaceLandmark control can only be returned in\n> metadata.\n> > +\n> > + Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.\n> > +\n> > + size: [n]\n> > +\n> > +\n> > ...\n> > diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp\n> > index 11d35321c..ce73ae9d7 100644\n> > --- a/src/libcamera/controls.cpp\n> > +++ b/src/libcamera/controls.cpp\n> > @@ -61,6 +61,7 @@ static constexpr size_t ControlValueSize[] = {\n> > [ControlTypeString] = sizeof(char),\n> > [ControlTypeRectangle] = sizeof(Rectangle),\n> > [ControlTypeSize] = sizeof(Size),\n> > + [ControlTypePoint] = sizeof(Point),\n> > };\n> >\n> > } /* namespace */\n> > @@ -255,6 +256,11 @@ std::string ControlValue::toString() const\n> > str += value->toString();\n> > break;\n> > }\n> > + case ControlTypePoint: {\n> > + const Point *value = reinterpret_cast *>(data);\n> > + str += value->toString();\n> > + break;\n> > + }\n> > case ControlTypeNone:\n> > case ControlTypeString:\n> > break;\n> > --\n> > 2.46.0.469.g59c65b2a67-goog\n> >\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 780E2C324C\n\tfor ;\n\tTue, 3 Sep 2024 07:43:26 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 820BB634CB;\n\tTue, 3 Sep 2024 09:43:25 +0200 (CEST)","from mail-lj1-x234.google.com (mail-lj1-x234.google.com\n\t[IPv6:2a00:1450:4864:20::234])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C845A6345D\n\tfor ;\n\tTue, 3 Sep 2024 09:43:22 +0200 (CEST)","by mail-lj1-x234.google.com with SMTP id\n\t38308e7fff4ca-2f3f0bdbcd9so58392571fa.1\n\tfor ;\n\tTue, 03 Sep 2024 00:43:22 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"IPwb1yo+\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=chromium.org; s=google; t=1725349402; x=1725954202;\n\tdarn=lists.libcamera.org; \n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:from:to:cc:subject:date:message-id:reply-to;\n\tbh=cVpAMd56tB5KbJhiFP2hdZ69eYCBJcoW9psN0USo3mk=;\n\tb=IPwb1yo+E06ZJD5+UbeSn0MJVbnQfAH8jYgr9t1tRxPFgUZ2iU+rhTMxwbUwJm6NIj\n\tpmKo96xzyEIsBr58WMVyxqp5zuHXU88MkhOrEBmZjAfC1IhA1T1X1DZXg5sAeA1dBTzG\n\tpxfA5IT2paMyeY6RH8XXCcESl3OBwVGNr1Yqg=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1725349402; x=1725954202;\n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:x-gm-message-state:from:to:cc:subject:date:message-id\n\t:reply-to;\n\tbh=cVpAMd56tB5KbJhiFP2hdZ69eYCBJcoW9psN0USo3mk=;\n\tb=UvGjpIt3G8e2xM8FClGKOwgiS4tL7aypNqCBZ3LaOhb764FVwZQHKIXdCEZ69fwu8t\n\tzf1oPcT0bMvCvUhH8O8/VxIBqR3QoV4j9tBZMB1xGwrFsTWnpuK6vf6nk/9jD2yTDgza\n\tm+27K6Gtp1uXG1RnZ7CuFMKIYgua8i0RchNjJdEvS7ys/275ZNAJsf/ticJgKeVF+hCo\n\tpsr59l968K6O0GlGJR426dS5umfQarxncCFfxMkaUkrAU4Ha+xyItRYj/lT+/E5Lqe7d\n\tf+qJr0Ls+V4EGDy4BSf+6s4VLskffFu/7dJF4Xj7iEBWMBmOOOuMq0Gm3DTmSzoW1Klq\n\tgEmw==","X-Gm-Message-State":"AOJu0Yw+Aznk4E55KGAryExxz7KZNaUoo4AR+jp2JT4m9Fo+a1occ/lc\n\t2OIgAXXcSedfEMiDfukYTw4gQv46A4ZniBumHR9p3puRe4VoRq4PqxR0sco6VIhaG0ZQqwmYsWs\n\tU42VQGYS2fGGZItK4ZrEUQVmoqCplqFmS1miQ","X-Google-Smtp-Source":"AGHT+IFoLdjV+n6Cc67R+Lj02ztzh0+RvJf1e97aN3KNYXA2k6Ra9UcnB+omClrap5EFNtNpLh/Z5NVZj3zAY9IRrmE=","X-Received":"by 2002:a2e:b892:0:b0:2f3:b71a:1e91 with SMTP id\n\t38308e7fff4ca-2f636a14a73mr48788991fa.17.1725349401688;\n\tTue, 03 Sep 2024 00:43:21 -0700 (PDT)","MIME-Version":"1.0","References":"<20240830210313.30691-1-chenghaoyang@chromium.org>\n\t<20240830210313.30691-3-chenghaoyang@chromium.org>\n\t","In-Reply-To":"","From":"Cheng-Hao Yang ","Date":"Tue, 3 Sep 2024 15:43:10 +0800","Message-ID":"","Subject":"Re: [PATCH v3 2/3] libcamera: Add face detection controls","To":"Jacopo Mondi ","Cc":"libcamera-devel@lists.libcamera.org, \n\tYudhistira Erlandinata ,\n\tbecker hsieh ","Content-Type":"multipart/alternative; boundary=\"000000000000338caf062132367b\"","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":31076,"web_url":"https://patchwork.libcamera.org/comment/31076/","msgid":"","date":"2024-09-03T07:59:42","subject":"Re: [PATCH v3 2/3] libcamera: Add face detection controls","submitter":{"id":117,"url":"https://patchwork.libcamera.org/api/people/117/","name":"Cheng-Hao Yang","email":"chenghaoyang@chromium.org"},"content":"Sorry, Jacopo, I regret:\n\nOn Tue, Sep 3, 2024 at 3:43 PM Cheng-Hao Yang \nwrote:\n\n> Hi Jacopo,\n>\n> The updates will be applied in the next patch.\n>\n> On Sat, Aug 31, 2024 at 10:19 PM Jacopo Mondi <\n> jacopo.mondi@ideasonboard.com> wrote:\n>\n>> Hi\n>>\n>> On Fri, Aug 30, 2024 at 09:00:20PM GMT, Harvey Yang wrote:\n>> > From: Yudhistira Erlandinata \n>> >\n>> > Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,\n>> > and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting\n>> > FaceDetectFaceLandmark.\n>> >\n>> > Signed-off-by: Yudhistira Erlandinata \n>> > Co-developed-by: becker hsieh \n>> > Co-developed-by: Harvey Yang \n>> > ---\n>> > include/libcamera/controls.h | 6 +++\n>> > src/libcamera/control_ids_draft.yaml | 63 ++++++++++++++++++++++++++++\n>> > src/libcamera/controls.cpp | 6 +++\n>> > 3 files changed, 75 insertions(+)\n>> >\n>> > diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h\n>> > index 7c2bb2872..bf1b8609c 100644\n>> > --- a/include/libcamera/controls.h\n>> > +++ b/include/libcamera/controls.h\n>> > @@ -34,6 +34,7 @@ enum ControlType {\n>> > ControlTypeString,\n>> > ControlTypeRectangle,\n>> > ControlTypeSize,\n>> > + ControlTypePoint,\n>> > };\n>> >\n>> > namespace details {\n>> > @@ -87,6 +88,11 @@ struct control_type {\n>> > static constexpr ControlType value = ControlTypeSize;\n>> > };\n>> >\n>> > +template<>\n>> > +struct control_type {\n>> > + static constexpr ControlType value = ControlTypePoint;\n>> > +};\n>> > +\n>> > template\n>> > struct control_type> : public\n>> control_type> {\n>> > };\n>> > diff --git a/src/libcamera/control_ids_draft.yaml\n>> b/src/libcamera/control_ids_draft.yaml\n>> > index 9bef5bf15..23e09bff8 100644\n>> > --- a/src/libcamera/control_ids_draft.yaml\n>> > +++ b/src/libcamera/control_ids_draft.yaml\n>> > @@ -227,4 +227,67 @@ controls:\n>> > value. All of the custom test patterns will be static\n>> (that is the\n>> > raw image must not vary from frame to frame).\n>> >\n>> > + - FaceDetectMode:\n>> > + type: uint8_t\n>> > + description: |\n>> > + Reporting mode of face detection.\n>> > +\n>> > + enum:\n>> > + - name: FaceDetectModeOff\n>> > + value: 0\n>> > + description: |\n>> > + Pipeline should not report face detection result.\n>> > + - name: FaceDetectModeSimple\n>> > + value: 1\n>> > + description: |\n>> > + Pipeline should at least report faces boundary rectangles,\n>> in\n>> > + metadata FaceDetectFaceRectangles, and confidence score,\n>> > + in metadata FaceDetectFaceScores, for each of them.\n>> > + FaceDetectFaceLandmark is optional.\n>> > +\n>> > + The number of faces in each list must be the same.\n>> > +\n>> > + \\sa FaceDetectFaceRectangles\n>> > + \\sa FaceDetectFaceScores\n>> > + \\sa FaceDetectFaceLandmark\n>> > +\n>> > + - FaceDetectFaceRectangles:\n>> > + type: Rectangle\n>> > + description: |\n>> > + Boundary rectangle of the detected faces in format:\n>> > + [..., xmin_i, ymin_i, xmax_i, ymax_i, ...], where (0,0) is\n>> top-left\n>> > + of active pixel area.\n>> > + The number of values should be 3 * the number of faces.\n>>\n>> Aren't there 4 entries per face ?\n>>\n> Right, sorry. Updated.\n>\nIt should be 1 * the number of faces, as the unit we use is Rectangle.\nWe don't need to describe the boundary again here as well, as it's\nAndroid's definition.\n\n\n>\n>\n>>\n>> > +\n>> > + The FaceDetectFaceRectangles control can only be returned in\n>> metadata.\n>> > +\n>> > + size: [n]\n>> > +\n>> > + - FaceDetectFaceScores:\n>> > + type: uint8_t\n>> > + description: |\n>> > + Confidence score of each of the detected faces by face\n>> detector.\n>> > + The range of score is [0, 100].\n>> > + The FaceDetectFaceScores control can only be returned in\n>> metadata.\n>> > + The number of values should be the number of faces.\n>> > +\n>> > + Currently identical to ANDROID_STATISTICS_FACE_SCORES.\n>> > +\n>> > + size: [n]\n>> > +\n>> > + - FaceDetectFaceLandmark:\n>> > + type: Point\n>> > + description: |\n>> > + Array of human face landmark coordinates in format:\n>> > + [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],\n>> > + with i = index of face.\n>> > + The number of values should be 6 * the number of faces.\n>>\n>> I count 3. but looking at the implementation each landmark is a point\n>> in (x, y) form.\n>>\n> Right, updated to [left_eye_i_x, left_eye_i_y, ...].\n>\nIt should be 3, and the format remains as\n`[..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...]`.\n\n\n>\n>\n>>\n>> > +\n>> > + The FaceDetectFaceLandmark control can only be returned in\n>> metadata.\n>> > +\n>> > + Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.\n>> > +\n>> > + size: [n]\n>> > +\n>> > +\n>> > ...\n>> > diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp\n>> > index 11d35321c..ce73ae9d7 100644\n>> > --- a/src/libcamera/controls.cpp\n>> > +++ b/src/libcamera/controls.cpp\n>> > @@ -61,6 +61,7 @@ static constexpr size_t ControlValueSize[] = {\n>> > [ControlTypeString] = sizeof(char),\n>> > [ControlTypeRectangle] = sizeof(Rectangle),\n>> > [ControlTypeSize] = sizeof(Size),\n>> > + [ControlTypePoint] = sizeof(Point),\n>> > };\n>> >\n>> > } /* namespace */\n>> > @@ -255,6 +256,11 @@ std::string ControlValue::toString() const\n>> > str += value->toString();\n>> > break;\n>> > }\n>> > + case ControlTypePoint: {\n>> > + const Point *value = reinterpret_cast> *>(data);\n>> > + str += value->toString();\n>> > + break;\n>> > + }\n>> > case ControlTypeNone:\n>> > case ControlTypeString:\n>> > break;\n>> > --\n>> > 2.46.0.469.g59c65b2a67-goog\n>> >\n>>\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 7E30ABF415\n\tfor ;\n\tTue, 3 Sep 2024 07:59:57 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 74391634E4;\n\tTue, 3 Sep 2024 09:59:56 +0200 (CEST)","from mail-lj1-x234.google.com (mail-lj1-x234.google.com\n\t[IPv6:2a00:1450:4864:20::234])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E6B9C6345D\n\tfor ;\n\tTue, 3 Sep 2024 09:59:54 +0200 (CEST)","by mail-lj1-x234.google.com with SMTP id\n\t38308e7fff4ca-2f3fea6a0a9so44140091fa.0\n\tfor ;\n\tTue, 03 Sep 2024 00:59:54 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"LLdf8WPg\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=chromium.org; s=google; t=1725350394; x=1725955194;\n\tdarn=lists.libcamera.org; \n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:from:to:cc:subject:date:message-id:reply-to;\n\tbh=fwZekGkU0QDMGdiBwHaWDA+exHM33oEpEvs6+rb3U7c=;\n\tb=LLdf8WPgB3JEf7BHUb2dIFkev80nWQjHf0fzBDpmcTgOlxygHsAzjljjJg6Z7aEic9\n\t1KrNzYdMN9PviBt0XEi2JYRb7rqmCN2ZJzT8KUe34lALgPozTn/EPaLe/24CcrCcueOA\n\t+bppFLtyZ+Q6XN8yg0Xilc7q83hTZtlNhDnYY=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1725350394; x=1725955194;\n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:x-gm-message-state:from:to:cc:subject:date:message-id\n\t:reply-to;\n\tbh=fwZekGkU0QDMGdiBwHaWDA+exHM33oEpEvs6+rb3U7c=;\n\tb=gC8F8s1H8euPWU8bDr7SCI1e9msNdvQFXx/xKhA1O/Hp4J5InqEernl8T7Ik/Aaob9\n\t+1xPobTHYOrokTKzXi52A7fPMaigebGP2OT2vREUl+q2ioCflpZ3UtJDVtCdh1SGhI2N\n\tPEXrrZg53MHtkzKipY/7oNbkdU7PWVeZYfllxZp29Z/2TRMjZ8iNqJlYMSPbOttE6LRF\n\tkxGFm/eD1L7enRsi9avWDKLHL+nmzeWYDT37o95eXRWWCG6d154QIQ34Vhbr3EhV8SQT\n\tuyeuuYK5kpCNBLxizI6c4MWDi2zQxXpuvj+dU3bf/6L04wbrjUoPgDVQitc5k9/AfaZo\n\t0S2w==","X-Gm-Message-State":"AOJu0YyKmE/WSxhLsLckur6KRZVT6HYNPkNNhkW4JejYuFkN7RHsiqw6\n\tuWdYPsgZBq9sb6zsYjAY7kWyXi/ASgjdq7IvpH1w2Ey+2+aGHgUv6qLL1qNDWH7InT8D/nahv1O\n\tT/Hlk/7duHJLmsXWlQQPJ9ylDB6ErOm7T1XV80+kUgE31faaqHbLk","X-Google-Smtp-Source":"AGHT+IGZyUYUVc15u+xlErvDvlCtg6VEd7cyPwcSpXNHl/0kT0GwVaNFU0L9kNS/1518L+ZswI7ws1rkfoC4587+Qnk=","X-Received":"by 2002:a2e:a585:0:b0:2f5:b59:191c with SMTP id\n\t38308e7fff4ca-2f612ae5af1mr54829451fa.17.1725350393921;\n\tTue, 03 Sep 2024 00:59:53 -0700 (PDT)","MIME-Version":"1.0","References":"<20240830210313.30691-1-chenghaoyang@chromium.org>\n\t<20240830210313.30691-3-chenghaoyang@chromium.org>\n\t\n\t","In-Reply-To":"","From":"Cheng-Hao Yang ","Date":"Tue, 3 Sep 2024 15:59:42 +0800","Message-ID":"","Subject":"Re: [PATCH v3 2/3] libcamera: Add face detection controls","To":"Jacopo Mondi ","Cc":"libcamera-devel@lists.libcamera.org, \n\tYudhistira Erlandinata ,\n\tbecker hsieh ","Content-Type":"multipart/alternative; boundary=\"00000000000057d594062132712f\"","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":31153,"web_url":"https://patchwork.libcamera.org/comment/31153/","msgid":"","date":"2024-09-10T04:52:17","subject":"Re: [PATCH v3 2/3] libcamera: Add face detection controls","submitter":{"id":117,"url":"https://patchwork.libcamera.org/api/people/117/","name":"Cheng-Hao Yang","email":"chenghaoyang@chromium.org"},"content":"Hi Jacopo,\n\nOn Sat, Aug 31, 2024 at 10:09 PM Jacopo Mondi \nwrote:\n\n> Hi Harvey\n> On Fri, Aug 30, 2024 at 09:00:20PM GMT, Harvey Yang wrote:\n> > From: Yudhistira Erlandinata \n> >\n> > Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,\n> > and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting\n> > FaceDetectFaceLandmark.\n> >\n> > Signed-off-by: Yudhistira Erlandinata \n> > Co-developed-by: becker hsieh \n> > Co-developed-by: Harvey Yang \n> > ---\n> > include/libcamera/controls.h | 6 +++\n> > src/libcamera/control_ids_draft.yaml | 63 ++++++++++++++++++++++++++++\n> > src/libcamera/controls.cpp | 6 +++\n> > 3 files changed, 75 insertions(+)\n> >\n> > diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h\n> > index 7c2bb2872..bf1b8609c 100644\n> > --- a/include/libcamera/controls.h\n> > +++ b/include/libcamera/controls.h\n> > @@ -34,6 +34,7 @@ enum ControlType {\n> > ControlTypeString,\n> > ControlTypeRectangle,\n> > ControlTypeSize,\n> > + ControlTypePoint,\n> > };\n> >\n> > namespace details {\n> > @@ -87,6 +88,11 @@ struct control_type {\n> > static constexpr ControlType value = ControlTypeSize;\n> > };\n> >\n> > +template<>\n> > +struct control_type {\n> > + static constexpr ControlType value = ControlTypePoint;\n> > +};\n> > +\n> > template\n> > struct control_type> : public\n> control_type> {\n> > };\n> > diff --git a/src/libcamera/control_ids_draft.yaml\n> b/src/libcamera/control_ids_draft.yaml\n> > index 9bef5bf15..23e09bff8 100644\n> > --- a/src/libcamera/control_ids_draft.yaml\n> > +++ b/src/libcamera/control_ids_draft.yaml\n> > @@ -227,4 +227,67 @@ controls:\n> > value. All of the custom test patterns will be static (that\n> is the\n> > raw image must not vary from frame to frame).\n> >\n> > + - FaceDetectMode:\n> > + type: uint8_t\n> > + description: |\n> > + Reporting mode of face detection.\n> > +\n> > + enum:\n> > + - name: FaceDetectModeOff\n> > + value: 0\n> > + description: |\n> > + Pipeline should not report face detection result.\n> > + - name: FaceDetectModeSimple\n> > + value: 1\n> > + description: |\n> > + Pipeline should at least report faces boundary rectangles,\n> in\n> > + metadata FaceDetectFaceRectangles, and confidence score,\n> > + in metadata FaceDetectFaceScores, for each of them.\n> > + FaceDetectFaceLandmark is optional.\n> > +\n> > + The number of faces in each list must be the same.\n> > +\n> > + \\sa FaceDetectFaceRectangles\n> > + \\sa FaceDetectFaceScores\n> > + \\sa FaceDetectFaceLandmark\n>\n> Currently identical to ANDROID_STATISTICS_FACE_DETECT_MODE ?\n>\nHmm, in this patch we lack the full mode. Let's add the full mode in the\ncontrol ids, without any pipeline handler supporting it.\n\n\n>\n> > +\n> > + - FaceDetectFaceRectangles:\n> > + type: Rectangle\n> > + description: |\n> > + Boundary rectangle of the detected faces in format:\n> > + [..., xmin_i, ymin_i, xmax_i, ymax_i, ...], where (0,0) is\n> top-left\n> > + of active pixel area.\n> > + The number of values should be 3 * the number of faces.\n> > +\n> > + The FaceDetectFaceRectangles control can only be returned in\n> metadata.\n>\n> Currently indendtical to ANDROID_STATISTICS_FACE_RECTANGLES ?\n>\nDone\n\n\n> > +\n> > + size: [n]\n> > +\n> > + - FaceDetectFaceScores:\n> > + type: uint8_t\n> > + description: |\n> > + Confidence score of each of the detected faces by face detector.\n> > + The range of score is [0, 100].\n> > + The FaceDetectFaceScores control can only be returned in\n> metadata.\n> > + The number of values should be the number of faces.\n> > +\n> > + Currently identical to ANDROID_STATISTICS_FACE_SCORES.\n> > +\n> > + size: [n]\n> > +\n> > + - FaceDetectFaceLandmark:\n> > + type: Point\n> > + description: |\n> > + Array of human face landmark coordinates in format:\n> > + [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],\n> > + with i = index of face.\n> > + The number of values should be 6 * the number of faces.\n> > +\n> > + The FaceDetectFaceLandmark control can only be returned in\n> metadata.\n> > +\n> > + Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.\n> > +\n> > + size: [n]\n> > +\n> > +\n>\n> Additional empy line\n>\nRemoved.\n\n\n>\n> > ...\n> > diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp\n> > index 11d35321c..ce73ae9d7 100644\n> > --- a/src/libcamera/controls.cpp\n> > +++ b/src/libcamera/controls.cpp\n> > @@ -61,6 +61,7 @@ static constexpr size_t ControlValueSize[] = {\n> > [ControlTypeString] = sizeof(char),\n> > [ControlTypeRectangle] = sizeof(Rectangle),\n> > [ControlTypeSize] = sizeof(Size),\n> > + [ControlTypePoint] = sizeof(Point),\n> > };\n> >\n> > } /* namespace */\n> > @@ -255,6 +256,11 @@ std::string ControlValue::toString() const\n> > str += value->toString();\n> > break;\n> > }\n> > + case ControlTypePoint: {\n> > + const Point *value = reinterpret_cast *>(data);\n> > + str += value->toString();\n> > + break;\n> > + }\n> > case ControlTypeNone:\n> > case ControlTypeString:\n> > break;\n> > --\n> > 2.46.0.469.g59c65b2a67-goog\n> >\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 69CDFBF415\n\tfor ;\n\tTue, 10 Sep 2024 04:52:30 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 28872634FB;\n\tTue, 10 Sep 2024 06:52:30 +0200 (CEST)","from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com\n\t[IPv6:2a00:1450:4864:20::22e])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 04B2B618F2\n\tfor ;\n\tTue, 10 Sep 2024 06:52:29 +0200 (CEST)","by mail-lj1-x22e.google.com with SMTP id\n\t38308e7fff4ca-2f759688444so30510591fa.1\n\tfor ;\n\tMon, 09 Sep 2024 21:52:28 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"Jgyc/E1a\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=chromium.org; s=google; t=1725943948; x=1726548748;\n\tdarn=lists.libcamera.org; \n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:from:to:cc:subject:date:message-id:reply-to;\n\tbh=A6tl7p6JWAvRL+CdYDopACZlNgjKfyrG1zAsU0Q+T4s=;\n\tb=Jgyc/E1aObn0Xk4apzJHo4Hh3W4X18TnSde1/WQ4o12weTgztYu+iqmUQIF+eq4RhV\n\t3+2NtR/ucgNtV9KQ1IIR3hz997GrFTSRkrJlM/a2hVQLcaOjOFK5/COroZ8Mms0GO6yq\n\t80ppnfqYdTi3MsVviS4GZQvFm23+0qkLfP0T8=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1725943948; x=1726548748;\n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:x-gm-message-state:from:to:cc:subject:date:message-id\n\t:reply-to;\n\tbh=A6tl7p6JWAvRL+CdYDopACZlNgjKfyrG1zAsU0Q+T4s=;\n\tb=BGZ/rLWfdjlym3sqNX2fk+eEIFPs4f3m7AyqiqnzpsOJ/7jeo5MpYWKedjs37VaWIG\n\t1lxROgCIgJXg9XYnAXVX8/PbuXJlGhHB54JFxX/F5ZO/0dDUCaMO/PvlzoOBqLfG4t/1\n\tVS2bUH66TieqKppp9yCZrnpOfh5R60Vl83qP/AJsFvSzNY0PlZsPkbSZz89twiS5V4nB\n\tj+14amJjyPlWT6hu78N/FCG+VvUhNTHzQLBbQ+0Whi1a5K2rKrUQsBbEYRWB/4xv3U/T\n\tE9aG+qwRzvAx8cQN5bRuWvKmH8kEhaarUE3PwS8Rr4ui3eML8bUDyMW3PmFjt5SwNOnh\n\t3jbA==","X-Gm-Message-State":"AOJu0YxIYqn6FK27cRO9hglB9hFvbFQIY2yA0ga0wJsxkPlwqGyHBsKJ\n\tFu0QB/kw6SSyDAwcTFTYKuZuuWgOmqaVT6SiGLwHBb2hQXIc/1n9ln4Z9wRrrA4Bv7P9kpqphbW\n\t8M3dIIF2VLkAC8MnqwHNcaaZT/NUu88ONw5Ejg0Qwl4SjExI=","X-Google-Smtp-Source":"AGHT+IEVZVD8gJ/qnISPG6CTlSTp6HCarfA5CuK8OO+8KeC/bpBCK83oB6HnZNPDtkAJ7gGYEpmexnQBen0tk1A7hII=","X-Received":"by 2002:a2e:a983:0:b0:2ef:1db2:c02a with SMTP id\n\t38308e7fff4ca-2f751eb2464mr107357041fa.6.1725943948025;\n\tMon, 09 Sep 2024 21:52:28 -0700 (PDT)","MIME-Version":"1.0","References":"<20240830210313.30691-1-chenghaoyang@chromium.org>\n\t<20240830210313.30691-3-chenghaoyang@chromium.org>\n\t","In-Reply-To":"","From":"Cheng-Hao Yang ","Date":"Tue, 10 Sep 2024 12:52:17 +0800","Message-ID":"","Subject":"Re: [PATCH v3 2/3] libcamera: Add face detection controls","To":"Jacopo Mondi ","Cc":"libcamera-devel@lists.libcamera.org, \n\tYudhistira Erlandinata ,\n\tbecker hsieh ","Content-Type":"multipart/alternative; boundary=\"000000000000ecba2e0621bca3f4\"","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]