[{"id":31466,"web_url":"https://patchwork.libcamera.org/comment/31466/","msgid":"","date":"2024-09-30T11:14:07","subject":"Re: [PATCH v8 2/3] libcamera: Add face detection controls","submitter":{"id":143,"url":"https://patchwork.libcamera.org/api/people/143/","name":"Jacopo Mondi","email":"jacopo.mondi@ideasonboard.com"},"content":"Hi Harvey\n\nOn Wed, Sep 25, 2024 at 08:12:25AM GMT, Harvey Yang wrote:\n> From: Yudhistira Erlandinata \n>\n> Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,\n> and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting\n> FaceDetectFaceLandmark.\n>\n> Signed-off-by: Yudhistira Erlandinata \n> Co-developed-by: becker hsieh \n> Co-developed-by: Harvey Yang \n> Reviewed-by: Jacopo Mondi \n> ---\n> include/libcamera/controls.h | 6 ++\n> src/libcamera/control_ids_draft.yaml | 88 ++++++++++++++++++++++++++++\n> src/libcamera/controls.cpp | 6 ++\n> 3 files changed, 100 insertions(+)\n>\n> diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h\n> index 7c2bb287..bf1b8609 100644\n> --- a/include/libcamera/controls.h\n> +++ b/include/libcamera/controls.h\n> @@ -34,6 +34,7 @@ enum ControlType {\n> \tControlTypeString,\n> \tControlTypeRectangle,\n> \tControlTypeSize,\n> +\tControlTypePoint,\n> };\n>\n> namespace details {\n> @@ -87,6 +88,11 @@ struct control_type {\n> \tstatic constexpr ControlType value = ControlTypeSize;\n> };\n>\n> +template<>\n> +struct control_type {\n> +\tstatic constexpr ControlType value = ControlTypePoint;\n> +};\n> +\n> template\n> struct control_type> : public control_type> {\n> };\n> diff --git a/src/libcamera/control_ids_draft.yaml b/src/libcamera/control_ids_draft.yaml\n> index 9bef5bf1..6a472b48 100644\n> --- a/src/libcamera/control_ids_draft.yaml\n> +++ b/src/libcamera/control_ids_draft.yaml\n> @@ -227,4 +227,92 @@ controls:\n> value. All of the custom test patterns will be static (that is the\n> raw image must not vary from frame to frame).\n>\n> + - FaceDetectMode:\n> + type: uint8_t\n\nThis is not valid anymore since the introduction of\ne6da224926b0 (\"libcamera: controls: Handle enum values without a cast\")\n\nI'll make it a int32_t and slightly tweak the text descriptions to\nalign them to existing draft controls.\n\nI'll re-send a new version of these patches on top of an upate to the\nRectangle's top-left point documentation.\n\nThanks\n j\n\n> + description: |\n> + Reporting mode of face detection.\n> +\n> + Currently identical to ANDROID_STATISTICS_FACE_DETECT_MODE.\n> +\n> + \\sa FaceDetectFaceRectangles\n> + \\sa FaceDetectFaceScores\n> + \\sa FaceDetectFaceLandmarks\n> + \\sa FaceDetectFaceIds\n> +\n> + enum:\n> + - name: FaceDetectModeOff\n> + value: 0\n> + description: |\n> + Pipeline should not report face detection result.\n> +\n> + - name: FaceDetectModeSimple\n> + value: 1\n> + description: |\n> + Pipeline should at least report FaceDetectFaceRectangles and\n> + FaceDetectFaceScores for each detected face.\n> + FaceDetectFaceLandmarks and FaceDetectFaceIds are optional.\n> +\n> + - name: FaceDetectModeFull\n> + value: 2\n> + description: |\n> + Pipeline should report all face controls, including\n> + FaceDetectFaceRectangles, FaceDetectFaceScores,\n> + FaceDetectFaceLandmarks, and FaceDeteceFaceIds.\n> +\n> + - FaceDetectFaceRectangles:\n> + type: Rectangle\n> + description: |\n> + Boundary rectangles of the detected faces.\n> + The number of values should be the number of faces reported in\n> + FaceDetectFaceRectangles.\n> +\n> + The FaceDetectFaceRectangles control can only be returned in metadata.\n> +\n> + Currently identical to ANDROID_STATISTICS_FACE_RECTANGLES.\n> +\n> + size: [n]\n> +\n> + - FaceDetectFaceScores:\n> + type: uint8_t\n> + description: |\n> + Confidence score of each of the detected faces by face detector.\n> + The range of score is [0, 100].\n> + The FaceDetectFaceScores control can only be returned in metadata.\n> + The number of values should be the number of faces reported in\n> + FaceDetectFaceRectangles.\n> +\n> + Currently identical to ANDROID_STATISTICS_FACE_SCORES.\n> +\n> + size: [n]\n> +\n> + - FaceDetectFaceLandmarks:\n> + type: Point\n> + description: |\n> + Array of human face landmark coordinates in format:\n> + [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],\n> + with i = index of face.\n> + The number of values should be 3 * the number of faces reported in\n> + FaceDetectFaceRectangles.\n> +\n> + The FaceDetectFaceLandmarks control can only be returned in metadata.\n> +\n> + Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.\n> +\n> + size: [n]\n> +\n> + - FaceDetectFaceIds:\n> + type: int32_t\n> + description: |\n> + Each detected face is given a unique ID that is valid for as long as\n> + the face is visible to the camera device. A face that leaves the field\n> + of view and later returns may be assigned a new ID.\n> + The number of values should be the number of faces reported in\n> + FaceDetectFaceRectangles.\n> +\n> + The FaceDetectFaceIds control can only be returned in metadata.\n> +\n> + Currently identical to ANDROID_STATISTICS_FACE_IDS.\n> +\n> + size: [n]\n> +\n> ...\n> diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp\n> index dba74404..c452e607 100644\n> --- a/src/libcamera/controls.cpp\n> +++ b/src/libcamera/controls.cpp\n> @@ -60,6 +60,7 @@ static constexpr size_t ControlValueSize[] = {\n> \t[ControlTypeString]\t\t= sizeof(char),\n> \t[ControlTypeRectangle]\t\t= sizeof(Rectangle),\n> \t[ControlTypeSize]\t\t= sizeof(Size),\n> +\t[ControlTypePoint]\t\t= sizeof(Point),\n> };\n>\n> } /* namespace */\n> @@ -254,6 +255,11 @@ std::string ControlValue::toString() const\n> \t\t\tstr += value->toString();\n> \t\t\tbreak;\n> \t\t}\n> +\t\tcase ControlTypePoint: {\n> +\t\t\tconst Point *value = reinterpret_cast(data);\n> +\t\t\tstr += value->toString();\n> +\t\t\tbreak;\n> +\t\t}\n> \t\tcase ControlTypeNone:\n> \t\tcase ControlTypeString:\n> \t\t\tbreak;\n> --\n> 2.46.0.792.g87dc391469-goog\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A44A5C3257\n\tfor ;\n\tMon, 30 Sep 2024 11:14:15 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9AFB363512;\n\tMon, 30 Sep 2024 13:14:14 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 77CC963502\n\tfor ;\n\tMon, 30 Sep 2024 13:14:12 +0200 (CEST)","from ideasonboard.com (unknown [95.131.45.214])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id E7DBA39F;\n\tMon, 30 Sep 2024 13:12:40 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"AGzOaeDH\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1727694761;\n\tbh=SU4ZUVzeLFmNw5aJ9942Qr/b0kOc1QrGpGZDL/iVLJI=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=AGzOaeDHB+hD7wsHCj1HxjizklVO5cjCV5svbgMK7juqrkr2OlFqocKqKwC9swDD3\n\t+VY8ZSA7B1/Iqle6tLpXoUA8TjYu1RsTg4rshjRmgJIWxk7+ER3aa1QX/uuO8oAO8P\n\toIk2uCV6E8MVTbk6y25doHtW/2LJjC4amNRQlS40=","Date":"Mon, 30 Sep 2024 13:14:07 +0200","From":"Jacopo Mondi ","To":"Harvey Yang ","Cc":"libcamera-devel@lists.libcamera.org, \n\tYudhistira Erlandinata ,\n\tbecker hsieh , \n\tJacopo Mondi ","Subject":"Re: [PATCH v8 2/3] libcamera: Add face detection controls","Message-ID":"","References":"<20240925081551.3265942-1-chenghaoyang@google.com>\n\t<20240925081551.3265942-3-chenghaoyang@google.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20240925081551.3265942-3-chenghaoyang@google.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]