[{"id":31314,"web_url":"https://patchwork.libcamera.org/comment/31314/","msgid":"","date":"2024-09-23T14:51:52","subject":"Re: [PATCH v6 2/3] libcamera: Add face detection controls","submitter":{"id":148,"url":"https://patchwork.libcamera.org/api/people/148/","name":"Cheng-Hao Yang","email":"chenghaoyang@google.com"},"content":"On Mon, Sep 23, 2024 at 10:49 PM Harvey Yang \nwrote:\n\n> From: Yudhistira Erlandinata \n>\n> Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,\n> and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting\n> FaceDetectFaceLandmark.\n>\n> Signed-off-by: Yudhistira Erlandinata \n> Co-developed-by: becker hsieh \n> Co-developed-by: Harvey Yang \n> ---\n> include/libcamera/controls.h | 6 ++\n> include/libcamera/meson.build | 3 +-\n> src/libcamera/control_ids_android.yaml | 98 ++++++++++++++++++++++++++\n> src/libcamera/control_ranges.yaml | 1 +\n> src/libcamera/controls.cpp | 6 ++\n> 5 files changed, 113 insertions(+), 1 deletion(-)\n> create mode 100644 src/libcamera/control_ids_android.yaml\n>\n> diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h\n> index 7c2bb287..bf1b8609 100644\n> --- a/include/libcamera/controls.h\n> +++ b/include/libcamera/controls.h\n> @@ -34,6 +34,7 @@ enum ControlType {\n> ControlTypeString,\n> ControlTypeRectangle,\n> ControlTypeSize,\n> + ControlTypePoint,\n> };\n>\n> namespace details {\n> @@ -87,6 +88,11 @@ struct control_type {\n> static constexpr ControlType value = ControlTypeSize;\n> };\n>\n> +template<>\n> +struct control_type {\n> + static constexpr ControlType value = ControlTypePoint;\n> +};\n> +\n> template\n> struct control_type> : public\n> control_type> {\n> };\n> diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build\n> index a969a95d..4fff14d2 100644\n> --- a/include/libcamera/meson.build\n> +++ b/include/libcamera/meson.build\n> @@ -34,6 +34,7 @@ libcamera_headers_install_dir = get_option('includedir')\n> / libcamera_include_dir\n>\n> controls_map = {\n> 'controls': {\n> + 'android': 'control_ids_android.yaml',\n> 'draft': 'control_ids_draft.yaml',\n> 'core': 'control_ids_core.yaml',\n> 'rpi/vc4': 'control_ids_rpi.yaml',\n> @@ -55,7 +56,7 @@ foreach mode, entry : controls_map\n> files_list = []\n> input_files = []\n> foreach vendor, header : entry\n> - if vendor != 'core' and vendor != 'draft'\n> + if vendor != 'core' and vendor != 'draft' and vendor != 'android'\n> if vendor not in pipelines\n> continue\n> endif\n> diff --git a/src/libcamera/control_ids_android.yaml\n> b/src/libcamera/control_ids_android.yaml\n> new file mode 100644\n> index 00000000..8b0d624f\n> --- /dev/null\n> +++ b/src/libcamera/control_ids_android.yaml\n> @@ -0,0 +1,98 @@\n> +# SPDX-License-Identifier: LGPL-2.1-or-later\n> +#\n> +# Copyright (C) 2024, Google Inc.\n> +#\n> +%YAML 1.1\n> +---\n> +# Unless otherwise stated, all controls are bi-directional, i.e. they can\n> be\n> +# set through Request::controls() and returned out through\n> Request::metadata().\n> +vendor: android\n> +controls:\n> + - FaceDetectMode:\n> + type: uint8_t\n> + description: |\n> + Reporting mode of face detection.\n> +\n> + Currently identical to ANDROID_STATISTICS_FACE_DETECT_MODE.\n> +\n> + \\sa FaceDetectFaceRectangles\n> + \\sa FaceDetectFaceScores\n> + \\sa FaceDetectFaceLandmarks\n> + \\sa FaceDetectFaceIds\n> +\n> + enum:\n> + - name: FaceDetectModeOff\n> + value: 0\n> + description: |\n> + Pipeline should not report face detection result.\n> + - name: FaceDetectModeSimple\n> + value: 1\n> + description: |\n> + Pipeline should at least report FaceDetectFaceRectangles and\n> + FaceDetectFaceScores for each detected faces.\n> + FaceDetectFaceLandmarks and FaceDetectFaceIds is optional.\n> +\n> + - name: FaceDetectModeFull\n> + value: 2\n> + description: |\n> + Pipeline should report all face controls, including\n> + FaceDetectFaceRectangles, FaceDetectFaceScores,\n> + FaceDetectFaceLandmarks, and FaceDeteceFaceIds.\n> +\n> + - FaceDetectFaceRectangles:\n> + type: Rectangle\n> + description: |\n> + Boundary rectangles of the detected faces.\n> + The number of values should be the number of faces reported in\n> + FaceDetectFaceRectangles.\n> +\n> + The FaceDetectFaceRectangles control can only be returned in\n> metadata.\n> +\n> + Currently identical to ANDROID_STATISTICS_FACE_RECTANGLES.\n> +\n> + size: [n]\n> +\n> + - FaceDetectFaceScores:\n> + type: uint8_t\n> + description: |\n> + Confidence score of each of the detected faces by face detector.\n> + The range of score is [0, 100].\n> + The FaceDetectFaceScores control can only be returned in metadata.\n> + The number of values should be the number of faces reported in\n> + FaceDetectFaceRectangles.\n> +\n> + Currently identical to ANDROID_STATISTICS_FACE_SCORES.\n> +\n> + size: [n]\n> +\n> + - FaceDetectFaceLandmarks:\n> + type: Point\n> + description: |\n> + Array of human face landmark coordinates in format:\n> + [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],\n> + with i = index of face.\n> + The number of values should be 3 * the number of faces reported in\n> + FaceDetectFaceRectangles.\n> +\n> + The FaceDetectFaceLandmarks control can only be returned in\n> metadata.\n> +\n> + Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.\n> +\n> + size: [n]\n> +\n> + - FaceDetectFaceIds:\n> + type: int32_t\n> + description: |\n> + Each detected face is given a unique ID that is valid for as long\n> as\n> + the face is visible to the camera device. A face that leaves the\n> field\n> + of view and later returns may be assigned a new ID.\n> + The number of values should be the number of faces reported in\n> + FaceDetectFaceRectangles.\n> +\n> + The FaceDetectFaceIds control can only be returned in metadata.\n> +\n> + Currently identical to ANDROID_STATISTICS_FACE_IDS.\n> +\n> + size: [n]\n> +\n> +...\n> diff --git a/src/libcamera/control_ranges.yaml\n> b/src/libcamera/control_ranges.yaml\n> index d42447d0..5c6ca414 100644\n> --- a/src/libcamera/control_ranges.yaml\n> +++ b/src/libcamera/control_ranges.yaml\n> @@ -14,5 +14,6 @@ ranges:\n> # Raspberry Pi vendor controls\n> rpi: 20000\n> # Next range starts at 30000\n> + android: 30000\n>\n\nSorry, I should've updated the comments as well.\nWill update in the next version with other changes to prevent\nspamming.\n\n\n>\n> ...\n> diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp\n> index dba74404..c452e607 100644\n> --- a/src/libcamera/controls.cpp\n> +++ b/src/libcamera/controls.cpp\n> @@ -60,6 +60,7 @@ static constexpr size_t ControlValueSize[] = {\n> [ControlTypeString] = sizeof(char),\n> [ControlTypeRectangle] = sizeof(Rectangle),\n> [ControlTypeSize] = sizeof(Size),\n> + [ControlTypePoint] = sizeof(Point),\n> };\n>\n> } /* namespace */\n> @@ -254,6 +255,11 @@ std::string ControlValue::toString() const\n> str += value->toString();\n> break;\n> }\n> + case ControlTypePoint: {\n> + const Point *value = reinterpret_cast *>(data);\n> + str += value->toString();\n> + break;\n> + }\n> case ControlTypeNone:\n> case ControlTypeString:\n> break;\n> --\n> 2.46.0.792.g87dc391469-goog\n>\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 0D5D9C3257\n\tfor ;\n\tMon, 23 Sep 2024 14:52:31 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id AD3F063510;\n\tMon, 23 Sep 2024 16:52:30 +0200 (CEST)","from mail-ed1-x535.google.com (mail-ed1-x535.google.com\n\t[IPv6:2a00:1450:4864:20::535])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2E1C36037E\n\tfor ;\n\tMon, 23 Sep 2024 16:52:29 +0200 (CEST)","by mail-ed1-x535.google.com with SMTP id\n\t4fb4d7f45d1cf-5c247dd0899so18688a12.1\n\tfor ;\n\tMon, 23 Sep 2024 07:52:29 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=google.com header.i=@google.com\n\theader.b=\"URmUTGrL\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=google.com; s=20230601; t=1727103149; x=1727707949;\n\tdarn=lists.libcamera.org; \n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:from:to:cc:subject:date:message-id:reply-to;\n\tbh=UaC+lk5VkXYyIN9ZQgBu3X+XSncK0fQwJLEtm/51oHk=;\n\tb=URmUTGrLEc69cA05XDErx2oLr94pzq/X7JUg5V3IC7BvzAWevi+oAw/Vb66UUIYSe+\n\tfvBAYMS5cmIJJAUPHIaLX+5yjoXrJvCYpCRddVqoU4VCS0sv213o3X4n1Sv/lx701EfR\n\tJAmNMIItuvh1nYhdQKatdOAQTGq7VcyiOZ6DgMoIuTPqkdZxmOuOS5BYADYN6A2Mqmk4\n\t9TfUYr16RVCxCVuFuSX2D4vb2yAPIHzEeOdw9S5TwG5p0kDDsbX8daiYV5Vi3FTyIEU9\n\tllAelx8otXXoye09z2wkVGooHtdonkVwAYrjo7i9wjCFRZET2LAQqDybxaSuAW5EPZ0l\n\tI24g==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1727103149; x=1727707949;\n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:x-gm-message-state:from:to:cc:subject:date:message-id\n\t:reply-to;\n\tbh=UaC+lk5VkXYyIN9ZQgBu3X+XSncK0fQwJLEtm/51oHk=;\n\tb=icwS8T2F4zsbLs4RPjkuCdVJdYehqj7tZxzZsTDfxl5EjiDnrbv7qxBzGQI4M/VMN4\n\tNPiNacUiZ5BnVyc3vWCLO2latJO6SMQC26wDWvQOrakWND9BiCL9JnQzscNwGqUR6zPX\n\tt4UnTHl9NxuJxLiBD40bc4lZy+btW6Drj//NQM+q8vilke85Eqv+uwBuVaibl03xThyD\n\tqiIHRwZYuKJxmGoaRXLUK6moveRYWzoVrxa1jcdNjNAmRFJcScPnyoE6OPJXgcSS8M7N\n\t4G7g3SW1D3MrQ/aPZCGK7fnZelLmhsMi0UDFjJs2hnwfgZ8YN8k/rrTcuCRaPx1Jhvgr\n\tj9fw==","X-Gm-Message-State":"AOJu0Yxt4erIdrwxtuqbVATWBOz3kXxNsCHjq7YaZ5GRJvAIjZwVKXJ5\n\tR+e4zhfn3vg87PsLgXGDBs0TVK1GDSwBU/PkBgPvj2L2WfNh1ge0H7z2dBxLm2i9TO3CFVZje6X\n\tjjGPBB146qdNADH9oSlQcVhs4/cSgScm/i3jCQq/fno7nd9+1UzCw","X-Google-Smtp-Source":"AGHT+IHRISTkGnoXoGrCZQV1iPu0deYRS7y2UfJA3tuHA+dHEKc4ndyGa7W+o0UVKZ1D3lq/qpY0051Ug6XfkCXaxOM=","X-Received":"by 2002:a05:6402:1ec5:b0:5c2:2d47:2868 with SMTP id\n\t4fb4d7f45d1cf-5c5b77faf0emr333858a12.6.1727103148457; Mon, 23 Sep 2024\n\t07:52:28 -0700 (PDT)","MIME-Version":"1.0","References":"<20240923144921.1729183-1-chenghaoyang@google.com>\n\t<20240923144921.1729183-3-chenghaoyang@google.com>","In-Reply-To":"<20240923144921.1729183-3-chenghaoyang@google.com>","From":"Cheng-Hao Yang ","Date":"Mon, 23 Sep 2024 22:51:52 +0800","Message-ID":"","Subject":"Re: [PATCH v6 2/3] libcamera: Add face detection controls","To":"Harvey Yang ","Cc":"libcamera-devel@lists.libcamera.org, \n\tYudhistira Erlandinata ,\n\tbecker hsieh ","Content-Type":"multipart/alternative; boundary=\"000000000000a818550622ca89ca\"","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]