From patchwork Tue Sep 24 11:02:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 21344 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D025EC0F1B for ; Tue, 24 Sep 2024 11:04:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 68BB663510; Tue, 24 Sep 2024 13:04:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="F7J2m8eL"; dkim-atps=neutral Received: from mail-pg1-x530.google.com (mail-pg1-x530.google.com [IPv6:2607:f8b0:4864:20::530]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B1B146350B for ; Tue, 24 Sep 2024 13:04:03 +0200 (CEST) Received: by mail-pg1-x530.google.com with SMTP id 41be03b00d2f7-7e6b738acd5so280989a12.0 for ; Tue, 24 Sep 2024 04:04:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1727175842; x=1727780642; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nD1aTGFoZhor1a0602dxtuqbeRDiOtQLq2EYFIgL/IE=; b=F7J2m8eLCTBdWzZ43yMy3urRtz4MYQ47Nz/DYznWLjyTh2Ds3DTCydaiXBiAb8fwj9 tyb+0VRloAPddmxo16RyPCMhbTa9AX0RdE0xT56zPYG9Grq7dCIpCksmLE+gYoo2w6Ht 5kLv/v0obggtfNuHFOatKSRYuuXT4wUm8hBtw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727175842; x=1727780642; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nD1aTGFoZhor1a0602dxtuqbeRDiOtQLq2EYFIgL/IE=; b=TCL/q4aAZtpesat6GjcfBCzFGOPUbN94CJoN6s486mM0EOCev5vFcPh7ZWU47P16Ob Ha8hIeePcqmE0PvhJkxuHYWHK2kRJlUqmSWJUINj/7pbJfGELc4q73WkaixErswKaffj mdx6ISMOuXN9kqqMtqgj7r0UqXPytJFGCDXitnwfeRy51y9jyWvxXuFiguQqnsXTCYai awl0fuiZXVajno9fAnHLC8aFnxfua2VKppKNxrdyz6dBuZwnsVeJz6ZBAMrGwsVDy/m4 k0L26TRKD1rOIeJaG/KVwESpoh+teo5vbEeFQFMpwW0ep/qgo8bfVHBvE5kzhZzg/W1k mOiw== X-Gm-Message-State: AOJu0YwIafGsx/2JFKCFp+ZdlEmaayb/I6zXGbQmrbvhk6pj4SsmnwWC 6/gDHa/2jC4SsQlUhq8hnl3hMurWlTtRXpwwRr7To5lYJ9ltJdZ9lMNEoMzjAEdMaVyScq2mwb4 = X-Google-Smtp-Source: AGHT+IGzvYP/LLry8S2fgbwzQ4FQ/tzj8rWvO5lQeLg/d5ilMhUDHOWDSARKHCGXrio72lSAVzks1Q== X-Received: by 2002:a05:6a20:e605:b0:1d1:13de:68c6 with SMTP id adf61e73a8af0-1d30ca220ddmr18164417637.29.1727175841658; Tue, 24 Sep 2024 04:04:01 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (208.158.221.35.bc.googleusercontent.com. [35.221.158.208]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71afc9833c3sm993326b3a.182.2024.09.24.04.04.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Sep 2024 04:04:01 -0700 (PDT) From: Harvey Yang X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Yudhistira Erlandinata , becker hsieh , Harvey Yang Subject: [PATCH v7 2/3] libcamera: Add face detection controls Date: Tue, 24 Sep 2024 11:02:45 +0000 Message-ID: <20240924110353.2025397-3-chenghaoyang@google.com> X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog In-Reply-To: <20240924110353.2025397-1-chenghaoyang@google.com> References: <20240924110353.2025397-1-chenghaoyang@google.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Yudhistira Erlandinata Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores, and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting FaceDetectFaceLandmark. Signed-off-by: Yudhistira Erlandinata Co-developed-by: becker hsieh Co-developed-by: Harvey Yang Reviewed-by: Jacopo Mondi --- include/libcamera/controls.h | 6 ++ include/libcamera/meson.build | 3 +- src/libcamera/control_ids_android.yaml | 98 ++++++++++++++++++++++++++ src/libcamera/control_ranges.yaml | 4 +- src/libcamera/controls.cpp | 6 ++ 5 files changed, 115 insertions(+), 2 deletions(-) create mode 100644 src/libcamera/control_ids_android.yaml diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h index 7c2bb287..bf1b8609 100644 --- a/include/libcamera/controls.h +++ b/include/libcamera/controls.h @@ -34,6 +34,7 @@ enum ControlType { ControlTypeString, ControlTypeRectangle, ControlTypeSize, + ControlTypePoint, }; namespace details { @@ -87,6 +88,11 @@ struct control_type { static constexpr ControlType value = ControlTypeSize; }; +template<> +struct control_type { + static constexpr ControlType value = ControlTypePoint; +}; + template struct control_type> : public control_type> { }; diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index a969a95d..4fff14d2 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -34,6 +34,7 @@ libcamera_headers_install_dir = get_option('includedir') / libcamera_include_dir controls_map = { 'controls': { + 'android': 'control_ids_android.yaml', 'draft': 'control_ids_draft.yaml', 'core': 'control_ids_core.yaml', 'rpi/vc4': 'control_ids_rpi.yaml', @@ -55,7 +56,7 @@ foreach mode, entry : controls_map files_list = [] input_files = [] foreach vendor, header : entry - if vendor != 'core' and vendor != 'draft' + if vendor != 'core' and vendor != 'draft' and vendor != 'android' if vendor not in pipelines continue endif diff --git a/src/libcamera/control_ids_android.yaml b/src/libcamera/control_ids_android.yaml new file mode 100644 index 00000000..8b0d624f --- /dev/null +++ b/src/libcamera/control_ids_android.yaml @@ -0,0 +1,98 @@ +# SPDX-License-Identifier: LGPL-2.1-or-later +# +# Copyright (C) 2024, Google Inc. +# +%YAML 1.1 +--- +# Unless otherwise stated, all controls are bi-directional, i.e. they can be +# set through Request::controls() and returned out through Request::metadata(). +vendor: android +controls: + - FaceDetectMode: + type: uint8_t + description: | + Reporting mode of face detection. + + Currently identical to ANDROID_STATISTICS_FACE_DETECT_MODE. + + \sa FaceDetectFaceRectangles + \sa FaceDetectFaceScores + \sa FaceDetectFaceLandmarks + \sa FaceDetectFaceIds + + enum: + - name: FaceDetectModeOff + value: 0 + description: | + Pipeline should not report face detection result. + - name: FaceDetectModeSimple + value: 1 + description: | + Pipeline should at least report FaceDetectFaceRectangles and + FaceDetectFaceScores for each detected faces. + FaceDetectFaceLandmarks and FaceDetectFaceIds is optional. + + - name: FaceDetectModeFull + value: 2 + description: | + Pipeline should report all face controls, including + FaceDetectFaceRectangles, FaceDetectFaceScores, + FaceDetectFaceLandmarks, and FaceDeteceFaceIds. + + - FaceDetectFaceRectangles: + type: Rectangle + description: | + Boundary rectangles of the detected faces. + The number of values should be the number of faces reported in + FaceDetectFaceRectangles. + + The FaceDetectFaceRectangles control can only be returned in metadata. + + Currently identical to ANDROID_STATISTICS_FACE_RECTANGLES. + + size: [n] + + - FaceDetectFaceScores: + type: uint8_t + description: | + Confidence score of each of the detected faces by face detector. + The range of score is [0, 100]. + The FaceDetectFaceScores control can only be returned in metadata. + The number of values should be the number of faces reported in + FaceDetectFaceRectangles. + + Currently identical to ANDROID_STATISTICS_FACE_SCORES. + + size: [n] + + - FaceDetectFaceLandmarks: + type: Point + description: | + Array of human face landmark coordinates in format: + [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...], + with i = index of face. + The number of values should be 3 * the number of faces reported in + FaceDetectFaceRectangles. + + The FaceDetectFaceLandmarks control can only be returned in metadata. + + Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS. + + size: [n] + + - FaceDetectFaceIds: + type: int32_t + description: | + Each detected face is given a unique ID that is valid for as long as + the face is visible to the camera device. A face that leaves the field + of view and later returns may be assigned a new ID. + The number of values should be the number of faces reported in + FaceDetectFaceRectangles. + + The FaceDetectFaceIds control can only be returned in metadata. + + Currently identical to ANDROID_STATISTICS_FACE_IDS. + + size: [n] + +... diff --git a/src/libcamera/control_ranges.yaml b/src/libcamera/control_ranges.yaml index d42447d0..9d406f10 100644 --- a/src/libcamera/control_ranges.yaml +++ b/src/libcamera/control_ranges.yaml @@ -13,6 +13,8 @@ ranges: draft: 10000 # Raspberry Pi vendor controls rpi: 20000 - # Next range starts at 30000 + # Android metadata related controls + android: 30000 + # Next range starts at 40000 ... diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp index dba74404..c452e607 100644 --- a/src/libcamera/controls.cpp +++ b/src/libcamera/controls.cpp @@ -60,6 +60,7 @@ static constexpr size_t ControlValueSize[] = { [ControlTypeString] = sizeof(char), [ControlTypeRectangle] = sizeof(Rectangle), [ControlTypeSize] = sizeof(Size), + [ControlTypePoint] = sizeof(Point), }; } /* namespace */ @@ -254,6 +255,11 @@ std::string ControlValue::toString() const str += value->toString(); break; } + case ControlTypePoint: { + const Point *value = reinterpret_cast(data); + str += value->toString(); + break; + } case ControlTypeNone: case ControlTypeString: break;