From patchwork Fri Aug 30 21:00:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 21125 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 76644C323E for ; Fri, 30 Aug 2024 21:03:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1D29563469; Fri, 30 Aug 2024 23:03:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Znn+t7K0"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D08B66341E for ; Fri, 30 Aug 2024 23:03:16 +0200 (CEST) Received: by mail-wr1-x429.google.com with SMTP id ffacd0b85a97d-37189d8e637so1427218f8f.3 for ; Fri, 30 Aug 2024 14:03:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1725051796; x=1725656596; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G4Ba3c7hJrcxKDqyLM6k3uK9nWWuDQPgRDSBllLf93U=; b=Znn+t7K0GAUgJ3mYyU1es3M17E7h7Hu4T5zObvWyyrz7HZYQGkT4QSrIs90132+93O GIpYlqZ2A/ArcaWv6zjkOhge6X4FKctOL931k/G8rsDib0CD26NHYfnIHTF5+47V5ZnO qkKMoY/qRM7zpUWbXuiBWVlj1gABdeGYCgIa4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725051796; x=1725656596; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G4Ba3c7hJrcxKDqyLM6k3uK9nWWuDQPgRDSBllLf93U=; b=g+CrcFGmMwy5S2+faznJmaCmHkyT4Y15nIxmedT0AVjtjReXdFm1tJkOaDEV52NM5U u4q/YEEySuJYh8bPqVFka/yPsdBmGyy2twVl5Pf4wQh6N2qbu5mJ6W9S2f+y2lCZny64 3i9KU+wNtOR4JYD2zImqnHEsA7FGIYa11POwMoD+Q9NhDnaZgF73qxiphsqyEW0rZGg8 8XTMNdCzPn6QQbFmhsgQ4ErJ1wq+LbRR5y+L9AzJnJ3Sx0zgUCTSIm2KAhMo5YJqjXvP P057CD3+SeMwFPtEkWRje0On2lPesnQ3vgLlvZwFxKwinNcJtAC1naZL4Tqvd5MwZ3Mq kZcw== X-Gm-Message-State: AOJu0YwWxbX0igYGzBeRyy+M3q32jKS8hB5HWLgjj1OgDQzwEmkpB0N5 DzioBwF837kgsEWSNbitrBUOiat+3stDf6E66Y5tJ8pJkG0YM94qjMxo/quf4GgiIHq3bu4FMZc QoA== X-Google-Smtp-Source: AGHT+IEfVzOuCShs8QkBLfsmcCXuk7hNSeaf71IT7P9hmMhUOjiHuXwtzG4n9jmLwajBDbbjtrxjTQ== X-Received: by 2002:adf:cd0c:0:b0:374:ba3f:d00c with SMTP id ffacd0b85a97d-374bf1e4d69mr89063f8f.59.1725051795874; Fri, 30 Aug 2024 14:03:15 -0700 (PDT) Received: from chenghaoyang-germany.c.googlers.com.com (161.126.77.34.bc.googleusercontent.com. [34.77.126.161]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3749efb37efsm4935529f8f.109.2024.08.30.14.03.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Aug 2024 14:03:15 -0700 (PDT) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Yudhistira Erlandinata , Harvey Yang Subject: [PATCH v3 1/3] libcamera: Add rectangle two-point constructor Date: Fri, 30 Aug 2024 21:00:19 +0000 Message-ID: <20240830210313.30691-2-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog In-Reply-To: <20240830210313.30691-1-chenghaoyang@chromium.org> References: <20240830210313.30691-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Yudhistira Erlandinata Add a Rectangle constructor that accepts two points: topLeft and bottomRight. Signed-off-by: Yudhistira Erlandinata Co-developed-by: Harvey Yang Reviewed-by: Jacopo Mondi --- include/libcamera/geometry.h | 2 ++ src/libcamera/geometry.cpp | 14 ++++++++++++++ 2 files changed, 16 insertions(+) diff --git a/include/libcamera/geometry.h b/include/libcamera/geometry.h index 3e6f0f5d7..dc56f180f 100644 --- a/include/libcamera/geometry.h +++ b/include/libcamera/geometry.h @@ -262,6 +262,8 @@ public: { } + constexpr Rectangle(const Point &topLeft, const Point &bottomRight); + int x; int y; unsigned int width; diff --git a/src/libcamera/geometry.cpp b/src/libcamera/geometry.cpp index 000151364..029b8dad2 100644 --- a/src/libcamera/geometry.cpp +++ b/src/libcamera/geometry.cpp @@ -629,6 +629,20 @@ std::ostream &operator<<(std::ostream &out, const SizeRange &sr) * \param[in] size The desired Rectangle size */ +/** + * \fn Rectangle::Rectangle(const Point &topLeft, const Point &bottomRight) + * \brief Construct a Rectangle with the two given points + * \param[in] topLeft The top-left corner + * \param[in] bottomRight The bottom-right corner + */ +constexpr Rectangle::Rectangle(const Point &topLeft, const Point &bottomRight) + : x(topLeft.x), y(topLeft.y), + width(bottomRight.x - x), + height(bottomRight.y - y) +{ + ASSERT(bottomRight.x >= x && bottomRight.y >= y); +} + /** * \var Rectangle::x * \brief The horizontal coordinate of the rectangle's top-left corner From patchwork Fri Aug 30 21:00:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 21126 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0FB1BC32C2 for ; Fri, 30 Aug 2024 21:03:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DB1C263471; Fri, 30 Aug 2024 23:03:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="m6j3JRmx"; dkim-atps=neutral Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6002C61E4A for ; Fri, 30 Aug 2024 23:03:17 +0200 (CEST) Received: by mail-wm1-x32a.google.com with SMTP id 5b1f17b1804b1-42bbbff40bbso8872195e9.2 for ; Fri, 30 Aug 2024 14:03:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1725051797; x=1725656597; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r/I/asyNDPgFivOcw3MnOP+duUL4cdodULyhKmUniNk=; b=m6j3JRmxFh9hBxvmBID2Q62NkbQyo8qFwhtXaWSQq1zOM6fdvo27KnI+HdaydE8IGZ 7WhAM9dHSvsxzzn/jSg54TMbDbA5VPmKhm04aVN9hwoHDgmRKRvR+W8DaoqNBFT6kvW+ WHSgu2a4tuPM6HPF1o7NW6MyolUUaxxBVbaLU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725051797; x=1725656597; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=r/I/asyNDPgFivOcw3MnOP+duUL4cdodULyhKmUniNk=; b=T881DNY8IjZfxULXy+rFPX61psQBKxy0V9t4fhNPYcEfULcAr+7hx1X01xyI09wD2t EmNiW2+unHH2PyzuzbzIHwYREl0kUPLrKk5HajWWf2p8Bus8pY9cZiGs+H2+RQ4Bf0wJ WKHq+sJpSrB7to8lkvLVJS0PifwlKcjEDH5tlwUM7S/oJ2fYX2ttFEfdYFH5/3sfjRi+ O0pCXZBNEQGgvToZvb0he2pPQM1bD8MfeG2yn/dRmggtPmRTV/LGp0E0cUuFrxUPMkgJ iqQcyF5TaqMwWv7R1UybqMN7Q1pff65vLwJuCaXqL2l8ggxABVDZ/0f5fDfIhWaX0ThS fsow== X-Gm-Message-State: AOJu0YxM03ZAWI9hZh+06V3/LjwUJJMdeBIlb4JD2XcX4/6hyz4eZQ7v denRmxrKJ/aaY2p7YlilPrrhra4OYTIh/CO/MPKKy4Qv+cioPrtjyX4OlEpzSYzYvlPYKqYEiWh 8tA== X-Google-Smtp-Source: AGHT+IHQznniGc0rKRGuRGQJriVB6WM6hAcyb1WL+lJJ5+TMXFOxA7Q0kNLyhnf45BpaF37XaNM+gg== X-Received: by 2002:a05:600c:1c9e:b0:426:61af:e1d6 with SMTP id 5b1f17b1804b1-42bbb44023emr24955075e9.29.1725051796598; Fri, 30 Aug 2024 14:03:16 -0700 (PDT) Received: from chenghaoyang-germany.c.googlers.com.com (161.126.77.34.bc.googleusercontent.com. [34.77.126.161]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3749efb37efsm4935529f8f.109.2024.08.30.14.03.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Aug 2024 14:03:16 -0700 (PDT) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Yudhistira Erlandinata , becker hsieh , Harvey Yang Subject: [PATCH v3 2/3] libcamera: Add face detection controls Date: Fri, 30 Aug 2024 21:00:20 +0000 Message-ID: <20240830210313.30691-3-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog In-Reply-To: <20240830210313.30691-1-chenghaoyang@chromium.org> References: <20240830210313.30691-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Yudhistira Erlandinata Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores, and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting FaceDetectFaceLandmark. Signed-off-by: Yudhistira Erlandinata Co-developed-by: becker hsieh Co-developed-by: Harvey Yang --- include/libcamera/controls.h | 6 +++ src/libcamera/control_ids_draft.yaml | 63 ++++++++++++++++++++++++++++ src/libcamera/controls.cpp | 6 +++ 3 files changed, 75 insertions(+) diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h index 7c2bb2872..bf1b8609c 100644 --- a/include/libcamera/controls.h +++ b/include/libcamera/controls.h @@ -34,6 +34,7 @@ enum ControlType { ControlTypeString, ControlTypeRectangle, ControlTypeSize, + ControlTypePoint, }; namespace details { @@ -87,6 +88,11 @@ struct control_type { static constexpr ControlType value = ControlTypeSize; }; +template<> +struct control_type { + static constexpr ControlType value = ControlTypePoint; +}; + template struct control_type> : public control_type> { }; diff --git a/src/libcamera/control_ids_draft.yaml b/src/libcamera/control_ids_draft.yaml index 9bef5bf15..23e09bff8 100644 --- a/src/libcamera/control_ids_draft.yaml +++ b/src/libcamera/control_ids_draft.yaml @@ -227,4 +227,67 @@ controls: value. All of the custom test patterns will be static (that is the raw image must not vary from frame to frame). + - FaceDetectMode: + type: uint8_t + description: | + Reporting mode of face detection. + + enum: + - name: FaceDetectModeOff + value: 0 + description: | + Pipeline should not report face detection result. + - name: FaceDetectModeSimple + value: 1 + description: | + Pipeline should at least report faces boundary rectangles, in + metadata FaceDetectFaceRectangles, and confidence score, + in metadata FaceDetectFaceScores, for each of them. + FaceDetectFaceLandmark is optional. + + The number of faces in each list must be the same. + + \sa FaceDetectFaceRectangles + \sa FaceDetectFaceScores + \sa FaceDetectFaceLandmark + + - FaceDetectFaceRectangles: + type: Rectangle + description: | + Boundary rectangle of the detected faces in format: + [..., xmin_i, ymin_i, xmax_i, ymax_i, ...], where (0,0) is top-left + of active pixel area. + The number of values should be 3 * the number of faces. + + The FaceDetectFaceRectangles control can only be returned in metadata. + + size: [n] + + - FaceDetectFaceScores: + type: uint8_t + description: | + Confidence score of each of the detected faces by face detector. + The range of score is [0, 100]. + The FaceDetectFaceScores control can only be returned in metadata. + The number of values should be the number of faces. + + Currently identical to ANDROID_STATISTICS_FACE_SCORES. + + size: [n] + + - FaceDetectFaceLandmark: + type: Point + description: | + Array of human face landmark coordinates in format: + [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...], + with i = index of face. + The number of values should be 6 * the number of faces. + + The FaceDetectFaceLandmark control can only be returned in metadata. + + Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS. + + size: [n] + + ... diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp index 11d35321c..ce73ae9d7 100644 --- a/src/libcamera/controls.cpp +++ b/src/libcamera/controls.cpp @@ -61,6 +61,7 @@ static constexpr size_t ControlValueSize[] = { [ControlTypeString] = sizeof(char), [ControlTypeRectangle] = sizeof(Rectangle), [ControlTypeSize] = sizeof(Size), + [ControlTypePoint] = sizeof(Point), }; } /* namespace */ @@ -255,6 +256,11 @@ std::string ControlValue::toString() const str += value->toString(); break; } + case ControlTypePoint: { + const Point *value = reinterpret_cast(data); + str += value->toString(); + break; + } case ControlTypeNone: case ControlTypeString: break; From patchwork Fri Aug 30 21:00:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 21127 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CBDDBC32D5 for ; Fri, 30 Aug 2024 21:03:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C24FD634A6; Fri, 30 Aug 2024 23:03:22 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="e1cWtsEc"; dkim-atps=neutral Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 40EFD6341E for ; Fri, 30 Aug 2024 23:03:18 +0200 (CEST) Received: by mail-wr1-x434.google.com with SMTP id ffacd0b85a97d-3718b5e9c4fso1332583f8f.0 for ; Fri, 30 Aug 2024 14:03:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1725051797; x=1725656597; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7AoRbpu06lGzR6tK3ziKLG0KVhF/Fx8lESirawDRrys=; b=e1cWtsEcP8c3lBtWuOEMSyTyqQB6m9U7UjoLMJdBHu4YFhrmJzp8s6lOXAZYsXDbfy EzFkurRUOphOfjCJkL61deWxVH6jX2cbWGfU/RCSWkh+bwOqK2aSVydGBElBV/fzaEK8 IVoDL8TXq65CTOpEg93wBkI2fAF1Ka5AZRBvs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725051797; x=1725656597; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7AoRbpu06lGzR6tK3ziKLG0KVhF/Fx8lESirawDRrys=; b=rFzDYRNM1GnVLM6RBvdvD27Gm+mvtnEfpPDwK6fIzMrTjJcL/dIieEWAx3Qs7sHyoK o1UHPvDuhj2GfJCUW4AyWw7NUOYYNquUJIWQsFL1UiuIRj/+3uCCxtHhlsYyp0oHT8/s FjvsNa0uyg8we1GGasNQJGmkIb7yAvItdEsR772M+OOCzEI1fRtIeu+wVjN2ueZ5Xykb 2SaVSoo6jINTAuF7JX41aAvSI0RPRdNeVymdl6KWW4HLlpxi1IXcLCUg+8mG19Rtkslq VQ7n8dYYI+b6LLIXtDodsdwgqzh40M7tnc05Y4uS418OjEQ+l7FFkr2Hc8BGqwmZujT5 WISA== X-Gm-Message-State: AOJu0YwsWfow4rAFdqNW5E0PBTJhvi6P7XdFeakM/w6SgrWBqtwu3ge2 yX+1yPD/NmFFXfSupt+5+1qV8pT9fA3PVuPLG8WMgy6eeeYUoHQppqN3pKGTgkHk2USU1T5Rt8U qlw== X-Google-Smtp-Source: AGHT+IE31b2YSlLUsqCfbmhbuf527wTAmByyDNHBbvDhQu83VStZubGEbrUPvTmWpc5TDcc6VW4n7g== X-Received: by 2002:a05:6000:18a6:b0:368:2f01:307a with SMTP id ffacd0b85a97d-3749b57c76emr7495417f8f.46.1725051797094; Fri, 30 Aug 2024 14:03:17 -0700 (PDT) Received: from chenghaoyang-germany.c.googlers.com.com (161.126.77.34.bc.googleusercontent.com. [34.77.126.161]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3749efb37efsm4935529f8f.109.2024.08.30.14.03.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Aug 2024 14:03:16 -0700 (PDT) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Yudhistira Erlandinata Subject: [PATCH v3 3/3] libcamera: android: Add face detection control support Date: Fri, 30 Aug 2024 21:00:21 +0000 Message-ID: <20240830210313.30691-4-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog In-Reply-To: <20240830210313.30691-1-chenghaoyang@chromium.org> References: <20240830210313.30691-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Allow Android HAL adapter to pass the face detection metadata control to the pipeline and also send face detection metadata to the camera client if the pipeline generates it. Signed-off-by: Yudhistira Erlandinata Co-developed-by: Harvey Yang --- src/android/camera_capabilities.cpp | 45 +++++++++++++++++++-- src/android/camera_device.cpp | 61 ++++++++++++++++++++++++++++- 2 files changed, 100 insertions(+), 6 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 71043e127..a89a3115d 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -7,6 +7,8 @@ #include "camera_capabilities.h" +#include + #include #include #include @@ -1176,11 +1178,46 @@ int CameraCapabilities::initializeStaticMetadata() maxFrameDuration_); /* Statistics static metadata. */ - uint8_t faceDetectMode = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; - staticMetadata_->addEntry(ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES, - faceDetectMode); - int32_t maxFaceCount = 0; + auto iter = camera_->controls().find(controls::draft::FaceDetectMode.id()); + if (iter != camera_->controls().end()) { + const ControlInfo &faceDetectCtrlInfo = iter->second; + std::vector faceDetectModes; + bool hasFaceDetection = false; + + for (const auto &value : faceDetectCtrlInfo.values()) { + uint8_t mode = value.get(); + uint8_t androidMode = 0; + + switch (mode) { + case controls::draft::FaceDetectModeOff: + androidMode = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; + break; + case controls::draft::FaceDetectModeSimple: + androidMode = ANDROID_STATISTICS_FACE_DETECT_MODE_SIMPLE; + hasFaceDetection = true; + break; + default: + LOG(HAL, Fatal) << "Received invalid face detect mode: " << mode; + } + faceDetectModes.push_back(androidMode); + } + if (hasFaceDetection) { + /* + * \todo Create new libcamera controls to query max + * possible faces detected. + */ + maxFaceCount = 10; + staticMetadata_->addEntry( + ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES, + faceDetectModes.data(), faceDetectModes.size()); + } + } else { + uint8_t faceDetectMode = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; + staticMetadata_->addEntry(ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES, + faceDetectMode); + } + staticMetadata_->addEntry(ANDROID_STATISTICS_INFO_MAX_FACE_COUNT, maxFaceCount); diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 493f66e7b..87b087160 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -15,6 +15,7 @@ #include #include +#include #include #include @@ -22,6 +23,7 @@ #include #include #include +#include #include #include "system/graphics.h" @@ -813,6 +815,11 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) controls.set(controls::ScalerCrop, cropRegion); } + if (settings.getEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, &entry)) { + const uint8_t *data = entry.data.u8; + controls.set(controls::draft::FaceDetectMode, data[0]); + } + if (settings.getEntry(ANDROID_SENSOR_TEST_PATTERN_MODE, &entry)) { const int32_t data = *entry.data.i32; int32_t testPatternMode = controls::draft::TestPatternModeOff; @@ -1540,8 +1547,9 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons value32 = ANDROID_SENSOR_TEST_PATTERN_MODE_OFF; resultMetadata->addEntry(ANDROID_SENSOR_TEST_PATTERN_MODE, value32); - value = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; - resultMetadata->addEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, value); + settings.getEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, &entry); + resultMetadata->addEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, + entry.data.u8, 1); value = ANDROID_STATISTICS_LENS_SHADING_MAP_MODE_OFF; resultMetadata->addEntry(ANDROID_STATISTICS_LENS_SHADING_MAP_MODE, @@ -1580,6 +1588,55 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons resultMetadata->addEntry(ANDROID_SENSOR_FRAME_DURATION, *frameDuration * 1000); + const auto &faceDetectRectangles = + metadata.get(controls::draft::FaceDetectFaceRectangles); + if (faceDetectRectangles) { + const Span rectangles = *faceDetectRectangles; + std::vector flatRectangles; + for (const Rectangle &rect : rectangles) { + flatRectangles.push_back(rect.x); + flatRectangles.push_back(rect.y); + flatRectangles.push_back(rect.x + rect.width); + flatRectangles.push_back(rect.y + rect.height); + } + resultMetadata->addEntry( + ANDROID_STATISTICS_FACE_RECTANGLES, flatRectangles); + } + + const auto &faceDetectFaceScores = + metadata.get(controls::draft::FaceDetectFaceScores); + if (faceDetectRectangles && faceDetectFaceScores) { + const Span &scores = *faceDetectFaceScores; + if (scores.size() != faceDetectRectangles->size()) { + LOG(HAL, Error) << "Pipeline returned wrong number of face scores; " + << "Expected: " + << faceDetectRectangles->size() + << ", got:" << scores.size(); + } + resultMetadata->addEntry(ANDROID_STATISTICS_FACE_SCORES, scores); + } + + const auto &faceDetectFaceLandmarks = + metadata.get(controls::draft::FaceDetectFaceLandmark); + if (faceDetectFaceScores && faceDetectRectangles && + faceDetectFaceLandmarks) { + const auto &landmarks = *faceDetectFaceLandmarks; + size_t expectedLandmarks = faceDetectRectangles->size() * 3; + if (landmarks.size() != expectedLandmarks) { + LOG(HAL, Error) << "Pipeline returned wrong number of face landmarks; " + << "Expected: " << expectedLandmarks + << ", got: " << landmarks.size(); + } + + std::vector androidLandmarks; + for (const auto &landmark : landmarks) { + androidLandmarks.push_back(landmark.x); + androidLandmarks.push_back(landmark.y); + } + resultMetadata->addEntry( + ANDROID_STATISTICS_FACE_LANDMARKS, androidLandmarks); + } + const auto &scalerCrop = metadata.get(controls::ScalerCrop); if (scalerCrop) { const Rectangle &crop = *scalerCrop;