From patchwork Mon Sep 30 19:59:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21440 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6824CC3257 for ; Mon, 30 Sep 2024 19:59:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1173A63518; Mon, 30 Sep 2024 21:59:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uD8UHZfE"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9538A63516 for ; Mon, 30 Sep 2024 21:59:31 +0200 (CEST) Received: from ideasonboard.com (unknown [95.131.46.153]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EC1BBA2F; Mon, 30 Sep 2024 21:57:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1727726280; bh=x+I7ykB+ojLGHL4GByGX0cx9G+UWeYOz0fLsOocL+64=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uD8UHZfEQ2X7IkDXhTDOugEskqzUfyPH9DgcTPDvUhazFduGKjwZ5jRJ5+P/KtNaE vI4y0438bdqAtWIjXthcL/uWXGTPD8OKwLMw+m4u0iTMLTSmP7x53iRJrC4f/S2UfU wyLuVlp/8JYlQnbiYSVuwg2+0t5U4N9ZXHijnCXY= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Harvey Yang Subject: [PATCH v9 1/5] libcamera: geometry: Clarify Rectangle's top-left corner Date: Mon, 30 Sep 2024 21:59:09 +0200 Message-ID: <20240930195915.152187-2-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20240930195915.152187-1-jacopo.mondi@ideasonboard.com> References: <20240930195915.152187-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The libcamera::Rectangle class allows defining rectangles regardless of the orientation of the reference system where a rectangle is used in. This implies that, depending on the reference system in use, the rectangle's top-left corner, as defined by libcamera, doesn't correspond to the visual top-left position. ^ | | ------------------- | ^ | h | | | y| o---->------------- | w -------------------------------> (0,0) x (0,0) x ------------------------------> | w y| o---->------------- | | | h | v | | ------------------- | V Clarify that a Rectangle's top-left corner corresponds to the point with the smaller x and y coordinates and that the horizontal and vertical dimensions are obtained by positive increments along the corresponding axes. Signed-off-by: Jacopo Mondi Reviewed-by: Harvey Yang Reviewed-by: David Plowman --- src/libcamera/geometry.cpp | 93 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) diff --git a/src/libcamera/geometry.cpp b/src/libcamera/geometry.cpp index 000151364c7f..6eb432e5d803 100644 --- a/src/libcamera/geometry.cpp +++ b/src/libcamera/geometry.cpp @@ -595,6 +595,88 @@ std::ostream &operator<<(std::ostream &out, const SizeRange &sr) * Rectangles are used to identify an area of an image. They are specified by * the coordinates of top-left corner and their horizontal and vertical size. * + * libcamera canonically defines a rectangle's 'top-left' corner as the point + * with the smaller 'x' and smaller 'y' coordinates. Depending on the reference + * system where the rectangle is used this might not correspond to the visual + * top-left corner of the rectangle. + * + * The rectangle's horizontal and vertical dimensions are obtained by positively + * increments along the corresponding axes by the given horizontal and vertical + * sizes. + * + * Examples: + * + * \verbatim + + X = top-left corner + o = reference system origin point + + ^ + | + | ----------------- + | ^ | + | | | + | X--->------------- + | + o-------------------------------> + (0,0) + + + (0,0) + <-------------------------------o + | + -------------<---X | + | | | + | V | + ----------------- | + V + + + (0,0) + o-------------------------------> + | X--->------------- + | | | + | v | + | ----------------- + | + V + + ^ + | + ----------------- | + | ^ | + | | | + -------------<---X | + <-------------------------------o + (0,0) + + ^ + | + ------|----------- + ^ | | + | | | + X--->--|----------- + ---------------o----------------> + |(0,0) + | + | + | + | + + ^ + | + | + | + | + | + ---------------o----------------> + (0,0)| ------------ + | ^ | + | | | + | X--->-------- + | + \endverbatim + * * The measure unit of the rectangle coordinates and size, as well as the * reference point from which the Rectangle::x and Rectangle::y displacements * refers to, are defined by the context were rectangle is used. @@ -611,6 +693,8 @@ std::ostream &operator<<(std::ostream &out, const SizeRange &sr) * \param[in] x The horizontal coordinate of the top-left corner * \param[in] y The vertical coordinate of the top-left corner * \param[in] size The size + * + * The rectangle's top-left corner is the point with the smaller x and y values. */ /** @@ -620,6 +704,8 @@ std::ostream &operator<<(std::ostream &out, const SizeRange &sr) * \param[in] y The vertical coordinate of the top-left corner * \param[in] width The width * \param[in] height The height + * + * The rectangle's top-left corner is the point with the smaller x and y values. */ /** @@ -632,11 +718,15 @@ std::ostream &operator<<(std::ostream &out, const SizeRange &sr) /** * \var Rectangle::x * \brief The horizontal coordinate of the rectangle's top-left corner + * + * The rectangle's top-left corner is the point with the smaller x and y values. */ /** * \var Rectangle::y * \brief The vertical coordinate of the rectangle's top-left corner + * + * The rectangle's top-left corner is the point with the smaller x and y values. */ /** @@ -685,6 +775,9 @@ Point Rectangle::center() const /** * \fn Point Rectangle::topLeft() const * \brief Retrieve the coordinates of the top left corner of this Rectangle + * + * The rectangle's top-left corner is the point with the smaller x and y values. + * * \return The Rectangle's top left corner */ From patchwork Mon Sep 30 19:59:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21441 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2B316C3257 for ; Mon, 30 Sep 2024 19:59:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C0B5563517; Mon, 30 Sep 2024 21:59:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bNgolEag"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7933163517 for ; Mon, 30 Sep 2024 21:59:32 +0200 (CEST) Received: from ideasonboard.com (unknown [95.131.46.153]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7BE9DFB8; Mon, 30 Sep 2024 21:58:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1727726281; bh=k2+c5+LRuyKaS5oPPCgMPKAhfdHGfVjtixSJ+6RwFks=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bNgolEageXc5BgRBkMbhkJxVR80c0NUuLZEatHuxSBhj8ZLAB2Zgjh01w9dOf9xPq KafrMxKVHGcu2F/TMrkYsZ5OveKx3JFC57s7T/WKGVYnv+DvKfwZ/CB6g/rZKQ1bJJ /txTNUsq5iPtaJcDELZvYqXdRi4VQCwFYQ53xI5c= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Harvey Yang , Yudhistira Erlandinata , Harvey Yang Subject: [PATCH v9 2/5] libcamera: geometry: Add two-point Rectangle constructor Date: Mon, 30 Sep 2024 21:59:10 +0200 Message-ID: <20240930195915.152187-3-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20240930195915.152187-1-jacopo.mondi@ideasonboard.com> References: <20240930195915.152187-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Yudhistira Erlandinata Add a constructor to the Rectangle class that accepts two points. The constructed Rectangle spans all the space between the two given points. Signed-off-by: Yudhistira Erlandinata Co-developed-by: Harvey Yang Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Reviewed-by: Harvey Yang --- include/libcamera/geometry.h | 7 +++++ src/libcamera/geometry.cpp | 58 ++++++++++++++++++++++++++++++++++++ test/geometry.cpp | 14 +++++++++ 3 files changed, 79 insertions(+) diff --git a/include/libcamera/geometry.h b/include/libcamera/geometry.h index 3e6f0f5d7fab..2cc25f1facd9 100644 --- a/include/libcamera/geometry.h +++ b/include/libcamera/geometry.h @@ -262,6 +262,13 @@ public: { } + constexpr Rectangle(const Point &point1, const Point &point2) + : Rectangle(std::min(point1.x, point2.x), std::min(point1.y, point2.y), + std::max(point1.x, point2.x) - std::min(point1.x, point2.x), + std::max(point1.y, point2.y) - std::min(point1.y, point2.y)) + { + } + int x; int y; unsigned int width; diff --git a/src/libcamera/geometry.cpp b/src/libcamera/geometry.cpp index 6eb432e5d803..5e5e72e15d67 100644 --- a/src/libcamera/geometry.cpp +++ b/src/libcamera/geometry.cpp @@ -715,6 +715,64 @@ std::ostream &operator<<(std::ostream &out, const SizeRange &sr) * \param[in] size The desired Rectangle size */ +/** + * \fn Rectangle::Rectangle(const Point &point1, const Point &point2) + * \brief Construct a Rectangle with the two given points + * \param[in] point1 One of corners of the rectangle + * \param[in] point2 The opposite diagonal corner point of \a point1 + * + * Contruct a rectangle that spans the space between two given points. The + * position of the two given points is not relevant for the rectangle's + * contruction. + * + * \verbatim + + p1 = point1 + p2 = point2 + + ^ + | + | ----------------p2 + | | | + | | | + | p1---------------- + | + o-------------------------------> + (0,0) + + ^ + | + | p1---------------- + | | | + | | | + | ----------------p2 + | + o-------------------------------> + (0,0) + + ^ + | + | ----------------p1 + | | | + | | | + | p2---------------- + | + o-------------------------------> + (0,0) + + ^ + | + | p2---------------- + | | | + | | | + | ----------------p1 + | + o-------------------------------> + (0,0) + + \endverbatim + */ + /** * \var Rectangle::x * \brief The horizontal coordinate of the rectangle's top-left corner diff --git a/test/geometry.cpp b/test/geometry.cpp index 64169206ad16..5760fa3c885a 100644 --- a/test/geometry.cpp +++ b/test/geometry.cpp @@ -481,6 +481,20 @@ protected: return TestFail; } + Point topLeft(3, 3); + Point bottomRight(30, 30); + Point topRight(30, 3); + Point bottomLeft(3, 30); + Rectangle rect1(topLeft, bottomRight); + Rectangle rect2(topRight, bottomLeft); + Rectangle rect3(bottomRight, topLeft); + Rectangle rect4(bottomLeft, topRight); + + if (rect1 != rect2 || rect1 != rect3 || rect1 != rect4) { + cout << "Point-to-point construction failed" << endl; + return TestFail; + } + return TestPass; } }; From patchwork Mon Sep 30 19:59:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21442 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1330AC3257 for ; Mon, 30 Sep 2024 19:59:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5D2046351F; Mon, 30 Sep 2024 21:59:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="o80mdWfE"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ACCBF63502 for ; Mon, 30 Sep 2024 21:59:33 +0200 (CEST) Received: from ideasonboard.com (unknown [95.131.46.153]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CEBB21DD2; Mon, 30 Sep 2024 21:58:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1727726282; bh=nbeI3zAkGJuCUOeTHhU5AUvOYT4Cs/rB4Rg4Fv0C1/M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=o80mdWfEdGbuP+A0h6CtaUGIlmZ1rDn4RIX+u311bZQnXdzNfRgWtlQY0UXDgntdR kFwzSf8r8dIqJrqOKXayDZPrJSyZgHICsgAkI1n620KniNE3GBsbCYlOEK2f43QRMR pGgPhl3OTgNhhpesRLedCRy7mNiXdRspQpCr7Yzk= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Harvey Yang , Yudhistira Erlandinata , Becker Hsieh , Harvey Yang Subject: [PATCH v9 3/5] libcamera: controls: Add ControlTypePoint Date: Mon, 30 Sep 2024 21:59:11 +0200 Message-ID: <20240930195915.152187-4-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20240930195915.152187-1-jacopo.mondi@ideasonboard.com> References: <20240930195915.152187-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Yudhistira Erlandinata Add a control_type<> specialization for libcamera::Point to allow storing data of that type in a ControlValue instance. The new control type will be used by controls introduced in the next patches. Signed-off-by: Yudhistira Erlandinata Co-developed-by: Becker Hsieh Co-developed-by: Harvey Yang Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Reviewed-by: Harvey Yang Reviewed-by: Laurent Pinchart --- include/libcamera/controls.h | 6 ++++++ src/libcamera/controls.cpp | 6 ++++++ 2 files changed, 12 insertions(+) diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h index c5131870d402..ca60bbacad17 100644 --- a/include/libcamera/controls.h +++ b/include/libcamera/controls.h @@ -35,6 +35,7 @@ enum ControlType { ControlTypeString, ControlTypeRectangle, ControlTypeSize, + ControlTypePoint, }; namespace details { @@ -97,6 +98,11 @@ struct control_type { static constexpr std::size_t size = 0; }; +template<> +struct control_type { + static constexpr ControlType value = ControlTypePoint; +}; + template struct control_type> : public control_type> { static constexpr std::size_t size = N; diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp index ea4397302a21..62185d643ecd 100644 --- a/src/libcamera/controls.cpp +++ b/src/libcamera/controls.cpp @@ -60,6 +60,7 @@ static constexpr size_t ControlValueSize[] = { [ControlTypeString] = sizeof(char), [ControlTypeRectangle] = sizeof(Rectangle), [ControlTypeSize] = sizeof(Size), + [ControlTypePoint] = sizeof(Point), }; } /* namespace */ @@ -254,6 +255,11 @@ std::string ControlValue::toString() const str += value->toString(); break; } + case ControlTypePoint: { + const Point *value = reinterpret_cast(data); + str += value->toString(); + break; + } case ControlTypeNone: case ControlTypeString: break; From patchwork Mon Sep 30 19:59:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21444 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 63C64C3257 for ; Mon, 30 Sep 2024 19:59:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B7D2B6351A; Mon, 30 Sep 2024 21:59:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="sH4SSRkW"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DAC176350F for ; Mon, 30 Sep 2024 21:59:34 +0200 (CEST) Received: from ideasonboard.com (unknown [95.131.46.153]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DD63B280; Mon, 30 Sep 2024 21:58:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1727726283; bh=fR/rrzHwpUYNOMGH4aZsQ9vsD19NhkyTXn3VlsHNiyM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sH4SSRkW3Tc0OWvv73pc1JpqpAa2CsxFjlVO6Z5b2CVMj6QR572DP9c0WuEUmwZCI a+1eM0iiw4U1qyJ7SDHeKbK4GC/mnfBQJhbIXtDnutt6j+WKbs72+3VGGwQ7nxxo8I 93AshjamKM5iOz2mWzvJSHUYrZ16Kbz/haIQL8Vg= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Harvey Yang , Yudhistira Erlandinata , Becker Hsieh , Harvey Yang Subject: [PATCH v9 4/5] libcamera: control_ids_draft: Add face detection controls Date: Mon, 30 Sep 2024 21:59:12 +0200 Message-ID: <20240930195915.152187-5-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20240930195915.152187-1-jacopo.mondi@ideasonboard.com> References: <20240930195915.152187-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Yudhistira Erlandinata Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores, and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting FaceDetectFaceLandmark. Signed-off-by: Yudhistira Erlandinata Co-developed-by: Becker Hsieh Co-developed-by: Harvey Yang Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Reviewed-by: Harvey Yang --- src/libcamera/control_ids_draft.yaml | 82 ++++++++++++++++++++++++++++ 1 file changed, 82 insertions(+) diff --git a/src/libcamera/control_ids_draft.yaml b/src/libcamera/control_ids_draft.yaml index 9bef5bf15238..1b284257f601 100644 --- a/src/libcamera/control_ids_draft.yaml +++ b/src/libcamera/control_ids_draft.yaml @@ -227,4 +227,86 @@ controls: value. All of the custom test patterns will be static (that is the raw image must not vary from frame to frame). + - FaceDetectMode: + type: int32_t + description: | + Control to select the face detection mode used by the pipeline. + + Currently identical to ANDROID_STATISTICS_FACE_DETECT_MODE. + + \sa FaceDetectFaceRectangles + \sa FaceDetectFaceScores + \sa FaceDetectFaceLandmarks + \sa FaceDetectFaceIds + + enum: + - name: FaceDetectModeOff + value: 0 + description: | + Pipeline doesn't perform face detection and doesn't report any + control related to face detection. + - name: FaceDetectModeSimple + value: 1 + description: | + Pipeline performs face detection and reports the + FaceDetectFaceRectangles and FaceDetectFaceScores controls for each + detected face. FaceDetectFaceLandmarks and FaceDetectFaceIds are + optional. + - name: FaceDetectModeFull + value: 2 + description: | + Pipeline performs face detection and reports all the controls + related to face detection including FaceDetectFaceRectangles, + FaceDetectFaceScores, FaceDetectFaceLandmarks, and + FaceDeteceFaceIds for each detected face. + + - FaceDetectFaceRectangles: + type: Rectangle + description: | + Boundary rectangles of the detected faces. The number of values is + the number of detected faces. + + The FaceDetectFaceRectangles control can only be returned in metadata. + + Currently identical to ANDROID_STATISTICS_FACE_RECTANGLES. + size: [n] + + - FaceDetectFaceScores: + type: uint8_t + description: | + Confidence score of each of the detected faces. The range of score is + [0, 100]. The number of values should be the number of faces reported + in FaceDetectFaceRectangles. + + The FaceDetectFaceScores control can only be returned in metadata. + + Currently identical to ANDROID_STATISTICS_FACE_SCORES. + size: [n] + + - FaceDetectFaceLandmarks: + type: Point + description: | + Array of human face landmark coordinates in format [..., left_eye_i, + right_eye_i, mouth_i, left_eye_i+1, ...], with i = index of face. The + number of values should be 3 * the number of faces reported in + FaceDetectFaceRectangles. + + The FaceDetectFaceLandmarks control can only be returned in metadata. + + Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS. + size: [n] + + - FaceDetectFaceIds: + type: int32_t + description: | + Each detected face is given a unique ID that is valid for as long as the + face is visible to the camera device. A face that leaves the field of + view and later returns may be assigned a new ID. The number of values + should be the number of faces reported in FaceDetectFaceRectangles. + + The FaceDetectFaceIds control can only be returned in metadata. + + Currently identical to ANDROID_STATISTICS_FACE_IDS. + size: [n] + ... From patchwork Mon Sep 30 19:59:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21443 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8BF1DC3257 for ; Mon, 30 Sep 2024 19:59:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A43866351C; Mon, 30 Sep 2024 21:59:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Z2DtUP4C"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BCFE463519 for ; Mon, 30 Sep 2024 21:59:35 +0200 (CEST) Received: from ideasonboard.com (unknown [95.131.46.153]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 15E112302; Mon, 30 Sep 2024 21:58:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1727726284; bh=sPHQzatYw/HYcmBUg3qDI6+GLHmbXeFy/S5N85m61so=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Z2DtUP4CjI5z/Nkg7Kk6vdoCm9IuFLDtx22oAhUGxmsyrT8iPtFiSjjSnFdfSpH33 5bB8VyiNGnut+1yDbvUcuoirFLSBWj9I3iaPGKNyJS0dm/bR2z+IFI2pEanA9JYWe3 mhFf+qCc0qQUVh+mY5FJjscTmWcJQAJaHKv/ESnQ= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Harvey Yang , Harvey Yang Subject: [PATCH v9 5/5] libcamera: android: Add face detection control support Date: Mon, 30 Sep 2024 21:59:13 +0200 Message-ID: <20240930195915.152187-6-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20240930195915.152187-1-jacopo.mondi@ideasonboard.com> References: <20240930195915.152187-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang Allow Android HAL adapter to pass the face detection metadata control to the pipeline and also send face detection metadata to the camera client if the pipeline generates it. Signed-off-by: Yudhistira Erlandinata Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Reviewed-by: Harvey Yang --- src/android/camera_capabilities.cpp | 44 +++++++++++++++++-- src/android/camera_device.cpp | 66 ++++++++++++++++++++++++++++- 2 files changed, 104 insertions(+), 6 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 71043e127497..b161bc6b3ed6 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -11,6 +11,7 @@ #include #include #include +#include #include #include @@ -1176,11 +1177,46 @@ int CameraCapabilities::initializeStaticMetadata() maxFrameDuration_); /* Statistics static metadata. */ - uint8_t faceDetectMode = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; - staticMetadata_->addEntry(ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES, - faceDetectMode); - int32_t maxFaceCount = 0; + auto iter = camera_->controls().find(controls::draft::FaceDetectMode.id()); + if (iter != camera_->controls().end()) { + const ControlInfo &faceDetectCtrlInfo = iter->second; + std::vector faceDetectModes; + bool hasFaceDetection = false; + + for (const auto &value : faceDetectCtrlInfo.values()) { + int32_t mode = value.get(); + uint8_t androidMode = 0; + + switch (mode) { + case controls::draft::FaceDetectModeOff: + androidMode = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; + break; + case controls::draft::FaceDetectModeSimple: + androidMode = ANDROID_STATISTICS_FACE_DETECT_MODE_SIMPLE; + hasFaceDetection = true; + break; + default: + LOG(HAL, Fatal) << "Received invalid face detect mode: " << mode; + } + faceDetectModes.push_back(androidMode); + } + if (hasFaceDetection) { + /* + * \todo Create new libcamera controls to query max + * possible faces detected. + */ + maxFaceCount = 10; + staticMetadata_->addEntry( + ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES, + faceDetectModes.data(), faceDetectModes.size()); + } + } else { + uint8_t faceDetectMode = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; + staticMetadata_->addEntry(ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES, + faceDetectMode); + } + staticMetadata_->addEntry(ANDROID_STATISTICS_INFO_MAX_FACE_COUNT, maxFaceCount); diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 493f66e7b38f..a038131ae39e 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -22,6 +22,7 @@ #include #include #include +#include #include #include "system/graphics.h" @@ -813,6 +814,11 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) controls.set(controls::ScalerCrop, cropRegion); } + if (settings.getEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, &entry)) { + const uint8_t *data = entry.data.u8; + controls.set(controls::draft::FaceDetectMode, data[0]); + } + if (settings.getEntry(ANDROID_SENSOR_TEST_PATTERN_MODE, &entry)) { const int32_t data = *entry.data.i32; int32_t testPatternMode = controls::draft::TestPatternModeOff; @@ -1540,8 +1546,9 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons value32 = ANDROID_SENSOR_TEST_PATTERN_MODE_OFF; resultMetadata->addEntry(ANDROID_SENSOR_TEST_PATTERN_MODE, value32); - value = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; - resultMetadata->addEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, value); + if (settings.getEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, &entry)) + resultMetadata->addEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, + entry.data.u8, 1); value = ANDROID_STATISTICS_LENS_SHADING_MAP_MODE_OFF; resultMetadata->addEntry(ANDROID_STATISTICS_LENS_SHADING_MAP_MODE, @@ -1580,6 +1587,61 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons resultMetadata->addEntry(ANDROID_SENSOR_FRAME_DURATION, *frameDuration * 1000); + const auto &faceDetectRectangles = + metadata.get(controls::draft::FaceDetectFaceRectangles); + if (faceDetectRectangles) { + std::vector flatRectangles; + for (const Rectangle &rect : *faceDetectRectangles) { + flatRectangles.push_back(rect.x); + flatRectangles.push_back(rect.y); + flatRectangles.push_back(rect.x + rect.width); + flatRectangles.push_back(rect.y + rect.height); + } + resultMetadata->addEntry( + ANDROID_STATISTICS_FACE_RECTANGLES, flatRectangles); + } + + const auto &faceDetectFaceScores = + metadata.get(controls::draft::FaceDetectFaceScores); + if (faceDetectRectangles && faceDetectFaceScores) { + if (faceDetectFaceScores->size() != faceDetectRectangles->size()) { + LOG(HAL, Error) << "Pipeline returned wrong number of face scores; " + << "Expected: " << faceDetectRectangles->size() + << ", got: " << faceDetectFaceScores->size(); + } + resultMetadata->addEntry(ANDROID_STATISTICS_FACE_SCORES, + *faceDetectFaceScores); + } + + const auto &faceDetectFaceLandmarks = + metadata.get(controls::draft::FaceDetectFaceLandmarks); + if (faceDetectRectangles && faceDetectFaceLandmarks) { + size_t expectedLandmarks = faceDetectRectangles->size() * 3; + if (faceDetectFaceLandmarks->size() != expectedLandmarks) { + LOG(HAL, Error) << "Pipeline returned wrong number of face landmarks; " + << "Expected: " << expectedLandmarks + << ", got: " << faceDetectFaceLandmarks->size(); + } + + std::vector androidLandmarks; + for (const Point &landmark : *faceDetectFaceLandmarks) { + androidLandmarks.push_back(landmark.x); + androidLandmarks.push_back(landmark.y); + } + resultMetadata->addEntry( + ANDROID_STATISTICS_FACE_LANDMARKS, androidLandmarks); + } + + const auto &faceDetectFaceIds = metadata.get(controls::draft::FaceDetectFaceIds); + if (faceDetectRectangles && faceDetectFaceIds) { + if (faceDetectFaceIds->size() != faceDetectRectangles->size()) { + LOG(HAL, Error) << "Pipeline returned wrong number of face ids; " + << "Expected: " << faceDetectRectangles->size() + << ", got: " << faceDetectFaceIds->size(); + } + resultMetadata->addEntry(ANDROID_STATISTICS_FACE_IDS, *faceDetectFaceIds); + } + const auto &scalerCrop = metadata.get(controls::ScalerCrop); if (scalerCrop) { const Rectangle &crop = *scalerCrop;