From patchwork Wed Oct 2 05:49:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21466 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 82600BD80A for ; Wed, 2 Oct 2024 05:50:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D6CF963528; Wed, 2 Oct 2024 07:50:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Y/fALEiG"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1366B6350B for ; Wed, 2 Oct 2024 07:50:14 +0200 (CEST) Received: from ideasonboard.com (unknown [5.77.88.238]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 134F263F; Wed, 2 Oct 2024 07:48:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1727848121; bh=ZmVN2B7zYL3NekOhgCe3US8FkMSisJ8m755y49T2gqg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y/fALEiGHdnAh/RY8FFu68kLN89Q1aat5CfXU7DDAqjD/6odJFZXyqQeeloTrb1RX U4D+gmk9RFiGEuENA57lqoBWApMs32rxS1JUdM8yXyFvVqlV21Lb73mlcyWB7UXtQw hEJ9+ZENAZk8p18GojdtTYcHOX3RyOminp5841uI= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Jacopo Mondi , Harvey Yang , David Plowman Subject: [PATCH v10 1/5] libcamera: geometry: Clarify Rectangle's top-left corner Date: Wed, 2 Oct 2024 07:49:58 +0200 Message-ID: <20241002055002.6271-1-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20241001191354.55056-1-jacopo.mondi@ideasonboard.com> References: <20241001191354.55056-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The libcamera::Rectangle class allows defining rectangles regardless of the orientation of the reference system where a rectangle is used in. This implies that, depending on the reference system in use, the rectangle's top-left corner, as defined by libcamera, doesn't correspond to the visual top-left position. ^ | | ------------------- | ^ | h | | | y| o---->------------- | w -------------------------------> (0,0) x (0,0) x ------------------------------> | w y| o---->------------- | | | h | v | | ------------------- | V Clarify that a Rectangle's top-left corner corresponds to the point with the smaller x and y coordinates and that the horizontal and vertical dimensions are obtained by positive increments along the corresponding axes. Signed-off-by: Jacopo Mondi Reviewed-by: Harvey Yang Reviewed-by: David Plowman Reviewed-by: Laurent Pinchart --- src/libcamera/geometry.cpp | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/src/libcamera/geometry.cpp b/src/libcamera/geometry.cpp index 000151364c7f..85a7f53a6f6c 100644 --- a/src/libcamera/geometry.cpp +++ b/src/libcamera/geometry.cpp @@ -594,6 +594,8 @@ std::ostream &operator<<(std::ostream &out, const SizeRange &sr) * * Rectangles are used to identify an area of an image. They are specified by * the coordinates of top-left corner and their horizontal and vertical size. + * By convention, the top-left corner is defined as the corner with the lowest + * x and y coordinates, regardless of the origin and direction of the axes. * * The measure unit of the rectangle coordinates and size, as well as the * reference point from which the Rectangle::x and Rectangle::y displacements @@ -611,6 +613,8 @@ std::ostream &operator<<(std::ostream &out, const SizeRange &sr) * \param[in] x The horizontal coordinate of the top-left corner * \param[in] y The vertical coordinate of the top-left corner * \param[in] size The size + * + * The rectangle's top-left corner is the point with the smaller x and y values. */ /** @@ -620,6 +624,8 @@ std::ostream &operator<<(std::ostream &out, const SizeRange &sr) * \param[in] y The vertical coordinate of the top-left corner * \param[in] width The width * \param[in] height The height + * + * The rectangle's top-left corner is the point with the smaller x and y values. */ /** @@ -632,11 +638,15 @@ std::ostream &operator<<(std::ostream &out, const SizeRange &sr) /** * \var Rectangle::x * \brief The horizontal coordinate of the rectangle's top-left corner + * + * The rectangle's top-left corner is the point with the smaller x and y values. */ /** * \var Rectangle::y * \brief The vertical coordinate of the rectangle's top-left corner + * + * The rectangle's top-left corner is the point with the smaller x and y values. */ /** @@ -685,6 +695,9 @@ Point Rectangle::center() const /** * \fn Point Rectangle::topLeft() const * \brief Retrieve the coordinates of the top left corner of this Rectangle + * + * The rectangle's top-left corner is the point with the smaller x and y values. + * * \return The Rectangle's top left corner */ From patchwork Wed Oct 2 05:49:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21467 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6E156BD80A for ; Wed, 2 Oct 2024 05:50:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2BF0163524; Wed, 2 Oct 2024 07:50:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ZgQ7nnBq"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 502C163512 for ; Wed, 2 Oct 2024 07:50:15 +0200 (CEST) Received: from ideasonboard.com (unknown [5.77.88.238]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C5E72741; Wed, 2 Oct 2024 07:48:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1727848123; bh=Ap9U8t93dlAPWb1HEcjODNBNakr9/cjN29sKX2PRuqo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZgQ7nnBqpDEHg3/sCrdjNjQR34DTpEOnks0TVOg4EavrJrUcxVQ2fpElDOcou0/mO cxUOp4CdJ8XOQS9P4y3YyPnPhP0SrF/XjcGGzTJB+5MxicupLjbl75UnqqxnS40PfK 5NGdlRkghvQDR9k0lIY6iMB55gi/nEtyPGQS5Fvs= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Yudhistira Erlandinata , Harvey Yang , Jacopo Mondi Subject: [PATCH v10 2/5] libcamera: geometry: Add two-point Rectangle constructor Date: Wed, 2 Oct 2024 07:49:59 +0200 Message-ID: <20241002055002.6271-2-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20241002055002.6271-1-jacopo.mondi@ideasonboard.com> References: <20241001191354.55056-1-jacopo.mondi@ideasonboard.com> <20241002055002.6271-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Yudhistira Erlandinata Add a constructor to the Rectangle class that accepts two points. The constructed Rectangle spans all the space between the two given points. Signed-off-by: Yudhistira Erlandinata Co-developed-by: Harvey Yang Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Reviewed-by: Harvey Yang Reviewed-by: Laurent Pinchart --- include/libcamera/geometry.h | 9 +++++++++ src/libcamera/geometry.cpp | 7 +++++++ test/geometry.cpp | 14 ++++++++++++++ 3 files changed, 30 insertions(+) diff --git a/include/libcamera/geometry.h b/include/libcamera/geometry.h index 3e6f0f5d7fab..9ca5865a3d0d 100644 --- a/include/libcamera/geometry.h +++ b/include/libcamera/geometry.h @@ -262,6 +262,15 @@ public: { } + constexpr Rectangle(const Point &point1, const Point &point2) + : Rectangle(std::min(point1.x, point2.x), std::min(point1.y, point2.y), + static_cast(std::max(point1.x, point2.x)) - + static_cast(std::min(point1.x, point2.x)), + static_cast(std::max(point1.y, point2.y)) - + static_cast(std::min(point1.y, point2.y))) + { + } + int x; int y; unsigned int width; diff --git a/src/libcamera/geometry.cpp b/src/libcamera/geometry.cpp index 85a7f53a6f6c..90ccf8c19f97 100644 --- a/src/libcamera/geometry.cpp +++ b/src/libcamera/geometry.cpp @@ -635,6 +635,13 @@ std::ostream &operator<<(std::ostream &out, const SizeRange &sr) * \param[in] size The desired Rectangle size */ +/** + * \fn Rectangle::Rectangle(const Point &point1, const Point &point2) + * \brief Construct a Rectangle from two opposite corners + * \param[in] point1 One of corners of the rectangle + * \param[in] point2 The opposite corner of \a point1 + */ + /** * \var Rectangle::x * \brief The horizontal coordinate of the rectangle's top-left corner diff --git a/test/geometry.cpp b/test/geometry.cpp index 64169206ad16..5760fa3c885a 100644 --- a/test/geometry.cpp +++ b/test/geometry.cpp @@ -481,6 +481,20 @@ protected: return TestFail; } + Point topLeft(3, 3); + Point bottomRight(30, 30); + Point topRight(30, 3); + Point bottomLeft(3, 30); + Rectangle rect1(topLeft, bottomRight); + Rectangle rect2(topRight, bottomLeft); + Rectangle rect3(bottomRight, topLeft); + Rectangle rect4(bottomLeft, topRight); + + if (rect1 != rect2 || rect1 != rect3 || rect1 != rect4) { + cout << "Point-to-point construction failed" << endl; + return TestFail; + } + return TestPass; } }; From patchwork Wed Oct 2 05:50:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21468 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A7EDBBD80A for ; Wed, 2 Oct 2024 05:50:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 192EB63512; Wed, 2 Oct 2024 07:50:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="AqyNzjoQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 752DE63512 for ; Wed, 2 Oct 2024 07:50:16 +0200 (CEST) Received: from ideasonboard.com (unknown [5.77.88.238]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 78AC9A1A; Wed, 2 Oct 2024 07:48:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1727848124; bh=cGo0UfLYVNri6IQT1UX1pG+d9scY8pRhOjGqSSN8CfM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AqyNzjoQnGJofWzPxdaaQORIf//w6SnLQncT3DAnvoCXA7vCPzSI0MWmzy48dq3WS jOgaoWKGDHRPd6q3MzKJd667foxaSopEQnS3R3OJixgVOlHBWs8c4nqt7vloEGMH0t gsci1NkQ1mvl5YSPM05k62IAjaMGiDW2sFn3nHhE= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Yudhistira Erlandinata , Becker Hsieh , Harvey Yang , Jacopo Mondi , Laurent Pinchart Subject: [PATCH v10 3/5] libcamera: controls: Add ControlTypePoint Date: Wed, 2 Oct 2024 07:50:00 +0200 Message-ID: <20241002055002.6271-3-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20241002055002.6271-1-jacopo.mondi@ideasonboard.com> References: <20241001191354.55056-1-jacopo.mondi@ideasonboard.com> <20241002055002.6271-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Yudhistira Erlandinata Add a control_type<> specialization for libcamera::Point to allow storing data of that type in a ControlValue instance. The new control type will be used by controls introduced in the next patches. Signed-off-by: Yudhistira Erlandinata Co-developed-by: Becker Hsieh Co-developed-by: Harvey Yang Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Reviewed-by: Harvey Yang Reviewed-by: Laurent Pinchart --- include/libcamera/controls.h | 6 ++++++ src/libcamera/controls.cpp | 6 ++++++ 2 files changed, 12 insertions(+) diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h index c5131870d402..ca60bbacad17 100644 --- a/include/libcamera/controls.h +++ b/include/libcamera/controls.h @@ -35,6 +35,7 @@ enum ControlType { ControlTypeString, ControlTypeRectangle, ControlTypeSize, + ControlTypePoint, }; namespace details { @@ -97,6 +98,11 @@ struct control_type { static constexpr std::size_t size = 0; }; +template<> +struct control_type { + static constexpr ControlType value = ControlTypePoint; +}; + template struct control_type> : public control_type> { static constexpr std::size_t size = N; diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp index ea4397302a21..62185d643ecd 100644 --- a/src/libcamera/controls.cpp +++ b/src/libcamera/controls.cpp @@ -60,6 +60,7 @@ static constexpr size_t ControlValueSize[] = { [ControlTypeString] = sizeof(char), [ControlTypeRectangle] = sizeof(Rectangle), [ControlTypeSize] = sizeof(Size), + [ControlTypePoint] = sizeof(Point), }; } /* namespace */ @@ -254,6 +255,11 @@ std::string ControlValue::toString() const str += value->toString(); break; } + case ControlTypePoint: { + const Point *value = reinterpret_cast(data); + str += value->toString(); + break; + } case ControlTypeNone: case ControlTypeString: break; From patchwork Wed Oct 2 05:50:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21469 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 91A2ABD80A for ; Wed, 2 Oct 2024 05:50:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 54EE56352E; Wed, 2 Oct 2024 07:50:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="LeXr2cWm"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7220063529 for ; Wed, 2 Oct 2024 07:50:18 +0200 (CEST) Received: from ideasonboard.com (unknown [5.77.88.238]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CAFA6D52; Wed, 2 Oct 2024 07:48:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1727848125; bh=hcHSpTNPYJhc+NYbm+iZH/MUXm0wgDwtSsdUPpWvPXs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LeXr2cWmqRu9ZQPT86AFkyovIqJQoDmCFsi5YqY3JEkejIq/IUfYGM14Qci0xFcXo oi6ZBUHqzo6ZT1z+jGyIIArWgXpqgL+BO9/XStMG8FlrlweJl1zUT+jBoLdP8JXVrW 5EITIYgPOJyIFxIEX3vFz2rUfTuTzcOX7TtXJnZI= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Yudhistira Erlandinata , Becker Hsieh , Harvey Yang , Jacopo Mondi Subject: [PATCH v10 4/5] libcamera: control_ids_draft: Add face detection controls Date: Wed, 2 Oct 2024 07:50:01 +0200 Message-ID: <20241002055002.6271-4-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20241002055002.6271-1-jacopo.mondi@ideasonboard.com> References: <20241001191354.55056-1-jacopo.mondi@ideasonboard.com> <20241002055002.6271-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Yudhistira Erlandinata Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores, and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting FaceDetectFaceLandmark. Signed-off-by: Yudhistira Erlandinata Co-developed-by: Becker Hsieh Co-developed-by: Harvey Yang Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Reviewed-by: Harvey Yang --- src/libcamera/control_ids_draft.yaml | 82 ++++++++++++++++++++++++++++ 1 file changed, 82 insertions(+) diff --git a/src/libcamera/control_ids_draft.yaml b/src/libcamera/control_ids_draft.yaml index 9bef5bf15238..1b284257f601 100644 --- a/src/libcamera/control_ids_draft.yaml +++ b/src/libcamera/control_ids_draft.yaml @@ -227,4 +227,86 @@ controls: value. All of the custom test patterns will be static (that is the raw image must not vary from frame to frame). + - FaceDetectMode: + type: int32_t + description: | + Control to select the face detection mode used by the pipeline. + + Currently identical to ANDROID_STATISTICS_FACE_DETECT_MODE. + + \sa FaceDetectFaceRectangles + \sa FaceDetectFaceScores + \sa FaceDetectFaceLandmarks + \sa FaceDetectFaceIds + + enum: + - name: FaceDetectModeOff + value: 0 + description: | + Pipeline doesn't perform face detection and doesn't report any + control related to face detection. + - name: FaceDetectModeSimple + value: 1 + description: | + Pipeline performs face detection and reports the + FaceDetectFaceRectangles and FaceDetectFaceScores controls for each + detected face. FaceDetectFaceLandmarks and FaceDetectFaceIds are + optional. + - name: FaceDetectModeFull + value: 2 + description: | + Pipeline performs face detection and reports all the controls + related to face detection including FaceDetectFaceRectangles, + FaceDetectFaceScores, FaceDetectFaceLandmarks, and + FaceDeteceFaceIds for each detected face. + + - FaceDetectFaceRectangles: + type: Rectangle + description: | + Boundary rectangles of the detected faces. The number of values is + the number of detected faces. + + The FaceDetectFaceRectangles control can only be returned in metadata. + + Currently identical to ANDROID_STATISTICS_FACE_RECTANGLES. + size: [n] + + - FaceDetectFaceScores: + type: uint8_t + description: | + Confidence score of each of the detected faces. The range of score is + [0, 100]. The number of values should be the number of faces reported + in FaceDetectFaceRectangles. + + The FaceDetectFaceScores control can only be returned in metadata. + + Currently identical to ANDROID_STATISTICS_FACE_SCORES. + size: [n] + + - FaceDetectFaceLandmarks: + type: Point + description: | + Array of human face landmark coordinates in format [..., left_eye_i, + right_eye_i, mouth_i, left_eye_i+1, ...], with i = index of face. The + number of values should be 3 * the number of faces reported in + FaceDetectFaceRectangles. + + The FaceDetectFaceLandmarks control can only be returned in metadata. + + Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS. + size: [n] + + - FaceDetectFaceIds: + type: int32_t + description: | + Each detected face is given a unique ID that is valid for as long as the + face is visible to the camera device. A face that leaves the field of + view and later returns may be assigned a new ID. The number of values + should be the number of faces reported in FaceDetectFaceRectangles. + + The FaceDetectFaceIds control can only be returned in metadata. + + Currently identical to ANDROID_STATISTICS_FACE_IDS. + size: [n] + ... From patchwork Wed Oct 2 05:50:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21470 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4C232BD80A for ; Wed, 2 Oct 2024 05:50:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 45CDE6352D; Wed, 2 Oct 2024 07:50:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="qc2XapjE"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DD72A6352A for ; Wed, 2 Oct 2024 07:50:18 +0200 (CEST) Received: from ideasonboard.com (unknown [5.77.88.238]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 330A663F; Wed, 2 Oct 2024 07:48:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1727848126; bh=+NwwauNPwrHz7nmY/bKT5sxXDeiCk8+5e/M63V5w8Ks=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qc2XapjE+5qy8eehHevadT3ppEyLVAAOCcDX7iGzHYjq/hlZ/dUDCkAhnYlMwcJKC nVr9+HWwk8B827fBxFd3HZKVtFlr/lywqTmyZhY9IbB4PGsvmofbte77hITX2DKGYg eomERZVmVmAAHIzg3JcZeb31EOOxmd9wzy6Jfl6w= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Harvey Yang , Harvey Yang , Jacopo Mondi Subject: [PATCH v10 5/5] libcamera: android: Add face detection control support Date: Wed, 2 Oct 2024 07:50:02 +0200 Message-ID: <20241002055002.6271-5-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20241002055002.6271-1-jacopo.mondi@ideasonboard.com> References: <20241001191354.55056-1-jacopo.mondi@ideasonboard.com> <20241002055002.6271-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang Allow Android HAL adapter to pass the face detection metadata control to the pipeline and also send face detection metadata to the camera client if the pipeline generates it. Signed-off-by: Yudhistira Erlandinata Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Reviewed-by: Harvey Yang --- src/android/camera_capabilities.cpp | 44 +++++++++++++++++-- src/android/camera_device.cpp | 66 ++++++++++++++++++++++++++++- 2 files changed, 104 insertions(+), 6 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 71043e127497..b161bc6b3ed6 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -11,6 +11,7 @@ #include #include #include +#include #include #include @@ -1176,11 +1177,46 @@ int CameraCapabilities::initializeStaticMetadata() maxFrameDuration_); /* Statistics static metadata. */ - uint8_t faceDetectMode = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; - staticMetadata_->addEntry(ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES, - faceDetectMode); - int32_t maxFaceCount = 0; + auto iter = camera_->controls().find(controls::draft::FaceDetectMode.id()); + if (iter != camera_->controls().end()) { + const ControlInfo &faceDetectCtrlInfo = iter->second; + std::vector faceDetectModes; + bool hasFaceDetection = false; + + for (const auto &value : faceDetectCtrlInfo.values()) { + int32_t mode = value.get(); + uint8_t androidMode = 0; + + switch (mode) { + case controls::draft::FaceDetectModeOff: + androidMode = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; + break; + case controls::draft::FaceDetectModeSimple: + androidMode = ANDROID_STATISTICS_FACE_DETECT_MODE_SIMPLE; + hasFaceDetection = true; + break; + default: + LOG(HAL, Fatal) << "Received invalid face detect mode: " << mode; + } + faceDetectModes.push_back(androidMode); + } + if (hasFaceDetection) { + /* + * \todo Create new libcamera controls to query max + * possible faces detected. + */ + maxFaceCount = 10; + staticMetadata_->addEntry( + ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES, + faceDetectModes.data(), faceDetectModes.size()); + } + } else { + uint8_t faceDetectMode = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; + staticMetadata_->addEntry(ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES, + faceDetectMode); + } + staticMetadata_->addEntry(ANDROID_STATISTICS_INFO_MAX_FACE_COUNT, maxFaceCount); diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 493f66e7b38f..a038131ae39e 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -22,6 +22,7 @@ #include #include #include +#include #include #include "system/graphics.h" @@ -813,6 +814,11 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) controls.set(controls::ScalerCrop, cropRegion); } + if (settings.getEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, &entry)) { + const uint8_t *data = entry.data.u8; + controls.set(controls::draft::FaceDetectMode, data[0]); + } + if (settings.getEntry(ANDROID_SENSOR_TEST_PATTERN_MODE, &entry)) { const int32_t data = *entry.data.i32; int32_t testPatternMode = controls::draft::TestPatternModeOff; @@ -1540,8 +1546,9 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons value32 = ANDROID_SENSOR_TEST_PATTERN_MODE_OFF; resultMetadata->addEntry(ANDROID_SENSOR_TEST_PATTERN_MODE, value32); - value = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; - resultMetadata->addEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, value); + if (settings.getEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, &entry)) + resultMetadata->addEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, + entry.data.u8, 1); value = ANDROID_STATISTICS_LENS_SHADING_MAP_MODE_OFF; resultMetadata->addEntry(ANDROID_STATISTICS_LENS_SHADING_MAP_MODE, @@ -1580,6 +1587,61 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons resultMetadata->addEntry(ANDROID_SENSOR_FRAME_DURATION, *frameDuration * 1000); + const auto &faceDetectRectangles = + metadata.get(controls::draft::FaceDetectFaceRectangles); + if (faceDetectRectangles) { + std::vector flatRectangles; + for (const Rectangle &rect : *faceDetectRectangles) { + flatRectangles.push_back(rect.x); + flatRectangles.push_back(rect.y); + flatRectangles.push_back(rect.x + rect.width); + flatRectangles.push_back(rect.y + rect.height); + } + resultMetadata->addEntry( + ANDROID_STATISTICS_FACE_RECTANGLES, flatRectangles); + } + + const auto &faceDetectFaceScores = + metadata.get(controls::draft::FaceDetectFaceScores); + if (faceDetectRectangles && faceDetectFaceScores) { + if (faceDetectFaceScores->size() != faceDetectRectangles->size()) { + LOG(HAL, Error) << "Pipeline returned wrong number of face scores; " + << "Expected: " << faceDetectRectangles->size() + << ", got: " << faceDetectFaceScores->size(); + } + resultMetadata->addEntry(ANDROID_STATISTICS_FACE_SCORES, + *faceDetectFaceScores); + } + + const auto &faceDetectFaceLandmarks = + metadata.get(controls::draft::FaceDetectFaceLandmarks); + if (faceDetectRectangles && faceDetectFaceLandmarks) { + size_t expectedLandmarks = faceDetectRectangles->size() * 3; + if (faceDetectFaceLandmarks->size() != expectedLandmarks) { + LOG(HAL, Error) << "Pipeline returned wrong number of face landmarks; " + << "Expected: " << expectedLandmarks + << ", got: " << faceDetectFaceLandmarks->size(); + } + + std::vector androidLandmarks; + for (const Point &landmark : *faceDetectFaceLandmarks) { + androidLandmarks.push_back(landmark.x); + androidLandmarks.push_back(landmark.y); + } + resultMetadata->addEntry( + ANDROID_STATISTICS_FACE_LANDMARKS, androidLandmarks); + } + + const auto &faceDetectFaceIds = metadata.get(controls::draft::FaceDetectFaceIds); + if (faceDetectRectangles && faceDetectFaceIds) { + if (faceDetectFaceIds->size() != faceDetectRectangles->size()) { + LOG(HAL, Error) << "Pipeline returned wrong number of face ids; " + << "Expected: " << faceDetectRectangles->size() + << ", got: " << faceDetectFaceIds->size(); + } + resultMetadata->addEntry(ANDROID_STATISTICS_FACE_IDS, *faceDetectFaceIds); + } + const auto &scalerCrop = metadata.get(controls::ScalerCrop); if (scalerCrop) { const Rectangle &crop = *scalerCrop;