From patchwork Fri Nov 8 10:51:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21867 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4FB66BE173 for ; Fri, 8 Nov 2024 10:51:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 71E3D657A8; Fri, 8 Nov 2024 11:51:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Q105kilL"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5084065495 for ; Fri, 8 Nov 2024 11:51:25 +0100 (CET) Received: from ideasonboard.com (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8F3062C5; Fri, 8 Nov 2024 11:51:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731063075; bh=WYEJPIBvC/NBM3WoE0+7WLf031OG5u2aEzw0uhaVJJI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Q105kilL2NFF8l8mq9JXseqd4cWXejTfUtMCxEa8oabtP1J7HPpA9eGmquFe7gjT6 SVBI40Yn7gxDpxeNFpEalul1AQFUHLdE+xqKfjCESJ7SZ6nVX8mlzn9wNxC614RKVq RFVmv0ObAizGi6ZGWof2CMNSy0GCUp8KGavibD28= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , naush@raspberrypi.com, Kieran Bingham Subject: [PATCH v2 1/6] include: linux: Update headers for line-based embedded data support Date: Fri, 8 Nov 2024 11:51:11 +0100 Message-ID: <20241108105117.137121-2-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241108105117.137121-1-jacopo.mondi@ideasonboard.com> References: <20241108105117.137121-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Update the kernel headers with the definition of two device-specific line-based metadata formats, and with the definition of the MEDIA_PAD_FL_INTERNAL and V4L2_SUBDEV_ROUTE_FL_IMMUTABLE flags. The new definitions will allow to support handling line-based metadata streams exposed by the sensor driver through an internal sink pad. While the changes have not yet been collected in the official linux-media tree, they're available in the 'metadata' branch of https://git.linuxtv.org/sailus/media_tree.git, at revision: 8d8fdc8a8809 ("media: Documentation: Add subdev configuration models, raw sensor model"). The patches are expected to land in Linux v6.13. Signed-off-by: Jacopo Mondi Acked-by: Kieran Bingham --- include/linux/README | 2 +- include/linux/media-bus-format.h | 4 ++++ include/linux/media.h | 1 + include/linux/v4l2-subdev.h | 5 +++++ 4 files changed, 11 insertions(+), 1 deletion(-) diff --git a/include/linux/README b/include/linux/README index ef178681f6eb..077eaba30b42 100644 --- a/include/linux/README +++ b/include/linux/README @@ -1,4 +1,4 @@ # SPDX-License-Identifier: CC0-1.0 -Files in this directory are imported from next-media-rkisp1-20240814-14-ga043ea54bbb9 of the Linux kernel. Do not +Files in this directory are imported from v6.12-rc1-56-g8d8fdc8a8809 of the Linux kernel. Do not modify them manually. diff --git a/include/linux/media-bus-format.h b/include/linux/media-bus-format.h index d4c1d991014b..13e68c2ccb61 100644 --- a/include/linux/media-bus-format.h +++ b/include/linux/media-bus-format.h @@ -183,4 +183,8 @@ #define MEDIA_BUS_FMT_META_20 0x8006 #define MEDIA_BUS_FMT_META_24 0x8007 +/* Specific metadata formats. Next is 0x9003. */ +#define MEDIA_BUS_FMT_CCS_EMBEDDED 0x9001 +#define MEDIA_BUS_FMT_OV2740_EMBEDDED 0x9002 + #endif /* __LINUX_MEDIA_BUS_FORMAT_H */ diff --git a/include/linux/media.h b/include/linux/media.h index b5a77bbf4062..4a733b9beb27 100644 --- a/include/linux/media.h +++ b/include/linux/media.h @@ -206,6 +206,7 @@ struct media_entity_desc { #define MEDIA_PAD_FL_SINK (1U << 0) #define MEDIA_PAD_FL_SOURCE (1U << 1) #define MEDIA_PAD_FL_MUST_CONNECT (1U << 2) +#define MEDIA_PAD_FL_INTERNAL (1U << 3) struct media_pad_desc { __u32 entity; /* entity ID */ diff --git a/include/linux/v4l2-subdev.h b/include/linux/v4l2-subdev.h index 2347e266cf75..839b1329afb2 100644 --- a/include/linux/v4l2-subdev.h +++ b/include/linux/v4l2-subdev.h @@ -204,6 +204,11 @@ struct v4l2_subdev_capability { * on a video node. */ #define V4L2_SUBDEV_ROUTE_FL_ACTIVE (1U << 0) +/* + * Is the route immutable? The ACTIVE flag of an immutable route may not be + * unset. + */ +#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE (1U << 1) /** * struct v4l2_subdev_route - A route inside a subdev From patchwork Fri Nov 8 10:51:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21868 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EFFA2BE173 for ; Fri, 8 Nov 2024 10:51:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8E81D657AE; Fri, 8 Nov 2024 11:51:32 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HllcnW4s"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 93E2C60589 for ; Fri, 8 Nov 2024 11:51:25 +0100 (CET) Received: from ideasonboard.com (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F394D7E9; Fri, 8 Nov 2024 11:51:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731063076; bh=9RDNizsjn3o2mwoKS2stCXuUdQGqHVcv8Vx79zQ7o4s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HllcnW4sQY/15+yt+uFp4OAtx7sixDEkA4tLRH8zcvDcVcKskn7p+yzo2K2wSv6Hi bzVCzwZaZ1JRulhC2FG+WUYzbO1cBYyW3skRcv23SGSTmFdYSjexi9fDjJo8XVpYeE Dj46tKuivOFNtLYIBjmERDHK96P+x+cgDzmGYXDY= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , naush@raspberrypi.com Subject: [PATCH v2 2/6] include: linux: videodev2: Add generic line based pixel formats Date: Fri, 8 Nov 2024 11:51:12 +0100 Message-ID: <20241108105117.137121-3-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241108105117.137121-1-jacopo.mondi@ideasonboard.com> References: <20241108105117.137121-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the definition for the generic line based pixelformats. The formats has been added in upstream Linux by commit 1d9215233958 ("media: uapi: v4l: Add generic 8-bit metadata format definitions") which got merged in Linux v6.10. The formats however are not yet available to userspace, as they have been made only available to the kernel by commit d69c8429ea80 ("media: uapi: v4l: Don't expose generic metadata formats to userspace") to let the line-based metadata support stabilize before allowing applications to use it. With the forthcoming completion of the line-based metadata upstreaming manually add the generic line based pixel format to prepare libcamera to support them. Signed-off-by: Jacopo Mondi --- include/linux/videodev2.h | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h index 3829c0b60f52..1168f6ff40eb 100644 --- a/include/linux/videodev2.h +++ b/include/linux/videodev2.h @@ -840,6 +840,18 @@ struct v4l2_pix_format { /* The metadata format identifier for FE stats buffers. */ #define V4L2_META_FMT_RPI_FE_STATS v4l2_fourcc('R', 'P', 'F', 'S') +/* + * Line-based metadata formats. Remember to update v4l_fill_fmtdesc() when + * adding new ones! + */ +#define V4L2_META_FMT_GENERIC_8 v4l2_fourcc('M', 'E', 'T', '8') /* Generic 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_10 v4l2_fourcc('M', 'C', '1', 'A') /* 10-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_12 v4l2_fourcc('M', 'C', '1', 'C') /* 12-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_14 v4l2_fourcc('M', 'C', '1', 'E') /* 14-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_16 v4l2_fourcc('M', 'C', '1', 'G') /* 16-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_20 v4l2_fourcc('M', 'C', '1', 'K') /* 20-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_24 v4l2_fourcc('M', 'C', '1', 'O') /* 24-bit CSI-2 packed 8-bit metadata */ + /* priv field value to indicates that subsequent fields are valid. */ #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe From patchwork Fri Nov 8 10:51:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21870 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A2364BE173 for ; Fri, 8 Nov 2024 10:51:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1A8D0657A6; Fri, 8 Nov 2024 11:51:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="eKJjQyXQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C600965496 for ; Fri, 8 Nov 2024 11:51:26 +0100 (CET) Received: from ideasonboard.com (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 564849FF; Fri, 8 Nov 2024 11:51:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731063076; bh=FaUyCGztuMJ8PVzCxwgAOpRhi+P4dSLsfEtb8WP6tKk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eKJjQyXQjBLtfiPChEiHT7GfCDecCYkHS8mj5Q4qawihp2b0hgxBOHdnQVu+tX4S5 N1WaRnnWn3ErfGtfK9crX7ACYFAlkvB0ZBZRIbEgKmGUqkpxAO29jR8htu/RhVbRFg b2fKfGp4CzVDUpbK74wOhTrtinrZc4rhxltafEPM= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , naush@raspberrypi.com, Laurent Pinchart Subject: [PATCH v2 3/6] libcamera: v4l2_subdevice: Add new metadata formats Date: Fri, 8 Nov 2024 11:51:13 +0100 Message-ID: <20241108105117.137121-4-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241108105117.137121-1-jacopo.mondi@ideasonboard.com> References: <20241108105117.137121-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart Support the newly introduced V4L2 media bus formats for metadata. This includes generic metadata formats, and two sensor-specific embedded data formats. Signed-off-by: Laurent Pinchart Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- src/libcamera/v4l2_subdevice.cpp | 63 ++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 9f2ec4798407..f65dd6593c99 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -700,6 +700,69 @@ const std::map mediaBusFormatInfo{ .bitsPerPixel = 0, .colourEncoding = PixelFormatInfo::ColourEncodingRAW, } }, + { MEDIA_BUS_FMT_META_8, { + .name = "META_8", + .code = MEDIA_BUS_FMT_META_8, + .type = MediaBusFormatInfo::Type::Metadata, + .bitsPerPixel = 8, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW, + } }, + { MEDIA_BUS_FMT_META_10, { + .name = "META_10", + .code = MEDIA_BUS_FMT_META_10, + .type = MediaBusFormatInfo::Type::Metadata, + .bitsPerPixel = 10, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW, + } }, + { MEDIA_BUS_FMT_META_12, { + .name = "META_12", + .code = MEDIA_BUS_FMT_META_12, + .type = MediaBusFormatInfo::Type::Metadata, + .bitsPerPixel = 12, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW, + } }, + { MEDIA_BUS_FMT_META_14, { + .name = "META_14", + .code = MEDIA_BUS_FMT_META_14, + .type = MediaBusFormatInfo::Type::Metadata, + .bitsPerPixel = 14, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW, + } }, + { MEDIA_BUS_FMT_META_16, { + .name = "META_16", + .code = MEDIA_BUS_FMT_META_16, + .type = MediaBusFormatInfo::Type::Metadata, + .bitsPerPixel = 16, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW, + } }, + { MEDIA_BUS_FMT_META_20, { + .name = "META_20", + .code = MEDIA_BUS_FMT_META_20, + .type = MediaBusFormatInfo::Type::Metadata, + .bitsPerPixel = 20, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW, + } }, + { MEDIA_BUS_FMT_META_24, { + .name = "META_24", + .code = MEDIA_BUS_FMT_META_24, + .type = MediaBusFormatInfo::Type::Metadata, + .bitsPerPixel = 24, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW, + } }, + { MEDIA_BUS_FMT_CCS_EMBEDDED, { + .name = "CCS_EMBEDDED", + .code = MEDIA_BUS_FMT_CCS_EMBEDDED, + .type = MediaBusFormatInfo::Type::EmbeddedData, + .bitsPerPixel = 0, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW, + } }, + { MEDIA_BUS_FMT_OV2740_EMBEDDED, { + .name = "OV2740_EMBEDDED", + .code = MEDIA_BUS_FMT_CCS_EMBEDDED, + .type = MediaBusFormatInfo::Type::EmbeddedData, + .bitsPerPixel = 0, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW, + } }, }; } /* namespace */ From patchwork Fri Nov 8 10:51:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21869 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 52D7EC32A9 for ; Fri, 8 Nov 2024 10:51:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 52CF6657B0; Fri, 8 Nov 2024 11:51:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nFyFg5Oe"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 020D7654A0 for ; Fri, 8 Nov 2024 11:51:27 +0100 (CET) Received: from ideasonboard.com (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B9ACC2C5; Fri, 8 Nov 2024 11:51:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731063077; bh=saaBta40gmnAT0S3kuz2Tbk5SFo3Q/UvKdChhOnHZ0U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nFyFg5OesU80l7EwvvbJl7FYFez210znJeUjDJCKOvpAvEiy4VX4066JvcED+Fyen gwsNK8etwaKf1V4jUA4krAzMU3AsMXl61pozEp3g+8PZRuw8Mfj0qIawhfKgrCfHQ8 NRDPAThBcbAlCdhmuMwmzIzcTC2de+c1305Me1+I= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , naush@raspberrypi.com, Laurent Pinchart Subject: [PATCH v2 4/6] libcamera: v4l2_videodevice: Update to the new kernel metadata API Date: Fri, 8 Nov 2024 11:51:14 +0100 Message-ID: <20241108105117.137121-5-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241108105117.137121-1-jacopo.mondi@ideasonboard.com> References: <20241108105117.137121-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart With support for metadata in the streams API, the v4l2_meta_format structure has been extended with width, height and bytesperline fields. Support them in the V4L2VideoDevice getFormat() and setFormat() functions is the video device is meta capture device and if the pixel format is one of the generic line-based metadata formats. Signed-off-by: Laurent Pinchart Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- include/libcamera/internal/v4l2_pixelformat.h | 2 + src/libcamera/v4l2_pixelformat.cpp | 34 +++++++++++++ src/libcamera/v4l2_videodevice.cpp | 51 ++++++++++++++----- 3 files changed, 73 insertions(+), 14 deletions(-) diff --git a/include/libcamera/internal/v4l2_pixelformat.h b/include/libcamera/internal/v4l2_pixelformat.h index c836346bdc04..543eb21b3c07 100644 --- a/include/libcamera/internal/v4l2_pixelformat.h +++ b/include/libcamera/internal/v4l2_pixelformat.h @@ -49,6 +49,8 @@ public: static const std::vector & fromPixelFormat(const PixelFormat &pixelFormat); + bool isGenericLineBasedMetadata() const; + private: uint32_t fourcc_; }; diff --git a/src/libcamera/v4l2_pixelformat.cpp b/src/libcamera/v4l2_pixelformat.cpp index eb9ac2224fd1..e8b3eb9c1394 100644 --- a/src/libcamera/v4l2_pixelformat.cpp +++ b/src/libcamera/v4l2_pixelformat.cpp @@ -372,6 +372,40 @@ V4L2PixelFormat::fromPixelFormat(const PixelFormat &pixelFormat) return info.v4l2Formats; } +/** + * \brief Test if a V4L2PixelFormat is one of the line based generic metadata + * formats + * + * A limited number of metadata formats, the ones that represents generic + * line-based metadata buffers, need to have their width, height and + * bytesperline set by userspace. + * + * This function tests if the current V4L2PixelFormat is one of those. + * + * Note: It would have been nicer to store this information in a + * V4L2PixelFormat::Info instance, but as metadata format are not exposed to + * applications, there are no PixelFormat and DRM fourcc codes associated to + * them. + * + * \return True if the V4L2PixelFormat() is a generic line based format, false + * otherwise + */ +bool V4L2PixelFormat::isGenericLineBasedMetadata() const +{ + switch (fourcc_) { + case V4L2_META_FMT_GENERIC_8: + case V4L2_META_FMT_GENERIC_CSI2_10: + case V4L2_META_FMT_GENERIC_CSI2_12: + case V4L2_META_FMT_GENERIC_CSI2_14: + case V4L2_META_FMT_GENERIC_CSI2_16: + case V4L2_META_FMT_GENERIC_CSI2_20: + case V4L2_META_FMT_GENERIC_CSI2_24: + return true; + default: + return false; + } +} + /** * \brief Insert a text representation of a V4L2PixelFormat into an output * stream diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 14eba0561d6a..87cdbc3edf52 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -888,7 +888,7 @@ int V4L2VideoDevice::setFormat(V4L2DeviceFormat *format) int V4L2VideoDevice::getFormatMeta(V4L2DeviceFormat *format) { struct v4l2_format v4l2Format = {}; - struct v4l2_meta_format *pix = &v4l2Format.fmt.meta; + struct v4l2_meta_format *meta = &v4l2Format.fmt.meta; int ret; v4l2Format.type = bufferType_; @@ -898,25 +898,42 @@ int V4L2VideoDevice::getFormatMeta(V4L2DeviceFormat *format) return ret; } - format->size.width = 0; - format->size.height = 0; - format->fourcc = V4L2PixelFormat(pix->dataformat); + format->fourcc = V4L2PixelFormat(meta->dataformat); + format->planes[0].size = meta->buffersize; format->planesCount = 1; - format->planes[0].bpl = pix->buffersize; - format->planes[0].size = pix->buffersize; + + bool genericLineBased = caps_.isMetaCapture() && + format->fourcc.isGenericLineBasedMetadata(); + + if (genericLineBased) { + format->size.width = meta->width; + format->size.height = meta->height; + format->planes[0].bpl = meta->bytesperline; + } else { + format->size.width = 0; + format->size.height = 0; + format->planes[0].bpl = meta->buffersize; + } return 0; } int V4L2VideoDevice::trySetFormatMeta(V4L2DeviceFormat *format, bool set) { + bool genericLineBased = caps_.isMetaCapture() && + format->fourcc.isGenericLineBasedMetadata(); struct v4l2_format v4l2Format = {}; - struct v4l2_meta_format *pix = &v4l2Format.fmt.meta; + struct v4l2_meta_format *meta = &v4l2Format.fmt.meta; int ret; v4l2Format.type = bufferType_; - pix->dataformat = format->fourcc; - pix->buffersize = format->planes[0].size; + meta->dataformat = format->fourcc; + meta->buffersize = format->planes[0].size; + if (genericLineBased) { + meta->width = format->size.width; + meta->height = format->size.height; + meta->bytesperline = format->planes[0].bpl; + } ret = ioctl(set ? VIDIOC_S_FMT : VIDIOC_TRY_FMT, &v4l2Format); if (ret) { LOG(V4L2, Error) @@ -929,12 +946,18 @@ int V4L2VideoDevice::trySetFormatMeta(V4L2DeviceFormat *format, bool set) * Return to caller the format actually applied on the video device, * which might differ from the requested one. */ - format->size.width = 0; - format->size.height = 0; - format->fourcc = V4L2PixelFormat(pix->dataformat); + format->fourcc = V4L2PixelFormat(meta->dataformat); format->planesCount = 1; - format->planes[0].bpl = pix->buffersize; - format->planes[0].size = pix->buffersize; + format->planes[0].size = meta->buffersize; + if (genericLineBased) { + format->size.width = meta->width; + format->size.height = meta->height; + format->planes[0].bpl = meta->bytesperline; + } else { + format->size.width = 0; + format->size.height = 0; + format->planes[0].bpl = meta->buffersize; + } return 0; } From patchwork Fri Nov 8 10:51:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21871 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0096DC32A9 for ; Fri, 8 Nov 2024 10:51:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 97C66657B4; Fri, 8 Nov 2024 11:51:35 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="oBiPQ4bv"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2C8A1654A1 for ; Fri, 8 Nov 2024 11:51:27 +0100 (CET) Received: from ideasonboard.com (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 28327353; Fri, 8 Nov 2024 11:51:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731063077; bh=6h1yEp6aQVkBEMxGMDPPOY+Z9mmfD6+Q/iTEzRq15zA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oBiPQ4bv+2LdjhnvvvZldzj1IURCT18Xv9V+1rfDHgVJhSQfZU1Jf3LrB1zqv/hct MyGqeFVqiBSLpZsog9Jz2Y9/eVJ/Jx0b8eAN6AEGbPOIlXagC/2aT3RhXdmtfD9gfb fR6LLeA0xOZ4UL9FwbrGO9SzYNCcFqlFstouMmSk= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , naush@raspberrypi.com, Laurent Pinchart Subject: [PATCH v2 5/6] libcamera: Add CameraSensor implementation for raw V4L2 sensors Date: Fri, 8 Nov 2024 11:51:15 +0100 Message-ID: <20241108105117.137121-6-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241108105117.137121-1-jacopo.mondi@ideasonboard.com> References: <20241108105117.137121-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart Add a new CameraSensorRaw implementation of the CameraSensor interface tailored to devices that implement the new V4L2 raw camera sensors API. This new class duplicates code from the CameraSensorLegacy class. The two classes will be refactored to share code. Signed-off-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- Changes since combined RFC: - Use std::abs() from cmath instead that fabs() from math.h - Set factory priority explicitly - Set flipsAlterBayerOrder_ - Fix build warning due to missing default case in cfa switch - Check for read-only hblank using V4L2_CTRL_FLAG_READ_ONLY --- Documentation/Doxyfile-internal.in | 1 + src/libcamera/sensor/camera_sensor_raw.cpp | 1055 ++++++++++++++++++++ src/libcamera/sensor/meson.build | 1 + 3 files changed, 1057 insertions(+) create mode 100644 src/libcamera/sensor/camera_sensor_raw.cpp diff --git a/Documentation/Doxyfile-internal.in b/Documentation/Doxyfile-internal.in index b5ad7e7ff6c7..5343bc2b131c 100644 --- a/Documentation/Doxyfile-internal.in +++ b/Documentation/Doxyfile-internal.in @@ -26,6 +26,7 @@ EXCLUDE = @TOP_SRCDIR@/include/libcamera/base/span.h \ @TOP_SRCDIR@/src/libcamera/ipc_pipe_unixsocket.cpp \ @TOP_SRCDIR@/src/libcamera/pipeline/ \ @TOP_SRCDIR@/src/libcamera/sensor/camera_sensor_legacy.cpp \ + @TOP_SRCDIR@/src/libcamera/sensor/camera_sensor_raw.cpp \ @TOP_SRCDIR@/src/libcamera/tracepoints.cpp \ @TOP_BUILDDIR@/include/libcamera/internal/tracepoints.h \ @TOP_BUILDDIR@/include/libcamera/ipa/soft_ipa_interface.h \ diff --git a/src/libcamera/sensor/camera_sensor_raw.cpp b/src/libcamera/sensor/camera_sensor_raw.cpp new file mode 100644 index 000000000000..4c653121d547 --- /dev/null +++ b/src/libcamera/sensor/camera_sensor_raw.cpp @@ -0,0 +1,1055 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas on Board Oy. + * + * camera_sensor_raw.cpp - A raw camera sensor using the V4L2 streams API + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/camera_lens.h" +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/camera_sensor_properties.h" +#include "libcamera/internal/formats.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/sysfs.h" +#include "libcamera/internal/v4l2_subdevice.h" + +namespace libcamera { + +class BayerFormat; +class CameraLens; +class MediaEntity; +class SensorConfiguration; + +struct CameraSensorProperties; + +enum class Orientation; + +LOG_DECLARE_CATEGORY(CameraSensor) + +class CameraSensorRaw : public CameraSensor, protected Loggable +{ +public: + CameraSensorRaw(const MediaEntity *entity); + ~CameraSensorRaw(); + + static std::variant, int> + match(MediaEntity *entity); + + const std::string &model() const override { return model_; } + const std::string &id() const override { return id_; } + + const MediaEntity *entity() const override { return entity_; } + V4L2Subdevice *device() override { return subdev_.get(); } + + CameraLens *focusLens() override { return focusLens_.get(); } + + const std::vector &mbusCodes() const override { return mbusCodes_; } + std::vector sizes(unsigned int mbusCode) const override; + Size resolution() const override; + + V4L2SubdeviceFormat getFormat(const std::vector &mbusCodes, + const Size &size) const override; + int setFormat(V4L2SubdeviceFormat *format, + Transform transform = Transform::Identity) override; + int tryFormat(V4L2SubdeviceFormat *format) const override; + + int applyConfiguration(const SensorConfiguration &config, + Transform transform = Transform::Identity, + V4L2SubdeviceFormat *sensorFormat = nullptr) override; + + const ControlList &properties() const override { return properties_; } + int sensorInfo(IPACameraSensorInfo *info) const override; + Transform computeTransform(Orientation *orientation) const override; + BayerFormat::Order bayerOrder(Transform t) const override; + + const ControlInfoMap &controls() const override; + ControlList getControls(const std::vector &ids) override; + int setControls(ControlList *ctrls) override; + + const std::vector & + testPatternModes() const override { return testPatternModes_; } + int setTestPatternMode(controls::draft::TestPatternModeEnum mode) override; + +protected: + std::string logPrefix() const override; + +private: + LIBCAMERA_DISABLE_COPY(CameraSensorRaw) + + std::optional init(); + int initProperties(); + void initStaticProperties(); + void initTestPatternModes(); + int applyTestPatternMode(controls::draft::TestPatternModeEnum mode); + + const MediaEntity *entity_; + std::unique_ptr subdev_; + + struct Streams { + V4L2Subdevice::Stream sink; + V4L2Subdevice::Stream source; + }; + + struct { + Streams image; + std::optional edata; + } streams_; + + const CameraSensorProperties *staticProps_; + + std::string model_; + std::string id_; + + V4L2Subdevice::Formats formats_; + std::vector mbusCodes_; + std::vector sizes_; + std::vector testPatternModes_; + controls::draft::TestPatternModeEnum testPatternMode_; + + Size pixelArraySize_; + Rectangle activeArea_; + BayerFormat::Order cfaPattern_; + bool supportFlips_; + bool flipsAlterBayerOrder_; + Orientation mountingOrientation_; + + ControlList properties_; + + std::unique_ptr focusLens_; +}; + +/** + * \class CameraSensorRaw + * \brief A camera sensor based on V4L2 subdevices + * + * This class supports single-subdev sensors with a single source pad and one + * or two internal sink pads (for the image and embedded data streams). + */ + +CameraSensorRaw::CameraSensorRaw(const MediaEntity *entity) + : entity_(entity), staticProps_(nullptr), supportFlips_(false), + flipsAlterBayerOrder_(false), properties_(properties::properties) +{ +} + +CameraSensorRaw::~CameraSensorRaw() = default; + +std::variant, int> +CameraSensorRaw::match(MediaEntity *entity) +{ + /* Check the entity type. */ + if (entity->type() != MediaEntity::Type::V4L2Subdevice || + entity->function() != MEDIA_ENT_F_CAM_SENSOR) { + libcamera::LOG(CameraSensor, Debug) + << entity->name() << ": unsupported entity type (" + << utils::to_underlying(entity->type()) + << ") or function (" << utils::hex(entity->function()) << ")"; + return { 0 }; + } + + /* Count and check the number of pads. */ + static constexpr uint32_t kPadFlagsMask = MEDIA_PAD_FL_SINK + | MEDIA_PAD_FL_SOURCE + | MEDIA_PAD_FL_INTERNAL; + unsigned int numSinks = 0; + unsigned int numSources = 0; + + for (const MediaPad *pad : entity->pads()) { + switch (pad->flags() & kPadFlagsMask) { + case MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_INTERNAL: + numSinks++; + break; + + case MEDIA_PAD_FL_SOURCE: + numSources++; + break; + + default: + libcamera::LOG(CameraSensor, Debug) + << entity->name() << ": unsupported pad " << pad->index() + << " type " << utils::hex(pad->flags()); + return { 0 }; + } + } + + if (numSinks < 1 || numSinks > 2 || numSources != 1) { + libcamera::LOG(CameraSensor, Debug) + << entity->name() << ": unsupported number of sinks (" + << numSinks << ") or sources (" << numSources << ")"; + return { 0 }; + } + + /* + * The entity matches. Create the camera sensor and initialize it. The + * init() function will perform further match checks. + */ + std::unique_ptr sensor = + std::make_unique(entity); + + std::optional err = sensor->init(); + if (err) + return { *err }; + + return { std::move(sensor) }; +} + +std::optional CameraSensorRaw::init() +{ + /* Create and open the subdev. */ + subdev_ = std::make_unique(entity_); + int ret = subdev_->open(); + if (ret) + return { ret }; + + /* + * 1. Identify the pads. + */ + + /* + * First locate the source pad. The match() function guarantees there + * is one and only one source pad. + */ + unsigned int sourcePad = UINT_MAX; + + for (const MediaPad *pad : entity_->pads()) { + if (pad->flags() & MEDIA_PAD_FL_SOURCE) { + sourcePad = pad->index(); + break; + } + } + + /* + * Iterate over the routes to identify the streams on the source pad, + * and the internal sink pads. + */ + V4L2Subdevice::Routing routing = {}; + ret = subdev_->getRouting(&routing, V4L2Subdevice::TryFormat); + if (ret) + return { ret }; + + bool imageStreamFound = false; + + for (const V4L2Subdevice::Route &route : routing) { + if (route.source.pad != sourcePad) { + LOG(CameraSensor, Error) << "Invalid route " << route; + return { -EINVAL }; + } + + /* Identify the stream type based on the supported formats. */ + V4L2Subdevice::Formats formats = subdev_->formats(route.source); + + std::optional type; + + for (const auto &[code, sizes] : formats) { + const MediaBusFormatInfo &info = + MediaBusFormatInfo::info(code); + if (info.isValid()) { + type = info.type; + break; + } + } + + if (!type) { + LOG(CameraSensor, Warning) + << "No known format on pad " << route.source; + continue; + } + + switch (*type) { + case MediaBusFormatInfo::Type::Image: + if (imageStreamFound) { + LOG(CameraSensor, Error) + << "Multiple internal image streams (" + << streams_.image.sink << " and " + << route.sink << ")"; + return { -EINVAL }; + } + + imageStreamFound = true; + streams_.image.sink = route.sink; + streams_.image.source = route.source; + break; + + case MediaBusFormatInfo::Type::Metadata: + /* + * Skip metadata streams that are not sensor embedded + * data. The source stream reports a generic metadata + * format, check the sink stream for the exact format. + */ + formats = subdev_->formats(route.sink); + if (formats.size() != 1) + continue; + + if (MediaBusFormatInfo::info(formats.cbegin()->first).type != + MediaBusFormatInfo::Type::EmbeddedData) + continue; + + if (streams_.edata) { + LOG(CameraSensor, Error) + << "Multiple internal embedded data streams (" + << streams_.edata->sink << " and " + << route.sink << ")"; + return { -EINVAL }; + } + + streams_.edata = { route.sink, route.source }; + break; + + default: + break; + } + } + + if (!imageStreamFound) { + LOG(CameraSensor, Error) << "No image stream found"; + return { -EINVAL }; + } + + LOG(CameraSensor, Debug) + << "Found image stream " << streams_.image.sink + << " -> " << streams_.image.source; + + if (streams_.edata) + LOG(CameraSensor, Debug) + << "Found embedded data stream " << streams_.edata->sink + << " -> " << streams_.edata->source; + + /* + * 2. Enumerate and cache the media bus codes, sizes and colour filter + * array order for the image stream. + */ + + /* + * Get the native sensor CFA pattern. It is simpler to retrieve it from + * the internal image sink pad as it is guaranteed to expose a single + * format, and is not affected by flips. + */ + V4L2Subdevice::Formats formats = subdev_->formats(streams_.image.sink); + if (formats.size() != 1) { + LOG(CameraSensor, Error) + << "Image pad has " << formats.size() + << " formats, expected 1"; + return { -EINVAL }; + } + + uint32_t nativeFormat = formats.cbegin()->first; + const BayerFormat &bayerFormat = BayerFormat::fromMbusCode(nativeFormat); + if (!bayerFormat.isValid()) { + LOG(CameraSensor, Error) + << "Invalid native format " << nativeFormat; + return { 0 }; + } + + cfaPattern_ = bayerFormat.order; + + /* + * Retrieve and cache the media bus codes and sizes on the source image + * stream. + */ + formats_ = subdev_->formats(streams_.image.source); + if (formats_.empty()) { + LOG(CameraSensor, Error) << "No image format found"; + return { -EINVAL }; + } + + /* Populate and sort the media bus codes and the sizes. */ + for (const auto &[code, ranges] : formats_) { + /* Drop non-raw formats (in case we have a hybrid sensor). */ + const MediaBusFormatInfo &info = MediaBusFormatInfo::info(code); + if (info.colourEncoding != PixelFormatInfo::ColourEncodingRAW) + continue; + + mbusCodes_.push_back(code); + std::transform(ranges.begin(), ranges.end(), std::back_inserter(sizes_), + [](const SizeRange &range) { return range.max; }); + } + + if (mbusCodes_.empty()) { + LOG(CameraSensor, Debug) << "No raw image formats found"; + return { 0 }; + } + + std::sort(mbusCodes_.begin(), mbusCodes_.end()); + std::sort(sizes_.begin(), sizes_.end()); + + /* + * Remove duplicate sizes. There are no duplicate media bus codes as + * they are the keys in the formats map. + */ + auto last = std::unique(sizes_.begin(), sizes_.end()); + sizes_.erase(last, sizes_.end()); + + /* + * 3. Query selection rectangles. Retrieve properties, and verify that + * all the expected selection rectangles are supported. + */ + + Rectangle rect; + ret = subdev_->getSelection(streams_.image.sink, V4L2_SEL_TGT_CROP_BOUNDS, + &rect); + if (ret) { + LOG(CameraSensor, Error) << "No pixel array crop bounds"; + return { ret }; + } + + pixelArraySize_ = rect.size(); + + ret = subdev_->getSelection(streams_.image.sink, V4L2_SEL_TGT_CROP_DEFAULT, + &activeArea_); + if (ret) { + LOG(CameraSensor, Error) << "No pixel array crop default"; + return { ret }; + } + + ret = subdev_->getSelection(streams_.image.sink, V4L2_SEL_TGT_CROP, + &rect); + if (ret) { + LOG(CameraSensor, Error) << "No pixel array crop rectangle"; + return { ret }; + } + + /* + * 4. Verify that all required controls are present. + */ + + const ControlIdMap &controls = subdev_->controls().idmap(); + + static constexpr uint32_t mandatoryControls[] = { + V4L2_CID_ANALOGUE_GAIN, + V4L2_CID_CAMERA_ORIENTATION, + V4L2_CID_EXPOSURE, + V4L2_CID_HBLANK, + V4L2_CID_PIXEL_RATE, + V4L2_CID_VBLANK, + }; + + ret = 0; + + for (uint32_t ctrl : mandatoryControls) { + if (!controls.count(ctrl)) { + LOG(CameraSensor, Error) + << "Mandatory V4L2 control " << utils::hex(ctrl) + << " not available"; + ret = -EINVAL; + } + } + + if (ret) { + LOG(CameraSensor, Error) + << "The sensor kernel driver needs to be fixed"; + LOG(CameraSensor, Error) + << "See Documentation/sensor_driver_requirements.rst in the libcamera sources for more information"; + return { ret }; + } + + /* + * Verify if sensor supports horizontal/vertical flips + * + * \todo Handle horizontal and vertical flips independently. + */ + const struct v4l2_query_ext_ctrl *hflipInfo = subdev_->controlInfo(V4L2_CID_HFLIP); + const struct v4l2_query_ext_ctrl *vflipInfo = subdev_->controlInfo(V4L2_CID_VFLIP); + if (hflipInfo && !(hflipInfo->flags & V4L2_CTRL_FLAG_READ_ONLY) && + vflipInfo && !(vflipInfo->flags & V4L2_CTRL_FLAG_READ_ONLY)) { + supportFlips_ = true; + + if (hflipInfo->flags & V4L2_CTRL_FLAG_MODIFY_LAYOUT || + vflipInfo->flags & V4L2_CTRL_FLAG_MODIFY_LAYOUT) + flipsAlterBayerOrder_ = true; + } + + if (!supportFlips_) + LOG(CameraSensor, Debug) + << "Camera sensor does not support horizontal/vertical flip"; + + /* + * 5. Discover ancillary devices. + * + * \todo This code may be shared by different V4L2 sensor classes. + */ + for (MediaEntity *ancillary : entity_->ancillaryEntities()) { + switch (ancillary->function()) { + case MEDIA_ENT_F_LENS: + focusLens_ = std::make_unique(ancillary); + ret = focusLens_->init(); + if (ret) { + LOG(CameraSensor, Error) + << "Lens initialisation failed, lens disabled"; + focusLens_.reset(); + } + break; + + default: + LOG(CameraSensor, Warning) + << "Unsupported ancillary entity function " + << ancillary->function(); + break; + } + } + + /* + * 6. Initialize properties. + */ + + ret = initProperties(); + if (ret) + return { ret }; + + /* + * 7. Initialize controls. + */ + + /* + * Set HBLANK to the minimum to start with a well-defined line length, + * allowing IPA modules that do not modify HBLANK to use the sensor + * minimum line length in their calculations. + */ + const struct v4l2_query_ext_ctrl *hblankInfo = subdev_->controlInfo(V4L2_CID_HBLANK); + if (hblankInfo && !(hblankInfo->flags & V4L2_CTRL_FLAG_READ_ONLY)) { + ControlList ctrl(subdev_->controls()); + + ctrl.set(V4L2_CID_HBLANK, static_cast(hblankInfo->minimum)); + ret = subdev_->setControls(&ctrl); + if (ret) + return ret; + } + + ret = applyTestPatternMode(controls::draft::TestPatternModeEnum::TestPatternModeOff); + if (ret) + return { ret }; + + return {}; +} + +int CameraSensorRaw::initProperties() +{ + model_ = subdev_->model(); + properties_.set(properties::Model, utils::toAscii(model_)); + + /* Generate a unique ID for the sensor. */ + id_ = sysfs::firmwareNodePath(subdev_->devicePath()); + if (id_.empty()) { + LOG(CameraSensor, Error) << "Can't generate sensor ID"; + return -EINVAL; + } + + /* Initialize the static properties from the sensor database. */ + initStaticProperties(); + + /* Retrieve and register properties from the kernel interface. */ + const ControlInfoMap &controls = subdev_->controls(); + + const auto &orientation = controls.find(V4L2_CID_CAMERA_ORIENTATION); + if (orientation != controls.end()) { + int32_t v4l2Orientation = orientation->second.def().get(); + int32_t propertyValue; + + switch (v4l2Orientation) { + default: + LOG(CameraSensor, Warning) + << "Unsupported camera location " + << v4l2Orientation << ", setting to External"; + [[fallthrough]]; + case V4L2_CAMERA_ORIENTATION_EXTERNAL: + propertyValue = properties::CameraLocationExternal; + break; + case V4L2_CAMERA_ORIENTATION_FRONT: + propertyValue = properties::CameraLocationFront; + break; + case V4L2_CAMERA_ORIENTATION_BACK: + propertyValue = properties::CameraLocationBack; + break; + } + properties_.set(properties::Location, propertyValue); + } else { + LOG(CameraSensor, Warning) << "Failed to retrieve the camera location"; + } + + const auto &rotationControl = controls.find(V4L2_CID_CAMERA_SENSOR_ROTATION); + if (rotationControl != controls.end()) { + int32_t propertyValue = rotationControl->second.def().get(); + + /* + * Cache the Transform associated with the camera mounting + * rotation for later use in computeTransform(). + */ + bool success; + mountingOrientation_ = orientationFromRotation(propertyValue, &success); + if (!success) { + LOG(CameraSensor, Warning) + << "Invalid rotation of " << propertyValue + << " degrees - ignoring"; + mountingOrientation_ = Orientation::Rotate0; + } + + properties_.set(properties::Rotation, propertyValue); + } else { + LOG(CameraSensor, Warning) + << "Rotation control not available, default to 0 degrees"; + properties_.set(properties::Rotation, 0); + mountingOrientation_ = Orientation::Rotate0; + } + + properties_.set(properties::PixelArraySize, pixelArraySize_); + properties_.set(properties::PixelArrayActiveAreas, { activeArea_ }); + + /* Color filter array pattern. */ + uint32_t cfa; + + switch (cfaPattern_) { + case BayerFormat::BGGR: + cfa = properties::draft::BGGR; + break; + case BayerFormat::GBRG: + cfa = properties::draft::GBRG; + break; + case BayerFormat::GRBG: + cfa = properties::draft::GRBG; + break; + case BayerFormat::RGGB: + cfa = properties::draft::RGGB; + break; + case BayerFormat::MONO: + default: + cfa = properties::draft::MONO; + break; + } + + properties_.set(properties::draft::ColorFilterArrangement, cfa); + + return 0; +} + +void CameraSensorRaw::initStaticProperties() +{ + staticProps_ = CameraSensorProperties::get(model_); + if (!staticProps_) + return; + + /* Register the properties retrieved from the sensor database. */ + properties_.set(properties::UnitCellSize, staticProps_->unitCellSize); + + initTestPatternModes(); +} + +void CameraSensorRaw::initTestPatternModes() +{ + const auto &v4l2TestPattern = controls().find(V4L2_CID_TEST_PATTERN); + if (v4l2TestPattern == controls().end()) { + LOG(CameraSensor, Debug) << "V4L2_CID_TEST_PATTERN is not supported"; + return; + } + + const auto &testPatternModes = staticProps_->testPatternModes; + if (testPatternModes.empty()) { + /* + * The camera sensor supports test patterns but we don't know + * how to map them so this should be fixed. + */ + LOG(CameraSensor, Debug) << "No static test pattern map for \'" + << model() << "\'"; + return; + } + + /* + * Create a map that associates the V4L2 control index to the test + * pattern mode by reversing the testPatternModes map provided by the + * camera sensor properties. This makes it easier to verify if the + * control index is supported in the below for loop that creates the + * list of supported test patterns. + */ + std::map indexToTestPatternMode; + for (const auto &it : testPatternModes) + indexToTestPatternMode[it.second] = it.first; + + for (const ControlValue &value : v4l2TestPattern->second.values()) { + const int32_t index = value.get(); + + const auto it = indexToTestPatternMode.find(index); + if (it == indexToTestPatternMode.end()) { + LOG(CameraSensor, Debug) + << "Test pattern mode " << index << " ignored"; + continue; + } + + testPatternModes_.push_back(it->second); + } +} + +std::vector CameraSensorRaw::sizes(unsigned int mbusCode) const +{ + std::vector sizes; + + const auto &format = formats_.find(mbusCode); + if (format == formats_.end()) + return sizes; + + const std::vector &ranges = format->second; + std::transform(ranges.begin(), ranges.end(), std::back_inserter(sizes), + [](const SizeRange &range) { return range.max; }); + + std::sort(sizes.begin(), sizes.end()); + + return sizes; +} + +Size CameraSensorRaw::resolution() const +{ + return std::min(sizes_.back(), activeArea_.size()); +} + +V4L2SubdeviceFormat +CameraSensorRaw::getFormat(const std::vector &mbusCodes, + const Size &size) const +{ + unsigned int desiredArea = size.width * size.height; + unsigned int bestArea = UINT_MAX; + float desiredRatio = static_cast(size.width) / size.height; + float bestRatio = FLT_MAX; + const Size *bestSize = nullptr; + uint32_t bestCode = 0; + + for (unsigned int code : mbusCodes) { + const auto formats = formats_.find(code); + if (formats == formats_.end()) + continue; + + for (const SizeRange &range : formats->second) { + const Size &sz = range.max; + + if (sz.width < size.width || sz.height < size.height) + continue; + + float ratio = static_cast(sz.width) / sz.height; + float ratioDiff = std::abs(ratio - desiredRatio); + unsigned int area = sz.width * sz.height; + unsigned int areaDiff = area - desiredArea; + + if (ratioDiff > bestRatio) + continue; + + if (ratioDiff < bestRatio || areaDiff < bestArea) { + bestRatio = ratioDiff; + bestArea = areaDiff; + bestSize = &sz; + bestCode = code; + } + } + } + + if (!bestSize) { + LOG(CameraSensor, Debug) << "No supported format or size found"; + return {}; + } + + V4L2SubdeviceFormat format{ + .code = bestCode, + .size = *bestSize, + .colorSpace = ColorSpace::Raw, + }; + + return format; +} + +int CameraSensorRaw::setFormat(V4L2SubdeviceFormat *format, Transform transform) +{ + /* Configure flips if the sensor supports that. */ + if (supportFlips_) { + ControlList flipCtrls(subdev_->controls()); + + flipCtrls.set(V4L2_CID_HFLIP, + static_cast(!!(transform & Transform::HFlip))); + flipCtrls.set(V4L2_CID_VFLIP, + static_cast(!!(transform & Transform::VFlip))); + + int ret = subdev_->setControls(&flipCtrls); + if (ret) + return ret; + } + + /* Apply format on the subdev. */ + int ret = subdev_->setFormat(streams_.image.source, format); + if (ret) + return ret; + + subdev_->updateControlInfo(); + return 0; +} + +int CameraSensorRaw::tryFormat(V4L2SubdeviceFormat *format) const +{ + return subdev_->setFormat(streams_.image.source, format, + V4L2Subdevice::Whence::TryFormat); +} + +int CameraSensorRaw::applyConfiguration(const SensorConfiguration &config, + Transform transform, + V4L2SubdeviceFormat *sensorFormat) +{ + if (!config.isValid()) { + LOG(CameraSensor, Error) << "Invalid sensor configuration"; + return -EINVAL; + } + + std::vector filteredCodes; + std::copy_if(mbusCodes_.begin(), mbusCodes_.end(), + std::back_inserter(filteredCodes), + [&config](unsigned int mbusCode) { + BayerFormat bayer = BayerFormat::fromMbusCode(mbusCode); + if (bayer.bitDepth == config.bitDepth) + return true; + return false; + }); + if (filteredCodes.empty()) { + LOG(CameraSensor, Error) + << "Cannot find any format with bit depth " + << config.bitDepth; + return -EINVAL; + } + + /* + * Compute the sensor's data frame size by applying the cropping + * rectangle, subsampling and output crop to the sensor's pixel array + * size. + * + * \todo The actual size computation is for now ignored and only the + * output size is considered. This implies that resolutions obtained + * with two different cropping/subsampling will look identical and + * only the first found one will be considered. + */ + V4L2SubdeviceFormat subdevFormat = {}; + for (unsigned int code : filteredCodes) { + for (const Size &size : sizes(code)) { + if (size.width != config.outputSize.width || + size.height != config.outputSize.height) + continue; + + subdevFormat.code = code; + subdevFormat.size = size; + break; + } + } + if (!subdevFormat.code) { + LOG(CameraSensor, Error) << "Invalid output size in sensor configuration"; + return -EINVAL; + } + + int ret = setFormat(&subdevFormat, transform); + if (ret) + return ret; + + /* + * Return to the caller the format actually applied to the sensor. + * This is relevant if transform has changed the bayer pattern order. + */ + if (sensorFormat) + *sensorFormat = subdevFormat; + + /* \todo Handle AnalogCrop. Most sensors do not support set_selection */ + /* \todo Handle scaling in the digital domain. */ + + return 0; +} + +int CameraSensorRaw::sensorInfo(IPACameraSensorInfo *info) const +{ + info->model = model(); + + /* + * The active area size is a static property, while the crop + * rectangle needs to be re-read as it depends on the sensor + * configuration. + */ + info->activeAreaSize = { activeArea_.width, activeArea_.height }; + + int ret = subdev_->getSelection(streams_.image.sink, V4L2_SEL_TGT_CROP, + &info->analogCrop); + if (ret) + return ret; + + /* + * IPACameraSensorInfo::analogCrop::x and IPACameraSensorInfo::analogCrop::y + * are defined relatively to the active pixel area, while V4L2's + * TGT_CROP target is defined in respect to the full pixel array. + * + * Compensate it by subtracting the active area offset. + */ + info->analogCrop.x -= activeArea_.x; + info->analogCrop.y -= activeArea_.y; + + /* The bit depth and image size depend on the currently applied format. */ + V4L2SubdeviceFormat format{}; + ret = subdev_->getFormat(streams_.image.source, &format); + if (ret) + return ret; + info->bitsPerPixel = MediaBusFormatInfo::info(format.code).bitsPerPixel; + info->outputSize = format.size; + + std::optional cfa = properties_.get(properties::draft::ColorFilterArrangement); + info->cfaPattern = cfa ? *cfa : properties::draft::RGB; + + /* + * Retrieve the pixel rate, line length and minimum/maximum frame + * duration through V4L2 controls. Support for the V4L2_CID_PIXEL_RATE, + * V4L2_CID_HBLANK and V4L2_CID_VBLANK controls is mandatory. + */ + ControlList ctrls = subdev_->getControls({ V4L2_CID_PIXEL_RATE, + V4L2_CID_HBLANK, + V4L2_CID_VBLANK }); + if (ctrls.empty()) { + LOG(CameraSensor, Error) + << "Failed to retrieve camera info controls"; + return -EINVAL; + } + + info->pixelRate = ctrls.get(V4L2_CID_PIXEL_RATE).get(); + + const ControlInfo hblank = ctrls.infoMap()->at(V4L2_CID_HBLANK); + info->minLineLength = info->outputSize.width + hblank.min().get(); + info->maxLineLength = info->outputSize.width + hblank.max().get(); + + const ControlInfo vblank = ctrls.infoMap()->at(V4L2_CID_VBLANK); + info->minFrameLength = info->outputSize.height + vblank.min().get(); + info->maxFrameLength = info->outputSize.height + vblank.max().get(); + + return 0; +} + +Transform CameraSensorRaw::computeTransform(Orientation *orientation) const +{ + /* + * If we cannot do any flips we cannot change the native camera mounting + * orientation. + */ + if (!supportFlips_) { + *orientation = mountingOrientation_; + return Transform::Identity; + } + + /* + * Now compute the required transform to obtain 'orientation' starting + * from the mounting rotation. + * + * As a note: + * orientation / mountingOrientation_ = transform + * mountingOrientation_ * transform = orientation + */ + Transform transform = *orientation / mountingOrientation_; + + /* + * If transform contains any Transpose we cannot do it, so adjust + * 'orientation' to report the image native orientation and return Identity. + */ + if (!!(transform & Transform::Transpose)) { + *orientation = mountingOrientation_; + return Transform::Identity; + } + + return transform; +} + +BayerFormat::Order CameraSensorRaw::bayerOrder(Transform t) const +{ + if (!flipsAlterBayerOrder_) + return cfaPattern_; + + /* + * Apply the transform to the native (i.e. untransformed) Bayer order, + * using the rest of the Bayer format supplied by the caller. + */ + BayerFormat format{ cfaPattern_, 8, BayerFormat::Packing::None }; + return format.transform(t).order; +} + +const ControlInfoMap &CameraSensorRaw::controls() const +{ + return subdev_->controls(); +} + +ControlList CameraSensorRaw::getControls(const std::vector &ids) +{ + return subdev_->getControls(ids); +} + +int CameraSensorRaw::setControls(ControlList *ctrls) +{ + return subdev_->setControls(ctrls); +} + +int CameraSensorRaw::setTestPatternMode(controls::draft::TestPatternModeEnum mode) +{ + if (testPatternMode_ == mode) + return 0; + + if (testPatternModes_.empty()) { + LOG(CameraSensor, Error) + << "Camera sensor does not support test pattern modes."; + return -EINVAL; + } + + return applyTestPatternMode(mode); +} + +int CameraSensorRaw::applyTestPatternMode(controls::draft::TestPatternModeEnum mode) +{ + if (testPatternModes_.empty()) + return 0; + + auto it = std::find(testPatternModes_.begin(), testPatternModes_.end(), + mode); + if (it == testPatternModes_.end()) { + LOG(CameraSensor, Error) << "Unsupported test pattern mode " + << mode; + return -EINVAL; + } + + LOG(CameraSensor, Debug) << "Apply test pattern mode " << mode; + + int32_t index = staticProps_->testPatternModes.at(mode); + ControlList ctrls{ controls() }; + ctrls.set(V4L2_CID_TEST_PATTERN, index); + + int ret = setControls(&ctrls); + if (ret) + return ret; + + testPatternMode_ = mode; + + return 0; +} + +std::string CameraSensorRaw::logPrefix() const +{ + return "'" + entity_->name() + "'"; +} + +REGISTER_CAMERA_SENSOR(CameraSensorRaw, 0) + +} /* namespace libcamera */ diff --git a/src/libcamera/sensor/meson.build b/src/libcamera/sensor/meson.build index f0d588977a4f..dce74ed6ac88 100644 --- a/src/libcamera/sensor/meson.build +++ b/src/libcamera/sensor/meson.build @@ -4,4 +4,5 @@ libcamera_internal_sources += files([ 'camera_sensor.cpp', 'camera_sensor_legacy.cpp', 'camera_sensor_properties.cpp', + 'camera_sensor_raw.cpp', ]) From patchwork Fri Nov 8 10:51:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21872 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 67CC7BE173 for ; Fri, 8 Nov 2024 10:51:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0B119657B6; Fri, 8 Nov 2024 11:51:37 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="SofcG7dn"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4312665499 for ; Fri, 8 Nov 2024 11:51:27 +0100 (CET) Received: from ideasonboard.com (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9534F7E9; Fri, 8 Nov 2024 11:51:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731063077; bh=fHBHZvwHYpRPxCDuT782iGohglgrIcdbLvMFRTbWd3Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SofcG7dnl4Uoktzd8UJ4ozRLHlSGgWZLk7sh6UV4fr//k0zkVSCl3jWWE5jiJFC/+ Gj8goxSPEn0HsKccbV4ZjD4Kj+a933d9dVJ2ZY8gGGHtegj94oyg3hSj3g2JEmGspH 97ergbR/IbDUXPtPJwgNuo9GuVsumJdQyxs+51CA= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , naush@raspberrypi.com, Laurent Pinchart Subject: [PATCH v2 6/6] libcamera: camera_sensor: Add support for embedded data Date: Fri, 8 Nov 2024 11:51:16 +0100 Message-ID: <20241108105117.137121-7-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241108105117.137121-1-jacopo.mondi@ideasonboard.com> References: <20241108105117.137121-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart Some sensors support producing and transmitting embedded data over a stream separate from the image stream. Add support for this feature in the CameraSensor interface, and implement it for the CameraSensorRaw class. The CameraSensorLegacy uses the default stub implementation, as the corresponding kernel drivers don't support embedded data. Signed-off-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- include/libcamera/internal/camera_sensor.h | 5 ++ src/libcamera/sensor/camera_sensor.cpp | 67 ++++++++++++++++++++ src/libcamera/sensor/camera_sensor_raw.cpp | 72 ++++++++++++++++++++++ 3 files changed, 144 insertions(+) diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index 8aafd82e6547..61bf1d7ca6de 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -61,6 +61,11 @@ public: Transform transform = Transform::Identity, V4L2SubdeviceFormat *sensorFormat = nullptr) = 0; + virtual V4L2Subdevice::Stream imageStream() const; + virtual std::optional embeddedDataStream() const; + virtual V4L2SubdeviceFormat embeddedDataFormat() const; + virtual int setEmbeddedDataEnabled(bool enable); + virtual const ControlList &properties() const = 0; virtual int sensorInfo(IPACameraSensorInfo *info) const = 0; virtual Transform computeTransform(Orientation *orientation) const = 0; diff --git a/src/libcamera/sensor/camera_sensor.cpp b/src/libcamera/sensor/camera_sensor.cpp index 54cf98b203fd..1217365b2a55 100644 --- a/src/libcamera/sensor/camera_sensor.cpp +++ b/src/libcamera/sensor/camera_sensor.cpp @@ -196,6 +196,73 @@ CameraSensor::~CameraSensor() = default; * error code otherwise */ +/** + * \brief Retrieve the image source stream + * + * Sensors that produce multiple streams do not guarantee that the image stream + * is always assigned number 0. This function allows callers to retrieve the + * image stream on the sensor's source pad, in order to configure the receiving + * side accordingly. + * + * \return The image source stream + */ +V4L2Subdevice::Stream CameraSensor::imageStream() const +{ + return { 0, 0 }; +} + +/** + * \brief Retrieve the embedded data source stream + * + * Some sensors produce embedded data in a stream separate from the image + * stream. This function indicates if the sensor supports this feature by + * returning the embedded data stream on the sensor's source pad if available, + * or an std::optional<> without a value otheriwse. + * + * \return The embedded data source stream + */ +std::optional CameraSensor::embeddedDataStream() const +{ + return {}; +} + +/** + * \brief Retrieve the format on the embedded data stream + * + * When an embedded data stream is available, this function returns the + * corresponding format on the sensor's source pad. The format may vary with + * the image stream format, and should therefore be retrieved after configuring + * the image stream. + * + * If the sensor doesn't support embedded data, this function returns a + * default-constructed format. + * + * \return The format on the embedded data stream + */ +V4L2SubdeviceFormat CameraSensor::embeddedDataFormat() const +{ + return {}; +} + +/** + * \brief Enable or disable the embedded data stream + * \param[in] enable True to enable the embedded data stream, false to disable it + * + * For sensors that support embedded data, this function enables or disables + * generation of embedded data. Some of such sensors always produce embedded + * data, in which case this function return -EISCONN if the caller attempts to + * disable embedded data. + * + * If the sensor doesn't support embedded data, this function returns 0 when \a + * enable is false, and -ENOSTR otherwise. + * + * \return 0 on success, or a negative error code otherwise + */ +int CameraSensor::setEmbeddedDataEnabled(bool enable) +{ + return enable ? -ENOSTR : 0; +} + /** * \fn CameraSensor::properties() * \brief Retrieve the camera sensor properties diff --git a/src/libcamera/sensor/camera_sensor_raw.cpp b/src/libcamera/sensor/camera_sensor_raw.cpp index 4c653121d547..9380ec7129f6 100644 --- a/src/libcamera/sensor/camera_sensor_raw.cpp +++ b/src/libcamera/sensor/camera_sensor_raw.cpp @@ -84,6 +84,11 @@ public: Transform transform = Transform::Identity, V4L2SubdeviceFormat *sensorFormat = nullptr) override; + V4L2Subdevice::Stream imageStream() const override; + std::optional embeddedDataStream() const override; + V4L2SubdeviceFormat embeddedDataFormat() const override; + int setEmbeddedDataEnabled(bool enable) override; + const ControlList &properties() const override { return properties_; } int sensorInfo(IPACameraSensorInfo *info) const override; Transform computeTransform(Orientation *orientation) const override; @@ -878,6 +883,73 @@ int CameraSensorRaw::applyConfiguration(const SensorConfiguration &config, return 0; } +V4L2Subdevice::Stream CameraSensorRaw::imageStream() const +{ + return streams_.image.source; +} + +std::optional CameraSensorRaw::embeddedDataStream() const +{ + if (!streams_.edata) + return {}; + + return { streams_.edata->source }; +} + +V4L2SubdeviceFormat CameraSensorRaw::embeddedDataFormat() const +{ + if (!streams_.edata) + return {}; + + V4L2SubdeviceFormat format; + int ret = subdev_->getFormat(streams_.edata->source, &format); + if (ret) + return {}; + + return format; +} + +int CameraSensorRaw::setEmbeddedDataEnabled(bool enable) +{ + if (!streams_.edata) + return enable ? -ENOSTR : 0; + + V4L2Subdevice::Routing routing{ 2 }; + + routing[0].sink = streams_.image.sink; + routing[0].source = streams_.image.source; + routing[0].flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE; + + routing[1].sink = streams_.edata->sink; + routing[1].source = streams_.edata->source; + routing[1].flags = enable ? V4L2_SUBDEV_ROUTE_FL_ACTIVE : 0; + + int ret = subdev_->setRouting(&routing); + if (ret) + return ret; + + /* + * Check if the embedded data stream has been enabled or disabled + * correctly. Assume at least one route will match the embedded data + * source stream, as there would be something seriously wrong + * otherwise. + */ + bool enabled = false; + + for (const V4L2Subdevice::Route &route : routing) { + if (route.source != streams_.edata->source) + continue; + + enabled = route.flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE; + break; + } + + if (enabled != enable) + return enabled ? -EISCONN : -ENOSTR; + + return 0; +} + int CameraSensorRaw::sensorInfo(IPACameraSensorInfo *info) const { info->model = model();