From patchwork Mon Mar 16 21:43:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 3118 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0A4EE60422 for ; Mon, 16 Mar 2020 22:43:21 +0100 (CET) Received: from pendragon.bb.dnainternet.fi (81-175-216-236.bb.dnainternet.fi [81.175.216.236]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 58ED1AD8; Mon, 16 Mar 2020 22:43:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1584395000; bh=OuYSMq66Kicl43m1yxRxpkunkmtmaYs0y8lU4TFqw5o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QDJ5plOIqFIlBsp6GBAPoICT1b7goiSx4nIc0Vmo/KSwD3GjupuOUGErFNn+m3ijN nNX/NYRkmMS50LHo2gq1qzPC4wQHm/SCH5c0PAAmKDUvebMBMGDor8E2WN+Ps3EVoT bhYpGzBsyoZP7qj+LxsRwmGM/FqjnsLLOieg1EFo= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: Martijn Braam , Mickael GUENE , Benjamin GAIGNARD , Andrey Konovalov Date: Mon, 16 Mar 2020 23:43:01 +0200 Message-Id: <20200316214310.27665-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> References: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 01/10] [DNI] include: linux: Extend VIDIOC_ENUM_FMT to support MC-centric devices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2020 21:43:21 -0000 This is an experimental change that hasn't been accepted in mainline yet. Its commit message for the Linux kernel is as follows. media: v4l2: Extend VIDIOC_ENUM_FMT to support MC-centric devices The VIDIOC_ENUM_FMT ioctl enumerates all formats supported by a video node. For MC-centric devices, its behaviour has always been ill-defined, with drivers implementing one of the following behaviours: - No support for VIDIOC_ENUM_FMT at all - Enumerating all formats supported by the video node, regardless of the configuration of the pipeline - Enumerating formats supported by the video node for the active configuration of the connected subdevice The first behaviour is obviously useless for applications. The second behaviour provides the most information, but doesn't offer a way to find what formats are compatible with a given pipeline configuration. The third behaviour fixes that, but with the drawback that applications can't enumerate all supported formats anymore, and have to modify the active configuration of the pipeline to enumerate formats. The situation is messy as none of the implemented behaviours are ideal, and userspace can't predict what will happen as the behaviour is driver-specific. To fix this, let's extend the VIDIOC_ENUM_FMT with a missing capability: enumerating pixel formats for a given media bus code. The media bus code is passed through the v4l2_fmtdesc structure in a new mbus_code field (repurposed from the reserved fields), and an additional flag is added to report if the driver supports this API extension. With this capability in place, applications can enumerate pixel formats for a given media bus code without modifying the active configuration of the device. The current behaviour of the ioctl is preserved when the new mbus_code field is set to 0, ensuring compatibility with existing userspace. This behaviour is now documented as mandatory for MC-centric devices as well as the traditional video node-centric devices. This allows applications to query MC-centric devices for all the supported pixel formats, as well as for the pixel formats corresponding to a given media bus code. Signed-off-by: Laurent Pinchart --- include/linux/videodev2.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h index ab40b3272ed2..120130c4aaae 100644 --- a/include/linux/videodev2.h +++ b/include/linux/videodev2.h @@ -769,13 +769,15 @@ struct v4l2_fmtdesc { __u32 flags; __u8 description[32]; /* Description string */ __u32 pixelformat; /* Format fourcc */ - __u32 reserved[4]; + __u32 mbus_code; /* Media bus code */ + __u32 reserved[3]; }; #define V4L2_FMT_FLAG_COMPRESSED 0x0001 #define V4L2_FMT_FLAG_EMULATED 0x0002 #define V4L2_FMT_FLAG_CONTINUOUS_BYTESTREAM 0x0004 #define V4L2_FMT_FLAG_DYN_RESOLUTION 0x0008 +#define V4L2_FMT_FLAG_MBUS_CODE 0x0010 /* Frame Size and frame rate enumeration */ /* From patchwork Mon Mar 16 21:43:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 3119 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A6E7960422 for ; Mon, 16 Mar 2020 22:43:21 +0100 (CET) Received: from pendragon.bb.dnainternet.fi (81-175-216-236.bb.dnainternet.fi [81.175.216.236]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E6008FC9; Mon, 16 Mar 2020 22:43:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1584395001; bh=FzmzqD4WiUI/9NFharnzhuMQCc1MhFQbgM2nIQOaTtw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qh8PqRQHSg3EUrY1axvtkIGpS0mvKN8A2VRxRfGokwUOMPJ4SIP7ZtFMHTRPTAAFT ejU1EPYpPNBztVSREAMq5ImpVZJwr/ERoplPgXFXJSB2Mwv/u9aYvr044kE79RuJnL /4thK+Xp1eAwCHoIq/JLlSlK5PELpHHCEdZaA7HY= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: Martijn Braam , Mickael GUENE , Benjamin GAIGNARD , Andrey Konovalov Date: Mon, 16 Mar 2020 23:43:02 +0200 Message-Id: <20200316214310.27665-3-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> References: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 02/10] libcamera: utils: Add string join function X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2020 21:43:21 -0000 Add a utils::join() function to join elements of a container into a string, with a separator and an optional conversion function if the elements are not implicitly convertible to std::string. Signed-off-by: Laurent Pinchart Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi --- src/libcamera/include/utils.h | 44 +++++++++++++++++++++++++++++++++++ src/libcamera/utils.cpp | 16 +++++++++++++ test/utils.cpp | 7 +++++- 3 files changed, 66 insertions(+), 1 deletion(-) diff --git a/src/libcamera/include/utils.h b/src/libcamera/include/utils.h index 940597760ee2..74ceed760cb3 100644 --- a/src/libcamera/include/utils.h +++ b/src/libcamera/include/utils.h @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -109,6 +110,49 @@ inline _hex hex(uint64_t value, unsigned int width) size_t strlcpy(char *dst, const char *src, size_t size); +#ifndef __DOXYGEN__ +template +std::string join(const Container &items, const std::string &sep, UnaryOp op) +{ + std::ostringstream ss; + bool first = true; + + for (typename Container::const_iterator it = std::begin(items); + it != std::end(items); ++it) { + if (!first) + ss << sep; + else + first = false; + + ss << op(*it); + } + + return ss.str(); +} + +template +std::string join(const Container &items, const std::string &sep) +{ + std::ostringstream ss; + bool first = true; + + for (typename Container::const_iterator it = std::begin(items); + it != std::end(items); ++it) { + if (!first) + ss << sep; + else + first = false; + + ss << *it; + } + + return ss.str(); +} +#else +template +std::string join(const Container &items, const std::string &sep, UnaryOp op = nullptr); +#endif + namespace details { class StringSplitter diff --git a/src/libcamera/utils.cpp b/src/libcamera/utils.cpp index f566e88cec5b..1d0e583756cf 100644 --- a/src/libcamera/utils.cpp +++ b/src/libcamera/utils.cpp @@ -291,6 +291,22 @@ details::StringSplitter::iterator details::StringSplitter::end() const return iterator(this, std::string::npos); } +/** + * \fn template \ + * std::string utils::join(const Container &items, const std::string &sep, UnaryOp op) + * \brief Join elements of a container in a string with a separator + * \param[in] items The container + * \param[in] sep The separator to add between elements + * \param[in] op A function that converts individual elements to strings + * + * This function joins all elements in the \a items container into a string and + * returns it. The \a sep separator is added between elements. If the container + * elements are not implicitly convertible to std::string, the \a op function + * shall be provided to perform conversion of elements to std::string. + * + * \return A string that concatenates all elements in the container + */ + /** * \fn split(const std::string &str, const std::string &delim) * \brief Split a string based on a delimiter diff --git a/test/utils.cpp b/test/utils.cpp index 58816f153066..2fca89ef3278 100644 --- a/test/utils.cpp +++ b/test/utils.cpp @@ -99,7 +99,7 @@ protected: return TestFail; } - /* utils::split() test. */ + /* utils::join() and utils::split() test. */ std::vector elements = { "/bin", "/usr/bin", @@ -111,6 +111,11 @@ protected: for (const auto &element : elements) path += (path.empty() ? "" : ":") + element; + if (path != utils::join(elements, ":")) { + cerr << "utils::join() test failed" << endl; + return TestFail; + } + std::vector dirs; for (const auto &dir : utils::split(path, ":")) From patchwork Mon Mar 16 21:43:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 3120 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9C8D262928 for ; Mon, 16 Mar 2020 22:43:22 +0100 (CET) Received: from pendragon.bb.dnainternet.fi (81-175-216-236.bb.dnainternet.fi [81.175.216.236]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 97E6A112C; Mon, 16 Mar 2020 22:43:21 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1584395002; bh=HY5E+dmJgzVhi3bW0GlrGoGcl2rHZOGUTpyfj6s9R6M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Hv5TcRUoxAz5rIWrq8kx4a6QEqHegYJ9PH9bB1ObTn6ywZPjxOS+P7vtq7CqDOWoB DHxowuQq3ONuLXeWxgVL2Nrfs3CfUdplAIPyztF70g+m3VY6X7zHZ1bZtR8XZn/skt jjBZRET3ZgTKu+NdMEWTk5A2c9lTCs39jwD9c0jY= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: Martijn Braam , Mickael GUENE , Benjamin GAIGNARD , Andrey Konovalov Date: Mon, 16 Mar 2020 23:43:03 +0200 Message-Id: <20200316214310.27665-4-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> References: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 03/10] libcamera: v4l2_subdevice: Extend [gs]etFormat() to specify format type X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2020 21:43:22 -0000 The V4L2 subdevice API exposes ACTIVE and TRY formats, but the V4L2Subdevice getFormat() and setFormat() operations only apply on ACTIVE formats. Extend them to support TRY formats as well. Signed-off-by: Laurent Pinchart Reviewed-by: Niklas Söderlund --- src/libcamera/include/v4l2_subdevice.h | 11 +++++++++-- src/libcamera/v4l2_subdevice.cpp | 23 +++++++++++++++++++---- 2 files changed, 28 insertions(+), 6 deletions(-) diff --git a/src/libcamera/include/v4l2_subdevice.h b/src/libcamera/include/v4l2_subdevice.h index 9c077674f997..4a04eadfb1f9 100644 --- a/src/libcamera/include/v4l2_subdevice.h +++ b/src/libcamera/include/v4l2_subdevice.h @@ -32,6 +32,11 @@ struct V4L2SubdeviceFormat { class V4L2Subdevice : public V4L2Device { public: + enum Whence { + ActiveFormat, + TryFormat, + }; + explicit V4L2Subdevice(const MediaEntity *entity); V4L2Subdevice(const V4L2Subdevice &) = delete; V4L2Subdevice &operator=(const V4L2Subdevice &) = delete; @@ -46,8 +51,10 @@ public: ImageFormats formats(unsigned int pad); - int getFormat(unsigned int pad, V4L2SubdeviceFormat *format); - int setFormat(unsigned int pad, V4L2SubdeviceFormat *format); + int getFormat(unsigned int pad, V4L2SubdeviceFormat *format, + Whence whence = ActiveFormat); + int setFormat(unsigned int pad, V4L2SubdeviceFormat *format, + Whence whence = ActiveFormat); static V4L2Subdevice *fromEntityName(const MediaDevice *media, const std::string &entity); diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 8b9da81e8ab3..fb9bdbce2610 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -95,6 +95,15 @@ const std::string V4L2SubdeviceFormat::toString() const * any device left open will be closed, and any resources released. */ +/** + * \enum V4L2Subdevice::Whence + * \brief Specify the type of format for getFormat() and setFormat() operations + * \var V4L2Subdevice::ActiveFormat + * \brief The format operation applies to ACTIVE formats + * \var V4L2Subdevice::TryFormat + * \brief The format operation applies to TRY formats + */ + /** * \brief Create a V4L2 subdevice from a MediaEntity using its device node * path @@ -184,12 +193,15 @@ ImageFormats V4L2Subdevice::formats(unsigned int pad) * \brief Retrieve the image format set on one of the V4L2 subdevice pads * \param[in] pad The 0-indexed pad number the format is to be retrieved from * \param[out] format The image bus format + * \param[in] whence The format to retrieve, ACTIVE or TRY * \return 0 on success or a negative error code otherwise */ -int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format) +int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format, + Whence whence) { struct v4l2_subdev_format subdevFmt = {}; - subdevFmt.which = V4L2_SUBDEV_FORMAT_ACTIVE; + subdevFmt.which = whence == ActiveFormat ? V4L2_SUBDEV_FORMAT_ACTIVE + : V4L2_SUBDEV_FORMAT_TRY; subdevFmt.pad = pad; int ret = ioctl(VIDIOC_SUBDEV_G_FMT, &subdevFmt); @@ -211,6 +223,7 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format) * \brief Set an image format on one of the V4L2 subdevice pads * \param[in] pad The 0-indexed pad number the format is to be applied to * \param[inout] format The image bus format to apply to the subdevice's pad + * \param[in] whence The format to set, ACTIVE or TRY * * Apply the requested image format to the desired media pad and return the * actually applied format parameters, as \ref V4L2Subdevice::getFormat would @@ -218,10 +231,12 @@ int V4L2Subdevice::getFormat(unsigned int pad, V4L2SubdeviceFormat *format) * * \return 0 on success or a negative error code otherwise */ -int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format) +int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format, + Whence whence) { struct v4l2_subdev_format subdevFmt = {}; - subdevFmt.which = V4L2_SUBDEV_FORMAT_ACTIVE; + subdevFmt.which = whence == ActiveFormat ? V4L2_SUBDEV_FORMAT_ACTIVE + : V4L2_SUBDEV_FORMAT_TRY; subdevFmt.pad = pad; subdevFmt.format.width = format->size.width; subdevFmt.format.height = format->size.height; From patchwork Mon Mar 16 21:43:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 3121 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D5CB7629AE for ; Mon, 16 Mar 2020 22:43:23 +0100 (CET) Received: from pendragon.bb.dnainternet.fi (81-175-216-236.bb.dnainternet.fi [81.175.216.236]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id ACD48AD8; Mon, 16 Mar 2020 22:43:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1584395003; bh=sYEj5f1OrBoH6UbQEUW5Sb/y9bAOVW0zGOQKWfBWLz4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OiJNVgy0gdNJITqmLOfc7yu9E6w/csyE9w0AjbdXQzMStMr3TBbC/3+sfRfo8Wdz3 nCzhb4z2Ccxe5RiJAlCLHYEM96fJiipC/SlyS1/hWzHL5DjbkajwLvNxW7Qw4PXyN7 LBCik0y6uVIpE1JjTv+mwZh4JKaqYOD5RuXRsLqI= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: Martijn Braam , Mickael GUENE , Benjamin GAIGNARD , Andrey Konovalov Date: Mon, 16 Mar 2020 23:43:04 +0200 Message-Id: <20200316214310.27665-5-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> References: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 04/10] libcamera: v4l2_videodevice: Support filtering formats by media bus code X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2020 21:43:24 -0000 Add support for the recent V4L2 extension to VIDIOC_ENUM_FMT that allows filtering pixel formats by media bus codes. Signed-off-by: Laurent Pinchart --- src/libcamera/include/v4l2_videodevice.h | 4 ++-- src/libcamera/v4l2_videodevice.cpp | 16 +++++++++++++--- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/src/libcamera/include/v4l2_videodevice.h b/src/libcamera/include/v4l2_videodevice.h index 2507daf23efa..7196cabd09c7 100644 --- a/src/libcamera/include/v4l2_videodevice.h +++ b/src/libcamera/include/v4l2_videodevice.h @@ -184,7 +184,7 @@ public: int getFormat(V4L2DeviceFormat *format); int setFormat(V4L2DeviceFormat *format); - ImageFormats formats(); + ImageFormats formats(uint32_t code = 0); int setCrop(Rectangle *rect); int setCompose(Rectangle *rect); @@ -222,7 +222,7 @@ private: int getFormatSingleplane(V4L2DeviceFormat *format); int setFormatSingleplane(V4L2DeviceFormat *format); - std::vector enumPixelformats(); + std::vector enumPixelformats(uint32_t code); std::vector enumSizes(unsigned int pixelFormat); int setSelection(unsigned int target, Rectangle *rect); diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index f17e0136b5cb..084394fa5380 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -896,16 +896,19 @@ int V4L2VideoDevice::setFormatSingleplane(V4L2DeviceFormat *format) /** * \brief Enumerate all pixel formats and frame sizes + * \param[in] code Restrict formats to this media bus code. * * Enumerate all pixel formats and frame sizes supported by the video device. + * If the \a code argument is not zero, only formats compatible with that media + * bus code will be enumerated. * * \return A list of the supported video device formats */ -ImageFormats V4L2VideoDevice::formats() +ImageFormats V4L2VideoDevice::formats(uint32_t code) { ImageFormats formats; - for (unsigned int pixelformat : enumPixelformats()) { + for (unsigned int pixelformat : enumPixelformats(code)) { std::vector sizes = enumSizes(pixelformat); if (sizes.empty()) return {}; @@ -921,7 +924,7 @@ ImageFormats V4L2VideoDevice::formats() return formats; } -std::vector V4L2VideoDevice::enumPixelformats() +std::vector V4L2VideoDevice::enumPixelformats(uint32_t code) { std::vector formats; int ret; @@ -930,11 +933,18 @@ std::vector V4L2VideoDevice::enumPixelformats() struct v4l2_fmtdesc pixelformatEnum = {}; pixelformatEnum.index = index; pixelformatEnum.type = bufferType_; + pixelformatEnum.mbus_code = code; ret = ioctl(VIDIOC_ENUM_FMT, &pixelformatEnum); if (ret) break; + if (code && !(pixelformatEnum.flags & V4L2_FMT_FLAG_MBUS_CODE)) { + LOG(V4L2, Error) + << "Media bus code filtering not supported by the device"; + return {}; + } + formats.push_back(pixelformatEnum.pixelformat); } From patchwork Mon Mar 16 21:43:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 3122 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 927AA62928 for ; Mon, 16 Mar 2020 22:43:25 +0100 (CET) Received: from pendragon.bb.dnainternet.fi (81-175-216-236.bb.dnainternet.fi [81.175.216.236]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E73151414; Mon, 16 Mar 2020 22:43:23 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1584395005; bh=oSPvaRvnGMjupYnjHIiXCmNYyNgKsUMDtpXeaTLY+pw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mdoS9ImYus5FsH36IFhxF9PMTPKwyi6ValaaCnwV6J8x/3f7HyHn2AxDXINFeIp4E rps/e5steJghYZxP4MhbqBy7YcI0GI1TEbCH4WHuWZ963jwrKey6Eob9ljx7BIb3K2 FGNegiZrOOTV6fy9bQJWYGLOMQ+LKrr8BQ7PZM8A= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: Martijn Braam , Mickael GUENE , Benjamin GAIGNARD , Andrey Konovalov Date: Mon, 16 Mar 2020 23:43:05 +0200 Message-Id: <20200316214310.27665-6-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> References: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 05/10] libcamera: v4l2_videodevice: Expose the device capabilities X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2020 21:43:25 -0000 Add a caps() function that exposes the V4L2 capabilities for the device. This is useful for generic code that can't hardcode any a priori knowledge of the device, such as in a simple pipeline handler. Signed-off-by: Laurent Pinchart Reviewed-by: Niklas Söderlund Reviewed-by: Jacopo Mondi --- src/libcamera/include/v4l2_videodevice.h | 2 ++ src/libcamera/v4l2_videodevice.cpp | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/src/libcamera/include/v4l2_videodevice.h b/src/libcamera/include/v4l2_videodevice.h index 7196cabd09c7..16fc6b60de02 100644 --- a/src/libcamera/include/v4l2_videodevice.h +++ b/src/libcamera/include/v4l2_videodevice.h @@ -182,6 +182,8 @@ public: const char *deviceName() const { return caps_.card(); } const char *busName() const { return caps_.bus_info(); } + const V4L2Capability &caps() const { return caps_; } + int getFormat(V4L2DeviceFormat *format); int setFormat(V4L2DeviceFormat *format); ImageFormats formats(uint32_t code = 0); diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 084394fa5380..f5a925d97b63 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -682,6 +682,12 @@ void V4L2VideoDevice::close() * \return The string containing the device location */ +/** + * \fn V4L2VideoDevice::caps() + * \brief Retrieve the device V4L2 capabilities + * \return The device V4L2 capabilities + */ + std::string V4L2VideoDevice::logPrefix() const { return deviceNode() + (V4L2_TYPE_IS_OUTPUT(bufferType_) ? "[out]" : "[cap]"); From patchwork Mon Mar 16 21:43:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 3123 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 73AB56299E for ; Mon, 16 Mar 2020 22:43:27 +0100 (CET) Received: from pendragon.bb.dnainternet.fi (81-175-216-236.bb.dnainternet.fi [81.175.216.236]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C1ED2AC1; Mon, 16 Mar 2020 22:43:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1584395007; bh=qp1IwdmpGTF15rtbkuVBmpldWAU1tfcElyuXFFKBvaE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iJwNDdYM1+hkTtSiJ4h5C21JMkzMTNONENzPwMl0iHF9uHGdMJSodyTsNFiEpStlj pawWw1y/se1UbasyKyFTJelg/vieXaDWFThQy/aM1LLFckbPjpU2A1vSWcR2ybRdXQ 4tvTh4oMlMfBQqidWddwPrDgajwWUPclqcRZvZP0= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: Martijn Braam , Mickael GUENE , Benjamin GAIGNARD , Andrey Konovalov Date: Mon, 16 Mar 2020 23:43:06 +0200 Message-Id: <20200316214310.27665-7-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> References: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 06/10] libcamera: v4l2_videodevice: Downgrade 4CC conversion errors to warnings X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2020 21:43:27 -0000 Failing to convert between V4L2 and DRM 4CCs indicates something is likely wrong, but doesn't necessarily prevent the camera from being usable. It may for instance only limit the number of supported formats. Downgrade the related log messages from LogError to LogWarning. Signed-off-by: Laurent Pinchart --- src/libcamera/v4l2_videodevice.cpp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index f5a925d97b63..28de3d2fe5c6 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -1626,7 +1626,7 @@ PixelFormat V4L2VideoDevice::toPixelFormat(uint32_t v4l2Fourcc) * class. Until we fix the logger, work around it. */ libcamera::_log(__FILE__, __LINE__, _LOG_CATEGORY(V4L2)(), - LogError).stream() + LogWarning).stream() << "Unsupported V4L2 pixel format " << utils::hex(v4l2Fourcc); return 0; @@ -1707,7 +1707,8 @@ uint32_t V4L2VideoDevice::toV4L2Fourcc(PixelFormat pixelFormat, bool multiplanar * \todo We can't use LOG() in a static method of a Loggable * class. Until we fix the logger, work around it. */ - libcamera::_log(__FILE__, __LINE__, _LOG_CATEGORY(V4L2)(), LogError).stream() + libcamera::_log(__FILE__, __LINE__, _LOG_CATEGORY(V4L2)(), + LogWarning).stream() << "Unsupported V4L2 pixel format " << utils::hex(pixelFormat); return 0; From patchwork Mon Mar 16 21:43:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 3124 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 876B262923 for ; Mon, 16 Mar 2020 22:43:28 +0100 (CET) Received: from pendragon.bb.dnainternet.fi (81-175-216-236.bb.dnainternet.fi [81.175.216.236]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9CFB7AD8; Mon, 16 Mar 2020 22:43:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1584395008; bh=6izEgDMmxR2rEEL5UKfYi0Sfs/I/Em6JXum7w1dTgBg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UyF6kUH9mvbpe/jLWtJqZ0g8alrs8uhikd30DGFF1uotdLYzQ8nvGq+Erb2cbpwC4 ast2xEAkyPwOmNHmd9zOrrqr0ll5FxoJFFQyqgNMWeAEoNJ7Lz0TsJkS6isMBfmPLX qbqXGcJgi8bBSJKLlwbk6xERPNY9M/e7etDiDRjY= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: Martijn Braam , Mickael GUENE , Benjamin GAIGNARD , Andrey Konovalov Date: Mon, 16 Mar 2020 23:43:07 +0200 Message-Id: <20200316214310.27665-8-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> References: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 07/10] libcamera: v4l2_videodevice: Map V4L2_PIX_FMT_GREY to DRM FourCC X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2020 21:43:28 -0000 DRM has a format for 8-bit greyscale data, DRM_FORMAT_R8. Despite the 'R' name, which comes from GL/Vulkan to mean single-channel data, the format maps to greyscale for display. We can thus map it to V4L2_PIX_FMT_GREY. Signed-off-by: Laurent Pinchart --- src/libcamera/v4l2_videodevice.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 28de3d2fe5c6..8e56e934b04f 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -1614,12 +1614,15 @@ PixelFormat V4L2VideoDevice::toPixelFormat(uint32_t v4l2Fourcc) case V4L2_PIX_FMT_NV21M: return DRM_FORMAT_NV21; + /* Greyscale formats. */ + case V4L2_PIX_FMT_GREY: + return DRM_FORMAT_R8; + /* Compressed formats. */ case V4L2_PIX_FMT_MJPEG: return DRM_FORMAT_MJPEG; /* V4L2 formats not yet supported by DRM. */ - case V4L2_PIX_FMT_GREY: default: /* * \todo We can't use LOG() in a static method of a Loggable @@ -1698,6 +1701,10 @@ uint32_t V4L2VideoDevice::toV4L2Fourcc(PixelFormat pixelFormat, bool multiplanar case DRM_FORMAT_NV21: return V4L2_PIX_FMT_NV21; + /* Greyscale formats. */ + case DRM_FORMAT_R8: + return V4L2_PIX_FMT_GREY; + /* Compressed formats. */ case DRM_FORMAT_MJPEG: return V4L2_PIX_FMT_MJPEG; From patchwork Mon Mar 16 21:43:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 3125 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 432CD62923 for ; Mon, 16 Mar 2020 22:43:29 +0100 (CET) Received: from pendragon.bb.dnainternet.fi (81-175-216-236.bb.dnainternet.fi [81.175.216.236]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 81750AC1; Mon, 16 Mar 2020 22:43:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1584395008; bh=UvNlc3+H7nmSZA9CeI63GH7eeffB/qxRdFaTWNnsQPo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DEijf6b/fdTFKVqF5AL/EflYoa6euufDhMlXRsJ49+FhHY8rZqsv8UgMpEpE9vPI/ TlfuioiPt6TAHp/OKJfW87hp+2AJj5hVW7kYsDGMbTeSZvqGa0ZBV5R/d5IndO61j7 IMsAv3+yuZfIzfdq+cGWH6NSsszff5vtj067qFCo= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: Martijn Braam , Mickael GUENE , Benjamin GAIGNARD , Andrey Konovalov Date: Mon, 16 Mar 2020 23:43:08 +0200 Message-Id: <20200316214310.27665-9-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> References: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 08/10] libcamera: pipeline: Add a simple pipeline handler X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2020 21:43:29 -0000 From: Martijn Braam This new pipeline handler aims at supporting any simple device without requiring any device-specific code. Simple devices are currently defined as a graph made of one or multiple camera sensors and a single video node, with each sensor connected to the video node through a linear pipeline. The simple pipeline handler will automatically parse the media graph, enumerate sensors, build supported stream configurations, and configure the pipeline, without any device-specific knowledge. It doesn't support configuration of any processing in the pipeline at the moment, but may be extended to support simple processing such as format conversion or scaling in the future. The only device-specific information in the pipeline handler is the list of supported drivers, required for device matching. We may be able to remove this in the future by matching with the simple pipeline handler as a last resort option, after all other pipeline handlers have been tried. Signed-off-by: Martijn Braam Signed-off-by: Laurent Pinchart --- Changes since v1: - Rebase on top of buffer API rework - Expose stream formats - Rework camera data config --- src/libcamera/pipeline/meson.build | 1 + src/libcamera/pipeline/simple/meson.build | 3 + src/libcamera/pipeline/simple/simple.cpp | 699 ++++++++++++++++++++++ 3 files changed, 703 insertions(+) create mode 100644 src/libcamera/pipeline/simple/meson.build create mode 100644 src/libcamera/pipeline/simple/simple.cpp diff --git a/src/libcamera/pipeline/meson.build b/src/libcamera/pipeline/meson.build index 0d466225a72e..606ba31a0319 100644 --- a/src/libcamera/pipeline/meson.build +++ b/src/libcamera/pipeline/meson.build @@ -5,3 +5,4 @@ libcamera_sources += files([ subdir('ipu3') subdir('rkisp1') +subdir('simple') diff --git a/src/libcamera/pipeline/simple/meson.build b/src/libcamera/pipeline/simple/meson.build new file mode 100644 index 000000000000..4945a3e173cf --- /dev/null +++ b/src/libcamera/pipeline/simple/meson.build @@ -0,0 +1,3 @@ +libcamera_sources += files([ + 'simple.cpp', +]) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp new file mode 100644 index 000000000000..2126799c54eb --- /dev/null +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -0,0 +1,699 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Laurent Pinchart + * Copyright (C) 2019, Martijn Braam + * + * simple.cpp - Pipeline handler for simple pipelines + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include +#include +#include + +#include "camera_sensor.h" +#include "device_enumerator.h" +#include "log.h" +#include "media_device.h" +#include "pipeline_handler.h" +#include "v4l2_subdevice.h" +#include "v4l2_videodevice.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(SimplePipeline) + +class SimplePipelineHandler; + +class SimpleCameraData : public CameraData +{ +public: + SimpleCameraData(PipelineHandler *pipe, MediaEntity *sensor, + MediaEntity *video); + + bool isValid() const { return sensor_ != nullptr; } + std::set streams() { return { &stream_ }; } + + int init(); + int setupLinks(); + int setupFormats(V4L2SubdeviceFormat *format, + V4L2Subdevice::Whence whence); + + struct Entity { + MediaEntity *entity; + MediaLink *link; + }; + + struct Configuration { + uint32_t code; + PixelFormat pixelFormat; + Size size; + }; + + Stream stream_; + std::unique_ptr sensor_; + std::list entities_; + + std::vector configs_; + std::map formats_; +}; + +class SimpleCameraConfiguration : public CameraConfiguration +{ +public: + SimpleCameraConfiguration(Camera *camera, SimpleCameraData *data); + + Status validate() override; + + const V4L2SubdeviceFormat &sensorFormat() { return sensorFormat_; } + +private: + /* + * The SimpleCameraData instance is guaranteed to be valid as long as + * the corresponding Camera instance is valid. In order to borrow a + * reference to the camera data, store a new reference to the camera. + */ + std::shared_ptr camera_; + const SimpleCameraData *data_; + + V4L2SubdeviceFormat sensorFormat_; +}; + +class SimplePipelineHandler : public PipelineHandler +{ +public: + SimplePipelineHandler(CameraManager *manager); + ~SimplePipelineHandler(); + + CameraConfiguration *generateConfiguration(Camera *camera, + const StreamRoles &roles) override; + int configure(Camera *camera, CameraConfiguration *config) override; + + int exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) override; + + int start(Camera *camera) override; + void stop(Camera *camera) override; + + bool match(DeviceEnumerator *enumerator) override; + + V4L2VideoDevice *video() { return video_; } + V4L2Subdevice *subdev(const MediaEntity *entity); + +protected: + int queueRequestDevice(Camera *camera, Request *request) override; + +private: + SimpleCameraData *cameraData(const Camera *camera) + { + return static_cast( + PipelineHandler::cameraData(camera)); + } + + int initLinks(); + + int createCamera(MediaEntity *sensor); + + void bufferReady(FrameBuffer *buffer); + + MediaDevice *media_; + V4L2VideoDevice *video_; + std::map subdevs_; + + Camera *activeCamera_; +}; + +/* ----------------------------------------------------------------------------- + * Camera Data + */ +SimpleCameraData::SimpleCameraData(PipelineHandler *pipe, MediaEntity *sensor, + MediaEntity *video) + : CameraData(pipe) +{ + int ret; + + /* + * Walk the pipeline towards the video node and store all entities + * along the way. + */ + MediaEntity *source = sensor; + + while (source) { + /* If we have reached the video node, we're done. */ + if (source == video) + break; + + /* Use the first output pad that has links. */ + MediaPad *sourcePad = nullptr; + for (MediaPad *pad : source->pads()) { + if ((pad->flags() & MEDIA_PAD_FL_SOURCE) && + !pad->links().empty()) { + sourcePad = pad; + break; + } + } + + if (!sourcePad) + return; + + /* Use the first link that isn't immutable and disabled. */ + MediaLink *sourceLink = nullptr; + for (MediaLink *link : sourcePad->links()) { + if ((link->flags() & MEDIA_LNK_FL_ENABLED) || + !(link->flags() & MEDIA_LNK_FL_IMMUTABLE)) { + sourceLink = link; + break; + } + } + + if (!sourceLink) + return; + + entities_.push_back({ source, sourceLink }); + + source = sourceLink->sink()->entity(); + + /* Avoid infinite loops. */ + auto iter = std::find_if(entities_.begin(), entities_.end(), + [&](const Entity &entity) { + return entity.entity == source; + }); + if (iter != entities_.end()) { + LOG(SimplePipeline, Info) << "Loop detected in pipeline"; + return; + } + } + + /* We have a valid pipeline, create the camera sensor. */ + sensor_ = std::make_unique(sensor); + ret = sensor_->init(); + if (ret) { + sensor_.reset(); + return; + } +} + +int SimpleCameraData::init() +{ + SimplePipelineHandler *pipe = static_cast(pipe_); + V4L2VideoDevice *video = pipe->video(); + int ret; + + /* + * Enumerate the possible pipeline configurations. For each media bus + * format supported by the sensor, propagate the formats through the + * pipeline, and enumerate the corresponding possible V4L2 pixel + * formats on the video node. + */ + for (unsigned int code : sensor_->mbusCodes()) { + V4L2SubdeviceFormat format{ code, sensor_->resolution() }; + + /* + * Setup links first as some subdev drivers take active links + * into account to propaget TRY formats. So is life :-( + */ + ret = setupLinks(); + if (ret < 0) + return ret; + + ret = setupFormats(&format, V4L2Subdevice::TryFormat); + if (ret < 0) + return ret; + + std::vector formats = + video->formats(format.mbus_code).formats(); + + LOG(SimplePipeline, Debug) + << "Adding configuration for " << format.size.toString() + << " in pixel formats [ " + << utils::join(formats, ", ", + [](unsigned int f) { return std::to_string(f); }) + << " ]"; + + /* + * Store the configuration in the formats_ map, mapping + * PixelFormat to configuration. Any previously stored value is + * overwritten, as the pipeline handler currently doesn't care + * about how a particular PixelFormat is achieved. + */ + for (unsigned int v4l2Format : formats) { + PixelFormat pixelFormat = video->toPixelFormat(v4l2Format); + if (!pixelFormat) + continue; + + Configuration config; + config.code = code; + config.pixelFormat = pixelFormat; + config.size = format.size; + + formats_[pixelFormat] = config; + } + } + + if (formats_.empty()) { + LOG(SimplePipeline, Error) << "No valid configuration found"; + return -EINVAL; + } + + return 0; +} + +int SimpleCameraData::setupLinks() +{ + int ret; + + /* + * Configure all links along the pipeline. Some entities may not allow + * multiple sink links to be enabled together, even on different sink + * pads. We must thus start by disabling all sink links (but the one we + * want to enable) before enabling the pipeline link. + */ + for (SimpleCameraData::Entity &e : entities_) { + for (MediaPad *pad : e.link->sink()->entity()->pads()) { + for (MediaLink *link : pad->links()) { + if (link == e.link) + continue; + + if ((link->flags() & MEDIA_LNK_FL_ENABLED) && + !(link->flags() & MEDIA_LNK_FL_IMMUTABLE)) { + ret = link->setEnabled(false); + if (ret < 0) + return ret; + } + } + } + + if (!(e.link->flags() & MEDIA_LNK_FL_ENABLED)) { + ret = e.link->setEnabled(true); + if (ret < 0) + return ret; + } + } + + return 0; +} + +int SimpleCameraData::setupFormats(V4L2SubdeviceFormat *format, + V4L2Subdevice::Whence whence) +{ + SimplePipelineHandler *pipe = static_cast(pipe_); + int ret; + + /* + * Configure the format on the sensor output and propagate it through + * the pipeline. + */ + ret = sensor_->setFormat(format); + if (ret < 0) + return ret; + + for (const Entity &e : entities_) { + MediaLink *link = e.link; + MediaPad *source = link->source(); + MediaPad *sink = link->sink(); + + if (source->entity() != sensor_->entity()) { + V4L2Subdevice *subdev = pipe->subdev(source->entity()); + subdev->getFormat(source->index(), format, whence); + } + + if (sink->entity()->function() != MEDIA_ENT_F_IO_V4L) { + V4L2Subdevice *subdev = pipe->subdev(sink->entity()); + subdev->setFormat(sink->index(), format, whence); + } + + LOG(SimplePipeline, Debug) + << "Link '" << source->entity()->name() + << "':" << source->index() + << " -> '" << sink->entity()->name() + << "':" << sink->index() + << " configured with format " << format->toString(); + } + + return 0; +} + +/* ----------------------------------------------------------------------------- + * Camera Configuration + */ + +SimpleCameraConfiguration::SimpleCameraConfiguration(Camera *camera, + SimpleCameraData *data) + : CameraConfiguration(), camera_(camera->shared_from_this()), + data_(data) +{ +} + +CameraConfiguration::Status SimpleCameraConfiguration::validate() +{ + Status status = Valid; + + if (config_.empty()) + return Invalid; + + /* Cap the number of entries to the available streams. */ + if (config_.size() > 1) { + config_.resize(1); + status = Adjusted; + } + + StreamConfiguration &cfg = config_[0]; + + /* Adjust the pixel format. */ + auto it = data_->formats_.find(cfg.pixelFormat); + if (it == data_->formats_.end()) + it = data_->formats_.begin(); + + PixelFormat pixelFormat = it->first; + const SimpleCameraData::Configuration &pipeConfig = it->second; + + if (cfg.pixelFormat != pixelFormat) { + LOG(SimplePipeline, Debug) << "Adjusting pixel format"; + cfg.pixelFormat = pixelFormat; + status = Adjusted; + } + + if (cfg.size != pipeConfig.size) { + LOG(SimplePipeline, Debug) + << "Adjusting size from " << cfg.size.toString() + << " to " << pipeConfig.size.toString(); + cfg.size = pipeConfig.size; + status = Adjusted; + } + + cfg.bufferCount = 3; + + return status; +} + +/* ----------------------------------------------------------------------------- + * Pipeline Handler + */ + +SimplePipelineHandler::SimplePipelineHandler(CameraManager *manager) + : PipelineHandler(manager), video_(nullptr) +{ +} + +SimplePipelineHandler::~SimplePipelineHandler() +{ + delete video_; +} + +CameraConfiguration *SimplePipelineHandler::generateConfiguration(Camera *camera, + const StreamRoles &roles) +{ + SimpleCameraData *data = cameraData(camera); + CameraConfiguration *config = + new SimpleCameraConfiguration(camera, data); + + if (roles.empty()) + return config; + + /* Create the formats map. */ + std::map> formats; + std::transform(data->formats_.begin(), data->formats_.end(), + std::inserter(formats, formats.end()), + [](const auto &format) -> decltype(formats)::value_type { + const PixelFormat &pixelFormat = format.first; + const Size &size = format.second.size; + return { pixelFormat, { size } }; + }); + + /* + * Create the stream configuration. Take the first entry in the formats + * map as the default, for lack of a better option. + */ + StreamConfiguration cfg{ StreamFormats{ formats } }; + cfg.pixelFormat = formats.begin()->first; + cfg.size = formats.begin()->second[0].max; + + config->addConfiguration(cfg); + + config->validate(); + + return config; +} + +int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) +{ + SimpleCameraConfiguration *config = + static_cast(c); + SimpleCameraData *data = cameraData(camera); + StreamConfiguration &cfg = config->at(0); + int ret; + + /* + * Configure links on the pipeline and propagate formats from the + * sensor to the video node. + */ + ret = data->setupLinks(); + if (ret < 0) + return ret; + + const SimpleCameraData::Configuration &pipeConfig = + data->formats_[cfg.pixelFormat]; + + V4L2SubdeviceFormat format{ pipeConfig.code, data->sensor_->resolution() }; + + ret = data->setupFormats(&format, V4L2Subdevice::ActiveFormat); + if (ret < 0) + return ret; + + /* Configure the video node. */ + uint32_t fourcc = video_->toV4L2Fourcc(cfg.pixelFormat); + + V4L2DeviceFormat outputFormat = {}; + outputFormat.fourcc = fourcc; + outputFormat.size = cfg.size; + + ret = video_->setFormat(&outputFormat); + if (ret) + return ret; + + if (outputFormat.size != cfg.size || outputFormat.fourcc != fourcc) { + LOG(SimplePipeline, Error) + << "Unable to configure capture in " << cfg.toString(); + return -EINVAL; + } + + cfg.setStream(&data->stream_); + + return 0; +} + +int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) +{ + unsigned int count = stream->configuration().bufferCount; + + return video_->exportBuffers(count, buffers); +} + +int SimplePipelineHandler::start(Camera *camera) +{ + SimpleCameraData *data = cameraData(camera); + unsigned int count = data->stream_.configuration().bufferCount; + + int ret = video_->importBuffers(count); + if (ret < 0) + return ret; + + ret = video_->streamOn(); + if (ret < 0) { + video_->releaseBuffers(); + return ret; + } + + activeCamera_ = camera; + + return 0; +} + +void SimplePipelineHandler::stop(Camera *camera) +{ + video_->streamOff(); + video_->releaseBuffers(); + activeCamera_ = nullptr; +} + +int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) +{ + SimpleCameraData *data = cameraData(camera); + Stream *stream = &data->stream_; + + FrameBuffer *buffer = request->findBuffer(stream); + if (!buffer) { + LOG(SimplePipeline, Error) + << "Attempt to queue request with invalid stream"; + return -ENOENT; + } + + return video_->queueBuffer(buffer); +} + +/* ----------------------------------------------------------------------------- + * Match and Setup + */ + +bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) +{ + static const char * const drivers[] = { + "imx7-csi", + "sun6i-csi", + }; + + for (const char *driver : drivers) { + DeviceMatch dm(driver); + media_ = acquireMediaDevice(enumerator, dm); + if (media_) + break; + } + + if (!media_) + return false; + + /* + * Locate sensors and video nodes. We only support pipelines with at + * least one sensor and exactly one video captude node. + */ + std::vector sensors; + std::vector videos; + + for (MediaEntity *entity : media_->entities()) { + switch (entity->function()) { + case MEDIA_ENT_F_CAM_SENSOR: + sensors.push_back(entity); + break; + + case MEDIA_ENT_F_IO_V4L: + if (entity->pads().size() == 1 && + (entity->pads()[0]->flags() & MEDIA_PAD_FL_SINK)) + videos.push_back(entity); + break; + + default: + break; + } + } + + if (sensors.empty()) { + LOG(SimplePipeline, Info) << "No sensor found"; + return false; + } + + if (videos.size() != 1) { + LOG(SimplePipeline, Info) + << "Pipeline with " << videos.size() + << " video capture nodes is not supported"; + return false; + } + + /* Locate and open the capture video node. */ + video_ = new V4L2VideoDevice(videos[0]); + if (video_->open() < 0) + return false; + + if (video_->caps().isMultiplanar()) { + LOG(SimplePipeline, Info) + << "V4L2 multiplanar devices are not supported"; + return false; + } + + video_->bufferReady.connect(this, &SimplePipelineHandler::bufferReady); + + /* + * Create one camera data instance for each sensor and gather all + * entities in all pipelines. + */ + std::vector> pipelines; + std::set entities; + + pipelines.reserve(sensors.size()); + + for (MediaEntity *sensor : sensors) { + std::unique_ptr data = + std::make_unique(this, sensor, + videos[0]); + if (!data->isValid()) { + LOG(SimplePipeline, Info) + << "No valid pipeline for sensor '" + << sensor->name() << "', skipping"; + continue; + } + + for (SimpleCameraData::Entity &entity : data->entities_) + entities.insert(entity.entity); + + pipelines.push_back(std::move(data)); + } + + if (entities.empty()) + return false; + + /* Create and open V4L2Subdev instances for all the entities. */ + for (MediaEntity *entity : entities) { + auto elem = subdevs_.emplace(std::piecewise_construct, + std::forward_as_tuple(entity), + std::forward_as_tuple(entity)); + V4L2Subdevice *subdev = &elem.first->second; + int ret = subdev->open(); + if (ret < 0) { + LOG(SimplePipeline, Error) + << "Failed to open " << subdev->deviceNode() + << ": " << strerror(-ret); + return false; + } + } + + /* Initialize each pipeline and register a corresponding camera. */ + for (std::unique_ptr &data : pipelines) { + int ret = data->init(); + if (ret < 0) + continue; + + std::shared_ptr camera = + Camera::create(this, data->sensor_->entity()->name(), + data->streams()); + registerCamera(std::move(camera), std::move(data)); + } + + return true; +} + +V4L2Subdevice *SimplePipelineHandler::subdev(const MediaEntity *entity) +{ + auto iter = subdevs_.find(entity); + if (iter == subdevs_.end()) + return nullptr; + + return &iter->second; +} + +/* ----------------------------------------------------------------------------- + * Buffer Handling + */ + +void SimplePipelineHandler::bufferReady(FrameBuffer *buffer) +{ + ASSERT(activeCamera_); + Request *request = buffer->request(); + completeBuffer(activeCamera_, request, buffer); + completeRequest(activeCamera_, request); +} + +REGISTER_PIPELINE_HANDLER(SimplePipelineHandler); + +} /* namespace libcamera */ From patchwork Mon Mar 16 21:43:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 3126 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D7BA06297B for ; Mon, 16 Mar 2020 22:43:29 +0100 (CET) Received: from pendragon.bb.dnainternet.fi (81-175-216-236.bb.dnainternet.fi [81.175.216.236]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3551CFC9; Mon, 16 Mar 2020 22:43:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1584395009; bh=BIKYf4ixBkRv9bMgw9GuSPzsRQR41cUy1FWPCe8iVWs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PLHFmrn205FO2ROXtgKmLszvz4UXxhK7pxIVHAJyFsNsWEULmDofzoXmCvbP6VO6J FyDx+84a3za2gjeXhuy/COz3g2slbnc42dlyVKROxkszVQHOCAAjHlXo6jxGHWFLYv KGcMzPc1sqN4icvlmqN3oJTD7J8EmNkXBVX3aaYE= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: Martijn Braam , Mickael GUENE , Benjamin GAIGNARD , Andrey Konovalov Date: Mon, 16 Mar 2020 23:43:09 +0200 Message-Id: <20200316214310.27665-10-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> References: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 09/10] libcamera: pipeline: simple: Add simple format converter X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2020 21:43:30 -0000 The simple format converter supports V4L2 M2M devices that convert pixel formats. Signed-off-by: Laurent Pinchart --- src/libcamera/pipeline/simple/converter.cpp | 210 ++++++++++++++++++++ src/libcamera/pipeline/simple/converter.h | 60 ++++++ src/libcamera/pipeline/simple/meson.build | 1 + 3 files changed, 271 insertions(+) create mode 100644 src/libcamera/pipeline/simple/converter.cpp create mode 100644 src/libcamera/pipeline/simple/converter.h diff --git a/src/libcamera/pipeline/simple/converter.cpp b/src/libcamera/pipeline/simple/converter.cpp new file mode 100644 index 000000000000..10798b6cf174 --- /dev/null +++ b/src/libcamera/pipeline/simple/converter.cpp @@ -0,0 +1,210 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Laurent Pinchart + * + * converter.cpp - Format converter for simple pipeline handler + */ + +#include "converter.h" + +#include + +#include +#include +#include + +#include "log.h" +#include "media_device.h" +#include "v4l2_videodevice.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(SimplePipeline); + +SimpleConverter::SimpleConverter(MediaDevice *media) + : m2m_(nullptr) +{ + /* + * Locate the video node. There's no need to validate the pipeline + * further, the caller guarantees that this is a V4L2 mem2mem device. + */ + const std::vector &entities = media->entities(); + auto it = std::find_if(entities.begin(), entities.end(), + [](MediaEntity *entity) { + return entity->function() == MEDIA_ENT_F_IO_V4L; + }); + if (it == entities.end()) + return; + + m2m_ = new V4L2M2MDevice((*it)->deviceNode()); + + m2m_->output()->bufferReady.connect(this, &SimpleConverter::outputBufferReady); + m2m_->capture()->bufferReady.connect(this, &SimpleConverter::captureBufferReady); +} + +SimpleConverter::~SimpleConverter() +{ + delete m2m_; +} + +int SimpleConverter::open() +{ + if (!m2m_) + return -ENODEV; + + return m2m_->open(); +} + +void SimpleConverter::close() +{ + if (m2m_) + m2m_->close(); +} + +std::vector SimpleConverter::formats(PixelFormat input) +{ + if (!m2m_) + return {}; + + V4L2DeviceFormat format; + format.fourcc = m2m_->output()->toV4L2Fourcc(input); + format.size = { 1, 1 }; + + int ret = m2m_->output()->setFormat(&format); + if (ret < 0) { + LOG(SimplePipeline, Error) + << "Failed to set format: " << strerror(-ret); + return {}; + } + + std::vector pixelFormats; + + for (unsigned int format : m2m_->capture()->formats().formats()) { + PixelFormat pixelFormat = m2m_->capture()->toPixelFormat(format); + if (pixelFormat) + pixelFormats.push_back(pixelFormat); + } + + return pixelFormats; +} + +int SimpleConverter::configure(PixelFormat inputFormat, + PixelFormat outputFormat, const Size &size) +{ + V4L2DeviceFormat format; + unsigned int fourcc; + int ret; + + fourcc = m2m_->output()->toV4L2Fourcc(inputFormat); + format.fourcc = fourcc; + format.size = size; + + ret = m2m_->output()->setFormat(&format); + if (ret < 0) { + LOG(SimplePipeline, Error) + << "Failed to set input format: " << strerror(-ret); + return ret; + } + + if (format.fourcc != fourcc || format.size != size) { + LOG(SimplePipeline, Error) + << "Input format not supported"; + return -EINVAL; + } + + fourcc = m2m_->capture()->toV4L2Fourcc(outputFormat); + format.fourcc = fourcc; + + ret = m2m_->capture()->setFormat(&format); + if (ret < 0) { + LOG(SimplePipeline, Error) + << "Failed to set output format: " << strerror(-ret); + return ret; + } + + if (format.fourcc != fourcc || format.size != size) { + LOG(SimplePipeline, Error) + << "Output format not supported"; + return -EINVAL; + } + + return 0; +} + +int SimpleConverter::exportBuffers(unsigned int count, + std::vector> *buffers) +{ + return m2m_->capture()->exportBuffers(count, buffers); +} + +int SimpleConverter::start(unsigned int count) +{ + int ret = m2m_->output()->importBuffers(count); + if (ret < 0) + return ret; + + ret = m2m_->capture()->importBuffers(count); + if (ret < 0) { + stop(); + return ret; + } + + ret = m2m_->output()->streamOn(); + if (ret < 0) { + stop(); + return ret; + } + + ret = m2m_->capture()->streamOn(); + if (ret < 0) { + stop(); + return ret; + } + + return 0; +} + +void SimpleConverter::stop() +{ + m2m_->capture()->streamOff(); + m2m_->output()->streamOff(); + m2m_->capture()->releaseBuffers(); + m2m_->output()->releaseBuffers(); +} + +int SimpleConverter::queueBuffers(FrameBuffer *input, FrameBuffer *output) +{ + int ret = m2m_->output()->queueBuffer(input); + if (ret < 0) + return ret; + + ret = m2m_->capture()->queueBuffer(output); + if (ret < 0) + return ret; + + return 0; +} + +void SimpleConverter::captureBufferReady(FrameBuffer *buffer) +{ + if (!outputDoneQueue_.empty()) { + FrameBuffer *other = outputDoneQueue_.front(); + outputDoneQueue_.pop(); + bufferReady.emit(other, buffer); + } else { + captureDoneQueue_.push(buffer); + } +} + +void SimpleConverter::outputBufferReady(FrameBuffer *buffer) +{ + if (!captureDoneQueue_.empty()) { + FrameBuffer *other = captureDoneQueue_.front(); + captureDoneQueue_.pop(); + bufferReady.emit(buffer, other); + } else { + outputDoneQueue_.push(buffer); + } +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/simple/converter.h b/src/libcamera/pipeline/simple/converter.h new file mode 100644 index 000000000000..a33071fa8578 --- /dev/null +++ b/src/libcamera/pipeline/simple/converter.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Laurent Pinchart + * + * converter.h - Format converter for simple pipeline handler + */ + +#ifndef __LIBCAMERA_PIPELINE_SIMPLE_CONVERTER_H__ +#define __LIBCAMERA_PIPELINE_SIMPLE_CONVERTER_H__ + +#include +#include +#include + +#include +#include + +namespace libcamera { + +class FrameBuffer; +class MediaDevice; +struct Size; +class V4L2M2MDevice; + +class SimpleConverter +{ +public: + SimpleConverter(MediaDevice *media); + ~SimpleConverter(); + + int open(); + void close(); + + std::vector formats(PixelFormat input); + + int configure(PixelFormat inputFormat, PixelFormat outputFormat, + const Size &size); + int exportBuffers(unsigned int count, + std::vector> *buffers); + + int start(unsigned int count); + void stop(); + + int queueBuffers(FrameBuffer *input, FrameBuffer *output); + + Signal bufferReady; + +private: + void captureBufferReady(FrameBuffer *buffer); + void outputBufferReady(FrameBuffer *buffer); + + V4L2M2MDevice *m2m_; + + std::queue captureDoneQueue_; + std::queue outputDoneQueue_; +}; + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_PIPELINE_SIMPLE_CONVERTER_H__ */ diff --git a/src/libcamera/pipeline/simple/meson.build b/src/libcamera/pipeline/simple/meson.build index 4945a3e173cf..8372f24e3788 100644 --- a/src/libcamera/pipeline/simple/meson.build +++ b/src/libcamera/pipeline/simple/meson.build @@ -1,3 +1,4 @@ libcamera_sources += files([ + 'converter.cpp', 'simple.cpp', ]) From patchwork Mon Mar 16 21:43:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 3127 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 72FF56297B for ; Mon, 16 Mar 2020 22:43:30 +0100 (CET) Received: from pendragon.bb.dnainternet.fi (81-175-216-236.bb.dnainternet.fi [81.175.216.236]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C95841961; Mon, 16 Mar 2020 22:43:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1584395010; bh=EhPtWClmMeBryNcfmhlq56VHPc4NCWvum1YLv6gJZlc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FspJl5lRUahpoXVn4ckDj3x4/bleBQ1ZTi1BfhOmPmjEX0C3Sq7UMfVDgY8mSVe7j FRqzWjO162LR0gkNMarporRk5w96rLAv12iSgkHe6KozUzXY0Lm3af2Uq8ofVkC5kl a8mggofPEIypz350oYmtm9mLWzPvj2ao3GIGLjmQ= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: Martijn Braam , Mickael GUENE , Benjamin GAIGNARD , Andrey Konovalov Date: Mon, 16 Mar 2020 23:43:10 +0200 Message-Id: <20200316214310.27665-11-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> References: <20200316214310.27665-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 10/10] libcamera: pipeline: simple: Integrate converter support X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2020 21:43:30 -0000 Add support for an optional format converter, supported by the SimpleConverter class. If a converter is available for the pipeline, it will be used to expose additional pixel formats. Signed-off-by: Laurent Pinchart --- src/libcamera/pipeline/simple/simple.cpp | 192 ++++++++++++++++++++--- 1 file changed, 174 insertions(+), 18 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 2126799c54eb..218b28e70eb2 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -31,12 +32,19 @@ #include "v4l2_subdevice.h" #include "v4l2_videodevice.h" +#include "converter.h" + namespace libcamera { LOG_DEFINE_CATEGORY(SimplePipeline) class SimplePipelineHandler; +struct SimplePipelineInfo { + const char *driver; + const char *converter; +}; + class SimpleCameraData : public CameraData { public: @@ -79,6 +87,8 @@ public: const V4L2SubdeviceFormat &sensorFormat() { return sensorFormat_; } + bool needConversion() const { return needConversion_; } + private: /* * The SimpleCameraData instance is guaranteed to be valid as long as @@ -89,6 +99,7 @@ private: const SimpleCameraData *data_; V4L2SubdeviceFormat sensorFormat_; + bool needConversion_; }; class SimplePipelineHandler : public PipelineHandler @@ -111,6 +122,7 @@ public: V4L2VideoDevice *video() { return video_; } V4L2Subdevice *subdev(const MediaEntity *entity); + SimpleConverter *converter() { return converter_; } protected: int queueRequestDevice(Camera *camera, Request *request) override; @@ -127,11 +139,17 @@ private: int createCamera(MediaEntity *sensor); void bufferReady(FrameBuffer *buffer); + void converterDone(FrameBuffer *input, FrameBuffer *output); MediaDevice *media_; V4L2VideoDevice *video_; std::map subdevs_; + SimpleConverter *converter_; + bool useConverter_; + std::vector> converterBuffers_; + std::queue converterQueue_; + Camera *activeCamera_; }; @@ -209,6 +227,7 @@ int SimpleCameraData::init() { SimplePipelineHandler *pipe = static_cast(pipe_); V4L2VideoDevice *video = pipe->video(); + SimpleConverter *converter = pipe->converter(); int ret; /* @@ -258,7 +277,13 @@ int SimpleCameraData::init() config.pixelFormat = pixelFormat; config.size = format.size; - formats_[pixelFormat] = config; + if (!converter) { + formats_[pixelFormat] = config; + continue; + } + + for (PixelFormat format : converter->formats(pixelFormat)) + formats_[format] = config; } } @@ -385,6 +410,8 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() status = Adjusted; } + needConversion_ = cfg.pixelFormat != pipeConfig.pixelFormat; + if (cfg.size != pipeConfig.size) { LOG(SimplePipeline, Debug) << "Adjusting size from " << cfg.size.toString() @@ -403,13 +430,14 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() */ SimplePipelineHandler::SimplePipelineHandler(CameraManager *manager) - : PipelineHandler(manager), video_(nullptr) + : PipelineHandler(manager), video_(nullptr), converter_(nullptr) { } SimplePipelineHandler::~SimplePipelineHandler() { delete video_; + delete converter_; } CameraConfiguration *SimplePipelineHandler::generateConfiguration(Camera *camera, @@ -473,22 +501,37 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) return ret; /* Configure the video node. */ - uint32_t fourcc = video_->toV4L2Fourcc(cfg.pixelFormat); + uint32_t fourcc = video_->toV4L2Fourcc(pipeConfig.pixelFormat); - V4L2DeviceFormat outputFormat = {}; - outputFormat.fourcc = fourcc; - outputFormat.size = cfg.size; + V4L2DeviceFormat captureFormat = {}; + captureFormat.fourcc = fourcc; + captureFormat.size = cfg.size; - ret = video_->setFormat(&outputFormat); + ret = video_->setFormat(&captureFormat); if (ret) return ret; - if (outputFormat.size != cfg.size || outputFormat.fourcc != fourcc) { + if (captureFormat.fourcc != fourcc || captureFormat.size != cfg.size) { LOG(SimplePipeline, Error) << "Unable to configure capture in " << cfg.toString(); return -EINVAL; } + /* Configure the converter if required. */ + useConverter_ = config->needConversion(); + + if (useConverter_) { + int ret = converter_->configure(pipeConfig.pixelFormat, + cfg.pixelFormat, cfg.size); + if (ret < 0) { + LOG(SimplePipeline, Error) + << "Unable to configure converter"; + return ret; + } + + LOG(SimplePipeline, Debug) << "Using format converter"; + } + cfg.setStream(&data->stream_); return 0; @@ -499,24 +542,47 @@ int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, { unsigned int count = stream->configuration().bufferCount; - return video_->exportBuffers(count, buffers); + /* + * Export buffers on the converter or capture video node, depending on + * whether the converter is used or not. + */ + if (useConverter_) + return converter_->exportBuffers(count, buffers); + else + return video_->exportBuffers(count, buffers); } int SimplePipelineHandler::start(Camera *camera) { SimpleCameraData *data = cameraData(camera); unsigned int count = data->stream_.configuration().bufferCount; + int ret; - int ret = video_->importBuffers(count); + if (useConverter_) + ret = video_->allocateBuffers(count, &converterBuffers_); + else + ret = video_->importBuffers(count); if (ret < 0) return ret; ret = video_->streamOn(); if (ret < 0) { - video_->releaseBuffers(); + stop(camera); return ret; } + if (useConverter_) { + ret = converter_->start(count); + if (ret < 0) { + stop(camera); + return ret; + } + + /* Queue all internal buffers for capture. */ + for (std::unique_ptr &buffer : converterBuffers_) + video_->queueBuffer(buffer.get()); + } + activeCamera_ = camera; return 0; @@ -524,8 +590,13 @@ int SimplePipelineHandler::start(Camera *camera) void SimplePipelineHandler::stop(Camera *camera) { + if (useConverter_) + converter_->stop(); + video_->streamOff(); video_->releaseBuffers(); + + converterBuffers_.clear(); activeCamera_ = nullptr; } @@ -541,6 +612,15 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) return -ENOENT; } + /* + * If conversion is needed, push the buffer to the converter queue, it + * will be handed to the converter in the capture completion handler. + */ + if (useConverter_) { + converterQueue_.push(buffer); + return 0; + } + return video_->queueBuffer(buffer); } @@ -550,16 +630,21 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) { - static const char * const drivers[] = { - "imx7-csi", - "sun6i-csi", + static const SimplePipelineInfo infos[] = { + { "imx7-csi", "pxp" }, + { "sun6i-csi", nullptr }, }; - for (const char *driver : drivers) { - DeviceMatch dm(driver); + MediaDevice *converter; + + for (const SimplePipelineInfo &info : infos) { + DeviceMatch dm(info.driver); media_ = acquireMediaDevice(enumerator, dm); - if (media_) + if (media_) { + DeviceMatch converterMatch(info.converter); + converter = acquireMediaDevice(enumerator, converterMatch); break; + } } if (!media_) @@ -614,6 +699,19 @@ bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) video_->bufferReady.connect(this, &SimplePipelineHandler::bufferReady); + /* Open the converter, if any. */ + if (converter) { + converter_ = new SimpleConverter(converter); + if (converter_->open() < 0) { + LOG(SimplePipeline, Warning) + << "Failed to open converter, disabling format conversion"; + delete converter_; + converter_ = nullptr; + } + + converter_->bufferReady.connect(this, &SimplePipelineHandler::converterDone); + } + /* * Create one camera data instance for each sensor and gather all * entities in all pipelines. @@ -688,12 +786,70 @@ V4L2Subdevice *SimplePipelineHandler::subdev(const MediaEntity *entity) void SimplePipelineHandler::bufferReady(FrameBuffer *buffer) { - ASSERT(activeCamera_); + /* + * If an error occurred during capture, or if the buffer was cancelled, + * complete the request, even if the converter is in use as there's no + * point converting an erroneous buffer. + */ + if (buffer->metadata().status != FrameMetadata::FrameSuccess) { + if (useConverter_) { + /* Requeue the buffer for capture. */ + video_->queueBuffer(buffer); + + /* + * Get the next user-facing buffer to complete the + * request. + */ + if (converterQueue_.empty()) + return; + + buffer = converterQueue_.front(); + converterQueue_.pop(); + } + + Request *request = buffer->request(); + completeBuffer(activeCamera_, request, buffer); + completeRequest(activeCamera_, request); + return; + } + + /* + * Queue the captured and the request buffer to the converter if format + * conversion is needed. If there's no queued request, just requeue the + * captured buffer for capture. + */ + if (useConverter_) { + if (converterQueue_.empty()) { + video_->queueBuffer(buffer); + return; + } + + FrameBuffer *output = converterQueue_.front(); + converterQueue_.pop(); + + converter_->queueBuffers(buffer, output); + return; + } + + /* Otherwise simply complete the request. */ Request *request = buffer->request(); completeBuffer(activeCamera_, request, buffer); completeRequest(activeCamera_, request); } +void SimplePipelineHandler::converterDone(FrameBuffer *input, + FrameBuffer *output) +{ + /* Complete the request. */ + ASSERT(activeCamera_); + Request *request = output->request(); + completeBuffer(activeCamera_, request, output); + completeRequest(activeCamera_, request); + + /* Queue the input buffer back for capture. */ + video_->queueBuffer(input); +} + REGISTER_PIPELINE_HANDLER(SimplePipelineHandler); } /* namespace libcamera */