From patchwork Wed May 4 13:21:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 15774 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BA089C3256 for ; Wed, 4 May 2022 13:22:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 58A146564E; Wed, 4 May 2022 15:22:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1651670522; bh=4xMOvcIug09O0pUCF151klKaOcYTIo+3JA46GmbdTTg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=gXK4pgAmLhou3P5/WqMd0MK0+AZ3rhwV2vTWpiB3hVQbp3DCTCD6Jvrr0cN/DTUMG jDTUrM/DXZDz3DxT3SNJdA/jSWg8Gd2s+Ut2aGYYO4MG98Kf/J1BDnQMS1h5kvgsWy iN08Tfns1yP5jd6K09SE+6rXiX9u7HDtLTtDInfMV0AhcKgy5JgpMtBR1pRCoGJtXS QTQObeEGpv/LNsMrZgeFHFLruYTgT4vktL0x2G7j8T8U/5kOnBvO7EsK7dYa8KXHCb qyN7u/QkdBnxvy2tl8/krex2qTqDPhGiP7QCK+nQQL26iFyiTRxPhKwQQyKpQt/Hzu 48OBW4OQg4jPQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EF4AE65642 for ; Wed, 4 May 2022 15:21:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="R4fErgmw"; dkim-atps=neutral Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 99B0E835 for ; Wed, 4 May 2022 15:21:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1651670519; bh=4xMOvcIug09O0pUCF151klKaOcYTIo+3JA46GmbdTTg=; h=From:To:Subject:Date:In-Reply-To:References:From; b=R4fErgmwcZT6b6IZTLk010qePCASATh8T9XP26OUWXupy7djYTrR5DfNefGU3/yef EqS5ZlVLXxlQasnm7rTBvXTFXN2gDD/KXT5CnQWhZbLgQwKiyLxJXSrkkxAUbyo/JJ lNj+DeR/aNSfSD5rgiZWBG2CLNJndlYaSdWJs6Fo= To: libcamera-devel@lists.libcamera.org Date: Wed, 4 May 2022 16:21:50 +0300 Message-Id: <20220504132154.9681-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220504132154.9681-1-laurent.pinchart@ideasonboard.com> References: <20220504132154.9681-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/5] libcamera: pipeline: simple: Improve debug message on config failure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When pipeline configuration fails, print the format returned by the kernel in addition to the one requested by libcamera, to ease debugging. Signed-off-by: Laurent Pinchart --- src/libcamera/pipeline/simple/simple.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 76bd228b5aba..22efc8f498dc 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -939,7 +939,8 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) captureFormat.size != pipeConfig->captureSize) { LOG(SimplePipeline, Error) << "Unable to configure capture in " - << pipeConfig->captureSize << "-" << videoFormat; + << pipeConfig->captureSize << "-" << videoFormat + << " (got " << captureFormat << ")"; return -EINVAL; } From patchwork Wed May 4 13:21:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 15775 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A4EE0C326C for ; Wed, 4 May 2022 13:22:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DD82D6564C; Wed, 4 May 2022 15:22:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1651670522; bh=n9gj6BTLGz9tumldNX/3y542S1gieNsp30b0qPBm8P4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=o+hC1VlS8XAhU0LfC/EbugqU+k4DalpmS1ZAqWOlozYw7BjZgS0Zz4Szp8jqmCX7F s1AnJtLaHxd9+q+5BWqffxS66Deb+N9fDdGyWwYXys0sQ0x/4DqOlU94tzKxhJDddX 9Pn1o1SAtvX8uf7XaKaDlfVI/bHiBNDyGhjB6T2WGshFZj/jtDQdvbviXQFW1P8g6v 102qnUSn05qPQ9yTd6oLiRbCGmnGZcq8QqSO/iWgwXOnjBQvALjzatJ6v0Gpl2Yn31 kMeR9b2XHEZWMUFPaO/meWS+OCTcbPRXvhwy0qfNbknov4FJcWIi5kN0OUi5FPX1pL bFQ2+0cPTPdnw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 59FE865641 for ; Wed, 4 May 2022 15:22:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="NyQm/TS1"; dkim-atps=neutral Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 00EF04A8 for ; Wed, 4 May 2022 15:21:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1651670520; bh=n9gj6BTLGz9tumldNX/3y542S1gieNsp30b0qPBm8P4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=NyQm/TS1SNevCv1IwFc/xbN+yVJu24sZRF2L09OCV0u6z24MraZdr+ML+0I//WSF9 cuVTfm6YKamvXISntSQOnvMe1uGe+mybjXdkBMl2JZvBKou33fqd0d8MfuS29qm/Hi l4pYNKqyztNjzsEimYW3FqKTmuhwRYbZrC9/Oifo= To: libcamera-devel@lists.libcamera.org Date: Wed, 4 May 2022 16:21:51 +0300 Message-Id: <20220504132154.9681-3-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220504132154.9681-1-laurent.pinchart@ideasonboard.com> References: <20220504132154.9681-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/5] libcamera: pipeline: simple: Document the pipeline traversal algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a section to the documentation at the top of the file to describe in a bit more details how the media graph is traversed. Signed-off-by: Laurent Pinchart --- src/libcamera/pipeline/simple/simple.cpp | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 22efc8f498dc..4f963df16315 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -95,6 +95,21 @@ LOG_DEFINE_CATEGORY(SimplePipeline) * valid pipeline configurations are found, a Camera is registered for the * SimpleCameraData instance. * + * Pipeline Traversal + * ------------------ + * + * During the breadth-first search, the pipeline is traversed from entity to + * entity, by following media graph links from source to sink, starting at the + * camera sensor. When reaching an entity (on its sink side), all its source + * pads are considered to continue the graph traversal. + * + * The shortest path between the camera sensor and a video node is stored in + * SimpleCameraData::entities_ as a list of SimpleCameraData::Entity structures, + * ordered along the data path from the camera sensor to the video node. The + * Entity structure stores a pointer to the MediaEntity, as well as information + * about how it is connected in that particular path for later usage when + * configuring the pipeline. + * * Pipeline Configuration * ---------------------- * From patchwork Wed May 4 13:21:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 15776 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6643CC3256 for ; Wed, 4 May 2022 13:22:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D969A65656; Wed, 4 May 2022 15:22:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1651670523; bh=hE0Dr1CqY/aox8FLLWv/Lak0S2JJYop4QweVrZJxg1A=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=WCWiLxnouxs/lDFHQYvvQj6HtIFV+Ju2xf3EE/zWhBOLUx1VXDupVXR8F5/G+fSOx N01Y4NffMjDoKYeQ2aYyEe4A5u0sCTwIbAlGA2tk3W5Nlx7+JgcKZHoz12hOsS3qFm kINZ0LUxNNRvl75S1AfT6/EDoBTihN5QUDFJilcG1ekO5axT6gJ5hkjl8G/SL6jZJ6 zJWbxpWnEN+xcGXUu9+WgwiW/0R88aP+nW7RfAO5vRoTB/G55dIgq9Y3ahcSdMOISp /dLfO4uqtdHAW48lk4vsNFHzvYM0sF2MFqVprCXKYPhO2JLh1v56y254vvvRiIj3mF CrfHOo9BNbLNw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AEDFD65641 for ; Wed, 4 May 2022 15:22:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Ctr296pK"; dkim-atps=neutral Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 59AF4835 for ; Wed, 4 May 2022 15:22:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1651670520; bh=hE0Dr1CqY/aox8FLLWv/Lak0S2JJYop4QweVrZJxg1A=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Ctr296pKSEBjuTRe39rO0gzC8U7JmzTuu7LoSzcntofNzRYIeOO3hq0hqnPyQ70kS x4SVDE952NE0sBi3Kh/wOEHXhkJxuC6YdH5zszOwbQi3aTXIumUy/nd4lQxmRRKsu9 3cri87eRVFl5Y/RuYXPDmjYAGZJZMqbsB2w1zFCw= To: libcamera-devel@lists.libcamera.org Date: Wed, 4 May 2022 16:21:52 +0300 Message-Id: <20220504132154.9681-4-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220504132154.9681-1-laurent.pinchart@ideasonboard.com> References: <20220504132154.9681-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/5] libcamera: pipeline: simple: Factor out format test to separate function X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To prepare for the implementation of a more complex format discovery method, factor out code that tries a pipeline configuration to a separate function. No functional change intended. Signed-off-by: Laurent Pinchart --- src/libcamera/pipeline/simple/simple.cpp | 110 ++++++++++++----------- 1 file changed, 59 insertions(+), 51 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 4f963df16315..19deb9447c96 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -250,6 +250,9 @@ public: std::queue> converterQueue_; private: + void tryPipeline(unsigned int code, const Size &size); + static std::vector routedSourcePads(MediaPad *sink); + void converterInputDone(FrameBuffer *buffer); void converterOutputDone(FrameBuffer *buffer); }; @@ -466,58 +469,11 @@ int SimpleCameraData::init() return ret; /* - * Enumerate the possible pipeline configurations. For each media bus - * format supported by the sensor, propagate the formats through the - * pipeline, and enumerate the corresponding possible V4L2 pixel - * formats on the video node. + * Generate the list of possible pipeline configurations by trying each + * media bus format supported by the sensor. */ - for (unsigned int code : sensor_->mbusCodes()) { - V4L2SubdeviceFormat format{}; - format.mbus_code = code; - format.size = sensor_->resolution(); - - ret = setupFormats(&format, V4L2Subdevice::TryFormat); - if (ret < 0) { - LOG(SimplePipeline, Debug) - << "Media bus code " << utils::hex(code, 4) - << " not supported for this pipeline"; - /* Try next mbus_code supported by the sensor */ - continue; - } - - V4L2VideoDevice::Formats videoFormats = - video_->formats(format.mbus_code); - - LOG(SimplePipeline, Debug) - << "Adding configuration for " << format.size - << " in pixel formats [ " - << utils::join(videoFormats, ", ", - [](const auto &f) { - return f.first.toString(); - }) - << " ]"; - - for (const auto &videoFormat : videoFormats) { - PixelFormat pixelFormat = videoFormat.first.toPixelFormat(); - if (!pixelFormat) - continue; - - Configuration config; - config.code = code; - config.captureFormat = pixelFormat; - config.captureSize = format.size; - - if (!converter_) { - config.outputFormats = { pixelFormat }; - config.outputSizes = config.captureSize; - } else { - config.outputFormats = converter_->formats(pixelFormat); - config.outputSizes = converter_->sizes(format.size); - } - - configs_.push_back(config); - } - } + for (unsigned int code : sensor_->mbusCodes()) + tryPipeline(code, sensor_->resolution()); if (configs_.empty()) { LOG(SimplePipeline, Error) << "No valid configuration found"; @@ -541,6 +497,58 @@ int SimpleCameraData::init() return 0; } +void SimpleCameraData::tryPipeline(unsigned int code, const Size &size) +{ + /* + * Propagate the format through the pipeline, and enumerate the + * corresponding possible V4L2 pixel formats on the video node. + */ + V4L2SubdeviceFormat format{}; + format.mbus_code = code; + format.size = size; + + int ret = setupFormats(&format, V4L2Subdevice::TryFormat); + if (ret < 0) { + /* Pipeline configuration failed, skip this configuration. */ + LOG(SimplePipeline, Debug) + << "Media bus code " << utils::hex(code, 4) + << " not supported for this pipeline"; + return; + } + + V4L2VideoDevice::Formats videoFormats = video_->formats(format.mbus_code); + + LOG(SimplePipeline, Debug) + << "Adding configuration for " << format.size + << " in pixel formats [ " + << utils::join(videoFormats, ", ", + [](const auto &f) { + return f.first.toString(); + }) + << " ]"; + + for (const auto &videoFormat : videoFormats) { + PixelFormat pixelFormat = videoFormat.first.toPixelFormat(); + if (!pixelFormat) + continue; + + Configuration config; + config.code = code; + config.captureFormat = pixelFormat; + config.captureSize = format.size; + + if (!converter_) { + config.outputFormats = { pixelFormat }; + config.outputSizes = config.captureSize; + } else { + config.outputFormats = converter_->formats(pixelFormat); + config.outputSizes = converter_->sizes(format.size); + } + + configs_.push_back(config); + } +} + int SimpleCameraData::setupLinks() { int ret; From patchwork Wed May 4 13:21:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 15777 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 22615C326D for ; Wed, 4 May 2022 13:22:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 558DC6565A; Wed, 4 May 2022 15:22:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1651670526; bh=JN1RXQayuzopNsDmpw3NQuT6eRGlbIl310dd8nGKdIk=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=P1DN7eEu0DOl1vwYRWAtV/y2/v20CZ5TG2F5u7NUXjtUN4gTg+0vT+z4oET2OciPb mcc3LoXOVcnCXft9bqocEcR8Vgak5esxn0OD8KWIVnxRqfavQLlWly7zKV6Vkkxgcl QNj3lOHcr5M65iLQ/bW5YYi6SA4hsWntS0E8h8sNfwhzIEWEqup3Xxnr9Y900cnbhI YQIFawHwxJ8W0uewY2chjtGIjSjP3eCNR0+yzFI9AadF2OqBSiJhsdZYUDkYvWbzK4 YDoGxfGDaZwl17hcz2Xa0fSCmnvkYFnQ1Xm2QTOR7H5sMYPSh2bQLxuOV2PboJOilt Z5yUwcC7eqpFA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 21C0265642 for ; Wed, 4 May 2022 15:22:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gXzw5uTm"; dkim-atps=neutral Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B47344A8 for ; Wed, 4 May 2022 15:22:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1651670520; bh=JN1RXQayuzopNsDmpw3NQuT6eRGlbIl310dd8nGKdIk=; h=From:To:Subject:Date:In-Reply-To:References:From; b=gXzw5uTmzP1DnNlHc7J7DNQ15z1t1bsCaMP6bDoZ780oOH2gVVLh+NiGdoSKlFR32 CUdTiMot1g56sjqZ3mOXX21lOWN1WN2q7O8BsqIhKfNErPpEXtUOAUJn7AOk/BxLXo d/3yYB09CIcAKXe5SDv1Pwt0ApGtB9t8OR/j7zV4= To: libcamera-devel@lists.libcamera.org Date: Wed, 4 May 2022 16:21:53 +0300 Message-Id: <20220504132154.9681-5-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220504132154.9681-1-laurent.pinchart@ideasonboard.com> References: <20220504132154.9681-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 4/5] libcamera: pipeline: simple: Store sensor resolution in configuration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When enumerating the supported configurations, store the corresponding sensor resolution in the SimpleCameraData::Configuration structure and use it when configuring the camera, instead of hardcoding the sensor full resolution. This prepares for support of downscaling in the camera sensor. Signed-off-by: Laurent Pinchart --- src/libcamera/pipeline/simple/simple.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 19deb9447c96..e7fda239d6b5 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -225,6 +225,7 @@ public: struct Configuration { uint32_t code; + Size sensorSize; PixelFormat captureFormat; Size captureSize; std::vector outputFormats; @@ -534,6 +535,7 @@ void SimpleCameraData::tryPipeline(unsigned int code, const Size &size) Configuration config; config.code = code; + config.sensorSize = size; config.captureFormat = pixelFormat; config.captureSize = format.size; @@ -935,7 +937,7 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) const SimpleCameraData::Configuration *pipeConfig = config->pipeConfig(); V4L2SubdeviceFormat format{}; format.mbus_code = pipeConfig->code; - format.size = data->sensor_->resolution(); + format.size = pipeConfig->sensorSize; ret = data->setupFormats(&format, V4L2Subdevice::ActiveFormat); if (ret < 0) From patchwork Wed May 4 13:21:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 15778 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A1761C326E for ; Wed, 4 May 2022 13:22:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F109D6564F; Wed, 4 May 2022 15:22:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1651670527; bh=6S82CbC3QGg3iKprxwbo6/cXnfsOIuf+cMBGA3qp2R0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=RXWt17ZzBXBqlyit8/ipkHHGeDNfzwSFjb/WvYbdCTqp19v0qAPDf/SUKs4WjB61e 8+LngsUClxs8SUlL1UZEOtDA6rv8N7b3Zj+fzHpYEdQOZKJrRZzcEbdM8MQSRCR+/q kejK0hJrKQqdxNb2pYz4QcjqYGAdZ3zFCtP4i33CMQfWaESpVJswxBf18hJvyYEcVy rLypCvAIalx3HBvBIV2W+C6RCIfLoDHUmVmIAF01KS2kcrzjzfsd4y/cv+twLRARxK HAfLDCV7NPuCHAyjdg0FwhKvXXoJc1MTFnwBz7FVq3uCefEVPYPWlRBDBgF5IPExGS 48u9z4lUkipxg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A47CF6564A for ; Wed, 4 May 2022 15:22:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Ds7vp8Kj"; dkim-atps=neutral Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1AB37835 for ; Wed, 4 May 2022 15:22:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1651670521; bh=6S82CbC3QGg3iKprxwbo6/cXnfsOIuf+cMBGA3qp2R0=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Ds7vp8KjlTfwFbael8MzNIB8RvVpTMOayrlA4yaxnbrzCH+ObwL94eytel3v0jzil OuQcZ9VNiMv6tAC8xJgzgOHUYw0787n0z0YiIzNcztLgHo0JpHx1yadqedbCX3qCpK XdBtJD9btIM7UcXVtF/Ep+VIM/NSVOBVRlYeNoIg= To: libcamera-devel@lists.libcamera.org Date: Wed, 4 May 2022 16:21:54 +0300 Message-Id: <20220504132154.9681-6-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220504132154.9681-1-laurent.pinchart@ideasonboard.com> References: <20220504132154.9681-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 5/5] libcamera: pipeline: simple: Support scaling on the sensor X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As the simple pipeline handler targets simple pipelines on the SoC side, it often gets used with platforms that have a YUV sensor capable of outputting different sizes. Extend the heuristics used for pipeline discovery and configuration to scale as much as possible on the sensor side, in order to minimize the required bandwidth. Signed-off-by: Laurent Pinchart --- src/libcamera/pipeline/simple/simple.cpp | 142 +++++++++++++++++------ 1 file changed, 109 insertions(+), 33 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index e7fda239d6b5..5858deb01ee4 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -115,16 +115,25 @@ LOG_DEFINE_CATEGORY(SimplePipeline) * * The simple pipeline handler configures the pipeline by propagating V4L2 * subdev formats from the camera sensor to the video node. The format is first - * set on the camera sensor's output, using the native camera sensor - * resolution. Then, on every link in the pipeline, the format is retrieved on - * the link source and set unmodified on the link sink. + * set on the camera sensor's output, picking a resolution supported by the + * sensor that best matches the needs of the requested streams. Then, on every + * link in the pipeline, the format is retrieved on the link source and set + * unmodified on the link sink. * - * When initializating the camera data, this above procedure is repeated for - * every media bus format supported by the camera sensor. Upon reaching the - * video node, the pixel formats compatible with the media bus format are - * enumerated. Each of those pixel formats corresponds to one possible pipeline - * configuration, stored as an instance of SimpleCameraData::Configuration in - * the SimpleCameraData::formats_ map. + * The best sensor resolution is selected using a heuristic that tries to + * minimize the required bus and memory bandwidth, as the simple pipeline + * handler is typically used on smaller, less powerful systems. To avoid the + * need to upscale, the pipeline handler picks the smallest sensor resolution + * large enough to accommodate the need of all streams. Resolutions that + * significantly restrict the field of view are ignored. + * + * When initializating the camera data, the above format propagation procedure + * is repeated for every media bus format and size supported by the camera + * sensor. Upon reaching the video node, the pixel formats compatible with the + * media bus format are enumerated. Each combination of the input media bus + * format, output pixel format and output size are recorded in an instance of + * the SimpleCameraData::Configuration structure, stored in the + * SimpleCameraData::configs_ vector. * * Format Conversion and Scaling * ----------------------------- @@ -243,7 +252,7 @@ public: V4L2VideoDevice *video_; std::vector configs_; - std::map formats_; + std::map> formats_; std::unique_ptr converter_; std::vector> converterBuffers_; @@ -471,26 +480,24 @@ int SimpleCameraData::init() /* * Generate the list of possible pipeline configurations by trying each - * media bus format supported by the sensor. + * media bus format and size supported by the sensor. */ - for (unsigned int code : sensor_->mbusCodes()) - tryPipeline(code, sensor_->resolution()); + for (unsigned int code : sensor_->mbusCodes()) { + for (const Size &size : sensor_->sizes(code)) + tryPipeline(code, size); + } if (configs_.empty()) { LOG(SimplePipeline, Error) << "No valid configuration found"; return -EINVAL; } - /* - * Map the pixel formats to configurations. Any previously stored value - * is overwritten, as the pipeline handler currently doesn't care about - * how a particular PixelFormat is achieved. - */ + /* Map the pixel formats to configurations. */ for (const Configuration &config : configs_) { - formats_[config.captureFormat] = &config; + formats_[config.captureFormat].push_back(&config); for (PixelFormat fmt : config.outputFormats) - formats_[fmt] = &config; + formats_[fmt].push_back(&config); } properties_ = sensor_->properties(); @@ -511,8 +518,10 @@ void SimpleCameraData::tryPipeline(unsigned int code, const Size &size) int ret = setupFormats(&format, V4L2Subdevice::TryFormat); if (ret < 0) { /* Pipeline configuration failed, skip this configuration. */ + format.mbus_code = code; + format.size = size; LOG(SimplePipeline, Debug) - << "Media bus code " << utils::hex(code, 4) + << "Sensor format " << format << " not supported for this pipeline"; return; } @@ -783,22 +792,70 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() status = Adjusted; } + /* Find the largest stream size. */ + Size maxStreamSize; + for (const StreamConfiguration &cfg : config_) + maxStreamSize.expandTo(cfg.size); + + LOG(SimplePipeline, Debug) + << "Largest stream size is " << maxStreamSize; + /* - * Pick a configuration for the pipeline based on the pixel format for - * the streams (ordered from highest to lowest priority). Default to - * the first pipeline configuration if no streams requests a supported + * Find the best configuration for the pipeline using a heuristic. + * First select the pixel format based on the streams (which are + * considered ordered from highest to lowest priority). Default to the + * first pipeline configuration if no streams requests a supported * pixel format. */ - pipeConfig_ = data_->formats_.begin()->second; + const std::vector *configs = + &data_->formats_.begin()->second; for (const StreamConfiguration &cfg : config_) { auto it = data_->formats_.find(cfg.pixelFormat); if (it != data_->formats_.end()) { - pipeConfig_ = it->second; + configs = &it->second; break; } } + /* + * \todo Pick the best sensor output media bus format when the + * requested pixel format can be produced from multiple sensor media + * bus formats. + */ + + /* + * Then pick, among the possible configuration for the pixel format, + * the smallest sensor resolution that can accommodate all streams + * without upscaling. + */ + const SimpleCameraData::Configuration *maxPipeConfig = nullptr; + pipeConfig_ = nullptr; + + for (const SimpleCameraData::Configuration *pipeConfig : *configs) { + const Size &size = pipeConfig->captureSize; + + if (size.width >= maxStreamSize.width && + size.height >= maxStreamSize.height) { + if (!pipeConfig_ || size < pipeConfig_->captureSize) + pipeConfig_ = pipeConfig; + } + + if (!maxPipeConfig || maxPipeConfig->captureSize < size) + maxPipeConfig = pipeConfig; + } + + /* If no configuration was large enough, select the largest one. */ + if (!pipeConfig_) + pipeConfig_ = maxPipeConfig; + + LOG(SimplePipeline, Debug) + << "Picked " + << V4L2SubdeviceFormat{ pipeConfig_->code, pipeConfig_->sensorSize, {} } + << " -> " << pipeConfig_->captureSize + << "-" << pipeConfig_->captureFormat + << " for max stream size " << maxStreamSize; + /* * Adjust the requested streams. * @@ -891,13 +948,32 @@ CameraConfiguration *SimplePipelineHandler::generateConfiguration(Camera *camera /* Create the formats map. */ std::map> formats; - std::transform(data->formats_.begin(), data->formats_.end(), - std::inserter(formats, formats.end()), - [](const auto &format) -> decltype(formats)::value_type { - const PixelFormat &pixelFormat = format.first; - const Size &size = format.second->captureSize; - return { pixelFormat, { size } }; - }); + + for (const SimpleCameraData::Configuration &cfg : data->configs_) { + for (PixelFormat format : cfg.outputFormats) + formats[format].push_back(cfg.outputSizes); + } + + /* Sort the sizes and merge any consecutive overlapping ranges. */ + for (auto &[format, sizes] : formats) { + std::sort(sizes.begin(), sizes.end(), + [](SizeRange &a, SizeRange &b) { + return a.min < b.min; + }); + + auto cur = sizes.begin(); + auto next = cur; + + while (++next != sizes.end()) { + if (cur->max.width >= next->min.width && + cur->max.height >= next->min.height) + cur->max = next->max; + else if (++cur != next) + *cur = *next; + } + + sizes.erase(++cur, sizes.end()); + } /* * Create the stream configurations. Take the first entry in the formats