From patchwork Sun Jun 12 15:23:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 16198 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7846CC3273 for ; Sun, 12 Jun 2022 15:23:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 944FC6563B; Sun, 12 Jun 2022 17:23:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1655047405; bh=VyiVX0meUYlRNfln26J7Lo+8p7G4tVTwjxezjqGQMdc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=kd/8CQc67/Q6CiAcddELdhoyHdzg3GAVKhisQ5k+qHf9foTFUANdCPY09YF0FSbWn 9i8FP3nsB+y50KEpHbNWG4hpdQajOvqi3q9SFCP0AMYKYglrlThZDeFyiti8JwW4K3 BlrYat60C62u/nxPkSCbeLyuJyuBo526CWffE19yXzZ+zmMFm/w+RniwSrMu9OH1/W 7nL9SYa7RCa3iqM25T+bDSQ1yjPdTU+loEF8syaEt+WmGvCGXtkPVl1MkukW/hyEYU gLE4W837Yb8/rGi+hJiYxUSyLuFNYTvBWqMANxMQgjln6H6mt0Tt/Hef9VDZFYK5n5 hiWAoOnQE3RPw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5412D600F4 for ; Sun, 12 Jun 2022 17:23:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="stO5Ekg1"; dkim-atps=neutral Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B9083825; Sun, 12 Jun 2022 17:23:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1655047404; bh=VyiVX0meUYlRNfln26J7Lo+8p7G4tVTwjxezjqGQMdc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=stO5Ekg1OIP/nxqJC3MkLWywuCWJJsj1TYfvOWXckk0oDHecf1HMF/ZaLCvGbDhM1 YMigVGn7t4IQNDgMwQMxvL91Bwb4WrOKnzP2jfF9aOCo4N42iXerJRFqWtYhb0nJXa LBy0v5cycDFQuRL4PHH07kqSZzlr69ZMimBTjiAk= To: libcamera-devel@lists.libcamera.org Date: Sun, 12 Jun 2022 18:23:07 +0300 Message-Id: <20220612152311.8408-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220612152311.8408-1-laurent.pinchart@ideasonboard.com> References: <20220612152311.8408-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/5] libcamera: pipeline: simple: Improve debug message on config failure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Cc: Dorota Czaplejewicz , Benjamin Schaaf Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When pipeline configuration fails, print the format returned by the kernel in addition to the one requested by libcamera, to ease debugging. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/simple/simple.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index de75465eb3b5..efb28ccd97e8 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -939,7 +939,8 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) captureFormat.size != pipeConfig->captureSize) { LOG(SimplePipeline, Error) << "Unable to configure capture in " - << pipeConfig->captureSize << "-" << videoFormat; + << pipeConfig->captureSize << "-" << videoFormat + << " (got " << captureFormat << ")"; return -EINVAL; } From patchwork Sun Jun 12 15:23:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 16199 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D578BC3274 for ; Sun, 12 Jun 2022 15:23:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7CFBA6563F; Sun, 12 Jun 2022 17:23:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1655047407; bh=tyHDn1gsDhR7Y+kOIC/MExFl5d/mTR0Q63bbCT4diGA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=BGWjxkZygc30fBsnmBMjOO+SG5obVP3JyPUCUc2WKL00F/bJf8HODr6pYyyYfnqZl SZGNOOC7JzQUcAB9CdmKBsCuy4Jivq9hpMICxJBpQDY5dQx8AXDn1Yy2MgQrLS2MTx ouniYP8jSkofMuknxgoZocF4CKsM5rlIf8SypKMBJPU+ckseGl5WsALs8LhK8oUg2P +PuGB+Ubjwe+hAPSqAminJsBzS1sQjOCyVPyjZIj2l4e1eovSsLmi2aP2/gXWN3Z/q f8mo/W2a+Wvz32FYNbhWo+Ia7e1Wb+b4TAqhE1Fov8wxUWb7eGZK/jyZVcA/dK2RQZ v95I1/31VvX8g== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 03E8B65633 for ; Sun, 12 Jun 2022 17:23:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PzY5EIt1"; dkim-atps=neutral Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 72968D24; Sun, 12 Jun 2022 17:23:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1655047404; bh=tyHDn1gsDhR7Y+kOIC/MExFl5d/mTR0Q63bbCT4diGA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PzY5EIt1+A6edanws6K9lWRnhkSSKBJVOOZZ+A9D7GcbDTKy/ekJuI2NzaCv0yTqB BaB843LVAXS/Q3hetrcexiCOka8B/8bGMcSY2v+WPxhOAzkltqQIqvt4YZ6bPT4oQx NjBpgmO1ha9UqYQweuRlnXPDMoIN5e6q4DfE9sjs= To: libcamera-devel@lists.libcamera.org Date: Sun, 12 Jun 2022 18:23:08 +0300 Message-Id: <20220612152311.8408-3-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220612152311.8408-1-laurent.pinchart@ideasonboard.com> References: <20220612152311.8408-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/5] libcamera: pipeline: simple: Document the pipeline traversal algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Cc: Dorota Czaplejewicz , Benjamin Schaaf Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a section to the documentation at the top of the file to describe in a bit more details how the media graph is traversed. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/simple/simple.cpp | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index efb28ccd97e8..b09368aee20b 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -95,6 +95,21 @@ LOG_DEFINE_CATEGORY(SimplePipeline) * valid pipeline configurations are found, a Camera is registered for the * SimpleCameraData instance. * + * Pipeline Traversal + * ------------------ + * + * During the breadth-first search, the pipeline is traversed from entity to + * entity, by following media graph links from source to sink, starting at the + * camera sensor. When reaching an entity (on its sink side), all its source + * pads are considered to continue the graph traversal. + * + * The shortest path between the camera sensor and a video node is stored in + * SimpleCameraData::entities_ as a list of SimpleCameraData::Entity structures, + * ordered along the data path from the camera sensor to the video node. The + * Entity structure stores a pointer to the MediaEntity, as well as information + * about how it is connected in that particular path for later usage when + * configuring the pipeline. + * * Pipeline Configuration * ---------------------- * From patchwork Sun Jun 12 15:23:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 16200 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 52631C3275 for ; Sun, 12 Jun 2022 15:23:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 05CB265636; Sun, 12 Jun 2022 17:23:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1655047408; bh=NEvw7Yb0BDqt535enhFpoKtHVCWV4h2LVLSYk7WADs0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=nclsfMlYSydR28u72ANXXh33tQpHJv5EuDBrtneF/x6Eo7p3PGOZZ5T87AF7CtGsa YW4BjJd2T8Ic5R3O5ysAYdpTTLd/Um610VomdAVYMkF3xlnnVkZ88acZw3lJNx4lQp X41zgGA3m724aAS36tGk6acGn3Q2YUhQUL7soMHKbxa95vR9Tn7UrMIORQAemTvJMt EeKD7gmK6J7BhY1FevKCtbfxsZ0RPZ14CC8piCG3nQ2YCqSuftS7pTHXSw60TleRCn L6K1xWxQq5jgTyMCn2fBRmfiBZ7IVfi6cmcLXSnlWfePMznAVAtBkI7LGMPBVwZqkc gFJUI2U/8Q9bg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 06E89600F4 for ; Sun, 12 Jun 2022 17:23:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="SeTrA16P"; dkim-atps=neutral Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2FC05DDE; Sun, 12 Jun 2022 17:23:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1655047405; bh=NEvw7Yb0BDqt535enhFpoKtHVCWV4h2LVLSYk7WADs0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SeTrA16PJqt9YR+DyiA2qHtwHsJvgl3lvSS020ayKUGjya1pmKDJ/tJj2ZHyE3lZP CVLpugj9GTyzZT2cgQhWQa4qLSsSbVeFvP0rNweoOWzbaph/lGqnOsMLlPmal3UnZJ wsQgznzJi2ohAdIQEoDHnqgML9u6TZ4kvSVRTlSM= To: libcamera-devel@lists.libcamera.org Date: Sun, 12 Jun 2022 18:23:09 +0300 Message-Id: <20220612152311.8408-4-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220612152311.8408-1-laurent.pinchart@ideasonboard.com> References: <20220612152311.8408-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/5] libcamera: pipeline: simple: Factor out format test to separate function X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Cc: Dorota Czaplejewicz , Benjamin Schaaf Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To prepare for the implementation of a more complex format discovery method, factor out code that tries a pipeline configuration to a separate function. No functional change intended. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- Changes since v1: - Drop unused function declaration - Document the tryPipeline() function --- src/libcamera/pipeline/simple/simple.cpp | 118 +++++++++++++---------- 1 file changed, 67 insertions(+), 51 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index b09368aee20b..3c90bdec60e0 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -250,6 +250,8 @@ public: std::queue> converterQueue_; private: + void tryPipeline(unsigned int code, const Size &size); + void converterInputDone(FrameBuffer *buffer); void converterOutputDone(FrameBuffer *buffer); }; @@ -466,58 +468,11 @@ int SimpleCameraData::init() return ret; /* - * Enumerate the possible pipeline configurations. For each media bus - * format supported by the sensor, propagate the formats through the - * pipeline, and enumerate the corresponding possible V4L2 pixel - * formats on the video node. + * Generate the list of possible pipeline configurations by trying each + * media bus format supported by the sensor. */ - for (unsigned int code : sensor_->mbusCodes()) { - V4L2SubdeviceFormat format{}; - format.mbus_code = code; - format.size = sensor_->resolution(); - - ret = setupFormats(&format, V4L2Subdevice::TryFormat); - if (ret < 0) { - LOG(SimplePipeline, Debug) - << "Media bus code " << utils::hex(code, 4) - << " not supported for this pipeline"; - /* Try next mbus_code supported by the sensor */ - continue; - } - - V4L2VideoDevice::Formats videoFormats = - video_->formats(format.mbus_code); - - LOG(SimplePipeline, Debug) - << "Adding configuration for " << format.size - << " in pixel formats [ " - << utils::join(videoFormats, ", ", - [](const auto &f) { - return f.first.toString(); - }) - << " ]"; - - for (const auto &videoFormat : videoFormats) { - PixelFormat pixelFormat = videoFormat.first.toPixelFormat(); - if (!pixelFormat) - continue; - - Configuration config; - config.code = code; - config.captureFormat = pixelFormat; - config.captureSize = format.size; - - if (!converter_) { - config.outputFormats = { pixelFormat }; - config.outputSizes = config.captureSize; - } else { - config.outputFormats = converter_->formats(pixelFormat); - config.outputSizes = converter_->sizes(format.size); - } - - configs_.push_back(config); - } - } + for (unsigned int code : sensor_->mbusCodes()) + tryPipeline(code, sensor_->resolution()); if (configs_.empty()) { LOG(SimplePipeline, Error) << "No valid configuration found"; @@ -541,6 +496,67 @@ int SimpleCameraData::init() return 0; } +/* + * Generate a list of supported pipeline configurations for a sensor media bus + * code and size. + * + * First propagate the media bus code and size through the pipeline from the + * camera sensor to the video node. Then, query the video node for all supported + * pixel formats compatible with the media bus code. For each pixel format, store + * a full pipeline configuration in the configs_ vector. + */ +void SimpleCameraData::tryPipeline(unsigned int code, const Size &size) +{ + /* + * Propagate the format through the pipeline, and enumerate the + * corresponding possible V4L2 pixel formats on the video node. + */ + V4L2SubdeviceFormat format{}; + format.mbus_code = code; + format.size = size; + + int ret = setupFormats(&format, V4L2Subdevice::TryFormat); + if (ret < 0) { + /* Pipeline configuration failed, skip this configuration. */ + LOG(SimplePipeline, Debug) + << "Media bus code " << utils::hex(code, 4) + << " not supported for this pipeline"; + return; + } + + V4L2VideoDevice::Formats videoFormats = video_->formats(format.mbus_code); + + LOG(SimplePipeline, Debug) + << "Adding configuration for " << format.size + << " in pixel formats [ " + << utils::join(videoFormats, ", ", + [](const auto &f) { + return f.first.toString(); + }) + << " ]"; + + for (const auto &videoFormat : videoFormats) { + PixelFormat pixelFormat = videoFormat.first.toPixelFormat(); + if (!pixelFormat) + continue; + + Configuration config; + config.code = code; + config.captureFormat = pixelFormat; + config.captureSize = format.size; + + if (!converter_) { + config.outputFormats = { pixelFormat }; + config.outputSizes = config.captureSize; + } else { + config.outputFormats = converter_->formats(pixelFormat); + config.outputSizes = converter_->sizes(format.size); + } + + configs_.push_back(config); + } +} + int SimpleCameraData::setupLinks() { int ret; From patchwork Sun Jun 12 15:23:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 16202 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 81A27C3277 for ; Sun, 12 Jun 2022 15:23:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7EBD065646; Sun, 12 Jun 2022 17:23:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1655047409; bh=r+X7VYZUsM/hMS4pjICoikt95yUzv6PyR6QOHrIcKIg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=XNCzIkgGcqCQNVPLMqXTtjBZK8mwVZRG6zZ97Fb/xK6c2i9g/1w5TPXRXGDEsi7o4 8sdTp7Ww0otOz0yd14R7HCIFhlnfOHjM+8JozmMgO7Fa5ORZuzvgV+4/pCEyFsfYBZ GxVf4TlOFLQrtGJGKUARnvpC/o2mgEVVQ3u36bgZwGQfeNl1xtK26yQQAFeD03eX1i Fl5WDayblMk4N2NEaxDzIgTcx/dFUSH9GMwjNxWfeNYsc5xJQIT/B+eGXLcUa39pZ4 a7O9GFW8yTnd3h9AiJAlvvWYBDXC0enl5NdJ9ueUWSA2mLk2BuuuEI0qgIj1mTFh/8 MOper4bAtin2w== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3936165633 for ; Sun, 12 Jun 2022 17:23:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PF5aME+0"; dkim-atps=neutral Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B615EDFC; Sun, 12 Jun 2022 17:23:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1655047406; bh=r+X7VYZUsM/hMS4pjICoikt95yUzv6PyR6QOHrIcKIg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PF5aME+00CGVyfyvLuY2vKvwBnmUrPYdtEUKAyinN8Xhw3F5ipmoNe670LA/XnDQs NYiAGovZkLXQDFV2xgepCNFViEHRQaIvTEbB104wyGg1xgWTN475QUsCJqQgafKDmp H3BiEogjOco2W0QH7dWtLjYojMti0qFrFGeivLos= To: libcamera-devel@lists.libcamera.org Date: Sun, 12 Jun 2022 18:23:10 +0300 Message-Id: <20220612152311.8408-5-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220612152311.8408-1-laurent.pinchart@ideasonboard.com> References: <20220612152311.8408-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 4/5] libcamera: pipeline: simple: Store sensor resolution in configuration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Cc: Dorota Czaplejewicz , Benjamin Schaaf Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When enumerating the supported configurations, store the corresponding sensor resolution in the SimpleCameraData::Configuration structure and use it when configuring the camera, instead of hardcoding the sensor full resolution. This prepares for support of downscaling in the camera sensor. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/simple/simple.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 3c90bdec60e0..8c48162d7ff0 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -225,6 +225,7 @@ public: struct Configuration { uint32_t code; + Size sensorSize; PixelFormat captureFormat; Size captureSize; std::vector outputFormats; @@ -542,6 +543,7 @@ void SimpleCameraData::tryPipeline(unsigned int code, const Size &size) Configuration config; config.code = code; + config.sensorSize = size; config.captureFormat = pixelFormat; config.captureSize = format.size; @@ -943,7 +945,7 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) const SimpleCameraData::Configuration *pipeConfig = config->pipeConfig(); V4L2SubdeviceFormat format{}; format.mbus_code = pipeConfig->code; - format.size = data->sensor_->resolution(); + format.size = pipeConfig->sensorSize; ret = data->setupFormats(&format, V4L2Subdevice::ActiveFormat); if (ret < 0) From patchwork Sun Jun 12 15:23:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 16201 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 78542C3276 for ; Sun, 12 Jun 2022 15:23:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 07EF265637; Sun, 12 Jun 2022 17:23:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1655047410; bh=w1dfCk6kzT2fHMHuuu7a0lEJZVJZ2WIuCdlhgqZoImY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=wZmAlvN1hAWf0OwIQEfowcp9ZWCxQkBxeV3sbVYyD9mWvdSbqctTOGH8W2BfNGXm0 kFwsnvXIzYy9SPV1RqL8/N0wCW27fRVlot6TIoEZiMMkH+b0CCTBqB07RmYjPujpEL tn2/nrZwiP7ywK/m8Y2OOVj5j0cjsLmPkxSmefjDZr/Hgmwa3Bmfbe249iKkcGSrJc A7TLppui3rGJV/1LJXJQBAPhayI7JCpmTU+t0aNIIbuXc/aUu8vc1DHIdlkrJWSta4 6dmubGEvHihM/v0+IJafcfpwVTfxETWC5d+cxMVXq9ccv77z9MFGyxmF6wao3+ibAJ FsmMUI+PaAAUA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E646765637 for ; Sun, 12 Jun 2022 17:23:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fo7xID/Z"; dkim-atps=neutral Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6CF3D825; Sun, 12 Jun 2022 17:23:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1655047406; bh=w1dfCk6kzT2fHMHuuu7a0lEJZVJZ2WIuCdlhgqZoImY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fo7xID/Z1w8VEiRerlXS0zduyLh+rLpraON/sj4llMonTZDjCbz3zGMTfzp9Sc4Wz k4uXa03HXVA9qQmGucird5YCUD70zz/NiRqtVG0LUR/39dWMshbVKCNnsDmZrRfZGI rrUw/Tc9AdOqRnX0ys2/IO1vL6ojBaLWAFc2UlSs= To: libcamera-devel@lists.libcamera.org Date: Sun, 12 Jun 2022 18:23:11 +0300 Message-Id: <20220612152311.8408-6-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220612152311.8408-1-laurent.pinchart@ideasonboard.com> References: <20220612152311.8408-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 5/5] libcamera: pipeline: simple: Support scaling on the sensor X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Cc: Dorota Czaplejewicz , Benjamin Schaaf Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As the simple pipeline handler targets simple pipelines on the SoC side, it often gets used with platforms that have a YUV sensor capable of outputting different sizes. Extend the heuristics used for pipeline discovery and configuration to scale as much as possible on the sensor side, in order to minimize the required bandwidth. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/simple/simple.cpp | 142 +++++++++++++++++------ 1 file changed, 109 insertions(+), 33 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 8c48162d7ff0..3a9f4d45e830 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -115,16 +115,25 @@ LOG_DEFINE_CATEGORY(SimplePipeline) * * The simple pipeline handler configures the pipeline by propagating V4L2 * subdev formats from the camera sensor to the video node. The format is first - * set on the camera sensor's output, using the native camera sensor - * resolution. Then, on every link in the pipeline, the format is retrieved on - * the link source and set unmodified on the link sink. + * set on the camera sensor's output, picking a resolution supported by the + * sensor that best matches the needs of the requested streams. Then, on every + * link in the pipeline, the format is retrieved on the link source and set + * unmodified on the link sink. * - * When initializating the camera data, this above procedure is repeated for - * every media bus format supported by the camera sensor. Upon reaching the - * video node, the pixel formats compatible with the media bus format are - * enumerated. Each of those pixel formats corresponds to one possible pipeline - * configuration, stored as an instance of SimpleCameraData::Configuration in - * the SimpleCameraData::formats_ map. + * The best sensor resolution is selected using a heuristic that tries to + * minimize the required bus and memory bandwidth, as the simple pipeline + * handler is typically used on smaller, less powerful systems. To avoid the + * need to upscale, the pipeline handler picks the smallest sensor resolution + * large enough to accommodate the need of all streams. Resolutions that + * significantly restrict the field of view are ignored. + * + * When initializating the camera data, the above format propagation procedure + * is repeated for every media bus format and size supported by the camera + * sensor. Upon reaching the video node, the pixel formats compatible with the + * media bus format are enumerated. Each combination of the input media bus + * format, output pixel format and output size are recorded in an instance of + * the SimpleCameraData::Configuration structure, stored in the + * SimpleCameraData::configs_ vector. * * Format Conversion and Scaling * ----------------------------- @@ -243,7 +252,7 @@ public: V4L2VideoDevice *video_; std::vector configs_; - std::map formats_; + std::map> formats_; std::unique_ptr converter_; std::vector> converterBuffers_; @@ -470,26 +479,24 @@ int SimpleCameraData::init() /* * Generate the list of possible pipeline configurations by trying each - * media bus format supported by the sensor. + * media bus format and size supported by the sensor. */ - for (unsigned int code : sensor_->mbusCodes()) - tryPipeline(code, sensor_->resolution()); + for (unsigned int code : sensor_->mbusCodes()) { + for (const Size &size : sensor_->sizes(code)) + tryPipeline(code, size); + } if (configs_.empty()) { LOG(SimplePipeline, Error) << "No valid configuration found"; return -EINVAL; } - /* - * Map the pixel formats to configurations. Any previously stored value - * is overwritten, as the pipeline handler currently doesn't care about - * how a particular PixelFormat is achieved. - */ + /* Map the pixel formats to configurations. */ for (const Configuration &config : configs_) { - formats_[config.captureFormat] = &config; + formats_[config.captureFormat].push_back(&config); for (PixelFormat fmt : config.outputFormats) - formats_[fmt] = &config; + formats_[fmt].push_back(&config); } properties_ = sensor_->properties(); @@ -519,8 +526,10 @@ void SimpleCameraData::tryPipeline(unsigned int code, const Size &size) int ret = setupFormats(&format, V4L2Subdevice::TryFormat); if (ret < 0) { /* Pipeline configuration failed, skip this configuration. */ + format.mbus_code = code; + format.size = size; LOG(SimplePipeline, Debug) - << "Media bus code " << utils::hex(code, 4) + << "Sensor format " << format << " not supported for this pipeline"; return; } @@ -791,22 +800,70 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() status = Adjusted; } + /* Find the largest stream size. */ + Size maxStreamSize; + for (const StreamConfiguration &cfg : config_) + maxStreamSize.expandTo(cfg.size); + + LOG(SimplePipeline, Debug) + << "Largest stream size is " << maxStreamSize; + /* - * Pick a configuration for the pipeline based on the pixel format for - * the streams (ordered from highest to lowest priority). Default to - * the first pipeline configuration if no streams requests a supported + * Find the best configuration for the pipeline using a heuristic. + * First select the pixel format based on the streams (which are + * considered ordered from highest to lowest priority). Default to the + * first pipeline configuration if no streams requests a supported * pixel format. */ - pipeConfig_ = data_->formats_.begin()->second; + const std::vector *configs = + &data_->formats_.begin()->second; for (const StreamConfiguration &cfg : config_) { auto it = data_->formats_.find(cfg.pixelFormat); if (it != data_->formats_.end()) { - pipeConfig_ = it->second; + configs = &it->second; break; } } + /* + * \todo Pick the best sensor output media bus format when the + * requested pixel format can be produced from multiple sensor media + * bus formats. + */ + + /* + * Then pick, among the possible configuration for the pixel format, + * the smallest sensor resolution that can accommodate all streams + * without upscaling. + */ + const SimpleCameraData::Configuration *maxPipeConfig = nullptr; + pipeConfig_ = nullptr; + + for (const SimpleCameraData::Configuration *pipeConfig : *configs) { + const Size &size = pipeConfig->captureSize; + + if (size.width >= maxStreamSize.width && + size.height >= maxStreamSize.height) { + if (!pipeConfig_ || size < pipeConfig_->captureSize) + pipeConfig_ = pipeConfig; + } + + if (!maxPipeConfig || maxPipeConfig->captureSize < size) + maxPipeConfig = pipeConfig; + } + + /* If no configuration was large enough, select the largest one. */ + if (!pipeConfig_) + pipeConfig_ = maxPipeConfig; + + LOG(SimplePipeline, Debug) + << "Picked " + << V4L2SubdeviceFormat{ pipeConfig_->code, pipeConfig_->sensorSize, {} } + << " -> " << pipeConfig_->captureSize + << "-" << pipeConfig_->captureFormat + << " for max stream size " << maxStreamSize; + /* * Adjust the requested streams. * @@ -899,13 +956,32 @@ CameraConfiguration *SimplePipelineHandler::generateConfiguration(Camera *camera /* Create the formats map. */ std::map> formats; - std::transform(data->formats_.begin(), data->formats_.end(), - std::inserter(formats, formats.end()), - [](const auto &format) -> decltype(formats)::value_type { - const PixelFormat &pixelFormat = format.first; - const Size &size = format.second->captureSize; - return { pixelFormat, { size } }; - }); + + for (const SimpleCameraData::Configuration &cfg : data->configs_) { + for (PixelFormat format : cfg.outputFormats) + formats[format].push_back(cfg.outputSizes); + } + + /* Sort the sizes and merge any consecutive overlapping ranges. */ + for (auto &[format, sizes] : formats) { + std::sort(sizes.begin(), sizes.end(), + [](SizeRange &a, SizeRange &b) { + return a.min < b.min; + }); + + auto cur = sizes.begin(); + auto next = cur; + + while (++next != sizes.end()) { + if (cur->max.width >= next->min.width && + cur->max.height >= next->min.height) + cur->max = next->max; + else if (++cur != next) + *cur = *next; + } + + sizes.erase(++cur, sizes.end()); + } /* * Create the stream configurations. Take the first entry in the formats