From patchwork Tue Mar 9 11:29:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marian Cichy X-Patchwork-Id: 11538 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 700CFBD80E for ; Tue, 9 Mar 2021 11:30:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1BE4368AA8; Tue, 9 Mar 2021 12:30:34 +0100 (CET) Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 97D2668A99 for ; Tue, 9 Mar 2021 12:30:31 +0100 (CET) Received: from dude02.hi.pengutronix.de ([2001:67c:670:100:1d::28]) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lJaZK-000840-SE; Tue, 09 Mar 2021 12:30:30 +0100 Received: from mci by dude02.hi.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1lJaZK-0007Sm-Jh; Tue, 09 Mar 2021 12:30:30 +0100 From: Marian Cichy To: libcamera-devel@lists.libcamera.org Date: Tue, 9 Mar 2021 12:29:47 +0100 Message-Id: <20210309112949.26251-1-m.cichy@pengutronix.de> X-Mailer: git-send-email 2.29.2 MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2001:67c:670:100:1d::28 X-SA-Exim-Mail-From: mci@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: libcamera-devel@lists.libcamera.org Subject: [libcamera-devel] [PATCH 0/2] pipeline: simple: Improve media-pipeline setup X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: graphics@pengutronix.de, Marian Cichy Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" These patches aim to improve the algorithm how SimplePipleine achieves the setup of the Media-Pipeline. Prior to these patches, the SimplePipeline starts at the sensor-device, then always chooses the first Source-Pad it finds. This is obviously just a mere guess how to find the capture device and may fail on some machines. With these patches, SimplePipeline will instead choose the shortest path from the sensor to a capture device. This makes sense as the shortest path is probably a direct path to a capture device, ignoring any IPU-components like encoders or translators. These patches were tested only on the imx-media driver, which is not yet supported by SimplePipeline. The imx-media driver needs some changes in order to work with libcamera. If testing with the imx-media driver is desired, one have to change the driver being media-centric first. For testing purposes, this can be done quick and dirty by applying the following hunk to the kernel: Regards, Marian Cichy Marian Cichy (2): media_entity: overload == operator to use media-entities in maps pipeline: simple: use uniform-cost search algorithm to setup media pipeline include/libcamera/internal/media_object.h | 2 + src/libcamera/pipeline/simple/simple.cpp | 118 +++++++++++----------- 2 files changed, 60 insertions(+), 60 deletions(-) --- a/drivers/staging/media/imx/imx-media-capture.c +++ b/drivers/staging/media/imx/imx-media-capture.c @@ -195,7 +195,7 @@ static int capture_enum_fmt_vid_cap(struct file *file, void *fh, if (ret) return ret; } else { - cc_src = imx_media_find_mbus_format(fmt_src.format.code, + cc_src = imx_media_find_mbus_format(f->mbus_code, PIXFMT_SEL_ANY); if (WARN_ON(!cc_src)) return -EINVAL; @@ -740,7 +740,7 @@ static struct video_device capture_videodev = { .release = video_device_release, .vfl_dir = VFL_DIR_RX, .tvnorms = V4L2_STD_NTSC | V4L2_STD_PAL | V4L2_STD_SECAM, - .device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING, + .device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING | V4L2_CAP_IO_MC, }; Secondly, if one wants to test these patches with the gstreamer libcamerasrc, it is also required to change the minimum buffers needed in the driver. The libcamerasrc seems to have an issue that it only queues 1 buffer, thus not starting the driver to stream. Applying this hunk to the kernel works around this issue, although frame drops are probably to be expected: --- a/drivers/staging/media/imx/imx-media-capture.c +++ b/drivers/staging/media/imx/imx-media-capture.c @@ -1042,7 +1042,7 @@ int imx_media_capture_device_register(struct imx_media_video_dev *vdev) vq->mem_ops = &vb2_dma_contig_memops; vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; vq->lock = &priv->mutex; - vq->min_buffers_needed = 2; + vq->min_buffers_needed = 1; vq->dev = priv->dev; ret = vb2_queue_init(vq); Finally, add imx-media to the supported Devices in libcamera: --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -51,6 +51,7 @@ static const SimplePipelineInfo supportedDevices[] = { { "imx7-csi", "pxp" }, { "qcom-camss", nullptr }, { "sun6i-csi", nullptr }, + { "imx-media", nullptr }, }; } /* namespace */